# Reply to ” Comment on ’Once more about the $K\bar{K}$ molecule approach to
the light scalars’ ”

N.N. Achasov<EMAIL_ADDRESS>A.V. Kiselev<EMAIL_ADDRESS>Laboratory
of Theoretical Physics, Sobolev Institute for Mathematics, Novosibirsk,
630090, Russia

###### Abstract

The need to regularize loop integrals in a manner that preserves gauge
invariance, for example, using the Pauli-Villars method, requires a
subtraction that in the large mass limit hides its high momentum origin. This
gives rise to the illusion that only nonrelativistic kaon loop momenta are
relevant, when in fact this is not the case, as we show.

###### pacs:

12.39.-x 13.40.Hq 13.66.Bc

The authors of Ref. Comment  adduced the following argument against our
criticism achasov2007  . Since the $\phi\to K^{+}K^{-}\to\gamma(a_{0}/f_{0})$
amplitude vanishes for gauge invariance when the photon momentum vanishes,
only those terms of the integrand have the physical sense which vanish with
vanishing the photon momentum. What actually happens is that cancellation of
contributions from different places of momentum (or co-ordinate) space is
realized. It is commonplace in electrodynamics. In particular, low energy
theorems are based on this. Discarding the integrand in the third term of Eq.
(3), the second term in Eq. (4), and the contribution of Eq. (5) the authors
of Comment  distort the physical significance of the $K^{+}K^{-}$ loop model
because these contributions represent the high momentum and charge flow
distributions of kaons. Below we show that the $K^{+}K^{-}$ loop model
describes the relativistic physics.

When basing the experimental investigations of the light scalar mesons
production in the $\phi$ radiative decays
$\phi\to\gamma\bigl{[}a_{0}(980)/f_{0}(980)\bigr{]}\to\gamma\bigl{[}(\pi^{0}\eta)/(\pi^{0}\pi^{0})\bigr{]}$,
there was suggested achasov1989  the kaon loop model $\phi\to
K^{+}K^{-}\to\gamma\bigl{[}a_{0}(980)/f_{0}(980)\bigr{]}$ with the pointlike
interaction, see Fig. 1. This model is used in the data treatment and ratified
by experiment. In Refs. achasov2001 ; achasov2003  an analysis of mechanisms
of decays under consideration was carried out, which gave the clear arguments
for this kaon loop model.

|  |
---|---|---
(a) | (b) | (c)
Figure 1: Diagrams contributing to the radiative decay amplitude.

Every diagram contribution in

$T\left\\{\phi(p)\to\gamma[a_{0}(q)/f_{0}(q)]\right\\}=(a)+(b)+(c)$ (1)

is divergent hence should be regularized in a gauge invariant manner, for
example, in the Pauli-Wilars one.

$\displaystyle\overline{T}\left\\{\phi(p)\to\gamma[a_{0}(q)/f_{0}(q)],M\right\\}=\overline{(a)}+\overline{(b)}+\overline{(c)}\,,$
$\displaystyle\overline{T}\left\\{\phi(p)\to\gamma[a_{0}(q)/f_{0}(q)],M\right\\}=\epsilon^{\nu}(\phi)\epsilon^{\mu}(\gamma)\overline{T}_{\nu\mu}(p,q)$
$\displaystyle=\epsilon^{\nu}(\phi)\epsilon^{\mu}(\gamma)\left[\overline{a}_{\nu\mu}(p,q)+\overline{b}_{\nu\mu}(p,q)+\overline{c}_{\nu\mu}(p,q)\right],$
(2)
$\displaystyle\overline{a}_{\nu\mu}(p,q)=-\frac{i}{\pi^{2}}\int\Biggl{\\{}\frac{(p-2r)_{\nu}(p+q-2r)_{\mu}}{(m_{K}^{2}-r^{2})[m_{K}^{2}-(p-r)^{2}][m_{K}^{2}-(q-r)^{2}]}$
$\displaystyle-\frac{(p-2r)_{\nu}(p+q-2r)_{\mu}}{(M^{2}-r^{2})[M^{2}-(p-r)^{2}][M^{2}-(q-r)^{2}]}\Biggr{\\}}dr\,,$
(3)
$\displaystyle\overline{b}_{\nu\mu}(p,q)=-\frac{i}{\pi^{2}}\int\Biggl{\\{}\frac{(p-2r)_{\nu}(p-q-2r)_{\mu}}{(m_{K}^{2}-r^{2})[m_{K}^{2}-(p-r)^{2}][m_{K}^{2}-(p-q-r)^{2}]}$
$\displaystyle-\frac{(p-2r)_{\nu}(p-q-2r)_{\mu}}{(M^{2}-r^{2})[M^{2}-(p-r)^{2}][M^{2}-(p-q-r)^{2}]}\Biggr{\\}}dr\,,$
(4) $\overline{c}_{\nu\mu}(p,q)=-\frac{i}{\pi^{2}}\,2g_{\nu\mu}\int
dr\Biggl{\\{}\frac{1}{(m_{K}^{2}-r^{2})[m_{K}^{2}-(q-r)^{2}]}-\frac{1}{(M^{2}-r^{2})[M^{2}-(q-r)^{2}]}\Biggr{\\}}\,,$
(5)

where $M$ is the regulator field mass. $M\to\infty$ in the end

$\overline{T}\bigl{[}\phi\to\gamma(a_{0}/f_{0}),M\to\infty\bigr{]}\to
T^{Phys}\bigl{[}\phi\to\gamma(a_{0}/f_{0})\bigr{]}\,.$ (6)

We can shift the integration variables in the regularized amplitudes and
easily check the gauge invariance condition

$\epsilon^{\nu}(\phi)k^{\mu}\overline{T}_{\nu\mu}(p,q)=\epsilon^{\nu}(\phi)(p-q)^{\mu}\overline{T}_{\nu\mu}(p,q)=0\,.$
(7)

It is instructive to consider how the gauge invariance condition

$\epsilon^{\nu}(\phi)\epsilon^{\mu}(\gamma)\overline{T}_{\nu\mu}(p,p)=0$ (8)

holds true,

$\displaystyle\epsilon^{\nu}(\phi)\epsilon^{\mu}(\gamma)\overline{T}_{\nu\mu}(p,p)=$
$\displaystyle\epsilon^{\nu}(\phi)\epsilon^{\mu}(\gamma)T^{\,m_{K}}_{\nu\mu}(p,p)-\,\epsilon^{\nu}(\phi)\epsilon^{\mu}(\gamma)T^{M}_{\nu\mu}(p,p)=(\epsilon(\phi)\epsilon(\gamma))(1-1)=0\,.$
(9)

The superscript $m_{K}$ refers to the nonregularized amplitude and the
superscript $M$ refers to the regulator field amplitude. So, the contribution
of the (a), (b), and (c) diagrams does not depend on a particle mass in the
loops ($m_{K}$ or $M$) at $p=q$ typical . But, the physical meaning of these
contributions is radically different. The
$\epsilon^{\nu}(\phi)\epsilon^{\mu}(\gamma)T^{\,m_{K}}_{\nu\mu}(p,p)$
contribution is caused by intermediate momenta (a few GeV) in the loops ,
whereas the regulator field contribution is caused fully by high momenta
($M\to\infty$) and teaches us how to allow for high $K$ virtualities in a
gauge invariant way.

Needless to say the integrand of
$\epsilon^{\nu}(\phi)\epsilon^{\mu}(\gamma)\overline{T}_{\nu\mu}(p,p)$ is not
equal to 0.

It is clear that

$\displaystyle\epsilon^{\nu}(\phi)\epsilon^{\mu}(\gamma)T^{M\to\infty}_{\nu\mu}(p,q)\to\epsilon^{\nu}(\phi)\epsilon^{\mu}(\gamma)T^{M\to\infty}_{\nu\mu}(p,p)$
$\displaystyle\equiv\epsilon^{\nu}(\phi)\epsilon^{\mu}(\gamma)T^{M}_{\nu\mu}(p,p)\equiv(\epsilon(\phi)\epsilon(\gamma))\,.$
(10)

So, the regulator field contribution tends to the subtraction constant when
$M\to\infty$.

The finiteness of the subtraction constant hides its high momentum origin and
gives rise to an illusion of a nonrelativistic physics in the $K^{+}K^{-}$
model with the pointlike interaction. See, for example, Ref. kalash ; see
Section 2 in this paper.

This work was supported in part by the Presidential Grant No. NSh-1027.2008.2
for Leading Scientific Schools and by RFFI Grant No. 07-02-00093 from the
Russian Foundation for Basic Research. A.V.K. thanks very much the Dynasty
Foundation and ICFPM for support, too.

## References

  * (1) Yu. S. Kalashnikova, A. E. Kudryavtsev, A. V. Nefediev, J. Haidenbauer, and C. Hanhart, Phys. Rev. D 78, 058501 (2008).
  * (2) N.N. Achasov and A.V. Kiselev, Phys. Rev. D 76, 077501 (2007)
  * (3) N.N. Achasov and V.N. Ivanchenko, Nucl. Phys. B 315, 465 (1989).
  * (4) N.N. Achasov and V.V. Gubin, Phys. Rev. D 63, 094007 (2001);
Yad. Fiz. 65, 1566 (2002) [Phys. At. Nucl. 65, 1528 (2002)].

  * (5) N.N. Achasov, Nucl. Phys. A 728, 425 (2003);
Yad. Fiz. 67, 1552 (2004) [Phys. At. Nucl. 67, 1529 (2004)].

  * (6) A typical example of such integrals is $2\int_{0}^{\infty}\frac{m^{2}x}{(x+m^{2})^{3}}dx=1$.
  * (7) Yu. S. Kalashnikova, A. E. Kudryavtsev, A. V. Nefediev, C. Hanhart, and J. Haidenbauer, Eur.Phys.J. A 24, 437 (2005).


# Pressure Test: Quantifying the impact of positive stress on companies from
online employee reviews

Sanja Šćepanović  Nokia Bell Labs, Cambridge, United Kingdom  Marios
Constantinides  Nokia Bell Labs, Cambridge, United Kingdom  Daniele Quercia
Nokia Bell Labs, Cambridge, United Kingdom  CUSP, Kings College London, United
Kingdom<EMAIL_ADDRESS>Seunghyun Kim  Georgia Tech, USA

###### Abstract

Workplace stress is often considered to be negative, yet lab studies on
individuals suggest that not all stress is bad. There are two types of stress:
_distress_ refers to harmful stimuli, while _eustress_ refers to healthy,
euphoric stimuli that create a sense of fulfillment and achievement. Telling
the two types of stress apart is challenging, let alone quantifying their
impact across corporations. By leveraging a dataset of 440K reviews about S&P
500 companies published during twelve successive years, we developed a deep
learning framework to extract stress mentions from these reviews. We proposed
a new methodology that places each company on a stress-by-rating quadrant
(based on its overall stress score and overall rating on the site), and
accordingly scores the company to be, on average, either a _low stress_ ,
_passive_ , _negative stress_ , or _positive stress_ company. We found that
(former) employees of positive stress companies tended to describe high-growth
and collaborative workplaces in their reviews, and that such companies’ stock
evaluations grew, on average, 5.1 times in 10 years (2009-2019) as opposed to
the companies of the other three stress types that grew, on average, 3.7 times
in the same time period. We also found that the four stress scores aggregated
every year—from 2008 to 2020 —closely followed the unemployment rate in the
U.S.: a year of positive stress (2008) was rapidly followed by several years
of negative stress (2009-2015), which peaked during the Great Recession
(2009-2011). These results suggest that automated analyses of the language
used by employees on corporate social-networking tools offer yet another way
of tracking workplace stress, allowing quantification of its impact on
corporations.

## Introduction

According to the American Institute of Stress, 40% of workers consider their
jobs to be stressful; a number that has significantly increased during the
COVID-19 pandemic [1]. The World Health Organization treats stress as the
number one health threat in the U.S., with more than 60% of doctor visits
being due to a stress-related issue [2]. Workplace stress is often linked to
lower motivation, poor performance, and decline in employees’ well-being [3],
while it is estimated to amount to 190 billions in healthcare costs in the
U.S. alone [4]. To currently track how its employees deal with stress, a large
company would typically recruit consultants who would then administer surveys
tailored to the company’s situation, which typically end up being costly [5],
and restricted to a limited pool of self-selected participants [6, 7, 8]. The
current situation points to the need of more research.

Stress is defined as “a set of physical and psychological responses to
external conditions or influences, known as stressors” [9]. According to
Lazarus [10], “any change, either good (eustress) or bad (distress), is
stressful, and whether it is a positive or a negative change, the
physiological response is the same.” To cope with workplace stress, an
employee has to cognitively acknowledge that a situation causes stress before
even attempting to manage it [11]. Kobasa’s framework of psychological
hardiness offers three main categories of coping strategies [12]: _commitment_
(having an active involvement in one’s own work with a sense of purpose),
_control_ (believing and acting instead of feeling helpless in front of
adversity), and _challenge_ (believing that change could be a source of
improvement). Kobasa posited that these categories could help individuals face
challenges and, as such, individuals could turn stressful events into
opportunities for personal growth [12]. However, despite having explored the
relation between stress and job performance for decades, researchers have not
yet established whether stress and performance are in a negative linear
relation, a positive linear relation, or an inverted-U relation [13].

To tackle this gap, we draw upon two streams of previous literature, that is,
literature about stress in the corporate context, and literature on how to
gauge stress from online data. In the literature about stress in the corporate
context, stress is often being portrayed as negative [14]; and as a leading
cause of death, poor work performance, and diminishing well-being [3]. More
recently, however, researchers have advocated that there exist another type of
stress: _positive stress_. The idea is that whether stress is positive or
negative depends on how an individual reacts to a stressor [15]. _‘One’s
stress mindset can be conceptualized as the extent to which one holds the
belief that stress has enhancing consequences for various stress-related
outcomes such as performance and productivity, health and well-being, and
learning and growth, or holds the belief that stress has debilitating
consequences for those outcomes[15]’_. Of prime importance is to distinguish
appraisal from stress mindset. Stress mindset describes the evaluation of the
nature of stress itself as positive or negative (i.e., enhancing or
debilitating) [15], whereas appraisal is about the evaluation of a particular
stressor as more or less stressful [16]. For example, one may appraise a
difficult task as highly stressful and have a stress debilitating mindset,
which, in turn, leads the individual to experience the situation as draining
(negative stress). By contrast, another individual may again consider the task
as highly stressful but have a stress enhancing mindset, leading the
individual to experience the situation as an opportunity for growth and
development (positive stress). While stress is often linked to depression [17,
18], several accounts posit that certain stressful experiences may
fundamentally change individuals for the better—a phenomenon referred to as
stress-related growth [15]. The experience of stress can enhance the
development of mental toughness, greater appreciation for life, and an
increased sense of meaningfulness [19, 20]. However, as Crum et al. [15]
pointed out, these conflicting findings in the stress literature suggest a
nuanced view of stress. A view that recognizes the debilitating nature of
stress on health and performance, but can also account for its enhancing
nature in specific circumstances. We hypothesized that the presence of both
positive and negative stress can be measured from digital data based on
previous literature that has done just that with different techniques upon
different datasets. Guntuku et al. [21] used the Linguistic Inquiry and Word
Count (LIWC) [22] dictionary’s features (e.g., topical categories, emotions,
parts-of-speech) to predict stress of social media (Facebook and Twitter)
users. Saha and De Choudhury [23] did a similar study but on Reddit and did so
in conjunction with gun violence events, and found specific stress markers to
be associated with linguistic changes about _“higher self pre-occupation and
death-related discussion.”_ Similar to our study, Vedant et al. [24] showed
that the use of language in employee reviews can be used to operationalize
organizational culture: the collection of values, expectations, and practices
that guide and inform employees’ actions within a company.

Based on these preliminary findings, we hypothesized that workplace stress is
reflected in company reviews. To explore this hypothesis, we placed companies
on a 2x2 coordinate system, based on their overall stress scores and overall
ratings on the company review site. This stress-by-rating quadrant effectively
divided companies in four stress types that we termed low stress, passive,
negative stress, and positive stress (Table 1 shows example reviews of
companies of each stress type). Low stress companies enjoy high overall
ratings and low stress scores. These are usually established organizations
that offer workplace flexibility, good pay, and bonuses. Passive companies are
characterized by low overall ratings and a small proportion of posts
mentioning stress. They tend to have high turnover, due to repetitive workload
and non-motivated employees. Negative stress companies are characterized by
low overall ratings and a high proportion of posts mentioning stress.
Employees of these companies are particularly unhappy as, in addition to the
unsatisfactory conditions, they also experience high pressure. Finally,
positive stress companies enjoy high overall ratings but also high stress
scores. These tend to be inspiring, reputable workplaces that attract
employees because of the collaborative atmosphere and career prospects despite
the pressure employees are subject to. The project website for our study is
found on https://social-dynamics.net/positive-stress.

### Data

After obtaining the U.S. unemployment rates between 2008 and 2020 from the
U.S. Bureau of Labor Statistics [25] and the S&P 500 stock market data
(including the 500 large capital U.S. companies with a cumulative market
capitalization to be around 70-80% of the total in the country) from the Yahoo
Finance portal [26], we matched that data with our company reviews. More
specifically, we obtained 440K geo-referenced posts on Glassdoor
(https://www.glassdoor.com), a popular company reviewing site about the S&P
500 companies published during twelve successive years, from 2008 to 2020. On
this site, current and, more likely, former employees of companies write
reviews about their own corporate experience, ranging from job interviews to
salaries to workplace culture. The site provided an overall rating of each
company based on employees’ reviews. As of 2021, there are 50M monthly
visitors on the platform, and 70M reviews about 1.3M companies. To ensure
quality reviews, the site employs the following three mechanisms. First, an
automatic (proprietary) and manual content moderation system is paired with
the possibility for users to flag content. Such a combined system partly
prevents fake reviews (e.g., a company unfairly forcing employees to leave
positive reviews). Second, every user wanting to browse others’ content has to
take the time to write one review. This requirement encourages the presence of
neutral reviews and partly prevents the so-called _non-response bias_ , a
situation in which the opinions of respondents are very different from those
of non-respondents. Third, the site allows for a maximum of one review per
employee per company per year, preventing any employee from contributing a
disproportionate number of reviews, and, in so doing, discouraging _sampling
bias_ , a situation in which the opinions of some members are more represented
in the data than those of others.

Table 1: Example reviews of companies of each stress type. Stress Type |  Review excerpt
---|---
Low stress |  My company walks its talk. It _[the company]_ takes care of customers and employees.
Negative stress |  There is a feeling of scarcity due to the constant reorganizations, pressure, and surprise layoffs. _[…]_ You could imagine how toxic the environment is.
Passive stress |  There is no regard for how the remaining work will get done, just how the bottom line looks at that moment in time. People are not treated as respected contributors to the organization. _[…]_ This is a very unstable, unhealthy, volatile, stressed out environment, with incredibly poor leadership.
Positive stress |  You have to be a very driven and self-motivated person to be successful here. If you are willing to commit and put in the extra effort and hard work, it will be extremely worth it. _[…]_ Every day is very busy and it can be stressful at times but its all worth it!.

## Methods

Figure 1: Placing companies on a stress-by-rating quadrant by detecting stress
mentions in reviews about a company using a state-of-the-art NLP deep-learning
framework (_Step 1_), placing the company in the rating-by-stress quadrant,
and computing its association with its stress type (i.e., with its quadrant)
(_Step 2_). To see how the association is computed, consider company $c$ shown
in _(b)_ to be of positive stress. $c$ is placed according to its
$z_{rating}(c,T)$ along the $x$-axis, and to its $z_{stress}(c,T)$ along the
$y$-axis. $R$ is the radius from the center to $c$’s point; $\alpha$ is the
angle between the radius line and the $x$-axis; $\beta$ is the angle between
the radius line and the $y$-axis; and $\gamma$ is the angle between the radius
line and the diagonal shown as a dotted line. The function $f(c,s,T)$ combines
$R$, $\alpha$, $\beta$, and $\gamma$, and accordingly scores $c$ to have a
high association weight with positive stress $s$ during period $T$ (darker
shades of colors), as $c$ is close to the quadrant’s diagonal, and distant
from the intersection point.

We extracted mentions related to stress using an NLP deep-learning tool, which
is trained to extract medical conditions from free-form text (Figure 1(a)). We
then designed a new methodology that placed the 500 S&P companies on a stress-
by-rating quadrant based on their overall ratings on the reviewing site on one
axis, and the presence of mentions related to stress in their reviews on the
other axis (Figure 1(b)). In so doing, we classified each company to be, on
average, either a _low stress_ , _passive_ , _negative stress_ , or _positive
stress_ company. We finally computed each company’s strength of membership to
its quadrant depending on whether the company is placed both close to the
diagonal and far from the (0,0) intersection point. The function $f(c,s,T)$,
which is expressed in Equation (3) and graphically depicted in Figure 1(b),
assigns a higher weight to company $c$ of stress type $s$, if $c$ is both
closer to the quadrant’s diagonal (i.e., it is farther from the remaining
quadrants) and more distant from the two axes’ intersection (i.e., it has
higher absolute overall rating and stress score values). We call $f(c,s,T)$ to
be company $c$’s association with stress type $s$ during $T$ since the higher
its value, the more $c$ is associated with stress type $s$ during $T$.

Extracting stress mentions from posts. To extract stress mentions, we used the
MedDL entity extraction module [27] (the left rectangle in Figure 1 (a)).
MedDL uses _contextual embeddings_ and a _deep BiLSTM-CRF sequence labeling
architecture_ (we used the default parameters as specified in [27]). The model
was pre-trained and evaluated on a labeled dataset of Reddit posts called
MedRed [27]. The MedRed dataset was split into train (50%), dev (25%), and
test (25%) sets. We evalued MedDL using the strict and relaxed $F1$-scores,
two commonly used performance metrics for entity extraction models. The
_strict $F1$-score_ counts only the _exact_ matches with the true labels as
correct, while the _relaxed $F1$-score_ takes into accoun the partially
matching extracted entities. We provide formulae for the two scores in
_Supplementary Information_ (SI Equation (1)). MedDL was compared against two
well-known entity extraction tools: MetaMap (a well-established tool [28] and
a de-facto baseline method for NLP studies related to health [29]) and
TaggerOne (a machine learning tool using semi-Markov models and a medical
lexicon [30]). The MedDL method achieved a strict/relaxed $F1$-score of
$.71$/$.85$ when extracting symptoms (Figure S4), outperforming both MetaMap
and TaggerOne by a large margin (the two have $F1$-scores of $.17/.48$ and
$.31/.58$, respectively). Furthermore, MedDL has shown generalizability when
applied on dataset (e.g., dream reports [31]) different than those it was
trained on (i.e., Reddit data).

Table 2: Top15 most frequent stress-related mentions identified on a company review site, and their frequency counts. Condition related to stress | # mentions | Example mention
---|---|---
stress | 3473 | _“Great company to work for, if you can handle stress.”_
high stress | 710 | _“High stress work environment, long work hours.”_
pressure | 447 | _“a lot of pressure to get things done.”_
burnout | 277 | _“[…], the ones who made the cut to stay are suffering from burnout.”_
understaffing | 99 | _“Somewhat job stability due to understaffing.”_
heavy workload | 58 | _“Lack of work/life balance, extremely heavy workload.”_
exhaustion | 58 | _“You will be pushed to the point of exhaustion […].”_
stress levels | 57 | _“[…] stress levels peak insanely when the store manager […].”_
overworked | 45 | _“At times, you can feel overworked and undervalued.”_
tension | 38 | _“There’s a lot of tension between coworkers because of commission.”_
high workload | 38 | _“[…] seeing many large set-backs which cause very high workload”_
extreme stress | 33 | _“Beware: extreme stress and pressure.”_
mental stress | 23 | _“[…] ends up giving you a lot of mental stress.”_
overload | 17 | _“No work life balance […], overloaded and benefits are not good.”_
pressure to perform | 9 | _“[…] a lot of pressure to perform, long working hours”_

We extracted stress mentions in three steps (further detailed in
_Supplementary Information_). First, we detected over 21K posts that mentioned
over 5K unique medical conditions. Most frequent medical conditions identified
include _stress_ , _pain_ , _headache_ , and _depression_. Second, we
inspected the top 200 most mentioned conditions and manually selected $31$ of
them that specifically reflect workplace stress (top 15 are shown in Table 2).
Third, we extracted all reviews mentioning any of the $31$ conditions. This
resulted in $7,338$ posts related to stress, which accounted for $1\%$ of all
posts. Despite this seemingly low number of posts, when aggregated, these
posts returned statistically significant results for our metrics, which are
described next.

Associating stress types with companies. We placed each S&P 500 company on a
stress-by-rating quadrant. More specifically, for each company $c$, we
computed its average review rating and its stress score:

$\displaystyle\textit{rating(c,T)}=\textit{$c$'s average review rating during
}T,$ $\displaystyle\textit{stress(c,T)}=\frac{\textit{\\# $c$'s posts related
to stress during }T}{\textit{total \\# $c$'s posts during }T}.$

where $T$ is set to initially include all the years under study (2009-2019).
To then ease comparability, we $z$-scored these two values:

$\displaystyle z_{rating}(c,T)$
$\displaystyle=\frac{rating(c,T)-\mu_{rating}(T)}{\sigma_{rating}(T)},$
$\displaystyle z_{stress}(c,T)$
$\displaystyle=\frac{stress(c,T)-\mu_{stress}(T)}{\sigma_{stress}(T)}.$

where $\mu_{rating}(T)$ and $\sigma_{rating}(T)$ are the average and standard
deviation of the review ratings for all companies (regardless of their stress
types) during the whole period $T$ (readily available on the company review
site), and $\mu_{stress}(T)$ and $\sigma_{stress}(T)$ are the average and
standard deviation of the stress scores for all companies during the whole
period $T$.

Each S&P 500 company was assigned to one of the four quadrants based on the
signs of their two z-scores (Figure 1(b)). For example, a company $c$ with a
negative $z_{rating}(c,T)$ and a positive $z_{stress}(c,T)$ would be placed in
the negative stress quadrant, while a company with a positive
$z_{rating}(c,T)$ and a positive $z_{stress}(c,T)$ would be placed in the
positive stress quadrant. The resulting quadrants are consequently four:

  * _Low Stress companies_ \- These enjoy high overall ratings and low stress scores. Their employees tended to think very positively about their workplace experience with comparatively fewer posts mentioning stress conditions.

  * _Passive companies_ – These are characterized by low overall ratings and a small proportion of posts mentioning stress. Their employees were mostly not satisfied with their jobs, but they also showed comparatively fewer signs of stress in their reviews.

  * _Negative stress companies_ – These are characterized by high stress scores and low overall ratings. Their employees mentioned stress conditions, while also scoring their workplace experience low.

  * _Positive stress companies_ – These enjoy high ratings despite high stress scores. Their employees mentioned stress yet did so in the context of high-pressure and highly rewarding work environments.

Once a company $c$ is placed in its quadrant (i.e., associated with its stress
type $s$), we needed to estimate its association with this quadrant, i.e.,
with $s$. For example, company $c$ with ($z_{rating}(c,T),z_{stress}(c,T)$)
equal to (3,3) is more strongly associated with positive stress, than what a
company with $(0.5,0.5)$ would be. To estimate $c$’s association with $s$, we
combined $c$’s two $z$-scores concerning review rating and stress score as
follows (and as depicted in Figure 1(b)):

$\displaystyle f(c,s,T)$
$\displaystyle=\left\\{\begin{array}[]{@{}ll@{}}l(z_{rating}(c,T),z_{stress}(c,T))=R/(\gamma+\pi),&\text{if}\
c\in s\text{ during }T;\\\ 0,&\text{if}\ c\notin s\text{, or }$c$\text{ has no
review during }T;\end{array}\right.$ (3) where: $\displaystyle R$
$\displaystyle=\sqrt{z_{rating}(c,T){}^{2}+z_{stress}{}(c,T)^{2}},$
$\displaystyle\gamma$
$\displaystyle=max((\alpha-\pi/4),(\beta-\pi/4)),\quad\quad$
$\displaystyle\alpha$
$\displaystyle=arccos(|z_{rating}(c,T){})|/R),\quad\quad\quad\quad\quad$
$\displaystyle\beta$ $\displaystyle=arccos(|z_{stress}(c,T){})|/R).$

where $T$ is initially set to include all the years under study, from 2009 to
2019. To ease understanding of the above formula, consider that function $l$,
on input of the two $z$-scores (i.e., the company’s two coordinates in the
quadrant), computes the extent to which company $c$ is on the diagonal and far
from the (0,0) intersection point (Figure 1(b)). It gives higher weights to
companies that are both closer to the quadrant’s diagonal (i.e., which are
farthest from the remaining quadrants) and more distant from the axes’
intersection (i.e., which have higher absolute rating/stress score values).

Computing stress scores over the years. For each year $y$, we quantified the
amount of a given stress type $s$ expressed in the posts produced in that
year. More specifically, we computed:

$\displaystyle m{(s,y)}=\sum_{c\in s}f(c,s,y)\times w{(c,y,s)},$ (4)

For all the companies of stress type $s$, we summed each company’s association
$f(c,s,y)$ with $s$ during year $y$ weighted by the presence of posts about
the company during $y$ (giving higher weights to companies whose employees
contributed more reviews in that year):

$w(c,y,s)=\left\\{\begin{array}[]{@{}ll@{}}\frac{\textit{\\# $c$'s posts in
year $y$}}{\textit{total \\# posts in year $y$}},&\text{if}\ c\in s\textit{ in
year }y;\\\ 0,&\text{if $c$ has no reviews in year $y$.}\end{array}\right.$
(5)

Associating topical categories with stress types. To identify relevant words
for each stress type, we run BERTopic [32], which is a state-of-the-art topic
modeling algorithm. A topic modeling algorithm is an unsupervised technique to
extract topics that appear frequently in a piece of text (in our case, a
post). The algorithm works in four sequential steps:

  1. 1.

converts each post into a 512-dimensional vector (called embedding) of
numerical values using a pre-trained BERT-based sentence transformer [33] (in
our case, we used the default model, that is, the “paraphrase-MiniLM-L6-v2”).
BERT (Bidirectional Encoder Representations from Transformers) is a state-of-
the-art transformer-based machine learning technique for natural language
processing (NLP), which takes into account the context of each word.

  2. 2.

reduces dimensionality using UMAP [34] (or Unification Map) for every
embedding, as many clustering algorithms handle high dimensionality poorly.
UMAP is arguably the best performing dimensionality reduction algorithm as it
keeps significant portion of the high-dimensional structure in lower
dimensionality.

  3. 3.

uses HDBSCAN [35] for clustering with the “UMAP” embeddings, resulting in
similar posts being clustered together. HDBSCAN is a density-based algorithm
that works well with UMAP as the structure is preserved in a lower-dimensional
space. Additionally, HDBSCAN does not force data points to clusters as it
considers them outliers.

  4. 4.

identifies keywords using the c-TF-IDF [32] score (Equation 6), and using that
score, derives topics from the identified keywords. To create a topic
representation, we took the top 3 keywords per topic based on their c-TF-IDF
scores. The higher the score, the more representative is as the score is a
proxy of information density.

$\textrm{c-TF-IDF}_{l}=\frac{k_{l}}{o_{l}}\times\frac{p}{\sum_{j}^{q}k_{j}}$
(6)

where the frequency of each keyword $k$ is extracted for each topic $l$ and
divided by the total number of keywords $o$. The total, unjoined, number of
posts $p$ is divided by the total frequency of keyword $k$ across all topics
$q$.

### Analysis plan

Our analysis plan unfolded in three steps. First, as an initial validation
step, we ascertained that stress was paraphrased in a company’s reviews
differently according to the company’s stress type. Second, we tested whether
the evolution of each stress score over the years tallied with large-scale
exogenous events such as the Great Recession. Third, we tested that a
company’s stress type is partly associated with its stock growth.

## Results

### Topics discussed in reviews of companies of different stress types

To ascertain whether the content of the reviews captured aspects specific to
the four stress types, we identified the top relevant words for each type by
running a topic modeling algorithm called BERTopic [32], and did so on four
distinct sets of reviews: each set contained reviews of all the companies of a
given stress type. This algorithm found the emergent topics in the four sets,
and Table 3 lists the the top three words for each topic. The top10 topics for
each quadrant are statistically associated with the quadrant. That is, based
on chi-square tests, each topic $l$ associated with quadrant $s$: has
frequency in $s$ always above zero (is dependent on $s$), and is independent
of any quadrant other than $s$. As detailed in _Supplementary Information_ ,
by inspecting these groups of words and corresponding representative reviews,
six annotators identified the emergence of three workplace themes:

_Career drivers_ (first set of rows in Table 3). Negative stress companies
were associated with words such as ‘overtime’, ‘mandatory, ‘shifts’, and the
typical workplace described in the reviews, according to our annotators, was
characterized by considerable emotional pressure. On the other hand, passive
companies were associated with words such as ‘vacation’, ‘pto’, and
‘vacation/sick’, and the corresponding reviews tended to deflect from the day-
to-day work and focus on activities outside work such as vacation and time
off. Low stress companies were associated with words such as ‘scheduling’,
‘flexibility’, and ‘autonomy’, and the typical workplace described in the
reviews was one in which employees cherished their sense of control over their
work. Finally, positive stress companies were associated with words such as
‘teamwork’, ‘supportive’, and ‘collaborative’, and the typical workplace in
the reviews was one with a collaborative and supportive culture.

_Industry or benefits_ (second set of rows in Table 3). Negative stress
companies were associated with words such as ‘discounts’, ‘sale’, ‘coupons’,
while positive stress companies were associated with words such as ‘gain’,
‘billions’, and ‘software’. Their reviews were effectively mentioning the
industry sectors they referred to: Consumer Discretionary (e.g., retail shops)
for the reviews of negative stress companies, and Information Technology for
those of positive stress ones. On the other hand, passive companies were
associated with words such as ‘insurance’, ‘espp’, and ‘hsa’, and, similarly,
low stress ones with words such as ‘401k’, ‘bonus’, and ‘retirement’; the
corresponding reviews indicated workplaces in which concerns about long-term
financial benefits rather than the presence of implicit incentives in one’s
own work were at the forefront.

_Emotional Aspects_ (third set of rows in Table 3). Negative stress companies
were associated with words such as ‘horrible’, ‘terrible’, and ‘awful’,
confirming, once again, the presence of emotional pressure. Passive companies
were instead associated with words such as ‘repetitive’, ‘turnover’, and
‘workload’, confirming the tedious nature of those workplaces. Low stress
companies were associated with words such as ‘fair’, ‘friendlygood’, and
‘pays’, and the corresponding reviews described a good work-life balance.
Finally, positive stress companies were associated with words such as
‘prestige’, ‘boost’, and ‘reputation’, and their reviews described high
performing, dynamic, and fast-paced workplaces.

| Negative stress | Passive | Low stress | Positive stress
---|---|---|---|---
Career drivers |  overtime |  vacation |  scheduling |  teamwork
mandatory |  pto |  flexibility |  supportive
shifts |  vacation/sick |  autonomy |  collaborative
Industry or benefits |  discounts |  insurance |  401k |  gain
sale |  espp |  bonus |  billions
coupons |  hsa |  retirement |  software
Emotional aspects |  horrible |  repetitive |  fair |  prestige
terrible |  turnover |  friendly/good |  boost
awful |  workload |  pays |  reputation
Table 3: Three-word groups present in the reviews of companies of the four
stress types. These groups were automatically found by BERTopic and speak to
three main workplace characteristics: career drivers, industry and benefits,
and emotional aspects. For each group, the top three words are shown together
with their normalized word importance. Abbreviations of words describing
monetary benefits include pto (paid time off); espp (employee stock purchase
plan); hsa (health savings account); 401k (a retirement savings and investing
plan that employers offer).

### Evolution of stress types and the Great Recession

After the preliminary validation step in which we ascertained that stress was
paraphrased in reviews differently according to the stress type, we tested
whether the evolution of each stress score over the years tallied with large-
scale exogenous events such as the Great Recession. We plotted the amount
$m(s,y)$ of each stress score $s$ in each year $y$ (as per Equation (9)), from
2008 to 2020 (top panel in Figure 2). The overall changes closely followed the
unemployment rates from the U.S. Bureau of Labor (bottom panel in Figure 2): a
year of positive stress (2008) was rapidly followed by several years of
negative stress (2009-2015), which peaked during the Great Recession
(2009-2011) during which the U.S. went through a loss of over 7.5 million jobs
and high unemployment rates [36].

Figure 2: The evolution of: _(top)_ the four types of stress; and _(bottom)_
the unemployment rate in the U.S., with the horizontal dashed line reflecting
pre-recession rate. The stress score per year is calculated using Equation
(9), and its standard deviations are shown with shaded lines.

(a)

(b)

Figure 3: _(a)_ Distribution across companies of the logarithm of stock growth
values from the average stock price in 2009 and that of 2019
(${stock\\_growth}_{[09-19]}=stock_{2019}/stock_{2009}$) showing the stock
growth is log-normally distributed. The average stock price for year $y$
($stock_{y}$) is calculated as the average of the daily Adjusted Closing
Prices for the year. _(b)_ Geometric mean of the stock growth values
$\bar{GM}({stock\\_growth}_{[09-19]})$ for increasing stress score percentiles
for the companies of a given stress type. Error bars represent geometric
standard error $GSE({stock\\_growth}_{[09-19]})=$
$\bar{GM}({stock\\_growth}_{[09-19]})/$
$\sqrt{N}\cdot\sigma(log({stock\\_growth}_{[09-19]}))$.

### Stock growth of companies of different stress types

Finally, we hypothesized that a company’s way of dealing with workplace stress
was partly _associated_ with performance. Given our data, we cannot study
whether stress _causes_ (poor) performance but can only study how the two are
associated. Also, there is no company performance measure that is solely
affected by a company’s stress culture. As such, our stress scores are
unlikely to be predictive of any company-wide performance measure. We opted
for long-term stock growth as our performance measure, not least because it is
publicly available and standardized across companies. However, such a growth
is partly affected by a company’s culture, and conflates endogenous factors
(e.g., productivity) with exogenous ones (e.g., financial cycles). Yet we
expected our stress measures to qualitatively describe different forms of
financial success, at least in the long term. To that end, we computed stock
growth during the full period of study, that is, between 2009 to 2019:

$\textrm{stock growth}_{[09-19]}(c)=\frac{stock(c)_{2019}}{stock(c)_{2009}}$
(7)

where $stock^{i}$ is the average adjusted closing price of company $c$’s stock
in year $i$. We chose long-term growth instead of short-term one (e.g., that
pertaining 2018-2019) to partly account for any potential influence of
exogenous events (e.g., Great Recession, market manipulation, incidental
growth/decline [37]). In _Supplementary Information_ , we show that the
results do not qualitatively change when considering the narrower 5-year
period from 2014 to 2019. Given a stress type $s$, we computed company $c$’s
_association_ $f(c,s,T)$ with $s$ during time period $T$ (initially set to the
whole period of study), consequently grouping all the companies of a given
stress type into their stress score percentiles (Figure 3b). As the
distribution of stock growth values across companies is heavy-tailed (Figure
3a), we used the geometric mean to average these values across companies. That
is, $GM({\textrm{stock growth}_{[09-19]}})=\Pi(\textrm{stock
growth}_{[09-19]}(c))^{1/n}$, where $c$ is a company in a specific _(stress
type,percentile)_ bin, and $n$ is the number of the companies in such a bin.
Positive stress companies enjoyed the highest stock growth with an average
value across all percentiles of $\bar{GM}(\textrm{stock
growth}_{[09-19]})=5.07$ (Figure 3b), while the average stock growth across
the other three types of companies was noticeably lower
($\bar{GM}({\textrm{stock growth}_{09-19}})=3.70$), with passive stress
companies exhibiting the lowest growth ($\bar{GM}({\textrm{stock
growth}_{09-19}})=3.42$). To ease the interpretation of such values, consider
the example of Equinix, a digital infrastructure company headquartered in
California, which our approach labeled to be a “positive stress” company. Its
stock price traded at 61$ in 2009 and its stock price climbed over 695% (i.e.,
its $\bar{GM}(\textrm{stock growth}_{[09-19]})$ was 7.95), trading at 485$ ten
years later.

## Discussion

### Limitations

This work has five main limitations. The first concerns the inability of
studying whether stress causes performance differences given the absence of
cross-lag data that links performance to a stress-related company-wide
indicator. Theoretically, we could run a lagged analysis as a linear
regression where the dependent variable is the company’s growth at different
time intervals. However, such an analysis is hard because of two main reasons:
_a)_ no fine-grained temporal granularity for reviews is possible as reviews
might be temporally misaligned since they could be posted after an employee
leaves the company, and _b)_ many, mostly smaller, companies have joined the
public reviewing site at later points in time, thus reviews will not cover all
12 years of analysis.

The second limitation is that the decreasing trend of stock growth may be
dependent on the two main aspects: company ratings and industry sector. These
two have little to do with the hypothesized relationship between stress and
performance. We therefore repeated our analyses by considering a company’s
overall website rating and its industry sector. As for ratings, we indeed
found increasing stock growth with increasing review ratings; still, positive
stress companies experienced the highest growth (Figure S6 in _Supplementary
Information_) compared to highly-rated companies. As for industry sectors, we
showed that tech companies were over-represented in the positive stress set,
and stock growth was partly driven by them (Figure S7 in _Supplementary
Information_). However, by separating companies by industry sector, we still
observed that positive stress companies grew more than the other three types
(Figure S8 in _Supplementary Information_).

The third limitation concerns data biases related to temporal granularity and
geographic representativeness. Upon new available data, future studies could
study workplace stress outside US, allowing for cross-cultural comparisons.

The fourth limitation has to do with nuances when rating a company (e.g.,
being satisfied with the use of the overall company rating and not its
composing dimensions). While on Glassdoor there are several rating fields
available, only the overall rating field was mandatory and hence provided
sufficient coverage for our analysis.

The fifth limitation is that the deep-learning model used to detect stress
mentions in posts is not always accurate. Our medical entity extraction model
has two main limitations. First, the model’s strict/relaxed accuracy is
.71/.85, and, even though it outperformed competitive baselines by a large
margin, it still is not perfect. To partly address this issue, our method
limits itself to textual mentions pertaining stress at work. Second, entity
extraction models such as ours are not always able to tell apart personal from
figurative health mentions (e.g., _‘I felt pain’ vs. ‘He was such a pain to
work with’_). This is still an active area of research. Yet our model is
relying on a large transformer model (e.g., contextual embeddings RoBERTa),
and, as such, it is less likely to make such errors than a simpler, keyword-
matching technique. Future studies could use some of the newly published
social media datasets [38] to further train our model to distinguish between
different _types of health mentions_.

### Implications

To place our work in the broader context of the literature, we point out three
main findings. Our first was that _company reviews contain linguistic markers
of four stress types_. Previous work found that stress of social media users
can be detected by analyzing their textual content, both on Twitter and Reddit
[21]. Another study by Saha and De Choudhury found that high levels of stress
markers were present in the use of language in Reddit comments posted by
university students who experienced gun violence events at their campuses.
This work showed that such linguistic changes are sufficiently subtle to
reflect four _different types of stress_ , that is, low, passive, positive,
and negative stress. Our second finding was that _stress over the years
tallied with large-scale exogenous events_. In particular, negative stress was
the most prevalent among the four stress types in recession years (both great
and mini recessions). This finding is in line with the literature linking
economic downturn with stress and mental health issues caused by job
instability [39], and speaks to the presence of linguistic markers reflecting
negative stress associated with country-level economic performance. Our third
finding was that _company stock growth is associated with positive stress._
This is a new finding, not least because of lack of data in the past. While
stock growth conflates endogenous factors (e.g., productivity) with exogenous
ones (e.g., financial cycles), we found that positive stress companies enjoyed
significantly stronger stock growth.

However, more work is needed to understand how to change a company’s culture
into one in which stressors could be used for one’s growth and self-
development. Given the recent wave of Great Resignation (i.e., the elevated
rate at which U.S. workers have quit their jobs [40]), questions relating to
corporate culture [41] and ways of retaining top talent are of utmost
importance. A recent study from Mercer, an American asset management firm,
found that elevated levels of employee turnover are not due to lack of
engagement at work but attributed to workplace culture and heightened
stressors. Therefore, organizations need to take immediate actions by
(re)assessing their workplace culture first and by then shifting it when
deemed appropriate, through training that fosters psychological safety and
cultivates one’s mindset towards positive stress. Traditionally, employee
well-being has been tracked with tailored surveys. Automated analyses of the
language used by employees on corporate social-networking tools might offer
yet another way of tracking workplace stress, which is sufficiently granular
to assess the impact of interventions in a company. Beyond the immediate use
of these findings for individual companies, several other stakeholders could
benefit from our methodology including government officials. As the
performance of the S&P 500 companies affects the broader U.S. economy,
recommended workplace practices could be established at state- or national-
level to improve work conditions.

## Data availability

We made our code and data available in a readily usable format on GitHub
(https://github.com/sanja7s/positive_stress_in_companies) to allow for
reproducibility. For each company, we shared the following attributes: company
name, #total reviews, #stress reviews, average rating, rating of work-life
balance, rating of career prospects, rating of the company, rating of the
culture, rating of the management, stress type, strength of association with
the stress type, stock values/growth for: 2009, 2012, 2014, 2019, and industry
sector.

## References

  * [1] Sugar, A.  Stay cool under pressure — without appearing cold.  _Harvard Business Review_ (2020).
  * [2] Nerurkar, A., Bitton, A., Davis, R. B., Phillips, R. S. & Yeh, G.  When physicians counsel about stress: Results of a national study.  _JAMA Internal Medicine_ 173, 76–77 (2013).
  * [3] Cartwright, S. & Cooper, C. L.  _Managing Workplace Stress_ , vol. 1 (Sage, 1997).
  * [4] Pal, P.  Battling the physical symptoms of stress.  _Harvard Business Review_ (2016).
  * [5] Vaske, J. J.  Advantages and disadvantages of internet surveys: Introduction to the special issue.  _Human Dimensions of Wildlife_ 16, 149–153 (2011).
  * [6] Duda, M. D. & Nobile, J. L.  The fallacy of online surveys: No data are better than bad data.  _Human Dimensions of Wildlife_ 15, 55–64 (2010).
  * [7] Gigliotti, L. M.  Comparison of an internet versus mail survey: A case study.  _Human Dimensions of Wildlife_ 16, 55–62 (2011).
  * [8] Fricker, R. D. & Schonlau, M.  Advantages and disadvantages of internet research surveys: Evidence from the literature.  _Field methods_ 14, 347–367 (2002).
  * [9] Selye, H.  _The Stress of Life_ (McGraw-Hill, 1956).
  * [10] Lazarus, R. S.  Toward better research on stress and coping.  _American Psychological Association_ (2000).
  * [11] Colligan, T. W. & Higgins, E. M.  Workplace stress: Etiology and consequences.  _Workplace Behavioral Health_ 21, 89–97 (2006).
  * [12] Kobasa, S. C.  Stressful life events, personality, and health: An inquiry into hardiness.  _Personality and Social Psychology_ 37, 1 (1979).
  * [13] Muse, L., Harris, S. & Feild, H.  Has the Inverted-U Theory of Stress and Job Performance Had a Fair Test?  _Journal of Human Performance_ 16, 349–364 (2003).
  * [14] Wallis, C., Mehrtens, R. & Thompson, D.  Stress: can we cope?  _Time_ 121, 48–54 (1983).
  * [15] Crum, A. J., Salovey, P. & Achor, S.  Rethinking stress: The role of mindsets in determining the stress response.  _Personality and Social Psychology_ 104, 716 (2013).
  * [16] Cohen, S., Kamarck, T. & Mermelstein, R.  A global measure of perceived stress.  _Journal of Health and Social Behavior_ 385–396 (1983).
  * [17] Hammen, C.  Stress and depression.  _Annual Review of Clinical Psychology_ 1, 293–319 (2005).
  * [18] Wang, J.  Work stress as a risk factor for major depressive episode (s).  _Psychological medicine_ 35, 865–871 (2005).
  * [19] Park, C. L. & Helgeson, V. S.  Introduction to the special section: growth following highly stressful life events–current status and future directions.  _Journal of consulting and clinical psychology_ 74, 791 (2006).
  * [20] Tedeschi, R. G. & Calhoun, L. G.  Posttraumatic growth: conceptual foundations and empirical evidence.  _Psychological inquiry_ 15, 1–18 (2004).
  * [21] Guntuku, S. C., Buffone, A., Jaidka, K., Eichstaedt, J. C. & Ungar, L. H.  Understanding and measuring psychological stress using social media.  In _Proceedings of the International AAAI Conference on Web and Social Media_ , vol. 13, 214–225 (2019).
  * [22] Pennebaker, J. W., Francis, M. E. & Booth, R. J.  Linguistic inquiry and word count: Liwc 2001.  _Mahway: Lawrence Erlbaum Associates_ 71, 2001 (2001).
  * [23] Saha, K. & De Choudhury, M.  Modeling stress with social media around incidents of gun violence on college campuses.  _Proceedings of the ACM on Human-Computer Interaction_ 1, 1–27 (2017).
  * [24] Das Swain, V. _et al._ Modeling organizational culture with workplace experiences shared on glassdoor.  In _Proceedings of the ACM CHI Conference on Human Factors in Computing Systems (CHI)_ , 1–15 (2020).
  * [25] U.S. Bureau of Labor Statistics.  https://www.bls.gov.
  * [26] Yahoo Finance portal.  https://finance.yahoo.com.
  * [27] Šćepanović, S., Martín-López, E., Quercia, D. & Baykaner, K.  Extracting medical entities from social media.  In _Proceedings of the ACM Conference on Health, Inference, and Learning (CHIL)_ , 170–181 (2020).
  * [28] Aronson, A. R. & Lang, F.-M.  An overview of metamap: historical perspective and recent advances.  _Journal of the American Medical Informatics Association_ 17, 229–236 (2010).
  * [29] Tutubalina, E., Miftahutdinov, Z., Nikolenko, S. & Malykh, V.  Medical concept normalization in social media posts with recurrent neural networks.  _Journal of Biomedical Informatics_ (2018).
  * [30] Leaman, R. & Lu, Z.  Taggerone: joint named entity recognition and normalization with semi-markov models.  _Bioinformatics_ 32, 2839–2846 (2016).
  * [31] Šćepanović, S., Aiello, L. M., Barrett, D. & Quercia, D.  Epidemic dreams: dreaming about health during the covid-19 pandemic.  _Royal Society open science_ 9, 211080 (2022).
  * [32] Grootendorst, M.  Bertopic: leveraging bert and c-tf-idf to create easily interpretable topics.  _URL https://doi. org/10.5281/zenodo_ 4381785.
  * [33] Devlin, J., Chang, M.-W., Lee, K. & Toutanova, K.  Bert: Pre-training of deep bidirectional transformers for language understanding.  _arXiv preprint arXiv:1810.04805_ (2018).
  * [34] McInnes, L., Healy, J. & Melville, J.  Umap: Uniform manifold approximation and projection for dimension reduction.  _arXiv preprint arXiv:1802.03426_ (2018).
  * [35] McInnes, L., Healy, J. & Astels, S.  hdbscan: Hierarchical density based clustering.  _Journal of Open Source Software_ 2, 205 (2017).
  * [36] Grusky, D. B., Western, B. & Wimer, C.  The consequences of the great recession.  _The Great Recession_ 3–20 (2011).
  * [37] Gamestop shares surge 100% in a day, reddit group rejoices (2021).
  * [38] Naseem, U., Kim, J., Khushi, M. & Dunn, A. G.  Identification of disease or symptom terms in reddit to improve health mention classification.  In _Proceedings of the ACM Web Conference 2022_ , 2573–2581 (2022).
  * [39] Mehta, K. _et al._ Depression in the us population during the time periods surrounding the great recession.  _The Journal of clinical psychiatry_ 76, 4221 (2015).
  * [40] How to manage the great resignation (2021).
  * [41] Reading corporate culture from the outside (2022).
  * [42] Huang, Z., Xu, W. & Yu, K.  Bidirectional lstm-crf models for sequence tagging.  _arXiv preprint arXiv:1508.01991_ (2015).
  * [43] Straková, J., Straka, M. & Hajic, J.  Neural architectures for nested ner through linearization.  In _Proceedings of the Conference of the Association for Computational Linguistics_ , 5326–5331 (2019).
  * [44] Akbik, A., Bergmann, T. & Vollgraf, R.  Pooled contextualized embeddings for Named Entity Recognition.  In _Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1_ , 724–728 (2019).
  * [45] Pennington, J., Socher, R. & Manning, C.  GloVe: Global vectors for word representation.  In _Proceedings of the conference on empirical methods in natural language processing_ , 1532–1543 (Association for Computational Linguistics, 2014).
  * [46] Mikolov, T., Sutskever, I., Chen, K., Corrado, G. S. & Dean, J.  Distributed representations of words and phrases and their compositionality.  In _In Proceedings of the Advances in neural information processing systems conference_ , 3111–3119 (2013).
  * [47] Braun, V. & Clarke, V.  Using thematic analysis in psychology.  _Qualitative Research in Psychology_ 3, 77–101 (2006).

## Supplementary Information

Figure 4: Number of posts (log) in our dataset between 2008 and 2020s. There
is no trend difference between all posts and those containing mentions related
to stress.

## Details of the dataset

We collected a total of 713,018 posts published for a company reviewing site
from the start of 2008 up until the first quarter of 2020 for the S&P 500
companies. We filtered out posts belonging to non-US based companies, yielding
a total of 439,163 posts across $399$ unique S&P 500 companies. The average
rating across companies ranged from a minimum value of $1.62$ up to a maximum
value of $5$ ($\mu=3.37,\sigma=0.40)$. While the overall fraction of stress
posts per company was $1.11\%$, this value ranged from $0\%$ up to $9.52\%$
across companies ($\mu=1\%,\sigma=1\%)$.

## Data representatives

A total of 439,163 posts were analyzed. These posts are about companies
distributed across all the 51 U.S. states (Table 4). The highest number of
posts were found in California (i.e., a total of 69,968 posts), while the
lowest in Wyoming (i.e., a total of 222 posts). The posts span across 11
industries classified according to the Global Industry Classification Standard
(GICS), with the highest number of posts for companies in Information
Technology, and the least number in Real Estate (Table 5). The posts were
written by managers, sales associates, software engineers, analysts, among
others (Table 6). Current employees make up 56% of the reviews, and the
remaining reviews are predominantly by former employees who held the job
within the last five years. The maximum annual number of posts between 2008
and 2020 was observed in 2016, while the lowest number of posts in 2009
(Figure 4).

Table 4: Number of posts and number of offices on the company reviewing site
across U.S. States, ranked by the number of posts published between 2008 and
2020 in descending order. The state of California had the most published
posts, while the state of Wyoming had the least published posts. The Pearson
correlation between the log number of posts and the number of companies per
state in our data is $.98$, while the correlation between the log number of
posts in our data and the log of population size across states is $.93$.

U.S. State | # posts | # offices
---|---|---
CA | 69968 | 340
TX | 43629 | 342
NY | 37515 | 313
IL | 25157 | 290
FL | 24082 | 283
GA | 17888 | 275
WA | 15672 | 239
NC | 14072 | 268
PA | 14064 | 271
OH | 12447 | 263
MA | 12355 | 253
AZ | 11834 | 228
NJ | 11561 | 245
VA | 11320 | 235
CO | 9408 | 249
MN | 8437 | 196
MI | 7953 | 237
MO | 7657 | 230
TN | 7165 | 221
OR | 6704 | 206
MD | 6610 | 207
IN | 5727 | 222
WI | 5006 | 181
CT | 4567 | 186
KY | 4224 | 190
UT | 3925 | 187

U.S. State | # posts | # offices
---|---|---
OK | 3772 | 174
DC | 3596 | 180
KS | 3459 | 169
SC | 3362 | 194
NV | 2815 | 163
LA | 2631 | 175
AL | 2546 | 171
DE | 2067 | 113
IA | 1849 | 133
RI | 1676 | 103
AR | 1666 | 149
NH | 1627 | 127
NE | 1564 | 139
ID | 1208 | 112
MS | 1138 | 127
NM | 1097 | 125
WV | 803 | 114
ME | 604 | 105
HI | 600 | 85
ND | 344 | 74
VT | 324 | 54
MT | 322 | 70
AK | 300 | 57
SD | 297 | 57
WY | 222 | 72

Figure 5: Number of posts (log) in our dataset versus state population (log). The states Washington DC and Rhode Island have more posts that what the population size would suggest. The line of best linear fit is shown in gray. U.S. states are shown with the two-code state abbreviation. Table 5: Number of posts across the Global Industry Classification Standard (GICS) sectors. More posts are generally found in sectors having more companies, as one expects. GICS Sector | # posts | # companies
---|---|---
Information Technology | 63198 | 52
Consumer Discretionary | 62395 | 40
Financials | 49955 | 42
Health Care | 36308 | 41
Consumer Staples | 26471 | 28
Industrials | 24074 | 43
Communication Services | 13842 | 13
Energy | 5510 | 20
Materials | 5269 | 21
Utilities | 3172 | 19
Real Estate | 2228 | 16
Table 6: Number of posts across roles and statuses. Employee Title | # posts
---|---
Sales Associate | 8006
Manager | 4536
Software Engineer | 4191
Customer Service Representative | 4058
Cashier | 3726
Director | 2819
Project Manager | 2365
Senior Manager | 2225
Senior Software Engineer | 2019
Associate | 1969
Store Manager | 1963
Assistant Manager | 1930
Pharmacy Technician | 1747
Analyst | 1660
Delivery Driver | 1619
Employee Status | # posts
Current Employee | 190876
Former Employee | 146449
Former Intern | 5207
Former Contractor | 3380
Current Intern | 2858
Figure 6: Correlation between the number of headquarters in each state in the
Fortune 500 list and the number of headquarters in each state in our dataset
(Spearman $r=.90$). The states of Nebraska (NE) and Arizona (AR) have fewer
headquarters than what the Fortune list would suggests.

## Description and evaluation of the deep-learning framework

To extract stress mentions, we used the MedDL entity extraction module [27]
(the left rectangle in Figure 1(a)). MedDL uses _contextual embeddings_ and a
_BiLSTM-CRF sequence labeling architecture_. The BiLSTM-CRF architecture [42]
is the deep-learning method commonly employed for accurately extracting
entities from text [43, 44], and consists of two layers. The first layer is a
BiLSMT network (the dashed rectangle in Figure 1(a)), which stands for Bi-
directional Long Short-Term Memory (LSTM). The outputs of the BiLSTM are then
passed to the second layer: the CRF layer (enclosed in the other dashed
rectangle). The predictions of the second layer (the white squares in Figure
1(a)) represent the output of the entity extraction module. To extract the
medical entities of symptoms and drug names, BiLSTM-CRF takes as input
representations of words (i.e., embeddings). The most commonly used embeddings
are Global Vectors for Word Representation (GloVe) [45] and Distributed
Representations of Words (word2vec) [46]. However, these do not take into
account a word’s context. The word ‘pressure’, for example, could be a stress
symptom at the workplace (e.g., ‘I felt constant pressure to deliver results’)
or could be used in the physics context (e.g., ‘The solid material found in
the centre of some planets at extremely high temperature and pressure’). To
account for context, _contextual embeddings_ are generally used. MedDL used
the RoBERTa embeddings as it had outperformed several others contextual
embeddings, including ELMo, BioBert and Clinical BERT [27].

Our evaluation metric is $F1$ score, which is the harmonic mean of precision
$P$ and recall $R$:

$F1=2\frac{P\cdot R}{P+R}$ (8) $P=\frac{\\#\textrm{correctly classified
medical entities}}{\\#\textrm{total entities classified as being medical}}$

and

$R=\frac{\\#\textrm{correctly classified medical entities}}{\\#\textrm{total
medical entities}}.$

For strict F-1 score, we counted as “correctly classified” only the entities
that were exactly matching the ground truth labels. For relaxed version of F-1
score, partially matching entities are also counted as correctly classified
(e.g., if the model extracts the entity “ _pain_ ” given the full mention of “
_strong pain_ ”). Also, given that our data comes with class imbalance (i.e.,
text tokens do not correspond equally to symptoms, or non-medical entities),
we corrected for that by computing $P$ and $R$ using micro-averages [ash13].
In so doing, we were able to compare Med-DL’s F1 scores with those of two
well-known entity extraction tools: MetaMap and TaggerOne. MetaMap is a well-
established tool for extracting medical concepts from text using symbolic NLP
and computational-linguistic techniques [28], and has become a de-facto
baseline method for NLP studies related to health [29]. TaggerOne is a machine
learning tool using semi-Markov models to jointly perform two tasks: entity
extraction and entity normalization. The tool does so using a medical lexicon
[30]. The MedDL pre-trained model was evaluated on a labeled dataset of Reddit
posts called MedRed. The MedRed dataset was split into train (50%), dev (25%),
and test (25%) sets. The MedDL method achieved a strict/relaxed $F1$-score of
$.71$/$.85$ when extracting symptoms (Figure SI 7), outperforming both MetaMap
and TaggerOne by a large margin (the two have $F1$-scores of $.17/.48$ and
$.31/.58$, respectively).

Figure 7: MedDL strict/relaxed F-1 score results when extracting medical
symptoms on the MedRed dataset compared to two competitive alternatives of
MetaMap and TaggerOne.

(a)

(b)

Figure 8:  _(a)_ Distribution across companies of the logarithm of stock
growth values from the average stock price in 2014 and that of 2019
(${stock\\_growth}_{[14-19]}=stock_{2019}/stock_{2014}$) showing the stock
growth is log-normally distributed. The average stock price for year $y$
($stock_{y}$) is calculated as the average of the daily Adjusted Closing
Prices for the year. _(b)_ Geometric mean of the stock growth values
$\bar{GM}({stock\\_growth}_{[14-19]})$ for increasing stress score percentiles
for the companies of a given stress type. Error bars represent geometric
standard error $GSE({stock\\_growth}_{[14-19]})=$
$\bar{GM}({stock\\_growth}_{[14-19]})/$
$\sqrt{N}\cdot\sigma(log({stock\\_growth}_{[14-19]}))$. Figure 9: Geometric
mean of the stock growth values $\bar{GM}({stock}\mbox{ }{growth}_{[09,19]})$
for different ratings percentiles for companies of the four stress types.
Error bars represent geometric standard error $GSE({stock}\mbox{
}{growth}_{[09,19]})=$ $\bar{GM}({stock}\mbox{
}{growth}_{[09,19]})/\sqrt{N}\cdot\sigma(log({stock}\mbox{
}{growth}_{[09,19]}))$.

(a) Low stress

(b) Passive

(c) Negative stress

(d) Positive stress

Figure 10: The number of companies per industry sector for the four stress
types. IT is more prominent among positive stress companies, while Health Care
among negative stress companies.

(a)

(b)

(c)

Figure 11: Geometric mean of the stock growth values
$\bar{GM}({stock\\_growth}_{[09-19]})$ for increasing stress score percentiles
for the companies in each of the three most present industry sectors: (a)
Information Technology, (b) Consumer Discretionary, and (c) Health Care. The
three sectors have sufficient data to ensure statistical significance for each
percentile bin. Error bars represent geometric standard error
$GSE({stock\\_growth}_{[09-19]})=$ $\bar{GM}({stock\\_growth}_{[09-19]})/$
$\sqrt{N}\cdot\sigma(log({stock\\_growth}_{[09-19]}))$.

Annotations of the words BERTopic found. For each topic, we identified the
three most representative words and submitted the reviews mentioning them to
six annotators. For example, we picked three reviews containing the words
‘overtime’, ‘mandatory’, and ‘shift’ for negative stress companies, and asked
six annotators to read them and describe what type of workplaces these reviews
would suggest. Upon collecting a total of 72 free-form responses (i.e., each
annotator described the reviews corresponding to the 12 topics), we conducted
a thematic analysis [47]. To identify overarching themes, we used a
combination of open coding and axial coding. We first applied open coding to
identify key concepts. Specifically, one of the authors read the responses and
marked them with keywords. We then used axial coding to identify relationships
between the most frequent keywords to summarize them into semantically
cohesive themes.

We found three high-level themes: _career drivers_ , _industry or benefits_ ,
and _emotional aspects_. In the reviews, each theme was paraphrased
differently depending on the four types of company stress, allowing us to
identify sub-themes. The _career drivers_ theme described what motivated
employees to go to work. Its sub-themes concerned companies whose employees
experienced ‘considerable emotional pressure’ (negative stress), tended to
‘focus on activities outside the work’ (passive), cherished ‘their sense of
control over their work’ (low stress), and enjoyed ‘a collaborative and
supportive workplace culture’ (positive stress). In the _industry or benefits_
theme, we identified sub-themes mentioning either the industry sectors of the
corresponding companies (e.g., Consumer Discretionary for negative stress, and
Information Technology for positive stress) or aspects concerning long-term
financial benefits (e.g., passive and low stress). Finally, in the _emotional
aspects_ theme, we identified sub-themes suggesting employees who experienced
‘emotional pressure’ (negative stress), ‘tedious work’ (passive), ‘good work-
life balance’ (low stress), or a ‘fast-paced, high-performing, and dynamic
workplace environment’ (positive stress).

## Evaluation of BERTopic results

We ran the topic modeling algorithm BERTopic [32] separately on the four sets
of reviews (each set containing reviews of the companies of a given stress
type). The fact that BERTopic discovered distinct topics in the four sets
reveals that stress is paraphrased differently in the sets. We calculated the
topical overlapping values for the different combinations of the four sets
(using the Jaccard similarity on the sets of keywords from the top ten topics
of each stress type), and found them to be (on average) as low as 0.08 (on a
scale ranging from 0 to 1).

## Evaluation of the four quadrants

To test whether the quadrant division of companies into four types was
meaningful, we manually inspected 30 posts taken at random from companies with
high stress, and found stress mentions in companies with low ratings to be
qualitatively different from those in companies with high ratings (e.g., a
review from a lowly rated company _“The pressure is constantly high, while
your work is not appreciated […] and it feels like the managers do not know
what they are doing.”_ versus a review from a highly rated company _“Happy
Employee. Best culture I have experienced, especially in a stressful job. […]
The job is hard, but nothing worth having comes easy.”_). Similarly, we found
qualitatively different review between companies with low stress and high
versus low ratings (e.g., a review from a highly rated company _“Solid company
offering Work From Home. […] decent options to choose for hours worked, great
tech support, all equipment supplied, always feel connected to team, strong
work ethic. ”_ , versus a review from a lowly rated company _“Sinking Ship due
to Horribly Managed […] Merger. At legacy X office, they managed to retain
some of the positive company culture leftover from the X days. The people are
still the best part of that office, but with the increasing turnover, layoffs
and “Hunger Games” management style, that is in danger of ending… ”_). As a
final validity check, we arranged companies along the two axes and clustered
them in an unsupervised way. We found four to be the best number of clusters.
More specifically, we applied k-means clustering, and searched for the optimal
number of clusters using the elbow method (Figure 12). The method involves
calculating the sum of squared distances between data points and the $k$
assigned clusters’ centroids, for an increasing number of clusters $k$. Once
this value stops decreasing significantly, it means that that the optimal
number of clusters is reached.

Figure 12: Inertia of Cosine k-Means versus number of clusters having the
“elbow” at k=$4$.

## Sensitivity of the results

Weighting the scores. We explored the effects of weighting the yearly scores
in:

$\displaystyle m{(s,y)}=\sum_{c\in s}f(c,s,y)\times w{(c,y,s)},$ (9)

by plotting the temporal scores without weights, i.e., where $w=1$. The result
is shown in Figure 13. The simple aggregation skews the results towards (the
long tail of) small companies as it considers a small company equal to a big
one.

Figure 13: The effects of weighting the yearly scores. _(top)_ The evolution
of temporal scores without weights, i.e., where $w=1$ for the four types of
stress; and _(bottom)_ the unemployment rate in the U.S., with the horizontal
dashed line reflecting pre-recession rate. The stress score per year is
calculated using Equation (9) with $w=1$.

Shorter-term growth. To test whether our results on stock growth are not
affected by exogenous events such as the Great Recession, we computed stock
growth for the narrower 5-year period between 2014 to 2019:

${stock}\mbox{ }{growth}_{[14-19]}=\frac{stock^{2019}}{stock^{2014}}$ (10)

where $stock_{i}$ is the average adjusted closing price of their stocks in
year $i$. Figure 8 shows that the trend remains qualitatively the same as that
in Figure 2, even when removing the Great Recession period. Positive stress
companies enjoyed the highest stock growth (with average value across all
percentiles being $\bar{GM}(\textrm{stock growth}_{[14-19]})=1.97$ as per
Figure 8 on the right), low stress companies had the second highest
($\bar{GM}(\textrm{stock growth}_{[14-19]})=1.53)$, while passive and negative
stress companies enjoyed the lowest growth ($\bar{GM}({\textrm{stock
growth}_{[14-19]}})=1.46$, and $1.45$, respectively).

Interaction effects between stress scores and review ratings. We tested
whether our observed stock growth was genuinely associated with positive
stress companies rather than being simply associated with highly-rated
companies. To this end, for each stress type, we plotted
$\bar{GM}({stock\\_growth}_{[09-19]})$ against different rating percentiles
(Figure 9). Highly rated companies experienced stock growth, yet there are
still significant differences across companies of different stress types: in
particular, positive stress companies of varying rating percentiles
consistently enjoyed the highest growth (the yellow line in Figure 9 is
consistently above the other three lines).

Growth per industry sectors. To test whether a specific industry sector is
predominant for a given stress type, we first plotted the number of companies
per industry sector according to the GICS classification (Figure 10).
Information Technology was more prominent among positive stress and low stress
companies, Health Care and Financials among negative stress ones, and
Industrials and Consumer Discretionary among passive ones. To then check
whether the distribution of industry sectors across the four types of stress
affected our findings for stock growth, we computed stock growth between 2009
and 2019, and did so for the three most frequent industry sectors separately
(i.e., Information Technology, Consumer Discretionary, and Health Care). We
chose those three sectors because each individually contained a sufficient
number of companies and, as such, allowed us to obtain statistical significant
results.

Stock growth was computed as $GM({\textrm{stock
growth}_{[09-19]}})=\Pi(\textrm{stock growth}_{[09-19]}(c))^{1/n}$, where $c$
is each company from a given industry sector (e.g., Information Technology) in
a specific _(stress type,percentile)_ bin, and $n$ is the number of the
companies in such a bin. For the three industry sectors, we plotted
$\bar{GM}({stock\\_growth})$ against different stress score percentiles
(Figure 11). In all three sectors, we observed that positive stress companies
had consistently higher stock growth compared to the other three stress types.

Percentage of stress posts. To test the sensitivity of our results to the
percentage of stress posts being considered, we repeated our analyses by
including only the companies with at least $r$ reviews. We found the optimal
threshold $r$ to be $280$, and did so as follows. To include at least half of
the total S&P 500 companies, the least number of reviews per company had to be
less than $r=350$. Then, for each $r=1,...,350$, we subset the companies
having at least $r$ reviews, and calculated the correlation between a
company’s rating and its positive stress score (for positive stress companies)
or its negative stress score (for negative stress companies), and did so for
each subset. We found that the absolute values of the correlations increased
with the number of reviews (Figure 14), as one expected, and there was a phase
shift at $r=280$ for positive stress companies ($\rho($company_rating,
positive_stress_association)=$.75$). The same applied to negative stress
companies (Figure 14). At this threshold, we were left with $287$ companies
out of $380$ companies in total. We repeated the calculations on this subset
of companies and, compared to our previously reported results, found even
stronger associations between: i) negative stress scores in the whole U.S. and
the Great Recession, and ii) a company’s positive stress score and its stock
growth.

Figure 14: Threshold selection. Correlation values between each of the two
stress scores and a company’s website overall rating ($y$-axis) for the
companies with at least $r$ reviews ($x$-axis). These values have a phase
shift at $r=280$ for positive stress companies (blue), matching the value of
the correlation for negative stress companies (red).

Combining stress and review scores. We fit three Ordinary Least Squares (OLS)
models to predict stock growth (Figure 15). In each model, we used the (log of
the) number of reviews as a control variable. In addition, (a) $M_{r}$ uses
the average rating score as the additional independent variable (baseline
model), (b) $M_{s}$ uses the stress score, and (c) $M_{r+s}$ uses both the
rating score and the stress score as additional independent variables. We
applied bootstrapping to ascertain the statistical significance of the results
by randomly subsampling a set of 120 companies 10 times. We observed a 78% and
a 192% increase in $M_{s}$ and in $M_{r+s}$ over the baseline model,
respectively.

Figure 15: Adjusted $R^{2}$ values of three OLS models with different
predictors: r is the rating score; s is the stress score; r+s is the rating
score and stress score. We applied bootstrapping to ascertain the statistical
significance of the results by randomly subsample a set of 120 companies 10
times. Average values and standard deviations are reported. We observed a 78%
and a 192% increase in $M_{s}$ and in $M_{r+s}$ over the baseline model,
respectively.