EXPLICIT ESTIMATES FOR THE STIRLING NUMBERS OF THE SECOND KIND ††footnotetext:
2020 Mathematics Subject Classification. 05A18, 60E05.  Keywords and Phrases.
Stirling numbers of the second kind, explicit estimates, geometric
distribution, exponential distribution

José A. Adell

We give explicit estimates for the Stirling numbers of the second kind
$S(n,m)$. With a few exceptions, such estimates are asymptotically sharp. The
form of these estimates varies according to $m$ lying in the central or non-
central regions of $\\{1,\ldots,n\\}$. In each case, we use a different
probabilistic representation of $S(n,m)$ in terms of well known random
variables to show the corresponding results.

##  1 Introduction

The Stirling numbers of the second kind $S(n,m)$ can be defined in various
equivalent ways (cf. Comtet (Com1974, , Chap. 5)). For instance,

(1) $z^{n}=\sum_{m=0}^{n}S(n,m)(z)_{m},\quad z\in\mathds{C},$

where $(z)_{m}=z(z-1)\cdots(z-m+1)$, $m=1,2,\ldots$ is the falling factorial
($(z)_{0}=1$) or, via their generating function, as

(2)
$\dfrac{(e^{z}-1)^{m}}{m!}=\sum_{n=m}^{\infty}S(n,m)\dfrac{z^{n}}{n!},\quad
z\in\mathds{C}.$

Since these numbers play an important role in different branches of
mathematics, one can find in the literature many papers devoted to obtain
manageable formulae for them. Such formulae date back at least to Jordan
Jor1991 , who obtained

(3) $S(n,m)\sim\dfrac{m^{n}}{m!},\qquad S(n,n-m)\sim\dfrac{n^{2m}}{2^{m}m!}.$

However, such estimates are acceptable only for $m$ quite small in comparison
with $n$. Asymptotic expansions for $S(n,m)$, as $n\to\infty$, according to
$m$ lying in the central or non-central regions of $\\{1,2,\ldots,n\\}$ have
been provided by many authors. In this regard, Sachkov (Sac1997, , p. 144)
obtained

$S(n,m)=\dfrac{m^{n}}{m!}\exp\left(\left(\dfrac{n}{m}-m\right)e^{-n/m}\right)(1+o(1)),\quad
m<n/\log n.$

Hsu Hsu1948  and Chelluri et al. CheRicTem2000  gave

$S(n,m)=\dfrac{n^{2(n-m)}}{2^{n-m}(n-m)!}\left(1+O(n^{-1/3})\right),\quad
n-n^{1/3}\leq m\leq n.$

Moser and Wyman MosWym1958  and Chelluri et al. CheRicTem2000  showed that

(4) $S(n,m)=\dfrac{n!(e^{R}-1)^{m}}{2R^{n}m!\sqrt{\pi
mRH}}\left(1+O(1/m)\right),\quad 0<\delta\leq m\leq n-n^{1/3},$

where $R$ is the solution to the equation

$R(1-e^{-R})^{-1}=\dfrac{n}{m},$

and

$H=e^{R}(e^{R}-1-R)/2(e^{R}-1)^{2}.$

More specific formulae for $S(n,n-n^{a})$, for $1/2<a<1$, can be found in
Louchard Lou2013 .

It seems that less attention has been paid to obtain explicit upper and lower
bounds for $S(n,m)$. As far as we know, Moser and Wyman MosWym1958  provided,
whenever $m=n-o(\sqrt{n})$,

(5)
$S(n,m)=\binom{n}{m}q^{-(n-m)}\left(\sum_{k=0}^{s}A_{k}(n-m)q^{k}+E_{s}\right),\quad
q=\dfrac{2}{n-m},$

where $0\leq s\leq n-m-1$, $A_{k}(r)$ is a polynomial in $r$ for each
$k=1,2,\ldots$, and

$|E_{s}|\leq\dfrac{(2(n-m)^{2}/5m)^{s+1}}{1-(2(n-m)^{2})/5m}.$

As upper and lower bounds with no essential restriction on $m$, we mention
that Rennie and Dobson RenDob1969  gave

$\dfrac{1}{2}(m^{2}+m+2)m^{n-m-1}-1\leq
S(n,m)\leq\dfrac{1}{2}\binom{n}{m}m^{n-m},\quad 1\leq m\leq n-1.$

More recently, using Stein-Chen Poisson approximation, Arratia and DeSalvo
ArrDeS2017  showed that

(6) $\binom{N}{m}e^{-2\mu}(1-e^{2\mu}D_{n,m})\leq
S(n,n-m)\leq\binom{N}{n}e^{-2\mu}(1+e^{2\mu}D_{n,m}),$

for $n\geq 3$ and $n\geq m\geq 2$, where

$N=\binom{n}{2},\quad\mu=\dfrac{\binom{m}{2}\binom{n}{3}}{\binom{N}{2}},$

and $D_{n,m}$ is an error term uniformly bounded by 1 which goes to zero if
$m=O(\sqrt{n})$. We emphasize that estimates (5) and (6) are asymptotically
sharp provided that $m=n-o(\sqrt{n})$ and $m=O(\sqrt{n})$, respectively.

The aim of this paper is twofold. On the one hand, to give explicit estimates
for $S(n,m)$ in the whole range of $m$. Moreover, we show in Section 5 that
our estimates are asymptotically sharp, excepting when $n-m\sim
m\log(mr_{m})$, with $r_{m}\sim r\in(0,\infty)$, case in which we only are
able to give upper an lower bounds of the same order of magnitude.

On the other hand, to open the door to possible extensions to more general
numbers, such as the Comtet numbers of the second kind. In fact, many of the
results reviewed above are based on formula (2), which seems difficult to
extend to other numbers. However, our starting point is the well known formula

(7)
$\sum_{j=m}^{\infty}S(j,m)z^{j}=\dfrac{z^{m}}{(1-z)(1-2z)\cdots(1-mz)},\quad
z\in\mathds{C},\quad|z|<1/m,$

which has a direct generalization to the Comtet numbers of the second kind
(see, for instance, Comtet Com1972  and El-Desouky et al. ElDCakShi2017 ).

To achieve our results, we give different probabilistic representations of
$S(n,m)$, whose usefulness depends on the values of $m$. Such representations,
written in terms of sums of independent random variables involving the
geometric and the exponential distributions, may be of independent interest.
Among them, we highlight the following

(8) $S(n,m)=\dfrac{m^{n}}{m!}P(V_{m-1}\leq n-m),\quad 2\leq m\leq n,$

where $V_{m-1}=X(1/m)+\cdots+X((m-1)/m)$ is a sum of $m-1$ independent random
variables such that $X(j/m)$ has the geometric distribution with failure
probability $j/m$ (a similar representation in terms of multinomial laws was
provided in AdeCar2021 ). Having in mind the first asymptotic formula in (3),
representation (8) gives explicit and sharp estimates for $S(n,m)$ for small
values of $m$ in a very simple way (see Theorem 3.2 in Section 3).

This would be the case for other values of $m$, if the random variable
$V_{m-1}$, properly standardized, would satisfy the central limit theorem.
Unfortunately, the limiting random variable $V$ is rather involved and does
not allow us to give a unified treatment for any value of $m$. For these
reasons, we provide different probabilistic representations of $S(n,m)$
according to $m$ lying in the central or non-central regions of
$\\{1,2,\ldots,n\\}$.

The paper is organized as follows. In the following section, we collect the
aforementioned probabilistic representations for $S(n,m)$. In Sections 3 and
4, we estimate $S(n,m)$ for $m$ in the non-central and in the central regions,
respectively, whereas Section 5 is devoted to discuss in detail the accuracy
of our estimates. The main results are stated in Theorems 3.1, 3.2, and 4.1.

##  2 Probabilistic representations

Let $\mathds{N}$ be the set of positive integers and
$\mathds{N}_{0}=\mathds{N}\cup\\{0\\}$. Throughout this paper, we always
assume that $n,m\in\mathds{N}$ with $m\leq n$ and $z\in\mathds{C}$. The
indicator function of the set $A$ is denoted by $1_{A}$, whereas $\mathds{E}$
stands for mathematical expectation. Unless otherwise specified, all of the
random variables appearing in a same expression are supposed to be mutually
independent.

To give suitable probabilistic representations of $S(n,m)$, we consider the
following random variables. In the first place, let $T$ be a random variable
having the exponential density $\rho(\theta)=e^{-\theta}$, $\theta\geq 0$. It
is well known that

(9)
$\mathds{E}T=\text{Var}(T)=1,\qquad\mathds{E}e^{zT}=\dfrac{1}{1-z},\quad|z|<1.$

If $(T_{j})_{1\leq j\leq m}$ is a finite sequence of independent copies of
$T$, we denote

(10) $S_{m}=T_{1}+2T_{2}+\cdots+mT_{m}.$

In the second place, let $X(q)$ be a random variable having the geometric
distribution with failure probability $q\in(0,1)$, that is,

(11) $P(X(q)=k)=pq^{k},\quad k\in\mathds{N}_{0},\quad p=1-q.$

It is known that

(12)
$\mathds{E}X(q)=\dfrac{q}{p},\quad\text{Var}(X(q))=\dfrac{q}{p^{2}},\quad\mathds{E}z^{X(q)}=\dfrac{p}{1-qz},\quad|z|<1/q.$

If $(X(q_{j}))_{1\leq j\leq m}$ is a finite sequence of independent random
variables such that each $X(q_{j})$ has the geometric distribution as in (11),
we denote

(13) $Y_{m}=X(q_{1})+\cdots+X(q_{m}).$

The following auxiliary result will be needed.

######  Lemma 2.1.

Let $f:\mathds{N}_{0}\to\mathds{R}$ be a bounded function and let $a>0$ be
such that $aq_{j}<1$, $j=1,\ldots,m$. Then,

$\mathds{E}a^{Y_{m}}f(Y_{m})=\mathds{E}a^{Y_{m}}\mathds{E}f(X(aq_{1})+\cdots+X(aq_{m})).$

###### Proof.

Set $Y_{m}=Y_{m-1}+X(q_{m})$. By (12) and the independence of the random
variables involved, we have

$\begin{split}&\dfrac{\mathds{E}a^{Y_{m}}f(Y_{m})}{\mathds{E}a^{Y_{m}}}=\dfrac{1}{\mathds{E}a^{Y_{m-1}}\mathds{E}a^{X(q_{m})}}\mathds{E}a^{Y_{m-1}+X(q_{m})}f(Y_{m-1}+X(q_{m}))\\\
&=\dfrac{1}{\mathds{E}a^{Y_{m-1}}}\,\dfrac{1-aq_{m}}{1-q_{m}}\sum_{k=0}^{\infty}(1-q_{m})(aq_{m})^{k}\mathds{E}a^{Y_{m-1}}f(Y_{m-1}+k)\\\
&=\dfrac{\mathds{E}a^{Y_{m-1}}f(Y_{m-1}+X(aq_{m}))}{\mathds{E}a^{Y_{m-1}}},\end{split}$

since $Y_{m-1}$ and $X(aq_{m})$ are independent. Thus, the conclusion follows
by induction on $m$. ∎

Denote

(14) $W_{m}(q)=X(q_{1})+\cdots+X(q_{m}),\quad q_{j}=\dfrac{qj}{m},\quad
j=1,\ldots,m,\quad 0<q<1.$

Various probabilistic representations of $S(n,m)$ in terms of the random
variables considered in (10) and (14) are provided in the following result.

######  Theorem 2.2.

For any $0<q<1$, we have

$\begin{split}S(n,m)&=\dfrac{\mathds{E}S_{m}^{n-m}}{(n-m)!}=\dfrac{m^{n}q^{m-n}}{\prod_{j=1}^{m}(m-qj)}\dfrac{1}{2\pi}\int_{-\pi}^{\pi}\mathds{E}e^{i\theta(W_{m}(q)-n+m)}\,d\theta\\\
&=\dfrac{m^{n}}{m!}P\left(X\left(1/m\right)+\cdots+X\left((m-1)/m\right)\leq
n-m\right),\end{split}$

where the last equality holds for $m\geq 2$.

###### Proof.

The first equality was shown in (AdeLek2019, , Thm. 3.1). We give here a short
proof of it for the sake of completeness. Starting from (7) and using (9) and
(10), we have for $|z|<1/m$

$\sum_{n=m}^{\infty}S(n,m)z^{n}=z^{m}\mathds{E}e^{zS_{m}}=z^{m}\sum_{j=0}^{\infty}\dfrac{\mathds{E}S_{m}^{j}}{j!}z^{j}=\sum_{n=m}^{\infty}\dfrac{\mathds{E}S_{m}^{n-m}}{(n-m)!}z^{n}.$

It therefore suffices to equate coefficients in this expression.

On the other hand, applying (7) with $z=qe^{i\theta}/m$ and recalling (12) and
(14), we see that

$\begin{split}&\sum_{k=m}^{\infty}S(k,m)\left(\dfrac{q}{m}\right)^{k}e^{i\theta
k}=\left(\dfrac{q}{m}\right)^{m}\dfrac{e^{i\theta
m}}{\displaystyle\prod_{j=1}^{m}\left(1-qj/m\right)}\prod_{j=1}^{m}\dfrac{1-qj/m}{1-qje^{i\theta}/m}\\\
&=\left(\dfrac{q}{m}\right)^{m}\dfrac{1}{\displaystyle\prod_{j=1}^{m}\left(1-qj/m\right)}\mathds{E}e^{i\theta(W_{m}(q)+m)}.\end{split}$

Multiplying both sides of this identity by $e^{-i\theta n}/2\pi$ and then
integrating over $[-\pi,\pi]$, we obtain the second equality in Theorem 2.2.

Finally, suppose that $m\geq 2$. Write (14) as

(15) $W_{m}(q)=Y_{m-1}+X(q),\qquad Y_{m-1}=X(q_{1})+\cdots+X(q_{m-1}).$

Since $W_{m}(q)$ is an $\mathds{N}_{0}$-valued random variable, we have (see,
for instance, Petrov (Pet1995, , p. 15))

(16)
$\begin{split}&\dfrac{1}{2\pi}\int_{-\pi}^{\pi}\mathds{E}e^{i\theta(W_{m}(q)-n+m)}\,d\theta=P(W_{m}(q)=n-m)\\\
&=\sum_{k=0}^{n-m}P(Y_{m-1}=k)P(X(q)=n-m-k)\\\
&=pq^{n-m}\sum_{k=0}^{n-m}\left(\dfrac{1}{q}\right)^{k}P(Y_{m-1}=k)=pq^{n-m}\mathds{E}\left(\dfrac{1}{q}\right)^{Y_{m-1}}1_{[0,n-m]}(Y_{m-1})\\\
&=pq^{n-m}\mathds{E}\left(\dfrac{1}{q}\right)^{Y_{m-1}}P\left(X\left(1/m\right)+\cdots+X\left((m-1)/m\right)\leq
n-m\right),\end{split}$

where the last equality follows from (15) and Lemma 2.1 with $a=q^{-1}$ and
$f=1_{[0,n-m]}$. On the other hand, it follows from (12) and (15) that

$\mathds{E}\left(\dfrac{1}{q}\right)^{Y_{m-1}}=\prod_{j=1}^{m-1}\dfrac{1-qj/m}{1-j/m}=\dfrac{1}{(m-1)!}\prod_{j=1}^{m-1}(m-qj).$

This, together with (16) and the second equality in Theorem 2.2, shows the
third one and completes the proof. ∎

For other probabilistic representations of $S(n,m)$, we refer the reader to
AdeCar2021  and AdeLek2019 .

##  3 Estimates in the non-central regions

Let $S_{m}$ be the random variable defined in (10). By (9), we have

(17)
$\begin{split}&\nu_{m}:=\mathds{E}S_{m}=\sum_{j=1}^{m}j=\dfrac{m(m+1)}{2},\\\
&\tau_{m}^{2}:=\text{Var}(S_{m})=\sum_{j=1}^{m}j^{2}=\dfrac{m(m+1)(2m+1)}{6}.\end{split}$

With these ingredients, our first main result is the following.

######  Theorem 3.1.

Assume that

(18) $3\leq n-m\leq\dfrac{\nu_{m}}{2\tau_{m}}.$

Then,

$\left|\dfrac{(n-m)!}{\nu_{m}^{n-m}}S(n,m)-1-\binom{n-m}{2}\dfrac{\tau_{m}^{2}}{\nu_{m}^{2}}\right|\leq
2e^{1/4}\left(\dfrac{2(n-m)\tau_{m}}{\nu_{m}}\right)^{3}.$

###### Proof.

Consider the standardized random variable

$Z_{m}=\dfrac{S_{m}-\nu_{m}}{\tau_{m}},$

which obviously satisfies $\mathds{E}Z_{m}=0$ and
$\text{Var}(Z_{m})=\mathds{E}Z_{m}^{2}=1$. We write

(19)
$\begin{split}&\mathds{E}S_{m}^{n-m}=\mathds{E}(\tau_{m}Z_{m}+\nu_{m})^{n-m}=\sum_{j=0}^{n-m}\binom{n-m}{j}\tau_{m}^{j}\mathds{E}Z_{m}^{j}\nu_{m}^{n-m-j}\\\
&=\nu_{m}^{n-m}\left(1+\binom{n-m}{2}\dfrac{\tau_{m}^{2}}{\nu_{m}^{2}}+\sum_{j=3}^{n-m}\binom{n-m}{j}\left(\dfrac{\tau_{m}}{\nu_{m}}\right)^{j}\mathds{E}Z_{m}^{j}\right).\end{split}$

By assumption (18), the modulus of the sum on the right-hand side in (19) is
bounded above by

(20)
$\begin{split}&\sum_{j=3}^{n-m}\left(\dfrac{2(n-m)\tau_{m}}{\nu_{m}}\right)^{j}\dfrac{\mathds{E}(|Z_{m}|/2)^{j}}{j!}\leq\left(\dfrac{2(n-m)\tau_{m}}{\nu_{m}}\right)^{3}\mathds{E}e^{|Z_{m}|/2}\\\
&\leq\left(\dfrac{2(n-m)\tau_{m}}{\nu_{m}}\right)^{3}\left(\mathds{E}e^{Z_{m}/2}+\mathds{E}e^{-Z_{m}/2}\right).\end{split}$

Using the identity

$-\log(1-z)-z=z^{2}g(z),\quad|g(z)|\leq 1,\quad|z|\leq 1/2,$

we have from (9), (10), and (17)

(21)
$\begin{split}&\mathds{E}e^{Z_{m}/2}=\prod_{j=1}^{m}\exp\left(\dfrac{j}{2\tau_{m}}(T_{j}-1)\right)\\\
&=\prod_{j=1}^{m}\exp\left(-\log\left(1-\dfrac{j}{2\tau_{m}}\right)-\dfrac{j}{2\tau_{m}}\right)\leq\prod_{j=1}^{m}\exp\left(\dfrac{j^{2}}{4\tau_{m}^{2}}\right)=e^{1/4}.\end{split}$

Similarly,

$\mathds{E}e^{-Z_{m}/2}\leq e^{1/4}.$

In view of the first equality in Theorem 2.2, the conclusion follows from
(19)–(21). ∎

The assumption $n-m\geq 3$ in (18) is not essential. Actually, if $n-m=1$ or
$n-m=2$, the first equality in Theorem 2.2 and (19) give us, respectively,

$\dfrac{S(n,n-1)}{\nu_{n-1}}=1,\quad\dfrac{2}{\nu_{n-2}^{2}}S(n,n-2)=1+\dfrac{\tau_{n-2}^{2}}{\nu_{n-2}^{2}}.$

Recall that the generalized harmonic numbers are defined as

(22) $H_{m,r}=\sum_{j=1}^{m}\dfrac{1}{j^{r}},\quad m,r\in\mathds{N}.$

We simply denote by $H_{m}:=H_{m,1}$ the harmonic numbers. Our second main
result is the following.

######  Theorem 3.2.

We have

$S(n,m)\leq\dfrac{m^{n}}{m!}.$

In addition, if $m\geq 2$ and $n\geq mH_{m}$, then

(23)
$\dfrac{m^{n}}{m!}\left(1-\dfrac{m^{2}H_{m,2}-mH_{m}}{(n-mH_{m}+1)^{2}}\right)\leq
S(n,m)\leq\dfrac{m^{n}}{m!}.$

###### Proof.

Since the first statement is an immediate consequence of the third equality in
Theorem 2.2, we will only show the second one. In the setting of Theorem 2.2,
denote

$V_{m-1}=X\left(1/m\right)+\cdots+X\left((m-1)/m\right),\quad m\geq 2.$

By (12), we see that

$\mathds{E}V_{m-1}=\sum_{j=1}^{m-1}\dfrac{j/m}{1-j/m}=\sum_{j=1}^{m-1}\dfrac{m}{m-j}-m+1=mH_{m-1}-m+1,$

as well as

(24)
$\begin{split}&\text{Var}(V_{m-1})=\sum_{j=1}^{m-1}\dfrac{j/m}{(1-j/m)^{2}}\\\
&=\sum_{j=1}^{m-1}\dfrac{m^{2}}{(m-j)^{2}}-\sum_{j=1}^{m-1}\dfrac{m}{m-j}=m^{2}H_{m,2}-mH_{m}.\end{split}$

Hence, by Chebyshev’s inequality, we have

$\begin{split}&P(V_{m-1}\geq n-m+1)=P(V_{m-1}-\mathds{E}V_{m-1}\geq
n-mH_{m-1})\\\
&\leq\dfrac{\text{Var}(V_{m-1})}{(n-mH_{m-1})^{2}}=\dfrac{m^{2}H_{m,2}-mH_{m}}{(n-mH_{m}+1)^{2}}.\end{split}$

Thus, the conclusion follows from the third equality in Theorem 2.2. ∎

##  4 Estimates in the central region

In this section, we assume that $m\geq 2$. Looking at (14), we choose
$q=q(n,m)\in(0,1)$ the unique solution to the equation

(25)
$n-m=\mathds{E}W_{m}(q)=\sum_{j=1}^{m}\dfrac{qj/m}{1-qj/m}=\sum_{j=1}^{m-1}\dfrac{qj/m}{1-qj/m}+\dfrac{q}{1-q},$

as follows from (12). Note that this failure probability $q=q(n,m)$ is well
defined, because the function $q\to\mathds{E}W_{m}(q)$, $0<q<1$, is strictly
increasing and goes from 0 to $\infty$. With this choice, we denote

(26) $W_{m}(q)=W_{m-1}(q)+X(q),\quad W_{m-1}(q)=X(q_{1})+\cdots+X(q_{m-1}).$

The variances of such random variables are denoted by

(27)
$\sigma_{m}^{2}(q)=\text{Var}(W_{m}(q)),\quad\sigma_{m-1}^{2}=\text{Var}(W_{m-1}(q)).$

By (12) and (26), we see that

(28)
$\sigma_{m}^{2}=\sigma_{m-1}^{2}+\dfrac{q}{(1-q)^{2}},\quad\sigma_{m-1}^{2}=\sum_{j=1}^{m-1}\dfrac{qj/m}{(1-qj/m)^{2}}.$

With these notations, we state our third main result.

######  Theorem 4.1.

Let $q$ be as in (25) and $p=1-q$. If $m\geq 2$, then

$\left|\dfrac{\prod_{j=1}^{m}(m-qj)}{m^{n}q^{n-m}}S(n,m)-\dfrac{1}{\sigma_{m}\sqrt{2\pi}}\right|\leq\dfrac{2+9\sqrt{2\pi}}{2\pi
p\sigma_{m}^{2}}+\dfrac{6\sqrt{2}}{\pi p\sigma_{m-1}\sigma_{m}}.$

In Lemma 5.1 below, we discuss the solution $q$ to equation (25) and give
sharp estimates of $\sigma_{m-1}$. The applicability of Theorem 4.1 is
considered in Cases 3 and 4 in the following section.

The proof of Theorem 4.1 is based upon the following three auxiliary results.
In the first place, for any $k\in\mathds{N}$, let $\beta_{k}$ be a random
variable having the beta density $\rho_{k}(x)=k(1-x)^{k-1}$, $0\leq x\leq 1$.
The Taylor’s formula with remainder in integral form can be written as

(29)
$e^{z}=\sum_{j=0}^{k-1}\dfrac{z^{j}}{j!}+\dfrac{z^{k}}{k!}\mathds{E}e^{z\beta_{k}},\quad
k\in\mathds{N}.$

We always assume that any random variable $\beta_{k}$ is independent of all
other random variables appearing in a same formula.

######  Lemma 4.2.

In the setting of Theorem 4.1, we have for any $\theta\in\mathds{R}$

$\mathds{E}e^{i\theta(W_{m}(q)-\mathds{E}W_{m}(q))}=\exp\left(-\dfrac{\sigma_{m}^{2}\theta^{2}}{2}-ia_{m}(\theta)\theta^{3}\right),$

where

(30) $|a_{m}(\theta)|\leq\dfrac{\sigma_{m}^{2}}{3p}.$

###### Proof.

Let $q_{j}=qj/m$ and $p_{j}=1-q_{j}$, $j=1,\ldots,m$. From (12), we have

(31)
$\mathds{E}e^{i\theta(X(q_{j})-\mathds{E}X(q_{j}))}=\exp\left(-\log(1-q_{j}e^{i\theta})+\log(1-q_{j})-i\theta\dfrac{q_{j}}{1-q_{j}}\right).$

With the help of (29), we write the exponent in (31) as

(32) $\begin{split}&\sum_{k=1}^{\infty}\dfrac{q_{j}^{k}}{k}(e^{i\theta
k}-1)-i\theta\sum_{k=1}^{\infty}q_{j}^{k}=\dfrac{1}{p_{j}}\sum_{k=1}^{\infty}p_{j}q_{j}^{k}\dfrac{e^{i\theta
k}-1-i\theta k}{k}\\\ &=\dfrac{1}{p_{j}}\mathds{E}\left(\dfrac{e^{i\theta
X(q_{j})}-1-i\theta X(q_{j})}{X(q_{j})}\right)\\\
&=\dfrac{1}{p_{j}}\left(-\dfrac{\theta^{2}}{2}\mathds{E}X(q_{j})-i\dfrac{\theta^{3}}{6}\mathds{E}(X(q_{j}))^{2}e^{i\theta
X(q_{j})\beta_{3}}\right)\\\
&=-\dfrac{\theta^{2}}{2}\dfrac{q_{j}}{p_{j}^{2}}-i\theta^{3}\dfrac{1}{6p_{j}}\mathds{E}(X(q_{j}))^{2}e^{i\theta
X(q_{j})\beta_{3}}.\end{split}$

By (27), (31), and (32), we get

(33)
$\begin{split}&\mathds{E}e^{i\theta(W_{m}(q)-\mathds{E}W_{m}(q))}=\prod_{j=1}^{m}\mathds{E}e^{i\theta(X(q_{j})-\mathds{E}X(q_{j}))}\\\
&=\exp\left(-\dfrac{\theta^{2}}{2}\sigma_{m}^{2}-i\theta^{3}a_{m}(\theta)\right),\end{split}$

where

$a_{m}(\theta)=\dfrac{1}{6}\sum_{j=1}^{m}\dfrac{1}{p_{j}}\mathds{E}(X(q_{j}))^{2}e^{i\theta
X(q_{j})\beta_{3}}.$

Since $p_{j}\geq p_{m}=p$, $j=1,\ldots,m$, we have from (12) and (28)

$\begin{split}&|a_{m}(\theta)|\leq\dfrac{1}{6p}\sum_{j=1}^{m}\mathds{E}(X(q_{j}))^{2}=\dfrac{1}{6p}\sum_{j=1}^{m}\left(\text{Var}(X(q_{j}))+(\mathds{E}X(q_{j}))^{2}\right)\\\
&=\dfrac{1}{6p}\sum_{j=1}^{m}\left(\dfrac{q_{j}}{p_{j}^{2}}+\dfrac{q_{j}^{2}}{p_{j}^{2}}\right)\leq\dfrac{\sigma_{m}^{2}}{3p}.\end{split}$

This, together with (33), shows the result. ∎

Let $Z$ be a random variable having the standard normal density

(34) $\rho(u)=\dfrac{1}{\sqrt{2\pi}}e^{-u^{2}/2},\quad u\in\mathds{R}.$

The even moments of $Z$ are given by

(35) $\mathds{E}Z^{2k}=\dfrac{(2k)!}{k!2^{k}},\quad k\in\mathds{N}_{0}.$

######  Lemma 4.3.

In the setting of Theorem 4.1, we have

$\left|\dfrac{1}{2\pi}\int_{|\theta|\leq
p}\mathds{E}e^{i\theta(W_{m}(q)-\mathds{E}W_{m}(q))}\,d\theta-\dfrac{1}{\sigma_{m}\sqrt{2\pi}}\right|\leq\dfrac{2+9\sqrt{2\pi}}{2\pi
p\sigma_{m}^{2}}.$

###### Proof.

By (34) and Lemma 4.2, we have

(36) $\begin{split}&\int_{|\theta|\leq
p}\mathds{E}e^{i\theta(W_{m}(q)-\mathds{E}W_{m}(q))}\,d\theta-\dfrac{\sqrt{2\pi}}{\sigma_{m}}\\\
&=\int_{|\theta|\leq
p}e^{-\sigma_{m}^{2}\theta^{2}/2-ia_{m}(\theta)\theta^{3}}\,d\theta-\int_{-\infty}^{\infty}e^{-\sigma_{m}^{2}\theta^{2}/2}\,d\theta\\\
&=\int_{|\theta|>p}e^{-\sigma_{m}^{2}\theta^{2}/2}\,d\theta+\int_{|\theta|\leq
p}e^{-\sigma_{m}^{2}\theta^{2}/2}\left(e^{-ia_{m}(\theta)\theta^{3}}-1\right)\,d\theta:=I+II.\end{split}$

Observe that

(37) $I\leq\dfrac{2}{p}\int_{p}^{\infty}\theta
e^{-\sigma_{m}^{2}\theta^{2}/2}\,d\theta=\dfrac{2}{p\sigma_{m}^{2}}e^{-(p\sigma_{m})^{2}/2}\leq\dfrac{2}{p\sigma_{m}^{2}}.$

On the other hand, using the inequality $|e^{z}-1|\leq|z|e^{|z|}$, we have
from (30)

$\begin{split}&|II|\leq\dfrac{\sigma_{m}^{2}}{3p}\int_{|\theta|\leq
p}|\theta|^{3}e^{-\sigma_{m}^{2}\theta^{2}/2}\,d\theta\leq\dfrac{3\sqrt{2\pi}}{p\sigma_{m}^{2}}\mathds{E}|Z|^{3}\\\
&\leq\dfrac{3\sqrt{2\pi}}{p\sigma_{m}^{2}}(\mathds{E}Z^{4})^{3/4}=\dfrac{3^{7/4}\sqrt{2\pi}}{p\sigma_{m}^{2}}\leq\dfrac{9\sqrt{2\pi}}{p\sigma_{m}^{2}}.\end{split}$

where we have used (35) and Hölder’s inequality. This, in conjunction with
(36) and (37), shows the result. ∎

In the following auxiliary result, we will need the identity

(38)
$\left(\sum_{j=1}^{m}x_{j}\right)^{2}=\sum_{j=1}^{m}x_{j}^{2}+2\sum_{1\leq
i<j\leq m}x_{i}x_{j},\quad x_{1},\ldots,x_{m}\in\mathds{R},$

as well as the inequality

(39)
$1-\cos\theta\geq\dfrac{\theta^{2}}{2}\left(1-\dfrac{\theta^{2}}{12}\right)\geq\dfrac{\theta^{2}}{2}\left(1-\dfrac{\pi^{2}}{12}\right)\geq\dfrac{\theta^{2}}{12},\quad|\theta|\leq\pi.$

######  Lemma 4.4.

In the setting of Theorem 4.1, we have

$\dfrac{1}{2\pi}\int_{p\leq|\theta|\leq\pi}\left|\mathds{E}e^{i\theta(W_{m}(q)-\mathds{E}W_{m}(q))}\right|\,d\theta\leq\dfrac{6\sqrt{2}}{\pi
p\sigma_{m-1}\sigma_{m}}.$

###### Proof.

We first claim that

(40)
$\left|\mathds{E}e^{i\theta(W_{m}(q)-\mathds{E}W_{m}(q))}\right|=\prod_{j=1}^{m}\dfrac{1}{\sqrt{1+2(1-\cos\theta)q_{j}/p_{j}^{2}}}.$

In fact, we have from (12)

$\left|\mathds{E}e^{i\theta(X(q_{j})-\mathds{E}X(q_{j}))}\right|^{2}=\left|\mathds{E}e^{i\theta
X(q_{j})}\right|^{2}=\dfrac{p_{j}^{2}}{|1-q_{j}e^{i\theta}|^{2}}=\dfrac{1}{1-2(1-\cos\theta)q_{j}/p_{j}^{2}},$

which implies (40). On the other hand, choosing $x_{j}=q_{j}/p_{j}^{2}$ in
(38) and recalling (28), we see that

$\sigma_{m}^{4}\leq\dfrac{q}{p^{2}}\sigma_{m}^{2}+2\sum_{1\leq i<j\leq
m}\dfrac{q_{i}}{p_{i}^{2}}\dfrac{q_{j}}{p_{j}^{2}},$

since $q_{j}/p_{j}^{2}\leq q/p^{2}$, $j=1,\ldots,m$. We therefore have from
(28)

$\begin{split}&\prod_{j=1}^{m}(1+2(1-\cos\theta)q_{j}/p_{j}^{2})\geq
4(1-\cos\theta)^{2}\sum_{1\leq i<j\leq
m}\dfrac{q_{i}}{p_{i}^{2}}\dfrac{q_{j}}{p_{j}^{2}}\\\ &\geq
2(1-\cos\theta)^{2}\sigma_{m}^{2}\left(\sigma_{m}^{2}-\dfrac{q}{p^{2}}\right)=2(1-\cos\theta)^{2}\sigma_{m}^{2}\sigma_{m-1}^{2}\geq
2\left(\dfrac{\theta^{2}\sigma_{m}\sigma_{m-1}}{12}\right)^{2},\end{split}$

where the last inequality follows from (39). By (40), this implies that

$\int_{p\leq|\theta|\leq\pi}\left|\mathds{E}e^{i\theta(W_{m}(q)-\mathds{E}W_{m}(q))}\right|\,d\theta\leq\dfrac{24}{\sigma_{m-1}\sigma_{m}\sqrt{2}}\int_{p}^{\pi}\dfrac{1}{\theta^{2}}\,d\theta\leq\dfrac{12\sqrt{2}}{p\sigma_{m-1}\sigma_{m}}.$

The proof is complete. ∎

###### Proof of Theorem 4.1.

In view of the second equality in Theorem 2.2, Theorem 4.1 is an immediate
consequence of Lemmas 4.3 and 4.4. ∎

##  5 Discussion

In this section, we discuss which one of the Theorems 3.1, 3.2, or 4.1 should
be applied to estimate $S(n,m)$, as $n\to\infty$, depending on the values of
$m$. To this end, we will need the following auxiliary result.

######  Lemma 5.1.

Let $q$ and $\sigma_{m-1}^{2}$ be as in (25) and (27), respectively, and let
$p=1-q$. Then,

(41)
$\dfrac{1}{mp}-\dfrac{1}{q}\log\left(1-q\dfrac{m-1}{m}\right)\leq\dfrac{n}{m}\leq\dfrac{1}{mp}-\dfrac{1}{q}\log(1-q)$

and

(42)
$\dfrac{m(m-1)}{mp+q}+\dfrac{m}{q}\log\left(\dfrac{(m-1)p+1}{m}\right)\leq\sigma_{m-1}^{2}\leq\dfrac{m}{p}+\dfrac{m}{q}\log\left(\dfrac{mp}{m-q}\right).$

###### Proof.

By definition (25), we have

$n-m=\sum_{j=1}^{m}\dfrac{1}{1-qj/m}-m,$

thus implying that

(43)
$\dfrac{n}{m}=\dfrac{1}{mp}+\dfrac{1}{m}\sum_{j=1}^{m-1}\dfrac{1}{1-qj/m}.$

Since the function $f(x)=(1-qx/m)^{-1}$, $0\leq x\leq m$, is increasing, we
get

$\begin{split}&-\dfrac{m}{q}\log\left(1-q\dfrac{m-1}{m}\right)=\int_{0}^{m-1}f(x)\,dx\leq\sum_{j=1}^{m-1}\dfrac{1}{1-qj/m}\\\
&\leq\int_{1}^{m}f(x)\,dx=-\dfrac{m}{q}\log\left(1-q\dfrac{m-1}{m-q}\right),\end{split}$

which, in conjunction with (43), implies (41). On the other hand, the function

$g(x)=\dfrac{qx/m}{(1-qx/m)^{2}}=\dfrac{1}{(1-qx/m)^{2}}-\dfrac{1}{1-qx/m},\quad
0\leq x\leq m,$

is increasing as well. Therefore,

$\begin{split}&\dfrac{m(m-1)}{mp+q}+\dfrac{m}{q}\log\left(1-q\dfrac{m-1}{m}\right)=\int_{0}^{m-1}g(x)\,dx\leq\sum_{j=1}^{m-1}\dfrac{qj/m}{(1-qj/m)^{2}}\\\
&\leq\int_{1}^{m}g(x)\,dx=\dfrac{n(m-1)}{(m-q)p}+\dfrac{m}{q}\log\left(1-q\dfrac{m-1}{m-q}\right).\end{split}$

In view of (28), this implies (42) and completes the proof. ∎

In the first place, note that if $m$ is bounded then Theorem 3.2 gives us an
estimate of

(44) $\dfrac{m!}{m^{n}}S(n,m)$

with main term 1 and remainder term of the order of $1/n^{2}$.

From now on, assume that $m$ tends to infinity. We use the notation $x_{m}\sim
y_{m}$ if

$0<\varliminf_{m\to\infty}\dfrac{x_{m}}{y_{m}}\leq\varlimsup_{m\to\infty}\dfrac{x_{m}}{y_{m}}<\infty.$

The following two cases are concerned with estimates in the non-central
regions.

Case 1: $n-m\leq m^{\alpha}$, for some $0<\alpha<1/2$. Under this assumption,
condition (18) is fulfilled, since $\nu_{m}/\tau_{m}\sim m^{1/2}$. Therefore,
we apply Theorem 3.1 to estimate

$\dfrac{(n-m)!}{\nu_{m}^{n-m}}S(n,m)$

with main term 1 and remainder term of the order of

$\left(\dfrac{(n-m)\tau_{m}}{\nu_{m}}\right)^{3}\sim\left(\dfrac{n-m}{\sqrt{m}}\right)^{3}.$

More generally, suppose that $n-m=o(\sqrt{m})$ or, equivalently,
$n-m=o(\sqrt{n})$. In such a case, the first equality in Theorem 2.2 and
formula (19) allows us to give the exact expression

(45)
$\begin{split}&\dfrac{(n-m)!}{\nu_{m}^{n-m}}S(n,m)=1+\binom{n-m}{2}\left(\dfrac{\tau_{m}}{\nu_{m}}\right)^{2}\\\
&+\sum_{j=3}^{n-m}\binom{n-m}{j}\left(\dfrac{\tau_{m}}{\nu_{m}}\right)^{j}\mathds{E}Z_{m}^{j}.\end{split}$

This is essentially formula (5) shown by Moser and Wyman MosWym1958 . Note
that the $j$-term in (45) has a coefficient depending on $\mathds{E}Z_{m}^{j}$
and order of magnitude $(o(\sqrt{m})/\sqrt{m})^{j}$.

Case 2: $n-m\geq m\log(mR_{m})$, with $R_{m}\to\infty$, as $m\to\infty$. In
this case, we use Theorem 3.2 to estimate (44) with main term 1 and remainder
term of the order of

(46) $\left(\dfrac{m}{n-mH_{m}}\right)^{2}$

Since $H_{m}\sim\log m$, the order of magnitude of (46) is at least $(\log
R_{m})^{-2}$.

In the following two cases, we apply Theorem 4.1 to give estimates in the
central region.

Case 3: $m^{\alpha}\leq n-m\leq m\log(mr_{m})$, where $0<\alpha<1/2$, and
$r_{m}\to 0$ and $mr_{m}\to\infty$, as $m\to\infty$. Equivalently,

(47) $1+\dfrac{1}{m^{1-\alpha}}\leq\dfrac{n}{m}\leq 1+\log(mr_{m}).$

In view of (41), we consider the increasing function

$\varphi(q)=\dfrac{1}{mp}-\dfrac{1}{q}\log(1-q),\quad 0<q<1.$

Note that

$\varphi(q)\sim 1+\dfrac{q}{2},\ q\to
0,\quad\varphi(q)\sim\dfrac{1}{mp}-\log(1-q),\ q\to 1,$

thus implying that

(48) $\begin{split}&\varphi\left(\dfrac{2}{m^{1-\alpha}}\right)\sim
1+\dfrac{1}{m^{1-\alpha}},\\\ &\varphi\left(1-\dfrac{1}{mr_{m}}\right)\sim
r_{m}+\log(mr_{m})\sim\log(mr_{m}),\end{split}$

thanks to the assumptions on $r_{m}$. As a consequence of (47) and (48), we
see that

(49) $\dfrac{1}{mr_{m}}\lesssim p\lesssim 1-\dfrac{2}{m^{1-\alpha}}.$

Let us consider the extreme cases in (49). Suppose that $p\sim 1/mr_{m}$. From
(28) and (42), we have

$\sigma_{m-1}^{2}\sim m^{2}r_{m},\quad\sigma_{m}^{2}\sim
m^{2}r_{m}^{2}+m^{2}r_{m}\sim m^{2}r_{m}\sim\sigma_{m-1}^{2}.$

Therefore, we apply Theorem 4.1 to estimate

(50) $\dfrac{\prod_{j=1}^{m}(m-qj)}{m^{n}q^{n-m}}S(n,m)$

with main term of the order of $1/\sigma_{m}\sim 1/m\sqrt{r_{m}}$ and a
remainder term of the order of

$\dfrac{1}{p\sigma_{m}^{2}}\sim\dfrac{1}{p\sigma_{m}\sigma_{m-1}}\sim\dfrac{1}{m}.$

Suppose that $p\sim 1-2/m^{1-\alpha}$. Again by (28) and (42), we have

$\sigma_{m}^{2}\sim\sigma_{m-1}^{2}\sim m.$

Thus, we use again Theorem 4.1 to estimate (50) with a main term of the order
of $1/\sigma_{m}\sim 1/\sqrt{m}$ and a remainder terms of the order of

$\dfrac{1}{p\sigma_{m}^{2}}\sim\dfrac{1}{p\sigma_{m}\sigma_{m-1}}\sim\dfrac{1}{m}.$

Finally, for $p$ in the range given by (49), we have a similar discussion: we
use Theorem 4.1 to estimate (50). The order of the main term may change, but
the remainder term has always the order of magnitude of $1/m$.

Case 4: $n-m\sim m\log(mr_{m})$, where $r_{m}\sim r$, for some $0<r<\infty$.
Proceeding as in Case 3, we have

$p\sim\dfrac{1}{m},\quad\sigma_{m}^{2}\sim\sigma_{m-1}^{2}\sim m^{2}.$

We apply Theorem 4.1 to estimate (50) with the following difference: the main
term and the remainder term have the same order of magnitude $1/m$. As a
consequence, Theorem 4.1 only gives us in this case upper and lower bounds for
(50) having the same order of magnitude, since

$\dfrac{1}{\sigma_{m}}\sim\dfrac{1}{\sigma_{m-1}}\sim\dfrac{1}{p\sigma_{m}^{2}}\sim\dfrac{1}{p\sigma_{m}\sigma_{m-1}}\sim\dfrac{1}{m}.$

Funding

This work was supported by Ministerio de Ciencia, Innovación y Universidades,
Project PGC2018-097621-B-100.

## References

  * (1) J. A. Adell, D. Cárdenas-Morales: On the uniqueness conjecture for the maximum Stirling numbers of the second kind. Results Math., 76(2) (2021), Paper No. 93, 8.
  * (2) J. A. Adell, A. Lekuona: Explicit expressions and integral representations for the Stirling numbers. A probabilistic approach. Adv. Difference Equ., (2019), Paper No. 398, 11.
  * (3) R. Arratia, S. DeSalvo: Completely effective error bounds for Stirling numbers of the first and second kinds via Poisson approximation. Ann. Comb., 21 (2017), 1–24.
  * (4) R. Chelluri, L. B. Richmond, N. M. Temme : Asymptotic estimates for generalized Stirling numbers. Analysis (Munich), 20 (2000), 1–13.
  * (5) L. Comtet: Nombres de Stirling généraux et fonctions symétriques. C. R. Acad. Sci. Paris Sér. A-B, 275 (1972), A747–A750.
  * (6) L. Comtet: Advanced combinatorics. The Art of Finite and Infinite Expansions. D. Reidel Publishing Co., Dordrecht, 1974.
  * (7) B. S. El-Desouky, N. P. Cakić, F. A. Shiha: New family of Whitney numbers. Filomat, 31(2) (2017), 309–320.
  * (8) L. C. Hsu: Note on an asymptotic expansion of the $n$th difference of zero. Ann. Math. Statistics, 19 (1948), 273–277.
  * (9) C. Jordan: Cours d’analyse de l’École polytechnique. Tome I. Éditions Jacques Gabay, Sceaux, 1991.
  * (10) G. Louchard: Asymptotics of the Stirling numbers of the second kind revisited. Appl. Anal. Discrete Math., 7 (2013), 193–210.
  * (11) L. Moser, M. Wyman: Stirling numbers of the second kind. Duke Math. J., 25 (1958), 29–43.
  * (12) V. V. Petrov: Limit theorems of probability theory. The Clarendon Press, Oxford University Press, New York, 1995.
  * (13) B. C. Rennie, A. J. Dobson: On Stirling numbers of the second kind. J. Combinatorial Theory, 7 (1969), 116–121.
  * (14) V. N. Sachkov: Probabilistic methods in combinatorial analysis. Cambridge University Press, Cambridge, 1997.


# Interpretability of Fine-grained Classification
of Sadness and Depression

Tiasa Singha Roy*, Priyam Basu  Aman Priyanshu
Manipal Insitute of Technology
{tiasa.singharoy, priyam.basu1<EMAIL_ADDRESS>Rakshit
Naidu
Carnegie Mellon University
<EMAIL_ADDRESS>
###### Abstract

While sadness is a human emotion that people experience at certain times
throughout their lives, inflicting them with emotional disappointment and
pain, depression is a longer term mental illness which impairs social,
occupational, and other vital regions of functioning making it a much more
serious issue and needs to be catered to at the earliest. NLP techniques can
be utilized for the detection and subsequent diagnosis of these emotions. Most
of the open sourced data on the web deal with sadness as a part of depression,
as an emotion even though the difference in severity of both is huge. Thus, we
create our own novel dataset illustrating the difference between the two. In
this paper, we aim to highlight the difference between the two and highlight
how interpretable our models are to distinctly label sadness and depression.
Due to the sensitive nature of such information, privacy measures need to be
taken for handling and training of such data. Hence, we also explore the
effect of Federated Learning (FL) on contextualised language models. The code
for this paper can be found at: 111https://github.com/tiasa2/Interpretability-
of-Federated-Learning-for-Fine-grained-Classification-of-Sadness-and-
Depression

Interpretability of Fine-grained Classification
of Sadness and Depression

Tiasa Singha Roy*, Priyam Basu* and Aman Priyanshu Manipal Insitute of
Technology {tiasa.singharoy, priyam.basu1<EMAIL_ADDRESS>

Rakshit Naidu Carnegie Mellon University<EMAIL_ADDRESS>

##  1 Introduction

Mental Health is defined as a “state of well-being in which individuals
realize their potential, cope with the normal stresses of life, work
productively, and contribute to their communities” by The World Health
Organization (WHO). Depression is a very common mental disease that a large
number of people throughout the world suffer from. According to research
conducted by the WHO, in 2020 more than 280 million people all over the world
suffer from depression (WHO). It acts as leading cause for disability and is
the most common form of neuro-psychiatric disorder () (WHO). According to
Substance Abuse and Mental Health Services Association, in 2018, adolescents
aged 12 to 17 years old had the highest rate of major depressive episodes
(14.4%) followed by young adults 18 to 25 years old (13.8%). Older adults aged
50 and older had the lowest rate of major depressive episodes (4.5%). Two-
thirds of those who commit suicide struggle with depression Team (2022).

Sadness, on the other hand, is an emotional state characterized by feelings of
unhappiness and low mood. A person may say they are feeling ‘depressed,’ but
if it goes away on its own and does not impact life in a big way, it probably
isn’t the illness of depression Canadian Mental Health Association (2022);
American Psychiatric Association . Unlike depression, which is persistent and
longer-lasting, sadness is temporary and transitory Holmes (2021). Hence, it
is very important for us to be able to detect the difference between the two
conditions as depression requires urgent care and treatment.

Figure 1: Pipeline of data creation system

Most of the open-sourced data related to depression available also contain
texts that imply just "sadness" as a part of it. Sadness related data, within
itself, also comprises of depressing texts as seen in the GoEmotions dataset
Demszky et al. (2020). Sentences such as _"I feel so much lonely"_ and _"I’m
so depressed"_ are found under the "sad" label. The converse, i.e sad texts
are also often found under Depression corpora. In the Benchmarking dataset
Basu et al. (2021), sentences like _"Forgot my cheese cake at work"_ and
_"Wish I had took holidays instead of being at work"_ , which imply the
narrator is sad but not depressed, are present under the "depression" label.
Motivated by such problems, we propose a novel dataset where we try to tackle
this problem by providing labelled samples of sad and depressing data
separately to enable private training of machine learning models for detection
of the presence of, as well as differentiation between, the two emotions. In
this paper, we intend to explore how interpretable detecting sadness and
depression are using common language models– given that the definitions of
both are very close to each other and yet, both are distinctly different. We
also investigate whether Federated Learning (FL) could be a potential solution
for training models on our sensitive dataset and if FL models lead to higher
interpretable models than the baseline models as FL invloves collective
aggregation of model updates.

##  2 Related Works

With the fast increase of social media usage, efforts for depression and other
emotion detection are being made as a part of a Sentiment analysis task. Park
et al. (2015) indicated that depressed Twitter users tend to post tweets
containing negative emotional sentiments more than healthy users. In their
paper AlSagri and Ykhlef (2020), the authors explored multiple ML
classification techniques for detecting depression on Twitter using content
and activity features. Random forest techniques have also been used for
detecting signs of sadness Cacheda et al. (2019). Deep learning approaches are
popularly used for sentiment analysis task Babu and Kanaga (2022) and have
been explored later in this paper. However, none of these works target the
contrast between sadness and depression. Hence, we create our own dataset
which has been discussed in the next section. Alongside their powerful
performance deep learning models also act as black-box prediction systems,
therefore explainable methods become important for interpretable communication
of model inference Ribeiro et al. (2016). The ability of LIME a model-agnostic
local method for interpretability introduction within text classification
systems, has become well known for its impactful performance Mardaoui and
Garreau (2021). We employ said XAI method for further substantiating our
study.

##  3 Sad-Depression Dataset Creation

###  3.1 Depression

####  3.1.1 Data Collection

  1. 1.

SDCNL dataset: This paper Haque et al. (2021) was presented at ICANN 2021,
proposing a dataset of depression vs suicide tweets, where the authors had
scraped data from posts, that belonged to the subreddits - r/SuicideWatch and
r/Depression and were labelled as suicidal and depressed respectively. Most of
the data-points in a paragraph format having more than 2000 characters. We
utilized the depression labelled posts in this dataset to create the initial
set of depression labelled texts in our dataset.

  2. 2.

Goodbye World Dataset: This work aimed at identifying individuals at risk of
suicide Hesamuel . The data collection here was similar to the one in the
previous SDCNL dataset, using the Reddit API to collect posts from
r/SuicideWatch and r/Depression. Unlike the SDCNL dataset, the data-points
available here were mostly in a sentence or sentence-couplet format and
thereby, better suited for the aim of our study. We used the posts from the
depression labelled file which were non-identical from the ones we had earlier
collected.

  3. 3.

Benchmarking Dataset: This used as a dataset to benchmark transformer models
Basu et al. (2021). It contains two labels - depression/sad(label 1) and non
depression or neutral(label 0). For the purpose of this work, we utilized the
data under the label depression.

####  3.1.2 Paraphrasing

As mentioned earlier, the data collected from SDCNL dataset contained majorly
paragraphs, with more than 2000 characters in each, describing the depression
context. Since our work aims to understand the distinction of context at a
sentence level, we utilize a paraphrasing approach to get a sentence
representation of the paragraph while retaining the equivalent contextual
meaning, with the intention to retain the depression context. Google’s Pegasus
transformer model Zhang et al. (2020) was used for this task because of its
abstractive summarization capabilities.

####  3.1.3 Semantic Matching

After paraphrasing, few of the paraphrased sentences were unable to capture
the context of their respective paragraphs accurately. To get only the
accurate sentences we utilized S-BERT Reimers and Gurevych (2019) to perform
semantic matching between the paraphrased sentences and their original
paragraphs. An arbitrary threshold value was set for selection of the correct
sentences, which were then added to the data from SDCNL and goodbye world
dataset to form the depression section of our dataset.

####  3.1.4 Data Annotation and Cleaning

For the final data under the "depression" label, we manually reviewed and
annotated the processed data. It was carried out to remove ambiguous samples
and to maintain the distinction between the two classes by re-labelling the
samples that fit better under the "sadness" label. To differentiate between
depression and sadness, the authors examined the context of the sentences in
question based on the distinction made by American Psychiatric Association
(APA) American Psychiatric Association  After this cleaning was carried out on
the data to remove special characters, external links and ’@’ tags. The
cleaned data was labelled as 1 for depression.

###  3.2 Sadness

####  3.2.1 Data Collection

  1. 1.

GoEmotions Dataset: Created by Google Research Demszky et al. (2020),
GoEmotions is a human-annotated dataset of 58k Reddit comments extracted from
popular English-language subreddits and labelled with 28 emotion categories.
It includes 12 positive, 11 negative, 4 ambiguous emotion categories and 1
“neutral”. Within the negative, we picked comments labeled under sadness for
the "sadness" label of our dataset.

  2. 2.

NLP-text-emotion dataset: This dataset Lukasgarbas  was combined from
dailydialog Li et al. (2017), isear uni (2022), and emotion-stimulus Ghazi et
al. (2015) to create a balanced dataset with 5 labels. The texts mainly
consist of short messages and dialog utterances. We utilized it for the
sadness related data for our dataset.

  3. 3.

Emotions dataset for NLP: Open sourced dataset on Kaggle Praveen (2020). This
was created as a dataset for NLP classification tasks utilizing CARER
methodology Saravia et al. (2018) to generate 6 label categories. For our
dataset, we used the data marked under the label - "sadness".

####  3.2.2 Pseudo-Labelling

Upon inspection of the collected data we found an issue unique to the sadness
labels. These labels contained, apart from sadness, sentences of neutral or
even positive sentiment. To improve the quality of data under this label, we
utilized a BERT Devlin et al. (2018) based classification model to assemble
the correct ones, by training on the benchmarking dataset. We picked this
dataset as it contained neutral or non depressing/sad samples(label 0) as well
as depressing/sad samples(label 1). The trained model was then used to
classify our collected "sadness" data to create pseudo labels to remove data
points that are closer in sentiment to the "neutral" data and retaining the
ones that have a closer relation to the depression/sad data. This allows us to
essentially extract potentially "sad" data from the original collected
samples. Finally all the sentences labelled as 1 are collected under the label
sadness in our final dataset.

####  3.2.3 Data Annotation and Cleaning

For the final data under the label "sadness", we again manually reviewed and
annotated the processed data. Similar to the depression data, this was done to
re-label any samples that are closer to depression rather than sadness or
remove samples with ambiguous context. The same source was utilized as used
for depression as a baseline definition while annotating in order to maintain
uniformity. We also removed a few neutral and positive context based samples
that remained. To retain only "sad" samples, we focused on the definition
provided by American Psychiatric Association  to remove all the samples that
did not align with the definition. Finally cleaning was carried out on the
data to remove special characters, external links and ’@’ tags. The cleaned
data was labelled as 0 for sadness.

###  3.3 Final Dataset

At the end, we took all the depression-labelled and sadness-labelled data-
points and combined them into a single dataset, which consisted of 3256
samples, of which 1914 samples were labelled as "sadness"(label 0) and 1342
samples under the label "depression"(label 1). It essentially follows a
0.58-0.42 class ratio, favouring effective training of classification models.

##  4 Interpretability of Federated Models

The rise in the use of machine learning models such as deep neural networks
for intent mining has led to more accurate predictions. Explainable AI
provides a set of methods to help us understand and interpret a model’s
decisions. In order to trust a model’s predictions, one must be able to
interpret the reasons behind its decisions. Several attempts have been made to
infer the text classification for language models. State of the art model-
agnostic explanations such as LIME (Local Interpretable Model-Agnostic
Explanations) have been used to enable better visualization and analysis of AI
models.

##  5 Experimental Results

LIME is model agnostic in nature, that is, one can use it for any machine
learning model. LIME allows interpretability to be supplemented to benchmark
models. In our venture to integrate explainable AI for sadness-depression
classification, we evaluated multiple models for performance comparison.

###  5.1 Baseline Results

Figure 2: LIME base interpretation of Sentences—Class:SAD

Figure 3: LIME base interpretation of Sentences—Class:DEPRESSION

We intially conduct a detailed benchmark inference, over BERT and RoBERTa
classification models. We provide these results in table 1.

Table 1: Performance of Baseline models Model Name | Accuracy
---|---
BERT | 91.9%
RoBERTa | 96.62%

Inferring the performance presented in Table 1, we can see that the RoBERTa
model is the best performing model and acts as the appropriate candidate for
further exploration using explainable AI. Alongside RoBERTa we also take a
look at the performance of BERT as a control for our analysis.

###  5.2 Federated Setting Results

We further reproduce these results in a federated setting. For this we employ
the federated averaging algorithm, we present these results in table 2. The
table discusses results in both an IID-setting and a Non-IID setting, allowing
us to substantiate the use of federated learning for privacy-enabled
classification of this sensitive data.

Table 2: Performance in Federated Setting Model Name | IID Setting | Non-IID Setting
---|---|---
BERT | 75.07% | 73.6%
RoBERTa | 81.85% | 83.66%

For our experiments, we synthetically simulate $K$—Total number of clients
clients, with $C$—Fraction of clients randomly chosen to perform computation
on each round, and $E$—Number of training passes each client makes over its
local dataset per round. For our inferential setting, we set $K=10,C=0.3$, and
$E=1$ for the experiments. Having produced successful convergence, with
satisfactory performance, we can claim the utility of federated learning and
explainable AI for sadness—depression classification.

We also demonstrate the utility and inference that LIME brings to this
federated system. In a study on topics as delicate as depression
classification, it becomes integral that over-seeing authorities/deployers
understand the inferential steps the model may be taking. We supplement
samples of explainability for Sadness and Depression detection in Figure 2 and
Figure 3. Figure 2 deliberates on the case of Sadness detection for the
sentence "I feel so emotional today," where the LIME interprets model decision
on perturbed word analysis. As one can see, the impact of the words "today"
and "i" hold significant weightage to model inference for class label 0 or
SAD. On the other hand, Figure 3 deliberates on the case of Depression
detection for the sentence "If I had a gun, I’d blow my fu*king brains out
right now," where LIME distinctly highlights the words "brains", "gun", and
"fu*king" which significantly impact the model’s prediction to be class label
1 or DEPRESSION. We can ascertain that certain words like "now," "had," and
"if" had an opposing effect; however, their impact wasn’t as consequential as
that of the aforementioned highlighted words. Employing interpretability,
therefore, allows more accessible and successful affirmation of model
predictions during deployment even within a federated setting, thereby
substantiating the experimental setting.

##  6 Conclusion

Sadness and depression both are sensitive attributes of one’s life. Social
media has given a new platform for people to vent their feelings, at the same
time, an opportunity for one to detect impactful signs early on. The
application of SOTA interpretable methods on this assortment of data will pave
a path towards the adoption of the same during real-world implementation. Our
use of LIME, which quantify local model explanations, allows us to display
their importance and relevance in depression sadness classification. For
future explorations, we would like to explore these explainable techniques for
the same. We would also like to investigate other techniques such as lexical
normalization to reduce noise and measure its impact on our predictions. We
believe that our work serves as a valuable resource for sadness and depression
classification. The integration with explainable as well as privacy-preserving
methods paves a new path towards future research and development.

## References

  * uni (2022) 2022\.  Swiss center for affective sciences swiss center for affective sciences.
  * AlSagri and Ykhlef (2020) Hatoon S AlSagri and Mourad Ykhlef. 2020.  Machine learning-based approach for depression detection in twitter using content and activity features.  _IEICE Transactions on Information and Systems_ , 103(8):1825–1832.
  * (3) APA American Psychiatric Association.  What is depression?
  * Babu and Kanaga (2022) Nirmal Varghese Babu and E Kanaga. 2022.  Sentiment analysis in social media data for depression detection using artificial intelligence: A review.  _SN Computer Science_ , 3(1):1–20.
  * Basu et al. (2021) Priyam Basu, Tiasa Singha Roy, Rakshit Naidu, Zumrut Muftuoglu, Sahib Singh, and Fatemehsadat Mireshghallah. 2021.  Benchmarking differential privacy and federated learning for bert models.  _arXiv preprint arXiv:2106.13973_.
  * Cacheda et al. (2019) Fidel Cacheda, Diego Fernandez, Francisco J Novoa, Victor Carneiro, et al. 2019\.  Early detection of depression: social network analysis and random forest techniques.  _Journal of medical Internet research_ , 21(6):e12554.
  * Canadian Mental Health Association (2022) BC Division Canadian Mental Health Association. 2022.  Difference between sadness and depression.
  * Demszky et al. (2020) Dorottya Demszky, Dana Movshovitz-Attias, Jeongwoo Ko, Alan Cowen, Gaurav Nemade, and Sujith Ravi. 2020.  Goemotions: A dataset of fine-grained emotions.  _arXiv preprint arXiv:2005.00547_.
  * Devlin et al. (2018) Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018.  Bert: Pre-training of deep bidirectional transformers for language understanding.  _arXiv preprint arXiv:1810.04805_.
  * Ghazi et al. (2015) Diman Ghazi, Diana Inkpen, and Stan Szpakowicz. 2015.  Detecting emotion stimuli in emotion-bearing sentences.  In _International Conference on Intelligent Text Processing and Computational Linguistics_ , pages 152–165. Springer.
  * Haque et al. (2021) Ayaan Haque, Viraaj Reddi, and Tyler Giallanza. 2021.  Deep learning for suicide and depression identification with unsupervised label correction.  In _International Conference on Artificial Neural Networks_ , pages 436–447. Springer.
  * (12) Hesamuel.  Hesamuel/goodbye_world.
  * Holmes (2021) Leonard Holmes. 2021.  Differences between sadness and clinical depression.
  * Li et al. (2017) Yanran Li, Hui Su, Xiaoyu Shen, Wenjie Li, Ziqiang Cao, and Shuzi Niu. 2017.  Dailydialog: A manually labelled multi-turn dialogue dataset.  In _Proceedings of The 8th International Joint Conference on Natural Language Processing (IJCNLP 2017)_.
  * (15) Lukasgarbas.  Lukasgarbas/nlp-text-emotion: Multi-class sentiment analysis lstm, finetuned bert.
  * Mardaoui and Garreau (2021) Dina Mardaoui and Damien Garreau. 2021.  An analysis of lime for text data.  In _Proceedings of The 24th International Conference on Artificial Intelligence and Statistics_ , volume 130 of _Proceedings of Machine Learning Research_ , pages 3493–3501. PMLR.
  * Park et al. (2015) Gregory Park, H Andrew Schwartz, Johannes C Eichstaedt, Margaret L Kern, Michal Kosinski, David J Stillwell, Lyle H Ungar, and Martin EP Seligman. 2015.  Automatic personality assessment through social media language. _Journal of personality and social psychology_ , 108(6):934.
  * Praveen (2020) Praveen. 2020.  Emotions dataset for nlp.
  * Reimers and Gurevych (2019) Nils Reimers and Iryna Gurevych. 2019.  Sentence-bert: Sentence embeddings using siamese bert-networks.  _arXiv preprint arXiv:1908.10084_.
  * Ribeiro et al. (2016) Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. 2016.  "why should i trust you?": Explaining the predictions of any classifier.
  * Saravia et al. (2018) Elvis Saravia, Hsien-Chi Toby Liu, Yen-Hao Huang, Junlin Wu, and Yi-Shin Chen. 2018\.  CARER: Contextualized affect representations for emotion recognition.  In _Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing_ , pages 3687–3697. Association for Computational Linguistics.
  * Team (2022) SingleCare Team. 2022.  Statistics about depression in the u.s.
  * (23) World Health Organization (WHO).  Depression data.
  * Zhang et al. (2020) Jingqing Zhang, Yao Zhao, Mohammad Saleh, and Peter Liu. 2020.  Pegasus: Pre-training with extracted gap-sentences for abstractive summarization.  In _International Conference on Machine Learning_ , pages 11328–11339. PMLR.