# Cellular cosheaf homology are cosheaf homology

Daisuke Kishimoto  Department of Mathematics, Kyoto University, Kyoto,
606-8502, Japan<EMAIL_ADDRESS>and  Yasutomo Yushima  Department of
Mathematics, Kyoto University, Kyoto, 606-8502, Japan<EMAIL_ADDRESS>
###### Abstract.

A cosheaf is the dual notion of a sheaf, but we cannot define its homology as
the formal dual of sheaf cohomology, in general, because of the lack of the
cosheafification. A cellular cosheaf is a contravariant functor from the face
poset of a CW complex to the category of abelian groups. We show that given a
cellular cosheaf $F$, there is a natural way to associate a cosheaf
$\widehat{F}$, for which we can define homology as the formal dual of sheaf
cohomology, such that the Borel-Moore homology of $F$ is isomorphic to the
homology of $\widehat{F}$ whenever the underlying CW complex of $F$ is a
simplicial complex.

###### Key words and phrases:

cosheaf, cellular cosheaf, cosheafification, cosheaf homology, Čech homology,
Borel-Moore homology, Alexandroff space

###### 2010 Mathematics Subject Classification:

55N30, 18F20

##  1\. Introduction

A _cosheaf_ is the dual notion of a sheaf, that is, a cosheaf is a covariant
functor from the poset of open sets of a given topological space into the
category of abelian groups, which preserves certain colimits. The first
appearance of cosheaves dates back, at least, to the 1960’s [3, 9], and since
then, they appear ubiquitously in broad areas of mathematics. However, we
cannot define the homology of a cosheaf as the formal dual of sheaf cohomology
as we cannot prove the existence of the cosheafification in general.

A _cellular cosheaf_ is a contravariant functor from the face poset of a CW
complex to the category of abelian groups. The Borel-Moore homology can be
defined for a cellular cosheaf, and is often called cellular cosheaf homology.
Recently, cosheaves and cellular cosheaves are intensely studied in applied
algebraic topology as they are used to describe time dependent change of
properties of spaces. See [5, 6, 7, 8, 11, 10, 12, 13, 15, 16, 18, 19]. By
definition, cellular cosheaves are not cosheaves, and so it is natural to
pose:

######  Problem 1.1.

Given a cellular cosheaf $F$, is there a natural way to associate a cosheaf
$\widehat{F}$ such that the Borel-Moore homology of $F$ and the homology of
$\widehat{F}$, if it were defined, are isomorphic?

It is well known that there is a one-to-one correspondence between posets and
$T_{0}$-Alexandroff spaces. We use this correspondence to study Problem 1.1,
and so we slightly generalize cellular cosheaves such that a cellular cosheaf
is defined to be a functor from a poset to the category of abelian groups. We
will extend the above correspondence to that there is a one-to-one
correspondence between equivalence classes of cellular cosheaves and cosheaves
on $T_{0}$-Alexandroff space (Theorem 2.8). Thus we study cosheaves on
$T_{0}$-Alexandroff spaces. The cosheafification cannot be constructed, in
general, simply by dualizing the sheafification as mentioned above. However,
as for precosheaves over $T_{0}$-Alexandroff spaces, we can prove the
following. See [5, 14] for other attempts to construct the cosheafification.

######  Theorem 1.2 (Theorem 3.3).

Each precosheaf on a $T_{0}$-Alexandroff space admits the cosheafification.

This enables us to construct a projective resolution for each cosheaf on a
$T_{0}$-Alexandroff space (Proposition 3.7), so that we can define its
homology as the formal dual of sheaf cohomology. Then we are able to compare
the Borel-Moore homology of a cellular cosheaf and the homology of the
corresponding cosheaf on a $T_{0}$-Alexandroff space. We will prove the
following answer to Problem 1.1.

######  Theorem 1.3 (Theorem 4.4).

Let $F$ be a cellular cosheaf on a simplicial complex, and let $\widehat{F}$
be the corresponding cosheaf on a $T_{0}$-Alexandroff space. Then the Borel-
Moore homology of $F$ is isomorphic to the homology of $\widehat{F}$.

We can import a plenty of results in sheaf theory into cosheaf homology if it
were defined. Then Theorem 1.3 would benefit especially applied algebraic
topology as the Borel-Moore homology of cellular cosheaves are frequently used
there.

To each poset $P$ we can associate a simplicial complex called an order
complex, and as we will see in Section 4, we can also associate to a cellular
cosheaf over a poset a cellular cosheaf on its order complex. Then we can
define the Borel-Moore homology for a cellular cosheaf over a poset as the one
for a cellular cosheaf on the associated order complex. We will see that this
definition is consistent with that for a cellular cosheaf on a regular CW
(Lemma 4.5). Then a result for a cellular cosheaf on regular CW complexes
analogous to Theorem 1.3 will be also obtained (Theorem 4.6).

### Acknowledgement

The first author was partially supported by JSPS KAKENHI Grant Numbers
JP17K05248 and JP19K03473.

##  2\. Cosheaves on $T_{0}$-Alexandroff spaces

This section studies a correspondence between cellular cosheaves and cosheaves
on $T_{0}$-Alexandroff spaces. This correspondence is based on the
correspondence between posets and $T_{0}$-Alexandroff spaces, which we recall
here. An _Alexandroff space_ is a topologuical space in which all
intersections of open sets, possibly infinitely many, are open, and a
$T_{0}$-space is a topological space such that given distinct two points,
there is an open set which contains exactly one of the two points. Let $X$ be
an Alexandroff space. Then for each $x$, there is the smallest open set
containing $x$, which we denote by $U_{x}$. Namely, $U_{x}$ is the
intersection of all open sets containing $x$. The $T_{0}$-property of an
Alexandroff space can be characterized as follows [17, Proposition 4].

######  Lemma 2.1.

An Alexandroff space is a $T_{0}$-space if and only if $U_{x}=U_{y}$ implies
$x=y$.

Let $X$ be a $T_{0}$-Alexandroff space. We define a poset from $X$. For
$x,y\in X$, we define $x\leq y$ if $U_{x}\subset U_{y}$. Clearly, the relation
$\leq$ satisfies reflexivity and transitivity. By Lemma 2.1, it also satisfies
antisymmetry, and so the relation $\leq$ is a partial order on $X$. Let
$\mathcal{P}(X)$ denote the poset $(X,\leq)$.

Next, we define a $T_{0}$-Alexandroff space from a poset. Let $P$ be a poset.
Then there is a topology $\mathcal{O}$ on $P$ whose open basis consists of
$P_{\geq x}=\\{y\geq x\mid y\in P\\}$ for $x\in P$. Let $\mathcal{X}(P)$
denote the topological space $(X,\mathcal{O})$.

######  Lemma 2.2.

For a poset $P$, $\mathcal{X}(P)$ is a $T_{0}$-Alexandroff space.

###### Proof.

Let $\\{U_{\lambda}\\}_{\lambda\in\Lambda}$ be a collection of open sets of
$\mathcal{X}(P)$. Note that a subset $U\subset P$ is an open set of
$\mathcal{X}(P)$ if and only if $x\in P$ belongs to $U$ whenever $x\geq y$ for
some $y\in U$. If $x\in P$ satisfies $x\geq y$ for some
$y\in\bigcap_{\lambda\in\Lambda}U_{\lambda}$, then $x\in U_{\lambda}$ for each
$\lambda\in\Lambda$, implying $x\in\bigcap_{\lambda\in\Lambda}U_{\lambda}$.
Thus $\bigcap_{\lambda\in\Lambda}U_{\lambda}$ is an open set of
$\mathcal{X}(P)$, and so $\mathcal{X}(P)$ is an Alexandroff space. Clearly,
$U_{x}=P_{\geq x}$ for each $x\in P$. Then $x\leq y$ whenever $U_{x}\subset
U_{y}$, implying $x=y$ whenever $U_{x}=U_{y}$. Thus by Lemma 2.1,
$\mathcal{X}(P)$ is a $T_{0}$-space, completing the proof. ∎

By definition, $\mathcal{P}(\mathcal{X}(P))=P$ for each poset $P$ and
$\mathcal{X}(\mathcal{P}(X))=X$ for each $T_{0}$-Alexandroff space $X$. Then
we obtain:

######  Theorem 2.3.

The assignments $\mathcal{P}$ and $\mathcal{X}$ give a one-to-one
correspondence between posets and $T_{0}$-Alexandroff spaces.

######  Remark 2.4.

As in [17], a map $X\to Y$ between $T_{0}$-Alexandroff spaces is continuous if
and only if the corresponding map $\mathcal{P}(X)\to\mathcal{P}(Y)$ is order-
preserving. Then Theorem 2.3 can be extended to an isomorphism between the
category of posets and the category of $T_{0}$-Alexandroff spaces.

We define cellular cosheaves and cosheaves. Let $\mathbf{Ab}$ denote the
category of abelian groups. We will always regard a poset $P$ as a category in
which objects are elements of $P$ and there is a unique morphism $x\to y$
whenever $x\leq y$.

######  Definition 2.5.

A _cellular cosheaf_ over a poset $P$ is a functor $F\colon P\to\mathbf{Ab}$.

For a topological space $X$, let $\operatorname{Open}(X)$ denote the poset of
open sets of $X$. Let $\mathtt{N}(\mathcal{U})$ denote the nerve of an open
cover $\mathcal{U}$.

######  Definition 2.6.

A _precosheaf_ on a topological space $X$ is a functor
$F\colon\operatorname{Open}(X)\to\mathbf{Ab}$. If the natural map

$\underset{\mathtt{N}(\mathcal{U})}{\operatorname{colim}}\,F\to F(U)$

is an isomorphism for each open set $U$ and each open cover $\mathcal{U}$ of
$U$, then $F$ is called a _cosheaf_.

We extend the correspondence of Theorem 2.3 to cellular cosheaves and
cosheaves. Let $F$ be a cellular cosheaf over a poset $P$. We define a
precosheaf $\widehat{F}$ on $\mathcal{X}(P)$ by

$\widehat{F}(U)=\underset{x\in U}{\operatorname{colim}}\,F(x)$

for each open set $U$ of $\mathcal{X}(P)$, where we regard $U$ as a subposet
of $P$ on the RHS.

######  Lemma 2.7.

Let $F$ be a cellular cosheaf over a poset $P$. Then $\widehat{F}$ is a
cosheaf.

###### Proof.

Let $U$ be an open set of $\mathcal{X}(P)$, and let $\mathcal{U}$ be an open
cover of $U$. By definition,
$\underset{V\in\mathtt{N}(\mathcal{U})}{\operatorname{colim}}\underset{x\in
V}{\operatorname{colim}}\,F(x)$ is the quotient

$\bigoplus_{x\in U}\bigoplus_{I_{x}}F(x)/\sim$

in which copies of $F(x)$ are identified and $a\sim b$ if $F(x<y)(a)=b$ for
$a\in F(x)$ and $b\in F(y)$, where $I_{x}$ is the set of elements of
$\mathtt{N}(\mathcal{U})$ including $U_{x}$. Then

$\underset{V\in\mathtt{N}(\mathcal{U})}{\operatorname{colim}}\underset{x\in
V}{\operatorname{colim}}\,F(x)\cong\underset{x\in
U}{\operatorname{colim}}\,F(x)$

and so we get

(2.1)
$\underset{\mathtt{N}(\mathcal{U})}{\operatorname{colim}}\,\widehat{F}=\underset{V\in\mathtt{N}(\mathcal{U})}{\operatorname{colim}}\underset{x\in
V}{\operatorname{colim}}\,F(x)\cong\underset{x\in
U}{\operatorname{colim}}\,F(x)=\widehat{F}(U).$

Thus $\widehat{F}$ is a cosheaf, as stated. ∎

Let $P$ be a poset. Then there is an injective order-preserving map

$\iota\colon P\to\operatorname{Open}(\mathcal{X}(P)),\quad x\mapsto U_{x}.$

Note that for a cosheaf $F$ on $\mathcal{X}(P)$, $\iota^{*}(F)$ is a cellular
cosheaf over a poset $P$. Let $\mathbf{Cosh}(X)$ denote the category of
cosheaves on a topological space $X$. Now we are ready to prove:

######  Theorem 2.8.

For a poset $P$, the functor

$[P,\mathbf{Ab}]\to\mathbf{Cosh}(\mathcal{X}(P)),\quad F\mapsto\widehat{F}$

is an equivalence of categories.

###### Proof.

We show that the functor
$\iota^{*}\colon\mathbf{Cosh}(\mathcal{X}(P))\to[P,\mathbf{Ab}]$ is the
inverse of the functor $\widehat{\cdot}$. Let $F\in[P,\mathbf{Ab}]$, up to
natural isomorphism. Since $x\in P$ is the terminal object of $U_{x}$,

$\iota^{*}(\widehat{F})(x)=\widehat{F}(U_{x})=\underset{y\in
U_{x}}{\operatorname{colim}}\,F(y)\cong F(x),$

implying $\iota^{*}\circ\widehat{\cdot}=1$. Let
$G\in\mathbf{Cosh}(\mathcal{X}(P))$. For
$U\in\operatorname{Open}(\mathcal{X}(P))$, let
$\mathcal{U}_{U}=\\{U_{x}\\}_{x\in U}$ be an open cover of $U$, and let
$U_{x_{0}\ldots x_{n}}=U_{x_{0}}\cap\cdots\cap U_{x_{n}}$. Since $G$ is a
cosheaf,

$\displaystyle G(U)$
$\displaystyle\cong\underset{\mathtt{N}(\mathcal{U}_{U})}{\operatorname{colim}}\,G$
$\displaystyle=\left(\bigoplus_{\begin{subarray}{c}x_{0},\ldots,x_{n_{1}}\in
U\\\ n_{1}\geq 0\end{subarray}}G(U_{x_{0}\cdots x_{n_{1}}})\right)/\sim$
$\displaystyle=\left(\bigoplus_{x_{\in}U}G(U_{x})\bigoplus_{\begin{subarray}{c}x_{0},\ldots,x_{n_{1}}\in
U\\\ n_{1}\geq 1\end{subarray}}G(U_{x_{0}\cdots x_{n_{1}}})\right)/\sim$
$\displaystyle\cong\left(\bigoplus_{x\in
U}G(U_{x})\bigoplus_{\begin{subarray}{c}x_{0},\ldots,x_{n_{1}}\in U\\\
n_{1}\geq 1\end{subarray}}\underset{\mathtt{N}(\mathcal{U}_{U_{x_{0}\cdots
x_{n_{1}}}})}{\operatorname{colim}}\,G\right)/\sim$
$\displaystyle\cong\left(\bigoplus_{x\in
U}G(U_{x})\bigoplus_{\begin{subarray}{c}x_{0},\ldots,x_{n_{1}}\in U\\\
n_{1}\geq
1\end{subarray}}\bigoplus_{\begin{subarray}{c}y_{0},\ldots,y_{n_{2}}\in
U_{x_{0}\cdots x_{n_{1}}}\\\ n_{2}\geq 0\end{subarray}}G(U_{y_{0}\cdots
y_{n_{2}}})\right)/\sim$ $\displaystyle\cong\left(\bigoplus_{x\in
U}G(U_{x})\bigoplus_{\begin{subarray}{c}x_{0},\ldots,x_{n_{1}}\in U\\\
n_{1}\geq 1\end{subarray}}\bigoplus_{y\in U_{x_{0}\cdots
x_{n_{1}}}}G(U_{y})\bigoplus_{\begin{subarray}{c}y_{0},\ldots,y_{n_{2}}\in
U_{x_{0}\cdots x_{n_{1}}}\\\ n_{2}\geq
1\end{subarray}}\underset{\mathtt{N}(\mathcal{U}_{U_{y_{0}\cdots
y_{n_{2}}}})}{\operatorname{colim}}\,G\right)/\sim$ $\displaystyle\;\;\vdots$
$\displaystyle\cong\left(\bigoplus_{x\in
U}\bigoplus_{J_{x}}G(U_{x})\right)/\sim$

where in the last term, copies of $G(U_{x})$ are identified and $a\sim b$ if
$G(U_{x}<U_{y})(a)=b$ for $a\in G(U_{x})$ and $b\in G(U_{y})$. Then we get

(2.2) $G(U)\cong\underset{x\in U}{\operatorname{colim}}\,G(U_{x})$

and so we obtain

$\widehat{\iota^{*}(G)}(U)=\underset{x\in
U}{\operatorname{colim}}\,\iota^{*}(G)(x)=\underset{x\in
U}{\operatorname{colim}}\,G(U_{x})\cong G(U),$

implying $\widehat{\cdot}\circ\iota^{*}=1$. Therefore the proof is complete. ∎

##  3\. Defining cosheaf homology

This section proves that we can define the homology of a cosheaf on a
$T_{0}$-Alexandroff space as the formal dual of sheaf cohomology. Let $F$ be a
precosheaf on a topological space $X$. The _costalk_ of $F$ at $x\in X$ is
defined by

$F_{x}=\underset{\mathtt{N}(x)}{\lim}\,F,$

where $\mathtt{N}(x)$ denotes the subposet of $\operatorname{Open}(X)$
consisting of all open sets containing $x$. If $X$ is an Alexandroff space,
then $U_{x}$ is the minimal element of $\mathtt{N}(x)$, implying

(3.1) $F_{x}\cong F(U_{x}).$

Then we need not take limits to get costalks. This simple property will be a
key in our study. There is a natural map

(3.2) $\rho(U)\colon\underset{x\in U}{\operatorname{colim}}\,F(U_{x})\to F(U)$

induced from the inclusions $U_{x}\to U$ for $x\in U$. By (2.2), we have:

######  Lemma 3.1.

Let $F$ be a cosheaf on a $T_{0}$-Alexandroff space $X$. Then for each open
set $U$ of $X$, the natural map (3.2) is an isomorphism.

Let $F,G$ be cosheaves on a topological space $X$, and let $\alpha\colon F\to
G$ be a homomorphism, that is, a natural transformation. Then there is the
induced map $\alpha_{x}\colon F_{x}\to G_{x}$ for each $x\in X$. If $X$ is an
Alexandroff space, then $\alpha_{x}=\alpha(U_{x})$ by (3.1). The following two
lemmas will be quite useful.

######  Lemma 3.2.

Let $F,G$ be cosheaves on a $T_{0}$-Alexandroff space $X$. For homomorphisms
$\alpha,\beta\colon F\to G$, $\alpha=\beta$ if and only if
$\alpha_{x}=\beta_{x}\colon F_{x}\to G_{x}$ for each $x\in X$.

###### Proof.

The if part is obvious, and so we prove the only if part. Let $U$ be an open
set of $X$, and let $\mathcal{U}=\\{U_{x}\\}_{x\in U}$ be an open cover of
$U$. By Lemma 3.1, the maps

$\underset{x\in U}{\operatorname{colim}}\,\alpha(U_{x}),\,\underset{x\in
U}{\operatorname{colim}}\,\beta(U_{x})\colon\underset{x\in
U}{\operatorname{colim}}\,F(U_{x})\to\underset{x\in
U}{\operatorname{colim}}\,G(U_{x})$

are identified with $\alpha(U)$ and $\beta(U)$, respectively. Thus since
$\alpha_{x}=\alpha(U_{x})$ and $\beta_{x}=\beta(U_{x})$, we get $\alpha=\beta$
whenever $\alpha_{x}=\beta_{x}$ for each $x\in X$. Therefore the proof is
finished. ∎

Now we define the cosheafification of a precosheaf as the formal dual of the
sheafification. Let $F$ be a precosheaf on a topological space $X$. The
_cosheafification_ of $F$ is a cosheaf $F^{+}$ on $X$ such that there is a
homomorphism $\pi\colon F^{+}\to F$ satisfying that for any homomorphism
$\alpha\colon G\to F$ for a cosheaf $G$, there is a unique homomorphism
$\tilde{\alpha}\colon G\to F^{+}$ satisfying a commutative diagram

$\textstyle{F^{+}\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\scriptstyle{\pi}$$\textstyle{G\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\scriptstyle{\alpha}$$\scriptstyle{\tilde{\alpha}}$$\textstyle{F.}$

######  Theorem 3.3.

Every precosheaf on a $T_{0}$-Alexandroff space admits the cosheafification.

###### Proof.

Let $F$ be a precosheaf on a $T_{0}$-Alexandroff space $X$. We define a
precosheaf $F^{+}$ on $X$ by

$F^{+}(U)=\underset{x\in U}{\operatorname{colim}}\,F(U_{x})$

for an open set $U$ of $X$. Let $\mathcal{U}$ be an open cover of an open set
$U$ of $X$. As in (2.1), we have

$\underset{\mathtt{N}(\mathcal{U})}{\operatorname{colim}}\,F^{+}=\underset{V\in\mathtt{N}(\mathcal{U})}{\operatorname{colim}}\,F^{+}(V)=\underset{V\in\mathtt{N}(\mathcal{U})}{\operatorname{colim}}\underset{x\in
V}{\operatorname{colim}}\,F(U_{x})\cong\underset{x\in
U}{\operatorname{colim}}\,F(U_{x})=F^{+}(U).$

Then $F^{+}$ is a cosheaf. We define a homomorphism $\pi\colon F^{+}\to F$ by
$\pi(U)=\rho(U)$, where $\rho(U)$ is as in (3.2). Let $\alpha\colon G\to F$ be
a homomorphism from a cosheaf $G$. Then there is a commutative diagram

$\textstyle{\underset{x\in
U}{\operatorname{colim}}\,G(U_{x})\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\scriptstyle{\underset{x\in
U}{\operatorname{colim}}\,\alpha_{x}}$$\scriptstyle{\rho(U)}$$\textstyle{\underset{x\in
U}{\operatorname{colim}}\,F(U_{x})\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\scriptstyle{\rho(U)}$$\textstyle{F^{+}(U)\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\scriptstyle{\pi(U)}$$\textstyle{G(U)\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\scriptstyle{\alpha(U)}$$\textstyle{F(U)\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\textstyle{F(U),}$

where the maps $\rho$ are as in (3.2). The left $\rho(U)$ is an isomorphism by
Lemma 3.1, so that we can define a map

$\tilde{\alpha}(U)\colon G(U)\xrightarrow[\cong]{\rho(U)^{-1}}\underset{x\in
U}{\operatorname{colim}}\,G(U_{x})\xrightarrow{\underset{x\in
U}{\operatorname{colim}}\,\alpha_{x}}\underset{x\in
U}{\operatorname{colim}}\,F(U_{x})=F^{+}(U)$

which is natural with respect to $U$. Thus we get a homomorphism
$\tilde{\alpha}\colon G\to F^{+}$. Clearly, we have
$\pi\circ\tilde{\alpha}=\alpha$. Since $\pi_{x}\colon F_{x}^{+}\to F_{x}$ is a
natural isomorphism such that $\pi_{x}\circ\tilde{\alpha}_{x}=\alpha_{x}$, the
uniqueness of $\tilde{\alpha}$ follows from Lemma 3.2, completing the proof. ∎

For a homomorphism $\alpha\colon F\to G$ of cosheaves on a topological space
$X$, we define a precosheaf $\operatorname{Ker}\alpha$ on $X$ by

$(\operatorname{Ker}\alpha)(U)=\operatorname{Ker}\\{\alpha(U)\colon F(U)\to
G(U)\\}$

for an open set $U$ of $X$. The important point here is that the precosheaf
$\operatorname{Ker}\alpha$ is not necessarily a cosheaf, even when $X$ is a
$T_{0}$-Alexandroff space.

######  Example 3.4.

Let $X=\\{a,b,c\\}$ be a topological space whose open sets are
$\emptyset,\\{a\\},\\{a,b\\},\\{a,c\\},X$. Clearly, $X$ is a
$T_{0}$-Alexandroff space. Define a precosheaf $F,G$ on $X$ by

$F(U)=\begin{cases}\mathbb{Z}&U\neq\emptyset\\\
0&U=\emptyset\end{cases}\quad\text{and}\quad
G(U)=\begin{cases}\mathbb{Z}&U=\\{a\\}\\\ 0&U\neq\\{a\\},\end{cases}$

where the maps $F(U<V)$ are the identity map of $\mathbb{Z}$ for
$U\neq\emptyset$. We can easily verify that $F,G$ are cosheaves. There is a
unique surjective homomorphism $\alpha\colon F\to G$, and we have

$(\operatorname{Ker}\alpha)(U)=\begin{cases}\mathbb{Z}&U\neq\emptyset,\\{a\\}\\\
0&U=\emptyset,\\{a\\}.\end{cases}$

Then it follows that

$\underset{\mathtt{N}(\mathcal{U}_{X})}{\operatorname{colim}}\operatorname{Ker}\alpha=\mathbb{Z}^{2}.$

On the other hand, $(\operatorname{Ker}\alpha)(X)=\mathbb{Z}$, and thus
$\operatorname{Ker}\alpha$ is not a cosheaf.

######  Remark 3.5.

The precosheaf defined by using the cokernel is a cosheaf because colimits
commute with colimits.

We construct a projective resolution for a cosheaf on a $T_{0}$-Alexandroff
space by dualizing the standard construction of an injective resolution for a
sheaf. We say that a sequence of cosheaves

$F\xrightarrow{\alpha}G\xrightarrow{\beta}H$

on a topological space $X$ is exact if
$F_{x}\xrightarrow{\alpha_{x}}G_{x}\xrightarrow{\beta_{x}}H_{x}$ is an exact
sequence of abelian groups for each $x\in X$. A resolution of a cosheaf $F$ is
an exact sequence of cosheaves

$\cdots\to R_{n}\to R_{n-1}\to\cdots\to R_{0}\to F\to 0$

which we denote by $R_{\bullet}\to F\to 0$, and a homotopy between resolutions
for $F$ can be defined in the obvious way. We say that a homomorphism of
cosheaves $\alpha\colon F\to G$ is surjective if $F\xrightarrow{\alpha}G\to 0$
is exact, and injective if $0\to F\xrightarrow{\alpha}G$ is exact. A cosheaf
$P$ on a topological space $X$ is _projective_ if for any homomorphism
$\alpha\colon P\to F$ and any surjective homomorphism $\beta\colon G\to F$,
there is a commutative diagram

$\textstyle{G\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\scriptstyle{\beta}$$\textstyle{P\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\scriptstyle{\alpha}$$\textstyle{F.}$

A projective resolution for a cosheaf $F$ is a resolution $P_{\bullet}\to F\to
0$ such that each $P_{n}$ is projective.

We consider the formal dual of a skyscraper sheaf. Let $X$ be a topologiacal
space. For $x\in X$ and an abelian group $A$, we define a precosheaf
$\mathtt{sky}(x,A)$ by

$\mathtt{sky}(x,A)(U)=\begin{cases}A&x\in U\\\ 0&x\not\in U\end{cases}$

for an open set $U$ of $X$. Clearly, $\mathtt{sky}(x,A)$ is a cosheaf.

######  Lemma 3.6.

Let $X$ be a topological space. For $x\in X$ and a projective abelian group
$P$, $\mathtt{sky}(x,P)$ is a projective cosheaf.

###### Proof.

Let $F,G$ be cosheaves on $X$. Suppose that we are given a homomorphism
$\alpha\colon\mathtt{sky}(x,P)\to F$ and a surjective homomorphism
$\beta\colon G\to F$. Then since $P$ is projective and
$\mathtt{sky}(x,P)_{x}\cong P$, there is a commutative diagram

$\textstyle{G_{x}\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\scriptstyle{\beta_{x}}$$\textstyle{\mathtt{sky}(x,P)_{x}\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\scriptstyle{\alpha_{x}}$$\scriptstyle{\tilde{\alpha}_{x}}$$\textstyle{F_{x}.}$

Let $U$ be an open set of $X$. For $x\in U$, let $\tilde{\alpha}(U)$ denote
the composite

$\mathtt{sky}(x,P)(U)\cong\mathtt{sky}(x,P)_{x}\xrightarrow{\tilde{\alpha}_{x}}G_{x}\xrightarrow{\rm
proj}G(U),$

and for $x\not\in U$, let $\tilde{\alpha}(U)=0\colon\mathtt{sky}(x,P)(U)=0\to
G(U)$. Then we get a homomorphism $\tilde{\alpha}\colon\mathtt{sky}(x,P)\to G$
satisfying a commutative diagram

$\textstyle{G\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\scriptstyle{\beta}$$\textstyle{\mathtt{sky}(x,P)\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\scriptstyle{\alpha}$$\scriptstyle{\tilde{\alpha}}$$\textstyle{F.}$

Thus $\mathtt{sky}(x,P)$ is a projective cosheaf, as stated. ∎

######  Proposition 3.7 (cf. [4, Chapter II, Theorem 3.2]).

Every cosheaf on a $T_{0}$-Alexandroff space admits a projective resolution
which is unique, up to homotopy.

###### Proof.

Let $F$ be a cosheaf on a $T_{0}$-Alexandroff space $X$. Let $\alpha\colon
G\to F$ be a homomorphism of cosheaves. Then since
$(\operatorname{Ker}\alpha)^{+}_{x}\cong(\operatorname{Ker}\alpha)_{x}$ for
each $x\in X$, the composition of homomorphisms
$(\operatorname{Ker}\alpha)^{+}\xrightarrow{\pi}\operatorname{Ker}\alpha\to G$
is injective. Thus it is sufficient to prove that there is a surjective
homomorphism $P\to F$, where $P$ is a projective cosheaf.

For each $x\in X$, there is a surjection $P_{x}\to F_{x}$, where $P_{x}$ is a
projective abelian group. Then as in the proof of Lemma 3.6, we can define a
homomorphism $\alpha(x)\colon\mathtt{sky}(x,P_{x})\to F$ which is surjective
on the costalks at $x$. Since the direct sum of projective cosheaves is a
projective cosheaf, $\bigoplus_{x\in X}\mathtt{sky}(x,P_{x})$ is a projective
cosheaf by Lemma 3.6. Clearly, the homomorphism

$\bigoplus_{x\in X}\alpha(x)\colon\bigoplus_{x\in X}\mathtt{sky}(x,P_{x})\to
F$

is surjective. The uniqueness of a projective resolution, up to homotopy, can
be proved by the standard argument in homological algebra, completing the
proof. ∎

By Proposition 3.7, we are able to define the homology of a cosheaf on a
$T_{0}$-Alexandroff space.

######  Definition 3.8.

Let $F$ be a cosheaf on a $T_{0}$-Alexandroff space $X$. The homology of $F$,
denoted by $H_{*}(X;F)$, is defined by the homology of a chain complex

$\cdots\to P_{n}(X)\to P_{n-1}(X)\to\cdots\to P_{0}(X)\to 0,$

where $P_{\bullet}\to F\to 0$ is a projective resolution for $F$.

##  4\. Čech homology and Borel-Moore homology

This section studies the Čech homology of a cosheaf on a $T_{0}$-Alexandroff
space, and compare it with the Borel-Moore homology to prove Theorem 1.3.

###  4.1. Čech homology

Let $F$ be a precosheaf on a topological space $X$, and let $\mathcal{U}$ be
an open cover of $X$. For
$U_{\alpha_{0}},\ldots,U_{\alpha_{n}}\in\mathcal{U}$, let
$U_{\alpha_{0}\cdots\alpha_{n}}=U_{\alpha_{0}}\cap\cdots\cap U_{\alpha_{n}}$.
We define

$\check{C}_{n}(\mathcal{U};F)=\bigoplus_{\alpha_{0},\ldots,\alpha_{n}}F(U_{\alpha_{0}\cdots\alpha_{n}}),$

where the sum is taken over all non-empty $U_{\alpha_{0}\cdots\alpha_{n}}$
such that $\alpha_{0},\ldots,\alpha_{n}$ are distinct. We define a map
$\partial\colon\check{C}_{n}(\mathcal{U};F)\to\check{C}_{n-1}(\mathcal{U};F)$
by

$\partial(x)=\sum_{i=0}^{n}(-1)^{i}F(U_{\alpha_{0}\cdots\alpha_{n}}<U_{\alpha_{0}\cdots\widehat{\alpha_{i}}\cdots\alpha_{n}})(x)$

for $x\in\check{C}_{n}(\mathcal{U};F)$. It is straightforward to check
$\partial^{2}=0$. Then we get a chain complex
$(\check{C}_{*}(\mathcal{U};F),\partial)$, and its homology is denoted by

$\check{H}_{*}(\mathcal{U};F)$

and called the _Čech homology_ of $F$ with respect to $\mathcal{U}$.

We compare Čech homology and cosheaf homology as well as Čech cohomology and
sheaf cohomology. Let $\mathcal{U}$ be an open cover of a topological space
$X$. For an open set $U$ of $X$, let

$\mathcal{U}|_{U}=\\{U\cap V\mid V\in\mathcal{U}\\}$

be an open cover of $U$. For a precosheaf $F$, we can define a precosheaf
$C_{n}(\mathcal{U};F)$ on $X$ by

$C_{n}(\mathcal{U};F)(U)=\check{C}_{n}(\mathcal{U}|_{U};F|_{U})$

for an open set $U$ of $X$.

######  Lemma 4.1 (cf. [4, Chapter III, Theorem 4.9]).

Let $F$ be a cosheaf on a $T_{0}$-Alexandroff space $X$, and let $\mathcal{U}$
be an open cover of $X$. Then the sequence

$C_{\bullet}(\mathcal{U};F)\to F\to 0$

is a resolution.

###### Proof.

For an open cover $\mathcal{V}$ of an open set $V$ of $X$, we have

$\displaystyle\underset{\mathtt{N}(\mathcal{V})}{\operatorname{colim}}\,C_{n}(\mathcal{U};F)$
$\displaystyle=\underset{W\in\mathtt{N}(\mathcal{V})}{\operatorname{colim}}\bigoplus_{\alpha_{0},\ldots,\alpha_{n}}F(U_{\alpha_{0}\cdots\alpha_{n}}\cap
W)$
$\displaystyle\cong\bigoplus_{\alpha_{0},\ldots,\alpha_{n}}\underset{W\in\mathtt{N}(\mathcal{V})}{\operatorname{colim}}F(U_{\alpha_{0}\cdots\alpha_{n}}\cap
W)$
$\displaystyle\cong\bigoplus_{\alpha_{0},\ldots,\alpha_{n}}F(U_{\alpha_{0}\cdots\alpha_{n}}\cap
V)$ $\displaystyle=C_{n}(\mathcal{U};F)(V),$

implying $C_{n}(\mathcal{U};F)$ is a cosheaf. We can see that the sequence
$C_{\bullet}(\mathcal{U};F)\to F\to 0$ is exact by substituting $U_{x}$,
completing the proof. ∎

We dualize a flasque sheaf. A cosheaf $F$ on a topological space $X$ is called
_flasque_ if $F(V<U)\colon F(V)\to F(U)$ is injective for all open sets
$V\subset U$ of $X$. The following lemma follows at once from the definition
of a flasque cosheaf.

######  Lemma 4.2 (cf. [4, Chapter II, Theorem 5.5]).

If $F$ is a flasque cosheaf on a $T_{0}$-Alexandroff space $X$, then for
$*\geq 1$,

$H_{*}(X;F)=0.$

Now we compare Čech homology and cosheaf homology.

######  Theorem 4.3 (cf. [4, Chapter III, Theorem 4.13]).

Let $F$ be a cosheaf on a $T_{0}$-Alexandroff space $X$, and let
$\mathcal{U}=\\{U_{\alpha}\\}_{\alpha\in A}$ be an open cover of $X$. If

$H_{*}(U_{\alpha_{0}\cdots\alpha_{n}};F|_{U_{\alpha_{0}\cdots\alpha_{n}}})=0$

for $*\geq 1$ and $n\geq 0$, then there is an isomorphism

$\check{H}_{*}(\mathcal{U};F)\cong H_{*}(X;F).$

###### Proof.

Let $P_{\bullet}\to F\to 0$ be the projective resolution constructed in the
proof of Proposition 3.7. Clearly, the direct sum of flasque cosheaves is
flasque. Then since the cosheaf $\mathtt{sky}(x,A)$ is flasque, each $P_{n}$
is flasque. Thus $C_{*}(\mathcal{U};P_{n})$ is flasque too. Consider a
commutative diagram

---
$\textstyle{\vdots\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\textstyle{\vdots\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\textstyle{\vdots\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\textstyle{\cdots\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\textstyle{\check{C}_{1}(\mathcal{U};P_{1})\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\textstyle{\check{C}_{1}(\mathcal{U};P_{0})\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\textstyle{\check{C}_{1}(\mathcal{U};F)\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\textstyle{0}$$\textstyle{\cdots\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\textstyle{\check{C}_{0}(\mathcal{U};P_{1})\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\textstyle{\check{C}_{0}(\mathcal{U};P_{0})\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\textstyle{\check{C}_{0}(\mathcal{U};F)\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\textstyle{0}$$\textstyle{\cdots\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\textstyle{P_{1}(X)\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\textstyle{P_{0}(X)\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\textstyle{F(X)\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\textstyle{0}$$\textstyle{0}$$\textstyle{0}$$\textstyle{0.}$

By Lemma 4.2 and the assumption
$H_{*}(U_{\alpha_{0}\cdots\alpha_{n}};F|_{U_{\alpha_{0}\cdots\alpha_{n}}})=0$
for $*\geq 1$ and $n\geq 0$, all sequences but the bottom and the right ones
are exact. Then we obtain the isomorphism in the statement. ∎

###  4.2. Borel-Moore homology

Let $X$ be a CW complex, and let $P(X)$ denote the face poset of $X$. We
abbreviate a cellular cosheaf over $P(X)^{\mathrm{op}}$ by a cellular cosheaf
on $X$. Let $F$ be a cellular cosheaf on $X$. We set

$C_{n}^{\mathrm{BM}}(X;F)=\bigoplus_{\begin{subarray}{c}\sigma\in P(X)\\\
\dim\sigma=n\end{subarray}}F(\sigma)$

and define a map $\partial\colon C_{n}^{\mathrm{BM}}(X;F)\to
C_{n-1}^{\mathrm{BM}}(X;F)$ by

$\partial(x)=\sum_{\dim\tau=n-1}[\sigma:\tau]F(\sigma>\tau)(x)$

for $x\in F(\sigma)$ and $\sigma\in P_{n}(X)$ with $\dim\sigma=n$, where
$[\sigma:\tau]$ denotes the coincidence number. Then it is straightforward to
check $\partial^{2}=0$, so that we get a chain complex
$(C_{*}^{\mathrm{BM}}(X;F),\partial)$. The _Borel-Moore homology_

$H^{\mathrm{BM}}_{*}(X;F)$

is defined to be the homology of the above chain complex. Now we prove:

######  Theorem 4.4.

Let $F$ be a cellular cosheaf on a simplicial complex $K$. Then there is an
isomorphism

$H^{\mathrm{BM}}_{*}(K;F)\cong H_{*}(\mathcal{X}(P(K));\widehat{F}).$

###### Proof.

Let $V$ denote the vertex set of $K$, and let $\mathcal{U}=\\{U_{v}\\}_{v\in
V}$ be an open cover of $\mathcal{X}(P(K))$. If $U_{v_{0}}\cap\cdots\cap
U_{v_{n}}\neq\emptyset$ for distinct $v_{0},\ldots,v_{n}$, then vertices
$v_{0},\ldots,v_{n}$ form a simplex $[v_{0},\ldots,v_{n}]$ of $K$ such that
$U_{v_{0}}\cap\cdots\cap U_{v_{n}}=U_{[v_{0},\ldots,v_{n}]}$. Thus
$(\check{C}_{*}(\mathcal{U};\widehat{F}),\partial)\cong(C^{\mathrm{BM}}_{*}(K;F),\partial)$,
implying

(4.1) $\check{H}_{*}(\mathcal{U};\widehat{F})\cong H^{\mathrm{BM}}_{*}(K;F).$

Let $P_{\bullet}\to\widehat{F}\to 0$ be a projective resolution for
$\widehat{F}$. Then by (3.1), the chain complex $\cdots\to
P_{n}(U_{[v_{0},\ldots,v_{n}]})\to P_{n-1}(U_{[v_{0},\ldots,v_{n}]})\to\cdots$
is identified with a sequence of costalks
$\cdots\to(P_{n})_{[v_{0},\ldots,v_{n}]}\to(P_{n-1})_{[v_{0},\ldots,v_{n}]}\to\cdots$
which is exact by definition. Then we get

$H_{*}(U_{v_{0}}\cap\cdots\cap U_{v_{n}};\widehat{F}|_{U_{v_{0}}\cap\cdots\cap
U_{v_{n}}})=H_{*}(U_{[v_{0},\ldots,v_{n}]};\widehat{F}|_{U_{[v_{0},\ldots,v_{n}]}})=0$

for $*\geq 1$. Thus by Theorem 4.3, there is an isomorphism

(4.2) $\check{H}_{*}(\mathcal{U};\widehat{F})\cong
H_{*}(\mathcal{X}(P(K));\widehat{F}).$

Therefore by combining (4.1) and (4.2), the proof is finished. ∎

We discuss the Borel-Moore homology of a general cellular cosheaf. Let $P$ be
a poset. Recall that the order complex $\Delta(P)$ is the geometric
realization of an abstract simplicial complex whose simplices are finite
sequences $x_{0}<\cdots<x_{n}$ in $P$. It is well known that
$X\cong\Delta(P(X))$ whenever $X$ is a regular CW complex, and a relation
between functors $\Delta$ and $\mathcal{X}$ can be found in [1, Section 1].
Let $F$ be a cellular cosheaf over a poset $P$. Then we can define a cellular
cosheaf $\Delta(F)$ on $\Delta(P)$ by

$\Delta(F)(x_{0}<\cdots<x_{n})=F(x_{n}).$

Then we can define the Borel-Moore homology of $F$ by
$H^{\mathrm{BM}}_{*}(\Delta(P);\Delta(F))$, which is also known as the higher
colimits of $F$ (see [2]). Thus by Theorem 4.4, the Borel-Moore homology of
$F$ is also identified with the homology of the naturally associated cosheaf.
The above definition of the Borel-Moore homology for a functor over a general
poset is consistent with the case of a functor over the face poset of a
regular CW complex by the following dual version of [5, Theorem 11.27].

######  Lemma 4.5.

Let $F$ be a cellular cosheaf on a regular CW complex $X$. Then there is an
isomorphism

$H_{*}^{\mathrm{BM}}(X;F)\cong H_{*}^{\mathrm{BM}}(\Delta(P(X));\Delta(F)).$

Thus by Theorem 4.4, we obtain:

######  Theorem 4.6.

Let $F$ be a cellular cosheaf on a regular CW complex $X$. Then there is an
isomorphism

$H_{*}^{\mathrm{BM}}(X;F)\cong
H_{*}(\mathcal{X}(P(\Delta(P(X))));\widehat{\Delta(F)}).$

## References

  * [1] J.A. Barmak, Algebraic Topology of Finite Topological Spaces and Applications, Lecture Notes in Mathematics 2032, Springer-Verlag, Berlin, 2011.
  * [2] A. Bousfield and D. Kan, Homotopy Limits, Completions and Localizations, Lecture Notes in Mathematics 304, Springer-Verlag, Berlin, 1972.
  * [3] G.E. Bredon, Cosheaves and homology, Pacific J. Math. 25 (1968), no. 1, 1-32.
  * [4] G.E. Bredon, Sheaf Theory, Graduate Texts in Mathematics 170 (2nd ed.), Springer-Verlag, Berlin, New York, 1997.
  * [5] J.M. Curry, Sheaves, cosheaves and applications, https://arxiv.org/abs/1303.3255.
  * [6] J.M. Curry, Topological data analysis and cosheaves, Jpn. J. Ind. Appl. Math. 32 (2015), no. 2, 333-371.
  * [7] J.M. Curry, Dualities between cellular sheaves and cosheaves, J. Pure Appl. Algebra 222 (2018), no. 4, 966-993.
  * [8] J. Curry and A. Patel, Classification of constructible cosheaves, Theory Appl. Categ. 35 (2020), paper no. 27, 1012-1047.
  * [9] R. Deheuvels, Homologie des ensembles ordonnés et des espaces topologiques, Bull. Soc. Math. France 90 (1962), 261-321.
  * [10] J. Hansen and R. Ghrist, Toward a spectral theory of cellular sheaves, J. Appl. Comput. Topol. 3 (2019), no. 4, 315-358.
  * [11] R. Ghrist, Elementary Applied Topology, ed. 1.0, Createspace, 2014.
  * [12] S. Mansourbeigi, Sheaf theory approach to distributed applications: analysing heterogeneous data in air traffic monitoring, Int. J. Data Sci. Anal. 3 (2017), no. 5, 34-39.
  * [13] V. Nanda, Local cohomology and stratification, Found. Comput. Math. 20 (2020), no. 2, 195-222.
  * [14] A.V. Prasolov, Precosheaves of pro-sets and abelian pro-groups are smooth, Topology Appl. 159 (2012), no. 5, 1339-1356.
  * [15] M. Robinson, Cosheaf representations of relations and Dowker complexes, J. Appl. Comput. Topol. (2021), open access.
  * [16] V. de Silva, E. Munch, and A. Patel. Categorified Reeb graphs, Discrete Comput. Geom. 55 (2016), 854-906.
  * [17] R.E. Stong, Finite topological spaces, Trans. Amer. Math. Soc. 123 (1966), no. 2, 325-340.
  * [18] H.R. Yoon, Cellular sheaves and cosheaves for distributed topological data analysis, PhD thesis, University of Pennsylvania, 2018.
  * [19] H.R. Yoon and R. Ghrist, Persistence by parts: multiscale feature detection via distributed persistent homology, https://arxiv.org/abs/2001.01623.


# Discovering Mental Health Research Topics with Topic Modeling

Xin Gao  Cem Sazara

###### Abstract

Mental health significantly influences various aspects of our daily lives, and
its importance has been increasingly recognized by the research community and
the general public, particularly in the wake of the COVID-19 pandemic. This
heightened interest is evident in the growing number of publications dedicated
to mental health in the past decade. In this study, our goal is to identify
general trends in the field and pinpoint high-impact research topics by
analyzing a large dataset of mental health research papers. To accomplish
this, we collected abstracts from various databases and trained a customized
Sentence-BERT based embedding model leveraging the BERTopic framework. Our
dataset comprises 96,676 research papers pertaining to mental health, enabling
us to examine the relationships between different topics using their
abstracts. To evaluate the effectiveness of the model, we compared it against
two other state-of-the-art methods: Top2Vec model and LDA-BERT model. The
model demonstrated superior performance in metrics that measure topic
diversity and coherence. To enhance our analysis, we also generated word
clouds to provide a comprehensive overview of the machine learning models
applied in mental health research, shedding light on commonly utilized
techniques and emerging trends. Furthermore, we provide a GitHub
link***https://github.com/stella-gao/Mental-Health-Research-Paper-Dataset to
the dataset used in this paper, ensuring its accessibility for further
research endeavors.

Data Visualization, Datasets, Topic Modeling, BERT, Mental Health

##  1 Introduction

The COVID-19 pandemic, which has significantly impacted our lifestyle for
nearly two years, has led to a rise in psychosocial stressors and mental
health problems. Consequently, there has been a notable surge in mental health
research as a response to these challenges. To understand the specific topics
studied by the research community, we employed topic modeling methods on the
titles and abstracts of conference and journal research papers focused on
mental health field. Our primary objective is to identify studies aimed at
improving mental health and analyze the prominent research topics of the past
decade.

To accomplish this, we collected abstracts from various databases, including
arXiv, ACM, bioRxiv, medRxiv, and PubMed, spanning the period from Jan 2010 to
March 2023. This extensive dataset reveals a growing interest in mental
health-related research during the last decade, with a significant peak
occurring during the COVID-19 pandemic. Our study aims to identify key trends
and significant research topics by analyzing a dataset of 96,676 research
papers. To extract meaningful insights from the dataset, we employed a
Sentence-BERT based embedding model called BERTopic (Grootendorst, 2022).
BERTopic generates document embeddings and clusters them into semantically
coherent topics, enabling further analysis. By applying this model, we can
identify specific concepts associated with each topic, providing a basis for
further analysis and investigation. For instance, the topic related to suicide
prominently includes terms such as “suicidal,” “ideation,” and “attempt.”
These identified topics may help uncover interdisciplinary connections and
foster collaboration among different fields.

In evaluating our approach, we conducted performance evaluation using various
metrics (Ferdinand Kapl, 2022). These metrics included TD (Topic
Distinctiveness/Diversity), measuring topic uniqueness and diversity. Inv. RBO
(Inverted Rank-Biased Overlap) evaluates topic coherence and word order
similarity. NPMI (Normalized Pointwise Mutual Information) measures semantic
coherence within topics by calculating the average pairwise similarity between
words within a topic. Cv (Coefficient of Topic Coherence) assesses coherence
among top-ranked words. Higher metric values indicate better topic coherence
and interpretability. By evaluating these metrics, we optimized topic modeling
by experimenting with hyperparameters. Our goal was to find the configuration
that yielded topics with high coherence, diversity, and interpretability, as
indicated by the metrics mentioned above. These performance evaluation metrics
provided valuable insights into the quality of the generated topics and guided
our decision-making process in selecting the values that optimized the topic
modeling results. We evaluated the BERTopic based model’s effectiveness by
comparing it against two other state-of-the-art methods, namely Top2Vec
(Angelov, 2020) and LDA-BERT (Basmatkar & Maurya, 2022) model. The BERTopic
based model outperformed the others in metrics such as Inv. RBO and Cv. By
examining the relationships between different topics using the abstracts, we
gained valuable insights into the landscape of mental health research. To
enhance our analysis, we designed a custom prompt to access the OpenAI API
(OpenAI, 2023a) and utilized the power of the GPT-3.5-turbo model (OpenAI,
2023b) to retrieve the machine learning methods employed in the research
papers. Additionally, we generated a word cloud to visually represent our
findings, emphasizing the most commonly used and emerging machine learning
techniques in mental health research. The resulting word cloud offers a
comprehensive overview of the machine learning models applied in mental health
research, shedding light on both commonly utilized techniques and emerging
trends. This approach can provide valuable insights into the adoption and
evolution of computational techniques in addressing mental health challenges.

##  2 Related Work

Topic modeling has been a popular method in natural language processing for
some time, even preceding the advent of deep learning approaches (Wallach,
2006; Chauhan & Shah, 2021). Recently, there has been significant effort and
high interest in improving mental health in the research community.
Researchers have used a variety of topic modeling techniques, including Latent
Dirichlet Allocation (LDA) and Top2Vec, to identify latent themes and topics
in mental health content (Gaur et al., 2018; Yanchuk et al., 2022). Some
studies have also explored the use of advanced natural language processing
methods, such as using BERT (Bidirectional Encoder Representations from
Transformers) as a text mining approach for mental health prediction (Zeberga
et al., 2022). The authors employed word2vec and BERT with Bi-LSTM to
effectively analyze and detect depression and anxiety signs from social media
posts of Reddit and Twitter. One challenge in topic modeling for mental health
research papers is ensuring that the identified topics are clinically
meaningful and relevant. In addition, the quality of the data used for
analysis, such as social media data, can vary widely, making it difficult to
draw accurate conclusions.

Studies using topic modeling techniques have provided insights into mental
health issues and identified hidden themes and patterns in mental health
content. Lin et al. (Lin et al., 2022) compared different neural topic
modeling methods in learning the topical propensities of various psychiatric
conditions from psychotherapy session transcripts parsed from speech
recordings. Leung et al. (Leung & Khalvati, 2022) aimed to identify
psychosocial stressors during the COVID-19 pandemic by applying natural
language processing (NLP) to social media data and analyzing the trend in the
prevalence of stressors at different stages of the pandemic. Gong et al. (Gong
& Poellabauer, 2017) proposed a novel topic modeling-based approach to perform
context-aware analysis of recordings, outperforming context-unaware methods
and challenge baselines for all metrics. Rio-Chanona et al. (del Rio-Chanona
et al., 2022) used the NRC emotion lexicon method for sentiment analysis and a
structural topic model to study the causes of the 2021 Great Resignation and
investigate changes in work- and quit-related posts on Reddit.

This study fills a research gap by exploring mental health research papers
using topic modeling. We investigate the effectiveness of the Sentence-BERT
based embedding model and compare it with other topic modeling methods. Given
BERT’s success in natural language processing tasks, we aim to leverage its
potential for analyzing mental health content.

##  3 Topic Modeling Methods

We utilized the open-source Python library BERTopic (Grootendorst, 2022) that
facilitates topic modeling using the BERT model. This method utilizes pre-
trained language models to extract embeddings from text which gives it an
advantage over the classical approaches such as Latent Dirichlet Allocation
(LDA) (Blei et al., 2003) and Non-Negative Matrix Factorization (NMF) (Lee &
Seung, 2000) that describe documents with Bag-of-Words representation. There
are three main steps in this method:

Extracting document embeddings.

This model uses a pre-trained Sentence-BERT (SBERT) (Reimers & Gurevych, 2019)
model to extract document embeddings. Sentence-BERT is considered a state-of-
the art method to extract document embeddings.

Dimensionality reduction and clustering.

The extracted embeddings are not directly used to build the final topics.
First, they go through dimensionality reduction with the Uniform Manifold
Approximation and Projection (UMAP) (McInnes et al., 2018) method. UMAP method
has proven to be an effective method to preserve local and global features of
high dimensional data in their reduced dimensions. The embeddings after the
dimension reduction are clustered with the Hierarchical Density-Based Spatial
Clustering of Applications (HDBSCAN) (McInnes et al., 2017) method.

Building topic representations.

In order to differentiate one topic from another, a modified TF-IDF procedure
is used. In the regular TF-IDF representation, each word is assigned a weight
that considers a local term: Term Frequency (TF) and Global term: Inverse
Document Frequency (IDF). As shown in equation (1), term frequency $tf_{t,d}$
is calculated for term t and document d and inverse document frequency is
calculated as the logarithm of the number of documents N in the corpus divided
by the total number of documents that contain term t ($df_{t}$).

$W_{t,d}=tf_{t,d}\cdot\log{\left(\frac{1+N}{df_{t}}\right)}$ (1)

BERTopic, uses a modified version of TF-IDF as shown in equation (2). Term
frequencies $tf_{t,c}$ are calculated by concatenating all documents in one
cluster and considering it as a single cluster document c. IDF is also
modified and replaced by an inverse cluster frequency term. It is calculated
by taking the logarithm of the average number of words per cluster (A) divided
by the frequency of term t across all clusters ($tf_{t}$). This term measures
how much information a term provides to a specific cluster. Overall, this TF-
IDF based approach generates topic-word distributions for each cluster of
documents.

$W_{t,c}=tf_{t,c}\cdot\log{\left(\frac{1+A}{tf_{t}}\right)}$ (2)

The dynamic model follows a similar procedure. The main idea is that the same
topics exist over different times, but they are represented differently at
different times. The dynamic model extends the same TF-IDF equation to
specific time steps. Term frequency is calculated using the documents
available at time i and the IDF term stays the same as before. Its formula is
given in equation (3). This equation allows fast calculations as the local
representations can be quickly calculated without recreating the clusters and
embeddings.

$W_{t,c,i}=tf_{t,c,i}\cdot\log{\left(\frac{1+A}{tf_{t}}\right)}$ (3)

We compared the BERTopic based model with two other topic modeling techniques:
Top2Vec and LDA-BERT. Top2Vec is an algorithm designed for topic modeling ad
semantic search tasks. It aims to automatically detect and extract topics from
a given text corpus while generating jointly embedded vectors for topics and
documents. Top2Vec and BERTopic share some similarities. They both use
pretrained embedding models to generate word embeddings which are also then
reduced by UMAP and clustered by HDBSCAN. The difference is that Top2Vec
produces clean and coherent topic representations by excluding noisy documents
that are not firmly assigned to any specific topic. It effectively captures
underlying themes in noisy or unstructured data. LDA-BERT is a hybrid approach
for topic modeling that combines Latent Dirichlet Allocation (LDA) with
Sentence BERT, an autoencoder and K-means clustering. It aims to enhance the
topic discovery process and extract meaningful themes from a collection of
sentences or documents, leading to improved topic coherence and
interpretability.

##  4 Dataset

We compiled a comprehensive dataset of mental health-related papers by
extracting them from well-known open access archives and multiple databases,
namely arXiv, ACM, bioRxiv, medRxiv, and PubMed. To ensure a thorough
collection, we utilized a search query specifically tailored for mental health
keywords. This query, consisting of 26 relevant keywords such as Agoraphobia,
Anxiety Disorder, Attention-Deficit/Hyperactivity Disorder [ADHD], Autism
Spectrum Disorder [ASD], Post-Traumatic Stress Disorder [PTSD], and
Schizophrenia, was constructed based on mental health statistics provided by
the National Institute of Mental
Health†††https://www.nimh.nih.gov/health/statistics. By combining these
keywords using logical operators (AND and OR), we ensured the matching of
relevant keywords in both the title and the abstract of research paper
metadata. This approach allowed us to capture a broader range of necessary
keywords for comprehensive retrieval.

The data collection process followed a systematic approach, starting with
PubMed, a prominent biomedical literature database known for its extensive
coverage of mental health-related research papers. Subsequently, we gathered
papers from bioRxiv, medRxiv, arXiv, and ACM databases to ensure a
comprehensive representation of diverse perspectives from both medical and
computational domains. This sequential approach allowed us to build a robust
dataset comprising 96,676 distinct research papers related to mental health.

The dataset distribution from different databases is as follows: 75,842 papers
from PubMed, 12,237 papers from bioRxiv, 7,522 papers from medRxiv, 559 papers
from arXiv, and 516 papers from ACM. Table 1 displays the frequency of papers
published in the field of mental health from Jan 2010 to March 2023. Notably,
we observed a significant growth in mental health research papers over the
last decade, indicating an increasing focus on mental health in both real-
world and research communities. Since 2020, a remarkable increase has been
observed in multiple mental health categories, including depression, anxiety,
and covid or pandemic related mental disorder. The observed surge in research
and focus on mental health reflects the growing recognition of the profound
impact of the pandemic on psychological well-being and the pressing need to
develop effective strategies for support and intervention. These findings
emphasize the critical importance of prioritizing mental health care and
resilience-building efforts, both during times of crisis and beyond, to
mitigate the adverse effects of such challenging circumstances.

Table 1: Paper Frequency by Year in Mental Health

Year | Frequency | Year | Frequency | Year | Frequency
---|---|---|---|---|---
2023 | 413 | 2022 | 21846 | 2021 | 20663
2020 | 13814 | 2019 | 8993 | 2018 | 1688
2017 | 6257 | 2016 | 2745 | 2015 | 2196
2014 | 4595 | 2013 | 1924 | 2012 | 5339
2011 | 4758 | 2010 | 1445 |  |

Figure 1: 2D UMAP Visualization of BERTopic Clusters

##  5 Methods and Results

Figure 2: Word Scores of Top 8 Topics from BERTopic Figure 3: Similarity
Matrix between Topics in BERTopic Figure 4: Hierarchical Clustering of Top 50
Most Frequent Mental Health Research Topics Figure 5: Temporal Evolution of
Machine Learning Techniques in Mental Health Research: A Comparative WordCloud
Analysis Across Different Years (2013-2017, 2019-2022) Table 2: Prompt
Provided to OpenAI gpt-3.5-turbo Model for Filtering Primary Machine Learning
Models

Role | Content
---|---
system | “You are a helpful text summarization assistant.”
user |  | “Given title and abstract of the research paper in the
---
format [Title:Abstract], generate the machine learning
model name if they used machine learning technique in
the format [Model: your ML model] for the {user_text}”

Table 3: Performance Comparison of BERTopic, LDA-BERT and Top2Vec methods using four evaluation metrics Algorithm | No. Topics | TD | Inv. RBO | NPMI | Cv
---|---|---|---|---|---
BERTopic | 202 | 0.7208 | 0.9948 | 0.2892 | 0.8068
LDA-BERT | 100 | 0.2117 | 0.7634 | 0.0772 | 0.5814
Top2Vec | 134 | 0.4978 | 0.9669 | -0.0358 | 0.4654

In our study on mental health research papers, we employed the BERTopic
framework to analyze the combined content of the title and abstract sections.
By focusing on these key sections, we aimed to extract the primary topics and
themes while minimizing noise and irrelevant details.

To ensure the integrity and consistency of our dataset, we conducted
preprocessing steps such as lowercase conversion, standardizing text format,
and eliminating irrelevant information like URLs, mentions, and hashtags. We
retained only alphabetic characters for further analysis.

Following the data cleaning process, we applied several tokenization methods
to transform the text into a sequence of tokens. We removed stop words,
expanded contractions, and performed lemmatization on the remaining tokens.
These preprocessing steps aimed to enhance the quality of the text data and
ensure that the topic modeling method could generate precise and meaningful
insights from the dataset.

We utilized the all-MiniLM-L6-v2 pre-trained embedding model from HuggingFace
hub as the foundation to train the model‡‡‡https://huggingface.co/sentence-
transformers/all-MiniLM-L6-v2. This embedding model is based on the latest
advancements in sentence-transformers and has been trained on a vast and
diverse dataset comprising over 1 billion training pairs. Its extensive
training ensures that it excels as a sentence-transformers model, mapping
sentences and paragraphs into a high-dimensional vector space of 384
dimensions, striking a balance between performance and efficiency.

Throughout our experimentation, we carefully identified the optimal settings
that yielded the most coherent and diverse topics. To capture different levels
of semantic information within the text, we considered unigrams, bigrams, and
trigrams during the topic generation process. This approach allowed us to
extract meaningful insights at various granularities. Additionally, we
employed the HDBSCAN clustering algorithm, known for its density-based
clustering capabilities, which automatically determines the number of clusters
without the need for explicit specification.

To further fine-tune the model, we conducted extensive experiments with
various hyperparameter values. One crucial consideration was ensuring that the
generated topics were substantiated by a sufficient number of associated
documents. Therefore, we set a minimum requirement of 50 documents per topic,
ensuring that the topics were representative of significant patterns within
the data. Similarly, we imposed a minimum threshold of 50 documents for each
cluster, guaranteeing that the clusters contained substantial data for
meaningful analysis and interpretation. These thresholds helped maintain the
robustness and significance of the identified topics and clusters.

We performed a series of experiments by varying these hyperparameters and
assessed the quality of the generated topics based on coherence, diversity,
and meaningfulness. We evaluated the resulting topics using established
metrics such as topic coherence, topic uniqueness, and semantic similarity.
Starting with different values for the minimum requirement, we iteratively
adjusted the threshold and analyzed the impact on the generated topics. We
measured the coherence of the topics using metrics like NPMI and Cv, ensuring
that the topics exhibited meaningful associations between words within each
cluster. We also considered the diversity and representativeness of the
topics, ensuring that they captured a significant number of documents and
covered a wide range of mental health-related concepts.

Similarly, for the UMAP projection, we experimented with different values for
the number of nearest neighbors. We projected the documents onto various
dimensional spaces and assessed the quality of the resulting embeddings. We
examined the distribution of documents in the reduced space and evaluated the
preservation of both local and global structure within the data. We paid
particular attention to the semantic similarity between documents, as captured
by the cosine distance metric, and assessed the effectiveness of different
nearest neighbor values in retaining relevant information. By systematically
fine-tuning these hyperparameters and carefully evaluating the quality of the
generated topics and embeddings, we identified the values of 50 associated
documents per topic, 50 documents per cluster, and 20 nearest neighbors for
the UMAP projection that yielded the most coherent, diverse, and meaningful
results. These values were selected to ensure that the topics and clusters
were robust, representative, and capable of providing comprehensive insights
into the mental health research landscape.

The trained BERTopic model yielded 202 topics. These topics represent clusters
of related words and phrases that frequently co-occur in the text and have
been ranked based on their coherence and distinctiveness scores. Analyzing
these topics provides valuable insights into the major themes and trends
present in the dataset. Furthermore, utilizing BERTopic allows us to explore
the relationships between different topics and visualize them.

To visualize the relationships between the generated topics, we applied UMAP
once again to reduce the dimensionality of the topic embeddings to 2 and
plotted them as a scatterplot. In Figure 1, we present a 2D UMAP visualization
of the clusters generated by BERTopic. This visualization offers a
comprehensive view of the distribution of topics in a 2D space, providing a
visual summary of the relationships between different topics and shedding
light on the overall structure of the dataset. The scatterplot clearly
exhibits distinct clusters of topics, where topics that are closer to each
other in the plot belong to the same cluster. The spread and distribution of
topics in the scatterplot reflect the diversity of the mental health research
landscape. The scatterplot demonstrates a wide range of themes and subtopics,
with topics appearing in different regions of the plot. This diversity
indicates the presence of various research directions and perspectives within
the dataset. By considering the distinct clusters, topic proximity, and
overall distribution of topics in the scatterplot, we can gain insights into
the major themes and trends in mental health research.

Figure 2 provides valuable insights into the key themes and topics in mental
health research, serving as a useful resource for identifying relevant words
and topics for further analysis. The word scores associated with the top 8
mental health-related topics indicate the relevance of specific words to each
topic. On the x-axis, the word scores range from 0 to 1, while the y-axis
represents the topics numbered from 0 to 7. Higher word scores indicate a
stronger association between the word and the corresponding topic.

Each topic is represented by a cluster of words, and the word scores indicate
the degree to which each word is associated with the topic. As seen in the
figure, topic 2 represents anxiety during pandemic, words such as “pandemic”,
“anxiety”, “health”, “psychological” and “worker” have high scores, indicating
their strong association with this topic. Similarly, in topic 3, which is
related to sleep quality and insomnia, words such as “sleep”, “insomnia”,
“sleep quality”, and “sleep disturbance” have high scores. In addition, we
observed that Topic 4 was related to stigma and mental illness, while Topic 6
was related to genetic factors, such as genes, schizophrenia, variants, and
disorders. These findings suggest that there are multiple factors that
contribute to mental illness and schizophrenia, including both social and
biological factors.

Figure 3 shows the similarity matrix between topics, which provides a valuable
insight into the relationships between different topics in a given field. By
analyzing the similarity matrix, we can identify clusters of topics that are
closely related to each other and those that are more distant. This
information can help us understand the underlying structure of the field and
identify the most important topics and their interconnections. The fact that
topics 6 (genetic gene schizophrenia) and 22 (variant gene cell) have a high
similarity score 0.56 that suggests there may be a relationship between the
variant gene cell and schizophrenia, which has been reported in several
studies. Similarly, the high similarity score of 0.54 between topics 2
(pandemic anxiety health) and 8 (student university medication) indicates a
potential relationship or overlap between these two topics. This finding
suggests that there may be a connection between pandemic-related anxiety and
issues related to students, universities, and medication. For example, the
paper titled “Anxiety and Stress Levels Associated With COVID-19 Pandemic of
University Students in Turkey: A Year After the Pandemic” published in the
journal Frontiers in Psychiatry provides relevant information regarding the
potential relationship between pandemic anxiety and university students. The
research investigated the psychological impact of the pandemic on students and
how it affected their mental health and well-being. The paper provides
insights into the specific context of university students in Turkey and their
experiences during the pandemic. It sheds light on the potential connection
between pandemic anxiety and the well-being of students in a university
setting. The paper could potentially contribute to the understanding of how
pandemic-related anxiety impacts the mental health of university students and
explore potential interventions, such as the use of medication, to mitigate
these effects. By identifying such relationships through the similarity
matrix, researchers can gain a better understanding of the underlying
structure of the field and the interconnections between different topics. This
information can be valuable for various purposes, such as identifying
important topics, exploring potential research directions, or discovering
novel connections between seemingly unrelated topics.

Figure 4 depicts the hierarchical clustering of the Top 50 mental health-
related topics. Hierarchical clustering allows topics to be organized into
hierarchical structures or clusters based on their similarity, allowing for a
more nuanced exploration of topic relationships. It can also help in
summarizing large sets of topics by identifying representative topics within
each cluster. For instance, topics 1 and 15 are found to be nearly identical,
sharing similar content and underlying themes. Similarly, topics 17 and 36
revolve around the child-parent relationship and its impact on autism. The
high similarity and hierarchical relationship between these two topics
highlights the significant influence that the child-parent relationship plays
a crucial role in the manifestation of autism. Furthermore, topics 9, 16, and
24 focus on distinct substances, namely tobacco, cannabis, and opioid use,
respectively. These topics fall under the broader category of substance abuse.

In Figure 5, we harnessed the power of the gpt-3.5-turbo model from OpenAI API
by providing it with a designed prompt. The prompt guided the API to assist us
in identifying and filtering the primary machine learning methods employed in
these research papers, as shown in Table 2. We further processed the data to
generate a WordCloud figure. This figure visually represents the prevalence
and prominence of different machine learning models within the field of mental
health research across the specified time period. Each model is represented by
a word, with the size of the word indicating its frequency and significance in
the literature.

Our analysis revealed notable patterns in the usage of machine learning models
over time. We observed a significant increase in the application of machine
learning techniques as the years progressed. Additionally, prior to 2017,
traditional machine learning methods were predominantly employed. However,
after 2017, there was a remarkable surge in the utilization of neural
networks, accompanied by the incorporation of natural language processing
(NLP), computer vision (CV), and reinforcement learning (RL) algorithms. The
WordCloud figure serves as a visual representation of these trends, offering a
comprehensive snapshot of the evolving landscape of machine learning models in
mental health research.

To evaluate the performance of the different topic models, we used a set of
comprehensive metrics that assess topic coherence, diversity, and clustering
quality. One of the metrics we employed was the NPMI (Normalized Pointwise
Mutual Information), which measures the coherence of topic words by evaluating
their association strength. A higher NPMI score, ranging from -1 to 1,
indicates a stronger association among the topic words. We also utilized the
Cv (Coefficient of Topic Coherence) measure, which calculates coherence using
a larger sliding window and indirect cosine similarity. This measure has shown
a high correlation with human judgment and ranges from 0 to 1, with 1
representing perfect association.

In addition to coherence, we measured the diversity of the topics using two
metrics. The first metric, TD (Topic Diversity), calculates the percentage of
unique topic words, with a score of 1 indicating that all topic words are
different. The second metric, Inv. RBO (Inverted Rank-Biased Overlap), also
measures topic diversity. It is a rank-weighted measure that gives less weight
to words at higher ranks. Again, this metric ranges from 0 to 1, with 1
indicating all different topic words.

We presented the evaluation results of the BERTopic, LDA-BERT, and Top2Vec
algorithms in Table 3. The results show that BERTopic outperformed the other
algorithms in terms of TD and Cv scores. It achieved a significantly higher TD
score of 0.7208 and Topic Coherence scores of 0.2892 (NPMI) and 0.8068 (Cv).
Interestingly, Top2Vec performed well in terms of Inv. RBO with a score of
0.9669, although there is a smaller gap between Top2Vec and BERTopic in this
metric.

##  6 Conclusion and Future Work

In conclusion, our study analyzed a large dataset of mental health research
papers to identify trends and high-impact research topics. The heightened
interest in mental health, particularly in response to the COVID-19 pandemic,
has led to a surge in publications dedicated to this field. By leveraging a
Sentence-BERT based embedding model and employing topic modeling techniques,
we successfully examined topic evolution. The model demonstrated superior
performance in diverse topic modeling metrics, indicating its effectiveness in
generating distinct, coherent, and interpretable topics. Additionally, by
leveraging OpenAI GPT API, we extracted and analyzed machine learning models
mentioned in the research papers, revealing computational techniques prevalent
in mental health research. The resulting word cloud offered a comprehensive
overview of the commonly utilized techniques and emerging trends. Overall, our
findings contribute to understanding the landscape of mental health research
landscape and provide insights for researchers and practitioners seeking to
address mental health challenges using computational techniques.

We plan to expand beyond research papers to include diverse data sources like
clinical records, social media data, and patient narratives in our future
work. Incorporating these sources can offer a holistic view of mental health
revealing new research directions and intervention strategies.

## References

  * Angelov (2020) Angelov, D.  Top2vec: Distributed representations of topics.  _arXiv preprint arXiv:2008.09470_ , 2020.
  * Basmatkar & Maurya (2022) Basmatkar, P. and Maurya, M.  An overview of contextual topic modeling using bidirectional encoder representations from transformers.  In _Proceedings of Third International Conference on Communication, Computing and Electronics Systems: ICCCES 2021_ , pp. 489–504. Springer, 2022.
  * Blei et al. (2003) Blei, D. M., Ng, A. Y., and Jordan, M. I.  Latent dirichlet allocation.  _Journal of machine Learning research_ , 3(Jan):993–1022, 2003.
  * Chauhan & Shah (2021) Chauhan, U. and Shah, A.  Topic modeling using latent dirichlet allocation: A survey.  _ACM Computing Surveys (CSUR)_ , 54(7):1–35, 2021\.
  * del Rio-Chanona et al. (2022) del Rio-Chanona, R. M., Hermida-Carrillo, A., Sepahpour-Fard, M., Sun, L., Topinkova, R., and Nedelkoska, L.  Mental health concerns prelude the great resignation: Evidence from social media.  _arXiv preprint arXiv:2208.07926_ , 2022.
  * Ferdinand Kapl (2022) Ferdinand Kapl, Yuyin Lang, B. S.  Otmisc: Our topic modeling is super cool, 2022.  URL https://github.com/berksudan/OTMISC-Topic-Modeling-Tool.
  * Gaur et al. (2018) Gaur, M., Kursuncu, U., Alambo, A., Sheth, A., Daniulaityte, R., Thirunarayan, K., and Pathak, J.  ” let me tell you about your mental health!” contextualized classification of reddit posts to dsm-5 for web-based intervention.  In _Proceedings of the 27th ACM International Conference on Information and Knowledge Management_ , pp. 753–762, 2018.
  * Gong & Poellabauer (2017) Gong, Y. and Poellabauer, C.  Topic modeling based multi-modal depression detection.  In _Proceedings of the 7th annual workshop on Audio/Visual emotion challenge_ , pp. 69–76, 2017.
  * Grootendorst (2022) Grootendorst, M.  Bertopic: Neural topic modeling with a class-based tf-idf procedure.  _arXiv preprint arXiv:2203.05794_ , 2022.
  * Lee & Seung (2000) Lee, D. and Seung, H. S.  Algorithms for non-negative matrix factorization.  _Advances in neural information processing systems_ , 13, 2000.
  * Leung & Khalvati (2022) Leung, Y. T. and Khalvati, F.  Exploring covid-19 related stressors using topic modeling.  _arXiv preprint arXiv:2202.00476_ , 2022.
  * Lin et al. (2022) Lin, B., Bouneffouf, D., Cecchi, G., and Tejwani, R.  Neural topic modeling of psychotherapy sessions.  _arXiv preprint arXiv:2204.10189_ , 2022.
  * McInnes et al. (2017) McInnes, L., Healy, J., and Astels, S.  hdbscan: Hierarchical density based clustering.  _J. Open Source Softw._ , 2(11):205, 2017.
  * McInnes et al. (2018) McInnes, L., Healy, J., and Melville, J.  Umap: Uniform manifold approximation and projection for dimension reduction.  _arXiv preprint arXiv:1802.03426_ , 2018.
  * OpenAI (2023a) OpenAI.  OpenAI API.  OpenAI, 2023a.  URL https://openai.com.  Accessed on May 31, 2023.
  * OpenAI (2023b) OpenAI.  GPT-3.5-turbo Model.  2023b.  URL https://openai.com.  Accessed on May 31, 2023.
  * Reimers & Gurevych (2019) Reimers, N. and Gurevych, I.  Sentence-bert: Sentence embeddings using siamese bert-networks.  _arXiv preprint arXiv:1908.10084_ , 2019.
  * Wallach (2006) Wallach, H. M.  Topic modeling: beyond bag-of-words.  In _Proceedings of the 23rd international conference on Machine learning_ , pp. 977–984, 2006.
  * Yanchuk et al. (2022) Yanchuk, A., Trifan, A., Fajarda, O., and Oliveira, J. L.  Automatic classification of stigmatizing articles of mental illness: The case of portuguese online newspapers.  In _New Trends in Database and Information Systems: ADBIS 2022 Short Papers, Doctoral Consortium and Workshops: DOING, K-GALS, MADEISD, MegaData, SWODCH, Turin, Italy, September 5–8, 2022, Proceedings_ , pp. 332–343. Springer, 2022.
  * Zeberga et al. (2022) Zeberga, K., Attique, M., Shah, B., Ali, F., Jembre, Y. Z., and Chung, T.-S.  A novel text mining approach for mental health prediction using bi-lstm and bert model.  _Computational Intelligence and Neuroscience_ , 2022, 2022.

##  Appendix A Appendix

Figure 6 presents an intriguing analysis of the changing landscape of mental
health topics over a span of twelve years. Through the utilization of
WordCloud visualizations, this study aims to capture the key themes and trends
surrounding mental health discussions across multiple years, spanning from
2011 to 2022. By comparing WordClouds from different time periods, we gain
valuable insights into the evolving nature of public discourse surrounding
mental health, shedding light on the shifting focus and priorities within this
critical field. This figure serves as a visual representation, enabling
researchers and practitioners to observe and explore the dynamic nature of
mental health concerns, ultimately facilitating a deeper understanding of the
societal factors that shape our attitudes and responses to mental well-being.

Figure 6: Temporal Evolution of Mental Health Topics: A Comparative WordCloud
Analysis Across Different Years (2011-2022)


#  Life of PII – A PII Obfuscation Transformer

Ajinkya Deshmukh
Synechron Data Practice (Pune, India)
<EMAIL_ADDRESS>
& Saumya Banthia
Synechron Data Practice (Charlotte, US)
<EMAIL_ADDRESS>
& Anantha Sharma
Synechron Data Practice (Charlotte, US)
<EMAIL_ADDRESS>
###### Abstract

Protecting sensitive information is crucial in today’s world of Large Language
Models (LLMs) and data-driven services. One common method used to preserve
privacy is by using data perturbation techniques to reduce overreaching
utility of (sensitive) Personal Identifiable Information (PII) data while
maintaining its statistical and semantic properties. Data perturbation methods
often result in significant information loss, making them impractical for use.
In this paper, we propose ’Life of PII’, a novel Obfuscation Transformer
framework for transforming PII into faux-PII while preserving the original
information, intent, and context as much as possible. Our approach includes an
API to interface with the given document, a configuration-based obfuscator,
and a model based on the Transformer architecture, which has shown high
context preservation and performance in natural language processing tasks and
LLMs.

Our Transformer-based approach learns mapping between the original PII and its
transformed faux-PII representation, which we call "obfuscated" data. Our
experiments demonstrate that our method, called Life of PII, outperforms
traditional data perturbation techniques in terms of both utility preservation
and privacy protection. We show that our approach can effectively reduce
utility loss while preserving the original information, offering greater
flexibility in the trade-off between privacy protection and data utility. Our
work provides a solution for protecting PII in various real-world
applications.

##  1 Introduction

The Use of LLMs like Chat-GPT is increasing in the world, people from all
backgrounds have started to use these tools to keep at pace with the world and
to make their lives easier. Many financial, insurance companies and investment
banks want to use these LLMs to satisfy their requirements. One of the main
challenges which they are facing is maintaining their data privacy because
these LLMs are not yet hostable on their private servers. Most financial
institution’s data is rich in PII and therefore carries with it an extra risk,
sending it out. One solution to this problem is to transform and obfuscate
this data prior to sending it to the LLMs like Chat-GPT, get the response from
the LLMs and then again re-transform this obfuscated response to get the final
answer. This is what our approach discusses as part of this paper.

In a previous work, we had explored transformer-based models for use in
Question Answering [1]. This gives us the opportunity and scope to provide a
solution to this problem. Our approach provides transforming and re-
transforming of data within the organization maintaining data privacy. It also
ensures that transformed and obfuscated data retains its semantic meaning.
This paper focuses on the use of python dependency modules and Natural
Language Processing techniques to provide transforming and re-transforming
facilities.

##  2 Methodology

To transform the data, three transformation techniques were implemented which
are User Provided Tokens (UPT) Transformation, Named Entity Recognition (NER)
Transformation and Part of Speech (PoS) Transformation and checked response
from LLMs for different combinations of these transformation techniques. Flow
for text transformation and LLM question and answering system is as below:

Figure 1: Proposed Text Transformation Flow for LLMs Question and Answering

The step for synonyms replacement is carried out to ensure transformation of
that word and also to avoid different transformation for the same word.
Transformation techniques used are as below:

###  2.1 UPT Transformation

UPT Transformation consists of providing words (which user wants to hide)
inside configuration and providing tokens for those words. Then after applying
UPT Transformation, these words will be replaced with the tokens user has
provided. For example, when user provides token ‘D202’ for the word ‘Krypton’
and token ‘Meridian’ for word ‘Eastern Richard’. Result of this transformation
will be:

Original Text |  The Eastern Richard Company Monthly Status Report states that it is performing good, but Project Krypton has a red status.
---|---
UPT Transformed Text |  The Meridian Company Monthly Status Report states that it is performing good, but Project D202 has a red status.

As this transformation hid information user wanted to hide and at the same
time ensured semantic meaning. Word ‘Eastern Richard’ is hiding with word
‘Meridian’ and ‘Meridian’ is also keeping the semantic meaning as it sounds
like one company.

###  2.2 NER Transformation

NER Transformation consists of identifying if there are any named entities in
text [2] and replacing these named entities with some tokens. Considering same
example as above, result of NER transformation will be:

Original Text |  The Eastern Richard Company Monthly Status Report states that it is performing good, but Project Krypton has a red status.
---|---
NER Transformed Text |  N0 states that it is performing good, but N1 has a red status. .

It can be observed that named entities found in the original text are ‘The
Eastern Richard Company Monthly Status Report’ and ‘Project Krypton’ which
were replaced with ‘N0’ and ‘N1’.

###  2.3 PoS Transformation

PoS Transformation consists of identifying if there are parts of speech like
nouns in the text [2] and replacing these nouns with some tokens. Considering
same example as above, result of PoS transformation will be:

Original Text |  The Eastern Richard Company Monthly Status Report states that it is performing good, but Project Krypton has a red status.
---|---
PoS Transformed Text |  The P0 Report states that it is performing good, but P1 has a red status.

It can be observed that PoS I.e., nouns found in the original text are
‘Eastern Richard Company Monthly Status’ and ‘Project Krypton’ which were
replaced with ‘P0’ and ‘P1’.

###  2.4 IL

When user follows procedure of applying transformation technique on the data,
sending this data to LLM, getting transformed response, applying re
transformation and obtaining final response, it is possible that user can lose
some information in the response he obtains from the LLMs due to provision of
transformed data to LLMs compared with the response user has otherwise
obtained without any transformation. This can be understood with the example,
suppose ‘Response1’ is the response user obtained from LLMs without any
transformation and ‘Response2’ is the re-transformed response user obtained
from LLMs using some transformation technique.

Response1 = "According to our analysis, the company’s revenue for the first
quarter of 2023 increased by 15% compared to the same period last year,
reaching a total of $10 million. This growth was driven by a 20 % increase in
sales of our flagship product, which accounted for 60% of the total revenue.
However, operating expenses also increased by 10%, mainly due to higher
marketing and research and development costs. As a result, the company’s net
profit for the quarter was $1.2 million, a 12% increase from last year.
Overall, the company’s performance for the quarter was positive, but we
recommend monitoring expenses closely to maintain profitability. "

Response2= " According to our analysis, the company’s revenue for the first
quarter of 2023 increased compared to the same period last year. This growth
was driven by a 20% increase in sales of our flagship product, which accounted
for 60% of the total revenue. However, operating expenses also increased,
mainly due to higher marketing and research and development costs. As a
result, the company’s net profit for the quarter was increased from last year.
Overall, the company’s performance for the quarter was positive, but we
recommend monitoring expenses closely to maintain profitability. "

As this can be seen from responses, Response2 lacks some of the important
information like ‘company’s revenue increased by 15%’, ‘company’s revenue is
of $10 million’, ‘operating expenses also increased by 10%’,’ company’s net
profit for the quarter was $1.2 million, a 12% increase’ which were there in
Response1. So, this information which is lost from Response2 as compared with
Response1 is called ‘Information loss.’ This information can be crucial at
many times so transformation techniques need to be chosen in such a way that
the user will be having minimum information loss or information loss which can
be tolerated. Information Loss (IL) is expressed in percentage like 20%
information loss. Two methods were used here for the calculation of the IL.
Information loss is expressed as

IL = (0.5*Manual information loss) + (0.5*Similarity based Information loss)

Similarity based Information loss (ILS): This method uses hugging face
sentence transformer model to calculate cosine similarity between our two
responses. (e.g., Response 1 and Response 2). It converts the Responses into
embedding and then compares how similar these sentences are based on cosine
similarity and comes back with similarity score ranging from 0 to 1. ILS is
calculated as

ILS = 1 – Similarity score

Manual information loss (ILM): One drawback of ILS is it can give information
loss more than 0% even if both responses meanings are same. For example, if
user provides context as “Mango is Fruit”, ask question as “What is Mango?”
then one response can be “Mango is a fruit” and other response can be “A
fruit”. Now both these responses are giving the right answer to the question
and there is no information loss here, but ILS can still give information loss
greater than 0 as these two responses are not same and occupies different
positions in the vector space when compared from cosine similarity. To
overcome this, ILM is also taken into consideration, adding human into loop.
Manual Information loss is the loss in which human analyses both responses
(e.g., Response1 and Response2), compares loss of information and then
provides information loss ranging from 0 to 1. Here for comparison, a person
can consider information which he/she finds important, it can be some figures,
names or anything. Method of calculation of ILM is given as

ILM = Information loss from response obtained after use of Transformation
Technique / Total important information in response obtained without use of
Transformation Technique

Consider Response1 and Response2 to understand ILM. So important information
lost from Response2 are figures like “15%”, “total of $10 million”, “by 10%”,
“$1.2 million” and “a 12% increase”. So, this count is 5. Total important
information in Response1 is “15%”, “total of $10 million”, “by 10%”, “$1.2
million” and “a 12% increase”, “first quarter of 2023”, “20% increase”,
“flagship product”, “60% of the total revenue” and count of this is 9. So, ILM
is 5/9 = 0.55

###  2.5 Transformation cycle

A complete process of applying Transformation Technique to data, sending this
data to the LLMs, obtaining transformed response from LLMs, applying re-
transformation on this response and obtaining final response is called as
‘Transformation cycle’. This term is used in paper for ease of understanding
and to avoid describing this cycle again and again.

###  2.6 STT

When Transformation cycle is applied using UPT Transformation and suppose user
has provided tokens for the words on which LLMs are trained on, then it can be
possible that LLMs can use their trained understanding as a response, which
means those are sensitive to some of tokens. This can be understood with one
example, suppose user has given token ‘Mango’ for the word ‘Rose’.
Transformation cycle is as below.

Original Text |  Rose is a flowering plant that is widely recognized for its beauty, fragrance, and symbolic significance. It belongs to the family Rosaceae and is native to Asia but is now cultivated in many parts of the world. Rose comes in a variety of colors, such as red, pink, yellow, and white, and is commonly used in gardens, bouquets, and various decorative arrangements.
---|---
UPT Transformed Text |  Mango is a flowering plant that is widely recognized for its beauty, fragrance, and symbolic significance. It belongs to the family Rosaceae and is native to Asia but is now cultivated in many parts of the world. Mango comes in a variety of colors, such as red, pink, yellow, and white, and is commonly used in gardens, bouquets, and various decorative arrangements.
Question |  What is Rose?
---|---
UPT Transformed question along with transformed context |  The Mango is a flowering plant that is widely recognized for its beauty, fragrance, and symbolic significance. It belongs to the family Rosaceae and is native to Asia but is now cultivated in many parts of the world. Mango comes in a variety of colors, such as red, pink, yellow, and white, and is commonly used in gardens, bouquets, and various decorative arrangements. What is Mango?
Transformed Response from LLMs |  I’m sorry, but the information you have provided about Mango is not accurate. Mango is not a member of the Rosaceae family; it belongs to the Anacardiaceous family. Mango is also not typically used in gardens or decorative arrangements, as it is a fruit that is typically eaten fresh or used in cooking. Mango comes in a variety of colors, including green, yellow, and orange, but not pink or white. Mango trees do produce flowers, but the fruit is the most well-known and widely used part of the plant.
Final Re-transformed Response |  I’m sorry, but the information you have provided about Rose is not accurate. Rose is not a member of the Rosaceae family; it belongs to the Anacardiaceous family. Rose is also not typically used in gardens or decorative arrangements, as it is a fruit that is typically eaten fresh or used in cooking. Rose comes in a variety of colors, including green, yellow, and orange, but not pink or white. Rose trees do produce flowers, but the fruit is the most well-known and widely used part of the plant.

So, user has provided information that ‘Mango is Flower’. This is user
provided knowledge, this information can conflict with the information of
‘Mango is fruit’ on which LLMs are already trained on. When user asks to LLMs
that ‘what is Mango?’ then it gives response as ‘Mango is fruit’ so it means
that even if user has provided information that ‘Mango is flower’, still it is
using data on which it is pre-trained on which means that it is sensitive to
this kind of information. So likewise, LLMs can be sensitive to many types of
words and will give a different response than expected, so this is called
Sensitivity to Transformation Technique (STT), and this should be considered
into effect before giving tokens otherwise user will be getting responses
which will come from LLMs trained understanding rather than information user
is providing to it. Also, LLMs can give out of context answers from data on
which it is already trained on. STT is expressed as either of these formats
like yes or no, 100% or 0%, 1 or 0. STT of 100% or 1 means LLMs are sensitive
to Transformation Technique and will use its pre-trained knowledge than using
our provided information to generate the response.

###  2.7 Prompt Engineering

Prompt engineering refers to the process of designing effective prompts or
input examples that help LLMs learn to perform a specific task. It can also be
seen as a way of communicating more effectively with LLMs, such that the
resulting output adheres more closely to the context and constraints within
which the problem needs to be addressed.

This is a need more than a good to have, as LLMs are prone to hallucinating -
generating real looking responses that are factually inaccurate. As part of
our technique, we use Prompt Engineering for two main reasons:

  * •

To act as a constraint, making the output relevant to the context - LLMs give
out of context answers in their vanilla state.

  * •

To return a pre-defined response if the question goes out of context - We do
this by POS Transformation-prompting, which means adding an instructional
prompt at the end of each context and question pair. There are better ways to
do this, such as prompt evaluation, which would be beyond the scope of
discussion in this paper.

##  3 Experimental Results

We have compared responses from LLMs for different combinations of UPT
Transformation, NER Transformation and PoS Transformation. In different
combinations, Transformation Technique is used alone, or different stages are
applied like performing other Transformation Technique on top of the first
Transformation Technique. As an example, UPT +NER means first UPT
Transformation is applied and then NER Transformation. At the time of re-
transformation also the same stages of re-transformation will be applied to
get the clarified response. Like first NER re-transformation will be done and
then UPT re-transformation to obtain response. We obtained responses for three
kinds of questions.

  1. 1.

Pointed questions: These are straightforward questions having one- or two-line
answers. Sample question will be ‘who is owner of Facebook?’ and answer will
be ‘The owner of Facebook is Mark Zuckerberg.’

  2. 2.

Key questions: These are more complex questions having more than 2 lines of
answers. Example question will be ‘What are key accomplishments.

  3. 3.

Summarizing questions: These are questions for summarizing reports. Example
question will be ‘Summarize this report’.

We asked 40 questions belonging to the above three kinds to LLM (LLM
considered here is Chat-GPT), calculated STT, ILM, ILS and IL for all these
responses and calculated average of these measures for each Transformation
Technique and these are presented in Table 1.1.

Transformation Technique |  STT |  ILM |  ILS |  IL
---|---|---|---|---
UPT |  0.00% |  1.28% |  12.70% |  6.99%
NER |  7.69% |  35.90% |  16.40% |  26.15%
PoS |  2.56% |  13.85% |  14.12% |  13.98%
UPT +NER |  2.56% |  28.97% |  27.94% |  28.46%
UPT \+ PoS |  0.00% |  22.82% |  19.95% |  21.39%
NER \+ PoS |  5.13% |  32.95% |  26.72% |  29.84%
UPT \+ NER \+ PoS |  0.00% |  43.08% |  33.80% |  38.44%

Table 1.1: Experiment Result for questions

From Table 1.1, IL is observed minimum for the UPT and maximum for the UPT
+NER +PoS. This seems obvious as it has more stages of Transformation
Technique applied over it. All Other techniques give IL in similar range. From
the above information losses which were obtained based on the final responses
from LLMs, it is observed that we are outperforming traditional data
perturbation techniques in terms of both utility preservation and privacy
protection as LLMs has given these responses based on our Transformation
Technique. Also, STT is observed in Transformation Technique like NER, PoS,
UPT +NER and NER +PoS. It means that for these techniques for few of
questions, LLMs was sensitive or has given answer from its data on which it
was pre-trained on.

Table 1.2 shows IL and STT for the final responses for questions when Prompt
Engineering is used.

Transformation Technique |  STT |  ILM |  ILS |  IL
---|---|---|---|---
UPT |  0.00% |  1.28% |  12.70% |  6.99%
NER |  0.00% |  34.21% |  23.62% |  28.48%
PoS |  2.56% |  13.85% |  14.12% |  13.98%
UPT \+ NER |  0.00% |  28.97% |  29.82% |  29.39%
UPT \+ PoS |  0.00% |  22.82% |  19.95% |  21.39%
NER \+ PoS |  0.00% |  32.95% |  31.14% |  32.04%
UPT \+ NER \+ PoS |  0.00% |  43.08% |  33.80% |  38.44%

Table 1.2: Experiment Result for questions with Prompt Engineering

As it can be observed that Prompt Engineering has reduced STT to zero almost
for all the techniques.

##  4 Conclusion

We have provided and tested Life of PII for sending information to LLMs
ensuring protection of critical information and keeping the semantic meaning
thereby ensuring appropriate responses from the LLMs. Selection of
Transformation Technique to use depends on the type of use case. For use cases
where only a few terms/information needs to be protected and requires minimum
information loss, UPT can be used. Use cases where protecting most of the
information is a top priority and information loss is on less priority, UPT
+NER +PoS can be used. If balance of both protecting information as well as
less information loss is required, then other Transformation Technique can be
used. Also, to ensure LLMs give response within context, prompt engineering
can be used. Currently in UPT, tokens are manually provided by the user to
have full control over the technique which can make the process a little
slower. In the future we will work on automating token provision process so
that it can make process faster and remove task of user to add appropriate
tokens.

## References

  * [1] Chejui Liao, Tabish Maniar, Sravanajyothi N, and Anantha Sharma.  Techniques to Improve Q&A Accuracy with Transformer-based models on Large Complex Documents, 2020.  https://arxiv.org/abs/2009.12695.
  * [2] Peng Qi, Yuhao Zhang, Yuhui Zhang, Jason Bolton, and Christopher D. Manning.  Stanza : A Python Natural Language Processing Toolkit for Many Human Languages.  https://nlp.stanford.edu/pubs/qi2020stanza.pdf.