# Sub-posets in $\omega^{\omega}$ and The Strong Pytkeev∗ Property

Ziqin Feng  Department of Mathematics and Statistics
Auburn University
Auburn, AL 36849<EMAIL_ADDRESS>and  Naga Chandra Padmini Nukala
Department of Mathematics and Statistics
Auburn University
Auburn, AL 36849<EMAIL_ADDRESS>
###### Abstract.

Tukey order are used to compare the cofinal complexity of partially order sets
(posets). We prove that there is a $2^{\mathfrak{c}}$-sized collection of sub-
posets in $2^{\omega}$ which forms an antichain in the sense of Tukey
ordering. Using the fact that any boundedly-complete sub-poset of
$\omega^{\omega}$ is a Tukey quotient of $\omega^{\omega}$, we answer two open
questions published in [FKL16].

The relation between $P$-base and strong Pytkeev∗ property is investigated.
Let $P$ be a poset equipped with a second-countable topology in which every
convergent sequence is bounded. Then we prove that any topological space with
a $P$-base has the strong Pytkeev∗ property. Furthermore, we prove that every
uncountably-dimensional locally convex space (lcs) with a $P$-base contains an
infinite-dimensional metrizable compact subspace. Examples in function spaces
are given.

###### Key words and phrases:

Tukey order, strong Pytkeev∗ property, $\omega^{\omega}$-base,
$\mathcal{K}(M)$-base, $P$-base, locally convex space (lcs), posets, function
spaces

###### 2010 Mathematics Subject Classification:

54D70, 06A06, 46B50

##  1\. Introduction

The neighborhood base at a point $x$ in a topological space $X$ is a poset
ordered by inverse set-inclusion. Posets, for example $\omega$,
$\omega^{\omega}$, and $\mathcal{K}(M)$ with $M$ being a separable metric
space, are used to measure the cofinal complexity of the neighborhood base at
a point. Here, $\omega^{\omega}$ is the family of all sequences of natural
numbers in the pointwise partial order, i.e. $f\leq g$ if and only if
$f(n)\leq g(n)$ for all $n\in\omega$; $\mathcal{K}(M)$ is the collection of
compact subsets of $M$ ordered by set-inclusion. Tukey order (see the
definition in Section 2) which was introduced in [Tuk40] early 20th century is
a powerful tool to compare the cofinal complexity of posets. Using Tukey
order, a space is first-countable at a point $x$ if the neighborhood base
poset at $x$ is a Tukey quotient of $\omega$. All the topological spaces in
this paper are assumed to be Tychonoff.

Let $P$ be an arbitrary poset. In general, a topological space $X$ is defined
to have a neighborhood $P$-base at an $x\in X$ if the neighborhood base poset
at $x$ is a Tukey quotient of $P$; and, the space $X$ has a $P$-base if it has
a neighborhood $P$-base at each $x\in X$. Clearly, a topological group has a
$P$-base if it has a neighborhood $P$-base at the identity. The concept of
$\omega^{\omega}$-base (or, $\mathfrak{G}$-base) was introduced in [FKLS06]
for locally convex spaces. Topological spaces with an $\omega^{\omega}$-base
have been intensively studied in recent years (see [Banakh2019], [BL18],
[LPT17], and [GKL15]). The authors in [DF20] investigated the compact spaces
with a $P$-base for some other posets, mainly $\mathcal{K}(M)$ where $M$ is a
separable metric space. In this paper, we consider the sub-posets in
$\omega^{\omega}$ and the relation between $P$-base and the strong Pytkeev∗
networks (see definition in Section 4). After the characterization of quasi-
barrreled lcs with an $\omega^{\omega}$-base was given in [CKS03], lcs with an
$\omega^{\omega}$-base attracted lots of attentions in functional analysis. In
the end of this paper, we investigate the property of compact subsets in lcs
with $P$-bases for some general poset $P$.

This paper is organized in the following way. In Section 3, we discuss the
sub-posets in $2^{\omega}$ and prove that there is a $2^{\mathfrak{c}}$-sized
collection of sub-posets in $2^{\omega}$ which forms an antichain in the sense
of Tukey ordering. Then, we answer two open questions posted in [FKL16] by
showing that: 1) any topological group with a $\Sigma_{2}$-base admits an
$\omega^{\omega}$-base; and 2), any separable metric space $M$ is polish if
$\mathcal{K}(M)\leq_{T}\Sigma$ for any unbounded and boundedly-complete proper
sub-poset $\Sigma$ in $\omega^{\omega}$.

In Section 4, we discuss the strong Pytkeev∗ property (see the definition in
Section 4). We give a sufficient condition for a free filter on a countable
set to be meager and then use it to prove that any space $X$ with a
neighborhood $P$-base at $x$ has a strong Pytkeev∗ network at the point given
that $P$ is a directed set equipped with a second-countable topology in which
every convergent sequence in $P$ is bounded. Hence any topological space with
such a $P$-base has the strong Pytkeev∗ property. We give an example of a
topological space with a $\mathcal{K}(\mathbb{Q})$-base but doesn’t have the
the strong Pytkeev property, here $\mathbb{Q}$ is the space of rationals. Then
we show that each uncountably-dimensional locally convex space (lcs) $E$ with
a $P$-base contains an infinite-dimensional metrizable compact subspace if $P$
is a directed set equipped with a second-countable topology in which every
convergent sequence in $P$ is bounded. This extended Theorem 1.2 in [BKP20].
Examples in function spaces are given.

##  2\. Preliminaries

For any topological space $M$, the poset $\mathcal{K}(M)$ is the collection
compact subsets of $X$ ordered by set-inclusion $\subseteq$. Hence, a
topological space $X$ has a neighborhood $\mathcal{K}(M)$-base at a point
$x\in X$, if there exists a neighborhood base
$\\{\mathcal{U}_{K}(x):{K\in\mathcal{K}(M)}\\}$ at $x$ such that
$\mathcal{U}_{L}(x)\subseteq\mathcal{U}_{K}(x)$ for all $K\subseteq L$ in
$\mathcal{K}(M)$.

For any topological space $X$, $C(X)$ is the collection of all continuous
real-valued functions. We use $C_{k}(X)$ ($C_{p}(X)$) to denote the
topological space $C(X)$ endowed with the compact-open topology (resp.,
pointwise convergence topology). A basic open neighborhood of $f\in C_{k}(X)$
($C_{p}(X)$) is the form $B(f,K,\epsilon)=\\{g:|f(x)-g(x)|<\epsilon\text{ for
all }x\in K\\}$ where $\epsilon>0$ and $K$ is a compact (resp., finite) subset
in $X$.

A filter on a set $X$ is a collection $\mathcal{F}$ of non-empty subsets of
$X$ which is closed under finite intersections and taking supersets in $X$. A
free filter is a filter with empty intersection. Identifying each subset of
$X$ with its characteristic function in $2^{X}$, we can equip $\mathcal{P}(X)$
with a compact Hausdorff topology. If $X$ is countable, we could identify
$\mathcal{P}(X)$ with $2^{\omega}$ which is a separable metric space.

Let $P$ be a directed poset, i.e. for any points $p,p^{\prime}\in P$, there
exists a point $q\in P$ such that $p\leq q$ and $p^{\prime}\leq q$. A subset
$C$ of $P$ is _cofinal_ in $P$ if for any $p\in P$, there exists a $q\in C$
such that $p\leq q$. Then $\text{cof}(P)=\min\\{|C|:C\text{ is cofinal in
}P\\}$. We also define $\text{add}(P)=\min\\{|Q|:Q\text{ is unbounded in
}P\\}$. Given two directed sets $P$ and $Q$, we say $Q$ is a Tukey quotient of
$P$, denoted by $P\geq_{T}Q$, if there is a map $\phi:P\rightarrow Q$, a Tukey
quotient, which carries cofinal subsets of $P$ to cofinal subsets of $Q$. A
map $\psi:Q\rightarrow P$ is said to be a Tukey map if $\psi(U)$ is unbounded
in $P$ whenever $U$ is unbounded in $Q$. It is known that $Q$ is a Tukey
quotient of $P$ if and only if there is a Tukey map from $Q$ to $P$. If $P$
and $Q$ are mutually Tukey quotients, we say that $P$ and $Q$ are Tukey
equivalent, denoted by $P=_{T}Q$. Gartside and Mamatelashvili in [GM16]
constructed a $2^{\mathfrak{c}}$-sized antichain in
$\mathcal{K}(\mathcal{M})=\\{\mathcal{K}(M):M\text{ is a separable and
metrizable space}\\}$.

######  Proposition 2.1.

Let $Q$ be a sub-poset of a poset $P$. If $Q$ is cofinal in $P$, then
$P=_{T}Q$.

###### Proof.

Let $Q$ be a cofinal subset of $P$. It is clear that the inclusion mapping
from $Q$ to $P$ is both a Tukey quotient and a Tukey map. Hence $Q$ and $P$
are Tukey equivalent. ∎

The posets in our discussion are naturally equipped with a topology, for
example, $\omega^{\omega}$ with product topology and $\mathcal{K}(M)$ with
Vietoris topology. One property of topological posets which plays important
role in several proofs is that every convergent sequence is bounded. This
property is weaker than being $\aleph_{0}$-directed, i.e., every countable set
is bounded. It is straightforward to see that $\omega^{\omega}$ with product
topology satisfies this property. It is proved [Banakh2019] that for any
monotone function $\phi:\omega^{\omega}\rightarrow P$ with cof$(P)\leq\omega$
and $f\in\omega^{\omega}$ there is an open neighborhood $O$ of $f$ such that
$\phi(O)$ is bounded. Next we prove this result holds for any poset with a
first-countable topology in which every convergent sequence is bounded.

######  Proposition 2.2.

Let $P$ be a directed set equipped with a first-countable topology such that
every convergent sequence is bounded. Suppose that $Q$ is a directed set with
$\text{add}(Q)=\omega$ and $f$ is any monotone mapping from $P$ to $Q$. For
any $p\in P$, there exists a neighborhood $B$ of $p$ such that
$f(B)=\\{f(b):b\in B\\}$ is bounded in $Q$.

###### Proof.

Let $\\{q_{n}:n\in\omega\\}$ be an unbounded subset in the poset $Q$. Take a
monotone mapping $f$ from $P$ to $Q$. Fix $p\in P$ and a countable decreasing
local basis $\\{B_{n}:n\in\omega\\}$ at $p$.

Suppose, for a contradiction, that $f(B_{n})$ is unbounded for each
$n\in\omega$. For each $n\in\omega$, pick $b_{n}\in B_{n}$ such that
$f(b_{n})\geq q_{n}$. Then $\\{b_{n}:n\in\omega\\}$ is a sequence in $P$ which
converges to $p$. By the assumption, there is a $p^{\ast}\in P$ such that
$p^{\ast}\geq b_{n}$ for each $n$ and $p^{\ast}\geq p$. Since $f$ is monotone,
$f(p^{\ast})\geq q_{n}$ for each $n\in\omega$. This is a contradiction because
$\\{q_{n}:n\in\omega\\}$ is unbounded in the directed set $Q$. ∎

If $M$ is a separable metric space, the poset $\mathcal{K}(M)$ is also a
separable metric space with the Hausdorff metric. It is straightforward to see
that if $\\{K_{n}:n\in\omega\\}$ is a sequence in $\mathcal{K}(M)$ converging
to $K$ then $K\cup\bigcup\\{K_{n}:n\in\omega\\}$ is also compact. Hence every
convergent sequence in $\mathcal{K}(M)$ is bounded above. By the lemma above,
we get the following corollary.

######  Corollary 2.3.

Let $M$ be a separable metric space. For every monotone function
$f:\mathcal{K}(M)\rightarrow Q$ where cof$(P)\leq\omega$ and every
$K\in\mathcal{K}(M)$, there exists a neighborhood $B$ of $K$ such that $f(B)$
is bounded.

A family $\mathcal{N}$ of subsets of a topological space $X$ is said to be a
Pytkeev-network at $x$ if for any neighborhood $O_{x}$ of $x$ and each subset
$A\subset X$ with $x\in\overline{A}$ there is a set $N\in\mathcal{N}$ such
that $x\in N\subset O_{x}$ , $N\cap A\neq\emptyset$, and moreover $N\cap A$ is
infinite if the set $A$ accumulates at $x$. A space $X$ is said to have the
strong Pytkeev property at $x\in X$ if it has a countable Pytkeev-network at
$x$; and, we say $X$ has the strong Pytkeev property if it has the strong
Pytkeev property at any $x\in X$.

##  3\. Sub-posets in $2^{\omega}$ and $\omega^{\omega}$

In this section, we consider the Tukey class of sub-posets in $2^{\omega}$,
more specifically its relation with the Tukey class of $\mathcal{K}(M)$ where
$M$ is a separable metric space.

######  Proposition 3.1.

For any separable metric space $M$, there is a sub-poset $\Sigma$ in
$2^{\omega}$ such that $\Sigma=_{T}\mathcal{K}(M)$.

###### Proof.

Let $M$ be a separable metric space and $D=\\{d_{n}:n\in\omega\\}$ be an
enumeration of a countable dense subspace of $M$. Define
$\Sigma=\\{Z\subset\omega:\overline{\\{d_{n}:n\in Z\\}}\text{ is compact in
}M\\}$ and a map $\phi:\Sigma\rightarrow\mathcal{K}(M)$ such that
$\phi(Z)=\overline{\\{d_{n}:n\in Z\\}}$. We claim that $\phi$ is both a Tukey
quotient and a Tukey map from $\Sigma$ to $\phi(\Sigma)$, i.e.,
$\Sigma=_{T}\phi(\Sigma)$.

Let $\mathcal{C}$ be a cofinal subset of $\Sigma$. Pick a $K\in\phi(\Sigma)$.
Then take any $Z\in\Sigma$ such that $\phi(Z)=K$. Then there is a
$C\in\mathcal{C}$ such that $C\supseteq Z$. Hence $K\subset\phi(C)$.
Therefore, $\phi(\mathcal{C})$ is cofinal in $\phi(\Sigma)$. Hence $\phi$ is a
Tukey quotient.

Let $\mathcal{U}$ be an unbounded subset of $\Sigma$. Assume, for a
contradiction, that $\phi(\mathcal{U})$ is bounded in $\phi(\Sigma)$, i.e.,
there is a compact subset $K$ of $M$ such that $K\in\phi(\Sigma)$ and
$\phi(Z)\subset K$ for each $Z\in\mathcal{U}$. Define
$Z_{0}=\bigcup\\{Z\in\Sigma:\phi(Z)\subset K\\}$. Then
$\overline{\\{d_{n}:n\in Z_{0}\\}}\subseteq K$, hence, it is compact.
Therefore, $Z_{0}\in\Sigma$ and $Z_{0}\supset Z$ for any $Z\in\mathcal{U}$
which contradicts with the unboundedness of $\mathcal{U}$. Therefore $\phi$ is
a Tukey map.

Then we prove that $\phi(\Sigma)$ is cofinal in $\mathcal{K}(M)$, hence by
Proposition 2.1, $\phi(\Sigma)=_{T}\mathcal{K}(M)$, furthermore,
$\Sigma=_{T}\mathcal{K}(M)$. Fix $K\in\mathcal{K}(M)$. Let
$\\{k_{i}:i\in\omega\\}$ be a countable dense subset of $K$. For each
$i\in\omega$, we fix a sequence $\\{a_{i,j}:j\in\omega\\}$ in $D$ such that
the sequence converges to $k_{i}$ for each $i\in\omega$ and the distance
between $a_{i,j}$ and $k_{i}$ is $\leq 1/j$ . Let
$S=\\{a_{i,j}:i,j\in\omega\text{ with }j\geq i\\}$. Then we define
$Z_{K}=\\{n:a_{i,j}=d_{n}\text{ for all }i,j\in\omega\text{ with }j\geq i\\}$
and $H=\overline{\\{d_{n}:n\in Z_{K}\\}}$. Clearly $K\subset H$. We claim that
$H$ is compact. Let $\mathcal{O}$ be an open cover for $H$. Then, there exists
a finite subcollection $\mathcal{O}^{\prime}$ of $\mathcal{O}$ such that
$K\subseteq\bigcup\mathcal{O}^{\prime}$. Then it is straightforward to verify
that $H\setminus\bigcup\mathcal{O}^{\prime}$ is finite, hence $H$ is compact.
Therefore, $Z_{K}$ is in $\Sigma$ and this shows that $\phi(\Sigma)$ is
cofinal in $\mathcal{K}(M)$. ∎

######  Theorem 3.2.

There is a $2^{\mathfrak{c}}$-sized family, $\mathcal{A}$, of sub-posets in
$2^{\omega}$ such that for distinct $P$ and $Q$ from $\mathcal{A}$ we have
$P\ngeq_{T}Q$ and $P\nleq_{T}Q$.

###### Proof.

Let $\mathcal{M}$ be a $2^{\mathfrak{c}}$-sized collection of separable metric
spaces such that for any pair $M,N\in\mathcal{M}$,
$\mathcal{K}(M)\neq_{T}\mathcal{K}(N)$. By Lemma 3.1, we pick a subset
$\Sigma_{M}$ of $2^{\omega}$ for each $M\in\mathcal{M}$ such that
$\Sigma_{M}=_{T}\mathcal{K}(M)$. Then $\\{\Sigma_{M}:M\in\mathcal{M}\\}$ is a
$2^{\mathfrak{c}}$-sized antichain in the collection $\mathcal{P}(2^{\omega})$
in term of Tukey ordering. ∎

A subset $\Sigma$ of $\omega^{\omega}$ is boundedly complete if each subset of
$\Sigma$ has a bound in $\Sigma$ given that it is bounded in
$\omega^{\omega}$. A $\Sigma$-base is called a $\Sigma_{2}$-base if $\Sigma$
is a boundedly-complete sub-poset in $\omega^{\omega}$. The concept of
$\Sigma_{2}$-bases was introduced in [FKL16]. Next result shows that any
boundedly-complete sub-poset of $\omega^{\omega}$ is a Tukey quotient of
$\omega^{\omega}$.

######  Theorem 3.3.

Any boundedly complete sub-poset of $\omega^{\omega}$ is a Tukey quotient of
$\omega^{\omega}$.

###### Proof.

Let $\Sigma$ be a boundedly complete sub-poset of $\omega^{\omega}$. Consider
the inclusion map $i:\Sigma\rightarrow\omega^{\omega}$. Because $\Sigma$ is
boundedly complete, a subset in $\Sigma$ is unbounded if and only if it is
unbounded in $\omega^{\omega}$. Hence the inclusion map is a Tukey map. Hence
$\Sigma$ is a Tukey quotient of $\omega^{\omega}$, i.e.,
$\Sigma\leq_{T}\omega^{\omega}$. ∎

Let $P$ and $Q$ be posets such that $P\leq_{T}Q$. It is straightforward to see
that any space with a $P$-base also has a $Q$-base (see Proposition 2.1 in
[DF20]). Hence, we get the following result.

######  Corollary 3.4.

Every topological space with a $\Sigma_{2}$-base admits an
$\omega^{\omega}$-base, i.e., a $\mathfrak{G}$-base.

As a special case, any topological group with a $\Sigma_{2}$-base admits an
$\omega^{\omega}$-base. This gives a positive answer to Problem 23 in [FKL16].

In his book [Ch74], Christensen proved (without using the Tukey order
notation) that: for any separable metric space $M$,
$\omega^{\omega}\geq_{T}\mathcal{K}(M)$ if and only if $M$ is Polish (in other
words, completely metrizable). By this result and Theorem 3.3, we prove the
following result which answers Problem 24 in [FKL16].

######  Theorem 3.5.

Let $M$ be a separable metric space admitting a compact ordered covering of
$M$ indexed by an unbounded and boundedly-complete proper subset of
$\omega^{\omega}$ that swallows the compact sets of $M$. Then the space $M$ is
a Polish space.

###### Proof.

Let $\Sigma$ be any unbounded and boundedly-complete proper subset of
$\omega^{\omega}$. Let $\\{K_{f}:f\in\Sigma\\}$ be a $\Sigma$-ordered compact
covering which swallows the compact sets of $M$, i.e.,
$\\{K_{f}:f\in\Sigma\\}$ is cofinal in $\mathcal{K}(M)$. By Proposition 2.1,
$\\{K_{f}:f\in\Sigma\\}=_{T}\mathcal{K}(M)$. The underlying mapping from
$\Sigma$ to $\\{K_{f}:f\in\Sigma\\}$ is clearly a Tukey quotient, hence
$\Sigma\geq_{T}\\{K_{f}:f\in\Sigma\\}$, furthermore,
$\Sigma\geq_{T}\mathcal{K}(M)$. Therefore,
$\omega^{\omega}\geq_{T}\mathcal{K}(M)$ by Theorem 3.3. Therefore, the space
$M$ is Polish by Christensen’s theorem. ∎

##  4\. The Strong Pytkeev∗ Property

In this section, we investigate the relation between $P$-bases and the strong
Pytkeev∗ property. The strong Pytkeev∗ property is introduced in [Banakh2016],
and also in [Banakh2019]; and it is implied by the strong Pytkeev property. It
is proved in [Banakh2016] that any topological space $X$ has the strong
Pytkeev property if and only if it is countably tight and has the strong
Pytkeev∗ property. A family $\mathcal{N}$ of subsets of $X$ is said to be a
Pytkeev∗-network (or an s∗)-network) at $x$ if for any neighborhood $O_{x}$ of
$x$ and any sequence $\\{x_{n}:n\in\omega\\}$ that accumulates at $x$ there
exists $N\in\mathcal{N}$ such that $N\subset O_{x}$ and the set $\\{n:x_{n}\in
N\\}$ is infinite. A space $X$ is said to have the strong Pytkeev∗ property at
$x\in X$ if it has a countable Pytkeev∗-network at $x$; and, we say $X$ has
the strong Pytkeev∗ property if it has the strong Pytkeev∗ property at any
$x\in X$.

First we give a sufficient condition for a free filter on a countable set
being meager. It is known (see [Banakh2019]) that for a submetrizable space
$X$, $X$ has a compact resolution if and only if it is analytic. Also each
analytic free filter on a countable set is meager. Hence, every free filter on
a countable set with a compact resolution is meager. A topological space has a
compact resolution if it has a $\omega^{\omega}$-ordered compact covering,
i.e., a family $\\{K_{f}:f\in\omega^{\omega}\\}$ of compact subsets of $X$
such that $X=\bigcup\\{K_{f}:f\in\omega^{\omega}\\}$ and $K_{f}\subseteq
K_{g}$ for every $f\leq g$ in $\omega^{\omega}$. With an extra condition, this
result could be extended to directed set equipped with a second-countable
topology in which every convergent sequence is bounded.

For the space $2^{X}$, a basic open neighborhood could be represented as
$[I,A]=\\{S:I\subset S\subset X\text{ and }S\cap A=\emptyset\\}$ where $I$ and
$A$ are finite subsets of $X$.

######  Proposition 4.1.

Let $P$ be a directed set equipped with a second-countable topology in which
every convergent sequence is bounded. A free filter $\mathcal{F}$ on a
countable set is meager if it has a $P$-ordered compact covering
$\mathcal{K}=\\{K_{p}:p\in P\\}$ with $\bigcap K_{p}$ being infinite for each
$p\in P$.

###### Proof.

Let $\mathcal{F}$ be a free filter on $\omega$ and $\mathcal{K}=\\{K_{p}:p\in
P\\}$ be a $P$-ordered compact covering of $\mathcal{F}$. We first show that
for each $p\in P$, there is a neighborhood $O$ of $p$ such that
$\bigcup\\{K_{q}:q\in O\\}$ is nowhere dense in $2^{\omega}$. Fix $p\in P$.
Assume, for a contradiction, that $\overline{\bigcup\\{K_{q}:q\in O\\}}$ has
nonempty interior for each neighborhood $O$ of $p$. We pick a decreasing local
base $\\{O_{n}:n\in\omega\\}$ at $p$ and define $C_{n}=\bigcup\\{K_{q}:q\in
O_{n}\\}$ for each $n\in\omega$. Then, for each $n\in\omega$ there exist
finite subsets $I_{n}$ and $A_{n}$ of $\omega$ such that
$[I_{n},A_{n}]\subset\overline{C_{n}}$. Without loss of generality, we could
assume that $I_{n}\neq\emptyset$ and $\max I_{n+1}\geq\max I_{n}+n$ for each
$n\in\omega$. Fix $n\in\omega$. The finite set $I_{n}$ is an element in
$\overline{C_{n}}$. Then $U_{n}=[I_{n},\\{m:\max I_{n}<m\leq\max I_{n+1}\\}]$
is a basic neighbor of $I_{n}$ in $2^{\omega}$. Hence, there exists a
$p_{n}\in O_{n}$ and an element $G_{n}\in K_{p_{n}}$ such that $G_{n}\in
U_{n}$. Note that $m\notin G_{n}$ for any $m$ with $\max I_{n}<m\leq\max
I_{n+1}$. Hence if $m>\max I_{0}$, $m\notin\bigcap\\{G_{n}:n\in\omega\\}$,
i.e., $\bigcap\\{G_{n}:n\in\omega\\}$ is finite. Since
$\\{O_{n}:n\in\omega\\}$ is a decreasing local base at $p$, the sequence
$\\{p_{n}:n\in\omega\\}$ converges to $p$. By the assumption, there is a
$p^{\ast}\in P$ such that $p^{\ast}\geq p_{n}$ for each $n\in\omega$.
Therefore, $G_{n}\in K_{p^{\ast}}$ for each $n\in\omega$. Then $\bigcap
K_{p^{\ast}}\subset\bigcap\\{G_{n}:n\in\omega\\}$ is finite which is a
contradiction.

Let $\mathcal{B}=\\{B_{n}:n\in\omega\\}$ be a countable base for $P$ and
$\mathcal{H}$ be the collection of $\bigcup\\{K_{q}:q\in B_{n}\\}$ which is
nowhere dense. By the result above, $\mathcal{F}=\bigcup\mathcal{H}$, hence
$\mathcal{F}$ is a meager set. ∎

We would need the famous characterization of meager filters due to Talagrand
in [Tal80]. A filter $\mathcal{F}$ on a countable set $X$ is meager subset of
the powerset $\mathcal{P}(X)$ if and only if there exists a finite-to-one map
$\phi:X\rightarrow\omega$ such that $\phi(F)$ has a finite complement in
$\omega$ for any $F\in\mathcal{F}$. A function from $X$ to $Y$ is said to be
finite-to-one if $f^{-1}(y)$ is finite for each $y\in Y$.

######  Theorem 4.2.

Let $P$ be a directed set equipped with a second-countable topology in which
every convergent sequence is bounded. If the topological space $X$ has a
$P$-base at $x$, then it has a countable Pytkeev∗-network at $x$.

###### Proof.

Fix a countable basis $\mathcal{B}=\\{B_{m}:m\in\omega\\}$ for $P$. Let
$\\{U_{p}:p\in P\\}$ be a $P$-ordered local base at $x$. Define
$U_{B_{m}}=\bigcap\\{U_{p}:p\in B_{m}\\}$. We show that
$\\{U_{B_{m}}:m\in\omega\\}$ is a s∗-network at $x$. Fix a sequence
$\\{x_{n}:n\in\omega\\}$ which accumulates at $x$. It is sufficient to show
that for every $p\in P$, there is an $m\in\omega$ such that $p\in B_{m}$ and
$U_{B_{m}}$ contains infinitely many terms in the sequence
$\\{x_{n}:n\in\omega\\}$.

If $\\{n:x_{n}=x\\}$ is infinite, then any $B_{m}$ with $p\in B_{m}$ satisfies
that $U_{B_{m}}$ contains infinite many terms in the sequence
$\\{x_{n}:n\in\omega\\}$. Without loss of generality, we assume that
$x_{n}\neq x$ for each $n\in\omega$. For each $p\in P$, define
$F(U_{p})=\\{n:x_{n}\in U_{p}\\}$; and then, let $\mathcal{F}$ be the filter
generated by $\\{F(U_{p}):p\in P\\}$. Clearly $\mathcal{F}$ is a free filter.
For each $p\in P$, define $K_{p}=\\{F:F\in\mathcal{F}\text{ and }F\supset
F(U_{p})\\}$. Then $K_{p}$ is compact and $\bigcap K_{p}=F(U_{p})$ is infinite
for each $p\in P$; also, $K_{p}\subseteq K_{p^{\prime}}$ if $p\leq
p^{\prime}$. So $\mathcal{F}$ is a free filter with a $P$-ordered compact
covering which satisfies the requirements in Lemma 4.1, hence it is a meager
set.

By Talagrand’s characterization of meager filters, there is a finite-to-one
mapping $\phi:\omega\rightarrow\omega$ such that $\phi(F)$ has finite
complement in $\omega$ for each $F\in\mathcal{F}$. Fix $p\in P$. Lastly, we
show that there is a $B_{m}\in\mathcal{B}$ such that $p\in B_{m}$ and
$U_{B_{m}}$ contains infinitely terms in the sequence
$\\{x_{n}:n\in\omega\\}$. Let $\\{O_{k}:k\in\omega\\}$ be a subcollection of
$\mathcal{B}$ which is a decreasing local base at $p$. Assume, for a
contradiction, that $U_{O_{k}}$ is finite for each $k\in\omega$. Let
$J_{k}=\\{n:x_{n}\in U_{O_{k}}\\}$. Because $\phi$ is finite-to-one, there is
a $y_{k}\in\omega$ such that $\phi^{-1}(y_{k})\neq\emptyset$ and
$J_{k}\cap\phi^{-1}(y_{k})$ is finite for each $k\in\omega$; moreover, we
could assume that $y_{k}<y_{k+1}$ for each $k\in\omega$. Because $P$ is a
directed set and $\phi^{-1}(y_{k})$ is finite for each $k\in\omega$, there
exists a $p_{k}\in O_{k}$ such that $x_{i}\notin U_{p_{k}}$ for each
$i\in\phi^{-1}(y_{k})$, i.e., $F(U_{p_{k}})\cap\phi^{-1}(y_{k})=\emptyset$ .
Clearly, $\\{p_{k}:k\in\omega\\}$ is a sequence in $P$ which converges to $p$.
Pick $p^{\ast}\in P$ such that $p^{\ast}\geq p$ and $p^{\ast}\geq p_{k}$ for
each $k\in\omega$, hence $F(U_{p^{\ast}})$ is a subset of $F(U_{p_{k}})$ for
each $k\in\omega$. Therefore, $\phi^{-1}(y_{k})\cap F(U_{p^{\ast}})=\emptyset$
for each $k\in\omega$, i.e., $\\{y_{k}:k\in\omega\\}$ is in the complement of
$\phi(F(U_{p^{\ast}}))$. This contradicts the finite-to-one property of the
function $\phi$. ∎

######  Corollary 4.3.

Let $P$ be a directed set equipped with a second-countable topology in which
every convergent sequence in $P$ is bounded. If the topological space $X$ has
a $P$-base, then it has strong Pytkeev∗ property.

Applying this result to topological groups, we obtain the following corollary.

######  Corollary 4.4.

Let $P$ be a directed set equipped with a second-countable topology in which
every convergent sequence in $P$ is bounded. If topological group $G$ with a
$P$-base at the identity, then $G$ has the strong Pytkeev∗ property.

It is proved in [sak08] that $C_{p}(X)$ has the strong Pytkeev property if and
only if $X$ is countable, i.e., $C_{p}(X)$ is metrizable. Hence, it is
reasonable to investigate a function space with compact-open topology. It is
straightforward to verify that $C_{k}(M)$ has a $\mathcal{K}(M)$-base for any
second-countable space $M$, hence the strong Pytkeev∗ property by the
corollary above. It is proved in [F20] that if $M$ is second-countable,
$C_{k}(M)$ is countably tight, hence has the strong Pytkeev property. So we
greatly generalized Theorem 1 in [TZ09] which shows that
$C_{k}(\omega^{\omega})$ has the strong Pytkeev property.

######  Theorem 4.5.

Let $M$ be any separable metric space. Then $C_{k}(M)$ has the strong Pytkeev
property.

Next, we give an example of $C_{k}(X)$ which has a $\mathcal{K}(M)$-base for
some separable metric space $M$, hence the strong Pytkeev∗ property, but it
doesn’t have the strong Pytkeev property.

######  Example 4.6.

There is a topological group with a $\mathcal{K}(M)$-base for some separable
metric space $M$, but not the strong Pytkeev property.

###### Proof.

Consider the function space $C_{k}(\omega_{1})$. Using the fact
$\omega_{1}\leq_{T}\mathcal{K}(\mathbb{Q})$ (see [GM17]), it is
straightforward to verify that $C_{k}(\omega_{1})$ has a
$\omega_{1}\times\omega$-base, hence a
$\mathcal{K}(\mathbb{Q})\times\omega$-base where $\mathbb{Q}$ is the space of
rationals. Therefore, $C_{k}(\omega_{1})$ has a $\mathcal{K}(\mathbb{Q})$-base
since $\mathcal{K}(\mathbb{Q})=_{T}\mathcal{K}(\mathbb{Q})\times\omega$.
Therefore, $C_{k}(\omega_{1})$ has the strong Pytkeev∗ property by the
corollary above. It is known (see [BL17]) that the strong Pytkeev property
implies countable tightness. Because $C_{k}(\omega_{1})$ is not countably
tight (see Example 9 in [F20]), it doesn’t have the strong Pytkeev property. ∎

It has been proven that any topological group with the strong Pytkeev property
admits a quasi-$\mathfrak{G}$-base $\\{U_{f}:f\in\Sigma\\}$ of the identity,
i.e., a neighborhood $\Sigma$-base of the identity for some sub-poset
$\Sigma\subset\omega^{\omega}$. Motivated by Theorem 4.2, it is natural to ask
the following question.

######  Question 4.7.

Let $G$ be a topological group with the strong Pytkeev property. Is it true
that $G$ has a $\Sigma$-base where $\Sigma$ is some sub-poset in
$\omega^{\omega}$ such that every convergent sequence is bounded?

Cascales and Orihuela in [CO87] proved that the precompact subsets of any lcs
with an $\omega^{\omega}$-base is metrizable. This result is generalized in
[Feng20] which proves that any precompact set of a topological group with a
$P$-base is metrizable if $P$ satisfies calibre $(\omega_{1},\omega)$. A
directed set $P$ satisfies calibre $(\omega_{1},\omega)$ if every uncountable
set in $P$ contains a countable bounded subset. It is straightforward to
verify the following result.

######  Proposition 4.8.

Let $P$ be a directed set equipped with a second-countable topology in which
every convergent sequence is bounded. Then $P$ has calibre
($\omega_{1},\omega$).

###### Proof.

Let $S$ be an uncountable subset of $P$. Since $P$ is second-countable, $S$
has a cluster point in $P$, namely, $p_{\ast}$. Then there is a sequence
$\\{p_{n}:n\in\omega\\}\subset S$ which converges to $p_{\ast}$ which is
bounded by the assumption. Hence $P$ has calibre ($\omega_{1},\omega$). ∎

It is known (see corollary 15.5 in [KKL11]) that a barrelled lcs with an
$\omega^{\omega}$-base is metrizable if and only if it doesn’t contains a copy
of $\varphi$ which is an $\aleph_{0}$-dimensional vector space endowed with
the finest locally convex topology. Also, any compact subset of $\varphi$ is
finite-dimensional. Motivated by these results, it is natural to investigate
the class of lcs in which every infinite-dimensional subspace contains an
infinite-dimensional compact metrizable subset. In [BKP20], one of the main
results is that every uncountably-dimensional lcs with an
$\omega^{\omega}$-base contains an infinite-dimensional compact metrizable
subset. Hence $C_{k}(X)$ satisfies this property if $X$ is an infinite
Tychonoff space containing a compact resolution that swallows the compact sets
[FK13, Therem 2]. Next, with the help of Theorem 4.2, we greatly expand this
class of lcs by showing that every uncountably-dimensional lcs with a $P$-base
is also in this class if $P$ is a poset equipped with a second-countable
topology such that every convergent sequence in $P$ is bounded.

We start with some definitions needed. Let $\kappa,\lambda$ be cardinals. We
say that a space $X$ is $(\kappa,\lambda)_{p}$-equiconvergent at a point $x\in
X$ if for any indexed family $\\{x_{\alpha}:\alpha<\kappa\\}$ of sequences
converging to $x$, there is a $\lambda$-sized subset $\Lambda$ such that for
each neighborhood $O_{x}$ of $x$ there is an $n\in\omega$ such that
$\\{\alpha\in\Lambda:x_{\alpha}(n)\notin O_{x}\\}$ is finite . We say $X$ is
$(\kappa,\lambda)_{p}$-equiconvergent if it is
$(\kappa,\lambda)_{p}$-equiconvergent at any point in $X$.

A lcs $E$ is said to have $(\kappa,\lambda)$-tall bornology if every
$\kappa$-sized subset $A$ contains a $\lambda$-sized bounded subset. It is
known (see [BKP20]) that any lcs which is $(\kappa,\lambda)_{p}$-equicovergent
has $(\kappa,\lambda)$-tall bornology. The authors in [BKP20] also prove that
any topological space with countable cs∙-network at a point $x$ is
$(\omega_{1},\omega)_{p}$-equiconvergent at $x$. A family $\mathcal{N}$ of
subsets of $X$ is said to be a cs∙-network at a point $x\in X$ if for any
neighborhood $O_{x}$ of $x$ and any sequence $\\{x_{n}:n\in\omega\\}$
converging to $x$ there exists $N\in\mathcal{N}$ such that $N\subset O_{x}$
and $N$ contains some point $x_{n}$ in the sequence. Clearly any countable
Pytkeev∗ network at a point $x$ is also a countable cs∙-network at $x$. Hence
any space with the strong Pytkeev∗ property has a countable cs∙-network at
each point in the space.

It is proved [BKP20] that for any lcs $E$ each compact subset of $E$ has
finite topological dimension if and only if each bounded linearly independent
subset of $E$ is finite. We use this result to prove the following theorem.

######  Theorem 4.9.

Let $P$ be a directed set equipped with a second-countable topology in which
every convergent sequence in $P$ is bounded. Every uncountably-dimensional lcs
$E$ with a $P$-base contains an infinite-dimensional metrizable compact
subspace.

###### Proof.

Let $E$ be a lcs with a $P$-base for some directed set equipped with second-
countable topology in which every convergent sequence in $P$ is bounded. By
Corollary 4.4, $E$ has the strong Pytkeev∗ property; hence it has cs∙-network
at each point at any $x\in E$. Therefore, $E$ is
$(\omega_{1},\omega)_{p}$-equiconvergent. By Proposition 3.3 in [BKP20], $E$
has $(\omega_{1},\omega)$-tall bornology, i.e., every uncountable set in $E$
contains an infinite bounded set. Let $H$ be an uncountable Hamel basis for
$E$, then $H$ contains an infinite bounded linearly independent set. Hence $E$
contains an infinite-dimensional compact subspace by Theorem 2.1 in [BKP20]
(the result mentioned above). By Lemma 4.8, $P$ has calibre
$(\omega_{1},\omega)$. Hence any such compact subspace in $E$ is metrizable by
Theorem 5.4 in [Feng20]. ∎

Applying the theorem above, we obtain the following examples.

######  Example 4.10.

Every uncountable-dimensional subspace of $C_{k}(\mathbb{Q})$ contains an
infinite-dimensional metrizable compact subspace.

Since the function space $C_{k}(\omega_{1})$ has a
$\mathcal{K}(\mathbb{Q})$-base, we get the following example.

######  Example 4.11.

Every uncountable-dimensional subspace of $C_{k}(\omega_{1})$ contains an
infinite-dimensional metrizable compact subspace.

Acknowledgement. The authors would like to express their gratitude to the
referee for all his/her valuable comments and suggestions which lead to the
improvements of the paper.

## References


# Hopeful_Men@LT-EDI-EACL2021: Hope Speech Detection Using Indic
Transliteration and Transformers

Ishan Sanjeev Upadhyay* {ishan.sanjeev<EMAIL_ADDRESS>Nikhil
E* {ishan.sanjeev<EMAIL_ADDRESS>Anshul Wadhawan
<EMAIL_ADDRESS>Radhika Mamidi<EMAIL_ADDRESS>
###### Abstract

This paper aims to describe the approach we used to detect hope speech in the
HopeEDI dataset. We experimented with two approaches. In the first approach,
we used contextual embeddings to train classifiers using logistic regression,
random forest, SVM, and LSTM based models.The second approach involved using a
majority voting ensemble of 11 models which were obtained by fine-tuning pre-
trained transformer models (BERT, ALBERT, RoBERTa, IndicBERT) after adding an
output layer. We found that the second approach was superior for English,
Tamil and Malayalam. Our solution got a weighted F1 score of 0.93, 0.75 and
0.49 for English,Malayalam and Tamil respectively. Our solution ranked first
in English, eighth in Malayalam and eleventh in Tamil.

††footnotetext: *These authors contributed equally to this work

##  1 Introduction

The spread of hate speech on social media is a problem that still exists
today. While there have been attempts made at hate speech detection (Schmidt
and Wiegand, 2017; Lee et al., 2018) to stop the spread of negativity, this
form of censorship can also be misused to obstruct rights and freedom of
speech. Furthermore, hate speech tends to spread faster than non-hate speech
(Mathew et al., 2019).While there has been a growing amount of marginalized
people looking for support online (Gowen et al., 2012; Wang and Jurgens,
2018), there has been a substantial amount of hate towards them too (Mondal et
al., 2017). Therefore, detecting and promoting content that reduces hostility
and increases hope is important. Hope speech detection can be seen as a rare
positive mining task because hope speech constitutes a low percentage of
overall content (Palakodety et al., 2020a). There has been work done on hope
speech or help speech detection before that has used logistic regression and
active learning techniques (Palakodety et al., 2020a, b). In our paper, we
will be doing the hope speech detection task on the HopeEDI dataset
(Chakravarthi, 2020) which consists of user comments from Youtube in English,
Tamil and Malayalam.

In this paper, we will first look at the task definition, followed by the
methodology used. We will then look at the experiments and results followed by
conclusion and future work.

##  2 Task Definition

The given problem is a comment level classification task for the
identification of ”hope speech” within YouTube comments, wherein they are to
be classified as ”Hope speech”, ”Not hope speech” and ”Not in intended
language”. The data provided in the task was annotated at a per-comment basis
wherein a comment could be composed of more than one sentence.

##  3 Methodology

Pre-processing ModuleLanguage Detection ModuleTransliteration Module (Indian
Languages)Hope Speech Detection Module

Figure 1:  Methodology Pipeline

This section talks about the methodology that we have used to solve the task.
As shown in Figure 1, the pipeline involves preprocessing, language detection,
transliteration (for Indian languages), and hope speech detection. These steps
are described in this section.

###  3.1 Pre-processing Module

The preprocessing module involved the following:

  * •

Removing special characters and excess whitespaces

  * •

Removing emojis

  * •

Make text lowercase.

These steps were taken to make the text more uniform. Special characters like
“@” and “#” were removed because they did not serve as good features for
classification and language detection. Emojis were removed because they were
sparsely used in the dataset.

###  3.2 Language Detection Module

The task involves classifying text into hope, not-hope and not-language.
Language detection module marks the not-language sentences. We use Google’s
language detection library (Shuyo, 2010) to do this. The Tamil and Malayalam
datasets are code-mixed. Inter-sentential,intra-sentential, tag code-mixing
and code-mixing between Latin and native script is observed in the Tamil and
Malayalam datasets. Google’s language detection library does not work on such
code-mixed data. Since the Tamil and Malayalam sentences involve code-mixed
data, language detection can not be done on them using the Google language
detection library. We observed that sentences that were marked as not-Tamil
and not-Malayalam were mostly English sentences with some of them being Hindi
and other languages. Hence, we adopted a heuristic where we marked sentences
as not-Tamil or not-Malayalam if the sentences were detected to be in English
or Hindi, other sentences were assumed to belong to the respective language.

###  3.3 Transliteration Module

After language detection, sentences that are classified to be in Tamil and
Malayalam undergo transliteration. Tamil and Malayalam text have code-mixing
between Latin and native script, hence transliteration is done to make the
entire text in the native script. This step is also important because it makes
the text closer to the kind of text IndicBert is trained on. Transliteration
was done by using the indic-transliteration library
††https://github.com/sanskrit-coders/indic_transliteration.

###  3.4 Hope Speech Detection Module

After preprocessing and transliteration (for Indian languages), the text is
sent to the hope speech detection module. The hope speech detection module is
responsible for predicting if a text is hope speech or not hope speech. We
have used the following for our experiment.

####  3.4.1 Models

Transformers (Vaswani et al., 2017) have performed well in various natural
language processing (NLP) tasks. Unlike recurrent neural networks (RNN),
transformers are non-sequential (ie. sentences are processed as a whole rather
than word by word) and use self-attention at each input time step. Hence, they
do not suffer from long dependency issues. Query, Key and Value are three
different ways in which input vectors are used in the self-attention
mechanism. The attention score for every input vector is calculated using a
compatibility function which takes as input the query vector and all the keys.
The final output is a weighted sum of values where the weights are the
attention scores calculated by the compatibility function. We have used the
following transformers in our experiment. We chose RoBERTa for our final model
in English and ALBERT (IndicBERT) for Tamil and Malayalam.

BERT (Devlin et al., 2019) is based on the transformer architecture. Using its
multi-layer encode module,It is able to jointly utilize both left and right
contexts across all layers to pre-train its bidirectional representations.
BERT is trained on two unsupervised prediction tasks, next sentence prediction
and masked language modelling. We have fine tuned “bert-base-uncased” model on
the dataset for one of our experiments.

RoBERTa (Liu et al., 2019) is a transformer architecture which is based on
optimizations made to the BERT approach. It trains on more data and bigger
batches, removes next sentence prediction objective that BERT used, trains on
longer sequences and introduces dynamic masking (ie. mask tokens change during
training epochs). RoBERTa outperforms BERT and XLNet on the GLUE benchmark.
For the hope speech classification task in English, we fine-tuned the
“roberta-base” model on the provided data. The roberta-base model is trained
on 160 GB of English text from five different datasets.

ALBERT (Lan et al., 2020) is a transformer architecture based on BERT but with
fewer parameters. There are two key changes made to ALBERT. The first is
factorized embeddings parameterization, which decomposes the large vocabulary
embedding matrix into smaller matrices. This makes it easier to grow hidden
size without increasing the parameter size of vocabulary embeddings. This step
leads to a reduction in parameters by 80% compared to BERT. The second is
cross-layer parameter sharing, which prevents the parameters from increasing
as the depth of the network increases. We fine-tuned the IndicBERT model for
the hope speech classification task in Tamil and Malayalam. We used IndicBERT
(Kakwani et al., 2020) which is a multilingual ALBERT model pre-trained on 12
major Indian languages. We also fine-tuned ”albert-base-v2” model for our
experiment in English.

LSTM Long Short-Term Memory (Hochreiter and Schmidhuber, 1997) networks seek
to solve the short-term memory or vanishing gradient problem that RNNs face.
They do so by having internal gates that regulate the flow of information.
Information flows through a mechanism known as cell states. The cell can make
decisions about what to store, what to forget and what the next hidden state
should be. This is done through internal mechanisms called gates which contain
sigmoid activations.

Random Forest Classifier Random forests (Breiman, 2001) use an ensemble of a
large number of decision trees generally trained with the bagging method.
These decision trees are created using random subsamples of the given dataset
with replacement (bootstrap dataset) and a random subset of the features. New
samples are classified by choosing the prediction made by most decision trees
(majority voting).

Support Vector Machine Support vector machine (SVM) (Hearst, 1998) is a
supervised learning method that can be used for classification or regression.
We have used SVM for classification. The objective of the SVM classification
algorithm is to find the hyper-plane that most accurately differentiates two
classes that have been plotted on a f dimensional plane where f is the number
of features.

Logistic Regression Logistic regression (McCullagh and Nelder, 1989)is a
statistical model used for binary classification. It does so by using a
logistic function to model the binary outcome. It can be extended for
multiclass classification problems.

####  3.4.2 Ensemble Process

Ensembles can help make better predictions by reducing the spread of
predictions. Hence, lowering variance and improving accuracy. We used a voting
based ensemble method where we trained N models on N different training and
validation data obtained by random shuffling. We then chose the majority
voting as the merging technique to produce our final prediction y. In majority
voting, the final prediction y is decided based on which prediction is made by
the majority of the models . We made two ensembles, one each of 7 models and
11 models and chose the ensemble that gave the best weighted F1 score.

##  4 Experiments

Initially, the entire database is preprocessed to remove extra tab spaces,
punctuations, emojis, mentions and links. In the case of Malayalam and Tamil,
we also transliterate the entire database. Then we distributed our
experimentation procedure into two different approaches. In the first
approach, we finetune our pre-trained masked language models using the train
and validation splits for the purpose of making them more suitable to the
subsequent classification task. Thereafter, contextual embeddings for each
sentence in the dataset are produced by calculating the average of the second
to last hidden layer for every single token in the sentence. We then trained
Logistic Regression, Random Forest, SVM and RNN-based classifier models using
these embeddings. In the second approach, all the sentences are encoded into
tokens using the respective tokenizers and then we add a linear layer on top
of the pre-trained model layers after dropout. All the layers of the devised
model are then trained such that the error is back propagated through the
entire architecture and the pre-trained weights of the model are modified to
reflect the new database. For both these approaches, we then calculated
predictions for the test split and reported performance metrics. For English,
we try out three different pre-trained models: ”roberta-base”, ”bert-base-
uncased”, and ”albert-base-v2” for both the approaches. For Tamil and
Malayalam however, only the IndicBERT model is applicable for either approach.

Language |  Database | Hope |  Not Hope |  Other Lang.
---|---|---|---|---
English |  Train | 1962 |  20778 |  22
|  Dev | 242 |  2569 |  2
Tamil |  Train | 6327 |  7872 |  1961
|  Dev | 757 |  998 |  263
Malayalam |  Train | 1668 |  6205 |  691
|  Dev | 190 |  784 |  96

Table 1:  Data distribution by class

| English | Tamil | Malayalam
---|---|---|---
Training | 22762 | 16160 | 2564
Development | 2843 | 2018 | 1070
Test | 2846 | 2020 | 1071
Total | 28451 | 20198 | 10705

Table 2:  Data distribution by language

###  4.1 Dataset

The HopeEDI dataset consists of Youtube comments marked as “hope”, “not hope”
and “other language” in three languages: English, Tamil and Malayalam. The
distribution of hope, not hope and other language tag in the training and
development datasets is shown in table 1. The ratio of hope to not hope is
around 0.09 in English, 0.26 in Malayalam and 0.79 in Tamil. Table 2 shows the
data distribution between training, development and test datasets. There are a
total of 28,451 comments in English, 10,705 comments in Malayalam and 20,198
comments in Tamil. Data in Tamil and Telugu has code-mixing. In the English
dataset, there are instances where English comments are annotated as not
English. For example, “Fox News is pure Garbage!” is annotated as not English
in the training set. This contributes some noise to the English dataset.

Model | Method | Macro | Weighted | Macro | Weighted | Macro | Weighted
---|---|---|---|---|---|---|---
| Used | Precision | Precision | Recall | Recall | F1-Score | F1-Score
BERT | E + LR | 0.778 | 0.911 | 0.656 | 0.924 | 0.695 | 0.913
| E + RF | 0.834 | 0.902 | 0.526 | 0.916 | 0.528 | 0.881
| E + SVM | 0.771 | 0.86 | 0.489 | 0.866 | 0.488 | 0.837
| E + LSTM | 0.456 | 0.833 | 0.500 | 0.913 | 0.477 | 0.871
| FT | 0.759 | 0.915 | 0.728 | 0.915 | 0.742 | 0.915
ALBERT | E + LR | 0.703 | 0.881 | 0.538 | 0.912 | 0.549 | 0.883
| E + RF | 0.832 | 0.900 | 0.506 | 0.914 | 0.491 | 0.874
| E + SVM | 0.456 | 0.833 | 0.500 | 0.913 | 0.477 | 0.871
| E + LSTM | 0.657 | 0.878 | 0.571 | 0.905 | 0.591 | 0.887
| FT | 0.755 | 0.916 | 0.705 | 0.924 | 0.725 | 0.919
RoBERTa | E + LR | 0.794 | 0.914 | 0.657 | 0.926 | 0.700 | 0.915
| E + RF | 0.840 | 0.905 | 0.535 | 0.917 | 0.544 | 0.885
| E + SVM | 0.821 | 0.899 | 0.517 | 0.915 | 0.512 | 0.878
| E + LSTM | 0.791 | 0.918 | 0.693 | 0.928 | 0.729 | 0.921
| FT | 0.753 | 0.915 | 0.748 | 0.922 | 0.745 | 0.923
Table 3:  Metrics for English language Model | Method | Macro | Weighted | Macro | Weighted | Macro | Weighted
---|---|---|---|---|---|---|---
| Used | Precision | Precision | Recall | Recall | F1-Score | F1-Score
Indic | E + LR | 0.473 | 0.482 | 0.484 | 0.520 | 0.441 | 0.464
-BERT | E + RF | 0.511 | 0.516 | 0.506 | 0.544 | 0.458 | 0.482
| E + SVM | 0.278 | 0.309 | 0.500 | 0.556 | 0.357 | 0.397
| E + LSTM | 0.591 | 0.587 | 0.501 | 0.557 | 0.364 | 0.403
| FT | 0.635 | 0.637 | 0.627 | 0.636 | 0.623 | 0.629
Table 4:  Metrics for Tamil language Model | Method | Macro | Weighted | Macro | Weighted | Macro | Weighted
---|---|---|---|---|---|---|---
| Used | Precision | Precision | Recall | Recall | F1-Score | F1-Score
Indic | E + LR | 0.645 | 0.729 | 0.501 | 0.790 | 0.447 | 0.699
-BERT | E + RF | 0.395 | 0.623 | 0.499 | 0.788 | 0.440 | 0.696
| E + SVM | 0.386 | 0.610 | 0.492 | 0.777 | 0.433 | 0.683
| E + LSTM | 0.367 | 0.579 | 0.471 | 0.745 | 0.413 | 0.652
| FT | 0.776 | 0.842 | 0.743 | 0.842 | 0.756 | 0.837
Table 5:  Metrics for Malayalam language

###  4.2 System Settings

In the first approach, we run the task of masked language modelling on our
database for 4 epochs for each of the 5 model-database combinations.
Afterwards, the sentence input token length is limited to 512 and the
embeddings extracted by evaluation on the input sequences by the model are of
length 768. The RNN based classifier is composed of an LSTM layer and two
dense layers. In the second approach, the encoded sentences are in the form of
a data loader class, containing the respective input IDs and attention masks,
with a batch size of 16. These are then passed into a model that implements a
dropout of 30% and the output from the final linear layer is used for
classification.

###  4.3 Evaluation Metrics

We used F1 and weighted F1 scores for evaluating our model.

$F1\>Score=2\times\frac{(precision\times recall)}{(precision+recall)}$

Weighted F1 scores are calculated by taking the F1 scores for each label and
then doing a weighted average by the number of true instances of each label.

$weighted\>F1=\frac{(F1_{i}\times y_{i}+F1_{j}\times y_{j})}{(y_{i}+y_{j})}$

$y_{i}$ and $y_{j}$ are the number of true instances of class $i$ and class
$j$ respectively and $F1_{i}$ and $F1_{j}$ are the F1 scores of class $i$ and
$j$ respectively.

###  4.4 Results

Our experimentation involved two approaches. In the first approach, we used
contextual embeddings (E) to train classifiers using logistic regression (LR),
random forest (RF), SVM, and LSTM based models. In the second approach we used
an ensemble of 11 models which were generated by fine-tuning (FT) pre-trained
transformer models after adding an output layer. We used majority voting to
get our final prediction. Results for both the approaches on our test split
generated from the provided train and dev datasets are reported in Tables 3, 4
and 5 for English, Tamil and Malayalam respectively. We report the macro-
averaged and weighted recall, precision and F1-score for each possible model-
method combination. While the weighted F1 scores are more representative of
how well a model performs, the disparity between the weighted and macro-
averaged scores demonstrates how disproportionate a certain model’s
effectiveness is in predicting the different classes. For English, the second
approach involving finetuning is the best performing one for each of the
models tested, closely followed by the Logistic Regression and LSTM-based
methods in the first approach. The roberta-base model seems to have a slight
edge over the other two tested models. For Tamil and Malayalam, the second
approach is still the best performer, but by a greater margin.

##  5 Conclusion and Future Work

In this paper, we presented our approach for hope speech detection in English,
Tamil, and Malayalam on the HopeEDI dataset. We used two approaches. The first
approach involved using contextual embeddings to train various classifiers.
The second approach involved using a majority voting ensemble of 11 models
which were obtained by fine-tuning pre-trained transformer models. The second
approach using the roberta-base model was the best performing model for
English, giving a weighted F1 score of 0.93. The second approach using
IndicBERT model gave the best performance for Tamil and Malayalam, giving a
weighted F1 score of 0.75 for Malayalam and 0.49 for Tamil. In the future, we
plan to fine-tune transformers pre-trained on code mixed data. Data
augmentation methods like synonym replacement and random insertion could be
used to fine-tune the model on more data.

## References

  * Breiman (2001) Leo Breiman. 2001.  Random forests.  _Mach. Learn._ , 45(1):5–32.
  * Chakravarthi (2020) Bharathi Raja Chakravarthi. 2020.  HopeEDI: A multilingual hope speech detection dataset for equality, diversity, and inclusion.  In _Proceedings of the Third Workshop on Computational Modeling of People’s Opinions, Personality, and Emotion’s in Social Media_ , pages 41–53, Barcelona, Spain (Online). Association for Computational Linguistics.
  * Devlin et al. (2019) Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019.  BERT: Pre-training of deep bidirectional transformers for language understanding.  In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)_ , pages 4171–4186, Minneapolis, Minnesota. Association for Computational Linguistics.
  * Gowen et al. (2012) Kris Gowen, Matthew Deschaine, Darcy Gruttadara, and Dana Markey. 2012.  Young adults with mental health conditions and social networking websites: Seeking tools to build community. _Psychiatric Rehabilitation Journal_ , 35(3):245–250.
  * Hearst (1998) Marti A. Hearst. 1998.  Support vector machines.  _IEEE Intelligent Systems_ , 13(4):18–28.
  * Hochreiter and Schmidhuber (1997) Sepp Hochreiter and Jürgen Schmidhuber. 1997.  Long short-term memory.  _Neural Computation_ , 9(8):1735–1780.
  * Kakwani et al. (2020) Divyanshu Kakwani, Anoop Kunchukuttan, Satish Golla, Gokul N.C., Avik Bhattacharyya, Mitesh M. Khapra, and Pratyush Kumar. 2020.  IndicNLPSuite: Monolingual corpora, evaluation benchmarks and pre-trained multilingual language models for Indian languages.  In _Findings of the Association for Computational Linguistics: EMNLP 2020_ , pages 4948–4961, Online. Association for Computational Linguistics.
  * Lan et al. (2020) Zhenzhong Lan, Mingda Chen, Sebastian Goodman, Kevin Gimpel, Piyush Sharma, and Radu Soricut. 2020.  Albert: A lite bert for self-supervised learning of language representations.
  * Lee et al. (2018) Younghun Lee, Seunghyun Yoon, and Kyomin Jung. 2018.  Comparative studies of detecting abusive language on Twitter.  In _Proceedings of the 2nd Workshop on Abusive Language Online (ALW2)_ , pages 101–106, Brussels, Belgium. Association for Computational Linguistics.
  * Liu et al. (2019) Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019.  Roberta: A robustly optimized bert pretraining approach.
  * Mathew et al. (2019) Binny Mathew, Ritam Dutt, Pawan Goyal, and Animesh Mukherjee. 2019.  Spread of hate speech in online social media.  In _Proceedings of the 10th ACM Conference on Web Science_ , WebSci ’19, page 173–182, New York, NY, USA. Association for Computing Machinery.
  * McCullagh and Nelder (1989) P. McCullagh and J.A. Nelder. 1989.  _Generalized Linear Models, Second Edition_.  Chapman and Hall/CRC Monographs on Statistics and Applied Probability Series. Chapman & Hall.
  * Mondal et al. (2017) Mainack Mondal, Leandro Araújo Silva, and Fabrício Benevenuto. 2017.  A measurement study of hate speech in social media.  In _Proceedings of the 28th ACM Conference on Hypertext and Social Media_ , HT ’17, page 85–94, New York, NY, USA. Association for Computing Machinery.
  * Palakodety et al. (2020a) Shriphani Palakodety, Ashiqur R. KhudaBukhsh, and Jaime G. Carbonell. 2020a.  Hope speech detection: A computational analysis of the voice of peace.
  * Palakodety et al. (2020b) Shriphani Palakodety, Ashiqur R. KhudaBukhsh, Jaime G. Carbonell, Shriphani Palakodety, Ashiqur R. KhudaBukhsh, and Jaime G. Carbonell. 2020b.  Voice for the voiceless: Active sampling to detect comments supporting the rohingyas.  _Proceedings of the AAAI Conference on Artificial Intelligence_ , 34(01):454–462.
  * Schmidt and Wiegand (2017) Anna Schmidt and Michael Wiegand. 2017.  A survey on hate speech detection using natural language processing.  In _Proceedings of the Fifth International Workshop on Natural Language Processing for Social Media_ , pages 1–10, Valencia, Spain. Association for Computational Linguistics.
  * Shuyo (2010) Nakatani Shuyo. 2010.  Language detection library for java.
  * Vaswani et al. (2017) Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017.  Attention is all you need.
  * Wang and Jurgens (2018) Zijian Wang and David Jurgens. 2018.  It’s going to be okay: Measuring access to support in online communities.  In _Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing_ , pages 33–45, Brussels, Belgium. Association for Computational Linguistics.