# Quantum transport of a spin-1 chiral fermion

Risako Kikuchi  Department of Physics, Nagoya University, Nagoya 464-8602,
Japan  Takumi Funato  Center for Spintronics Research Network, Keio
University, Yokohama 223-8522, Japan  Kavli Institute for Theoretical
Sciences, University of Chinese Academy of Sciences, Beijing, 100190, China
Ai Yamakage  Department of Physics, Nagoya University, Nagoya 464-8602, Japan

###### Abstract

We theoretically study the quantum transport in a three-dimensional spin-1
chiral fermion system in the presence of impurity scattering. Within the self-
consistent Born approximation, we find peak structure of the density of states
and significant suppression of electrical conductivity around the zero energy.
The zero-energy conductivity depends less on impurity concentration, unlike a
Weyl fermion. These properties originate from the flat band structure of
spin-1 chiral fermion.

##  I Introduction

A variety of chiral quasiparticles beyond Weyl and Dirac fermions can emerge
in crystalline material, protected by symmetry and topological charges [1].
Spin-1 chiral fermion is a prominent realization of such an exotic particle,
which hosts the trivial band with nearly flat dispersion, called flat band, in
addition to the Dirac cone. Angle-Resolved Photo Emission Spectroscopy (ARPES)
measurements have observed a spin-1 chiral fermion in a chiral crystal CoSi
[2, 3, 4, 5, 6]. A two-dimensional (approximate) version of spin-1 chiral
fermion has also been predicted [7] in the context of the $\alpha$–T3
lattices. Quantum phenomena, including spin transport [8] and optical
responses [9, 10, 11, 12, 13, 14, 15, 16], can be anomalous, stemming from the
peculiar electronic states of a spin-1 fermion. Research has also spilled over
into quadratic dispersion with the spin-1 structure [17, 18, 19]. A
fundamental example of such phenomena is the electrical conductivity of
quantum transport.

Chiral fermions of Dirac and Weyl fermions exhibit peculiar transport
properties on zero energy [20, 21, 22]. For two-dimensional massless Dirac
fermions, representing low-energy electrons in graphene, the electrical
conductivity remains finite and independent of the relaxation time at their
gapless point, though the density of states vanishes [23, 24, 25, 26, 27, 28,
29]. Weyl fermions in the three spatial dimensions, on the other hand, show a
specific quantum transport phenomenon strongly depending on the impurity
concentration at the Weyl point located on the zero energy [30, 31, 32, 33,
34]. Interestingly, a finite(long)-range disorder potential (does not) gives
rise to the semimetal–metal transition. These studies imply that the quantum
transport phenomena on the zero energy possibly depend on the spatial
dimension, disorder type, and symmetry/topology of chiral fermions. Therefore,
a spin-1 chiral fermion, characterized by the flat band, monopole charge 2,
and orthogonal symmetry class, is expected to show a distinct quantum
transport character. Recent years have seen remarkable progress in our
understanding of two-dimensional spin-1 systems [35, 36, 37, 38]. Thus, we can
address quantum transport phenomena in three-dimensional spin-1 systems.

In this work, we theoretically analyze the quantum transport of three-
dimensional spin-1 chiral fermion under impurity potentials, based on the
Boltzmann transport theory and the self-consistent Born approximation (SCBA)
in linear response theory. As a result, we find some characteristic phenomena
near the zero energy by SCBA intrinsic to the spin-1 nature. There is a peak
structure of the density of states, which originates from a flat band of the
spin-1 fermion, depending on the impurity concentration. We also find
significant suppression of conductivity near the zero energy. This phenomenon
of low-energy transport is caused by the vanishing group velocity of the flat
band and the interband effect between the flat band and the Dirac cone.

The paper is organized as follows. Our model for a spin-1 chiral fermion in
the presence of impurity is introduced in Sec. II. The conductivity is
calculated within the Boltzmann theory in Sec. III and the SCBA with the
current vertex correction in Sec. IV. The self-consistent equation for self
energy and the Bethe-Salpeter equation for vertex correction are explicitly
derived. Comparison of our results to related works and some remarks are
discussed in Sec. V. And Sec. VI summarizes this work.

##  II model

We consider a three-dimensional spin-1 chiral fermion system, described by

$\displaystyle\hat{\mathcal{H}}=\hbar v\hat{\bm{S}}\cdot\bm{k},$ (1)

where $\bm{k}$ is the electron wavenumber and $v$ is the Fermi velocity.
$\bm{S}=(\hat{S}_{x},\hat{S}_{y},\hat{S}_{z})$ are the spin operator:

$\displaystyle\hat{S}_{x}$ $\displaystyle=\pmqty{undef},$ (2)
$\displaystyle\hat{S}_{y}$ $\displaystyle=\pmqty{undef},$ (3)
$\displaystyle\hat{S}_{z}$ $\displaystyle=\begin{pmatrix}0&0&0\\\ 0&0&i\\\
0&-i&0\\\ \end{pmatrix}.$ (4)

The eigenenergy is given by

$\displaystyle\epsilon_{\lambda,\bm{k}}=\hbar v\lambda k,$ (5)

where $\lambda$ is the label for the conduction band ($\lambda=1$), the flat
band ($\lambda=0$), and the valence band ($\lambda=-1$).

We assume two impurity potentials, the Gaussian and delta function potentials,
which represent finite-range and short-range disorders, respectively. Note
that the length scale of impurity may strongly affect the transport property
of chiral fermion systems, as seen in graphene. The Gaussian potential is
defined by

$\displaystyle U(\bm{r})$ $\displaystyle=$ $\displaystyle\frac{\pm
u_{0}}{(\sqrt{\pi}d_{0})^{3}}\exp(-\frac{r^{2}}{d_{0}^{2}}),$ (6)

where $d_{0}$ is the characteristic length scale and $\pm u_{0}$ is the
strength of the impurity potential. The sign $\pm$ means to assume that the
numbers of positive and negative valued impurities are the same, so the Fermi
level is fixed, irrelevant to the impurity concentration. The delta function
potential is defined by

$\displaystyle U(\bm{r})=\pm u_{0}\delta(\bm{r}),$ (7)

which corresponds to the short-range limit ($d_{0}\to 0$) of the Gaussian
potential. Their Fourier transforms are obtained to be

$\displaystyle u(\bm{k})=\pm u_{0}\exp(-\frac{k^{2}}{q_{0}^{2}}),$ (8)

with $q_{0}=2/d_{0}$ and $u(\bm{k})=\pm u_{0}$, respectively. To avoid the
ultraviolet divergence, we need to regularize the delta function potential
with a hard cutoff as

$\displaystyle u(\bm{k})=\pm u_{0}\theta(q_{0}-k),$ (9)

where $q_{0}$ is the inverse of the length that characterizes the impurity
potential.

Isotropic disorder potentials are characterized by the moment of scattering
angle as

$\displaystyle
V_{n}^{2}(k,k^{\prime})=2\pi\int_{-1}^{1}d(\cos\theta_{\bm{kk^{\prime}}})|u(\bm{k}-\bm{k^{\prime}})|^{2}\cos^{n}\theta_{\bm{kk^{\prime}}},$
(10)

where $\theta_{\bm{kk^{\prime}}}$ is the angle between $\bm{k}$ and
$\bm{k^{\prime}}$. We also define a parameter characterizing the scattering
strength

$\displaystyle W=\frac{q_{0}n_{\text{i}}u_{0}^{2}}{\hbar^{2}v^{2}},$ (11)

where $n_{\text{i}}$ is the number of scatterers per unit volume.

##  III Boltzmann transport theory

We calculate the conductivity from the Boltzmann transport theory. The
scattering probability $W_{\lambda^{\prime}\bm{k}^{\prime},\lambda\bm{k}}$ is
given by the Fermi’s golden rule as

$\displaystyle
W_{\lambda^{\prime}\bm{k^{\prime}},\lambda\bm{k}}=\frac{2\pi}{\hbar}n_{\text{i}}|\bra{\lambda^{\prime},\bm{k^{\prime}}}U\ket{\lambda,\bm{k}}|^{2}\delta(\epsilon_{\lambda^{\prime},\bm{k^{\prime}}}-\epsilon_{\lambda,\bm{k}}).$
(12)

The transport relaxation time $\tau_{\mathrm{tr}}$ is defined by

$\displaystyle\frac{1}{\tau_{\rm{tr}}(\epsilon_{\lambda,\bm{k}})}=\sum_{\lambda^{\prime}}\int\frac{d\bm{k^{\prime}}}{(2\pi)^{3}}(1-\cos\theta_{\bm{k^{\prime}}\bm{k}})W_{\lambda^{\prime}\bm{k^{\prime}},\lambda\bm{k}}.$
(13)

The density of states of the linear dispersion ($\lambda=\pm 1$) in the clean
limit is given by

$\displaystyle D_{0}(\epsilon)=\frac{\epsilon^{2}}{2\pi^{2}(\hbar
v)^{3}}\mbox{\quad for\quad}\epsilon\neq 0,$ (14)

and that of the flat band is ($\lambda=0$) described by the delta function and
diverges for $\epsilon=0$. The conductivity at the zero temperature is
obtained to be

$\displaystyle\sigma_{B}(\epsilon)$ $\displaystyle=$
$\displaystyle\frac{e^{2}v^{2}}{3}D_{0}(\epsilon)\tau_{\mathrm{tr}}(\epsilon).$
(15)

The transport relaxation time is written as

$\displaystyle\frac{1}{\tau_{\mathrm{tr}}(\epsilon)}$
$\displaystyle=\frac{n_{\text{i}}}{2\pi\hbar^{2}v}\int_{-1}^{1}\int_{0}^{\infty}dk^{\prime}d(\cos\theta_{\bm{kk^{\prime}}})k^{\prime
2}(1-\cos\theta_{\bm{kk^{\prime}}})$
$\displaystyle\quad\times\frac{(\cos\theta_{\bm{kk}^{\prime}}+1)^{2}}{4}\delta(k-k^{\prime})|u(\bm{k}-\bm{k^{\prime}})|^{2}$
$\displaystyle=\frac{n_{\text{i}}\epsilon^{2}}{4(2\pi)^{2}\hbar(\hbar
v)^{3}}\bigl{[}V_{0}^{2}(\epsilon/\hbar v,\epsilon/\hbar
v)+V_{1}^{2}(\epsilon/\hbar v,\epsilon/\hbar v)$ $\displaystyle-
V_{2}^{2}(\epsilon/\hbar v,\epsilon/\hbar v)-V_{3}^{2}(\epsilon/\hbar
v,\epsilon/\hbar v)\bigr{]},$ (16)

where the momentum is located on the Fermi surface, $k=\epsilon/\hbar v$.

Figure 1: (Color online) Electrical conductivity of a spin-1 fermion for $W=2$
(red line), $W=5$ (green line), and $W=10$ (blue line), derived by the
Boltzmann equation with the Gaussian potential.

As a result, we find the conductivity

$\displaystyle\sigma_{B}(\epsilon)$ $\displaystyle=$
$\displaystyle\frac{e^{2}v^{2}\hbar}{\pi n_{\text{i}}u_{0}^{2}},$ (17)

for the delta function potential with $q_{0}\to\infty$. And

$\displaystyle\sigma_{B}(\epsilon)$ $\displaystyle=$
$\displaystyle\frac{8}{3\pi}\dfrac{e^{2}}{\hbar}\dfrac{1}{n_{\text{i}}u_{0}^{2}}h\left(\dfrac{\epsilon}{\hbar
vq_{0}}\right),$ (18)

for the Gaussian potential, where we define

$\displaystyle
h(x)=\dfrac{64x^{8}}{32x^{4}-16x^{2}+3-(8x^{2}+3)\exp(-8x^{2})}.$ (19)

Figure 1 shows the conductivity for the Gaussian potential, which is a
monotonically increasing function of the Fermi energy and takes a finite value
at the zero energy

$\displaystyle\sigma_{B}(\epsilon\rightarrow 0)=\frac{e^{2}v^{2}\hbar}{\pi
n_{\text{i}}u_{0}^{2}},$ (20)

which coincides with that of the delta function potential. The Fermi-energy
dependence of conductivity stems from the wavenumber dependence of the
potential. The delta and Gaussian-function potentials are constant and
exponentially decreasing functions of the wavenumber. Therefore the
conductivity is constant and an increasing function for the delta and Gaussian
potentials, respectively.

##  IV The linear response theory (SCBA)

Next, we calculate the density of states and conductivity in a self-consistent
manner to consider the effect of level broadening induced by impurity.

###  IV.1 Formulation

Assuming that the impurity distribution is uniformly random, the impurity-
averaged Green function is given by

$\displaystyle\hat{G}(\bm{k},\epsilon+is0)$ $\displaystyle=$
$\displaystyle\frac{1}{\epsilon\hat{S}_{0}-\hbar
vk\hat{\bm{S}}\cdot\bm{n}-\hat{\Sigma}(\bm{k},\epsilon+is0)},$ (21)

where $\bm{n}=\bm{k}/k$ is the unit vector and $\hat{S}_{0}$ is the identity
matrix. The sign $s$ refers to the retarded ($s=1$) and advanced ($s=-1$)
Green’s functions. The self-consistent equation for the self-energy is written
as

$\displaystyle\hat{\Sigma}(\bm{k},\epsilon+is0)=\int\frac{d\bm{k^{\prime}}}{(2\pi)^{3}}n_{\text{i}}|u(\bm{k}-\bm{k^{\prime}})|^{2}\hat{G}(\bm{k^{\prime}},\epsilon+is0).$
(22)

The density of states per unit volume is calculated as

$\displaystyle
D(\epsilon)=-\frac{1}{\pi}\imaginary\int\frac{d\bm{k}}{(2\pi)^{3}}\Tr\hat{G}(\bm{k},\epsilon+i0).$
(23)

The conductivity by the Kubo formula is written as

$\displaystyle\sigma(\epsilon)$ $\displaystyle=-\frac{\hbar
e^{2}v^{2}}{4\pi}\sum_{s,s^{\prime}=\pm
1}ss^{\prime}\int\frac{d\bm{k^{\prime}}}{(2\pi)^{3}}\text{Tr}[\hat{S}_{x}\hat{G}(\bm{k^{\prime}},\epsilon+is0)$
$\displaystyle\quad\times\hat{J}_{x}(\bm{k^{\prime}},\epsilon+is0,\epsilon+is^{\prime}0)\hat{G}(\bm{k^{\prime}},\epsilon+is^{\prime}0)],$
(24)

where $\hat{J}_{x}(\bm{k},\epsilon,\epsilon^{\prime})$ is the current density
flowing in the $x$ direction reinforced with the vertex correction and is
determined to be a solution of the following Bethe-Salpeter equation

$\displaystyle\hat{J}_{x}(\bm{k},\epsilon,\epsilon^{\prime})$
$\displaystyle=\hat{S}_{x}+\int\frac{d\bm{k^{\prime}}}{(2\pi)^{3}}n_{\text{i}}|u(\bm{k}-\bm{k^{\prime}})|^{2}\hat{G}(\bm{k^{\prime}},\epsilon)$
$\displaystyle\quad\times\hat{J}_{x}(\bm{k^{\prime}},\epsilon,\epsilon^{\prime})\hat{G}(\bm{k^{\prime}},\epsilon^{\prime}).$
(25)

Since $(\hat{\bm{S}}\cdot\bm{n})^{3}=(\hat{\bm{S}}\cdot\bm{n})$ for the spin–1
representation matrices, the self energy is expressed as

$\displaystyle\hat{\Sigma}(\bm{k},\epsilon)$
$\displaystyle=\Sigma_{1}(k,\epsilon)\hat{S}_{0}+\Sigma_{2}(k,\epsilon)(\hat{\bm{S}}\cdot\bm{n})+\Sigma_{3}(k,\epsilon)(\hat{\bm{S}}\cdot\bm{n})^{2}.$
(26)

Using the above expansion, Eq. (21) is rewritten as

$\displaystyle\hat{G}(\bm{k},\epsilon)$
$\displaystyle=\frac{1}{X(k,\epsilon)\hat{S}_{0}+Y(k,\epsilon)\hat{\bm{S}}\cdot\bm{n}+Z(k,\epsilon)(\hat{\bm{S}}\cdot\bm{n})^{2}}$
$\displaystyle=x(k,\epsilon)\hat{S}_{0}+y(k,\epsilon)(\hat{\bm{S}}\cdot\bm{n})+z(k,\epsilon)(\hat{\bm{S}}\cdot\bm{n})^{2},$
(27)

where

$\displaystyle X(k,\epsilon)$ $\displaystyle=\epsilon-\Sigma_{1}(k,\epsilon),$
(28) $\displaystyle Y(k,\epsilon)$ $\displaystyle=-\hbar
vk-\Sigma_{2}(k,\epsilon),$ (29) $\displaystyle Z(k,\epsilon)$
$\displaystyle=-\Sigma_{3}(k,\epsilon),$ (30)

and

$\displaystyle x(k,\epsilon)$ $\displaystyle=\frac{1}{X(k,\epsilon)},$ (31)
$\displaystyle y(k,\epsilon)$
$\displaystyle=-\frac{Y(k,\epsilon)}{(X(k,\epsilon)+Z(k,\epsilon))^{2}-Y(k,\epsilon)^{2}},$
(32) $\displaystyle z(k,\epsilon)$
$\displaystyle=\frac{Y(k,\epsilon)^{2}-Z(k,\epsilon)(X(k,\epsilon)+Z(k,\epsilon))}{((X(k,\epsilon)+Z(k,\epsilon))^{2}-Y(k,\epsilon)^{2})X(k,\epsilon)}.$
(33)

Here we reduce the above expressions for the self energy to a form more
convenient to solve the self-consistent equation. Substituting Eq. (27) into
Eq. (22), we get

$\displaystyle\hat{\Sigma}(\bm{k},\epsilon+is0)$
$\displaystyle=\hat{S}_{0}\int\frac{k^{\prime
2}dk^{\prime}}{(2\pi)^{3}}n_{\text{i}}\left[V_{0}^{2}(k,k^{\prime})x(k^{\prime},\epsilon+is0)\right.$
$\displaystyle\hskip
70.0001pt\left.+(V_{0}^{2}(k,k^{\prime})-V_{2}^{2}(k,k^{\prime}))z(k^{\prime},\epsilon+is0)\right]$
$\displaystyle\quad+(\hat{\bm{S}}\cdot\bm{n})\int\frac{k^{\prime
2}dk^{\prime}}{(2\pi)^{3}}n_{\text{i}}V_{1}^{2}(k,k^{\prime})y(k^{\prime},\epsilon+is0)$
$\displaystyle\quad+(\hat{\bm{S}}\cdot\bm{n})^{2}\int\frac{k^{\prime
2}dk^{\prime}}{(2\pi)^{3}}n_{\text{i}}^{2}\left(\frac{3}{2}V_{2}^{2}(k,k^{\prime})-\frac{1}{2}V_{0}^{2}(k,k^{\prime})\right)$
$\displaystyle\hskip 100.00015pt\times z(k^{\prime},\epsilon+is0),$ (34)

with the help of useful relations shown in Appendix A. Comparing this with Eq.
(26), the self-consistent equation is decomposed into the three equations as

$\displaystyle\Sigma_{1}(k,\epsilon+is0)=\int\frac{k^{\prime
2}dk^{\prime}}{(2\pi)^{3}}n_{\text{i}}\bigl{[}V_{0}^{2}(k,k^{\prime})x(k^{\prime},\epsilon+is0)$
$\displaystyle\hskip
40.00006pt+(V_{0}^{2}(k,k^{\prime})-V_{2}^{2}(k,k^{\prime}))z(k^{\prime},\epsilon+is0)\bigr{]},$
(35) $\displaystyle\Sigma_{2}(k,\epsilon+is0)=\int\frac{k^{\prime
2}dk^{\prime}}{(2\pi)^{3}}n_{\text{i}}V_{1}^{2}(k,k^{\prime})y(k^{\prime},\epsilon+is0),$
(36) $\displaystyle\Sigma_{3}(k,\epsilon+is0)=\int\frac{k^{\prime
2}dk^{\prime}}{(2\pi)^{3}}n_{\text{i}}\left(\frac{3}{2}V_{2}^{2}(k,k^{\prime})-\frac{1}{2}V_{0}^{2}(k,k^{\prime})\right)$
$\displaystyle\hskip 120.00018pt\times z(k^{\prime},\epsilon+is0).$ (37)

Substituting the self energy into Eq. (23), the density of states is written
as

$\displaystyle
D(\epsilon)=-\frac{1}{\pi}\imaginary\int\frac{d\bm{k}}{(2\pi)^{3}}\left(\frac{1}{X(k,\epsilon+i0)}\right.$
$\displaystyle\quad+\frac{1}{X(k,\epsilon+i0)+Y(k,\epsilon+i0)+Z(k,\epsilon+i0)}$
$\displaystyle\quad\left.+\frac{1}{X(k,\epsilon+i0)-Y(k,\epsilon+i0)+Z(k,\epsilon+i0)}\right).$
(38)

In addition, the Bethe-Salpeter equation is simplified into an easier form.
The current vertex $\hat{J}_{x}(\bm{k},\epsilon,\epsilon^{\prime})$ is
expanded to eight terms as

$\displaystyle\hat{J}_{x}(\bm{k},\epsilon,\epsilon^{\prime})$
$\displaystyle=\hat{S}_{x}J_{0}(k,\epsilon,\epsilon^{\prime})+n_{x}(\hat{\bm{S}}\cdot\bm{n})^{2}J_{1}(k,\epsilon,\epsilon^{\prime})$
$\displaystyle+n_{x}(\hat{\bm{S}}\cdot\bm{n})J_{2}(k,\epsilon,\epsilon^{\prime})+(\hat{\bm{S}}\cdot\bm{n})^{2}\hat{S}_{x}J_{3}(k,\epsilon,\epsilon^{\prime})$
$\displaystyle+\hat{S}_{x}(\hat{\bm{S}}\cdot\bm{n})^{2}J_{4}(k,\epsilon,\epsilon^{\prime})+(\hat{\bm{S}}\cdot\bm{n})\hat{S}_{x}J_{5}(k,\epsilon,\epsilon^{\prime})$
$\displaystyle+\hat{S}_{x}(\hat{\bm{S}}\cdot\bm{n})J_{6}(k,\epsilon,\epsilon^{\prime})+n_{x}\hat{S}_{0}J_{7}(k,\epsilon,\epsilon^{\prime}),$
(39)

by using Eqs. (53)–(57) for Eq. (25) (see Appendix A). The matrix-form Bethe-
Salpeter equation reduces to the eight equations (see Appendix B) for the
expansion coefficients $J_{0}$–$J_{7}$. Substituting them into Eq. (24), the
conductivity is written as

$\displaystyle\sigma(\epsilon)=\frac{2\hbar
e^{2}v^{2}}{3}\int_{0}^{\infty}\frac{k^{\prime
2}dk^{\prime}}{(2\pi)^{3}}\mbox{Re}\left[-\frac{J_{0}^{++}+J_{1}^{++}+J_{2}^{++}+J_{3}^{++}+J_{4}^{++}+J_{5}^{++}+J_{6}^{++}+J_{7}^{++}}{(X+Y+Z)^{2}}\right.$
$\displaystyle-\frac{J_{0}^{++}-J_{1}^{++}+J_{2}^{++}+J_{3}^{++}+J_{4}^{++}-J_{5}^{++}-J_{6}^{++}-J_{7}^{++}}{(X-Y+Z)^{2}}-\frac{2J_{0}^{++}+J_{3}^{++}+J_{4}^{++}+J_{5}^{++}+J_{6}^{++}}{X(X+Y+Z)}$
$\displaystyle-\frac{2J_{0}^{++}+J_{3}^{++}+J_{4}^{++}-J_{5}^{++}-J_{6}^{++}}{X(X-Y+Z)}+\frac{J_{0}^{+-}+J_{1}^{+-}+J_{2}^{+-}+J_{3}^{+-}+J_{4}^{+-}+J_{5}^{+-}+J_{6}^{+-}+J_{7}^{+-}}{|X+Y+Z|^{2}}$
$\displaystyle+\frac{J_{0}^{+-}-J_{1}^{+-}+J_{2}^{+-}+J_{3}^{+-}+J_{4}^{+-}-J_{5}^{+-}-J_{6}^{+-}-J_{7}^{+-}}{|X-Y+Z|^{2}}+\frac{J_{0}^{+-}+J_{4}^{+-}+J_{6}^{+-}}{X(X^{*}+Y^{*}+Z^{*})}$
$\displaystyle+\frac{J_{0}^{+-}+J_{4}^{+-}-J_{6}^{+-}}{X(X^{*}-Y^{*}+Z^{*})}\left.+\frac{J_{0}^{+-}+J_{3}^{+-}+J_{5}^{+-}}{X^{*}(X+Y+Z)}+\frac{J_{0}^{+-}+J_{3}^{+-}-J_{5}^{+-}}{X^{*}(X-Y+Z)}\right],$
(40)

where
$J_{i}^{ss^{\prime}}=J_{i}(k^{\prime},\epsilon+is0,\epsilon+is^{\prime}0)$,
$X=X(k^{\prime},\epsilon+i0)$, and so on.

###  IV.2 Numerical Calculations

The self-consistent equations derived above cannot be generally solved in an
analytic way, but we can obtain the solution by numerical iteration [29]. We
discretize the wavenumber as

$\displaystyle dk_{j}$
$\displaystyle=k_{c}\frac{j}{\sum_{j=1}^{j_{\text{max}}}j},\quad
k_{j}=\frac{1}{2}dk_{j}+\sum_{j^{\prime}=1}^{j-1}dk_{j^{\prime}},$ (41)

where $j=1,2,...,j_{\text{max}}$ and $k_{c}$ is the cutoff wavenumber.
Hereafter, we fix $j_{\text{max}}=500$.

###  IV.3 Density of states

Figure 2: (Color online) Quantum transport for $W=2$ (red line), $W=5$ (green
line), and $W=10$ (blue line), derived by the SCBA. Density of states for
(a-1) the delta function potential and (a-2) Gaussian potential
($k_{c}=q_{0}$). Conductivity for (b-1) the delta function potential and (b-2)
the Gaussian potential ($k_{c}=q_{0}$). (c-1) and (c-2) are the enlargements
near the zero energy of (b-1) and (b-2), respectively.

The density of states and conductivity are obtained by SCBA. We fix
$k_{c}=q_{0}$ in the following. Note that the results do not explicitly depend
on $q_{0}$ because the density of states and the conductivity are functions of
$\epsilon/(q_{0}\hbar v)$ and $W$, and is normalized by $q_{0}^{2}/\hbar v$
and $e^{2}q_{0}/\hbar$, respectively.

Figures 2(a-1) and 2(a-2) show the density of states as a function of the
Fermi energy for the delta function and Gaussian potentials, respectively. We
can see a pronounced peak structure around the zero energy. This peak stems
from the flat band at $\epsilon=0$ and is broaden by the impurity potential.
Away from the zero energy, the density of states is approximately proportional
to $\epsilon^{2}$, which comes from the linear dispersion bands and is
essentially the same as those in the clean limit.

###  IV.4 Conductivity

Figures 2(b-1) and 2(b-2) show the conductivity as a function of the Fermi
energy for the delta function and Gaussian potentials, respectively. In the
high-energy region, for both the delta function/Gaussian potential, the
conductivity increases monotonically and reaches nearly the values given in
the Boltzmann theory. However, the conductivity is significantly suppressed in
the vicinity of the zero energy. The energy range of the suppressed area is as
wide as that of the peak of the density of states. This implies that the flat
band plays a crucial role in suppressing conductivity.

In addition, the suppressed conductivity exhibits a behavior specific to the
length scale of potential. Figures 2(c-1) and 2(c-2) show enlarged views of
the suppressed area. The conductivity for the delta function potential [Fig.
2(c-1)] shows a small peak, whereas that for the Gaussian potential [Fig.
2(c-2)] is nearly constant with a tiny dip on the zero energy. This difference
is due to the vertex corrections $J_{1}-J_{7}$ in Eq. (39). We clarified that
the suppressed conductivity for the Gaussian potential without vertex
correction ($J_{0}=1$, $J_{1}=J_{2}=...=J_{7}=0$) makes a small peak, as for
the delta function potential. Namely, the vertex correction plays a vital role
in the low-energy regime. Unlike the Gaussian potential, the conductivity for
the delta function potential produces a small peak and is not flattened by the
vertex correction, which is given by $J_{0}\neq 0$ and
$J_{1}=J_{2}=...=J_{7}=0$. Note that the suppressed conductivity shows less
dependence on $W$ than the unsuppressed area in the higher energy.

In this study, we assume zero temperature. The conductivity at a finite
temperature is given by $\int d\epsilon(-\partial
f/\partial\epsilon)\sigma(\epsilon)$. The Fermi-energy dependence of the
conductivity is smeared by the Fermi distribution $f$. In spin-1 systems,
since the conductivity has a large Fermi-energy dependence, the temperature
dependence may be nonmonotonic.

###  IV.5 Interband effect

Here we clarify the details of the characteristic behavior near zero energy
and its origin. To do so, we decompose the conductivity into intraband and
interband contributions, which are defined by

$\displaystyle\sigma_{\text{intra}}(\epsilon)$ $\displaystyle=-\frac{\hbar
e^{2}v^{2}}{4\pi}\sum_{s,s^{\prime}=\pm
1}ss^{\prime}\int\frac{d\bm{k^{\prime}}}{(2\pi)^{3}}\bigl{(}S_{cc}G^{s}_{c}J^{ss^{\prime}}_{cc}G^{s^{\prime}}_{c}$
$\displaystyle+S_{vv}G^{s}_{v}J^{ss^{\prime}}_{vv}G^{s^{\prime}}_{v}\bigr{)},$
(42)

and

$\displaystyle\sigma_{\text{inter}}(\epsilon)$ $\displaystyle=-\frac{\hbar
e^{2}v^{2}}{4\pi}\sum_{s,s^{\prime}=\pm
1}ss^{\prime}\int\frac{d\bm{k^{\prime}}}{(2\pi)^{3}}\bigl{(}S_{0c}G^{s}_{c}J^{ss^{\prime}}_{c0}G^{s^{\prime}}_{0}$
$\displaystyle+S_{c0}G^{s}_{0}J^{ss^{\prime}}_{0c}G^{s^{\prime}}_{c}+S_{0v}G^{s}_{v}J^{ss^{\prime}}_{v0}G^{s^{\prime}}_{0}$
$\displaystyle+S_{v0}G^{s}_{0}J^{ss^{\prime}}_{0v}G^{s^{\prime}}_{v}\bigr{)},$
(43)

where the subscripts $c$, $0$, and $v$ denote the conduction, flat, and
valence bands in the band basis, respectively. They are obtained by
diagonalizing the Green’s function matrix as

$\displaystyle\hat{U}^{\dagger}\hat{G}(\bm{k},\epsilon+is0)\hat{U}=\begin{pmatrix}G^{s}_{c}&0&0\\\
0&G^{s}_{0}&0\\\ 0&0&G^{s}_{v}\\\ \end{pmatrix}.$ (44)

In this basis, the velocities $S_{x}$ and $J_{x}$ are written as

$\displaystyle\hat{U}^{\dagger}\hat{S}_{x}\hat{U}=\begin{pmatrix}S_{cc}&S_{c0}&0\\\
S_{0c}&0&S_{0v}\\\ 0&S_{v0}&S_{vv}\\\ \end{pmatrix},$ (45)

and

$\displaystyle\hat{U}^{\dagger}\hat{J_{x}}(k,\epsilon+is0,\epsilon+is^{\prime}0)\hat{U}=\begin{pmatrix}J^{ss^{\prime}}_{cc}&J^{ss^{\prime}}_{c0}&0\\\
J^{ss^{\prime}}_{0c}&J^{ss^{\prime}}_{00}&J^{ss^{\prime}}_{0v}\\\
0&J^{ss^{\prime}}_{v0}&J^{ss^{\prime}}_{vv}\\\ \end{pmatrix}.$ (46)

The flat band has zero velocity leading to the absence of intraband terms. The
interband terms between the conduction and valence bands are also zero.

Figure 3: (Color online) The conductivity from the intraband transition of the
Dirac cone (black line) and from the interband transition between the Dirac
cone and the flat band (red line) for $W=5$ derived by the SCBA. (a) delta
function potential, (b) Gaussian potential, $k_{c}=q_{0}$ and (c) Gaussian
potential, $k_{c}=3q_{0}$

Figure 3 shows the decomposed conductivities in the band basis. One can see a
small peak of the interband term and suppression of the intraband terms near
$\epsilon=0$. Namely, the interband conductivity is more dominant than the
intraband one. This suggests that disorder effects are decisive for the low-
energy transport behavior.

###  IV.6 Cutoff dependence

Figure 4: (Color online) Quantum transport for $k_{c}=q_{0}$ (purple line),
$k_{c}=2q_{0}$ (green line) and $k_{c}=3q_{0}$ (blue line) derived by the
SCBA. (Gaussian potential, $W=5$) (a) Density of states, (b) Conductivity and
(c) Conductivity near the suppressed area

Figure 4 is the result of varying the value of $k_{c}$ in Eq. (41) for the
Gaussian potential. Note that we fix $W=5$ here. One can see that the peak of
the density of states shown in Fig. (4) rapidly increases as $k_{c}$
increases. Our model has an entirely flat band, which is a simple
approximation for that in the actual material. Namely, the cutoff momentum
$k_{c}$ corresponds to the range over which the band can be approximated as
flat. Consequently, as $k_{c}$ increases, the contribution from the flat band
also increases, resulting in a large peak structure in the density of states.

The conductivity shown in Fig. 4(b) indicates that the energy region showing
the suppressed conductivity ($\sigma\sim 0.011e^{2}q_{0}/\hbar$) becomes wider
as $k_{c}$ increases for the same reason as for the enhancement of the density
of states discussed above. We also find that the conductivity at $\epsilon\sim
0$ does not change significantly with $k_{c}$ [Fig. 4(c)]. As the density of
states increases, the number of scattering channels also increases, and as a
result, the conductivity is expected to remain nearly constant.

##  V Discussion

###  V.1 Comparison of results from Boltzmann equation and SCBA

Figure 5: (Color online) The conductivity ($W=2$) derived by the Boltzmann
equation (dashed line) and the SCBA (solid line) for (a) the delta function
potential and (b) Gaussian potential ($k_{c}=q_{0}$).

Figure 5 compares the conductivity derived from the Boltzmann equation with
that derived from SCBA. The specific behavior of the low-energy region
(suppressed conductivity) derived from SCBA cannot be derived from the
Boltzmann equation for the following reasons. The suppressed conductivity
originates from the interband effects of the Dirac cone and the flat band
broadened by impurity scattering. The spectral broadening is not considered in
the Boltzmann equation hence the interband effect does not contribute to the
conductivity, owing to the energy conservation. SCBA is the simplest method to
take spectral broadening into account, and it provides a simple way to
understand interband conduction phenomena in spin-1 systems.

###  V.2 Related studies

Figure 6: (Color online) Conductivity on the zero energy for the delta
function potential (black line) and Gaussian potential ($k_{c}=q_{0}$:purple
line, $k_{c}=2q_{0}$:green line and $k_{c}=3q_{0}$:blue line) derived by the
SCBA.

In a three-dimensional Weyl fermion under the finite range (Gaussian) impurity
potential, the conductivity at the Weyl point changes its behavior
significantly depending on the scattering strength, indicating the
semimetal–metal transition [30]. In the spin-1 fermion system, on the
contrary, the conductivity at $\epsilon=0$ has a much smaller dependence on
the scattering strength, showing no transition.

A spin-1 fermion in two spatial dimension is not protected by symmetry but
approximately emerges. In the two-dimensional spin-1 fermion with the delta
function disorder potential, the density of states has a peak, and the
conductivity is suppressed near the zero energy [35] like in the three-
dimensional system. The behavior of the conductivity suppression differs
between the 2D and 3D systems, with the 2D system showing a gradual increase
as energy approaches zero, whereas the 3D system shows a rapid increase and
makes a small peak. Furthermore, we elucidate a dependence of the length scale
of impurity of the conductivity for the three-dimensional system. The short-
range (delta function) potential makes a peak, while the finite-range
(Gaussian) one does not. This low-energy spectral property of conductivity can
be essential to comparing and understanding experimental results in the
future.

###  V.3 On experimental realization

The present study has shown the Fermi-energy dependences of the density of
states and the conductivity. Here we comment on relation to experimental
realization. A direct way to tune the Fermi energy is to use the gating of a
thin-film spin-1 fermion material. Alternatively, the Fermi energy could be
varied by doping the bulk material, although this is not easy to control
continuously. A chiral crystal CoSi hosts a spin-1 chiral fermion on the
Brillouin zone center in the absence of spin-orbit interaction, which would be
a good platform to observe the quantum transport of the spin-1 fermion.
However, in addition to the spin-1 fermion at the $\Gamma$ point, there is a
double Weyl fermion at the $R$ point, and one must take the additional
contribution from them into the transport properties. Further research on this
is desirable.

##  VI Conclusion

This study clarified the quantum transport theory for a spin-1 chiral fermion
with disorder potential within the SCBA in the presence of the current vertex
correction. As a result, we found some characteristic phenomena originated
from the flat-band structure near the zero energy. One is a peak structure of
the density of states. The other is the suppression of conductivity. It has
been discussed that these apparently contradicting behaviors are reconciled by
considering the vanishing group velocity of the flat band. In addition, we
have found that the interband effect, which is inherent to the spin-1 band
structure, significantly contributes to the electrical conductivity.

These results provide the basis for clarifying the quantum transport phenomena
of spin-1 fermions. Furthermore, it is suggested that nontrivial impurity
effects are latent in chiral fermions beyond Dirac and Weyl fermions, and it
will be an interesting problem to clarify quantum transport phenomena in more
diverse chiral-fermion systems.

Note added. We became aware of a very recent work which studies the density of
states [39], Fermi arcs and surface Berry curvature for the spin-1 chiral
fermion in the presence of disorder. We have found the same qualitative
behavior of the density of states.

###### Acknowledgements.

This work is supported by JSPS KAKENHI for Grants (Grants No. JP20K03835 and
No. JP21K20356) and the Sumitomo Foundation (Grant No. 190228).

##  Appendix A Useful relations

Let $\bm{n}_{\perp 1}$, $\bm{n}_{\perp 2}$, and $\bm{n}$ be the three-
dimensional unit vectors that are perpendicular mutually. Let $S_{x}$,
$S_{y}$, and $S_{z}$ be the $3\times 3$ spin-1 representation matrices
introduced in the main text. One finds the following useful relations.

$\displaystyle(\hat{\bm{S}}\cdot\bm{n})^{3}=(\hat{\bm{S}}\cdot\bm{n}),$ (47)
$\displaystyle(\hat{\bm{S}}\cdot\bm{n})^{2}\hat{S}_{i}(\hat{\bm{S}}\cdot\bm{n})^{2}=(\hat{\bm{S}}\cdot\bm{n})\hat{S}_{i}(\hat{\bm{S}}\cdot\bm{n})=n_{i}(\hat{\bm{S}}\cdot\bm{n}),$
(48) $\displaystyle(\hat{\bm{S}}\cdot\bm{n}_{\perp
1})^{2}+(\hat{\bm{S}}\cdot\bm{n}_{\perp
2})^{2}+(\hat{\bm{S}}\cdot\bm{n})^{2}=2\hat{S}_{0},$ (49)
$\displaystyle(\hat{\bm{S}}\cdot\bm{n}_{\perp
1})\hat{S}_{i}(\hat{\bm{S}}\cdot\bm{n}_{\perp
1})+(\hat{\bm{S}}\cdot\bm{n}_{\perp
2})\hat{S}_{i}(\hat{\bm{S}}\cdot\bm{n}_{\perp 2})$
$\displaystyle+(\hat{\bm{S}}\cdot\bm{n})\hat{S}_{i}(\hat{\bm{S}}\cdot\bm{n})=\hat{S}_{i},$
(50) $\displaystyle(\hat{\bm{S}}\cdot\bm{n}_{\perp
1})\hat{S}_{i}\hat{S}_{j}(\hat{\bm{S}}\cdot\bm{n}_{\perp
1})+(\hat{\bm{S}}\cdot\bm{n}_{\perp
2})\hat{S}_{i}\hat{S}_{j}(\hat{\bm{S}}\cdot\bm{n}_{\perp 2})$
$\displaystyle+(\hat{\bm{S}}\cdot\bm{n})\hat{S}_{i}\hat{S}_{j}(\hat{\bm{S}}\cdot\bm{n})=-\hat{S}_{j}\hat{S}_{i}+2\delta_{ij}\hat{S}_{0}.$
(51)

An arbitrary unit vector $\bm{n}^{\prime}$ is expressed as

$\displaystyle\bm{n^{\prime}}=\bm{n}_{\perp 1}\sin\theta\cos\phi+\bm{n}_{\perp
2}\sin\theta\sin\phi+\bm{n}\cos\theta,$ (52)

where $\theta$ denotes the angle between $\bm{n}$ and $\bm{n^{\prime}}$ and
$\phi$ denotes the azimuth angle in the $\bm{n}_{\perp 1}$-$\bm{n}_{\perp 2}$
plane.

Using Eqs. (47)–(51), we can calculate the following integrals as

$\displaystyle\int_{0}^{2\pi}\int_{0}^{\pi}d\theta
d\phi|u(\bm{k}-\bm{k^{\prime}})|^{2}(\hat{\bm{S}}\cdot\bm{n^{\prime}})=(\hat{\bm{S}}\cdot\bm{n})V_{1}^{2}(k,k^{\prime}),$
(53) $\displaystyle\int_{0}^{2\pi}\int_{0}^{\pi}d\theta
d\phi|u(\bm{k}-\bm{k^{\prime}})|^{2}(\hat{\bm{S}}\cdot\bm{n^{\prime}})^{2}$
$\displaystyle=\left(\frac{3}{2}V_{2}^{2}(k,k^{\prime})-\frac{1}{2}V_{0}^{2}(k,k^{\prime})\right)(\hat{\bm{S}}\cdot\bm{n})^{2}$
$\displaystyle\quad+\left(V_{0}^{2}(k,k^{\prime})-V_{2}^{2}(k,k^{\prime})\right)\hat{S}_{0},$
(54) $\displaystyle\int_{0}^{2\pi}\int_{0}^{\pi}d\theta
d\phi|u(\bm{k}-\bm{k^{\prime}})|^{2}n^{\prime}_{x}\hat{S}_{0}=V_{1}^{2}(k,k^{\prime})n_{x}\hat{S}_{0},$
(55) $\displaystyle\int_{0}^{2\pi}\int_{0}^{\pi}d\theta
d\phi|u(\bm{k}-\bm{k^{\prime}})|^{2}n^{\prime}_{x}(\hat{\bm{S}}\cdot\bm{n^{\prime}})$
$\displaystyle=\left(\frac{3}{2}V_{2}^{2}(k,k^{\prime})-\frac{1}{2}V_{0}^{2}(k,k^{\prime})\right)n_{x}(\hat{\bm{S}}\cdot\bm{n})$
$\displaystyle\quad+\frac{1}{2}(V_{0}^{2}(k,k^{\prime})-V_{2}^{2}(k,k^{\prime}))\hat{S}_{x},$
(56) $\displaystyle\int_{0}^{2\pi}\int_{0}^{\pi}d\theta
d\phi|u(\bm{k}-\bm{k^{\prime}})|^{2}n^{\prime}_{x}(\hat{\bm{S}}\cdot\bm{n^{\prime}})^{2}$
$\displaystyle=\frac{1}{2}(V_{1}^{2}(k,k^{\prime})-V_{3}^{2}(k,k^{\prime}))\hat{S}_{x}(\hat{\bm{S}}\cdot\bm{n})$
$\displaystyle\quad+\frac{1}{2}(V_{1}^{2}(k,k^{\prime})-V_{3}^{2}(k,k^{\prime}))(\hat{\bm{S}}\cdot\bm{n})\hat{S}_{x}$
$\displaystyle\quad+(V_{1}^{2}(k,k^{\prime})-V_{3}^{2}(k,k^{\prime}))n_{x}\hat{S}_{0}$
$\displaystyle\quad+\left(\frac{5}{2}V_{3}^{2}(k,k^{\prime})-\frac{3}{2}V_{1}^{2}(k,k^{\prime})\right)n_{x}(\hat{\bm{S}}\cdot\bm{n})^{2}.$
(57)

##  Appendix B The Bethe-Salpeter equation

From Appendix A, the Bethe-Salpeter equation becomes Eq. (82) where
$J_{i}=J_{i}(k,\epsilon+is0,\epsilon+is^{\prime}0),J^{\prime}_{i}=J_{i}(k^{\prime},\epsilon+is0,\epsilon+is^{\prime}0),V_{i}^{2}=V_{i}^{2}(k,k^{\prime}),x=x(k^{\prime},\epsilon+is0),x^{\prime}=x(k^{\prime},\epsilon+is^{\prime}0)$
and so on.

$\displaystyle\left(\begin{array}[]{c}J_{0}\\\ J_{1}\\\ J_{2}\\\ J_{3}\\\
J_{4}\\\ J_{5}\\\ J_{6}\\\ J_{7}\end{array}\right)$ $\displaystyle=$
$\displaystyle\left(\begin{array}[]{c}1\\\ 0\\\ 0\\\ 0\\\ 0\\\ 0\\\ 0\\\
0\end{array}\right)+\int\displaystyle\frac{k^{\prime
2}dk^{\prime}}{(2\pi)^{3}}n_{\text{i}}\begin{pmatrix}V_{0}^{2}&0&\frac{1}{2}V_{0}^{2}-\frac{1}{2}V_{2}^{2}&V_{0}^{2}-V_{2}^{2}&V_{0}^{2}-V_{2}^{2}&0&0&0\\\
0&\frac{5}{2}V_{3}^{2}-\frac{3}{2}V_{1}^{2}&0&0&0&0&0&0\\\
0&0&\frac{3}{2}V_{2}^{2}-\frac{1}{2}V_{0}^{2}&0&0&0&0&0\\\
0&0&0&\frac{3}{2}V_{2}^{2}-\frac{1}{2}V_{0}^{2}&0&0&0&0\\\
0&0&0&0&\frac{3}{2}V_{2}^{2}-\frac{1}{2}V_{0}^{2}&0&0&0\\\
0&\frac{1}{2}V_{1}^{2}-\frac{1}{2}V_{3}^{2}&0&0&0&V_{1}^{2}&0&0\\\
0&\frac{1}{2}V_{1}^{2}-\frac{1}{2}V_{3}^{2}&0&0&0&0&V_{1}^{2}&0\\\
0&V_{1}^{2}-V_{3}^{2}&0&0&0&0&0&V_{1}^{2}\\\
\end{pmatrix}\hat{T}\left(\begin{array}[]{c}J_{0}^{\prime}\\\
J_{1}^{\prime}\\\ J_{2}^{\prime}\\\ J_{3}^{\prime}\\\ J_{4}^{\prime}\\\
J_{5}^{\prime}\\\ J_{6}^{\prime}\\\ J_{7}^{\prime}\end{array}\right),$ (82)

where the matrix $\hat{T}$ is defined as

$\displaystyle\hat{T}=\begin{pmatrix}xx^{\prime}&0&0&0&0&0&0&0\\\
T_{01}&T_{11}&T_{21}&T_{31}&T_{41}&T_{51}&T_{61}&T_{71}\\\
T_{02}&T_{12}&T_{22}&T_{32}&T_{42}&T_{52}&T_{62}&T_{72}\\\
zx^{\prime}&0&0&T_{33}&0&yx^{\prime}&0&0\\\
xz^{\prime}&0&0&0&T_{44}&0&xy^{\prime}&0\\\
yx^{\prime}&0&0&yx^{\prime}&0&T_{55}&0&0\\\
xy^{\prime}&0&0&0&xy^{\prime}&0&T_{66}&0\\\ 0&0&0&0&0&0&0&xx^{\prime}\\\
\end{pmatrix}.$ (84)

Here, the matrix elements $T_{ij}$ in the second line of $\hat{T}$ are given
by

$\displaystyle T_{01}$ $\displaystyle=yz^{\prime}+zy^{\prime},$ (85)
$\displaystyle T_{11}$
$\displaystyle=xx^{\prime}+xz^{\prime}+yy^{\prime}+zx^{\prime}+zz^{\prime},$
(86) $\displaystyle T_{21}$
$\displaystyle=xy^{\prime}+yx^{\prime}+yz^{\prime}+zy^{\prime},$ (87)
$\displaystyle T_{31}$ $\displaystyle=xy^{\prime}+yz^{\prime}+zy^{\prime},$
(88) $\displaystyle T_{41}$
$\displaystyle=yx^{\prime}+yz^{\prime}+zy^{\prime},$ (89) $\displaystyle
T_{51}$ $\displaystyle=xz^{\prime}+yy^{\prime}+zz^{\prime},$ (90)
$\displaystyle T_{61}$ $\displaystyle=yy^{\prime}+zx^{\prime}+zz^{\prime},$
(91) $\displaystyle T_{71}$
$\displaystyle=xz^{\prime}+yy^{\prime}+zx^{\prime}+zz^{\prime},$ (92)

$T_{ij}$ in the third line are given by

$\displaystyle T_{02}$ $\displaystyle=yy^{\prime}+zz^{\prime},$ (93)
$\displaystyle T_{12}$
$\displaystyle=xy^{\prime}+yx^{\prime}+yz^{\prime}+zy^{\prime},$ (94)
$\displaystyle T_{22}$
$\displaystyle=xx^{\prime}+xz^{\prime}+yy^{\prime}+zx^{\prime}+zz^{\prime},$
(95) $\displaystyle T_{32}$
$\displaystyle=xz^{\prime}+yy^{\prime}+zz^{\prime},$ (96) $\displaystyle
T_{42}$ $\displaystyle=yy^{\prime}+zx^{\prime}+zz^{\prime},$ (97)
$\displaystyle T_{52}$ $\displaystyle=xy^{\prime}+yz^{\prime}+zy^{\prime},$
(98) $\displaystyle T_{62}$
$\displaystyle=yx^{\prime}+yz^{\prime}+zy^{\prime},$ (99) $\displaystyle
T_{72}$ $\displaystyle=xy^{\prime}+yx^{\prime}+yz^{\prime}+zy^{\prime},$ (100)

and, the others are given by

$\displaystyle T_{33}$ $\displaystyle=xx^{\prime}+zx^{\prime},$ (101)
$\displaystyle T_{44}$ $\displaystyle=xx^{\prime}+xz^{\prime},$ (102)
$\displaystyle T_{55}$ $\displaystyle=xx^{\prime}+zx^{\prime},$ (103)
$\displaystyle T_{66}$ $\displaystyle=xx^{\prime}+xz^{\prime}.$ (104)

By solving these eight self-consistent equations, $J_{0}$–$J_{7}$ are
determined.

## References

  * Bradlyn  _et al._ [2016] B. Bradlyn, J. Cano, Z. Wang, M. G. Vergniory, C. Felser, R. J. Cava, and B. A. Bernevig, Beyond dirac and weyl fermions: Unconventional quasiparticles in conventional crystals, Science 353, aaf5037 (2016).
  * Tang  _et al._ [2017] P. Tang, Q. Zhou, and S.-C. Zhang, Multiple Types of Topological Fermions in Transition Metal Silicides, Phys. Rev. Lett. 119, 206402 (2017).
  * Takane  _et al._ [2019] D. Takane, Z. Wang, S. Souma, K. Nakayama, T. Nakamura, H. Oinuma, Y. Nakata, H. Iwasawa, C. Cacho, T. Kim, K. Horiba, H. Kumigashira, T. Takahashi, Y. Ando, and T. Sato, Observation of chiral fermions with a large topological charge and associated fermi-arc surface states in cosi, Phys. Rev. Lett. 122, 076402 (2019).
  * Sanchez  _et al._ [2019] D. S. Sanchez, I. Belopolski, T. A. Cochran, X. Xu, J.-X. Yin, G. Chang, W. Xie, K. Manna, V. Süß, C.-Y. Huang, N. Alidoust, D. Multer, S. S. Zhang, N. Shumiya, X. Wang, G.-Q. Wang, T.-R. Chang, C. Felser, S.-Y. Xu, S. Jia, H. Lin, and M. Z. Hasan, Topological chiral crystals with helicoid-arc quantum states, Nature 567, 500 (2019).
  * Rao  _et al._ [2019] Z. Rao, H. Li, T. Zhang, S. Tian, C. Li, B. Fu, C. Tang, L. Wang, Z. Li, W. Fan, J. Li, Y. Huang, Z. Liu, Y. Long, C. Fang, H. Weng, Y. Shi, H. Lei, Y. Sun, T. Qian, and H. Ding, Observation of unconventional chiral fermions with long Fermi arcs in CoSi, Nature 567, 496 (2019).
  * Yuan  _et al._ [2019] Q.-Q. Yuan, L. Zhou, Z.-C. Rao, S. Tian, W.-M. Zhao, C.-L. Xue, Y. Liu, T. Zhang, C.-Y. Tang, Z.-Q. Shi, Z.-Y. Jia, H. Weng, H. Ding, Y.-J. Sun, H. Lei, and S.-C. Li, Quasiparticle interference evidence of the topological Fermi arc states in chiral fermionic semimetal CoSi, Sci Adv 5, eaaw9485 (2019).
  * Shen  _et al._ [2010] R. Shen, L. B. Shao, B. Wang, and D. Y. Xing, Single dirac cone with a flat band touching on line-centered-square optical lattices, Phys. Rev. B 81, 041410(R) (2010).
  * Tang  _et al._ [2021] K. Tang, Y.-C. Lau, K. Nawa, Z. Wen, Q. Xiang, H. Sukegawa, T. Seki, Y. Miura, K. Takanashi, and S. Mitani, Spin Hall effect in a spin-1 chiral semimetal, Phys. Rev. Research 3, 033101 (2021).
  * Flicker  _et al._ [2018] F. Flicker, F. de Juan, B. Bradlyn, T. Morimoto, M. G. Vergniory, and A. G. Grushin, Chiral optical response of multifold fermions, Phys. Rev. B 98, 155145 (2018).
  * Sánchez-Martínez  _et al._ [2019] M.-Á. Sánchez-Martínez, F. de Juan, and A. G. Grushin, Linear optical conductivity of chiral multifold fermions, Phys. Rev. B 99, 155145 (2019).
  * Habe [2019] T. Habe, Dynamical conductivity in the multiply degenerate point-nodal semimetal CoSi, Phys. Rev. B 100, 245131 (2019).
  * Xu  _et al._ [2020] B. Xu, Z. Fang, M.-Á. Sánchez-Martínez, J. W. F. Venderbos, Z. Ni, T. Qiu, K. Manna, K. Wang, J. Paglione, C. Bernhard, C. Felser, E. J. Mele, A. G. Grushin, A. M. Rappe, and L. Wu, Optical signatures of multifold fermions in the chiral topological semimetal CoSi, Proc. Natl. Acad. Sci. 117, 27104 (2020).
  * Ni  _et al._ [2020] Z. Ni, B. Xu, M.-Á. Sánchez-Martínez, Y. Zhang, K. Manna, C. Bernhard, J. W. F. Venderbos, F. de Juan, C. Felser, A. G. Grushin, and L. Wu, Linear and nonlinear optical responses in the chiral multifold semimetal RhSi, npj Quantum Materials 5, 1 (2020).
  * Ni  _et al._ [2021] Z. Ni, K. Wang, Y. Zhang, O. Pozo, B. Xu, X. Han, K. Manna, J. Paglione, C. Felser, A. G. Grushin, F. de Juan, E. J. Mele, and L. Wu, Giant topological longitudinal circular photo-galvanic effect in the chiral multifold semimetal CoSi, Nat. Commun. 12, 154 (2021).
  * Kaushik and Cano [2021] S. Kaushik and J. Cano, Magnetic photocurrents in multifold Weyl fermions, Phys. Rev. B 104, 155149 (2021).
  * Dey and Ghosh [2022] B. Dey and T. K. Ghosh, Dynamical polarization, optical conductivity and plasmon mode of a linear triple component fermionic system, J. Phys. Condens. Matter 34, 255701 (2022).
  * Nandy  _et al._ [2019] S. Nandy, S. Manna, D. Călugăru, and B. Roy, Generalized triple-component fermions: Lattice model, Fermi arcs, and anomalous transport, Phys. Rev. B Condens. Matter 100, 235201 (2019).
  * Chen and Wang [2021] G. Chen and C. M. Wang, Optical conductivities in triple fermions with different monopole charges, J. Phys. Condens. Matter 34, 105303 (2021).
  * Pal  _et al._ [2022] O. Pal, B. Dey, and T. K. Ghosh, Berry curvature induced anisotropic magnetotransport in a quadratic triple-component fermionic system, J. Phys. Condens. Matter 34, 155702 (2022).
  * Hosur and Qi [2013] P. Hosur and X. Qi, Recent developments in transport phenomena in Weyl semimetals, C. R. Phys. 14, 857 (2013).
  * Armitage  _et al._ [2018] N. P. Armitage, E. J. Mele, and A. Vishwanath, Weyl and Dirac semimetals in three-dimensional solids, Rev. Mod. Phys. 90, 015001 (2018).
  * Gorbar  _et al._ [2020] E. V. Gorbar, V. A. Miransky, I. A. Shovkovy, and P. O. Sukhachov,  _Electronic Properties of Dirac and Weyl Semimetals_ (World Scientific Publishing Company Pte Limited, 2020).
  * Fradkin [1986] E. Fradkin, Critical behavior of disordered degenerate semiconductors. II. Spectrum and transport properties in mean-field theory, Phys. Rev. B 33, 3263 (1986).
  * Ludwig  _et al._ [1994] A. W. W. Ludwig, M. P. A. Fisher, R. Shankar, and G. Grinstein, Integer quantum hall transition: An alternative approach and exact results, Phys. Rev. B 50, 7526 (1994).
  * Shon and Ando [1998] N. H. Shon and T. Ando, Quantum Transport in Two-Dimensional Graphite System, J. Phys. Soc. Jpn. 67, 2421 (1998).
  * Ziegler [1998] K. Ziegler, Delocalization of 2D Dirac Fermions: The Role of a Broken Supersymmetry, Phys. Rev. Lett. 80, 3113 (1998).
  * Tworzydło  _et al._ [2006] J. Tworzydło, B. Trauzettel, M. Titov, A. Rycerz, and C. W. J. Beenakker, Sub-Poissonian shot noise in graphene, Phys. Rev. Lett. 96, 246802 (2006).
  * Katsnelson [2006] M. I. Katsnelson, Zitterbewegung, chirality, and minimal conductivity in graphene, Eur. Phys. J. B 51, 157 (2006).
  * Noro  _et al._ [2010] M. Noro, M. Koshino, and T. Ando, Theory of Transport in Graphene with Long-Range Scatterers, J. Phys. Soc. Jpn. 79, 094713 (2010).
  * Ominato and Koshino [2014] Y. Ominato and M. Koshino, Quantum transport in a three-dimensional weyl electron system, Phys. Rev. B 89, 054202 (2014).
  * Kobayashi  _et al._ [2014] K. Kobayashi, T. Ohtsuki, K.-I. Imura, and I. F. Herbut, Density of states scaling at the semimetal to metal transition in three dimensional topological insulators, Phys. Rev. Lett. 112, 016402 (2014).
  * Nandkishore  _et al._ [2014] R. Nandkishore, D. A. Huse, and S. L. Sondhi, Rare region effects dominate weakly disordered three-dimensional Dirac points, Phys. Rev. B 89, 245110 (2014).
  * Ominato and Koshino [2015] Y. Ominato and M. Koshino, Quantum transport in three-dimensional weyl electron system in the presence of charged impurity scattering, Phys. Rev. B 91, 035202 (2015).
  * Ominato and Koshino [2016] Y. Ominato and M. Koshino, Magnetotransport in Weyl semimetals in the quantum limit: Role of topological surface states, Phys. Rev. B 93, 245304 (2016).
  * Vigh  _et al._ [2013] M. Vigh, L. Oroszlány, S. Vajna, P. San-Jose, G. Dávid, J. Cserti, and B. Dóra, Diverging dc conductivity due to a flat band in a disordered system of pseudospin-1 dirac-weyl fermions, Phys. Rev. B 88, 161413(R) (2013).
  * Häusler [2015] W. Häusler, Flat-band conductivity properties at long-range Coulomb interactions, Phys. Rev. B 91, 041102(R) (2015).
  * Yang  _et al._ [2019] Z. Yang, W. Chen, Q. W. Shi, and Q. Li, Quantum conductivity correction in a two-dimensional disordered pseudospin-1 system, Phys. Rev. B 99, 134204 (2019).
  * Burgos  _et al._ [2022] R. Burgos, J. H. Warnes, and G. C. Arteaga, Semiclassical anisotropic transport theory in two dimensional pseudo-spin one system, Physica E 136, 114998 (2022).
  * Hsu  _et al._ [2022] H.-C. Hsu, I. C. Fulga, and J.-S. You, Disorder effects on triple point fermions, arXiv preprint arXiv:2205.12003 (2022).


Copyright for this paper by its authors.
Use permitted under Creative Commons License Attribution 4.0 International (CC
BY 4.0).

FIRE 2020: Forum for Information Retrieval Evaluation, December 16-20
December, 2020, Hyderabad, India

# WLV-RIT at HASOC-Dravidian-CodeMix-FIRE2020: Offensive Language
Identification in Code-switched YouTube Comments

Tharindu Ranasinghe  University of Wolverhampton, UK
<EMAIL_ADDRESS>Sarthak Gupte<EMAIL_ADDRESS>Marcos
Zampieri<EMAIL_ADDRESS>Ifeoma Nwogu<EMAIL_ADDRESS>Rochester
Institute of Technology, USA

(2020)

###### Abstract

This paper describes the WLV-RIT entry to the Hate Speech and Offensive
Content Identification in Indo-European Languages (HASOC) shared task 2020.
The HASOC 2020 organizers provided participants with annotated datasets
containing social media posts of code-mixed in Dravidian languages (Malayalam-
English and Tamil-English). We participated in task 1: Offensive comment
identification in Code-mixed Malayalam Youtube comments. In our methodology,
we take advantage of available English data by applying cross-lingual
contextual word embeddings and transfer learning to make predictions to
Malayalam data. We further improve the results using various fine tuning
strategies. Our system achieved 0.89 weighted average F1 score for the test
set and it ranked 5th place out of 12 participants.

###### keywords:

offensive language identification hate speech text classification code-
switching

##  1 Introduction

Offensive content is pervasive in social media putting users of various
platforms at risk [1]. The pervasiveness of such content motivated the
development of several systems capable of identifying offensive posts in a
number of languages [2, 3]. Once identified, these posts can be then set aside
for human moderation or deleted from online platforms mitigating risks to
their users [4].

Recent studies have addressed many types of offensive content such as online
abuse [5, 6], aggression [7], cyberbullying [8, 9], and hate speech [10, 11].
International workshops and competitions such as HatEval 2019 [12], OffensEval
2019 and 2020 [13, 14], co-located with SemEval, have been organized in the
last two years attracting a large number of participants. Most high performing
system in these competitions used neural networks and contextual word
embeddings such as BERT [15].

In this paper we describe the WLV-RIT entry to the the HASOC 2020 shared task
which featured Malayalam-English code-switched data. Building on the
experience of recent high performing models submitted to the OffensEval
competitions, we use a transformer-based architecture described in detail in
Section 3. The HASOC 2020 code-switching dataset is a particularly challenging
one for state-of-the-art offensive language detection systems and we make use
of transfer learning techniques that have been recently applied to project
predictions from English to resource-poorer languages with great success [16].

##  2 Task description and Datasets

The goal of this task is to identify the offensive language of the code-mixed
dateset of comments/posts in Dravidian Languages (Tamil-English and Malayalam-
English) collected from social media. Each comment/post is annotated with an
offensive language label at the comment/post level. The dataset has been
collected from YouTube comments [17]. We participated in task 1 which is a
message-level label classification task; given a YouTube comment in Code-mixed
(Mixture of Native and Roman Script) Tamil and Malayalam, systems have to
classify whether a post is offensive or not-offensive. To the best of our
knowledge, this is the first dataset to be released for offensive language
detection in Dravidian Code-Mixed text [17].

In addition to the dataset provided by the organisers we also used an English
Offensive Language Identification Dataset (OLID) [18] used in the SemEval-2019
Task 6 (OffensEval) [13] for transfer learning experiments which are
describing in Section 3. OLID is arguably one of the most popular offensive
language datasets. It contains manually annotated tweets with the following
three-level taxonomy and labels:

  * A:

Offensive language identification - offensive vs. non-offensive;

  * B:

Categorization of offensive language - targeted insult or thread vs.
untargeted profanity;

  * C:

Offensive language target identification - individual vs. group vs. other.

We adopted the transfer learning strategy similar to previous recent work
[16]. We believe that the flexibility provided by the hierarchical annotation
model of OLID allows us to map OLID level A (offensive vs. non-offensive ) to
labels in the HASOC Malayalam-English dataset.

##  3 Methods

The methodology applied in this work is divided in two parts. Subsection 3.1
describes traditional machine learning applied to this task and in Subsection
3.2 we describe the transformer models used.

The motivation behind our methodology is the recent success that the
transformers had in wide range of NLP tasks like language generation [15],
sequence classification [19, 20], word similarity [21], named entity
recognition [22], question and answering [23] etc. The main idea of the
methodology is that we train a classification model with several transformer
models in-order to identify offensive texts. However, the transformer models
are known to be resource intensive requiring fairly large datasets [15],
therefore, we also experimented with several traditional machine learning
models

###  3.1 Traditional Machine Learning Methods

In the first part of the methodology, we used traditional machine learning
models. We experimented with three models; Multinomial Naive Bayes [24],
Support Vector Machines [25], and Random Forest [26]. The models take an input
vector created using Bag-of-words and outputs a label, either offensive or
non-offensive. The models for Multinomial Naive Bayes, SVM and Random Forest
were implemented using the Scikit-learn [27].

#### Data Preprocessing

We performed three preprocessing techniques; removing punctuations, removing
emojis and lemmatising the English words. This was done with the use of the
NLTK (Natural Language Toolkit) library [28] in Python.

#### Hyper Parameter Optimisation

Optimisation of hyper parameters was performed on SVM and random forest only.
For SVM, the hyper parameters fine-tuned were alpha, random state and max
iteration, where alpha represents regularisation, random state is used for
shuffling of the data and max iteration denotes number of passes through the
training data which is also known as epochs. Optimal values achieved were
alpha=0.001, random state=5, max iteration=15. For random forest, only one
hyper parameter was used which is n-estimator that denotes number of decision
trees created. Optimal value achieved for number of trees was 500.

###  3.2 Transformers Models

As the second part of the methodology, we used Transformer models. Transformer
architectures have been trained on general tasks like language modelling and
then can be fine-tuned for classification tasks [29]. They take an input of a
sequence and outputs the representation of the sequence. The sequence has one
or two segments that the first token of the sequence is always [CLS] which
contains the special classification embedding and another special token [SEP]
is used for separating segments. For text classification tasks, Transformer
models take the final hidden state h of the first token [CLS] as the
representation of the whole sequence [29]. A simple softmax classifier is
added to the top of the transformer model to predict the probability of a
class as shown in Equation 1 where W is the task-specific parameter matrix.
The architecture diagram of the classification is shown in Figure 1

$p(c|\textbf{h})=softmax(W\textbf{h})$ (1)

#### Transformers

We experimented two pretrained transformer models; BERT [15] and XLM-ROBERTA
[30] We used the HuggingFace’s implementation of the transformer models [31]
and the pre-trained models available in the HuggingFace model
repository.111HuggingFace model repository - https://huggingface.co/models
These models were used mainly considering their support to Malayalam language.
For BERT we used the BERT multilingual model (BERT-M) and for XLM-ROBERTA
(XLM-R) we used the XLM-R-Large model. Both models support 104 languages
including Malayalam. The interesting fact about XLM-R is that it is very
compatible in monolingual benchmarks while achieving best results in cross-
lingual benchmarks at the same time [30].

Figure 1: Transformer Text Classification Architecture

#### Transfer Learning

The main idea of the transfer learning strategy is that we train a
classification model on a resource rich language, typically English, using a
transformer model and perform transfer learning on a less resource language.
We trained the classification model on the first level of OLID [32] and then
we save the weights of the transformer model as well as the softmax layer. We
use this saved weights from English to initialise the weights when we are
training the classification model for Malayalam. This strategy has improved
the performance of different languages with less resources for offensive
language identification such as Hindi, Bengali etc [16]. Therefore we
experimented with this strategy to see whether it improves the results for
Malayalam too. According to the recent research, cross-lingual transformers
have slight edge when using this transfer-learning strategy [16].

#### Data Preprocessing

The data preprocessing for this task was kept fairly minimal to make it
portable for other languages too. We only followed one data preprocessing
technique; converting emojis to text. Emojis are found to play a key role in
expressing emotions in the context of social media [33]. But, we cannot assure
the existence of embeddings for emojis in pretrained models. Therefore as a
preprocessing step, we converted emojis to text. For this conversion we used
the Python libraries demoji 222demoji repository -
https://github.com/bsolomon1124/demojis and emoji 333emoji repository -
https://github.com/carpedm20/emoji. demoji returns a normal descriptive text
and emoji returns a specifically formatted text. For an example, the
conversion of ☺ is ‘slightly smiling face’ using demoji and
‘:slightly_smiling_face:’ using emoji. Considering that demoji returns a
normal text, we used demoji to convert the emojis to text.

#### Fine-tuning

To improve the models, we experimented different fine-tuning strategies:
majority class self-ensemble, average self-ensemble, language modelling, which
are described below. These fine tuning strategies have shown promising results
in recent shared tasks [34].

  1. 1.

Self-Ensemble (SE) \- Self-ensemble is found as a technique which results
better performance than the performance of a single model [35]. In this
approach, same model architecture is trained or fine-tuned with different
random seeds or train-validation splits. Then the output of each model is
aggregated to generate the final results. As the aggregation methods, we
analysed majority-class and average in this research. The number of models
used with self-ensemble will be denoted by $N$.

     * •

Majority-class SE (MSE) \- As the majority class, we computed the mode of the
classes predicted by each model. Given a data instance, following the softmax
layer, a model predicts probabilities for each class and the class with
highest probability is taken as the model predicted class.

     * •

Average SE (ASE) \- In average SE, final probability of class $c$ is
calculated as the average of probabilities predicted by each model as in
Equation 2 where h is the final hidden state of the [CLS] token. Then the
class with highest probability is selected as the final class.

$p_{ASE}(c|h)=\frac{\sum_{k=1}^{N}p_{k}(c|h)}{N}$ (2)

  2. 2.

Language Modelling (LM) \- As language modelling, we retrained the transformer
model on task dataset before fine-tuning it for the downstream task; text
classification. This training is took place according with the model’s initial
trained objective. Following this technique model understanding on the task
data can be improved.

#### Implementation

We used a Nvidia Tesla K80 GPU to train the models. We mainly fine tuned the
learning rate and number of epochs of the classification model manually to
obtain the best results for the validation set. We obtained $1e^{-}5$ as the
best value for learning rate and 3 as the best value for number of epochs for
all the languages. Training for English language took around 1 hour while
training for Malayalam took around 30 minutes.

##  4 Results and Evaluation

In this section, we report the experiments we conducted and their results. As
informed by the task organisers, we used Weighted Average F1 score to measure
the model performance. We also report Precision, Recall and F1 score for each
class label as well the Macro F1 score in the results tables. Results in
Tables 1 \- 5 are computed on validation dataset. Finally, in Section 4.1 we
report the results provided by organisers to our models, for the test set.

Table 1 shows the results we gained with traditional machine learning
algorithms. Out of the three traditional machine learning algorithms Random
Forest performed best, providing us with 0.93 weighted average F1 score. In
the experiments we did with transformers, initially we focused on the impact
of transfer learning when used with different transformer models and the
obtained results are summarised in Table 2. According to the results XLM-R
with transfer learning outperformed other models. Also we could notice that
transfer learning improved both models; BERT and XLM-R.

| Non Hate Offensive | Hate Offensive | Weighted Average |
---|---|---|---|---
Model | P | R | F1 | P | R | F1 | P | R | F1 | F1 Macro
Random Forest | 0.93 | 0.99 | 0.96 | 0.92 | 0.68 | 0.78 | 0.93 | 0.93 | 0.93 | 0.87
Linear SVM | 0.93 | 0.98 | 0.96 | 0.88 | 0.68 | 0.77 | 0.92 | 0.93 | 0.92 | 0.86
Mult. Naive Bayes | 0.90 | 0.98 | 0.94 | 0.88 | 0.53 | 0.66 | 0.90 | 0.90 | 0.89 | 0.80

Table 1: Results for offensive language detection with traditional ML models.
For each model, Precision (P), Recall (R), and F1 are reported on all classes,
and weighted averages. Macro-F1 is also listed.

| Non Hate Offensive | Hate Offensive | Weighted Average |
---|---|---|---|---
Model | P | R | F1 | P | R | F1 | P | R | F1 | F1 Macro
XLM-R (TL) | 0.91 | 0.96 | 0.94 | 0.77 | 0.59 | 0.67 | 0.89 | 0.89 | 0.89 | 0.80
BERT-m (TL) | 0.90 | 0.95 | 0.93 | 0.76 | 0.57 | 0.65 | 0.88 | 0.88 | 0.87 | 0.78
XLM-R | 0.89 | 0.98 | 0.93 | 0.79 | 0.40 | 0.53 | 0.88 | 0.88 | 0.86 | 0.74
BERT-m | 0.88 | 0.97 | 0.92 | 0.78 | 0.38 | 0.51 | 0.86 | 0.87 | 0.85 | 0.72

Table 2: Results for offensive language detection with default settings on
Transformers. For each model, Precision (P), Recall (R), and F1 are reported
on all classes, and weighted averages. Macro-F1 is also listed. TL indicated
the Transfer Learning experiments

The self ensemble methods were experimented using all the transformer models
and obtained results are summarised in Tables 3 and 4. In most experiments,
ASE has given a higher F1 than MSE and it improved the results over the
default settings. With that fine tuning strategy too XLM-R with transfer
learning outperformed all the other models.

The language modeling fine tuning strategy were experimented using all the
transformer models and obtained results are summarised in Table 5. These
experimented were done on top of ASE fine tuning strategy since it provided
better results than the default settings. Results show that language modeling
clearly improved the results. In fact, the best result from our experiments
were shown when XLM-R model with transfer learning fine tuned with ASE and
language modeling.

| Non Hate Offensive | Hate Offensive | Weighted Average |
---|---|---|---|---
Model | P | R | F1 | P | R | F1 | P | R | F1 | F1 Macro
XLM-R (TL) | 0.91 | 0.96 | 0.94 | 0.77 | 0.59 | 0.67 | 0.89 | 0.89 | 0.89 | 0.80
BERT-m (TL) | 0.90 | 0.95 | 0.93 | 0.76 | 0.57 | 0.65 | 0.88 | 0.88 | 0.87 | 0.78
XLM-R | 0.89 | 0.98 | 0.93 | 0.79 | 0.42 | 0.55 | 0.88 | 0.88 | 0.86 | 0.76
BERT-m | 0.88 | 0.97 | 0.92 | 0.78 | 0.40 | 0.53 | 0.87 | 0.87 | 0.85 | 0.74

Table 3: Results for offensive language detection with MSE on Transformers.
For each model, Precision (P), Recall (R), and F1 are reported on all classes,
and weighted averages. Macro-F1 is also listed. TL indicated the Transfer
Learning experiments

| Non Hate Offensive | Hate Offensive | Weighted Average |
---|---|---|---|---
Model | P | R | F1 | P | R | F1 | P | R | F1 | F1 Macro
XLM-R (TL) | 0.92 | 0.97 | 0.95 | 0.78 | 0.60 | 0.68 | 0.90 | 0.90 | 0.90 | 0.81
BERT-m (TL) | 0.91 | 0.96 | 0.94 | 0.77 | 0.58 | 0.66 | 0.89 | 0.89 | 0.88 | 0.79
XLM-R | 0.90 | 0.99 | 0.94 | 0.80 | 0.43 | 0.56 | 0.89 | 0.89 | 0.87 | 0.77
BERT-m | 0.89 | 0.98 | 0.93 | 0.79 | 0.41 | 0.54 | 0.88 | 0.88 | 0.86 | 0.75

Table 4: Results for offensive language detection with ASE on Transformers.
For each model, Precision (P), Recall (R), and F1 are reported on all classes,
and weighted averages. Macro-F1 is also listed. TL indicated the Transfer
Learning experiments

| Non Hate Offensive | Hate Offensive | Weighted Average |
---|---|---|---|---
Model | P | R | F1 | P | R | F1 | P | R | F1 | F1 Macro
XLM-R (TL) | 0.94 | 0.99 | 0.97 | 0.82 | 0.63 | 0.69 | 0.93 | 0.93 | 0.93 | 0.85
BERT-m (TL) | 0.92 | 0.97 | 0.95 | 0.78 | 0.59 | 0.67 | 0.91 | 0.91 | 0.91 | 0.82
XLM-R | 0.91 | 0.99 | 0.95 | 0.81 | 0.44 | 0.56 | 0.90 | 0.90 | 0.88 | 0.78
BERT-m | 0.890 | 0.99 | 0.94 | 0.80 | 0.42 | 0.55 | 0.89 | 0.89 | 0.87 | 0.76

Table 5: Results for offensive language detection with ASE and Language
Modeling on Transformers. For each model, Precision (P), Recall (R), and F1
are reported on all classes, and weighted averages. Macro-F1 is also listed.
TL indicated the Transfer Learning experiments

###  4.1 Submission Results

Considering the evaluation results on the validation set, we selected the
fine-tuned XLM-R(TL) model with ASE + language modeling as our official
submission to the HASOC task. According to the results provided by the
organisers, our best model has scored 0.89 weighted average F1 score on the
test set and ranked 5th out of 12 participants.

##  5 Analysis

In addition to the experiments described in this paper, we carried out a
qualitative analysis on the dataset to find interesting patterns and
observations. In the training data out of 3,200 tweets only 567 were labelled
offensive and the remaining 2,633 were labelled as not-offensive. The use of
English words were minimal although there are many tweets which are in
Malayalam language but written in Roman script. When analysing the tweets
labelled as offensive, we observed that there are many tweets in the dataset
which are actually not-offensive but labelled as offensive. Free English
translations of some examples include: Spent 4 years proclaiming to be a Royal
Mech. There are 25k dislikes from Ikka (Mammooty) fans, you are free to unlike
and cry. Nice, looks like a TV drama series from SuryaTV (a Malayalam
channel). Have you no shame defaming a reputed hospital?

We observed that between 20% and 25% of the tweets which are labelled as
offensive are similar to the example shown above which has certainly impacted
the performance of the models.

##  6 Conclusion

In this paper we have presented the system submitted by the WLV-RIT team to
the HASOC 2020 - Offensive Language Identification - Dravidian Code Mix Task 1
at FIRE 2020. Following a recent study [16], we have shown that the XLM-R with
transfer learning is the most successful transformer model from several
transformer models we experimented. It should be noted that the traditional
machine learning models comes very close to the performance of the transformer
models. We have shown that the best traditional machine learning algorithm we
experimented; Random Forest outperforms the majority of our transformer model
based experiments. This can be due to properties of the dataset or due to the
fact that a low-resource language like Malayalam is under represented in
multilingual pre-trained models. With several fine tuning strategies, XLM-R
with transfer learning provides the best result for the validation set.
Finally, our approach achieved 5th place in the leaderboard for the test set.

## Acknowledgements

We would like to thank the HASOC organizers for running this interesting
shared task and for replying promptly to all our inquiries. We further thank
the anonymous reviewers for their insightful feedback.

## References

  * Bauman et al. [2013] S. Bauman, R. B. Toomey, J. L. Walker,  Associations among bullying, cyberbullying, and suicide in high school students,  Journal of adolescence 36 (2013) 341–350.
  * Kumar et al. [2020] R. Kumar, A. K. Ojha, S. Malmasi, M. Zampieri,  Evaluating aggression identification in social media,  in: Proceedings of TRAC, 2020.
  * Pitenis et al. [2020] Z. Pitenis, M. Zampieri, T. Ranasinghe,  Offensive Language Identification in Greek,  in: Proceedings of LREC, 2020.
  * Risch and Krestel [2018] J. Risch, R. Krestel,  Delete or not Delete? Semi-Automatic Comment Moderation for the Newsroom,  in: Proceedings of TRAC, 2018.
  * Nobata et al. [2016] C. Nobata, J. Tetreault, A. Thomas, Y. Mehdad, Y. Chang,  Abusive language detection in online user content,  in: Proceedings of WWW, 2016.
  * Founta et al. [2018] A. M. Founta, C. Djouvas, D. Chatzakou, I. Leontiadis, J. Blackburn, G. Stringhini, A. Vakali, M. Sirivianos, N. Kourtellis,  Large scale Crowdsourcing and Characterization of Twitter Abusive Behavior,  in: Proceedings of ICWSM, 2018.
  * Kumar et al. [2018] R. Kumar, A. K. Ojha, S. Malmasi, M. Zampieri,  Benchmarking Aggression Identification in Social Media,  in: Proceedings of TRAC, 2018.
  * Chelmis et al. [2017] C. Chelmis, D.-S. Zois, M. Yao,  Mining patterns of cyberbullying on twitter,  in: Proceedings of ICDMW, 2017.
  * Yao et al. [2018] M. Yao, C. Chelmis, D.-S. Zois,  Cyberbullying detection on instagram with optimal online feature selection,  in: Proceedings of ASONAM, 2018.
  * Malmasi and Zampieri [2017] S. Malmasi, M. Zampieri,  Detecting Hate Speech in Social Media,  in: Proceedings of the International Conference Recent Advances in Natural Language Processing (RANLP), 2017, pp. 467–472.
  * Mathew et al. [2019] B. Mathew, A. Illendula, P. Saha, S. Sarkar, P. Goyal, A. Mukherjee,  Temporal effects of unmoderated hate speech in gab,  arXiv preprint arXiv:1909.10966 (2019).
  * Basile et al. [2019] V. Basile, C. Bosco, E. Fersini, D. Nozza, V. Patti, F. M. R. Pardo, P. Rosso, M. Sanguinetti,  Semeval-2019 task 5: Multilingual detection of hate speech against immigrants and women in twitter,  in: Proceedings of SemEval, 2019\.
  * Zampieri et al. [2019] M. Zampieri, S. Malmasi, P. Nakov, S. Rosenthal, N. Farra, R. Kumar,  SemEval-2019 Task 6: Identifying and Categorizing Offensive Language in Social Media (OffensEval),  in: Proceedings of SemEval, 2019\.
  * Zampieri et al. [2020] M. Zampieri, P. Nakov, S. Rosenthal, P. Atanasova, G. Karadzhov, H. Mubarak, L. Derczynski, Z. Pitenis, Ç. Çöltekin,  Semeval-2020 task 12: Multilingual offensive language identification in social media (offenseval 2020),  in: Proceedings of SemEval, 2020\.
  * Devlin et al. [2019] J. Devlin, M.-W. Chang, K. Lee, K. Toutanova,  Bert: Pre-training of deep bidirectional transformers for language understanding,  in: Proceedings of NAACL, 2019.
  * Ranasinghe and Zampieri [2020] T. Ranasinghe, M. Zampieri,  Multilingual offensive language identification with cross-lingual embeddings,  in: Proceedings of EMNLP, 2020.
  * Chakravarthi et al. [2020] B. R. Chakravarthi, M. A. Kumar, J. P. McCrae, P. B, S. KP, T. Mandl,  Overview of the track on "hasoc-offensive language identification- dravidiancodemix",  in: Proceedings of FIRE, 2020.
  * Zampieri et al. [2019] M. Zampieri, S. Malmasi, P. Nakov, S. Rosenthal, N. Farra, R. Kumar,  Predicting the Type and Target of Offensive Posts in Social Media,  in: Proceedings of NAACL, 2019.
  * Ranasinghe and Hettiarachchi [2020] T. Ranasinghe, H. Hettiarachchi,  BRUMS at SemEval-2020 Task 12 : Transformer based Multilingual Offensive Language Identification in Social Media,  in: Proceedings of SemEval, 2020\.
  * Ranasinghe et al. [2019] T. Ranasinghe, M. Zampieri, H. Hettiarachchi,  BRUMS at HASOC 2019: Deep Learning Models for Multilingual Hate Speech and Offensive Language Identification,  In Proceedings of FIRE (2019).
  * Hettiarachchi and Ranasinghe [2020] H. Hettiarachchi, T. Ranasinghe,  Brums at semeval-2020 task 3: Contextualised embeddings for predicting the (graded) effect of context in word similarity,  in: Proceedings of SemEval, 2020\.
  * Liang et al. [2020] C. Liang, Y. Yu, H. Jiang, S. Er, R. Wang, T. Zhao, C. Zhang,  Bond: Bert-assisted open-domain named entity recognition with distant supervision,  in: Proceedings of SIGKDD, 2020.
  * Yang et al. [2019] W. Yang, Y. Xie, A. Lin, X. Li, L. Tan, K. Xiong, M. Li, J. Lin,  End-to-end open-domain question answering with BERTserini,  in: Proceedings of NAACL, 2019.
  * Kibriya et al. [2004] A. M. Kibriya, E. Frank, B. Pfahringer, G. Holmes,  Multinomial naive bayes for text categorization revisited,  in: Australasian Joint Conference on Artificial Intelligence, Springer, 2004, pp. 488–499.
  * Cortes and Vapnik [1995] C. Cortes, V. Vapnik,  Support-vector networks,  Machine learning 20 (1995) 273–297.
  * Liaw et al. [2002] A. Liaw, M. Wiener, et al.,  Classification and regression by randomforest,  R news 2 (2002) 18–22.
  * Pedregosa et al. [2011] F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, et al.,  Scikit-learn: Machine learning in python,  the Journal of machine Learning research 12 (2011) 2825–2830.
  * Bird et al. [2009] S. Bird, E. Klein, E. Loper, Natural language processing with Python: analyzing text with the natural language toolkit, " O’Reilly Media, Inc.", 2009.
  * Sun et al. [2019] C. Sun, X. Qiu, Y. Xu, X. Huang,  How to fine-tune bert for text classification?,  in: M. Sun, X. Huang, H. Ji, Z. Liu, Y. Liu (Eds.), Chinese Computational Linguistics, Springer International Publishing, 2019, pp. 194–206.
  * Conneau et al. [2019] A. Conneau, K. Khandelwal, N. Goyal, V. Chaudhary, G. Wenzek, F. Guzmán, E. Grave, M. Ott, L. Zettlemoyer, V. Stoyanov,  Unsupervised cross-lingual representation learning at scale,  arXiv preprint arXiv:1911.02116 (2019).
  * Wolf et al. [2019] T. Wolf, L. Debut, V. Sanh, J. Chaumond, C. Delangue, A. Moi, P. Cistac, T. Rault, R. Louf, M. Funtowicz, J. Brew,  Huggingface’s transformers: State-of-the-art natural language processing,  ArXiv abs/1910.03771 (2019).
  * Zampieri et al. [2019] M. Zampieri, S. Malmasi, P. Nakov, S. Rosenthal, N. Farra, R. Kumar,  Predicting the type and target of offensive posts in social media,  in: Proceedings of NAACL, 2019.
  * Hettiarachchi and Ranasinghe [2019] H. Hettiarachchi, T. Ranasinghe,  Emoji powered capsule network to detect type and target of offensive posts in social media,  in: Proceedings of RANLP, 2019.
  * Hettiarachchi and Ranasinghe [2020] H. Hettiarachchi, T. Ranasinghe,  Infominer at wnut-2020 task 2: Transformer-based covid-19 informative tweet extraction,  in: Proceedings of W-NUT, 2020.
  * Xu et al. [2020] Y. Xu, X. Qiu, L. Zhou, X. Huang,  Improving bert fine-tuning via self-ensemble and self-distillation,  arXiv preprint arXiv:2002.10345 (2020).