# Emergent SU(8) Dirac semimetal and novel proximate phases of spin-orbit
coupled fermions on a honeycomb lattice

Basudeb Mondal<EMAIL_ADDRESS>International Centre for
Theoretical Sciences, Tata Institute of Fundamental Research, Bengaluru
560089, India.  Vijay B. Shenoy<EMAIL_ADDRESS>Centre for Condensed Matter
Theory, Department of Physics, Indian Institute of Science, Bengaluru 560012,
India.  Subhro Bhattacharjee<EMAIL_ADDRESS>International Centre for
Theoretical Sciences, Tata Institute of Fundamental Research, Bengaluru
560089, India.

###### Abstract

Emergent Dirac fermions provide the starting point for understanding the
plethora of novel condensed matter phases. The nature of the associated phases
and phase transitions crucially depends on both the emergent symmetries as
well as the implementation of the microscopic ones on the low-energy Dirac
fermions. Here, we show that $j=3/2$ electrons in spin-orbit coupled materials
on honeycomb lattice can give rise to SU(8) symmetric Dirac semimetals with
symmetry implementation very different from that of graphene. This non-trivial
embedding of the microscopic symmetries in the low energy is reflected in the
nature of phases proximate to the Dirac semimetal. Such phases can arise from
finite short-range electron-electron interactions. In particular, we identify
24 such phases – divided into three classes – and their low energy properties
obtained by condensing particle-number conserving fermion bilinears that break
very different microscopic symmetries and/or are topologically protected by
symmetries. The latter includes interesting generalisations of quantum spin-
Hall phases. Remarkably some of the resultant phases still support a sub-set
of gapless fermions– protected by a sub-group of SU(8) – resulting in
interesting density wave semimetals. Near the phase transitions to such
density wave semimetals, the surviving gapless fermions strongly interact with
the bosonic order parameter field and give rise to novel quantum critical
points. Our study is applicable to a wide class of $d^{1}$ and $d^{3}$
transition metals with strong spin-orbit coupling and predicts that such
materials can harbour a very rich interplay of symmetries and competing
interactions in the intermediate correlation regime.

##  I Introduction

Massless Dirac fermions arise in a variety of condensed matter systems Vafek
and Vishwanath (2014); Goerbig and Montambaux (2017). Perhaps the most well-
known is the recently studied – both experimentally and theoretically –
example of monolayer graphene Wallace (1947); Novoselov  _et al._ (2005);
Zhang  _et al._ (2005); Geim (2009); Castro Neto  _et al._ (2009); Goerbig
(2011); Das Sarma  _et al._ (2011); Novoselov (2011); Geim (2011) where such
Dirac fermions arise as a low energy limit of electrons hopping on the
honeycomb lattice. More generally such Dirac fermions may arise in a variety
of other two and three dimensional lattices Wehling  _et al._ (2014); Vafek
and Vishwanath (2014); Young and Kane (2015); Hirata  _et al._ (2021) relevant
for several materials including organic semiconductors like $\alpha$-(BEDT-
TTF)2I3 Kobayashi  _et al._ (2007); Goerbig  _et al._ (2008); Hirata  _et al._
(2021), the d${}_{x^{2}-y^{2}}$-wave superconductor in cuprates Wollman  _et
al._ (1995); Kirtley  _et al._ (1995); Volovik (1993); Wang and MacDonald
(1995); Balents  _et al._ (1998), Dirac and Weyl semimetals Wan  _et al._
(2011); Young  _et al._ (2012) and surface of 3D topological insulators Moore
and Balents (2007); Roy (2009); Fu  _et al._ (2007); Hasan and Kane (2010); Qi
and Zhang (2011); Chen  _et al._ (2009). These low energy Dirac fermions have
indelible signatures in a plethora of low energy experiments of these
candidate materials as is evident in the integer quantum Hall effect Pacile
_et al._ (2008); Zhang  _et al._ (2005) as well as other spectroscopic and
transport properties Li  _et al._ (2009); Nair  _et al._ (2008); Li  _et al._
(2008); Levy  _et al._ (2010); Das Sarma  _et al._ (2011); Checkelsky  _et
al._ (2009) of monolayer graphene, surface transport of 3D topological
insulators Bardarson and Moore (2013) or spectroscopy of $d$-wave
superconductors Wollman  _et al._ (1995); Kirtley  _et al._ (1995).

Dirac fermions also arise in a somewhat different context as low energy
theories of certain quantum spin liquids (QSL). Indeed in U(1) Dirac QSLs, low
energy fermionic spinons– minimally coupled to an emergent U(1) gauge field–
have free Dirac dispersion with enhanced symmetries at low energies within
parton mean-field theories Hermele  _et al._ (2005). While in this case of
QED3, the fluctuations of the gauge field ultimately lead to the destruction
of the quasi-particles Affleck and Marston (1988); Marston and Affleck (1989);
Rantner and Wen (2001); Wen (2002); Hermele  _et al._ (2004, 2005); Franz and
Tešanović (2001); Rantner and Wen (2002); Hermele  _et al._ (2005), the
proximate ordered phases can be obtained by condensing appropriate spinon
bilinears that gap out the Dirac spinons. Also, such effective Dirac theories,
with or without dynamic gauge fields have also been recently discussed in
context of $(2+1)$ dimensional fermionic particle-vortex dualities emanating
out of conjectures of Dirac composite fermions in half-filled Landau level in
quantum Hall systems Son (2015).

An equally important question pertains to the nature of the different phases
obtained Boyack  _et al._ (2021) upon gapping out the Dirac fermions via
short-ranged four-fermion interactions/other bosonic fields or via external
perturbations such as originating from substrate effects in graphene Zhou  _et
al._ (2007). For the former, a typical effect of such interactions is to
condense a fermion-bilinear that dynamically generates mass for the gapless
Dirac fermions for a finite strength of the interaction. The nature of the
resultant gapped phases 111This phase can have other gapless mode such as
Goldstone boson if a continuous symmetry is broken. Boyack  _et al._ (2021);
Ryu  _et al._ (2009); Herbut  _et al._ (2009a); Tanaka and Hu (2005); Hermele
_et al._ (2005) as well as the theory of the associated phase transition from
the proximate Dirac semimetal via Gross-Neveu-Yukawa Gross and Neveu (1974);
Zinn-Justin (1991); Sachdev (1999); Boyack  _et al._ (2021); Herbut (2006)
field theories have received considerable attention in a wide array of
condensed matter settings and allows for systematic understanding of novel
quantum phase transitions including Landau forbidden deconfined quantum
criticality Senthil  _et al._ (2004); Senthil and Fisher (2006).

A central aspect of the above diverse physics is the implementation of
microscopic symmetries on the low-energy Dirac fermions. These ultraviolet
(UV) symmetries typically consist of lattice symmetries, time reversal as well
as possible spin-rotation symmetries of the electrons occupying the underlying
atomic orbitals. Material-dependent microscopic energetics allow for different
implementations of these UV symmetries on the underlying low energy Dirac
fermions opening up avenues to probe the Dirac semimetal Das Sarma  _et al._
(2011) as well as stabilize novel proximate phases. In regards to the latter,
masses that preserve microscopic symmetries, but gap out the fermions, have
recently provided crucial insights in the development of the theory of
symmetry-protected topological phases (SPTs) Kane and Mele (2005); Haldane
(1988). In a large class of fermionic SPTs, this has been made possible via
the interplay of atomic spin-orbit coupling Witczak-Krempa  _et al._ (2013)
and electron correlations that allow for such non-trivial implementation of
usual condensed matter symmetries on the low energy degrees of freedom.
Central to our interest here is the possibility of realising a new class of
spin-orbitally coupled Dirac fermions such that the UV symmetries are
implemented in a novel way allowing for new material platforms for the
interplay of symmetries and interactions. What, then, are the nature of the
gapped phases in these spin-orbit coupled two-dimensional Dirac materials?

In this paper, we present a new material-relevant platform for realizing spin-
orbit (SO)-assisted SU(8) Dirac fermions in two spatial dimensions and discuss
a plethora of rich phases proximate to such a Dirac semimetal. Somewhat
counterintuitively, the underlying microscopics involving strong spin-orbit
coupling (SOC), octahedral crystal field effect on $d$-orbitals, and hopping
pathways can lead to a large symmetry enhancement resulting in gapless Dirac
semimetal for non-interacting electrons with an emergent SU(8) symmetry at low
energy or infrared (IR). This can be realized, for example, in systems
containing transition metal ions with strong SOC in $d^{1}$ electronic
configuration on a honeycomb lattice in an edge-sharing octahedral motif (see
Fig. 1), which leads to a quarter filled $j=3/2$ atomic orbitals as low energy
electronic degree of freedom. This lattice is fairly common in the context of
honeycomb Iridates as well as $\alpha$-RuCl3 where the transition metal ion
has a $d^{5}$ configuration leading to $j=1/2$ moments with possible relevance
to Kitaev QSLs. A much more relevant material in the same class, for the
present purpose, is ZrCl3 where Zr3+ is in a $4d^{1}$ configuration. Our
calculations, however, are quite general and show that generally interesting
low-energy physics can emerge in a relevant parameter regime for $d^{1}$
systems and its particle-hole conjugate $d^{3}$, which we expect will be
synthesized in the near future.

The low energy SU(8) Dirac fermions – described by a free Dirac action of
$N_{F}=4$ flavours of $4$-component Dirac fermions (Eq. 1)– obtained here
fundamentally differ from a large flavour/spin generalisation of graphene in
terms of the implementation of lattice transformations which, due to strong
SOC, is intertwined with the spin-rotations. This is reflected in the low
energy theory via the transformation of the Dirac modes under various lattice
symmetries. A direct fallout of this non-trivial symmetry implementation is
observed in the phases that are proximate to the SU(8) Dirac semimetal and can
be obtained from it for finite electron-electron interactions and/or
additional hopping perturbations via breaking of the SU(8) symmetry by
condensing various fermion bilinears. In this work, we study the 64 particle-
hole bilinears (i.e. bilinears that do not carry a net electronic charge)
consisting of the SU(8) singlet and the 63 dimensional adjoint multiplet. This
adjoint multiplet is further broken down into singlets, doublets, and triplets
by the UV symmetry group and corresponds to different ordered phases. For the
present symmetry realisation we find that the 64 bilinears result in 24
different phases proximate to the SU(8) Dirac semimetal.

Typically condensation of the bilinears gaps out the single-fermion spectrum
resulting in a broken symmetry phase with a single-fermion gap. We show
several examples of such gapped broken symmetry insulators, typically with
different spin/charge modulations. In addition, this work shows interesting
instances where the condensate leaves intact a sub-set of the gapless Dirac
fermions which describe various types of spin density wave semimetals.
Notably, while the bilinears necessarily break the IR symmetry of the free
Dirac theory– the SU(8) and parity or time reversal, they may still be
invariant under all or some microscopic/UV symmetry transformations. Indeed,
the classification of such masses that are allowed by microscopic symmetries
and gap out the fermions, results in SPTs with gapped fermionic spectrum in
the bulk and gapless edges whose low energy bulk theory is given by various
Chern-Simons actions that characterise the appropriate quantized Hall response
of such insulators. In the present context, we find several such interaction-
driven SPTs, variously dubbed as topological Mott insulators Raghu  _et al._
(2008); Grover and Senthil (2008) both in the presence and absence of time
reversal symmetry. These include interesting generalisations of the quantum
spin-Hall phases as well as newer ones such as the quantum spin-quadrupole
Hall phase, with the latter being odd under time reversal, unlike the former.
A more subtle version realised in the present case includes a ferro spin-
quadrupole ordered insulator with spin-octupole filtered edge currents (Sec.
VIII.1.1) in addition to a ferro spin-octupole ordered insulator with
anomalous charge quantum Hall effect (Sec. VIII.1.2). These anomalous
fermionic masses, along with the conventional (microscopic) symmetry-breaking
ones – made possible by non-trivial implementation of UV symmetries on the
Dirac fermions – provide a rich phase diagram for the phases and associated
phase transitions proximate to the SO coupled SU(8) Dirac semimetal.

The classification of the bilinears naturally opens the floodgate of questions
regarding the nature of concomitant quantum transitions primarily out of the
Dirac semimetal into one of the 24 symmetry-broken phases. As mentioned above,
these transitions occur at a finite value of the short-range four fermion
interactions (Eq. 43). For continuous phase transitions, the resultant Gross-
Neveu-Yukawa type critical theories are obtained by decoupling the four-
fermion interaction along a particular bilinear channel resulting in coupling
between the Dirac fermions with the bosonic order parameter corresponding to
the fermion bilinear which, in turn, gain dynamics on integrating out the
higher energy fermions Herbut  _et al._ (2009b); Vafek and Vishwanath (2014).
The properties of such critical points when the fermions are gapped out across
the transition can be understood within various renormalisation group schemes
such as $3-\epsilon$ expansions Herbut  _et al._ (2009b) or Large $N_{f}$
expansions Herbut (2006). We shall encounter several possible examples of such
transitions and mention some of the expected fallouts in respective places
while the details will be taken up elsewhere. A particular class of transition
worth mentioning involves the possibility of gapping out only a subset of
fermions across the transition, typically resulting in density-wave semimetals
of various types. Such transitions between two different semimetals serve as
examples of (semi)metallic criticality, which have been of recent interest
Seifert  _et al._ (2020); Liu  _et al._ (2022).

A more subtle structure in the phase diagram arises in the form of unnecessary
phase transitions Bi and Senthil (2019) or more precisely, unnecessary multi-
critical points. These are observed in two or more lattice triplets whose
components are made up of incompatible mass matrices (i.e., they do not
mutually anticommute). For such a triplet, isolated points of gaplessness (See
Fig. 8) occur on a $2$-sphere denoting the mass manifold for the triplet.
These isolated gapless points denote unnecessary multi-critical points since
any two generic gapped points on this sphere can also be connected entirely by
avoiding these gapless isolated points and hence avoiding the transition
altogether. Such unnecessary multi-critical points can be understood as a
fallout of the particular embedding of the microscopic symmetries in the
emergent SU(8) (see Fig. 9).

Finally, the bilinears corresponding to different order parameters may carry
fermionic modes at their topological defects Abanov and Wiegmann (2000);
Grover and Senthil (2008); Ryu  _et al._ (2009). The simplest is the gapless
chiral edge fermions associated with the domain walls of the Integer Chern
insulator or the anomalous Hall insulator. These lead to Chern-Simons terms in
the action once the fermions are integrated out in the gapped phase and
account for the gapless edge modes. In addition, we also find a slew of
generalised Quantum spin-Hall insulators with edge/domain walls carrying
gapless fermion modes captured by mutual Chern-Simons terms. The above
analysis is easily extended to other topological defects including vortices
and skyrmions. In particular, for the Skyrmions of a triplet quantum spin-
octupole order parameter, we find the corresponding skyrmions are bosonic and
carry four units of electronic charge. Thus, condensing such skyrmions
naturally gives rise to a charge-$4e$ superconductor.

Considering the length and the number of results that we present in this work,
we start with an overview of the results that summarises the work and helps
the reader navigate the text.

Figure 1: (a) Splitting of the $d$-orbitals due to crystal field and SOC. (b)
Edge-sharing octahedra forming a honeycomb lattice. As elaborated in the main
text and in Appendix A.2, the honeycomb lattice lies in the [111] plane of the
Cartesian coordinate system whose projections are denoted by $X,Y$, and $Z$.

##  II Overview of the results

The focus of our work is crystalline systems on a honeycomb lattice formed out
of edge-sharing octahedra (Fig. 1) where the electronically active transition
metal ions, with strong atomic SOC, sit at the centres of such octahedra. Such
structures are quite common and occur in a several stacked SOC magnets of
recent interest such as the honeycomb Iridates A2IrO3 (A=Na, Li) Singh and
Gegenwart (2010); Liu  _et al._ (2011); Singh  _et al._ (2012), and ruthenates
$\alpha$-RuCl3 Banerjee  _et al._ (2016); Sandilands  _et al._ (2015); Plumb
_et al._ (2014); Choi  _et al._ (2012). In this work, we focus on such
honeycomb system where the $j=3/2$ electronic orbitals (Fig. 1) form the low
energy manifold and in particular with a single electron ($d^{1}$
configuration) in the $j=3/2$ orbitals. An example of such a situation is
$\alpha$-ZrCl3 Swaroop and Flengas (1964a, b); Yamada  _et al._ (2018) where
Cl- forms the edge sharing octahedral network with a Zr3+– in $d^{1}$
configuration – sits and gives rise to a quarter filled $j=3/2$ system. While
the fate of the low energy electronic phase of ZrCl3 is still to be settled
Yamada  _et al._ (2018); Ushakov  _et al._ (2020), we expect that a large
number of such materials exist whose physics is governed by various regimes of
electronic correlations. Our analysis will be applicable to the whole class of
such materials with $j=3/2$ orbitals in $d^{1}$ (and $d^{3}$, by particle-hole
symmetry) configuration.

###  II.1 Spin-orbit coupled SU(8) Dirac semimetal

The material set-up consists of the honeycomb system with electrons occupying
the $j=3/2$ atomic orbitals (Eq. 6) and at quarter filling (i.e., $d^{1}$
configuration) as detailed in Section III. The hopping Hamiltonian accounting
for the electron hopping via the ligands that form the octahedra (Eq. 9),
introduced in Ref. Yamada  _et al._ (2018), is – in an appropriately rotated
local basis (Eq. 17)– nothing but four copies of nearest neighbour hopping
model on a honeycomb lattice in $\pi$-flux at $1/4$th filling (Eq. 18). The
four copies stem from the four $j=3/2$ atomic orbitals, although the symmetry
is enhanced to SU(4). On solving this Hamiltonian we obtain two valleys (Eq.
23) of Dirac band-touching at the $1/4$th filling (Fig. 4). Note that these
Dirac points, unlike graphene, are not at the Brillouin zone (BZ) corners, but
at locations inside the magnetic BZ as shown in Fig. 4 with momenta given by
Eq. 23.

The low energy theory takes the canonical Dirac form with the (Euclidean)
Lagrangian density Kaplan (2009)

$\displaystyle\mathcal{L}_{0}=v_{F}\sum_{f=1}^{N_{F}}\bar{\chi}_{f}({\bf
r})(-i\not{\partial})\chi_{f}({\bf r})$ (1)

and the corresponding Hamiltonian is given by Eq. 25. Here,
$\not{\partial}\equiv\sum_{i=0}^{2}\gamma_{i}\partial_{i}$ with
$\gamma^{0},\gamma^{1}$ and $\gamma^{2}$ being $4\times 4$ matrices (see Eq.
29) that generate an Euclidean Clifford algebra, i.e.,
$\\{\gamma_{\mu},\gamma_{\nu}\\}=2\delta_{\mu\nu}$ and $v_{F}$ is the velocity
of the Dirac fermions. $\chi_{f}$ denotes $4-$component Dirac fermions (Eq.
26) with $N_{F}=4$ flavours (i.e., $f=1,2,\cdots,N_{F}$) with
$\bar{\chi}_{f}=i\chi^{\dagger}_{f}\gamma_{0}$. The four components of each
$\chi_{f}$ are given by

$\displaystyle\chi_{f\sigma\tau}$ (2)

where $\sigma=1,2$ corresponds to the two bands that touch at the Dirac points
and $\tau=\pm$ are the two Dirac valleys/nodes.

The free Dirac theory (Eq. 1) has a very large internal SU(2$N_{F}$) symmetry
in addition to the usual symmetries of the Dirac theory– emergent Lorentz
transformation and continuous translations as well as discrete charge
conjugation, parity (reflection in two dimension Semenoff (1984)) and time
reversal. In the present case, the emergent SU(8) symmetry is a combination of
the SU(4) flavour symmetry of the hopping model and the emergent chiral SU(2)
symmetry with the latter being similar to spinless graphene Ryu  _et al._
(2009).

This IR symmetry is, of course, emergent and much larger than the
microscopic/UV symmetries of the lattice theory. In various candidate
materials, different energy scales of the atomic orbitals provide for
different microscopic symmetry groups which are then embedded within the
larger IR symmetry group of the Dirac theory. Indeed, in the present case, the
strong SOC resulting in the $j=3/2$ atomic orbitals result in an
implementation of the honeycomb lattice symmetries on the resultant Dirac
fermions, distinct from graphene. In particular, in the present case, the
effect of SOC is manifested in the symmetry transformation of the $\chi$
spinors via mixing of the SU(4) flavours under lattice transformation (Table 5
and Eq. 42). This difference in embedding, coupled with the difference in the
location of the Dirac points, reflects in the properties of the present SO-
coupled Dirac semimetal and its proximate phases that are much richer than a
larger $N_{F}$ flavour mono-layer graphene.

###  II.2 Phases proximate to the SU(8) Dirac semimetal

The emergent SU(8) and relativistic invariance of the low energy theory
indicates that a large number of correlation functions decay identically at
long distances. These correlation functions correspond to a wide set of very
different phases as far as the microscopic symmetry breaking is concerned
Hermele  _et al._ (2005) and adding four-fermion interactions can lead to
spontaneously symmetry broken and/or symmetry-protected topological phases
proximate to the Dirac semimetal by favouring one of the channels.

The Dirac semimetal is perturbatively stable to short-range four-fermion
interactions (Eq. 43). However, as the strength of the four-fermion term is
increased it can favour the condensation of a particle-hole fermion bilinear
(Eq. 44)

$\displaystyle\langle\bar{\chi}_{f\sigma\tau}\chi_{f^{\prime}\sigma^{\prime}\tau^{\prime}}\rangle\neq
0$ (3)

that breaks the SU(8) symmetry spontaneously. There are 64 such bilinears
divided into a SU(8) scalar

$\displaystyle-i\langle\bar{\chi}\chi\rangle$ (4)

and the 63 fold adjoint multiplet of SU(8) (also given in Eq. 46)

$\displaystyle-i\langle\bar{\chi}\mathcal{P}_{a}\chi\rangle,~{}~{}~{}~{}~{}a=1,\cdots,63.$
(5)

and $\mathcal{P}_{a}$ being the SU(8) generators (Eq. 31) obtained by
combining the 15 generators of flavour SU(4), $\Sigma_{i}$ (defined below Eq.
10), and 3 generators of chiral SU(2), $\zeta_{j}$ (Eq. 30), along with their
respective identities.

Mean-field decomposition (exact in the limit $N_{F}\rightarrow\infty$ Moshe
and Zinn-Justin (2003); Sachdev (1999)) of the four-fermion interactions into
the bilinears (Eq. 45) generically lead to fully gapping of the Dirac fermions
resulting in a SU(8) symmetry broken insulator. However, we also find
situations where only a subset of fermions are gapped out leading, typically
to a density wave semimetal with a lower (than SU(8)) symmetry (see below).

Table 1: The SU(4) invariant chiral phases. The microscopic symmetry elements mentioned in this and the subsequent tables are defined in Table 5. Also, the $\zeta_{i}$ are generators of the chiral SU(2) and $\Sigma_{i}$ are generators of flavor SU(4) which are defined in Eq. 30 and Appendix C respectively. # | The Phase | Irrep | Broken | Fermion | Comments
---|---|---|---|---|---
|  |  | Microscopic | Bilinear |
|  |  | symmetries |  |
1. | Integer Chern | $\mathcal{A}^{o}_{2g}$ |  $\mathbf{C_{2}^{\prime}}$, $\mathbf{\sigma_{d}}$, TR | $-i\langle\bar{\chi}\chi\rangle$ | Fully gapped.
| insulator (ICI). |  |  |  | Charge Hall response
| (Sec. VI.1) |  |  |  | given by $N_{F}=4$
|  |  |  |  | CS theory (Eq. 73)
2. | Stripy | $\mathcal{T}^{e}_{1g}$ | $\mathbf{T_{1}},\mathbf{T_{2}},$ | $-i\langle\bar{\chi}\zeta_{1}\chi\rangle,$ | Fully gapped.
| charge density |  | $\mathbf{C_{3},S_{6},C_{2}^{\prime},\sigma_{d}}$ | $-i\langle\bar{\chi}\zeta_{2}\chi\rangle,$ | Stripy modulation
| wave (StCDW). |  |  | $-i\langle\bar{\chi}\zeta_{3}\chi\rangle$ | of electronic charge
| (Sec. VI.2) |  |  |  | density (Fig. 6)

A natural question, then, pertains to the nature of such phases obtained by
fermion-bilinear condensation that lie proximate to the Dirac semimetal. To
answer this question, the actual embedding of the microscopic/UV symmetries in
the emergent low-energy symmetries becomes important. We find that the 64
masses, break up into 27 irreducible representations representing 24 different
phases (Eqs. 51a-65f). The microscopic symmetries only allow for singlet,
doublet and triple irreducible representations (Irreps), and the 24 phases are
made up of six singlets, five doublets, and sixteen triplets.

The 24 phases can be subdivided into three groups depending on the
participation of the SU(4) flavour and the SU(2) chiral sectors in the fermion
bilinear (Eq. 44) which dictate their transformation properties under the
microscopic symmetries. These are :

Group-1 : Chiral masses : The chiral masses (Eqs. 51a-51b) are composed of
SU(4) flavour singlets. These four bilinears have a structure of
$-i\langle\bar{\chi}\zeta_{j}\chi\rangle$ where $j=0,1,2,3$ and we have
included the SU(8) scalar (Eq. 4) corresponding to the case of $\zeta^{0}$.
This singlet corresponds to ($N_{F}=4$) Integer Chern Insulator (ICI) and is a
SU(4) generalisation of the Haldane mass Haldane (1988) as given by Eq. 72.
The other three masses (Eq. 76) form a triplet under lattice symmetries that
correspond to three stripy charge-density-wave insulators (StCDW) (Fig. 6).
Notably, the presence of the $\pi$-flux breaks up the chiral masses as ${\bf
4}={\bf 1}\oplus{\bf 3}$, as opposed to graphene where the four chiral masses
are decomposed as ${\bf 4}={\bf 1}\oplus{\bf 1}\oplus{\bf 2}$ corresponding to
$N_{F}=2$ Integer Chern insulator, staggered charge density waves and two
Kekule orders respectively Ryu  _et al._ (2009). The summary of these four
masses is given in Table 1 and the details are discussed in Section VI.

Table 2: The flavor phases # | The Phase | Irrep | Broken | Fermion | Comments
---|---|---|---|---|---
|  |  | Microscopic | Bilinear |
|  |  | symmetries |  |
3. | Singlet | $\mathcal{A}^{e}_{1g}$ | None | $-i\langle\bar{\chi}\Sigma_{45}\chi\rangle$ | Fully Gapped.
| Quantum |  |  |  | Spin-octupole Hall response in
| spin-octupole |  |  |  | presence of electric field (Eq. 83) via
| Hall insulator. |  |  |  | spin-octupole filtered edges
| (Sec. VII.1.1) |  |  |  | protected by U(1)$\rtimes$ Z${}_{\rm 2}^{TR}$.
4. | Triplet | $\mathcal{T}^{e}_{1g}$ | $\mathbf{T_{1}},\mathbf{T_{2}},$ | $-i\langle\bar{\chi}\Sigma_{23}\chi\rangle,$ | Fully gapped. Spin-octupole
| Quantum |  |  $\mathbf{C_{3}}$, $\mathbf{S_{6}}$, | $-i\langle\bar{\chi}\Sigma_{13}\chi\rangle,$ | filtered edge currents.
| spin-octupole |  |  $\mathbf{C_{2}^{\prime}}$, $\mathbf{\sigma_{d}}$ | $-i\langle\bar{\chi}\Sigma_{12}\chi\rangle$ | The Skyrmion configurations of
| Hall insulator. |  |  |  | the triplet order parameter carry $4$
| (Inversion even) |  |  |  | units of electronic charge. Such
| (Sec. VII.1.2) |  |  |  | skyrmion condensation leads
|  |  |  |  | to a novel $4e$ superconductor.
5. | Triplet | $\mathcal{T}^{e}_{1u}$ | $\mathbf{T_{1}},\mathbf{T_{2}},$ | $-i\langle\bar{\chi}\Sigma_{34}\chi\rangle,$ | Non-compatible masses.
| Quantum |  |  $\mathbf{C_{3}}$, $\mathbf{S_{6}}$, | $-i\langle\bar{\chi}\left(-\frac{1}{2}\Sigma_{14}-\frac{\sqrt{3}}{2}\Sigma_{15}\right)\chi\rangle,$ | Generally fully gapped
| spin-octupole |  | $\mathbf{C_{2}^{\prime}},\mathbf{\sigma_{d}},\mathbf{I}$ | $-i\langle\bar{\chi}\left(-\frac{\sqrt{3}}{2}\Sigma_{25}+\frac{1}{2}\Sigma_{24}\right)\chi\rangle.$ | except for isolated points
| Hall insulators. |  |  |  | protected by symmetries.
| (Inversion odd) |  |  |  | Spin-octupole
| (Sec. VII.1.3) |  |  |  | filtered edge.
6. | Triplet | $\mathcal{T}^{e}_{2u}$ | $\mathbf{T_{1}},\mathbf{T_{2}},$ | $-i\langle\bar{\chi}\Sigma_{35}\chi\rangle,$ | Similar to entry No.5
| Quantum |  |  $\mathbf{C_{3}}$, $\mathbf{S_{6}}$, | $-i\langle\bar{\chi}\left(\frac{\sqrt{3}}{2}\Sigma_{24}+\frac{1}{2}\Sigma_{25}\right)\chi\rangle,$ | of this table but with different
| spin-octupole |  | $\mathbf{C_{2}^{\prime}},\mathbf{\sigma_{d}},\mathbf{I}$ | $-i\langle\bar{\chi}\left(\frac{\sqrt{3}}{2}\Sigma_{14}-\frac{1}{2}\Sigma_{15}\right)\chi\rangle.$ | spin-octupole current at the edges
| Hall insulator |  |  |  | protected by different set of symmetries.
| (Inversion odd) |  |  |  |
| (Sec. VII.1.3) |  |  |  |
7. | Doublet | $\mathcal{E}^{o}_{u}$ |  $\mathbf{C_{2}^{\prime}}$, $\mathbf{I}$, | $-i\langle\bar{\chi}\Sigma_{4}\chi\rangle,-i\langle\bar{\chi}\Sigma_{5}\chi\rangle$ | Fully gapped.
| Quantum |  |  $\mathbf{C_{3}}$, $\mathbf{S_{6}},$ |  | Vortices carry zero
| spin-quadrupole Hall |  |  $\mathbf{\sigma_{d}},$ TR |  | modes with charge and
| insulator. |  |  |  | quadrupole quantum numbers.
| (Sec. VII.2) |  |  |  |
8. | Triplet | $\mathcal{T}^{o}_{2g}$ | $\mathbf{T_{1}},\mathbf{T_{2}},$ | $-i\langle\bar{\chi}\Sigma_{1}\chi\rangle,$ | Fully gapped.
| Quantum |  |  $\mathbf{C_{3}}$, $\mathbf{S_{6}}$, | $-i\langle\bar{\chi}\Sigma_{2}\chi\rangle,$ | Quadrupole filtered edge
| spin-quadrupole Hall |  |  $\mathbf{C_{2}^{\prime}},\mathbf{\sigma_{d}}$, TR | $-i\langle\bar{\chi}\Sigma_{3}\chi\rangle$ | modes protected by U(1)
| insulator. |  |  |  | symmetry. TR broken.
| (Sec. VII.2) |  |  |  |

Group-2 : Flavour masses : The flavour masses are composed of chiral singlets.
There are fifteen such bilinears of the form
$-i\langle\bar{\chi}\Sigma_{j}\chi\rangle$ where $j$ runs over the fifteen
generators of SU(4) (listed in Appendix C). Ten of them are time reversal (TR)
even (Eq. 52) and transform into each other under an adjoint representation of
an SO(5) sub-group of the SU(4) flavour group while the other five are TR odd
(Eq. 53) and transform under a vector representation of the same SO(5). Under
lattice symmetries, the ten TR even masses break up into one singlet and three
triplets (Eqs. 62 and 63) while the five TR odd masses break up into a doublet
and a triplet (Eq. 64). All these 15 flavour masses correspond to six
generalised quantum spin-Hall phases Kane and Mele (2005). In particular, they
result in four TR even quantum spin-octupole Hall phases (Sec. VII.1) and two
TR odd quantum spin-quadrupole Hall phases (Sec. VII.2) with the latter being
an interesting analog of symmetry protected topological phases without time-
reversal symmetry. These masses are summarised in Table 2 while the details
are given in Sec. VII.

Group-3 : Mixed masses : The mixed masses are composed of non-trivial
combinations of flavour and chiral sectors. There are forty-five such
bilinears of the form $-i\langle\bar{\chi}\Sigma_{j}\zeta_{k}\chi\rangle$
where $j(k)$ runs over the 15(3) SU(4) flavour (SU(2) chiral) generators.
These masses, under lattice symmetries, are divided into 11 triplets, 4
doublets and 4 singlets as shown in Eq. 65. The nature of the resultant phases
is quite rich and corresponds to various types of dipolar, quadrupolar and
octupolar density waves that break both SU(4) flavour as well as lattice
symmetries. In particular, these masses describe four density wave patterns
for various types of spin-quadrupole and spin-octupole order parameters. These
are – (1) Uniform (ferro) order, (2) Staggered (“Neel”) order (Fig. 10), (3)
Stripy order (Fig. 6), and, (4) Zig-zag order (Fig. 11). Moreover, depending
on the fate of the fermions in these phases, they are divided into two classes
: (a) insulators, where all the Dirac fermions are gapped out (summarised in
Table 3), and, (b) semimetals, where a subset of Dirac fermions remain gapless
even after the condensation of the bilinear (summarised in Table 4). We
discuss them in detail in Section VIII. It is useful to note that amongst the
insulators, there are two singlets, $\mathcal{A}^{e}_{1g}$ and
$\mathcal{A}_{2g}^{o}$ (see Table 3; Secs. VIII.1.1 and VIII.1.2) that have
non-trivial gapless edge modes. While the former is time reversal even
corresponds to an Ising ferro spin-quadrupolar insulator with quantized spin-
octupole filtered edge modes that are captured by a non-trivial mutual CS term
(Eq. 124), the latter is time reversal odd and represents an anomalous Hall
insulator with quantized charge Hall response (Eq. 127).

In section IX that follows the main body of work presented in sections III-
VIII, we discuss the implications of the above results in a broader context
bringing forth the unique features and opportunities brought out by the
specific spin-orbit coupled system discussed in this paper. Various technical
details are summarised in different appendices.

Table 3: The mixed phases (insulators) # | The Phase | Irrep | Broken | Fermion | Comments
---|---|---|---|---|---
|  |  | Microscopic | Bilinear |
|  |  | symmetries |  |
9. | Singlet | $\mathcal{A}_{1g}^{e}$ | None | $-i\Braket{\bar{\chi}\left(\Sigma_{3}\zeta_{1}-\Sigma_{1}\zeta_{3}-\Sigma_{2}\zeta_{2}\right)\chi}/{\sqrt{3}}$ | Fully gapped.
| Spin-quadrupolar |  |  |  | Octupole ($\Sigma_{45}$)
| ferro Hall |  |  |  | filtered edge modes.
| insulator. |  |  |  |
| (Sec. VIII.1.1) |  |  |  |
10. | Singlet | $\mathcal{A}_{2g}^{o}$ |  $\mathbf{C_{2}^{\prime}}$, $\mathbf{\sigma_{d}}$,TR | $-i\Braket{\bar{\chi}\left(\Sigma_{12}\zeta_{1}-\Sigma_{23}\zeta_{3}+\Sigma_{13}\zeta_{2}\right)\chi}/{\sqrt{3}}$ | Fully gapped.
| Spin-octupolar |  |  |  | Gapless edges carry
| Anomalous ferro |  |  |  | qunatized charge current
| Hall Insulator. |  |  |  | via CS term.
| (Sec. VIII.1.2) |  |  |  |
11. | Doublet | $\mathcal{E}_{u}^{o}$ |  $\mathbf{C_{2}^{\prime}}$, $\mathbf{\sigma_{d}}$, $\mathbf{I}$, $\mathbf{C_{3}}$, $\mathbf{S_{6}}$ | See Eq. VIII.1.3 | Fully gapped.
| Spin-octupolar | (2 doublets) | TR | and Eq. VIII.1.3 | Both doublets correspond
| Néel insulator. |  |  |  | to the same phase.
| (Sec. VIII.1.3) |  |  |  | Vortices can carry
|  |  |  |  | non-trivial quantum
|  |  |  |  | number.
12. | Triplet | $\mathcal{T}_{2g}^{o}$ | $\mathbf{T_{1}},\mathbf{T_{2}},$ |  $-i\Braket{\bar{\chi}\Sigma_{45}\zeta_{1}\chi}$, | Fully gapped.
| Spin-octupolar |  |  $\mathbf{C_{3}}$, $\mathbf{S_{6}}$, $\mathbf{C_{2}^{\prime}},\mathbf{\sigma_{d}},$ |  $-i\Braket{\bar{\chi}\Sigma_{45}\zeta_{2}\chi}$, |
| stripy density wave |  | TR | $-i\Braket{\bar{\chi}\Sigma_{45}\zeta_{3}\chi}$ |
| (StDW) insulator. |  |  |  |
| (Sec. VIII.1.4) |  |  |  |
13. | Triplet | $\mathcal{T}_{1u}^{e}$ | $\mathbf{T_{1}},\mathbf{T_{2}},$ | $-i\Braket{\bar{\chi}\Sigma_{5}\zeta_{1}\chi}$ | Fully gapped,
| Spin-quadrupolar |  |  $\mathbf{C_{3}}$, $\mathbf{S_{6}}$, $\mathbf{C_{2}^{\prime}},\mathbf{\sigma_{d}},\mathbf{I}$ |  $-\frac{i}{2}\Braket{\bar{\chi}(-\sqrt{3}\Sigma_{4}+\Sigma_{5})\zeta_{3}\chi}$, | Masses are
| zig-zag density wave |  |  | $-\frac{i}{2}\Braket{\bar{\chi}(-\sqrt{3}\Sigma_{4}-\Sigma_{5})\zeta_{2}\chi}$ | non-comaptible,
| (ZDW) insulator. |  |  |  | Gapless modes
| (Sec. VIII.1.5) |  |  |  | for some special linear
|  |  |  |  | combinations.
14. | Triplet | $\mathcal{T}_{2u}^{e}$ | $\mathbf{T_{1}},\mathbf{T_{2}},$ |  $-i\Braket{\bar{\chi}\Sigma_{4}\zeta_{1}\chi}$, | Similar to entry No.13
| Spin-quadrupolar |  |  $\mathbf{C_{3}}$, $\mathbf{S_{6}}$, $\mathbf{C_{2}^{\prime}},\mathbf{\sigma_{d}},\mathbf{I}$ |  $-\frac{i}{2}\Braket{\bar{\chi}(\sqrt{3}\Sigma_{5}+\Sigma_{4})\zeta_{3}\chi}$, | in this table, but
| ZDW insulator |  |  | $-\frac{i}{2}\Braket{\bar{\chi}(\sqrt{3}\Sigma_{5}-\Sigma_{4})\zeta_{2}\chi}$ | different quadrupole
| (Sec. VIII.1.5) |  |  |  | operators are
|  |  |  |  | ordered.
15. | Triplet | $\mathcal{T}_{1u}^{o}$ | $\mathbf{T_{1}},\mathbf{T_{2}},$ | See | Both triplets correspond
| Spin-octupolar | (2 triplets) |  $\mathbf{C_{3}}$, $\mathbf{S_{6}}$, $\mathbf{C_{2}^{\prime}},\mathbf{\sigma_{d}},\mathbf{I},$ | Eq. VIII.1.6, VIII.1.6 | to the same phase.
| ZDW insulator. |  | TR |  | Fully gapped,
| (Sec. VIII.1.6) |  |  |  | Masses are non-compatible.
|  |  |  |  | Gapless modes appear
|  |  |  |  | for special linear
|  |  |  |  | combinations.
16. | Triplet | $\mathcal{T}_{2u}^{o}$ | $\mathbf{T_{1}},\mathbf{T_{2}},$ | See | Similar to entry No. 15
| Spin-octupolar | (2 triplets) |  $\mathbf{C_{3}}$, $\mathbf{S_{6}}$, $\mathbf{C_{2}^{\prime}},\mathbf{\sigma_{d}},\mathbf{I},$ | Eq.VIII.1.6, VIII.1.6 | in this table, but
| ZDW insulator |  | TR |  | different spin-octupole
| (Sec. VIII.1.6) |  |  |  | operators are ordered.
Table 4: The mixed phases (semimetals) # | The Phase | Irrep | Broken | Fermion | Comments
---|---|---|---|---|---
|  |  | Microscopic | Bilinear |
|  |  | symmetries |  |
17. | Singlet | $\mathcal{A}_{1u}^{o}$ | TR, $\mathbf{I},\mathbf{\sigma_{d}}$ | See Eq. VIII.2.1 | Have semimetallic features.
| Spin-octupolar |  |  |  | Can give rise to integer
| Néel semimetal. |  |  |  | QH phase which is different
| (Reflection odd) |  |  |  | from the ICI phase.
| (Sec. VIII.2.1) |  |  |  |
18. | Singlet | $\mathcal{A}_{2u}^{o}$ | TR, $\mathbf{I}$, $\mathbf{C_{2}^{\prime}}$ | See Eq. VIII.2.1 | Similar to entry No.17
| Spin-octupolar |  |  |  | in this table, but different
| Néel semimetal |  |  |  | spin-octupole operator is ordered.
| (Reflection even) |  |  |  |
| (Sec. VIII.2.1) |  |  |  |
19. | Triplet | $\mathcal{T}_{1g}^{o}$ | $\mathbf{T_{1}},\mathbf{T_{2}},$ | $-i\Braket{\bar{\chi}\left(-\Sigma_{13}\zeta_{3}-\Sigma_{23}\zeta_{2}\right)\chi}/\sqrt{2}$ | Masses are non-compatible,
| Spin-octupolar |  |  $\mathbf{C_{3}}$, $\mathbf{S_{6}}$, $\mathbf{C_{2}^{\prime}},\mathbf{\sigma_{d}},$ | $-i\Braket{\bar{\chi}\left(\Sigma_{12}\zeta_{2}-\Sigma_{13}\zeta_{1}\right)\chi}/\sqrt{2}$ | 8 fermionic modes are
| StDW semimetal |  | TR | $-i\Braket{\bar{\chi}\left(\Sigma_{23}\zeta_{1}+\Sigma_{12}\zeta_{3}\right)\chi}/\sqrt{2}$ | gapless, number of
| (Sec. VIII.2.2) |  |  |  | gapless modes are same for
|  |  |  |  | all linear combinations of
|  |  |  |  | the masses within a given
|  |  |  |  | triplet.
20. | Triplet | $\mathcal{T}_{2g}^{o}$ | $\mathbf{T_{1}},\mathbf{T_{2}},$ | $-i\Braket{\bar{\chi}\left(\Sigma_{13}\zeta_{3}-\Sigma_{23}\zeta_{2}\right)\chi}/\sqrt{2}$ | Masses are non-compatible,
| Spin-octupolar |  |  $\mathbf{C_{3}}$, $\mathbf{S_{6}}$, $\mathbf{C_{2}^{\prime}},\mathbf{\sigma_{d}},$ | $-i\Braket{\bar{\chi}\left(-\Sigma_{12}\zeta_{2}-\Sigma_{13}\zeta_{1}\right)\chi}/\sqrt{2}$ | 4 fermionic modes are always
| StDW semimetal |  | TR | $-i\Braket{\bar{\chi}\left(-\Sigma_{23}\zeta_{1}+\Sigma_{12}\zeta_{3}\right)\chi}/\sqrt{2}$ | gapless,
| (Sec. VIII.2.2) |  |  |  | this number changes depending
|  |  |  |  | on the linear combinations of
|  |  |  |  | the masses.
21. | Triplet | $\mathcal{T}_{2g}^{e}$ | $\mathbf{T_{1}},\mathbf{T_{2}},$ | $-i\Braket{\bar{\chi}\left(-\Sigma_{1}\zeta_{2}+\Sigma_{2}\zeta_{3}\right)\chi}/\sqrt{2}$ | Similar to entry No.19
| Spin-quadrupolar |  |  $\mathbf{C_{3}}$, $\mathbf{S_{6}}$, $\mathbf{C_{2}^{\prime}},\mathbf{\sigma_{d}},$ | $-i\Braket{\bar{\chi}\left(\Sigma_{3}\zeta_{2}+\Sigma_{2}\zeta_{1}\right)\chi}/\sqrt{2}$ | in this table, but different
| StDW semimetal. |  |  | $-i\Braket{\bar{\chi}\left(\Sigma_{3}\zeta_{3}+\Sigma_{1}\zeta_{1}\right)\chi}/\sqrt{2}$ | spin-quadrupole operators
| (Sec. VIII.2.3) |  |  |  | are ordered.
22. | Triplet | $\mathcal{T}_{1g}^{e}$ | $\mathbf{T_{1}},\mathbf{T_{2}},$ | $-i\Braket{\bar{\chi}\left(-\Sigma_{1}\zeta_{2}-\Sigma_{2}\zeta_{3}\right)\chi}/\sqrt{2}$ | Similar to entry No.20
| Spin-quadrupolar |  |  $\mathbf{C_{3}}$, $\mathbf{S_{6}}$, $\mathbf{C_{2}^{\prime}},\mathbf{\sigma_{d}}$ | $-i\Braket{\bar{\chi}\left(-\Sigma_{3}\zeta_{2}+\Sigma_{2}\zeta_{1}\right)\chi}/\sqrt{2}$ | in this table, but different
| StDW semimetal |  |  | $-i\Braket{\bar{\chi}\left(\Sigma_{3}\zeta_{3}-\Sigma_{1}\zeta_{1}\right)\chi}/\sqrt{2}$ | spin-octupole operators are ordered.
| (Sec. VIII.2.3) |  |  |  |
23. | Doublet | $\mathcal{E}_{g}^{e}$ |  $\mathbf{C_{2}^{\prime}},\mathbf{\sigma_{d}}$, $\mathbf{C_{3}}$, $\mathbf{S_{6}}$ | See Eq. VIII.2.4 | Some bands remain gapless.
| spin-quadrupolar |  |  |  | Can give rise to integer QH
| ferro semimetal. |  |  |  | phase different the ICI phase
| (Sec. VIII.2.4) |  |  |  |
24. | Doublet | $\mathcal{E}_{g}^{o}$ |  $\mathbf{C_{2}^{\prime}},\mathbf{\sigma_{d}}$, $\mathbf{C_{3}}$, $\mathbf{S_{6}},$ | See Eq. VIII.2.5 | Some bands remain gapless
| Spin-octupolar |  | TR |  |
| ferro semimetal. |  |  |  |
| (Sec. VIII.2.5) |  |  |  |

##  III The Dirac theory for the $d^{1}$ systems on honeycomb lattice

The starting point of our analysis is the low energy single electron atomic
orbitals of the $d^{1}$ ion in an octahedral crystal field with strong atomic
SOC Romhányi  _et al._ (2017); Yamada  _et al._ (2018) as shown in Fig. 1. In
the absence of SOC, the single electron occupies the six-fold degenerate
$t_{2g}$ atomic orbitals $|d_{XY},\sigma\rangle,|d_{YZ},\sigma\rangle$ and
$|d_{ZX},\sigma\rangle$ with $\sigma=\uparrow,\downarrow$ while the high
energy $e_{g}$ orbitals remain empty and are projected out. The SOC, projected
on the $t_{2g}$ manifold gives $H_{\rm SOC}^{t_{2g}}=-\lambda~{}{\bf l\cdot
s}$ ($\lambda>0$), where $l=1$ is the effective orbital angular momentum of
the $t_{2g}$ orbitalsRomhányi  _et al._ (2017); Kim  _et al._ (2008) (see
Appendix A.1 for further details).

The strong SOC selects the low energy orbitals by splitting the six-fold
degeneracy into $4\oplus 2$ which corresponds to the lower $j=3/2$ and higher
$j=1/2$ (${\bf j=l+s}$) orbitals respectively with the splitting being
$3\lambda/2$ Kim  _et al._ (2008). For the $d^{1}$ configuration, the low
energy physics is therefore of a 1/4th filled $j=3/2$ orbitals (Fig. 1) with
single electron creation operators at the lattice site are given by Romhányi
_et al._ (2017); Yamada  _et al._ (2018)

$\displaystyle\psi^{\dagger}=\left(\psi^{\dagger}_{1/2},\psi^{\dagger}_{-1/2},\psi^{\dagger}_{3/2},\psi^{\dagger}_{-3/2}\right).$
(6)

The interplay of hopping and interaction of electrons occupying these four
orbitals then determine the low energy electronic properties of the system.

###  III.1 The lattice and microscopic symmetries

Consider such $j=3/2$ orbitals on a honeycomb geometry with edge-sharing
octahedra. In this geometry, the active atoms sit at the centre of each
octahedron and form the honeycomb lattice. It is useful to consider the
honeycomb lattice to lie in a plane perpendicular to the Cartesian $[111]$
direction (details in Appendix A.2) such that the three nearest neighbour
bonds are parallel to the three Cartesian planes shown in Fig. 1.
Correspondingly we denote these bonds as $x$, $y$, and $z$ bonds if they are
parallel to the $YZ$, $ZX$, and $XY$ planes respectively, following the, by
now popular nomenclature in the context of the Kitaev spin model Kitaev (2006)
on the honeycomb lattice Nanda  _et al._ (2020).

Figure 2: The filled circles are points of A sub-lattice and the hollow
circles are of B sub-lattice. The gray shaded area shows the two points of a
single unit cell with $\mathbf{b}_{1}$ and $\mathbf{b}_{2}$ being unit lattice
vectors (see Eq. 214).

The honeycomb net has a triangular Bravais lattice and a two-site unit cell
with two sub-lattices, $\mathcal{s}=A,B$ as shown in Fig. 2. Each point on the
honeycomb lattice is labeled by
$(\mathcal{u}_{1},\mathcal{u}_{2},\mathcal{s})$ where
$\mathcal{u}_{1},\mathcal{u}_{2}\in\mathbb{Z}$ denote the position of the unit
cell via

$\displaystyle{\bf r}=\mathcal{u}_{1}{\bf b}_{1}+\mathcal{u}_{2}{\bf b}_{2}$
(7)

with ${\bf b}_{1}$ and ${\bf b}_{2}$ are unit lattice vectors shown in Fig. 2.

| ${\bf r}(\mathcal{u}_{1},\mathcal{u}_{2},\mathcal{s})\rightarrow{\bf r}^{\prime}(\mathcal{u}_{1}^{\prime},\mathcal{u}_{2}^{\prime},\mathcal{s}^{\prime})$ | $\psi({\bf r})\to\psi^{\prime}({\bf r}^{\prime})$
---|---|---
${\bf T}_{1}$ | $(\mathcal{u}_{1},\mathcal{u}_{2},\mathcal{s})\rightarrow(\mathcal{u}_{1}+1,\mathcal{u}_{2},\mathcal{s})$ | $\psi\to\mathcal{U}_{\mathbf{T}_{1}}\psi$
${\bf T}_{2}$ | $(\mathcal{u}_{1},\mathcal{u}_{2},\mathcal{s})\rightarrow(\mathcal{u}_{1},\mathcal{u}_{2}+1,\mathcal{s})$ | $\psi\to\mathcal{U}_{\mathbf{T}_{2}}\psi$
${\bf C_{3}}$ | $(\mathcal{u}_{1},\mathcal{u}_{2},\mathcal{s})\to(\mathcal{u}_{2}-1,-\mathcal{u}_{1}-\mathcal{u}_{2}+\delta_{\mathcal{s},A},\mathcal{s})$ | $\psi\to\mathcal{U}_{\mathbf{C_{3}}}\psi$
${\bf S_{6}}$ | $(\mathcal{u}_{1},\mathcal{u}_{2},\mathcal{s})\to(\mathcal{u}_{1}+\mathcal{u}_{2}-\delta_{\mathcal{s},A},-\mathcal{u}_{1},\bar{\mathcal{s}})$ | $\psi\to\mathcal{U}_{\mathbf{S_{6}}}\psi$
${\bf C_{2}^{\prime}}$ | $(\mathcal{u}_{1},\mathcal{u}_{2},\mathcal{s})\to(\mathcal{u}_{2}-1,\mathcal{u}_{1}+1,\mathcal{s})$ | $\psi\to\mathcal{U}_{\mathbf{C_{2}}}\psi$
${\bm{\sigma_{d}}}$ | $(\mathcal{u}_{1},\mathcal{u}_{2},\mathcal{s})\to(-\mathcal{u}_{2},-\mathcal{u}_{1},\bar{\mathcal{s}})$ | $\psi\to\mathcal{U}_{\mathbf{\sigma_{d}}}\psi$
${\bf I}$ | $(\mathcal{u}_{1},\mathcal{u}_{2},\mathcal{s})\to(-\mathcal{u}_{1}-1,-\mathcal{u}_{2}+1,\bar{\mathcal{s}})$ | $\psi\to\mathcal{U}_{\mathbf{I}}\psi$
Table 5: The action of the microscopic symmetries on the $j=3/2$ orbitals (Eq.
6). Note $\bar{\mathcal{s}}=B(A)$ for $\mathcal{s}=A(B)$. The
$\mathcal{U}_{\mathbb{S}}$ ($\mathbb{S}$ represents some lattice symmetry) are
$4\times 4$ unitary matrices which are given in Appendix B (Eqs 223 \- 240).

To understand the lattice symmetries, we take the ideal honeycomb structure of
$\alpha$-ZrCl3 as a prototypical example as it has all the representative
symmetries. The point group of the $\alpha$-ZrCl3 lattice is $D_{3d}$ which
has 12 elements. Keeping in mind the geometry of the edge-sharing ligand
octahedra surrounding the active ion (Fig. 1), the generators of the lattice
symmetries of the system are as follows and their action on the lattice
coordinates are given in Table 5.

  * •

${\bf T}_{1},{\bf T}_{2}:$ Two-dimensional lattice translations of the
honeycomb lattice by ${\bf b}_{1}$ and ${\bf b}_{2}$ respectively.

  * •

${\bf C_{3}:}$ Rotations by angle $\frac{2\pi}{3}$ about the center of a
honeycomb plaquette.

  * •

${\bf S_{6}:}$ Rotations about the center of a honeycomb plaquette by angle
$\frac{\pi}{3}$ followed by a reflection about the honeycomb plane.

  * •

${\bf C^{\prime}_{2}}:$ Rotations by angle $\pi$ about the axes lying on the
honeycomb plane and passing through two opposite vertices of a honeycomb
plaquette. There are three of such axes. One of the ${\bf C_{2}^{\prime}}$
axes is parallel to the $z$-bonds (see Fig. 2).

  * •

${\bm{\sigma}_{d}}:$ Reflections about planes which are perpendicular to the
honeycomb plane and bisect the angle between two consecutive ${\bf
C^{\prime}_{2}}$ axes. There are three such planes. One of the planes is the
perpendicular bisector of one of the $z$-bonds in Fig 2.

  * •

${\bf I:}$ Inversion about the center of a honeycomb plaquette.

In addition, the system also has time reversal (TR) symmetry $\mathbb{T}$,
with

$\displaystyle\mathbb{T}^{2}=-1.$ (8)

In appendix B, we provide the details of the transformation of the $t_{2g}$
and the $j=3/2$ orbitals under the above symmetries.

###  III.2 The tight binding Hamiltonian for indirect hopping

Starting with the hopping Hamiltonian for the $t_{2g}$ orbitals and taking
into account the indirect hopping amplitudes via the ligand in the edge-
sharing geometry (Fig. 1), the effective Hamiltonian is obtained by projecting
it to the $j=3/2$ manifold using Eq. 210. This minimal hopping Hamiltonian for
the $j=3/2$ orbitals (Eq. 6) is given by

$\displaystyle
H=-\frac{t}{\sqrt{3}}\sum_{\braket{\mathbf{r,\mathcal{s};r^{\prime},\mathcal{s}^{\prime}}}}\psi^{\dagger}(\mathbf{r},\mathcal{s})U^{\mathcal{s}\mathcal{s}^{\prime}}_{\mathbf{rr^{\prime}}}\psi(\mathbf{r^{\prime}},\mathcal{s}^{\prime})+{\rm
h.c.}$ (9)

where $U^{\mathcal{s}\mathcal{s}^{\prime}}_{\mathbf{rr^{\prime}}}$ are hopping
amplitudes of overall strength $t$, on nearest neighbour bonds (hence only
between different sublattices) that are given by $4\times 4$ Hermitian
matrices which depend on the type ($x,y$ or $z$, see Fig. 2) of the
$\braket{\mathbf{r},\mathcal{s};\mathbf{r}^{\prime},\mathcal{s}^{\prime}}$
bond Yamada  _et al._ (2018) given by

$\displaystyle U^{AB}_{\mathbf{rr^{\prime}}}$ $\displaystyle\equiv
U_{x}=-\Sigma_{1}\text{ , if $\braket{\mathbf{r},A;\mathbf{r}^{\prime},B}=x$
}$ (10) $\displaystyle\equiv U_{y}=-\Sigma_{2}\text{ , if
$\braket{\mathbf{r},A;\mathbf{r}^{\prime},B}=y$ }$ $\displaystyle\equiv
U_{z}=-\Sigma_{3}\text{ , if $\braket{\mathbf{r},A;\mathbf{r}^{\prime},B}=z$
}$

Here, $\Sigma_{i}$ are sixteen $4\times 4$ traceless Hermitian matrices with
$\Sigma_{0}$ being the identity matrix and the rest being generators of SU(4).
They can be obtained by using $j=3/2$ matrices as shown in Appendix C.

The three $U_{\alpha}$(for $\alpha=x,y,z$) matrices square up to identity and
mutually anti-commute, i.e.,

$\displaystyle
U_{\alpha}^{2}=\mathbb{I}_{4}\equiv\Sigma_{0},~{}~{}~{}~{}\\{U_{\alpha},U_{\beta}\\}=2\delta_{\alpha\beta}\Sigma_{0}~{}~{}~{}~{}\forall~{}\alpha,\beta=x,y,z.$
(11)

Before proceeding to diagonalize Eq. 9 to obtain the electron band structure,
we identify the generic nature of the electron dispersion.

####  III.2.1 The SU(4) Symmetry and $\pi$-flux

As a first step, it is useful to consider the phase picked up by the electron
on encircling any closed loop of the lattice. Such loops are formed out of the
honeycomb plaquette consisting of six sites. The phase is given by the product
of the $U_{\alpha}$ matrices around a honeycomb plaquette and is generically
given by

$\displaystyle\prod_{\braket{\mathbf{r,\mathcal{s};r^{\prime},\mathcal{s}^{\prime}}}\in\hexagon}U^{\mathcal{s}\mathcal{s}^{\prime}}_{\mathbf{rr^{\prime}}}=\sum_{i}W_{i}~{}\Sigma_{i}$
(12)

where $W_{i}$ are the respective coefficients that denote a generic direction
in the SU(4) space. In the above sum, $i$ runs over the 16 indices of the
$\Sigma_{i}$ matrices defined in Appendix C.

Crucially, however, it was noticed in Ref. Yamada  _et al._ (2018) that the
explicit form of $U$-matrices (Eq. 10) give

$\displaystyle\prod_{\braket{\mathbf{r,\mathcal{s};r^{\prime},\mathcal{s}^{\prime}}}\in\hexagon}U^{\mathcal{s}\mathcal{s}^{\prime}}_{\mathbf{rr^{\prime}}}=-\Sigma_{0}$
(13)

such that no direction in SU(4) space is favoured and the system has an
underlying SU(4) symmetry. This SU(4) symmetry can be made manifest by
suitable site-dependent unitary rotations of the $\psi$ fermions (see below,
Eq. 17) Yamada  _et al._ (2018).

An equally important feature is the negative sign in Eq. 13 which shows that
such SU(4) fermions experience a $\pi$-flux through every hexagon. Thus the
above problem of $d^{1}$ fermions is that of SU(4) symmetric fermions hopping
on a honeycomb lattice with $\pi$-flux per plaquette. The non-trivial effect
of $\pi$-flux at $1/4$-th filling is already apparent by considering the
simpler case of spinless fermions on honeycomb lattice with $\pi$-flux at
quarter-filling. This, as we discuss below, leads to Dirac fermions at low
energy whose properties are quite different from those in graphene.

In the rest of this work, we uncover the interplay of SU(4) symmetry and the
$\pi$-flux that, along with electron-electron interactions, leads to rich low-
energy electronic properties of $d^{1}$ systems.

####  III.2.2 SU(4) diagonalization and the local basis

Following Ref. Yamada  _et al._ (2018), the SU(4) symmetry of the hopping
Hamiltonian in Eq. 9 can be made manifest by performing site-dependent (local)
unitary transformations on the fermions.

Figure 3: Honeycomb lattice with $\pi$-flux and the four-point magnetic unit-
cell in the gauge choice (Eq. 22): the dashed (continuous) bonds have
$\eta({\mathbf{\boldsymbol{\mathcalligra{r}}\,_{\mathcal{S}},\boldsymbol{\mathcalligra{r}}\,_{\mathcal{S}^{\prime}}^{\prime}}})=-1(+1)$
(see Eq. 19).

To obtain this manifestly SU(4) invariant basis, and also due to the
$\pi$-flux, it is useful to consider a four site magnetic unit cell as shown
in Fig. 3. The four sites, $A_{1},A_{2},B_{1},B_{2}$, in the magnetic unit
cell comprise two sites each of $A$ and $B$ sub-lattices of the underlying
honeycomb net. The lattice translation vectors for this magnetic unit-cell, as
shown in Fig. 3, are given in terms of the underlying honeycomb lattice
primitive vectors as

$\displaystyle{\bf R}_{1}={\bf b}_{1}+{\bf b}_{2};~{}~{}~{}~{}~{}~{}{\bf
R}_{2}={\bf b}_{1}-{\bf b}_{2}$ (14)

such that the sites with reference to the enlarged unit cell are given by

$\displaystyle\boldsymbol{\mathcalligra{r}}\,_{\mathcal{S}}=n_{x}\mathbf{R}_{1}+n_{y}\mathbf{R}_{2}+\mathbf{d}_{\mathcal{S}}\equiv\boldsymbol{\mathcalligra{r}}\,+{\bf
d}_{\mathcal{S}}.$ (15)

with $\mathcal{S}\in\\{A_{1},A_{2},B_{1},B_{2}\\}$ denotes the four sites in
the magnetic unit-cell, ${\bf d}_{\mathcal{S}}$ are the position vectors of
the $\mathcal{S}$-sublattice site w.r.t. the $B_{2}$ site (see Fig. 3) of the
same magnetic unit cell, $\boldsymbol{\mathcalligra{r}}\,$ labelled by
integers $n_{x}$ and $n_{y}$.

With this, we can now define new fermion annihilation operators given by

$\displaystyle\phi(\boldsymbol{\mathcalligra{r}}\,_{\mathcal{S}})=[\phi_{1}(\boldsymbol{\mathcalligra{r}}\,_{\mathcal{S}}),\phi_{2}(\boldsymbol{\mathcalligra{r}}\,_{\mathcal{S}}),\phi_{3}(\boldsymbol{\mathcalligra{r}}\,_{\mathcal{S}}),\phi_{4}(\boldsymbol{\mathcalligra{r}}\,_{\mathcal{S}})]^{T}$
(16)

as

$\displaystyle\phi(\boldsymbol{\mathcalligra{r}}\,_{\mathcal{S}})=\mathcal{G}(\boldsymbol{\mathcalligra{r}}\,_{\mathcal{S}})^{\dagger}\psi(\boldsymbol{\mathcalligra{r}}\,_{\mathcal{S}})$
(17)

where $\mathcal{G}(\boldsymbol{\mathcalligra{r}}\,_{\mathcal{S}})$ are
$4\times 4$ unitary matrices whose explicit forms are given in Appendix D. Any
many-body operator can be expressed either in the $\phi$ basis or $\psi$. In
this article, we use the terms “local basis” and “global basis” respectively
to refer to these two ways.

The Hamiltonian (Eq. 9) written in the local basis is

$\displaystyle H$
$\displaystyle=-\frac{t}{\sqrt{3}}\sum_{\braket{\boldsymbol{\mathcalligra{r}}\,_{\mathcal{S}},\boldsymbol{\mathcalligra{r}}\,^{\prime}_{\mathcal{S}^{\prime}}}}\eta({\mathbf{\boldsymbol{\mathcalligra{r}}\,_{\mathcal{S}},\boldsymbol{\mathcalligra{r}}\,_{\mathcal{S}^{\prime}}^{\prime}}})\phi^{\dagger}(\boldsymbol{\mathcalligra{r}}\,_{\mathcal{S}})\phi(\boldsymbol{\mathcalligra{r}}\,^{\prime}_{\mathcal{S}^{\prime}})+\text{
h.c. }$ (18)

which is manifestly SU(4) invariant and
$\eta({\mathbf{\boldsymbol{\mathcalligra{r}}\,_{\mathcal{S}},\boldsymbol{\mathcalligra{r}}\,_{\mathcal{S}^{\prime}}^{\prime}}})=\pm
1$ implementing the $\pi$-flux constraint of Eq. 13, via

$\displaystyle\prod_{\braket{\mathbf{\boldsymbol{\mathcalligra{r}}\,_{\mathcal{S}},\boldsymbol{\mathcalligra{r}}\,_{\mathcal{S}^{\prime}}^{\prime}}}\in\hexagon}\eta({\mathbf{\boldsymbol{\mathcalligra{r}}\,_{\mathcal{S}},\boldsymbol{\mathcalligra{r}}\,_{\mathcal{S}^{\prime}}^{\prime}}})=-1$
(19)

Fig. 3 shows a choice for
$\eta({\mathbf{\boldsymbol{\mathcalligra{r}}\,_{\mathcal{S}},\boldsymbol{\mathcalligra{r}}\,_{\mathcal{S}^{\prime}}^{\prime}}})$
which is given by

$\displaystyle\eta({\mathbf{\boldsymbol{\mathcalligra{r}}\,_{\mathcal{S}},\boldsymbol{\mathcalligra{r}}\,_{\mathcal{S}^{\prime}}^{\prime}}})=\left\\{\begin{array}[]{cl}-1&\text{
if }\mathcal{S}=B_{1},\mathcal{S}^{\prime}=A_{2}\text{ and
}\boldsymbol{\mathcalligra{r}}\,^{\prime}=\boldsymbol{\mathcalligra{r}}\,+\mathbf{R}_{2}\\\
+1&\text{ (otherwise)}\\\ \end{array}\right.$ (22)

Obviously, there are many other choices for
$\eta({\mathbf{\boldsymbol{\mathcalligra{r}}\,_{\mathcal{S}},\boldsymbol{\mathcalligra{r}}\,_{\mathcal{S}^{\prime}}^{\prime}}})$
which are related to each other through gauge transformations which correspond
to different signs of the
$\mathcal{G}(\boldsymbol{\mathcalligra{r}}\,_{\mathcal{S}})$ matrices with
respect to the ones introduced in Appendix D. An alternate choice for
$\eta({\mathbf{\boldsymbol{\mathcalligra{r}}\,_{\mathcal{S}},\boldsymbol{\mathcalligra{r}}\,_{\mathcal{S}^{\prime}}^{\prime}}})$
and indeed the magnetic unit cell is shown in Fig. 15. For the rest of our
calculation in the main text, we choose
$\eta({\mathbf{\boldsymbol{\mathcalligra{r}}\,_{\mathcal{S}},\boldsymbol{\mathcalligra{r}}\,_{\mathcal{S}^{\prime}}^{\prime}}})$
as given by Eq. 22.

####  III.2.3 The band structure

Figure 4: (a) Band structure for the Hamiltonian in Eq. 18 for the magnetic
unit cell consisting of four sites (Fig. 3). Each band is four-fold
degenerate. For $d^{1}$ system, the lowest four bands are occupied with the
chemical potential crossing the two Dirac points as shown. (b) Position of the
two Dirac points (Eq.23) in the chosen gauge (Fig. 3 and Eq. 22) in the
magnetic Brillouin zone (in red). The primitive Brillouin zone of the
hexagonal lattice is also drawn in blue.

Eq. 18 represents four copies of nearest neighbour hopping model on honeycomb
lattice in presence of $\pi$-flux. A single copy of such model at half filling
was studied in Ref. Ryu  _et al._ (2009); Bercx  _et al._ (2014). However, we
shall find that the underlying SU(4) symmetry in the present case and the
$1/4$th filling for $d^{1}$ materials (see below) along with SOC open up a new
regime of possibilities for the resultant system at low energies.

To disentangle the role of the SU(4) and the $\pi$-flux, it is useful to
consider a single flavour “spinless” version of Eq. 18 with
$\phi(\boldsymbol{\mathcalligra{r}}\,_{\mathcal{S}})$ being a single component
fermion. This is worked out in Appendices E and K. The resultant band
structure is shown in Fig. 4 and consists, for quarter filling, two linearly
dispersing band-touching points–Dirac cones– at

$\displaystyle\pm\mathbf{Q}=\pm\left[\frac{\pi}{6},-\frac{\pi}{2\sqrt{3}}\right].$
(23)

We label the two Dirac points (valleys) by the Ising variable $\tau=\pm$.
Similar Dirac points also occur at 3/4th filling by particle-hole symmetry of
the microscopic problem. Also note that there are four Dirac points at half
filling Bercx  _et al._ (2014) as is shown in Fig. 4. In the rest of the
discussion though we shall consider exclusively the vicinity of quarter
filling and the nature of the low energy Dirac fermions at the two valleys at
$\pm{\bf Q}$ given by Eq. 23.

Turning back to the case of $j=3/2$ orbitals in $d^{1}$ configuration (Eq.
18), the band structure is now four-fold degenerate due to the SU(4) symmetry
such that the lower four bands are completely filled with the chemical
potential again at the two Dirac cones given by Eq. 23. As remarked above, the
similar Dirac cones are also present for three-quarter filling and hence the
rest of our discussion is also applicable to materials with $d^{3}$ electronic
configuration.

At this point, we would like to take a small detour by discussing the above
band structure in the global basis (Eq. 6) which provides interesting
complementary insights into the results that follow in the rest of this paper.
This alternate insight arises from the observation that while for a single
flavour $\pi$-flux problem we are forced to use the magnetic unit cell (Fig.
3), for the four flavour version relevant to $d^{1}$ or $d^{3}$ systems, it is
possible to use the two-site primitive honeycomb unit-cell (Fig. 2) by
diagonalizing the Hamiltonian in Eq. 9. However, in this global basis neither
the SU(4), nor the $\pi$-flux is manifest but are mixed together non-
trivially. As a result, while all the lattice symmetries (Table 5), act in a
linear fashion, i. e., they are non-projective despite the fact that each
hexagonal plaquette hosts a $\pi$-flux as shown in Eq. (13), to overcome the
slightly more involved nature (arising explicit mixing of the lattice
symmetries and the SU(4) in the global basis) we use the local basis for most
of our discussion in the rest of the main-texts. Notwithstanding, the global
basis is useful to understand certain structures in our calculations which we
refer to at relevant places throughout the rest of the paper. The
considerations in the global basis are presented in Appendix J. Briefly, Bloch
diagonalizing the Hamiltonian in global basis (Eq. 9) obtains four bands
arising from the four $j=3/2$ orbitals (Eq. 6) and each two-fold degenerate
due to inversion symmetry. The first set of bands touches the second set of
bands at four distinct points with Dirac cone structure, see Fig. 14. With the
quarter filling of the bands the chemical potential is tuned to the Dirac
points at the four $\mathbf{Q}_{g}$ vectors, $\Gamma$, M1, M2, M3 termed as
valleys, in the original honeycomb Brillouin zone. This is to be contrasted
with the local basis where one obtains two valleys due to the doubling of the
unit cell, and the concomitant folding of the bands. One of the central
insights of the global basis is that under lattice symmetries such as ${\bf
S}_{6}$ (Table 5), only three of the Dirac cones sitting at the three $M$
points (Fig. 14) mix amongst themselves while the Dirac cone at the BZ center,
i.e. at $\Gamma$-point, remains isolated. This naturally distinguishes the
different valleys into two groups– one containing only the $\Gamma$ point cone
and the other containing the other three at the three nonequivalent $M$
points. As we shall see later, the above grouping is a fallout of the fact
that the microscopic lattice symmetries get embedded in a larger low energy IR
space group (see Section V) that allows up to three-dimensional
representations such that the above grouping is a block diagonalization of a
reducible representation, i.e., ${\bf 4=1\oplus 3}$. This insight will be
important in understanding a subset of partially gapless masses discussed in
section VIII.2.

###  III.3 Low-energy Dirac theory

Turning back to the local basis (Eq. 16 and Fig. 4), for $1/4$th filling, the
low energy theory is obtained by expanding the lattice fermions,
$\phi({\boldsymbol{\mathcalligra{r}}\,_{\mathcal{S}}})$, in terms of the soft-
modes, around the two Dirac points, $\pm{\bf Q}$, as

$\displaystyle\phi_{f}({\boldsymbol{\mathcalligra{r}}\,_{\mathcal{S}}})\sim\mathcal{W}^{(+)}_{\mathcal{S}\sigma}\chi_{f\sigma+}(\boldsymbol{\mathcalligra{r}}\,)e^{i\bf
Q\cdot\boldsymbol{\mathcalligra{r}}\,}+\mathcal{W}^{(-)}_{\mathcal{S}\sigma}\chi_{f\sigma-}(\boldsymbol{\mathcalligra{r}}\,)e^{-i\bf
Q\cdot\boldsymbol{\mathcalligra{r}}\,}$ (24)

where $\chi_{f\sigma\tau}({\bf x})$ are the soft modes in the continuum
evaluated at ${\bf x}=\boldsymbol{\mathcalligra{r}}\,$ with $f=1,\cdots,4$
denote the SU(4) flavour index, $\sigma=1,2$ is the particle or hole-like band
index and $\tau=\pm$ is the valley index coming from the two Dirac nodes at
$\pm{\bf Q}$. $\mathcal{W}^{(\pm)}_{\mathcal{S}\sigma}$ are two $4\times 2$
matrices (one at each valley, $\tau=\pm$) in the (magnetic) unit-cell
$(\mathcal{S})$-particle-hole ($\sigma$) space. The details are given in
Appendix E.

In terms of the soft modes, the low energy Hamiltonian takes the canonical
Dirac form in two spatial dimensions and is given by

$\displaystyle H_{D}=-iv_{F}\sum_{f=1}^{4}\int d^{2}{\bf
x}~{}\chi_{f}^{\dagger}(\mathbf{x})(\alpha_{1}\partial_{1}+\alpha_{2}\partial_{2})\chi_{f}(\mathbf{x})$
(25)

where $v_{F}=\frac{t\mathcal{l}}{\sqrt{2}}$ is the fermi velocity,
$\mathcal{l}$ is the length of each side of the hexagon and
$\partial_{i}=\partial/\partial x_{i}$ ($i=1,2$), with

$\displaystyle\chi_{f}(\mathbf{x})$
$\displaystyle=\left(\chi_{f1+},\chi_{f2+},\chi_{f1-},\chi_{f2-}\right)^{T}$
(26)

a 4-component spinor, one for each SU(4) flavour $f=1,2,3,4$, which can be
further stacked up to form a $16$-component spinor, and

$\displaystyle\alpha_{1}=\tau_{3}\sigma_{1},~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}\alpha_{2}=\tau_{0}\sigma_{2}$
(27)

are the two Dirac Matrices. Here $\tau_{\mu}$ and $\sigma_{\mu}$
($\mu=0,1,2,3$) are Pauli matrices that act in the valley space and
band/particle-hole space respectively.

The corresponding Euclidean action is given by

$\displaystyle{S}_{0}=\int d^{2}{\bf x}d\tau~{}\mathcal{L}_{0}$ (28)

where, $\mathcal{L}_{0}$ is given by Eq. 1 with $N_{F}=4$ and
$\bar{\chi}_{f}=i\chi^{\dagger}_{f}\gamma_{0}$ and

$\displaystyle\gamma_{0}=\tau_{3}\sigma_{3},~{}~{}~{}~{}~{}~{}~{}\gamma_{1}=\tau_{0}\sigma_{2}~{}~{}~{}~{}~{}~{}~{}\gamma_{2}=-\tau_{3}\sigma_{1}$
(29)

such that $\alpha_{1}=i\gamma_{0}\gamma_{1}$ and
$\alpha_{1}=i\gamma_{0}\gamma_{2}$. Here $\gamma_{0},\gamma_{1}$ and
$\gamma_{2}$ generate the Euclidean Clifford Algebra that satisfy
$\left\\{\gamma_{\mu},\gamma_{\nu}\right\\}=2\delta_{\mu\nu}$ with
$\mu,\nu=0,1,2$ Kaplan (2009).

The above low-energy free Dirac theory has a much larger symmetry compared to
the microscopic system. Firstly, Eq. 28 is invariant under SU(2)
transformations on each flavour of $\chi_{f}$ generated by

$\displaystyle\\{\tau_{3}\sigma_{0}/2,\tau_{1}\sigma_{2}/2,\tau_{2}\sigma_{2}/2\\}\equiv\\{\zeta_{1},\zeta_{2},\zeta_{3}\\}/2.$
(30)

This denotes rotation in the valley and band space similar to Dirac fermions
in graphene Herbut  _et al._ (2009a) which we refer to as chiral symmetry Ryu
_et al._ (2009). This, along with the manifest invariance under the SU(4)
flavour symmetry generated by $\Sigma_{i}$ (defined in Appendix. C), nominally
gives rise to an internal symmetry of SU(4) $\otimes$ SU(2). However, the
emergent internal symmetry is SU(8) which is generated by 63 traceless
Hermitian matrices, $\mathcal{P}_{b}$, that are obtained as

$\displaystyle\mathcal{P}_{b}=\Sigma_{i}\zeta_{j}$ (31)

where on the LHS, $b=1,2,\cdots,63$ which are made up of the fifteen SU(4)
generators, $\Sigma_{i}$, given in Appendix C and three SU(2) generators,
$\zeta_{j}$, defined in Eq. 30 along with the identities in the two spaces
$\Sigma_{0}$ and $\zeta_{0}$ respectively.

Under the SU(8) generated by the $16\times 16$ traceless Hermitian matrices,
$\mathcal{P}_{b}$, the spinors $\chi$ transform as

$\displaystyle\chi\rightarrow\exp(i\xi_{b}\mathcal{P}_{b})\chi$ (32)

where $\chi=(\chi_{1}^{T},\chi_{2}^{T},\chi_{3}^{T},\chi_{4}^{T})^{T}$ is the
16-component spinor with each $\chi_{f}~{}(f=1,2,3,4)$ given by Eq. 26. This
leads to the conservation of the SU(8) flavour current

$\displaystyle J_{\mu,b}=-i\bar{\chi}\gamma_{\mu}\mathcal{P}_{b}\chi$ (33)

i.e. $\partial_{\mu}J_{\mu,b}=0$ $\forall~{}b=1,\cdots,63$. This is to be
contrasted with SU(8) Dirac fermions realised in the $\pi$-flux phase on a
square lattice Hermele  _et al._ (2005) for fermionic spinons in a class of
quantum spin-liquids, where the resultant implementation of the symmetries on
the low energy fermions are very different. Crucially, in the present case,
the non-trivial SOC of the underlying orbitals results in the mixing of the
spin and the real spaces, under various lattice symmetries and time reversal
which leads to important observable consequences which are reflected in the
nature of the phases proximate to the semimetal, as we show below. In
addition, in the square lattice spin liquid problem, the spinons couple to an
emergent dynamic SU(2) gauge field which is absent in the present case.

In addition to the above internal SU(8), the free Dirac action of Eq. 28 has a
usual set of emergent space-time symmetries that include :

##### Emergent Lorentz symmetry

generated by the three matrices

$\displaystyle\gamma_{\mu\nu}=-\frac{i}{4}[\gamma_{\mu},\gamma_{\nu}]$ (34)

along with simultaneous rotations of the Euclidean space-time coordinates.
Under Lorentz transformation, the spinors transform as
$\chi\rightarrow\exp(i\Omega_{\mu\nu}\gamma_{\mu\nu})\chi$. Note that
$\gamma_{\mu}=\epsilon^{\mu\nu\lambda}\gamma_{\nu\lambda}$ where
$\mu,\nu,\lambda=0,1,2$.

##### Continuous spatial translation symmetry

under which the soft modes at the two valleys transform as

$\displaystyle{\bf T}_{\bf x_{0}}^{\rm
cont}:\left\\{\begin{array}[]{c}\chi_{+}({\bf
x})\rightarrow\chi^{\prime}_{+}({\bf x})=e^{i{\bf Q\cdot
x_{0}}}~{}\chi_{+}({\bf x-x_{0}})\\\ \chi_{-}({\bf
x})\rightarrow\chi^{\prime}_{-}({\bf x})=e^{-i{\bf Q\cdot
x_{0}}}~{}\chi_{-}({\bf x-x_{0}})\\\ \end{array}\right.$ (37)

where

$\displaystyle\chi_{\pm}({\bf x})=\frac{1}{2}(1\pm\zeta_{1})\chi({\bf x})$
(38)

are the two spinors associated with the two valleys respectively located at
${\bf\pm Q}$. Using Eq. 38, we can re-write the free Dirac action (Eq. 28) as
${S}_{0}={S}_{0}^{+}+{S}_{0}^{-}$ where ${S}_{0}^{\pm}=v_{F}\int d^{2}{\bf
x}d\tau~{}\bar{\chi}_{\pm}({\bf x})(-i\not{\partial})\chi_{\pm}({\bf x})$ are
the actions at the two valleys.

##### Emergent CPT symmetries :

The free Dirac action ${S}_{0}$ is also invariant under emergent charge
conjugation($\mathcal{C}$), emergent parity($\mathcal{P}$) and emergent time
reversal($\mathcal{T}^{\prime}$) symmetries. These symmetries act on the
spinors in the following way:

$\displaystyle\mathcal{C}:$
$\displaystyle\quad\chi(\mathbf{x},t)\to-i\gamma_{2}\gamma_{0}\bar{\chi}^{T}(\mathbf{x},t),$
(39a) $\displaystyle\mathcal{P}:$
$\displaystyle\quad\chi(x_{1},x_{2},t)\to-i\gamma_{1}\chi(-x_{1},x_{2},t),$
(39b) $\displaystyle\mathcal{T}^{\prime}:$
$\displaystyle\quad\chi(\mathbf{x},t)\to-i\gamma_{2}K\chi(\mathbf{x},-t),$
(39c)

with $K$ being the complex conjugation operator. Here we denote the emergent
time-reversal by $\mathcal{T}^{\prime}$ to distinguish it from the microscopic
time reversal operation defined in Eq. 42 (which we denote with $\mathbb{T}$
in Eq. 8).

###  III.4 Microscopic symmetries in the low energy theory

The enhanced IR symmetries provide important insights into the low-energy
physics including the properties of the Dirac semimetal and associated quantum
phase transitions into proximate symmetry broken phases. The latter is
determined by the underlying UV/microscopic symmetries. These UV symmetries
are embedded as a subgroup of the emergent (larger) IR symmetry group and are
implemented as a combination of the IR symmetry transformations (see, for
example, the discussion below Eq. 42 for the time-reversal symmetry). This is
particularly interesting in the present case where the underlying SOC mixes
the lattice and the $j=3/2$ flavour space such that the embedding of the
microscopic symmetries in the IR symmetry group can be rather intricate. It
is, therefore, useful to list the symmetry transformation of the low energy
Dirac fermions, $\chi$, under various microscopic symmetries discussed above.

The total electronic charge is conserved in the microscopic system. This U(1)
electronic charge conservation leads to the conservation of a current

$\displaystyle J^{charge}_{\mu}=-i\bar{\chi}\gamma_{\mu}\chi$ (40)

in the low energy Dirac theory, i.e., $\partial_{\mu}J_{\mu}^{charge}=0$.

On the other hand, the transformation of the low energy Dirac fermions,
$\chi$, under the discrete lattice symmetries (Table 5) as well as microscopic
time reversal (Eq. 8) have the generic form (see Appendix F for details)

$\displaystyle\chi({\bf x})\xrightarrow{\mathbb{S}}\chi^{\prime}({\bf
x}^{\prime})$
$\displaystyle=\left(\Omega^{f}_{\mathbb{S}}\otimes~{}\Omega^{c}_{\mathbb{S}}\right)\chi(\mathbb{S}^{-1}\mathbf{x})$
(41)

where $\mathbb{S}(={\bf
T_{1},T_{2},C_{3},S_{6},C_{2}^{\prime}},\bm{\sigma_{d}},{\bf I})$ stands for
the generators of the lattice symmetries listed in Table 5 and
$\Omega_{\mathbb{S}}^{f},\Omega_{\mathbb{S}}^{c}$ both are $4\times 4$ unitary
matrices that act on the SU(4) flavor space and the chiral space respectively.
The explicit form of these matrices is given in Appendix F.2. A central aspect
of Eq. 41 is the fact that because of underlying SOC, both
$\Omega^{f}_{\mathbb{S}}$ and $\Omega^{c}_{\mathbb{S}}$ are non-trivial
matrices for all the lattice symmetries.

Finally, under the microscopic time-reversal symmetry (Eq. 8), we have

$\displaystyle\mathbb{T}~{}:~{}\chi({\bf x},t)\rightarrow\chi^{\prime}({\bf
x},t)$ $\displaystyle=i\gamma^{1}~{}\Sigma_{13}\zeta_{2}~{}K~{}\chi({\bf
x},-t)$ (42)

such that $\mathbb{T}^{2}=-\mathbb{I}_{16}$ and thereby accounting for the
Kramers’ degeneracy for the $j=3/2$ orbitals. Notably, this transformation is
proportional to a simultaneous emergent time reversal, $\mathcal{T}^{\prime}$
(Eq. 39c) combined with a SU(8) rotation by $\Sigma_{13}\zeta_{2}$ and a
Lorentz boost.

##  IV Short range Interactions

Having described the free low energy theory for the electrons and its enhanced
IR symmetries, we now turn to the effect of interactions on them. More
precisely we consider the effect of short-range four-fermion interactions. A
generic form of such interaction Hamiltonian obtained from an underlying
multi-orbital Hubbard-type model for the lattice fermions is given by

$\displaystyle H_{\rm int}=$ $\displaystyle\int d^{2}{\bf x}d^{2}{\bf
x^{\prime}}~{}V_{ijkl}({\bf x-x^{\prime}})\chi_{i}({\bf
x})^{\dagger}\chi_{j}({\bf x^{\prime}})^{\dagger}\chi_{k}({\bf
x})\chi_{l}({\bf x^{\prime}})$
$\displaystyle~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}+\cdots$
(43)

where $V_{ijkl}$ denotes potential ($i,j,k,l$ collectively spans over the
different indices) and $\cdots$ corresponds to more irrelevant higher fermion
interactions. We assume that the interactions are short-ranged in the sense
$V_{ijkl}$ is only appreciable for ${\bf x}$ and ${\bf x^{\prime}}$ being
proximate with a suitable UV regulation. Further, we assume that the form of
$V$ is constrained enough that at low energy it has the full SU(8) symmetry.
Long-ranged Coulomb interactions as well as on-site Hubbard repulsion, for
example, have such SU(8), symmetry. This immediately means that even in the
presence of these short-range interactions, all the flavour currents
$J_{\mu,a}$ (Eq. 33), in addition to the electronic current $J^{charge}_{\mu}$
(Eq. 40) remain conserved unless spontaneously broken.

While short-range quartic interactions are perturbatively irrelevant at the
free Dirac fixed point Herbut (2006); Vafek and Vishwanath (2014), on cranking
them up they lead to phase transitions possibly gapping out the fermions and
associated spontaneous breaking of the symmetries of the free Dirac theory. In
the rest of this paper, we provide an understanding of the phases that are
obtained by condensing various fermion bilinears which do not carry a net
electronic charge, i.e. invariant under the U(1) described in Eq. 40 and
therefore have the form

$\displaystyle\Delta^{a}=-i\langle\bar{\chi}\mathcal{M}_{a}\chi\rangle\neq 0$
(44)

where $\mathcal{M}_{a}$ are $16\times 16$ mass matrices such that
$\gamma_{0}\mathcal{M}_{a}$ anti-commutes with both the Dirac matrices–
$\alpha_{1}$ and $\alpha_{2}$ – given by Eq. 27. . This leaves out another
important class of fermion bilinears symbolically of the form
$\langle\chi~{}\chi\rangle$ that describes different types of superconductors
which will be taken up elsewhere Chaubey  _et al._ .

For such a U(1) invariant massive phase, the mean-field Hamiltonian is given
by

$\displaystyle{S}_{MF}={S}_{0}+{S}_{m}=-i\int d^{2}{\bf
r}d\tau~{}\bar{\chi}({\bf
r})\left[v_{F}\not{\partial}-\Delta^{a}\mathcal{M}_{a}\right]\chi({\bf r}).$
(45)

There are 64 such linearly independent $\mathcal{M}_{a}$ matrices that can be
broadly classified into two classes according to their transformation
properties under SU(8). The first class contains a single SU(8) singlet given
by Eq. 4 while the second class corresponds to 63 SU(8) adjoint multiplet

$\displaystyle-i\langle\bar{\chi}\mathcal{P}_{a}\chi\rangle,~{}~{}~{}~{}~{}a=1\cdots,63.$
(46)

and $\mathcal{P}_{a}$ being the SU(8) generators (Eq. 31).

The microscopic operators that characterise the same broken symmetry – hence
can serve as valid order-parameters for appropriate symmetry broken phases–
have the same transformation properties as the bilinear and hence are
proportional to each other. In principle, the microscopic operators can also
get contributions from the conserved currents of the same symmetry, but
usually, such currents decay faster than the bilinears, and hence at long
distances, the correlation function of the microscopic fields is determined by
that of the field theory bilinear Hermele  _et al._ (2005).

The correspondence between the microscopic operators and the low-energy
fermion bilinears is obtained by comparing their symmetry transformations. In
particular, the transformation of the low-energy fermion bilinears under
microscopic symmetries can be used to systematically uncover the nature of the
phases proximate to the Dirac semimetal. Bilinears that are related by
microscopic symmetries together constitute a single phase. This leads to the
classification of the fermion masses in terms of broken microscopic symmetries
and/or anomalies. In the present case, we find that the 64 masses group
together to give rise to 24 phases which we now turn to understand in detail.

##  V Classification of the fermionic bilinears : phases and transitions

This task of classifying the bilinears according to their microscopic
symmetries (and hence identifying the phases) is much more involved compared
to the same problem in graphene Ryu  _et al._ (2009) since the SOC mixes the
$j=3/2$ flavour and the real spaces in a non-trivial way. As a result, the
lattice translations, ${\bf T_{1}},{\bf T_{2}}$ (Table 5) do not necessarily
commute with the point group symmetries such as ${\bf C_{3}},{\bf S_{6}},{\bf
C^{\prime}_{2}},\bm{\sigma_{d}},{\bf I}$ (Table 5) and microscopic time
reversal, $\mathbb{T}$, (Eq. 8). This is clear by looking at the
transformations of the Dirac spinor, $\chi$, under the above lattice
symmetries (Eq. 41 with the detailed forms given by Eqs. 322 \- 342). Hence we
need to analyse the action of the entire set of transformations generated by
the space group and microscopic time reversal on $\chi$ to understand the
transformation of the fermion bilinears in Eq. 44. The resultant symmetry
group, we dub as IR space group.

##### IR Space group :

To understand the structure of this IR space group, we note that the $j=3/2$
electron states transform under a double group representation of $D_{3d}$,
which has 24 elements. Since the translations do not commute with point group
transformations, corresponding to any element (say, $\mathbb{S}$) of the
double group of $D_{3d}$, there are four elements in the IR space group which
can be constructed as (say)
$\mathbb{S},\mathbf{T_{1}}\mathbb{S},\mathbb{S}\mathbf{T_{2}}$ and
$\mathbf{T_{1}}\mathbb{S}\mathbf{T_{2}}$ by composing it with translations,
${\bf T_{1}}$ and $\mathbf{T_{2}}$ (Table 5). So, the group of microscopic
symmetries that act on the spinors has 96 elements in total. These elements
can be divided into 20 conjugacy classes and hence there are 20 different
irreducible representations of the IR space group. Among these 20, only 10 has
$+$ve character for $2\pi$ rotation. Since the fermion bilinears are always
invariant under a $2\pi$ rotation, we consider only these irreps for the
classification of the masses.

Among these 10 irreps of the IR space group, four are 1-dimensional
($\mathcal{A}_{1g},\mathcal{A}_{2g},\mathcal{A}_{1u},\mathcal{A}_{2u}$), two
are 2-dimensional ($\mathcal{E}_{g},\mathcal{E}_{u}$) and four irreps are
3-dimensional
($\mathcal{T}_{1g},\mathcal{T}_{2g},\mathcal{T}_{1u},\mathcal{T}_{2u}$).
Following conventional notation, the subscripts $1(2)$ and $g(u)$ denote that
the irrep is even (odd) under rotation, $\mathbf{C_{2}^{\prime}}$ and
inversion, ${\bf I}$, respectively (Table. 5). Further, to incorporate
microscopic time reversal, $\mathbb{T}$ (Eq. 8), we will add a superscript
$e(o)$ (e.g., $\mathcal{A}_{1u}^{e(o)}$) to denote the particular irrep is
even (odd) under time-reversal. The details of these irreps are given in
Appendix G.

The central question we now turn to investigate in the rest of the paper are
the nature of the phases obtained by condensing the fermion bilinears
$\langle\bar{\chi}\mathcal{P}_{a}\chi\rangle$. Since this is decided by the
microscopic symmetries, we decompose the above bilinears in terms of the
irreducible representation of the microscopic symmetries Hermele  _et al._
(2005). We find that these 64 bilinears break up into 27 irreps of the space
group among which there are six 1-dimensional representations, five
2-dimensional representations and sixteen 3-dimensional representations. This
can be obtained as follows.

Starting with Eqs. 41 and 42, we can derive the action of the microscopic
symmetries on the members of the 64 fermion bilinears, which leads to the
following structure. Under the action of a lattice symmetry transformation
(say $\mathbb{S}$, corresponding to Table 5 and Eq. 41), a fermion bilinear of
the form $\bar{\chi}\mathcal{P}_{a}\chi$ (with $\mathcal{P}_{a}$ given by Eq.
31) transforms as

$\displaystyle-i\bar{\chi}\mathcal{P}_{a}\chi=\chi^{\dagger}\gamma_{0}\Sigma_{i}\zeta_{j}\chi\xrightarrow{\mathbb{S}}\chi^{\dagger}(\Omega_{\mathbb{S}}^{f\dagger}\Sigma_{i}\Omega_{\mathbb{S}}^{f})\otimes(\Omega_{\mathbb{S}}^{c\dagger}\gamma_{0}\zeta_{j}\Omega_{\mathbb{S}}^{c})\chi$
(47)

and under the action of microscopic time reversal (Eqs. 8 and 42), we get

$\displaystyle\chi^{\dagger}\gamma_{0}\Sigma_{i}\zeta_{j}\chi\xrightarrow{\mathbb{T}}\chi^{\dagger}(\Sigma_{13}\Sigma_{i}^{*}\Sigma_{13})\otimes(\tau_{1}\sigma_{0}~{}\gamma_{0}^{*}\zeta_{j}^{*}~{}\tau_{1}\sigma_{0})\chi$
(48)

The above structure shows that the action of each symmetry is implemented as a
product of the transformations in the flavour and chiral spaces, i.e., for
lattice symmetries,

$\displaystyle\Sigma_{i}$
$\displaystyle\xrightarrow{\mathbb{S}}\Omega_{\mathbb{S}}^{f\dagger}\Sigma_{i}\Omega_{\mathbb{S}}^{f};\quad\quad\gamma_{0}\zeta_{j}$
$\displaystyle\xrightarrow{S}\Omega_{S}^{c\dagger}\gamma_{0}\zeta_{j}\Omega_{S}^{c}$
(49)

and for time reversal

$\displaystyle\Sigma_{i}$
$\displaystyle\xrightarrow{\mathbb{T}}\Sigma_{13}\Sigma_{i}^{*}\Sigma_{13};\quad\quad\gamma_{0}\zeta_{j}$
$\displaystyle\xrightarrow{\mathbb{T}}\tau_{1}\sigma_{0}~{}\gamma_{0}^{*}\zeta_{j}^{*}~{}\tau_{1}\sigma_{0}.$
(50)

However, due to the SOC, the real space transformations get non-trivially
coupled to the flavour space and this resultant inter-locking is reflected in
the form of the transformation matrices, particularly in Eq. 49 where the
$\Omega_{S}^{f}$ reflects the degree of interlocking between real and flavour
spaces. In fact, it is this non-trivial structure that distinguishes the spin-
orbit coupled Dirac fermions– the topic of the present work– with multi-
flavour (larger $N_{F}$) generalisation of graphene where such SOC is usually
neglected (except for the spin-Hall effect Kane and Mele (2005) and related
phases where SOC is essential).

| Irrep | $\Sigma_{p}$ | $\mathbb{T}$
---|---|---|---
Singlets | $\mathcal{A}_{1g}^{e}$ | $\Sigma_{0}=\mathbf{1}_{4}$ | even
| $\mathcal{A}_{2g}^{o}$ | $\Sigma_{45}$ | odd
Doublet | $\mathcal{E}_{u}^{e}$ | $\\{\Sigma_{4},\Sigma_{5}\\}$ | even
Triplets | $\mathcal{T}_{1g}^{e}$ | $\\{\Sigma_{3},\Sigma_{1},-\Sigma_{2}\\}$ | even
| $\mathcal{T}_{2g}^{o}$ | $\\{\Sigma_{12},\Sigma_{23},\Sigma_{13}\\}$ | odd
| $\mathcal{T}_{1u}^{o}$ | $\\{\Sigma_{35},\frac{\sqrt{3}\Sigma_{14}}{2}-\frac{\Sigma_{15}}{2},\frac{\sqrt{3}\Sigma_{24}}{2}+\frac{\Sigma_{25}}{2}\\}$ | odd
| $\mathcal{T}_{2u}^{o}$ | $\\{\Sigma_{34},-\frac{\Sigma_{14}}{2}-\frac{\sqrt{3}\Sigma_{15}}{2},\frac{\Sigma_{24}}{2}-\frac{\sqrt{3}\Sigma_{25}}{2}\\}$ | odd
Table 6: Irreps of the matrices SU(4) flavour space.

Due to the direct product structure of the above transformations, we can
analyze the action of the microscopic symmetries on the flavor and the chiral
spaces separately and then use Clebsch-Gordon decomposition, e.g., if the
matrix $\Sigma_{i}$ transforms in some irreducible representation (say,
$\mathcal{D}_{1}$) and $\gamma_{0}\zeta_{j}$ transforms in some other irrep
(say, $\mathcal{D}_{2}$), then the bilinear
$\bar{\chi}\Sigma_{i}\zeta_{j}\chi$ transforms in the product representation
$\mathcal{D}_{1}\otimes\mathcal{D}_{2}$. This direct product representation is
reducible in general which then is reduced into a direct sum representation.

Table 6 shows the $\Sigma_{i}$ matrices in the SU(4) flavour space and their
irreducible representations under the IR space-group transformations as well
as TR. The transformations of the $\gamma_{0}\zeta_{j}$ (Eq. 30) matrices in
the SU(2) chiral space are written in Table 7. Details of their symmetry
transformations are given in Appendix G.

| Irrep | $\gamma_{0}\zeta_{j}$ | $\mathbb{T}$
---|---|---|---
Singlet | $\mathcal{A}_{2g}^{o}$ | $\gamma_{0}$ | odd
Triplet | $\mathcal{T}_{1g}^{e}$ | $\\{\gamma_{0}\zeta_{1},\gamma_{0}\zeta_{2},\gamma_{0}\zeta_{3}\\}$ | even
Table 7: Irreps for the matrices in the SU(2) chiral space.

The 64 fermion bilinears are subdivided into three groups depending on the
participation of the flavour, $\Sigma_{i}$ and the chiral elements,
$\zeta_{j}$, in the fermion bilinear (Eq. 44) which, in turn, dictate their
transformation properties under the microscopic symmetries. These are – (1)
Group-1 : the chiral masses composed of flavour singlets, (2) Group-2 : the
flavour masses composed of chiral singlets, and, (3) Group-3 : the mixed
masses which are composed of non-trivial combinations of both the flavour and
chiral sectors. Here we list the masses in the groups mentioned above. The
following sections contain a detailed discussion of their physics.

###  V.1 Group-1 : The chiral masses

There are four masses of the form $-i\langle\bar{\chi}\chi\rangle$ and
$-i\langle\bar{\chi}\zeta_{i}\chi\rangle$ ($i=1,2,3)$ that are invariant under
the SU(4) flavour symmetry and charge conservation which are broken down by
the lattice symmetries and TR as ${\bf 4}={\bf 1}\oplus{\bf 3}$, i.e.,

$\displaystyle\left[\mathcal{A}_{1g}^{e}\right]^{\Sigma}\otimes\left[\mathcal{A}_{2g}^{o}\right]^{\zeta}=\mathcal{A}_{2g}^{o}$
(51a)
$\displaystyle\left[\mathcal{A}_{1g}^{e}\right]^{\Sigma}\otimes\left[\mathcal{T}_{1g}^{e}\right]^{\zeta}=\mathcal{T}_{1g}^{e}$
(51b)

where $\left[\cdots\right]^{\Sigma}$ and $\left[\cdots\right]^{\zeta}$ denote
the two irreducible representations taken from Tables 6 and 7 respectively.

As mentioned in the overview (Table 1), the singlet represents an Integer
Chern insulator (ICI) phase, the triplet corresponds to the three stripy
charge density waves (CDW) (Fig. 6). Since the flavour index plays no role, we
can quantitatively compare the spinless version of the present problem
(Appendix K) with spinless electrons in graphene Ryu  _et al._ (2009); Hou
_et al._ (2007). In the case of graphene, the irreducible representation
splits up into ${\bf 4}={\bf 1}\oplus{\bf 1}\oplus{\bf 2}$ where the two
singlets represent the ICI phaseHaldane (1988); Qi  _et al._ (2006) and
staggered (Néel) CDW, and the doublet corresponds to the two Kekule patterns
Ryu  _et al._ (2009); Hou  _et al._ (2007). This is very different from the
present case and this provides a startling example where the microscopic SOC
changes the low energy symmetry implementation. We discuss these masses in
more detail in Sec. VI.

###  V.2 Group-2 : The flavour masses

There are 15 masses of the form $i\langle\bar{\chi}\Sigma_{j}\chi\rangle$
where $\Sigma_{j}$ are the 15 generators of SU(4) as given by Eq. 259 in
Appendix C. Under microscopic symmetries, they break up into six different
irreps, i.e., ${\bf 15=1\oplus 3\oplus 3\oplus 3\oplus 2\oplus 3}$ given by
Eqs. 62, 63 and 64. As pointed out in the overview section (Table 2) these
correspond to six generalised spin-Hall phases that are summarised in Table 2
while the details are given in Sec. VII.

In order to explore the nature of the resultant phases, it is useful to
understand in detail the mathematical structure of the implementation of the
various microscopic symmetries that break up the 15 flavour masses further
into different irreducible representations. Starting with TR, ten of the
flavour masses are TR even and are of the form

$\displaystyle-i\langle\bar{\chi}\Sigma_{j}\chi\rangle,~{}~{}~{}~{}~{}j=12,13,14,15,23,24,25,34,35~{}{\rm
and}~{}45.$ (52)

These transform into each other under an adjoint representation of an SO(5)
sub-group (generated by themselves, Eq. 52) of the SU(4) flavour group.

The other five are TR odd and transform under a vector representation of the
same SO(5) and are given by

$\displaystyle-i\langle\bar{\chi}\Sigma_{j}\chi\rangle~{}~{}\text{with}~{}~{}j=1,2,3,4~{}{\rm
and}~{}5$ (53)

Next, the lattice inversion (Table 5), $\mathbf{I}$, breaks each of the above
two sets further. Out of the 10 adjoint ones (Eq. 52), four

$\displaystyle-i\langle\bar{\chi}\Sigma_{j}\chi\rangle~{}~{}{\rm
with}~{}~{}j=45,12,13~{}{\rm and}~{}23$ (54)

are even under inversion, $\mathbf{I}$, while the other six

$\displaystyle-i\langle\bar{\chi}\Sigma_{j}\chi\rangle~{}~{}{\rm
with}~{}~{}j=14,15,24,25,34~{}{\rm and}~{}35$ (55)

are odd under it. For the 5 vector masses (Eq. 53), two

$\displaystyle-i\langle\bar{\chi}\Sigma_{j}\chi\rangle~{}~{}{\rm
with}~{}~{}j=4~{}{\rm and}~{}5$ (56)

are inversion odd, while three

$\displaystyle-i\langle\bar{\chi}\Sigma_{j}\chi\rangle~{}~{}{\rm
with}~{}~{}j=1,2~{}{\rm and}~{}3$ (57)

are even.

Each of the above four subsets (Eqs. 54-57) is individually closed under a
U(1) $\otimes$ SU(2) sub-group of the SO(5) (eq. 52) that is generated
respectively by

$\displaystyle\Sigma_{45}$ (58)

and

$\displaystyle\\{\Sigma_{12},\Sigma_{13},\Sigma_{23}\\}.$ (59)

In particular, in the first subset (Eq. 54), the first mass is a U(1)
$\otimes$ SU(2) singlet while the rest are only U(1) singlets that transform
as spin-1 under the SU(2). The three masses in Eq. 57 are U(1) singlets and an
SU(2) triplet, while the two masses in Eq. 56 are SU(2) singlets and transform
into each other under the U(1). Finally the six masses in Eq. 55, decompose
into two SU(2) triplets :

$\displaystyle\\{-i\langle\bar{\chi}\Sigma_{14}\chi\rangle,-i\langle\bar{\chi}\Sigma_{24}\chi\rangle,-i\langle\bar{\chi}\Sigma_{34}\chi\rangle\\}$
(60)

and

$\displaystyle\\{-i\langle\bar{\chi}\Sigma_{15}\chi\rangle,-i\langle\bar{\chi}\Sigma_{25}\chi\rangle,-i\langle\bar{\chi}\Sigma_{35}\chi\rangle\\}.$
(61)

The three components of the first triplet mix with their corresponding
components of the second triplet under the U(1) generated by Eq. 58.

Now, considering the other lattice symmetries the above four subsets (Eqs.
54-57) break up further into the irreps of the microscopic symmetry group as
follows.

The four masses in Eq. 54 break up into a singlet and a triplet as

$\displaystyle\left[\mathcal{A}_{2g}^{o}\right]^{\Sigma}\otimes\left[\mathcal{A}_{2g}^{o}\right]^{\zeta}=\mathcal{A}_{1g}^{e}$
(62a)
$\displaystyle\left[\mathcal{T}_{2g}^{o}\right]^{\Sigma}\otimes\left[\mathcal{A}_{2g}^{o}\right]^{\zeta}=\mathcal{T}_{1g}^{e}$
(62b)

whose explicit forms are given in Eqs. 79 and 89 respectively and correspond
to two different kinds of quantum spin-octupole phases discussed in Sec.
VII.1.1 and VII.1.2.

The six inversion-odd masses (Eq. 55), on the other hand, break up into two
triplets

$\displaystyle\left[\mathcal{T}_{1u}^{o}\right]^{\Sigma}\otimes\left[\mathcal{A}_{2g}^{o}\right]^{\zeta}=\mathcal{T}_{2u}^{e}$
(63a)
$\displaystyle\left[\mathcal{T}_{2u}^{o}\right]^{\Sigma}\otimes\left[\mathcal{A}_{2g}^{o}\right]^{\zeta}=\mathcal{T}_{1u}^{e}.$
(63b)

given by a linear combination of the two triplets in Eq. 60 and 61 as given by
Eqs. 96 and 100 respectively. These too correspond to spin-octupole Hall
phases, albeit with interesting fine-tuned gapless points for a special
combination of the three components of the triplets as discussed in Sec.
VII.1.3.

The doublet and the triplets in Eqs. 56 and 57 remain intact and result in

$\displaystyle\left[\mathcal{E}_{u}^{e}\right]^{\Sigma}\otimes\left[\mathcal{A}_{2g}^{o}\right]^{\zeta}=\mathcal{E}_{u}^{o}$
(64a)
$\displaystyle\left[\mathcal{T}_{1g}^{e}\right]^{\Sigma}\otimes\left[\mathcal{A}_{2g}^{o}\right]^{\zeta}=\mathcal{T}_{2g}^{o}$
(64b)

with explicit forms being given by Eqs. 112 and 116 respectively. These phases
break time-reversal symmetry and describe quantum spin-quadrupole Hall phases
as described in Sec. VII.2.

###  V.3 Group 3 : The mixed masses

Finally, the largest set of masses is obtained by taking the direct product of
the flavour multiplets and the chiral multiplets. There are 45 such masses of
the form $-i\langle\bar{\chi}\Sigma_{i}\zeta_{j}\chi\rangle$ where
$\Sigma_{i}$ are the fifteen generators of flavor SU(4) (see Appendix C) and
$j=1,2,3$. Therefore these masses transform into each other under
transformations of the SU(4) $\otimes$ SU(2) subgroup of SU(8) of the free
Dirac theory. Their group decomposition to irreducible representations under
the microscopic symmetries is given by

$\displaystyle\left[\mathcal{A}_{2g}^{o}\right]^{\Sigma}\otimes\left[\mathcal{T}_{1g}^{e}\right]^{\zeta}={\color[rgb]{1,0,0}\mathcal{T}_{2g}^{o}}$
(65a)
$\displaystyle\left[\mathcal{E}_{u}^{e}\right]^{\Sigma}\otimes\left[\mathcal{T}_{1g}^{e}\right]^{\zeta}={\color[rgb]{1,0,0}\mathcal{T}_{1u}^{e}}\oplus{\color[rgb]{1,0,0}\mathcal{T}_{2u}^{e}}$
(65b)
$\displaystyle\left[\mathcal{T}_{1g}^{e}\right]^{\Sigma}\otimes\left[\mathcal{T}_{1g}^{e}\right]^{\zeta}=\mathcal{T}_{1g}^{e}\oplus\mathcal{T}_{2g}^{e}\oplus\mathcal{E}_{g}^{e}\oplus{\color[rgb]{1,0,0}\mathcal{A}_{1g}^{e}}$
(65c)
$\displaystyle\left[\mathcal{T}_{2g}^{o}\right]^{\Sigma}\otimes\left[\mathcal{T}_{1g}^{e}\right]^{\zeta}=\mathcal{T}_{1g}^{o}\oplus\mathcal{T}_{2g}^{o}\oplus\mathcal{E}_{g}^{o}\oplus{\color[rgb]{1,0,0}\mathcal{A}_{2g}^{o}}$
(65d)
$\displaystyle\left[\mathcal{T}_{1u}^{o}\right]^{\Sigma}\otimes\left[\mathcal{T}_{1g}^{e}\right]^{\zeta}={\color[rgb]{1,0,0}\mathcal{T}_{1u}^{o}}\oplus{\color[rgb]{1,0,0}\mathcal{T}_{2u}^{o}}\oplus{\color[rgb]{1,0,0}\mathcal{E}_{u}^{o}}\oplus\mathcal{A}_{1u}^{o}$
(65e)
$\displaystyle\left[\mathcal{T}_{2u}^{o}\right]^{\Sigma}\otimes\left[\mathcal{T}_{1g}^{e}\right]^{\zeta}={\color[rgb]{1,0,0}\mathcal{T}_{1u}^{o}}\oplus{\color[rgb]{1,0,0}\mathcal{T}_{2u}^{o}}\oplus{\color[rgb]{1,0,0}\mathcal{E}_{u}^{o}}\oplus\mathcal{A}_{2u}^{o}.$
(65f)

The dimension of the representation depends non-trivially on the details of
the spin-orbital locking, which, in turn, is reflected in the nature of
different density wave phases that these masses lead to. These density wave
phases mainly come in two varieties. Out of the total of 45 mixed masses, 18
(marked in black in Eq. 65) generically have at least four gapless fermionic
modes protected by a subgroup of SU(8), often in conjunction with lattice
symmetries. Thus they describe different kinds of density wave Dirac semimetal
(summarised in Table 4). The rest 27 (marked in red in Eq. 65) generically
consists of density wave-insulators (summarised in Table 3). Two of the
insulators, both singlets – $\mathcal{A}^{e}_{1g}$ and $\mathcal{A}^{o}_{2g}$,
have edge modes whose signature is evident from appropriate Chern-Simons
terms.

Before delving into the details of the resultant phases in this category in
Section. VIII, we summarise the general structure of these masses and their
classification here under various microscopic symmetries leading up to the
decomposition in Eq. 65. To this end, starting with microscopic TR,
$\mathbb{T}$ (Eq. 8), the 45 bilinears are divided into two classes with 15 TR
even given by

$\displaystyle\\{-i\langle\bar{\chi}\Sigma_{i}\zeta_{1}\chi\rangle,-i\langle\bar{\chi}\Sigma_{i}\zeta_{2}\chi\rangle,-i\langle\bar{\chi}\Sigma_{i}\zeta_{3}\chi\rangle\\}$
(66)

where $i=1,2,3,4,5$ (same $\Sigma_{i}$ content as in Eq. 53) and 30 TR odd
given by

$\displaystyle\\{-i\langle\bar{\chi}\Sigma_{i}\zeta_{1}\chi\rangle,-i\langle\bar{\chi}\Sigma_{i}\zeta_{2}\chi\rangle,-i\langle\bar{\chi}\Sigma_{i}\zeta_{3}\chi\rangle\\}$
(67)

with the $i$ indices being given in Eq. 52.

Each of these two sets, under lattice inversion, $\mathbf{I}$ (Table. 5),
break up into two subsets as odd and even under $\mathbf{I}$. Out of the set
of 15 in Eq. 66 the six odd ones are given by

$\displaystyle\\{-i\langle\bar{\chi}\Sigma_{4}\zeta_{i}\chi\rangle,-i\langle\bar{\chi}\Sigma_{5}\zeta_{i}\chi\rangle\\}$
(68)

while nine even ones are given by

$\displaystyle\\{-i\langle\bar{\chi}\Sigma_{1}\zeta_{i}\chi\rangle,-i\langle\bar{\chi}\Sigma_{2}\zeta_{i}\chi\rangle,-i\langle\bar{\chi}\Sigma_{3}\zeta_{i}\chi\rangle\\}$
(69)

where $i=1,2,3$.

On the other hand, the set of 30 masses in Eq. 67 breaks up into two subsets.
One of the subsets contains twelve masses that are even under $\mathbf{I}$ and
is given by

$\displaystyle\\{-i\langle\bar{\chi}\Sigma_{45}\zeta_{i}\chi\rangle,-i\langle\bar{\chi}\Sigma_{12}\zeta_{i}\chi\rangle,-i\langle\bar{\chi}\Sigma_{23}\zeta_{i}\chi\rangle,-i\langle\bar{\chi}\Sigma_{13}\zeta_{i}\chi\rangle\\}.$
(70)

The other subset containing eighteen $\mathbf{I}$ odd masses is

$\displaystyle\\{-i\langle\bar{\chi}\Sigma_{14}\zeta_{i}\chi\rangle,-i\langle\bar{\chi}\Sigma_{15}\zeta_{i}\chi\rangle,-i\langle\bar{\chi}\Sigma_{24}\zeta_{i}\chi\rangle,$
$\displaystyle\quad-i\langle\bar{\chi}\Sigma_{25}\zeta_{i}\chi\rangle,-i\langle\bar{\chi}\Sigma_{34}\zeta_{i}\chi\rangle,-i\langle\bar{\chi}\Sigma_{35}\zeta_{i}\chi\rangle\\}.$
(71)

Further application of lattice symmetries (Appendix G) break these up into
singlets, doublets and triplets as follows. The TR even and inversion odd
subset (Eq. 68) of six decomposes into two triplets given by Eq. 65b which
correspond to two different zig-zag spin-quadrupolar density wave insulators
given by Eq. 135. Similarly, the nine TR and inversion even masses break up
into two triplets, one doublet and one singlet as given by Eq. 65c. They
represent spin-quadrupole density waves. While the singlet corresponds to an
insulator (Eq. 122) with quantized spin-octupole filtered edge modes, the rest
(Eqs. 188, VIII.2.3 and VIII.2.4) are partially gapless semimetals.

The subset of 12 TR odd and inversion even masses (Eq. 70) break up into three
triplets, one doublet and one singlet under the action of the lattice
symmetries (Eqs. 65a, 65d). The singlet (an anomalous Hall insulator) and one
of the triplets represent insulating ferro (uniform) (Eq. 125) and stripy (Eq.
132) density wave ordering of spin-octupoles respectively. The doublet (Eq.
VIII.2.5) corresponds to ferro spin-octupole semimatal and the other two
triplets (Eqs. 172, VIII.2.2) correspond to different stripy spin-octupole
density wave semimetals.

Finally, the 18 TR and inversion odd masses, break up under lattice symmetries
into four triplets, two doublets and two singlets given by Eqs. 65e and 65f.
Out of them, the two singlets (Eqs. VIII.2.1 and VIII.2.1) correspond to
staggered (“Néel”) spin-octupole density-wave semimetal. The two doublets
(Eqs. VIII.1.3 and VIII.1.3) on the other hand, both correspond to Néel spin-
octupole density wave insulators. As discussed below Eq. VIII.1.3, they can be
rotated into each other via a U(1) transformation generated by $\Sigma_{45}$
within the flavour space. Given this fact and they break the same symmetries,
the two doublets correspond to the same phase and are not distinct from each
other. Similar arguments hold for the two sets of triplets, each of which
represents zig-zag spin-octupole density wave insulators. The two
$\mathcal{T}^{o}_{1u}$ triplets (Eqs. VIII.1.6 and VIII.1.6) can be
continuously connected without change of symmetry and hence represent the same
phase. Similarly the two $\mathcal{T}^{o}_{2u}$ triplets (Eqs. VIII.1.6 and
VIII.1.6) give the same phase.

##  VI Group-1 : The chiral Masses

There are four chiral masses that are given in Eqs. 51a and 51b. Their
transformations under microscopic symmetries are given in Appendix G. In
accordance with Eqs. 51a and 51b, these are divided into a singlet (TR and
reflection odd) and a triplet (TR and inversion even) under the space-group
symmetries while they are all singlets under the flavour SU(4). Here we
discuss the physics of these masses.

###  VI.1 The SU(8) symmetric Integer Chern Insulator

Figure 5:  The mean field hopping model for the Integer Chern insulator. The
second neighbour hoppings (in blue) are generated via the spontaneous symmetry
breaking and the hopping amplitudes along the directions of the arrows (in
local basis) are $i=\sqrt{-1}$. The same hopping pattern also holds for the
singlet spin-octupole Hall mass leading to the octupolar Hall effect (Eq. 83)
except in that case the hopping amplitude is given by $i\Sigma_{45}$.

The SU(8) singlet mass,

$\displaystyle\Delta_{ICI}=-i\langle\bar{\chi}\chi\rangle,$ (72)

given by Eq. 51a is odd under the action of $\mathbb{T}$,
$\mathbf{C_{2}^{\prime}}$ and $\mathbf{\sigma_{d}}$. This suggests that this
mass is the analog of the Chern mass for graphene Haldane (1988); Qi  _et al._
(2006) which shows the integer quantum Hall effect and hence represents the
ICI phase. Indeed, minimally coupling the electrons in the massive Dirac
action (Eq. 45) for the above Chern mass, $\Delta_{ICI}$ to a U(1) probe gauge
field, $A_{\mu}$, that couples to the electronic charge and integrating out
the gapped fermions, we get a U(1) Chern-Simons term with the (Euclidean)
Lagrangian density :

$\displaystyle\mathcal{L}_{\rm
CS}=i\frac{N_{F}\rm{Sgn}[\Delta_{ICI}]}{4\pi}\epsilon^{\mu\nu\lambda}A_{\mu}\partial_{\nu}A_{\lambda}$
(73)

where $N_{F}=4$ is the number of SU(4) flavors (i.e. number of four component
Dirac fermion fields). Thus, all the flavors contribute the same amount to the
charge Hall conductivity resulting in, $\sigma_{xy}=N_{F}\frac{e^{2}}{2\pi}$
Thouless  _et al._ (1982).

The nature of symmetry breaking can be analysed by considering the low energy
projection of the microscopic current operators on the lattice Haldane (1988);
Raghu  _et al._ (2008). In particular, starting with the microscopic orbitals
in the local basis, $\phi(\boldsymbol{\mathcalligra{r}}\,_{\mathcal{S}})$,
given by Eq. 17, the hopping operator on the next nearest neighbour (NNN)
bonds– say the blue bond in Fig. 5 from site $B_{2}$ to $B_{1}$– is given by
Haldane (1988)

$\displaystyle\mathcal{B}_{B_{2}B_{1}}=\phi^{\dagger}(\boldsymbol{\mathcalligra{r}}\,_{B_{1}})\phi(\boldsymbol{\mathcalligra{r}}\,_{B_{2}})$
(74)

The form in the global basis (in terms of the $j=3/2$ orbitals) can be easily
obtained via Eq. 17 and using the forms of
$\mathcal{G}(\boldsymbol{\mathcalligra{r}}\,_{\mathcal{S}})$ given in Appendix
D.

In the low energy limit, Eq. 74 is equal to

$\displaystyle\mathcal{B}_{B_{2}B_{1}}=\frac{1}{2\sqrt{3}}\bar{\chi}~{}\chi+\cdots$
(75)

where $\cdots$ represent higher order terms. Therefore for $\Delta_{ICI}\neq
0$, we have an imaginary second neighbour hopping whose sign structure is
given by Fig. 5. This leads to finite bond current such that the total gauge
invariant loop current per hexagon is indeed zero (mod $2\pi$). The loop
currents, therefore, form a Z2 order parameter proportional to the mass,
$\Delta_{ICI}$. Such Z2 order parameters allow domain walls as one-dimensional
topological defects in two-dimensional systems across which the sign of the
mass changes. As is evident from Eq. 73, the edge modes have opposite
chirality in the two cases and hence the domain wall is associated with chiral
gapless edge modes that are exponentially localised along the domain wall
Callan Jr and Harvey (1985).

###  VI.2 The SU(4) symmetric stripy charge density wave Insulator

Figure 6: Stripy CDW for $-i\Braket{\bar{\chi}\zeta_{1}\chi}\neq 0$. Here red
circles and blue dots represent opposite charge densities at the honeycomb
sites.

The three other SU(4) flavour invariant masses given by Eq. 51b form a triplet
($\mathcal{T}^{e}_{1g}$) which is even under TR symmetry, $\mathbb{T}$ as well
as inversion, ${\bf I}$, about the plaquette centre. These are given by

$\displaystyle\\{-i\langle\bar{\chi}\zeta_{1}\chi\rangle,-i\langle\bar{\chi}\zeta_{2}\chi\rangle,-i\langle\bar{\chi}\zeta_{3}\chi\rangle\\}.$
(76)

They transform into each other under various lattice rotations and reflections
as a triplet as shown in Appendix G. $\zeta_{i}$ ($i=1,2,3$) of course
generates an SU(2) (see Eq. 30) which is broken down by the lattice symmetries
to a triplet for the space group.

The three bilinears have the same symmetry as that of the three stripy CDW
order as shown in Fig. 6 for $-i\langle\bar{\chi}\zeta_{1}\chi\rangle$. The
other two can be obtained by $\mathbf{C_{3}}$ rotations. Indeed the analysis
of the above three fermion bilinears in the global basis (Eq. 6) confirms the
symmetry analysis in identifying the above stripy CDW masses. In particular,
starting with the electron operators in the global basis i.e., $\psi$ (Eq. 6),
the projected charge density operators on different sub-lattices have the
following form :

$\displaystyle:\psi^{\dagger}(\boldsymbol{\mathcalligra{r}}\,_{\mathcal{S}})\psi(\boldsymbol{\mathcalligra{r}}\,_{\mathcal{S}}):~{}=\begin{cases}-i\bar{\chi}\xi_{1}\chi&\text{
For $\mathcal{S}=B_{2},A_{2}$ }\\\ i\bar{\chi}\xi_{1}\chi&\text{ For
$\mathcal{S}=A_{1},B_{1}$ }\end{cases}$ (77)

where $:\mathcal{O}:$ denotes normal ordering. Integrating out the gapped
fermions in the presence of the mass does not lead to a finite charge Hall
response.

The three matrices $\gamma_{0}\zeta_{i}$ ($i=1,2,3$) pairwise anti-commute
with each other such that the three stripy CDW masses are compatible in the
sense that the fermion gap does not close as the three masses are rotated into
each other under the chiral SU(2) (see Eq. 30) transformations generated by
$e^{i\theta\hat{\bf n}\cdot\mathbf{\zeta}}$. This would suggest that the order
parameter manifold is a unit sphere, $\mathcal{S}^{2}$ similar to collinear
magnetic ordering (with one important difference that the present order
parameter is even under TR unlike magnetic order). The above SU(2) is,
however, broken down by the lattice symmetries which, in terms of the order
parameter, selects out symmetry-allowed points on the sphere,
$\mathcal{S}^{2}$. In particular, the leading order anisotropy of the form

$\displaystyle\mathcal{L}_{aniso}\sim-\mathcal{w}_{1}\Delta_{1}\Delta_{2}\Delta_{3}+\mathcal{w}_{2}\left(\Delta_{1}^{4}+\Delta_{2}^{4}+\Delta_{3}^{4}\right)$
(78)

is symmetry allowed (see Table 11) in the effective action with $\Delta_{i}$
being the amplitude for the three components of the CDW (Eq. 76). This reduces
the order parameter manifold to discrete points on the sphere. The details of
the ordering depend on the signs of the couplings $\mathcal{w}_{1}$ and
$\mathcal{w}_{2}$. Due to the presence of the third-order invariant, the
transition out of the semimetal is expected to be first-order.

We conclude the discussion of the chiral masses with two points. First, it is
useful to compare the four chiral masses with the case of spinless fermions on
the honeycomb lattice with $\pi$-flux at one-quarter filling presented in
appendix. K. The presence of the $\pi$-flux breaks up the chiral space as
${\bf 4}={\bf 1}\oplus{\bf 3}$, as opposed to graphene where the chiral space
is decomposed as ${\bf 4}={\bf 1}\oplus{\bf 1}\oplus{\bf 2}$. In the present
case where the $\pi$-flux is a consequence of SOC, the above SU(4) singlet
masses can be thought of as four copies of the spinless case in Appendix K.
Second, while the three triplet stripy masses are compatible with each other,
i.e., the respective mass matrices mutually anticommute, all of them are
incompatible with the singlet ICI mass since the corresponding matrices (Table
7) pairwise commute. This ensures a phase transition Ryu  _et al._ (2009)
between the two phases which is accompanied by the change in the nature of
broken symmetry as well as the Chern-Simons level (Eq.73) from $N_{F}=4$ (in
the ICI) to $N_{F}=0$ (in the stripy CDW).

##  VII Group-2 : The flavour masses

The 15 flavour masses of the form $-i\langle\bar{\chi}\Sigma_{i}\chi\rangle$
are divided into six irreducible representations by microscopic symmetries
that are given by Eqs. 62, 63 and 64. Each of these 15 masses individually
breaks the flavor SU(4) down to U(1) $\otimes$ SO(4). However linear
combinations of them can reduce the symmetry further as we discuss below in
the case of each subgroup. A notable feature of these residual sub-groups is
that the generators depend on the particular direction of the mass matrix and
hence are locally defined in the space of the order parameters. This is
exactly like the case of a collinear ferromagnet/antiferromagnet where the
particular generator of the residual U(1) depends on the direction of the
ordering of the magnetic moments in the spin-space. We shall study the nature
of these for each of the six phases separately, including the action of the
lattice symmetries– including the spontaneously broken ones– as well as the
nature of the residual symmetry group.

Out of the 15 masses, the ten TR even ones (Eq. 52) correspond to four
different types of spin-octupole Hall phases while the five TR odd ones (Eq.
53) represent two spin-quadrupole Hall phases. We explain their features in
turn.

###  VII.1 Quantum spin-octupole Hall insulators

####  VII.1.1 $\mathcal{A}^{e}_{1g}$ Singlet mass

For the singlet (Eq. 62a), the mass is given by the fermion bilinear

$\displaystyle\mathcal{A}_{1g}^{e}~{}:$
$\displaystyle~{}-i\langle\bar{\chi}\Sigma_{45}\chi\rangle$ (79)

which fully gaps out all the fermions. This breaks the SU(4) flavour symmetry
down to U(1) $\otimes$ SO(4) with the U(1) being generated by $\Sigma_{45}$
and SO(4) by six other $\Sigma_{i}$’s that commute with $\Sigma_{45}$, i.e.,
$\\{\Sigma_{1},\Sigma_{2},\Sigma_{3},\Sigma_{12},\Sigma_{23},\Sigma_{13}\\}$.

In the microscopic $j=3/2$ basis (the transformation of the $\Sigma$ operators
from the local to global basis is given in Table 8), the $\Sigma_{45}$
operator is given by

$\displaystyle\Sigma_{45}$
$\displaystyle=-\frac{4}{3\sqrt{3}}\left(J_{x}J_{y}J_{z}+J_{y}J_{z}J_{x}+J_{z}J_{x}J_{y}-\frac{15i}{8}\right)$
(80)

where $(J_{x},J_{y},J_{z})$ are the $j=3/2$ spin operators in the global basis
(see Table 8) such that $\Sigma_{45}$ is a spin-octupole.

Starting with the Euclidean Dirac action in the presence of the mass term (Eq.
45), we can diagonalise the spinors $\chi$ in terms of the eigenstates of
$\Sigma_{45}$. The eigenstates consist of two pairs of Kramers doublets. The
two members of each doublet have opposite eigenvalues of $\Sigma_{45}$, i.e.
$\pm 1$. Each of the modes contributes to finite Hall edge current leading to
spin-octupole filtered Hall edge modes similar to the quantum spin-Hall effect
Kane and Mele (2005). The two TR partners carry current in the opposite
direction and backscattering within each TR pair is disallowed by TR symmetry
– again just like quantum spin Hall effect. However, the scattering between
the oppositely moving edge modes belonging to the two different Kramers
doublets is not allowed because they necessarily have opposite eigenvalues of
$\Sigma_{45}$.

A more formal derivation of the resultant symmetry-protected CS action is
obtained by coupling probe charge and spin-octupolar gauge fields to Eq. 45,
i.e., considering

$\displaystyle S[A_{c},A_{o}]=-i\int d^{2}{\bf r}d\tau~{}\bar{\chi}({\bf
r})\left[v_{F}\not{D}-\Delta~{}\Sigma_{45}\right]\chi({\bf r})$ (81)

where

$\displaystyle\not{D}=\gamma^{\mu}\left(i\partial_{\mu}-A_{c,\mu}-\Sigma_{45}A_{o,\mu}\right)$
(82)

where ${\bf A}_{c}$ and ${\bf A}_{o}$ are charge and spin-octupole probe gauge
fields respectively. Then integrating out the fermions leads to the mutual CS
term given by

$\displaystyle S^{mutual}_{CS}$
$\displaystyle=i\frac{N_{F}}{2\pi}~{}sgn(\Delta)~{}\int
d^{3}x~{}\epsilon^{\mu\nu\lambda}A_{c,\mu}\partial_{\nu}A_{o,\lambda}$ (83)

which characterises the quantum spin-octupolar Hall response.

The lattice version of this mass can be analyzed in a similar way as done for
the ICI mass. For that, we consider microscopic hopping operators on one of
the NNN bonds and project that to the low-energy sector. We again take the
blue bond in Fig. 5 from site $B_{2}$ to $B_{1}$ and write the following
hopping operator

$\displaystyle\mathcal{B}_{B_{2}B_{1}}^{(45)}=\phi^{\dagger}(\boldsymbol{\mathcalligra{r}}\,_{B_{1}})~{}\Sigma_{45}~{}\phi(\boldsymbol{\mathcalligra{r}}\,_{B_{2}}).$
(84)

In term of the low-energy spinors, this has the following form

$\displaystyle\mathcal{B}_{B_{2}B_{1}}^{(45)}=\frac{1}{2\sqrt{3}}\bar{\chi}~{}\Sigma_{45}\chi+\cdots.$
(85)

Thus the imaginary part of this hopping operator is proportional to the order
parameter in this phase. This shows that there are non-zero bond currents in
this phase. The hopping pattern on the other bonds are same as shown in Fig. 5
with the hopping amplitudes being $i\Sigma_{45}$ instead of $i$.

(a) Zig-zag edge

(b) Armchair edge

Figure 7: Spectrum for $-i\bar{\chi}\Sigma_{45}\chi$ with zig-zag and armchair
edges. Both of these plots show gapless states at $1/4th$ filling (shown by
black dashed line). To get this spectrum, we consider a honeycomb lattice with
cylindrical geometry where the edges of the cylinder have zigzag or armchair-
like boundaries. Here $k_{x}$ is lattice momentum along the periodic
direction. We take 32 magnetic unit cells along the length of the cylinder to
perform this numerical calculation.

The lattice Hamiltonian can now be used to check for the edge modes by
obtaining the spectrum with open boundary conditions. The spectrum on a
cylinder for the zig-zag and armchair edges are shown in Fig. 7.

Not surprisingly, in Fig. 7, such edge modes are also observed for both 3/4th
filling as well as 1/2 filling. While the case of 3/4th filling is expected to
result from the microscopic particle-hole transformation that maps
$1/4\leftrightarrow 3/4$, the physics of $1/2$ filling would be interesting to
understand in future.

Similar to the ICI phase, the order parameter for the spin-octupole Hall phase
is a $Z_{2}$ field and leads to gapless fermionic modes associated with the
domain walls of the order parameter. Note, however, that such a field is TR
even and is symmetric under all lattice transformations. Thus this mass is
naturally allowed by the microscopics. This is analogous to the Kane-Mele mass
Kane and Mele (2005) for graphene which is symmetry allowed but is
energetically suppressed due to the very small value of the SOC in that case.

####  VII.1.2 The $\mathcal{T}_{1g}^{e}$ triplet masses

The TR even triplet in Eq. 62b consists of three masses of the form

$\displaystyle\mathcal{T}_{1g}^{e}~{}:$
$\displaystyle~{}\left\\{\begin{array}[]{c}-i\braket{\bar{\chi}\Sigma_{12}\chi}\\\
-i\braket{\bar{\chi}\Sigma_{13}\chi}\\\
-i\braket{\bar{\chi}\Sigma_{23}\chi}\end{array}\right.$ (89)

which fully gaps out the Dirac fermions. In terms of the $j=3/2$ operators,
the three mass matrices are given by :

$\displaystyle\Sigma_{\alpha\beta}=\frac{7\epsilon^{\alpha\beta\gamma}}{3}\left(J_{\gamma}-\frac{4}{7}J_{\gamma}^{3}\right)$
(90)

where $\alpha,\beta,\gamma=1,2,3$ with $\alpha\neq\beta$ such that they are a
mixture of dipole and spin-octupole operators. Following Eq. 45, the generic
mass term is given by

$\displaystyle-i\left(\Delta_{1}\bar{\chi}\Sigma_{23}\chi+\Delta_{2}\bar{\chi}\Sigma_{13}\chi+\Delta_{3}\bar{\chi}\Sigma_{12}\chi\right)$
(91)

where $\Delta_{i}~{}(i=1,2,3)$ are the weights for each of the three
components. Hence such masses lie on a 2-sphere with directional cosines given
by
$\cos\theta_{i}=\Delta_{i}/\sqrt{\Delta_{1}^{2}+\Delta_{2}^{2}+\Delta_{3}^{2}}$
where the different points can be rotated into each other via the SU(2)
symmetry generated by Eq 59. With reference to Eq. 89, it is now clear that
this SU(2) corresponds to the continuous rotation amongst the three spin-
octupoles.

At each point on this mass sphere, the residual symmetry is U(1) $\otimes$
SO(4). However, the particular generators of this residual symmetry group
depend on the location of the point and are related to each other by the same
SU(2) transformations (Eq. 59). For example, the generators of the residual
symmetry at the point $[001]$ are given by

$\displaystyle\\{\Sigma_{12},\Sigma_{3},\Sigma_{4},\Sigma_{5},\Sigma_{34},\Sigma_{35},\Sigma_{45}\\}$
(92)

where the first generator corresponds to the U(1) (which is left-over of the
SU(2) (Eq. 59)) and the rest generate the SO(4). The residual groups at other
points on the mass sphere are obtained via SU(2) rotations generated by Eq.
59.

The phase breaks the spin-octupole SU(2) symmetry (Eq. 59) spontaneously and
results in quantum spin-octupole Hall effect that is protected by
U(1)$\rtimes$ Z${}_{\rm 2}^{TR}$ and is similar to the quantum Spin Hall phase
obtained via spontaneously broken spin-rotation symmetry discussed in Ref.
Grover and Senthil (2008) with interesting differences (see below). The
presence of non-trivial spin-octupole filtered edge states is confirmed by
calculating the mutual Hall response similar to Eq. 81 for the singlet case
above which leads to the mutual Chern-Simons action similar to Eq. 83.

The presence of the gapless edge-modes can also be checked by going back to
the mean-field Lattice Hamiltonian in presence of the lattice version of the
mass (not shown). The lattice version of the Hamiltonian corresponding to the
continuum bilinear $-i\langle\bar{\chi}\Sigma_{12}\chi\rangle$ is the same as
that for the quantum spin-octupolar Hall mass given in Eq. 84 with the hopping
matrix(i.e., $\Sigma_{45}$) replaced by $\Sigma_{12}$.

An interesting fallout of the present implementation of the symmetry is the
fact that the three component spin-octupolar order-parameter allows for
Skyrmion configurations. Following the calculations of Refs. Grover and
Senthil (2008); Abanov and Wiegmann (2000) (and references therein) it is
rather straightforward to show that such Skyrmions carry $N_{F}(=4)$ units of
electronic charge and are bosons. Hence, condensation of such Skyrmions within
a framework discussed in Ref. Grover and Senthil (2008) would lead to a novel
$4e$ superconductor with single electron excitations being gapped and the
magnetic flux is quantized in units of $hc/4e$ Berg  _et al._ (2009). This is
tantamount to the fractionalisation Senthil and Fisher (2000) of the
elementary BCS $hc/2e$-vortex. The above mechanism to obtain a $4e$
superconductor is rather novel and differs from the usual mechanism of BCS
superconductivity, where such a $4e$ superconductor is obtained by forming a
4-electron bound state and condensing them. The novel superconductor here
seems to be a natural consequence of the SOC-mediated symmetry implementation
in quarter-filled $j=3/2$ honeycomb lattices that allow binding of $4e$
charges to the topological texture of the spin-octupole order parameter.

####  VII.1.3 The $\mathcal{T}_{1u}^{e}$ and $\mathcal{T}_{2u}^{e}$ triplet
masses

The two inversion odd TR even triplets (Eq. 63) are given by

$\displaystyle\mathcal{T}^{e}_{1u}~{}:$
$\displaystyle~{}\left\\{\begin{array}[]{l}-i\braket{\bar{\chi}\Sigma_{34}\chi}\\\
-i\braket{\bar{\chi}\frac{\Sigma_{14}+\sqrt{3}\Sigma_{15}}{2}\chi}\\\
-i\braket{\bar{\chi}\frac{\Sigma_{24}-\sqrt{3}\Sigma_{25}}{2}\chi}\\\
\end{array}\right.$ (96) $\displaystyle\mathcal{T}^{e}_{2u}~{}:$
$\displaystyle~{}\left\\{\begin{array}[]{l}-i\braket{\bar{\chi}\Sigma_{35}\chi}\\\
-i\braket{\bar{\chi}\frac{\sqrt{3}\Sigma_{14}-\Sigma_{15}}{2}\chi}\\\
-i\braket{\bar{\chi}\frac{\sqrt{3}\Sigma_{24}+\Sigma_{25}}{2}\chi}\\\
\end{array}\right.$ (100)

In terms of the $j=3/2$ spin matrices, we have

$\displaystyle\Sigma_{34}$
$\displaystyle=\frac{2}{3}\left(J_{z}^{3}-\frac{13}{4}J_{z}\right)$ (101a)
$\displaystyle-\frac{1}{2}\Sigma_{14}-\frac{\sqrt{3}}{2}\Sigma_{15}$
$\displaystyle=\frac{2}{3}\left(J_{x}^{3}-\frac{13}{4}J_{x}\right)$ (101b)
$\displaystyle-\frac{\sqrt{3}}{2}\Sigma_{25}+\frac{1}{2}\Sigma_{24}$
$\displaystyle=-\frac{2}{3}\left(J_{y}^{3}-\frac{13}{4}J_{y}\right)$ (101c)

for the three $\mathcal{T}^{e}_{1u}$ masses and

$\displaystyle\Sigma_{35}=\frac{2}{3\sqrt{3}}\left[(J_{x}^{2}J_{z}+J_{x}J_{z}J_{x}+J_{z}J_{x}^{2})\right.$
$\displaystyle\left.\qquad\qquad\qquad\qquad-(J_{y}^{2}J_{z}+J_{y}J_{z}J_{y}+J_{z}J_{y}^{2})\right]$
(102a)
$\displaystyle\frac{\sqrt{3}}{2}\Sigma_{24}+\frac{1}{2}\Sigma_{25}=\frac{2}{3\sqrt{3}}\left[(J_{x}^{2}J_{y}+J_{x}J_{y}J_{x}+J_{y}J_{x}^{2})\right.$
$\displaystyle\left.\qquad\qquad\qquad\qquad-(J_{z}^{2}J_{y}+J_{z}J_{y}J_{z}+J_{y}J_{z}^{2})\right]$
(102b)
$\displaystyle\frac{\sqrt{3}}{2}\Sigma_{14}-\frac{1}{2}\Sigma_{15}=\frac{2}{3\sqrt{3}}\left[(J_{y}^{2}J_{x}+J_{y}J_{x}J_{y}+J_{x}J_{y}^{2})\right.$
$\displaystyle\left.\qquad\qquad\qquad\qquad-(J_{z}^{2}J_{x}+J_{z}J_{x}J_{z}+J_{x}J_{z}^{2})\right].$
(102c)

for the $\mathcal{T}^{e}_{2u}$ triplet. Hence these masses represent two
different sets of spin-octupole order. Note that while components of the two
triplets can be rotated into each other by a U(1) rotation generated by
$\Sigma_{45}$, the two triplets represent different phases since they have
different transformations under lattice reflection, $\mathbf{C_{2}^{\prime}}$.

The three masses in each of the triplets are incompatible, i.e. the matrices
$(m_{1},m_{2},m_{3})$ in Eq. 101a, 101b, 101c or 102a, 102b, 102c do not
mutually pair-wise anticommute. This results in an interesting structure for
the residual symmetry in the resultant massive phases. For a generic linear
combination of the three masses, similar to Eq. 91 but now for the
$\mathcal{T}^{e}_{1u}$ and $\mathcal{T}^{e}_{2u}$ triplets, i.e.,

$\displaystyle-i\left(\Delta_{1}\bar{\chi}m_{1}\chi+\Delta_{2}\bar{\chi}m_{2}\chi+\Delta_{3}\bar{\chi}m_{3}\chi\right),$
(103)

where $m_{1},m_{2},m_{3}$ are the three matrices in Eq. 101 or 102, the
flavour SU(4) is broken down to U(1) $\otimes$ U(1) $\otimes$ U(1). However,
to get more insights, it is useful to diagonalise the bilinear in Eq. 103 for
a generic point on the unit sphere described by the directional cosines
$\Delta_{i}/\sqrt{\Delta_{1}^{2}+\Delta_{2}^{2}+\Delta_{3}^{2}}$ (middle panel
of Fig. 8) to obtain

$\displaystyle-\bar{\chi}^{\prime}\mathcal{D}\chi^{\prime}$ (104)

where $\chi^{\prime}$ are the fermions in the diagonalised basis and

$\displaystyle\mathcal{D}=\begin{pmatrix}a_{1}\sigma_{3}&0\\\
0&a_{2}\sigma_{3}\end{pmatrix}\otimes\zeta_{0}$ (105)

with $a_{1},a_{2}$ are two real functions of $\Delta_{i}$s, $\sigma_{3}$ is
the third Pauli matrix, and $\zeta_{0}$ is the identity matrix that acts in
the valley-band space of the spinors, i.e. in the chiral SU(2) space (Eq. 30).

Figure 8: Figure shows the energy spectrum of low-energy fermions along the
$k_{x}=k_{y}$ line for different combinations of the masses in a triplet. Any
linear combination of the masses in a particular triplet can be represented on
the surface of a unit sphere shown at the center. The energy spectrum shown in
the upper panel of this figure corresponds to the $\mathcal{T}_{1u}^{e}$
triplets in Eq. 96, VIII.1.5 and the $\mathcal{T}_{2u}^{e}$ triplets in Eq.
100, VIII.1.5. The spectrum in (a), (b) and (c) in the upper panel correspond
to the spectrum at the points $A$, $C$ and $B$ on the sphere. Similarly, the
spectrum in the bottom panel corresponds to the $\mathcal{T}_{1u}^{o}$ and
$\mathcal{T}_{2u}^{o}$ triplets in Eq. VIII.1.6, VIII.1.6, VIII.1.6, VIII.1.6.
Here again, the spectrum in (d), (e) and (f) correspond to the spectrum at the
points $A$, $C$, and $B$ on the sphere.

In this diagonalised basis, it is easy to see that there are three linearly
independent matrices (other than the identity matrix) that commute with the
$\mathcal{D}$ matrix in Eq. 105. These are this matrix, $\mathcal{D}$, itself
and

$\displaystyle\begin{pmatrix}\sigma_{3}&0\\\
0&0\end{pmatrix}\otimes\zeta_{0},\quad\begin{pmatrix}0&0\\\
0&\sigma_{3}\end{pmatrix}\otimes\zeta_{0}.$ (106)

The above three matrices generate the residual U(1) $\otimes$ U(1) $\otimes$
U(1) symmetry on generic points on the sphere in the middle panel of Fig. 8
like $C$ and $D$. The first U(1) results in conserved flavour (spin-octupole)
currents along the NNN bonds with a flow pattern similar to that shown in Fig.
5. For such generic points, the fermions are fully gapped with each gapped
band being 4-fold degenerate (spectrum (b) in the top panel of Fig. 8). We can
calculate the edge response, which is given by a mutual CS action similar to
Eq. 83. These spin-octupole filtered edge modes are again protected by the
U(1)$\rtimes$ Z${}_{\rm 2}^{TR}$ as in the case of $\mathcal{T}^{e}_{1g}$ mass
discussed above.

Interestingly on putting two of the $\Delta_{i}$s to zero such as the point
$A$ in Fig. 8 (middle panel), while the above conclusions survive, the gapped
bands have an enhanced 8-fold degeneracy (as shown in (a) of the top panel of
Fig. 8) due to enhanced residual flavour symmetry of U(1) $\otimes$ SO(4).
From the perspective of Eq. 105, the numbers $a_{1}$ and $a_{2}$ becomes equal
at these points such that we can further basis transform
$\mathcal{D}\rightarrow(\sigma_{3}\otimes\sigma_{0})\otimes\zeta_{0}$. Now
there are six generators in addition to $\mathcal{D}$, that commute with the
mass which is given by

$\displaystyle\begin{pmatrix}\sigma_{i}&0\\\
0&\sigma_{i}\end{pmatrix}\otimes\zeta_{0},\quad\begin{pmatrix}\sigma_{i}&0\\\
0&-\sigma_{i}\end{pmatrix}\otimes\zeta_{0}.$ (107)

with $i=1,2,3$. This generates SU(2)$\otimes$ SU(2) $\equiv$ SO(4) in addition
to the U(1) generated by $\mathcal{D}$ itself.

A much more interesting situation arises when one moves from point $C$ (or
$D$) to point $B$ (in the middle panel of Fig. 8) which is characterised by

$\displaystyle|\Delta_{1}|=|\Delta_{2}|=|\Delta_{3}|,$ (108)

and is one of the eight isolated special points.

At these points, $a_{2}$ in Eq. 105 becomes zero. We have assumed
$a_{1}>a_{2}$ without any loss of generality. As a result four flavours of
fermions belonging to the $a_{2}$ block become gapless while four others
belonging to the $a_{1}$ block remain gapped. This leads to a partially gapped
state. The resultant spectrum is shown in (c) of the top panel in Fig. 8. It
is clear that at this special points when $a_{2}=0$, in addition to
$\mathcal{D}$ and the two matrices given by Eq. 106, two additional matrices

$\displaystyle\begin{pmatrix}0&0\\\
0&\sigma_{1}\end{pmatrix}\otimes\zeta_{0},\quad\begin{pmatrix}0&0\\\
0&\sigma_{2}\end{pmatrix}\otimes\zeta_{0}$ (109)

also commute with the mass matrix at this special point. The above two
matrices along with the last one of Eq. 106 generate a SU(2) such that at
these isolated points the symmetry is given by U(1) $\otimes$ U(1) $\otimes$
SU(2) and it is this last SU(2) which protects a subset of gapless Dirac
fermions. On moving away from these special points infinitesimally, the SU(2)