# Topological fine structure of an energy band

Hui Liu  Department of Physics, Stockholm University, AlbaNova University
Center, 106 91 Stockholm, Sweden  Cosma Fulga  Leibniz Institute for Solid
State and Materials Research, IFW Dresden, Helmholtzstrasse 20, 01069 Dresden,
Germany  Würzburg-Dresden Cluster of Excellence ct.qmat, 01062 Dresden,
Germany  Emil J. Bergholtz  Department of Physics, Stockholm University,
AlbaNova University Center, 106 91 Stockholm, Sweden  János K. Asbóth
Department of Theoretical Physics, Institute of Physics, Budapest University
of Technology and Economics, Műegyetem rkp. 3., H-1111 Budapest, Hungary
Wigner Research Centre for Physics, H-1525 Budapest, P.O. Box 49., Hungary

###### Abstract

A band with a nonzero Chern number cannot be fully localized by weak disorder.
There must remain at least one extended state, which “carries the Chern
number.” Here we show that a trivial band can behave in a similar way. Instead
of fully localizing, arbitrarily weak disorder leads to the emergence of two
sets of extended states, positioned at two different energy intervals, which
carry opposite Chern numbers. Thus, a single trivial band can show the same
behavior as two separate Chern bands. We show that this property is predicted
by a topological invariant called a “localizer index.” Even though the band as
a whole is trivial as far as the Chern number is concerned, the localizer
index allows access to a topological fine structure. This index changes as a
function of energy within the bandwidth of the trivial band, causing
nontrivial extended states to appear as soon as disorder is introduced. Our
work points to a previously overlooked manifestation of topology, which
impacts the response of systems to impurities beyond the information included
in conventional topological invariants.

_Introduction_ — Anderson localization [1, 2] is the quantum phase transition
across which increasing the disorder leads to the extended states of a
disordered system to acquire finite localization lengths, and a conductor
becomes insulating. Initially, the scaling theory of localization [3, 4]
predicted that such a transition would not occur in two-dimensional (2D)
systems that lack any symmetry, i.e., class A of the Altland-Zirnbauer
classification [5]. Instead, all states become localized as soon as disorder
is introduced, and a thermodynamically large 2D system is insulating even at
an infinitesimal disorder strength.

The discovery of the quantum Hall effect [6] and the subsequent development of
the theory of topological phases of matter [7, 8, 9] changed this paradigm. It
was realized that robust extended states are indeed possible in 2D, class A,
and their presence is a consequence of their topologically nontrivial
character. Thus, for a 2D system in class A, a band with nonzero Chern number
cannot be fully localized by weak disorder [10]. Instead, at least one
extended state that “carries the Chern number” must remain [11]. Using
terminology introduced by Laughlin [10], extended states at different energies
and carrying opposite Chern numbers “levitate” towards each other, eventually
“annihilating” in order to produce a trivial Anderson insulator. In contrast,
trivial bands are expected to fully localize even for infinitesimal disorder
strength, provided that the disorder is sufficiently generic. These two types
of behavior are shown schematically in Fig. 1(a).

Figure 1:  Panel (a): The paradigm of Anderson localization for 2D systems in
class A. A trivial band (middle, Chern number ${\cal C}=0$) is fully localized
by disorder, whereas nontrivial bands (${\cal C}=\pm 1$) lead to robust
extended states. The latter eventually meet and annihilate (star) for larger
disorder strength. The index ${\cal W}$ denotes the number of chiral edge
modes present at energies between/outside those of the extended states. Panel
(b): Our main result. The trivial band does not fully localize, but splits
into two branches of extended states carrying oposite Chern numbers. As we
show, this behavior is a consequence of its topological fine structure.

Here, we revisit Anderson localization of trivial bands of 2D systems, class
A, showing bands which cannot be localized by weak, generic disorder, even
though their Chern number vanishes. Instead, there are multiple, robust
extended states, which carry opposite Chern numbers, and which, as disorder is
further increased, participate in the levitation and annihilation process [see
Fig. 1(b)]. This behavior is still due to nontrivial topology, but it is the
consequence of an index that is more general than the Chern number – the
_localizer index_ [12, 13, 14, 15]. While the Chern number is a global
property characterizing the entirety of a band, the localizer index can be
evaluated for different energies within a band, thus providing access to a
topological fine structure. As we show, changes of this topological invariant
necessarily lead to the formation of extended states that are robust against
disorder.

Figure 2:  Top panels: band structure of the model in the absence of disorder.
We use a ribbon geometry, infinite along the $x$-direction and consisting of
60 unit cells along $y$. Black, red, and green colors indicate states
localized at the top edge, bottom edge, and in the bulk, respectively. Bottom
panels: Two-terminal transmission probability, plotted as a function of energy
$E$ and disorder strength $W$, for a $60\times 60$ unit cell system with
periodic boundary conditions in the $y$-direction. Darker regions correspond
to the presence of extended bulk states, helping us track the levitation and
annihilation process. Each point is obtained by averaging over 50 independent
disorder realizations.

_An example_ — We begin by illustrating our general conclusions using a simple
2D model: a two-band Chern insulator coupled to a single, trivial band. The
momentum-space Hamiltonian reads

$H(\mathbf{k})=\begin{pmatrix}h^{\phantom{{\dagger}}}_{11}(\mathbf{k})&h^{\phantom{{\dagger}}}_{12}(\mathbf{k})&v\\\
h_{12}^{*}(\mathbf{k})&-h^{\phantom{{\dagger}}}_{11}(\mathbf{k})&0\\\
v&0&0\end{pmatrix},$ (1)

where $\mathbf{k}=(k_{x},k_{y})$ is the 2D quasimomentum, and the matrix
element functions are $h_{11}(\mathbf{k})=2(\cos k_{x}-\cos k_{y})$, and
$h_{12}(\mathbf{k})=\sqrt{2}e^{-i\pi/4}(e^{ik_{x}}+e^{ik_{y}}+ie^{i(k_{x}+k_{y})}+1)$.
The upper-left $2\times 2$ block is a Chern insulator with Chern numbers
$\mathcal{C}=\pm 1$ for the upper and lower bands [16], whose hopping
amplitudes are used to set the unit of energy. The lower right element is the
single-band atomic insulator: a trivial flat band with a vanishing onsite
potential. It is coupled to the Chern insulator with an amplitude $v$.

In the regime of coupling strengths $0\leq v<8$, this three-band model has two
phases: a topological phase for small coupling, $0\leq v\lesssim 5.65$, and a
trivial phase for large coupling, $v\gtrsim 5.65$. In the topological phase
the top and bottom bands have Chern numbers $+1$ and $-1$, respectively,
whereas the middle band is trivial, with Chern number 0. In the trivial phase
all bands have a vanishing Chern number. These two phases are separated by a
topological phase transition, where the gaps between the 1st and 2nd, and the
2nd and 3rd band close simultaneously [see Fig. 2(a1-c1)].

We are interested in the what happens to the energy eigenstates of the model
as disorder is turned on and gradually increased. Thus, we add a random onsite
potential to the Hamiltonian, different for each of the three orbitals,
uniformly distributed in $[-W,W]$, with $W$ denoting the disorder strength. We
calculate the two-terminal transmission probability $G$ of a finite-size,
square system using the Kwant package [17] in order to numerically estimate
the energies at which extended states are present. The leads are attached to
the left and right boundaries of the system, and we connect the top and bottom
up by periodic boundary conditions along the transverse direction, so as to
pick up only the contribution of bulk states to the transmission probability.
For additional details of the numerical implementation, see the Supplemental
Material (SM) [18] and the code on Zenodo [19].

Localized states’ contribution to the transmission probability decays
exponentially with system size, whereas for extended bulk states this
contribution stays constant, or even grows, as system size is increased. Thus,
for a sufficiently large system (in our numerics, $60\times 60$ unit cells),
we can use $G(E,W)$ to locate energies where extended states exist [large
transmission probabilities, darker colors in Fig. 2(a2-c2)], and then track
the pattern of their levitation and annihilation.

When the atomic insulator and Chern insulator are weakly coupled, $v\lesssim
3$, the pattern of levitation and annihilation is unsurprising. As seen in
Fig. 2(a2), first the middle, trivial band fully localizes. The outer,
nontrivial bands leave behind extended states which levitate towards each
other and annihilate around $W\approx 8$. This conventional behavior parallels
that observed in earlier works on disordered topological phases [20, 21]. Also
for $v\gtrsim 5.65$, the system’s behavior is typical: all bands are trivial,
they all localize in the presence of disorder, and no levitation and
annihilation can be seen.

A qualitatively different behavior, however, can be seen for intermediate
coupling, $3.3\lesssim v\lesssim 5.65$. Here, instead of localizing, the
middle band produced two sets of extended states, located symmetrically around
$E=0$. These proceed to levitate away from $E=0$, and annihilate with the
extended states that originated from the top-most and bottom-most bands [see
Fig. 2(b2)]. The unconventional extended states emerging from the middle band
carry opposite Chern numbers, as we have checked by repeating the calculation
with open boundary conditions, and also by computing the scattering-matrix
topological invariant [22, 23, 24]. Further, by performing three-terminal
transport simulations and a finite-size scaling analysis (SM) [18], we have
checked that these extended states appear to persist for any disorder strength
$W\neq 0$, no matter how small. This, however, does not rule out the
possibility that in the thermodynamic limit these extended states only appear
at some small but nonzero disorder only, as in the case of the topological
Anderson insulator [25, 26]. To rule this out we use appropriate topological
invariants below.

_Topological fine structure_ — We now show that the phenomenon of extended
states emerging from the trivial band, observed in our example at coupling
$v\approx 4.5$, persists also in the thermodynamic limit. For this, we employ
a recently introduced tool for computing real-space topological invariants,
the “spectral localizer”, and quantities computed from it, the localizer
index, and the localizer gap [12, 13, 15, 14, 27, 28, 29, 30, 31]. We rely on
revealing what we call the topological fine structure of the middle band,
captured by these quantities. We briefly summarize these concepts below, and
clarify how they can show robustly delocalized states inside a seemingly
topologically trivial (Chern number 0) band. We then calculate these
quantities for our model, and prove that for $v\approx 4.5$, there must exist
at least two energies in the middle band where eigenstates cannot become
localized by weak disorder.

Our starting point is the spectral localizer ${\cal L}(r,E)$ [12, 13, 15, 14,
27, 28, 29, 30, 31], a matrix-valued function, defined for a finite-size
sample of the 2D system, class A, with open boundary conditions. It is a
continuous function of a reference position $\mathbf{r}=(r_{x},r_{y})$, which
is encoded via a complex number $r=r_{x}+ir_{y}$, and of the energy $E$. It is
a Hermitian matrix of size $2Nm\times 2Nm$ for a system with $N$ unit cells
and $m$ orbitals,

${\cal L}(r,E)=\begin{pmatrix}H-E&\kappa(X-iY-r^{*})\\\
\kappa(X+iY-r)&-H+E\end{pmatrix}.$ (2)

Here, $H$ is the Hamiltonian matrix of the finite-size system, $X$ and $Y$ are
the matrices of the position operators (more details in the SM [18]). The
dimensional parameter $\kappa$ has to be chosen to be small enough, see the SM
[18] and Ref. [12] for a more detailed description. In our case and with our
units this was fulfilled by setting $\kappa=0.25$.

The first piece of information provided by the spectral localizer matrix is
the so-called localizer index $Q(r,E)$. It is defined as the matrix signature
(${\rm sig}$, the number of positive eigenvalues minus the number of negative
eigenvalues) of the spectral localizer[13],

$\displaystyle Q(r,E)$ $\displaystyle=\frac{1}{2}{\rm sig}[\mathcal{L}(r,E)].$
(3)

At energy $E$ in a spectral gap or mobility gap, the localizer index is
identical to the number of chiral edge modes at the system boundary for any
value of $\mathbf{r}$ chosen deep within the systems’ bulk[12, 13, 14, 15].
Thus, under these conditions, it contains the same information as the Chern
number, and it does not depend on the reference position $\mathbf{r}$, as long
as $\mathbf{r}$ is chosen deep in the bulk.

We want to use the localizer index $Q(r,E)$ for energies $E$ inside an energy
band; here, its value depends on the reference position $\mathbf{r}$ inside
the bulk, but only if there are reference positions where at least one of the
eigenvalues of $\mathcal{L}(r,E)$ is 0. It can happen that there are no such
values of $\mathbf{r}$, and the localizer index is constant throughout the
bulk; under what circumstances this is expected to occur is currently not
understood [12, 13, 14, 15].

To check whether the localizer index is independent of the reference position,
we use the localizer gap $g_{\cal L}$ [12, 13, 14, 15]. This is at a given
energy $E$ the smallest absolute value of any of ${\cal L}(r,E)$’s
eigenvalues, i.e. the shortest distance from an eigenvalue of ${\cal L}(r,E)$
to zero, at any $\mathbf{r}$ inside the bulk, i.e.,

$\displaystyle g_{\cal L}(E)$
$\displaystyle=\\!\\!\min_{\mathbf{r}\in\text{bulk}}g_{\cal L}(r,E);$
$\displaystyle g_{\cal L}(r,E)$
$\displaystyle=\\!\\!\\!\min_{\lambda\in\sigma[\mathcal{L}(r,E)]}\left|\lambda\right|,$
(4)

where $\sigma[\mathcal{L}]$ is the set of eigenvalues of $\mathcal{L}$. We
characterize the topological fine structure of a band using the localizer
index $Q(r,E)$, evaluated at energies where the localizer gap
$g_{\mathcal{L}}(E)$ is nonzero, and hence, the index is independent of $r$,

$\displaystyle Q(E)$ $\displaystyle=Q(r,E)\quad\text{when
}g_{\mathcal{L}}(E)>0.$ (5)

Besides ensuring the localizer index depends only on energy and not reference
position, the localizer gap $g_{\mathcal{L}}(E)$ also quantifies the
robustness of the index. Given Eq. (3), the index $Q(r,E)$ cannot change under
any perturbation, unless that perturbation is large enough to close the
localizer gap, meaning that $g_{\cal L}(E)$ is reduced to zero. This is simply
a consequence of Weyl’s inequality: when perturbing $H\to\widetilde{H}$ and
thus ${\cal L}\to\widetilde{\cal L}$, $Q$ is unchanged as long as
$||\widetilde{\cal L}-{\cal L}||<g_{\cal L}$. Here, $||\cdot||$ is the 2-norm
of a matrix, the modulus of its largest eigenvalue.

The localizer index $Q(E)$ can reveal topological fine structure of an energy
band of a clean 2D system. It predicts the number of chiral edge states that
would be seen at $E$ on a large enough sample with open boundary conditions,
if weak disorder was added so as to localize the bulk states at energy $E$.
This follows from the previous paragraph, with the weak disorder being treated
as a perturbation.

If an energy band includes energy values $E_{1}<E_{2}$ where the localizer
indices differ, $Q(E_{1})\neq Q(E_{2})$, then there must exist an energy value
between them, $E_{1}<E<E_{2}$, where energy eigenstates remain extended under
weak disorder. This is necessarily true, because in the weakly disordered
system the only way the number of chiral edge states at $E_{1}$ and $E_{2}$
can be different if there is an energy between these values where the bulk gap
closes (a topological phase transition in the spectrum of the disordered
system). This shows how robustly extended states can emerge from a seemingly
topologically trivial band (Chern number 0).

Figure 3:  (a) Topological fine structure of the central energy band at
coupling $v=4.5$: Localizer gap $g_{\mathcal{L}}$ (dashed line, horizontal
axis) and localizer index $Q$ (shading and labels). Horizontal gray regions:
energy ranges in which the $g_{\cal L}=0$, thus $Q(E)$ cannot be evaluated.
Vertical red bars on left axis: energy intervals of the bulk bands. (b) Phase
diagram of the topological fine structure revealed using the localizer gap
$g_{\cal L}$, obtained from a $10\times 10$ unit cell system, plotted versus
energy $E$ and coupling $v$. Semi-transparent light blue shaded areas: energy
intervals of the bands. Vertical dashed line: the critical point $v\approx
3.3$ at which the topological phase transition of the middle band occurs with
no closing of the bulk gaps – as obtained from the transport simulations.
Dash-dotted line: $v=4.5$, corresponding to panel a). Dotted line: topological
phase transition with a change in the Chern numbers, and closing of the bulk
band gaps. All plots are with $\kappa=0.25$ and $g_{\cal L}$ was computed
taking a grid of $40\times 40$ reference positions $\mathbf{r}$ in the Wigner-
Seitz cell.

_Topological fine structure in our example_ — We now apply the formalism above
to our model, the Hamiltonian of Eq. (1), to show that the delocalized states
we observe in Fig. 2 are delocalized at weak disorder in the thermodynamic
limit. For the numerical work, we took square shaped systems of $20\times 20$
unit cells with open boundary conditions (we verified that repeating the
calculation for larger system sizes does not significantly alter the result).
We chose numerous energy values $-10<E<10$, and calculated the spectral
localizer, Eq. (2), and its eigenvalues of smallest magnitude over a grid of
40$\times$40 reference positions $\mathbf{r}$ in the central Wigner-Seitz unit
cell. Thus we obtained the localizer gap, Eq. (4). We note that, as proven in
Ref. [15] and verified by us in the SM [18], for a fixed set of Hamiltonian
parameters and $\mathbf{r}$ in the bulk, $g_{\cal L}(r,E)$ converges as system
size is increased, thus only the behavior of the system close to the position
encoded in $r$ influences $g_{\cal L}(r,E)$. We picked some $v,E$ values where
we also fully diagonalized the spectral localizer to obtain the localizer
index $Q(E)$. Since this topological invariant cannot change as long as the
localizer gap is nonzero, this numerically costly procedure was only needed at
few $(v,E)$ pairs.

First, we show our main result, the robustly delocalized states at $v=4.5$, in
Fig. 3(a). Here, in both spectral gaps we find $Q=-1$, consistent with the
Chern numbers of the bands. Remarkably, the localizer gap remains open (e.g.
$g_{\cal L}>0.02$) even for energies _within_ the central band, meaning that
the localizer index $Q(E)$ remains well defined for most energies in the band.
Moreover, the index changes from $Q=-1$ to $Q=0$ and then back to $Q=-1$ as
energy is scanned through the middle band. Thus there are two topological
phase transitions in that band: this is the topological fine structure,
predicting two energies where robustly extended states must occur. Here, these
transitions are at $E\simeq\pm 1.95$, which agrees with Fig 2(b2).

Second, we show a full phase diagram of the topological fine structure of
Hamiltonian, Eq. (1), obtained by repeating the calculation above for many
values of $v$, in Fig. 3(b). We find the nontrivial topological fine structure
of the middle band throughout the interval $3.3\lesssim v\lesssim 5.65$. This
is consistent with our weak disorder numerics which show robustly extended
states in the middle band also when adding additional perturbations to the
Hamiltonian, such as shifting the energy of the band (see SM [18]). Note that
the mismatch between the localizer gap closing and the closing of bulk band
gaps is a technical issue of choices of $\kappa$, see the SM [18] for more
details. Note that the localizer gap is 0 close to $E=0$ for all values of $v$
we consider, however, there is no topological phase transition in the spectrum
here, no robustly extended eigenstates are expected, since the localizer index
does not change across $E=0$.

We emphasize that the emergence of extended states from the trivial band is
the consequence of a topological phase transition occurring as $v$ crosses a
critical value, between 3.1 and 3.3, as seen in Fig. 3(b) as well as the
additional transmission calculation in SM [18]. However, this transition does
_not_ involve a closing of the bulk gaps of the system, and it is not
associated to a change in the Chern numbers of the bands.

_Conclusion and outlook_ — We have shown that a seemingly topologically
trivial band can exhibit a nontrivial topological fine structure, with a
generalization of the Chern number taking different values inside the band.
Such a fine structure implies that the band hosts eigenstates that are
robustly extended under weak disorder. The generalization of the Chern number
capturing the topological fine structure is the localizer index and the
localizer gap of the spectral localizer.

Our work will motivate future research in several directions. Saliently, while
we have only considered class A in two dimensions, it is an intriguing open
question what topological fine structure may exist in other symmetry classes
and dimensions. In this context we note that topological markers have recently
been generalized to odd-dimensional systems [32].

Another direction worth exploring is the interplay between the fine structure
topology and interactions. This, we conjecture, may lead to fractional Chern
insulators [33, 34], which usually, but not always [35], require a band with
nonzero Chern number. In the present context we envision that stable states
may occur at unconventional filling fractions. In particular, Laughlin-like
states may instead occur at even denominator band filling (corresponding to
odd denominator filling of the effective fine structure band).

On a more practical level, it would be interesting to identify further models
exhibiting the topological fine structure phenomenology. Given the generality
of our argumentation such models should indeed be ubiquitous. In particular,
it would be interesting to find a minimal model, presumably featuring only two
energy bands. Nevertheless, we emphasize that the present three band setting
is also realistic. In fact, attaching a trivial flat band to a Chern insulator
has already been realized in photonic systems [36, 37, 38]. We expect that our
work will motivate further experimental research in this direction.

Notions of band structure topology have profoundly changed the way we
understand phases of matter and altered the paradigm of localization.
Topological fine structure provides a natural next level of understanding of
these fundamental concepts.

_Acknowledgements_ — We thank Ulrike Nitzsche for technical assistance. ICF
acknowledges support from the Deutsche Forschungsgemeinschaft (DFG, German
Research Foundation) under Germany’s Excellence Strategy through the Würzburg-
Dresden Cluster of Excellence on Complexity and Topology in Quantum Matter –
_ct.qmat_ (EXC 2147, project-ids 390858490 and 392019). HL and EJB were
supported by the Swedish Research Council (VR, grant 2018-00313), the
Wallenberg Academy Fellows program (2018.0460) of the Knut and Alice
Wallenberg Foundation, and the Göran Gustafsson Foundation for Research in
Natural Sciences and Medicine. JKA acknowledges support by the National
Research Development and Innovation Office (NKFIH) through the OTKA Grant FK
132146.

## References

  * Anderson [1958] P. W. Anderson, Absence of diffusion in certain random lattices, Phys. Rev. 109, 1492–1505 (1958).
  * Evers and Mirlin [2008] F. Evers and A. D. Mirlin, Anderson transitions, Rev. Mod. Phys. 80, 1355–1417 (2008).
  * Abrahams  _et al._ [1979] E. Abrahams, P. W. Anderson, D. C. Licciardello, and T. V. Ramakrishnan, Scaling theory of localization: Absence of quantum diffusion in two dimensions, Phys. Rev. Lett. 42, 673–676 (1979).
  * Wegner [1979] F. Wegner, The mobility edge problem: Continuous symmetry and a conjecture, Z. Physik B 35, 207–210 (1979).
  * Altland and Zirnbauer [1997] A. Altland and M. R. Zirnbauer, Nonstandard symmetry classes in mesoscopic normal-superconducting hybrid structures, Phys. Rev. B 55, 1142–1161 (1997).
  * von Klitzing [1986] K. von Klitzing, The quantized Hall effect, Rev. Mod. Phys. 58, 519–531 (1986).
  * Qi and Zhang [2011] X.-L. Qi and S.-C. Zhang, Topological insulators and superconductors, Rev. Mod. Phys. 83, 1057–1110 (2011).
  * Hasan and Kane [2010] M. Z. Hasan and C. L. Kane, Colloquium: Topological insulators, Rev. Mod. Phys. 82, 3045–3067 (2010).
  * Chiu  _et al._ [2016] C.-K. Chiu, J. C. Y. Teo, A. P. Schnyder, and S. Ryu, Classification of topological quantum matter with symmetries, Rev. Mod. Phys. 88, 035005 (2016).
  * Laughlin [1984] R. B. Laughlin, Levitation of extended-state bands in a strong magnetic field, Phys. Rev. Lett. 52, 2304–2304 (1984).
  * Halperin [1982] B. I. Halperin, Quantized Hall conductance, current-carrying edge states, and the existence of extended states in a two-dimensional disordered potential, Phys. Rev. B 25, 2185–2190 (1982).
  * Cerjan and Loring [2022] A. Cerjan and T. A. Loring, Local invariants identify topology in metals and gapless systems, Phys. Rev. B 106, 064109 (2022).
  * Loring [2015] T. A. Loring, K-theory and pseudospectra for topological insulators, Annals of Physics 356, 383–416 (2015).
  * Loring and Schulz-Baldes [2018] T. Loring and H. Schulz-Baldes, The spectral localizer for even index pairings, arXiv e-prints , arXiv:1802.04517 (2018), arXiv:1802.04517 [math-ph] .
  * Loring and Schulz-Baldes [2017] T. Loring and H. Schulz-Baldes, Finite volume calculation of $K$-theory invariants, arXiv e-prints , arXiv:1701.07455 (2017), arXiv:1701.07455 [math-ph] .
  * Neupert  _et al._ [2011] T. Neupert, L. Santos, C. Chamon, and C. Mudry, Fractional quantum Hall states at zero magnetic field, Phys. Rev. Lett. 106, 236804 (2011).
  * Groth  _et al._ [2014] C. W. Groth, M. Wimmer, A. R. Akhmerov, and X. Waintal, Kwant: a software package for quantum transport, New J. Phys. 16, 063065 (2014).
  * [18] In this supplemental material, we show the transport settings, additional evidence from transport calculations, winding number calculations at strong disorder, localizer gap for $v=2$, more details on spectral localizer simulations, and a case study of energy shift. .
  * [19] H. Liu, I. C. Fulga, E. J. Bergholtz, and J. K. Asbóth, Topological fine structure of an energy band.
  * Bergman and Refael [2010] D. L. Bergman and G. Refael, Bulk metals with helical surface states, Phys. Rev. B 82, 195417 (2010).
  * Junck  _et al._ [2013] A. Junck, K. W. Kim, D. L. Bergman, T. Pereg-Barnea, and G. Refael, Transport through a disordered topological-metal strip, Phys. Rev. B 87, 235114 (2013).
  * Brouwer [1998] P. W. Brouwer, Scattering approach to parametric pumping, Phys. Rev. B 58, R10135–R10138 (1998).
  * Bräunlich  _et al._ [2010] G. Bräunlich, G. M. Graf, and G. Ortelli, Equivalence of topological and scattering approaches to quantum pumping, Communications in Mathematical Physics 295, 243–259 (2010).
  * Fulga  _et al._ [2012] I. C. Fulga, F. Hassler, and A. R. Akhmerov, Scattering theory of topological insulators and superconductors, Phys. Rev. B 85, 165409 (2012).
  * Li  _et al._ [2009] J. Li, R.-L. Chu, J. K. Jain, and S.-Q. Shen, Topological Anderson insulator, Phys. Rev. Lett. 102, 136806 (2009).
  * Groth  _et al._ [2009] C. W. Groth, M. Wimmer, A. R. Akhmerov, J. Tworzydło, and C. W. J. Beenakker, Theory of the topological Anderson insulator, Phys. Rev. Lett. 103, 196805 (2009).
  * Fulga  _et al._ [2016] I. C. Fulga, D. I. Pikulin, and T. A. Loring, Aperiodic weak topological superconductors, Phys. Rev. Lett. 116, 257002 (2016).
  * Schulz-Baldes and Stoiber [2022] H. Schulz-Baldes and T. Stoiber, Invariants of disordered semimetals via the spectral localizer, Europhysics Letters 136, 27001 (2022).
  * Liu and Fulga [2023] H. Liu and I. C. Fulga, Mixed higher-order topology: Boundary non-Hermitian skin effect induced by a Floquet bulk, Phys. Rev. B 108, 035107 (2023).
  * Schulz-Baldes and Stoiber [2023] H. Schulz-Baldes and T. Stoiber, Spectral localization for semimetals and Callias operators, Journal of Mathematical Physics 64, 081901 (2023), arXiv:2203.15014 [math-ph] .
  * Franca and Grushin [2023] S. Franca and A. G. Grushin, Obstructions in trivial metals as topological insulator zero-modes, arXiv e-prints , arXiv:2304.01983 (2023), arXiv:2304.01983 [cond-mat.mes-hall] .
  * Hannukainen  _et al._ [2022] J. D. Hannukainen, M. F. Martínez, J. H. Bardarson, and T. K. Kvorning, Local topological markers in odd spatial dimensions and their application to amorphous topological matter, Phys. Rev. Lett. 129, 277601 (2022).
  * Regnault and Bernevig [2011] N. Regnault and B. A. Bernevig, Fractional Chern insulator, Phys. Rev. X 1, 021014 (2011).
  * Liu and Bergholtz [2024] Z. Liu and E. J. Bergholtz, Recent developments in fractional Chern insulators, in  _Encyclopedia of Condensed Matter Physics (Second Edition)_, edited by T. Chakraborty (Academic Press, Oxford, 2024) second edition ed., pp. 515–538.
  * Simon  _et al._ [2015] S. H. Simon, F. Harper, and N. Read, Fractional Chern insulators in bands with zero Berry curvature, Phys. Rev. B 92, 195104 (2015).
  * Guglielmon and Rechtsman [2019] J. Guglielmon and M. C. Rechtsman, Broadband topological slow light through higher momentum-space winding, Phys. Rev. Lett. 122, 153904 (2019).
  * Mann and Alù [2021] S. A. Mann and A. Alù, Broadband topological slow light through Brillouin zone winding, Phys. Rev. Lett. 127, 123601 (2021).
  * Chen  _et al._ [2022] F. Chen, H. Xue, Y. Pan, M. Wang, Y. Hu, L. Zhang, Q. Chen, S. Han, G.-g. Liu, Z. Gao, P. Zhou, W. Yin, H. Chen, B. Zhang, and Y. Yang, Demonstration of broadband topological slow light, arXiv e-prints , arXiv:2208.07228 (2022), arXiv:2208.07228 [physics.app-ph] .


# Towards One-Shot Learning for Text Classification using Inductive Logic
Programming

Ghazal Afroozi Milani Department of Computer Science, University of Surrey,
United Kingdom Daniel Cyrus Department of Computer Science, University of
Surrey, United Kingdom Alireza Tamaddoni-Nezhad Department of Computer
Science, University of Surrey, United Kingdom
<EMAIL_ADDRESS>
###### Abstract

With the ever-increasing potential of AI to perform personalised tasks, it is
becoming essential to develop new machine learning techniques which are data-
efficient and do not require hundreds or thousands of training data. In this
paper, we explore an Inductive Logic Programming approach for one-shot text
classification. In particular, we explore the framework of Meta-Interpretive
Learning (MIL), along with using common-sense background knowledge extracted
from ConceptNet. Results indicate that MIL can learn text classification rules
from a small number of training examples, even one example. Moreover, the
higher complexity of chosen example for one-shot learning, the higher accuracy
of the outcome. Finally, we utilise two approaches, Background Knowledge
Splitting and Average One-Shot Learning, to evaluate our model on a public
News Category dataset. The outcomes validate MIL’s superior performance to the
Siamese net for one-shot learning from text.

##  1 Introduction

Machine learning, in particular, Deep Neural Networks (DNNs), has achieved
outstanding outcomes in numerous real world applications such as image and
text classification [8, 21]. However, these learning methods require a huge
number of training instances, which is not always possible to be provided in
advance. An automated software tool, for example, should be able to learn from
a small number of interactions with the user in order to be efficiently
customised with every new user’s requirement. Inductive Logic Programming
(ILP) and, in particular, Meta Interpretive Learning (MIL) can learn human-
readable hypotheses from a small amount of training data [18, 25]. This
capability is very promising for medical and industrial usage, especially when
we do not have access to a large amount of training data. The main
contribution of this study is to develop and evaluate a new learning
algorithm, Meta-Goal Learner (MGL), based on the framework of Meta-
Interpretive Learning (MIL), which can be utilised for one-shot learning from
textual data along with using a common-sense knowledge-base (i.e. ConceptNet)
as background knowledge for learning.

##  2 Inductive Logic Programming

Inductive Logic Programming (ILP) is a subfield of symbolic artificial
intelligence, which works based on inductive reasoning to detect
generalisations, rules, or models [14, 19]. In this method, the system learns
from training examples with the help of background information and using logic
programming [25]. Indeed, the primary purpose of ILP [14], like the other
types of machine learning, is to induce a model/ hypothesis that can
generalise training examples but, unlike them, uses logic programs to
represent data and learns relations [3].

####

Using logic programs brings several benefits to ILP. Firstly, as they
represent hypotheses in ILP and are related to relational databases, they can
support relational data like graphs. On the other hand, because of its
expressivity, it can learn complex relational theories like cellular automata,
event calculus theories, and Petri nets [3]. Secondly, it is similar to
natural language; therefore, readable and valuable for explainable AI and
scientific discovery ([18]).

####

Every standard ILP setting consists of three input sets, including Background
Knowledge (B), positive training examples ($E^{+}$), and negative training
examples ($E^{-}$) [12, 14]. The output is a Hypothesis $H$ such that:

  * •

It is necessary $\forall$ e $\in$ $E^{+}$ ${B\not\models}$ e

  * •

${H}$ is complete/ sufficient if $\forall$ e $\in$ $E^{+}$ and ${H\wedge
B\models}$ e

  * •

${H}$ is strong consistent if $\forall$ e $\in$ $E^{-}$ and ${H\wedge
B\not\models}$ e

  * •

${H}$ is weak consistent if ${H\wedge B\not\models\square}$

###  2.1 Meta-Interpretive Learning

Meta-Interpretive Learning (MIL) [16] is a type of Inductive Logic Programming
which allows learning logic programs from background knowledge, training
examples and a declarative bias called metarules. Metarules are datalog
clauses with variables quantified over predicate symbols (i.e., second-order
variables) [20, 11].

The advantage of MIL is its capability to (1) automatically introduce new sub-
definitions when it learns a predicate definition [17, 16]. This is referred
to as Predicate Invention. (2) learning recursive clauses. MIL uses metarules
to restrict hypotheses space. In other words, a meta-rule is a higher-order
expression in which the form of permitted clauses in the hypothesis space is
instructed[17, 13].

A metarule is a higher-order well formed formula as follows:

$\exists\sigma\forall\tau
P(s_{1},...,s_{m})\leftarrow...,Q_{i}(t_{1},...,t_{n}),...$
where $\sigma,\tau$ are disjoint variables, $\rho$ and $\zeta$ are sets of
predicate symbols and constants respectively
$P,Q_{i}\in\sigma\cup\tau\cup\rho$ and
$s_{1},...,s_{m},t_{1},...,t_{n}\in\sigma\cup\tau\cup\zeta.$ Metarules are
shown normally without quantifiers as:
$P(s_{1},...,s_{m})\leftarrow...,Q_{i}(t_{1},...,t_{n}),...$ . P and $Q_{i}$
are existentially quantified higher-order variables, while the other variables
present universally quantified variables.

The standard MIL problem [16] is defined as follows: Given metarules (M),
definite program background knowledge (B) and ground positive and negative
unit examples ($E^{+}$, $E^{-}$), MIL returns a higher-order datalog program
hypothesis (H) if one exists such that: $M\wedge B\wedge H\models E^{+}$ and
$M\wedge B\wedge H\not\models E^{-}$ is consistent.

####

In ILP, selecting proper Background Knowledge (B) is essential for appropriate
learning performance. However, this could be difficult or expensive as often
this should be done by experts. There are two main problems related to having
a balance background knowledge, including too little and too much Background
Knowledge. If using too little Background Knowledge, we might lose the target
hypothesis. Having too much irrelevant Background Knowledge is another ILP
challenge that can increase hypothesis space size and decrease learning
performance ([5]).

####

There are two methods to overcome this issue, one is to enable an ILP system
to invent a new predicate instead of missing background knowledge [24]. The
second method is to use transfer learning to discover knowledge that can help
mitigate the effects of background knowledge shortage [9].

##  3 Commonsense Knowledge and ConceptNet

Commonsense knowledge is an ordinary information that helps people make sense
of everyday situations without mentioning them in their communications. Due to
its implicitness, capturing this knowledge is beneficial for designing an
effective human-computer interface and various types of Artificial
Intelligence (AI) [7, 6]. These common-sense knowledge sources are in various
forms and about different types of knowledge. Ilievski et al. [7] divided them
into five categories, common-sense knowledge graphs ( e.g. ConceptNet and
Atomic), common knowledge graphs and ontologies( e.g. Wikidata and Yago ),
lexical resources ( e.g. WordNet and Roget), visual common-sense sources (
e.g. Visual Genome), and Corpora and language models ( e.g. GenericsKB and
Language models) as shown in Table 1.

Category | Source | Relations | Example
---|---|---|---
Commonsense KGs | ConceptNet | 37 | mother-related_to-family
| Atomic | 9 | Person X bakes bread-xEffect-eat food
| Glucose | 10 | someone A -eats- something A
| WebChild | 4groups | eating -type of- consumption
| Quasimodo | 78636 | pressure cooker-cook faster-food
| SenticNet | 1 | cold food -polarity- negative
| HasPartKB | 1 | dairy food -has part- vitamin
| Probase | 1 | apple -is a- food
| Isacore | 1 | snack food -is a- food
Common KGs | Wikidata | 6.7k | food -has quality- mouthfeel
| YAGO4 | 116 | banana chip -rdf:type- food
| DOLCE | 1 | n/p
| SUMO | 1614 | food -hyponym- food_product
Lexical resources | WordNet | 10 | food -hyponym- comfort food
| Roget | 2 | dish -synonym- food
| FrameNet | 8(f2f) | eating -evoke- Ingestion
| MetaNet | 14(f2f) | food -has role- food_consumer
| VerbNet | 36(roles) | eating -haspatient- comestible
Visual sources | Visual Genome | 42374 | food -on- plate
| Flickr30k | 1 | eating place -confers with- their kitchen
Corpora& LMs | GenericsKB | n/a | Animals search for food
| GPT-2 | n/a | eating at home will not lead to weight gain
| ChatGPT | n/a | Food provides nourishment: People eat food to..
|  |  | (huge amount of information about eating)
Table 1: Overview of common-sense knowledge sources partly taken from [7]

####

ConceptNet is a multilingual common-sense knowledge graph derived from the
crowd-sourced Open Mind Common Sense project and knowledge from other
resources like WordNet, DBPedia, Games with a purpose, Wiktionary, OpenCyc and
JMDict, connecting words and terms of natural language as nodes with labelled
weighted edges with 37 types of relations [7].

The standard definition of a Common-sense Knowledge Graph is as follows [27]:
G= (V,E,R) is a Common-sense Knowledge Graph, where V, E and R are the node,
edge and relation sets. Edges comprise triplets (h, r, t) where h and t are
head and tail entities connected by relation $r:E={(h,r,t)\mid h\in V,t\in
V,r\in R}$, and nodes are along with a free-text description.

These node relations in ConceptNet5.5 have been categorised [22] to two sub-
classes: Symmetric relations like LocatedNear, RelatedTo, SimilarTo, and
Asymmetric relations such as AtLocation, CapableOf, CreatedBy, DefinedAs,
HasA, and Entails [7]. We use relations of ”RelatedTo”, which demonstrate an
undefined meaning relation between two words. As shown in Figure 1, every node
represents a word extracted from ConceptNet and connected to another node by a
bidirectional edge. These symmetric edges show the relatedness of the
connected words and, therefore, are bidirectional.

Figure 1: An illustration of ConceptNet used by our system.

##  4 One-shot text classification using ILP and MIL

The basis of an interactive learning system is to learn from the user. This
challenge requires incremental learning techniques that help the system to
learn from time to time. In this section, we introduce Meta-Goal Learner
(MGL), a novel machine learning from user interactions. We explore Metagol[4]
learning engine for the task of few-shot learning in MGL.

MGL initially fetches the words of the task (i.e. sentence given by the user)
and obtains relevant background knowledge from ConceptNet. The system can then
learn and categorise the upcoming tasks, as shown in Figure 2. For instance,
when a user writes a task (e.g. ”call mother”), the system will wait for the
first answer/ label from the user and then will consider it as a positive
example. In this case, the positive example represents the task and its
category name. The system then generates relevant background knowledge as
follows:

(a) Model Prototype

(b) MGL illustration

Figure 2: Meta-Goal Learner.

  1. 1.

Tokenise the sentence and generate clauses as part of the background
knowledge:
e.g., contains(X,call). contains(X,mum).

  2. 2.

Send a request to ConceptNet web API for fetching the relations of the words
separately and adding them to the background knowledge:
e.g., related_to(call, phone). related_to(mother, family).

Given the positive example and background knowledge, the system will learn the
hypothesis using ILP. When the user adds the next sentence, the system will
follow steps one and two and checks with the hypotheses learned before. If
there is a match, the system will label the sentence as before; otherwise, the
system will try to learn a new classification rule using the current input
task as a positive example and examples from other tasks as negative examples.
In the next section, we present some of the experimental evaluations of the
learning algorithms which we have been exploring for integration within MGL.

###  4.1 ILP Experiments using the learning engine Aleph

In this experiment, we check whether it is possible to use the ILP learning
engine Aleph[23] for few-shot text classification. Examples and background
knowledge are defined as follows:

Examples We use two positive examples for each of the ”family”, ”work”, and
”sport” categories. For negative examples, we use every example of each
category as the negative example for the other category.

Background Knowledge To provide Background knowledge, we initially omit stop-
words using NLTK library. Then, the system generates predicates for each
sentence in two steps:

  1. 1.

Indicating every word of the sentences in a ”contains” predicate
(e.g. contains([registering,gym],gym)).

  2. 2.

Sending a request to ConceptNet API for each word of the sentence to fetch
their ”related” terms and then generating their ”related_to” predicate (e.g.
related_to(mother,family)).

Results and discussion Using the learning engine Aleph, three rules
(hypotheses) have been generated to classify the three categories.

Rule 1: category(A,family) :- contains(A,B), related_to(B,shop).
Rule 2: category(A,work) :- contains(A,B), related_to(B,letter).
Rule 3: category(A,sport) :- contains(A,B), related_to(B,exercise).

As explained above the training data for each category included 2 positive
examples. We also experimented with one positive example (one-shot learning)
but Aleph was unable to learn any rule. To evaluate the accuracy of the above
rules on separate test data, we use a different dataset with 29 positive
examples, including 15 ”family”, 9 ”work” and 5 ”sport” examples and the
negative test data for each group consists of examples from other groups
(i.e., 15 family examples used as negative for work, 9 work examples are used
as negative for sport, and 5 sport examples are used as negative for family)
and reached to the overall predictive accuracy of 64%.

###  4.2 MIL Experiments using the learning engine Metagol

In this experiment, we first check whether it is possible to learn a
hypothesis from one positive and one negative example using MIL then explore
the possibility of learning more general rules using predicate invention and
recursive rules in MIL.

We employ the learning engine Metagol [15] to utilise MIL with the same
Background Knowledge and examples applied in the Aleph experiment. In addition
to training examples and background knowledge, Metagol also needs metarules to
induce the hypothesis. We test our system with two different types of
metarules. At first, we utilise a chain metarule to obtain the learning
hypothesis. In the second experiment, we use indent, chain, and recursive
metarules along with using constants.

Results and discussion The system induces one hypothesis from one positive and
one negative example. Note that in this experiment, the positive example is a
simple case where the category name is directly related to one of the words in
the sentence. Consequently, one chain metarule is enough to generate the
hypothesis. For instance, in category([call, mother], family), ’mother’
directly connects to ’family’ with only one edge of ’related_to’ relation in
ConceptNet. Hence, the acquired hypothesis consists of one ’related_to’
predicate as follows:

category(A,B) :- contains(A,C), related_to(C,B).

This rule is similar to the rule which was learned by Aleph in the previous
section when we provided at least two positive examples for each category and
used variable in mode declaration but Aleph (unlike Metagol) was unable to
learn from just one positive example. However, the rule above cannot cover all
relevant positive examples as it only considers ’related_to’ once. To
demonstrate this, we provided more training examples (4 positive and 6
negative) which include cases where we need at least two levels of
connections, along with using chain metarule. Consequently, the system invents
a new predicate (i.e., category_1(A,B):- related_to(A,C),related_to(C,B).)
within its hypothesis set as follows:

category(A,B) :- contains(A,C), category_1(C,B).
category_1(A,B):- related_to(A,C),related_to(C,B).

####

The new rule can capture cases such as the category name ’home’ which does not
directly connect to ’mother’ but is connected via ’family’. However, in order
to have a general transitive rule for ’related_to’ we need to learn a
recursive rule.

Finally, we demonstrate that Metagol can be used to learn a recursive rule to
capture transitive ’related_to’ relations. In this experiment, we provide two
positive examples with several relatednesses in their structure. Moreover, we
use constant and variable at the indent metarule and consequently, the
following results including a complete recursive hypothesis are achieved:

category(A,B):-contains(A,C),category_1(C,B).
category_1(A,B):-related_to(A,C),category_1(C,B).
category_1(A,home):-related_to(A,home).

As shown above, it is notable that whatever a chosen example for one-shot
learning has more relatedness in its structure, i.e. higher complexity
compared to the other examples, it could achieve a complete recursive
hypothesis. This, in turn, is especially beneficial for choosing the best
possible shot to do one-shot learning.

To evaluate one-shot learning, we employ the same test dataset used for the
experiments with Aleph in the previous section. As evidenced by the results,
Metagol could achieve around 70% accuracy, while Aleph could not learn any
rule from one example.

In summary, we demonstrated that the Metagol learning engine can be used in
our system to learn text classification rules from one example (one-shot
learning). This feature clearly distinguishes Metagol (MIL) from Aleph
(standard ILP). Moreover, we also demonstrated learning recursive rules in
addition to predicate invention. However, this capability required metarules
defined for the given application. In future, we will explore new
implementations of MIL which can automatically generate the required metarules
from background knowledge and examples.

###  4.3 Experiments comparing MIL with Deep Learning

In this experiment, we use Siamese Network for few-shot learning for text
classification and compare the results with MIL from the previous sections. A
Siamese network includes two identical sub-networks with shared weights. Each
sub-network works simultaneously and compares its outputs at the end [2]. If
the input of the twin sub-networks is the same, they extract similar semantic
features, and the distance between their output will be less; otherwise, the
distance will be more significant. The Siamese Network used in this experiment
is similar to the CNN-based Siamese model described in [28] where each twin
sub-networks consist of a convolutional layer, a max pooling layer and a fully
connected layer. The model sets with a batch size of 50, and a learning rate
of 0.00001. To achieve optimal results, the network is trained over ten
epochs.

Following the approach described in [26], we evaluate the performance of the
Siamese network and compare the results with our one-shot learning approach
using Metagol. The training data for both systems are one positive example and
one or more negative examples (examples from other categories) and the test
data is the same as the one used in the previous section.

(a) Increasing negative samples

(b) Increasing positive samples

Figure 3: Comparison between the average predictive accuracy of MIL (Metagol)
vs Deep Learning (Siamese Net) on Task Classification dataset. (a) learning
with one positive example and one or more negative examples. (b) learning with
one or more positive examples, and the same number of negatives. The training
and test data have been randomly chosen from the Task Classification dataset
and the test data includes 15 positive and 14 negative examples.

As shown in Figure (3(a)), the Siamese model started from 47% average accuracy
with one positive and one negative example and stays around the default
accuracy after learning from 6 examples(50%). Our system achieved an average
accuracy of 70% from one positive and one negative example and maintained
higher average accuracy than the Siamese model throughout. Figure (3(b)),
depicts the average accuracy of our system and Siamese model in the case of an
increasing number of positive examples. It also confirms the outperformance of
our model over Siamese model.

(a) Increasing negative samples

(b) Increasing positive samples

Figure 4: Comparison between the average predictive accuracy of MIL (Metagol)
vs Deep Learning (Siamese Net) on News Category dataset. (a) learning with one
positive example and one or more negative examples. (b) learning with one or
more positive examples, and the same number of negatives. The training and
test data have been randomly chosen from News Category Dataset and the test
data includes 50 positive and 50 negative examples.

To evaluate our model on a publicly available dataset, we use News Category
Dataset [10] which contains news articles from different sources and covers
various topics such as sports, technology and the environment. We randomly
selected five positive examples from the environment news headlines to train
our model and test the gain hypotheses on randomly selected 50 positive and 50
negative samples. As shown in Figure 4, the results suggest the advantage of
our method based on MIL over the Siamese model when we have few training
examples, e.g. one positive and one negative example.

It is noticeable that for training and evaluating a large dataset, we use two
techniques as follows:

  1. 1.

To solve the problem of requiring large amounts of background knowledge for
training and testing, we use the background knowledge of each example
separately.

  2. 2.

To train our model on a large dataset with only one or a few examples (one-
shot learning), we randomly select 10 samples and make an average from several
possible one-shots to calculate the average of different types of one-shots
(ex: the average of possible one positive and one negative example shots or an
average of one positive and two negative examples etc).

##  5 Conclusion

Short text classification is challenging due to the freely constructed
sentence structures and their limited length. This study aims to provide a
novel algorithm to classify short texts called ”tasks” automatically based on
the user’s interests and without having any prior information about user
behaviours. Thus, the first research question sought to determine a reliable
source for providing pertinent background information to feed the system. We
utilised Conceptnet to prepare the required background information related to
the task. Our model was then designed based on different types of ILP and MIL
systems, including Aleph and Metagol. We also used a Siamese Network model to
compare with the deep learning methods. Siamese nets are the most popular DL
approaches for one or few-shot learning. We applied the Convolutional-Siamese
model with one positive and one or more negative examples. According to the
current results, the MIL-based approach achieved 70% accuracy for one-shot
learning, while the accuracy of ILP and Deep Learning was around default
accuracy (50%) in this task.

Finally, we used News Category Dataset as a publicly available dataset to
evaluate our model. This led us to utilise two techniques, including
Background Knowledge Splitting and the Average One Shot Learning approach to
train and evaluate the large dataset. The final results validated MIL’s
superior performance to the Siamese network for one-shot learning from text.

Supplemental Materials
The data and code used in the experiments can be found in the following Github
repository:
https://github.com/ghazalmilani/One-Shot-Learning-from-Text-ICLP2023.git

Acknowledgement
The first author would like to acknowledge EIT-Digital and Goalshaper Ltd,
especially Ms Tracey Carr for supporting her PhD studies. We also thank Dany
Varghese for the discussions related to this project.

## References

  * [1]
  * [2] Davide Chicco (2021): _Siamese Neural Networks: An Overview_ , pp. 73–94.  Springer US, New York, NY, 10.1007/978-1-0716-0826-5_3.
  * [3] Andrew Cropper & Sebastijan Dumančić (2022): _Inductive logic programming at 30: a new introduction_.  Journal of Artificial Intelligence Research 74, pp. 765–850, 10.1613/jair.1.13507.
  * [4] Andrew Cropper & Stephen H. Muggleton (2016): _Metagol System_.  https://github.com/metagol/metagol.  Available at https://github.com/metagol/metagol.
  * [5] Andrew Cropper & Sophie Tourret (2020): _Logical reduction of metarules_.  Machine Learning 109, pp. 1323–1369, 10.1007/s10994-019-05834-x.
  * [6] Ji Han, Serhad Sarica, Feng Shi & Jianxi Luo (2020): _Semantic Networks for Engineering Design: A Survey_.  arXiv:https://arxiv.org/abs/2012.07060.
  * [7] Filip Ilievski, Alessandro Oltramari, Kaixin Ma, Bin Zhang, Deborah L McGuinness & Pedro Szekely (2021): _Dimensions of commonsense knowledge_.  Knowledge-Based Systems 229, p. 107347, 10.1016/j.knosys.2021.107347.
  * [8] Yann LeCun, Yoshua Bengio & Geoffrey Hinton (2015): _Deep learning_.  nature 521(7553), pp. 436–444, 10.1038/nature14539.
  * [9] Dianhuan Lin, Eyal Dechter, Kevin Ellis, Joshua B Tenenbaum & Stephen H Muggleton (2014): _Bias reformulation for one-shot function induction_.  10.3233/978-1-61499-419-0-525.
  * [10] Rishabh Misra (2022): _News category dataset_.  arXiv preprint arXiv:2209.11429, 10.48550/arXiv.2209.11429.
  * [11] Rolf Morel, Andrew Cropper & C.-H. Luke Ong (2019): _Typed Meta-interpretive Learning of Logic Programs_.  In: JELIA, 10.1007/978-3-030-19570-0_13.
  * [12] Stephen Muggleton (2009): _Inverse entailment and progol_.  New Generation Computing 13, pp. 245–286, 10.1007/BF03037227.
  * [13] Stephen Muggleton, Wang-Zhou Dai, Claude Sammut, Alireza Tamaddoni-Nezhad, Jing Wen & Zhi-Hua Zhou (2018): _Meta-interpretive learning from noisy images_.  Machine Learning 107(7), pp. 1097–1118, 10.1007/s10994-018-5710-8.
  * [14] Stephen Muggleton & Luc de Raedt (1994): _Inductive Logic Programming: Theory and methods_.  The Journal of Logic Programming 19-20, pp. 629–679, 10.1016/0743-1066(94)90035-3.  Special Issue: Ten Years of Logic Programming.
  * [15] Stephen Muggleton, Dianhuan Lin, Niels Pahlavi & Alireza Tamaddoni-Nezhad (2014): _Meta-interpretive learning: Application to grammatical inference_.  Machine Learning 94, 10.1007/s10994-013-5358-3.
  * [16] Stephen Muggleton, Dianhuan Lin & Alireza Tamaddoni-Nezhad (2015): _Meta-interpretive learning of higher-order dyadic datalog: predicate invention revisited_.  Machine Learning 100, pp. 49–73, 10.1007/s10994-014-5471-y.
  * [17] Stephen H. Muggleton (2017): _Meta-Interpretive Learning: Achievements and Challenges (Invited Paper)_.  In Stefania Costantini, Enrico Franconi, William Van Woensel, Roman Kontchakov, Fariba Sadri & Dumitru Roman, editors: Rules and Reasoning, Springer International Publishing, Cham, pp. 1–6, 10.1007/978-3-319-61252-2_1.
  * [18] Stephen H Muggleton, Ute Schmid, Christina Zeller, Alireza Tamaddoni-Nezhad & Tarek Besold (2018): _Ultra-strong machine learning: comprehensibility of programs learned with ILP_.  Machine Learning 107, pp. 1119–1140, 10.1007/s10994-018-5707-3.
  * [19] S.H. Nienhuys-Cheng, R. de Wolf, J.G. Carbonell & J. Siekmann (1997): _Foundations of Inductive Logic Programming_.  Lecture Notes in Artificial Intelligence, Springer, 10.1007/3-540-62927-0.
  * [20] S. Patsantzis & S. Muggleton (2022): _Meta-interpretive learning as metarule specialisation_.  Machine Learning, 10.1007/s10994-022-06156-1.
  * [21] SJ Russell & Peter Norvig (2022): _Artificial Intelligence: A Modern Approach, 4th, Global ed_.
  * [22] Robyn Speer, Joshua Chin & Catherine Havasi (2017): _Conceptnet 5.5: An open multilingual graph of general knowledge_.  In: Proceedings of the AAAI conference on artificial intelligence, 31, 10.1609/aaai.v31i1.11164.
  * [23] A Srinivasan (2007): _Aleph: A learning engine for proposing hypotheses_.  Software available at http://web2. comlab. ox. ac. uk/oucl/research/areas/machlearn/Aleph/aleph. pl.  Available at https://www.cs.ox.ac.uk/activities/programinduction/Aleph/aleph.html.
  * [24] Andrew Cropper Stephen Muggleton (2015): _Can predicate invention compensate for incomplete background knowledge?_ In: Thirteenth Scandinavian Conference on Artificial Intelligence: SCAI 2015, 278, IOS Press, p. 27, 10.3233/978-1-61499-589-0-27.
  * [25] Alireza Tamaddoni-Nezhad, David Bohan, Ghazal Afroozi Milani, Alan Raybould & Stephen Muggleton (2021): _Human–Machine Scientific Discovery_ , 10.1093/oso/9780198862536.003.0015.
  * [26] Dany Varghese, Roman Bauer, Daniel Baxter-Beard, Stephen Muggleton & Alireza Tamaddoni-Nezhad (2021): _Human-like rule learning from images using one-shot hypothesis derivation_.  In: International Conference on Inductive Logic Programming, Springer, pp. 234–250, 10.1007/978-3-030-97454-1_17.
  * [27] Bin Wang, Guangtao Wang, Jing Huang, Jiaxuan You, Jure Leskovec & C-C Jay Kuo (2021): _Inductive learning on commonsense knowledge graph completion_.  In: 2021 International Joint Conference on Neural Networks (IJCNN), IEEE, pp. 1–8, 10.1109/IJCNN52387.2021.9534355.
  * [28] Leiming Yan, Yuhui Zheng & Jie Cao (2018): _Few-Shot Learning for Short Text Classification_.  Multimedia Tools Appl. 77(22), p. 29799–29810, 10.1007/s11042-018-5772-4.