# Optical-pump terahertz-probe spectroscopy of the topological crystalline
insulator Pb1-xSnxSe through the topological phase transition

Zhenyang Xiao  [<EMAIL_ADDRESS>Jiashu Wang  [  Xinyu Liu  [  Badih Assaf  [
David Burghoff  [

###### Abstract

Topological crystalline insulators—topological insulators whose properties are
guaranteed by crystalline symmetry—can potentially provide a promising
platform for terahertz optoelectronic devices, as their properties can be
tuned on demand when layered in heterostructures. We perform the first
optical-pump terahertz-probe spectroscopy of topological crystalline
insulators, using them to study the dynamics of Pb1-xSnxSe as a function of
temperature. At low temperatures, excitation of Dirac fermions leads to an
increase in terahertz transmission; from this negative photoconductivity, the
intrasubband relaxation rate of 6 ps is extracted. At high temperatures where
only massive fermions exist, the free-carrier losses induced by the pump
reduce the terahertz transmission for the duration of the 27 ps interband
lifetime. Both effects are present at temperatures near the topological-to-
trivial transition. Our experimental observations provide critical details for
potential applications of Pb1-xSnxSe and provide a direct measurement of the
topological character of Pb1-xSnxSe heterostructures.

###### keywords:

American Chemical Society, LaTeX

University of Notre Dame] Department of Electrical Engineering, University of
Notre Dame, Notre Dame University of Notre Dame] Department of Physics,
University of Notre Dame, Notre Dame University of Notre Dame] Department of
Physics, University of Notre Dame, Notre Dame University of Notre Dame]
Department of Physics, University of Notre Dame, Notre Dame University of
Notre Dame] Department of Electrical Engineering, University of Notre Dame,
Notre Dame IR,NMR,UV

##  1 Introduction

Recent progress in topological insulators (TIs) have pointed to them as
promising platforms for terahertz and far-infrared heterostructure devices1,
2, 3. TIs are novel materials with insulating bulk bandgaps and gapless
surface states, which are promising platforms for devices with novel
functionalities4, 5, 6. TI material can potentially be protected against
defects and can have tunable inverted bandgaps. There are two kinds of TI
materials, the canonical topological materials and the topological crystalline
insulators (TCI). The canonical topological materials are narrow-gap
semiconductors with an inverted band structure, such as bismuth or antimony
chalcogenides7. As for TCIs, they possess a crystalline symmetry that
guarantees the existence of an even number of surface states8. As one of the
well-known TCI materials, lead-tin-selenide (Pb1-xSnxSe) has recently been
reported as an interesting candidate for terahertz optoelectronic
heterostructures, as its properties can be tuned using either the Sn
concentration or the temperature 3. In addition, certain alloy fractions
remain inverted even at room temperature. In principle, this could lead
towards the creation of engineered topological states, whose wavefunctions are
designed to achieve a particular optical or electrical response. For example,
one can imagine terahertz sources9, 10, 11 or detectors12, 13 that operate at
higher temperatures by relying on states that are protected from scattering by
their topological nature, not by a reduction in overlap. However, there is
still a long way to go towards making optoelectronic devices out of these
novel materials. Understanding the carrier dynamics and photoconductivity is
the first obstacle for creating optoelectronic devices.

Optical-pump terahertz-probe spectroscopy (OPTP) and time-resolved angle
resolved photoemission spectroscopy (tr-ARPES) are common methods being used
in the TI dynamics study14, 15, 16, 17. Tr-ARPES is a powerful tool for
studying non-equilibrium quantum states, as it enables imaging of electron
dynamics of an optically excited state directly in momentum space18, 19.
However, the bulk band structure of TIs measured in this method is largely
masked by the topological surface states. In contrast, terahertz time-domain
spectroscopy (THz-TDS) and ultrafast optical-pump terahertz-probe spectroscopy
(OPTP) can directly interrogate the dynamics of free carriers by manipulating
the photoinduced transmittance. Since light penetrates into the TI material,
both bulk and surface carriers’ information can be obtained. They are
particularly useful for investigations of Dirac materials, as has been
demonstrated in studies of the photoinduced carrier dynamics in graphene and
Bi2Se3.20, 21, 22, 23, 24. OPTP systems can provide a time-resolved
measurement for studying the photoexcited carriers generated by ultrafast
infrared pulses, allowing the carrier dynamics related to the photoexcitation
and photorelaxation to both be extracted. As an important 2D material,
graphene has been extensively studied by OPTP measurements, and anomalous
negative photoinduced conductivity has been found. This negative
photoconductivity was first observed in graphene monolayers produced by
chemical vapor deposition (CVD) methods, and it was subsequently observed in
Bi2Se3 thin films as well25, 26.

In this work, we study the carrier dynamics of the TCI Pb1-xSnxSe as a
function of temperature using optical-pump terahertz-probe spectroscopy. At
low temperatures (T=29 K), we observe negative photoconductivity introduced by
the photoexcitation, similar to what has been observed in Bi2Se3 and graphene.
A decay time of about 6 ps is observed for the hot Dirac electrons causing the
negative conductivity. Both positive and negative photoconductivity are
observed at different pump delay positions at moderate temperatures (T=96,
163, and 230 K), which can be explained by the coexistence of the Dirac
fermions from the gapless surface states and the massive fermions from the
bulk states. At room temperature (T=297 K), only massive fermions are present,
as the gapless Dirac cone is gapped, and the induced photoconductivity decays
at a rate of about 27 ps. These observations provide critical details for
potential applications of Pb1-xSnxSe and provide a direct measurement of the
topological character of topological surface states.

##  2 Results and discussion

For this work, we grew a 50 nm quantum well of Pb1-xSnxSe of BaF2 (111) with a
5 nm EuSe capping layer to protect from oxidation. Pb1-xSnxSe is a well-known
TCI material that can exhibit a topological transition from a trivial
insulator to TCI under modification of chemical composition, temperature, or
pressure 7, while EuSe is topologically trivial. The layers are grown
following the procedure described in Ref. 27, directly on the BaF2 surface,
without the buffer layer stack. The band structure parameters of bulk
Pb1-xSnxSe has been investigated at different Sn compositions and
temperatures, with a band gap $E_{g}$ that can be approximately expressed by28

$\displaystyle E_{g}(x,T)=$ $\displaystyle
E_{g}(x,4.2)+2\left(\sqrt{22.5^{2}+\left(0.32\,$\mathrm{K}^{-1}$\right)^{2}(T-4.2\,$\mathrm{K}$)^{2}}-22.5\right)($\mathrm{m}\mathrm{e}\mathrm{V}$)$
(1) $\displaystyle
E_{g}(x,4.2)=2\left(72-710x+2440x^{2}-4750x^{3}\right)($\mathrm{m}\mathrm{e}\mathrm{V}$),$

where x is the composition fraction of the Sn, T is the temperature, and
$E_{g}(x,4.2)$ is the energy gap at 4.2 K. Equation 1 is most accurate when
$0<x<0.3$ and 4.2 K<T<200 K. The Sn content of the films can deduced from
X-ray diffraction measurements using Vegard’s law as done in Ref. 28. We find
x=0.21$\pm$0.1. We approximate the value of the energy gap at 4.2 K to be
$E_{g}=-30\pm 5$\mathrm{m}\mathrm{e}\mathrm{V}$$ from Equation 1. Hall
measurements indicate that the samples are p-doped at low temperature with an
estimated hole concentration of $7.8\times
10^{18}\,$\mathrm{c}\mathrm{m}$^{-3}$.

The band structure of the 50 nm Pb1-xSnxSe (x=0.21) can now be calculated
using an envelope function method29, previously used in Refs. 28 and 27.
Following the basis and model shown in Ref. 27, the k.p matrix Hamiltonian of
Pb1-xSnxSe, is utilized for this operation, with $k_{z}=-i\partial_{z}$. The
energy gap of the TCI well is fixed at $E_{g}=-30$
$\mathrm{m}\mathrm{e}\mathrm{V}$ at low temperature. The energy gap in the
barrier is fixed to 2 $\mathrm{e}\mathrm{V}$. Both the substrate and the
capping layer have a wide energy gap exceeding 1 $\mathrm{e}\mathrm{V}$,
however, we have checked that varying energy gap of the barriers in the model
above 1 $\mathrm{e}\mathrm{V}$ has a minimal impact on the band structure of
the Dirac states of the material. The band offsets of the conduction and the
valence bands are assumed equal and opposite in sign to account for the band
inversion that takes place at the interface.

We fix the k.p matrix along the growth direction to $\hbar\nu_{z}=2.6$
$\mathrm{e}\mathrm{V}\mathrm{\SIUnitSymbolAngstrom}$ and we assume the bulk
valleys to be isotropic so that $\nu_{z}=\nu_{x}$, as typically seen for
Pb1-xSnxSe with x>0.20.28 The model yields the band dispersion shown in Fig.
1a, with gapless Dirac surface states dispersing the energy gap formed by the
quantum well subbands. To model the temperature dependence of the band
structure, we use the energy dependent gap empirically determined from
Equation 1, a repeat of our calculations with $E_{g}(0.21,T)$ implemented in
the model for various T. The model yields the energy gap of the topological
surface states (TSS) in the topological regime as it evolves from 0
$\mathrm{m}\mathrm{e}\mathrm{V}$ up to a few meV when $E_{g}(0.21,T)$ is small
but negative. When $E_{g}(0.21,T)$ becomes positive, the model is equivalent
to that of a conventional narrow gap semiconductor and yields the energy
separation between the top most hole subband H1 and bottom most electron
subband E1. The evolution of the TSS gap and that of E1-H1 gap is shown in
Fig. 1b.

Figure 1: System and Pb1-xSnxSe (x=0.21) sample information. a. Calculated
band diagram of the Pb1-xSnxSe (x=0.21) film at 4.2 K (left), and zone center
($k_{t}=0$) wavefunctions (right). b. Gap information of Pb1-xSnxSe (x=0.21)
versus temperature. The bulk bandgap is represented by the blue squares, the
bandgaps of the topological surface states (TSSs) are represented by green
squares, and the red squares represent the E1 gap (the gap of the lowest
subband when trivial). c. Simplified version of the optical-pump terahertz-
probe spectroscopy (OPTP) measurement.

As can be seen in Fig. 1b, a topological phase transition is seen to happen at
110$\pm$15 K (the uncertainty is due to the propagated uncertainty of
$E_{g}$). When the temperature goes lower than this point, the topological
surface states (TSS) with a gapless Dirac point appears, and the carrier in
those states become so-called Dirac fermions. With our knowledge of the bulk
gap, we can compute the band dispersion of the Dirac states and its gap
acquisition through transition. The unique massless Dirac fermions in the
surface states are the key to understand the dynamics present in this
material. In order to study those dynamics, OPTP is used. The principle of
this system can be seen in Fig. 1c: a high-power short-pulse laser is used to
excite both a photoconductive antenna (PCA) and the sample, and the terahertz
generated by the PCA is passed through the sample and modified by the
photoexcited carriers. The terahertz signal is then detected using electro-
optic sampling in a ZnSe crystal and balanced detection. For this measurement,
the pulse used to excite the sample was at 800 nm (1.55 eV) with a duration of
86 fs. Because we were limited by the high repetition rate of our Ti:Sapphire
laser, the optical pump beam’s fluence at the sample was limited to 1.6
$\mu\mathrm{J}\mathrm{/}\mathrm{c}\mathrm{m}^{2}$ for all measurements. The
bandwidth of our system is about 6 THz, and the SNR is about 60 dB. A dual
lock-in amplifier scheme is used to remove long-term drift: the pump beam is
chopped, and two lock-in amplifiers are used to measure the transmitted
terahertz. One represents the difference in terahertz with the pump on versus
the pump off, while the other represents the average. These two signals can be
combined to produce the spectrum with the pump on versus the pump off.

Using the time-domain system, we first measured the transmittance of the
Pb1-xSnxSe (x=0.21) sample as the temperature is changed. As shown in Fig. 2a,
we focus on five temperature points, two of which (29 K and 96 K) are well
below the temperature point at which there exist topological surface states,
and three that are above (163 K, 230 K and 297 K). The transmittance is
calculated by comparing the signal through a substrate without the TCI to one
that is measured with the TCI. Due to the lower SNR at the two sides of the
spectrum, we select only the highest SNR region from 0.7 THz to 2 THz for
analysis—below the Reststrahlen band of the BaF2 substrate—and fit the
transmittance data using a Drude-Lorentz model with two resonant frequencies.
Two pronounced resonances are observed: one at 1.7 THz (7 meV) that is
relatively constant in temperature, and one lower-frequency branch that is
strongly temperature-dependent, below 1.2 THz (5 meV). Neither can be
attributed to direct electronic transitions, as the energies are too low to be
explained by intersubband absorption ($\Delta E>14$
$\mathrm{m}\mathrm{e}\mathrm{V}$ for all occupied subbands). We attribute the
constant-temperature resonance as arising from weak coupling to bulk TO
phonons, as this frequency occurs close to the value previously measured by
FTIR spectroscopy (50 $\mathrm{c}\mathrm{m}^{-1}$)30. The strongly
temperature-dependent resonance can perhaps be best explained as coupling to
shear vertical surface modes, which were previously found (in Pb0.7Sn0.3Se) to
have energies that strongly depend on the emergence of Dirac fermions,
shifting to low energies at temperatures below the transition 31. This effect
arises as a result of the strong electron-phonon interaction but requires
further study.

Figure 2:  a. Pb1-xSnxSe (x=0.21) transmittance at different temperatures.
Measurements are indicated by circles and the Drude-Lorentz fittings are
indicated by lines. b. Resonance frequencies as a function of temperature. The
widths of the curves represent one standard deviation.

After the transmittance spectra of the Pb1-xSnxSe sample were obtained using
normal time-domain spectroscopy, OPTP measurements were performed. By locating
the first stage of the TDS signal at the point marked by the black dashed line
in Fig. 3b and scanning the delay of the high-power infrared pump pulse,
information about the time-resolved carrier dynamics could be measured, as is
shown in Fig. 3a. Figs. 3b, 3c, and 3d shows the time scale of the detailed OP
signal. In Fig. 3c, one can see that the transmitted field reaches a maximum
at a pump-probe delay of 1.6 ps (with a 1 ps rise time), and then decays over
the next 6 ps. In order to investigate the frequency response, the change in
the transmitted signal induced by the pump was plotted at several pump-probe
delays. Each curve in Fig. 3d represents the transmitted pulse at the
different time points marked with the same color dash line in 3c. These are
also converted into the frequency domain. Note that the optical pump does not
induce a strong frequency dependence in the transmission, as is expected of
the optical properties of Dirac cones.

Figure 3:  a. Effect of an optical pump on the amplitude of the transmitted
terahertz pulse at different temperatures, offset for clarity. (The dashed
lines indicate the offset for each curve.) An increase in transmission is
notable at low temperatures, while only a decrease is evident at high
temperatures. b. Transmitted TDS pulse shape of Pb1-xSnxSe at 29 K. c. OP
pulse-induced difference of THz field through Pb1-xSnxSe at 29 K as a function
of pump-probe delay. d. Dual lock-in difference signal measured at different
time points marked in c, in both time (left) and frequency (right) domain.

Fig. 3a illustrates the photoexcited carrier dynamics as a function of
temperature. At room temperature (T=297 K), the transmittance of the sample
decreases as the pump pulse arrives at the sample. This is a well-explained
phenomenon that is justified by increase of the number of free carriers. At
this temperature point, the sample has a bandgap around 115 meV, which makes
its behavior similar to conventional semiconductors: only massive fermions are
present, the gapless Dirac cone doesn’t exist, and due to the increase of the
free carrier concentration, the transmittance decreases after photoexcitation.
The induced photoconductivity at room temperature decays at a rate of about 27
ps due to interband scattering. However, at low temperatures (T=29 K), an
increase in transmittance after the sample is excited is observed. This is an
anomalous phenomenon that has also been observed in some well-explored 2D
materials, such as Bi2Se3 and graphene.20, 21, 22, 23, 25, 32. A decay time of
about 6 ps is observed. A combination of positive and negative
photoconductivity is observed at different pump delay positions at moderate
temperatures (T=96, 163, 230 K) which can be explained by the coexistence of
the Dirac fermions from the gapless surface states and the massive fermions
from the bulk states. We also find that the strength of the pump-induced
signal is also changing with temperature. As the temperature is lowered, the
effect of the massive electrons is reduced, as the low fluence ensures that
the incident light is screened by the Dirac fermions.

To further explore the anomalous transmittance increase at low temperatures,
we calculate the photoconductivity of the sample at 29 K using33

$T(\omega)=\frac{1+n}{(1+n+Z_{0}G(\omega))e^{i\phi}},$ (2)

where $T(\omega)$ is the complex transmittance of the sample, n is the
refractive index of the substrate, $Z_{0}$ is the impedance of free space,
$G\left(\omega\right)$ is the conductivity, and $\phi$ is the phase difference
introduced by the thickness difference of the substrate and the Pb1-xSnxSe
sample.

Figure 4:  a. Real and imaginary part of the conductivity at 29 K with no
optical pump. b. Real and imaginary part of the photoinduced conductivity at
29 K and 297 K.

Fig. 4a shows the calculated real and imaginary part of the conductivity as a
function of frequency for the unpumped sample. The pumped sample is very
similar, so in Fig. 4b we plot the differential conductivity. To better
understand the physical origin of the photoinduced conductivity here, the
conductivity is fitted with a two-term Drude-Lorentz model, including a Drude
term, a Drude-Lorentz term (which modelling the TO phonon resonance), and a
frequency-independent contribution to the dielectric constant (which accounts
for the effect of higher-energy excitations):34, 26

$G_{j}(\omega)=\left(-\frac{\omega_{pD}^{2}}{i\omega-\Gamma_{D}}-\frac{i\omega\omega_{pDL}^{2}}{\omega_{DL}^{2}-\omega^{2}-i\omega\Gamma_{DL}}-i\left(\epsilon_{\infty}-1\right)\omega\right)\epsilon_{0}d,$
(3)

where $\omega_{pD}$ is the plasma frequency of the Drude oscillator,
$\Gamma_{D}$ is the scattering rate of the Drude oscillator,
$\epsilon_{\infty}$ is the dielectric constant at high frequency, d is the
sample thickness, $\omega_{pDL}$ and $\Gamma_{D}$ are the parameters for the
Lorentz oscillator.

The origin of the negative photoinduced conductivity can be understood by
considering the Drude conductivity in the low-frequency limit. The low-
frequency conductivity dynamics are governed by the Drude weight
$D_{Drude}=\frac{d\omega_{pD}^{2}}{(2\pi)^{2}}$ and the scattering rate
$\Gamma_{Drude}$, which is related to the carrier concentration and the
mobility respectively. The real part of the conductivity is determined by the
ratio of the Drude weight and the scattering rate. For conventional
semiconductors with bandgaps, the real part of photoinduced conductivity
typically increases since the photoinduced carriers increase the Drude weight.
For Dirac cones, this effect is typically offset by an increase in scattering
rate, as the photoexcited electrons become hot 26. At room temperature, the
Pb1-xSnxSe sample possesses only massive states, which makes it behave like
conventional semiconductors. However, as lower temperature, the surface states
become gapless and the increase of scattering rate becomes dominant, leading
to a negative photoinduced conductivity.

To confirm that the scattering rate increases, we fit the complex conductance
spectra to the same Drude-Lorentz model with and without the pump. While the
Drude weight $D_{Drude}$ of the pumped Pb1-xSnxSe (x=0.21) sample increased by
2.1%, consistent with a small increase in the density of free carriers, a much
larger increase of the scattering rate, $\Delta\Gamma_{D}$=47.8%, is observed.
As a result, the ratio $\frac{D_{Drude}}{\Gamma_{Drude}}$ acutally decreases.
This causes the real part of the photoinduced conductivity, $\delta
G\equiv{G_{pumped}-G_{unpumped}}$, to be negative as is shown in Fig. 4b. This
explains the anomalous increase of the signal after the photoexcitation.
Therefore, the 6 ps decay rate of the negative photoconductivity should be
interpreted as an intrasubband (i.e., intra-Dirac cone) relaxation rate, not
as a lifetime of the Dirac fermions.

##  3 Conclusion

In conclusion, we have used optical-pump terahertz probe spectroscopy to
characterize TCI Pb1-xSnxSe heterostructures. At low temperatures where the
topological transition happens, one observes a negative photoinduced
conductivity immediately after the photoexcitation, a result of the generation
of hot surface state Dirac fermions. The relaxation of these Dirac fermions is
relatively fast, occurring in about 6 ps. At room temperature, only normal
massive carriers are observed, and the transmittance behaves like a normal
photo-excited semiconductor, with a carrier lifetime of about 27 ps. At
moderate temperatures, both carriers are observed. Our observations provide
critical details for potential applications of Pb1-xSnxSe and provide a direct
measurement of the topological character of Pb1-xSnxSe thin films, which will
be of use for the characterization of more sophisticated bandstructure-
engineered heterostructures.

All authors acknowledge support from the NDnano seed grant program. D.B. and
Z.X acknowledge support from the AFOSR under grant no. FA9550-20-1-0192, from
the NSF under ECCS-2046772, and from ONR under N00014-21-1-2735. BA.A, J.W and
X.L acknowledge support from NSF1905277 and DMR-1905277.

## References

  * Fu 2011 Fu, L. Topological Crystalline Insulators. _Physical Review Letters_ 2011, _106_ , 106802
  * Zhang et al. 2010 Zhang, X.; Wang, J.; Zhang, S.-C. Topological insulators for high-performance terahertz to infrared applications. _Physical Review B_ 2010, _82_ , 245107
  * Krizman et al. 2018 Krizman, G.; Assaf, B.; Phuphachong, T.; Bauer, G.; Springholz, G.; Bastard, G.; Ferreira, R.; de Vaulchier, L.; Guldner, Y. Tunable Dirac interface states in topological superlattices. _Physical Review B_ 2018, _98_ , 075303
  * Hasan and Kane 2010 Hasan, M. Z.; Kane, C. L. Colloquium: topological insulators. _Reviews of modern physics_ 2010, _82_ , 3045
  * Qi and Zhang 2011 Qi, X.-L.; Zhang, S.-C. Topological insulators and superconductors. _Reviews of Modern Physics_ 2011, _83_ , 1057
  * Hasan and Moore 2011 Hasan, M. Z.; Moore, J. E. Three-dimensional topological insulators. _Annu. Rev. Condens. Matter Phys._ 2011, _2_ , 55–78
  * Woźny et al. 2020 Woźny, M.; Szuszkiewicz, W.; Dyksik, M.; Motyka, M.; Szczerbakow, A.; Bardyszewski, W.; Story, T.; Cebulski, J. Electron-phonon coupling and a resonant-like optical observation of a band inversion in topological crystalline insulator Pb $\\_${$1-x$} Sn $\\_x$ Se. _arXiv preprint arXiv:2003.11849_ 2020,
  * Hsieh et al. 2012 Hsieh, T. H.; Lin, H.; Liu, J.; Duan, W.; Bansil, A.; Fu, L. Topological crystalline insulators in the SnTe material class. _Nature communications_ 2012, _3_ , 1–7
  * Bosco et al. 2019 Bosco, L.; Franckié, M.; Scalari, G.; Beck, M.; Wacker, A.; Faist, J. Thermoelectrically cooled THz quantum cascade laser operating up to 210 K. _Applied Physics Letters_ 2019, _115_ , 010601
  * Kainz et al. 2019 Kainz, M. A.; Semtsiv, M. P.; Tsianos, G.; Kurlov, S.; Masselink, W. T.; Schönhuber, S.; Detz, H.; Schrenk, W.; Unterrainer, K.; Strasser, G., et al. Thermoelectric-cooled terahertz quantum cascade lasers. _Optics express_ 2019, _27_ , 20688–20693
  * Dhillon et al. 2017 Dhillon, S.; Vitiello, M.; Linfield, E.; Davies, A.; Hoffmann, M. C.; Booske, J.; Paoloni, C.; Gensch, M.; Weightman, P.; Williams, G., et al. The 2017 terahertz science and technology roadmap. _Journal of Physics D: Applied Physics_ 2017, _50_ , 043001
  * Palaferri et al. 2018 Palaferri, D.; Todorov, Y.; Gacemi, D.; Barbieri, S.; Li, L.; Davies, A.; Linfield, E.; Sirtori, C. Noise characterization of patch antenna THz photodetectors. _Applied physics letters_ 2018, _113_ , 161105
  * Micheletti et al. 2021 Micheletti, P.; Faist, J.; Olariu, T.; Senica, U.; Beck, M.; Scalari, G. Regenerative terahertz quantum detectors. _APL Photonics_ 2021, _6_ , 106102
  * Hsieh et al. 2011 Hsieh, D.; Mahmood, F.; McIver, J.; Gardner, D.; Lee, Y.; Gedik, N. Selective probing of photoinduced charge and spin dynamics in the bulk and surface of a topological insulator. _Physical Review Letters_ 2011, _107_ , 077401
  * Kumar et al. 2011 Kumar, N.; Ruzicka, B. A.; Butch, N.; Syers, P.; Kirshenbaum, K.; Paglione, J.; Zhao, H. Spatially resolved femtosecond pump-probe study of topological insulator Bi 2 Se 3. _Physical Review B_ 2011, _83_ , 235306
  * Sobota et al. 2012 Sobota, J. A.; Yang, S.; Analytis, J. G.; Chen, Y.; Fisher, I. R.; Kirchmann, P. S.; Shen, Z.-X. Ultrafast optical excitation of a persistent surface-state population in the topological insulator Bi 2 Se 3. _Physical review letters_ 2012, _108_ , 117403
  * Wang et al. 2012 Wang, Y.; Hsieh, D.; Sie, E.; Steinberg, H.; Gardner, D.; Lee, Y.; Jarillo-Herrero, P.; Gedik, N. Measurement of intrinsic Dirac fermion cooling on the surface of the topological insulator Bi 2 Se 3 using time-resolved and angle-resolved photoemission spectroscopy. _Physical Review Letters_ 2012, _109_ , 127401
  * Hajlaoui et al. 2013 Hajlaoui, M.; Papalazarou, E.; Mauchain, J.; Jiang, Z.; Miotkowski, I.; Chen, Y.; Taleb-Ibrahimi, A.; Perfetti, L.; Marsi, M. Time resolved ultrafast ARPES for the study of topological insulators: The case of Bi 2 Te 3. _The European Physical Journal Special Topics_ 2013, _222_ , 1271–1275
  * Christiansen et al. 2019 Christiansen, D.; Selig, M.; Malic, E.; Ernstorfer, R.; Knorr, A. Theory of exciton dynamics in time-resolved ARPES: Intra-and intervalley scattering in two-dimensional semiconductors. _Physical Review B_ 2019, _100_ , 205401
  * Shi et al. 2014 Shi, S.-F.; Tang, T.-T.; Zeng, B.; Ju, L.; Zhou, Q.; Zettl, A.; Wang, F. Controlling Graphene Ultrafast Hot Carrier Response from Metal-like to Semiconductor-like by Electrostatic Gating. _Nano Letters_ 2014, _14_ , 1578–1582
  * Hafez et al. 2017 Hafez, H. A.; Chai, X.; Sekine, Y.; Takamura, M.; Oguri, K.; Al-Naib, I.; Dignam, M.; Hibino, H.; Ozaki, T. Effects of environmental conditions on the ultrafast carrier dynamics in graphene revealed by terahertz spectroscopy. _Physical Review B_ 2017, _95_ , 165428
  * Jnawali et al. 2013 Jnawali, G.; Rao, Y.; Yan, H.; Heinz, T. F. Observation of a Transient Decrease in Terahertz Conductivity of Single-Layer Graphene Induced by Ultrafast Optical Excitation. _Nano Letters_ 2013, _13_ , 524–530
  * Lu et al. 2017 Lu, J.; Liu, H.; Sun, J. Negative terahertz photoconductivity in 2D layered materials. _Nanotechnology_ 2017, _28_ , 464001, Publisher: IOP Publishing
  * Yang et al. 2020 Yang, X.; Luo, L.; Vaswani, C.; Zhao, X.; Yao, Y.; Cheng, D.; Liu, Z.; Kim, R. H. J.; Liu, X.; Dobrowolska-Furdyna, M.; Furdyna, J. K.; Perakis, I. E.; Wang, C.; Ho, K.; Wang, J. Light Control of Surface–Bulk Coupling by Terahertz Vibrational Coherence in a Topological Insulator. _npj Quantum Materials_ 2020, _5_ , 1–7
  * Valdés Aguilar et al. 2015 Valdés Aguilar, R.; Qi, J.; Brahlek, M.; Bansal, N.; Azad, A.; Bowlan, J.; Oh, S.; Taylor, A. J.; Prasankumar, R. P.; Yarotski, D. A. Time-resolved terahertz dynamics in thin films of the topological insulator Bi2Se3. _Applied Physics Letters_ 2015, _106_ , 011901, Publisher: American Institute of Physics
  * Sim et al. 2014 Sim, S.; Brahlek, M.; Koirala, N.; Cha, S.; Oh, S.; Choi, H. Ultrafast terahertz dynamics of hot Dirac-electron surface scattering in the topological insulator Bi 2 Se 3. _Physical Review B_ 2014, _89_ , 165137
  * Wang et al. 2020 Wang, J.; Liu, X.; Bunker, C.; Riney, L.; Qing, B.; Bac, S.; Zhukovskyi, M.; Orlova, T.; Rouvimov, S.; Dobrowolska, M., et al. Weak antilocalization beyond the fully diffusive regime in Pb 1- x Sn x Se topological quantum wells. _Physical Review B_ 2020, _102_ , 155307
  * Krizman et al. 2018 Krizman, G.; Assaf, B.; Phuphachong, T.; Bauer, G.; Springholz, G.; de Vaulchier, L.; Guldner, Y. Dirac parameters and topological phase diagram of P b 1- x S n x Se from magnetospectroscopy. _Physical Review B_ 2018, _98_ , 245202
  * Bastard 1990 Bastard, G. Wave mechanics applied to semiconductor heterostructures. 1990,
  * Novikova et al. 2018 Novikova, N.; Yakovlev, V.; Kucherenko, I.; Vinogradov, V.; Aleschenko, Y.; Muratov, A.; Karczewski, G.; Chusnutdinow, S. Infrared Reflection Spectra of PbSnSe (x= 0.2, 0.34) Topological Insulator Films on a ZnTe/GaAs Substrate and the Vibrational Modes of Multilayer Structures. _Semiconductors_ 2018, _52_
  * Kalish et al. 2019 Kalish, S.; Chamon, C.; El-Batanouny, M.; Santos, L.; Sankar, R.; Chou, F. Contrasting the Surface Phonon Dispersion of Pb 0.7 Sn 0.3 Se in Its Topologically Trivial and Nontrivial Phases. _Physical review letters_ 2019, _122_ , 116101
  * Tielrooij et al. 2013 Tielrooij, K.; Song, J.; Jensen, S. A.; Centeno, A.; Pesquera, A.; Elorza, A. Z.; Bonn, M.; Levitov, L.; Koppens, F. Photoexcitation cascade and multiple hot-carrier generation in graphene. _Nature Physics_ 2013, _9_ , 248–252
  * Wu et al. 2013 Wu, L.; Brahlek, M.; Aguilar, R. V.; Stier, A.; Morris, C.; Lubashevsky, Y.; Bilbro, L.; Bansal, N.; Oh, S.; Armitage, N. A sudden collapse in the transport lifetime across the topological phase transition in (Bi 1- x In x) 2 Se 3. _Nature Physics_ 2013, _9_ , 410–414
  * Valdés Aguilar et al. 2013 Valdés Aguilar, R.; Wu, L.; Stier, A.; Bilbro, L.; Brahlek, M.; Bansal, N.; Oh, S.; Armitage, N. P. Aging and reduced bulk conductance in thin films of the topological insulator Bi2Se3. _Journal of Applied Physics_ 2013, _113_ , 153702


# MetaDT: Meta Decision Tree for Interpretable Few-Shot Learning

Baoquan Zhang, Hao Jiang, Xutao Li, Shanshan Feng, Yunming Ye, Rui Ye
Harbin Institute of Technology, Shenzhen
{zhangbaoquan<EMAIL_ADDRESS>{lixutao, victor_fengss,
<EMAIL_ADDRESS>
<EMAIL_ADDRESS>Corresponding author.

###### Abstract

Few-Shot Learning (FSL) is a challenging task, which aims to recognize novel
classes with few examples. Recently, lots of methods have been proposed from
the perspective of meta-learning and representation learning for improving FSL
performance. However, few works focus on the interpretability of FSL decision
process. In this paper, we take a step towards the interpretable FSL by
proposing a novel decision tree-based meta-learning framework, namely, MetaDT.
Our insight is replacing the last black-box FSL classifier of the existing
representation learning methods by an interpretable decision tree with meta-
learning. The key challenge is how to effectively learn the decision tree
(i.e., the tree structure and the parameters of each node) in the FSL setting.
To address the challenge, we introduce a tree-like class hierarchy as our
prior: 1) the hierarchy is directly employed as the tree structure; 2) by
regarding the class hierarchy as an undirected graph, a graph convolution-
based decision tree inference network is designed as our meta-learner to learn
to infer the parameters of each node. At last, a two-loop optimization
mechanism is incorporated into our framework for a fast adaptation of the
decision tree with few examples. Extensive experiments on performance
comparison and interpretability analysis show the effectiveness and
superiority of our MetaDT. Our code will be publicly available upon
acceptance.

##  1 Introduction

Deep convolutional neural network (CNN) has achieved great success on image
classification with abundant labeled data [16]. However, for many rare or new-
found objects, acquiring so much labeled data is unrealistic, which limits
their applications in practical scenarios such as drug discovery [1] and cold-
start recommendations [39]. In contrast, humans can quickly learn and
recognize novel classes from very few observations. To bridge the gap, Few-
Shot Learning (FSL) problem has been proposed and has attracted wide attention
recently. It targets at learning transferable knowledge from some base classes
with sufficient labeled samples, and then transferring the knowledge to
quickly learn a classifier for novel classes with few examples [44].

(a) Existing representation learning-based methods.

(b) Our MetaDT.

Figure 1: Different from existing representation learning methods (a), our
MetaDT replaces its black-box classifier by a decision tree with meta-learning
(b), which provides interpretability for FSL.

To address the FSL problem, various methods have been proposed, which can be
roughly divided into two categories: meta-learning based methods and
representation learning-based methods. The former aims to learn a task-
agnostic meta-learner (_e.g._ , a good initial model [13]) by constructing a
large number of few-shot tasks from base classes, and then leverages the meta-
learner to quickly learn/infer a classifier for novel classes. The latter aims
to learn transferable representations from base classes by designing a good
training strategy [30] or feature extractor [47], so that the novel classes
can be nicely recognized via a simple classifier (_e.g._ , cosine classifier
[9]). Though both types of methods have achieved promising performance on FSL,
they mainly focus on improving FSL performance, but have not paid sufficient
attention to the interpretability of FSL. In fact, such ability is very
important in risk-sensitive applications (_e.g._ , medical diagnosis [4]),
where the model not only needs to make a decision (_e.g._ , the clinical
outcome of a patient), but also to explain the reason of making such decision.

Recently, several works [6, 48, 43] attempt to explore interpretable FSL,
_i.e._ , recognizing novel classes meanwhile providing a clear decision
explanation for making the class prediction. However, almost all methods focus
on explaining how FSL models work from the perspective of heatmap
visualization, _e.g._ , visualizing the importance of image regions [48].
These methods only roughly locate and visualize the image region that the FSL
model focuses on, but it is still unclear why the FSL model makes such
decision.

To address the drawback, in this paper, we present a new perspective for
interpretable FSL by proposing a novel decision tree-based meta-learning
framework, namely, MetaDT. As shown in Figure 1(a) and 1(b), our key idea is
1) replacing the last black-box FSL classifier (_e.g._ , cosine classifier) of
the representation learning methods by an interpretable decision tree with
meta-learning; 2) introducing a tree-like class hierarchy consisting of few-
shot classes (_e.g._ , “English setter” or “Dalmatian”) and their superclasses
(_e.g._ , “Spotted dog”) as the decision tree, where each node in the tree is
a learnable model making each step decision (_e.g._ , “Is spotted dog?”); and
3) performing novel class prediction in a sequential decision manner from the
root of decision tree to its leafs. The advantage of such design is that a
sequence of intermediate decisions that lead up to a final class prediction
can be obtained, which helps to diagnose why the FSL model makes such
prediction. For example, as shown in Figure. 1(b), the image of “Labrador
retriever” is wrongly predicted to “Dalmatian”. By examining the intermediate
decisions, we can identify the reason for such wrong decision is that the
image is misclassified as “Spotted dog” when making the decision of “Is
spotted dog or solid dog?”.

Specifically, the key challenge of such idea is how to effectively learn the
decision tree (_i.e._ , the tree structure and the parameters of each node) in
the FSL setting. To address the challenge, we regard the introduced class
hierarchy as a determined structure prior of the decision tree. By viewing it
as an undirected graph, we design a graph convolution-based decision tree
inference network as our meta-learner to learn to infer the parameters of each
node. The advantage of such design is the class hierarchy relations and
abundant class semantics can be fully and effectively exploited in our meta-
learner. Besides, we incorporate a two-loop optimization mechanism [13] into
our framework for a fast adaptation. The outer-loop optimization aims to learn
good initial parameters for our meta-learner from abundant base classes. Then,
the inner-loop optimization applies the initial parameters to novel classes
for quickly inferring a task-specific decision tree with few examples. At
last, we perform the novel class prediction in a sequential decision manner
from the root of the task-specific decision tree to its leaf nodes.

Our main contributions can be summarized as follows:

  * •

We present a new perspective for interpretable FSL by replacing the black-box
FSL classifier with an interpretable decision tree, which provides clear
decision explanations for class prediction. To our knowledge, this is the
first work to explore decision trees on FSL.

  * •

We propose a novel decision tree-based meta-learning framework. Here, a graph
convolution-based decision tree inference network with class hierarchy is
carefully designed for effectively learning a decision tree. Its advantage is
the priors of class hierarchy (_i.e._ , class semantics and hierarchy
relations) can be fully exploited.

  * •

We conduct comprehensive experiments on miniImagenet, CIFAR-FS, and
tieredImagenet, which verify the effectiveness of our MetaDT. In addition,
extensive interpretability and visualization analyses show the superiority of
our MetaDT for interpretable FSL.

##  2 Related Work

###  2.1 Few-Shot Learning

Few-Shot Learning (FSL) aims to recognize novel classes with few labeled
samples. Existing methods can be roughly grouped into two categories: meta-
learning based approaches and representation learning-based approaches. The
meta-learning based approaches focus on learning a task-agnostic meta-learner
by constructing a large number of few-shot tasks from base classes, and then
leverage the meta-learner to quickly learn/infer an FSL classifier for
recognizing novel classes. The meta-learner can be a good initial model [13,
32], optimization algorithm [3, 53], embedding network [36, 21, 55], metric
strategy [45, 56, 23], or label propagation strategy[37, 34, 50, 31], etc. The
representation learning-based approaches aim to design a good feature
extractor [52, 47] or training strategy [11, 38, 8, 25] to learn transferable
representations from abundant base classes, so that the novel class samples
can be recognized by a simple cosine classifier [9] or logistic regression
classifier [30].

Recently, some studies attempt to introduce some external knowledge as
auxiliary priors to further boost the performance [22, 46, 35, 27, 54] of
existing FSL methods. For example, in [46, 22], the authors extend [36] by
introducing the semantic information of class names as priors, to enhance
class prototypes for FSL. In [27], they follow the representation learning-
based approaches and introduce class hierarchies as priors to further boost
cosine classifier [9] for FSL. Different from these methods, our MetaDT
focuses on interpretable FSL and introduces the priors of class hierarchy to
construct and infer a novel decision tree for interpretable FSL. Its advantage
is that more superclasses can be exploited to improve knowledge transfer and
detailed decision explanations can be provided for class prediction.

Figure 2: The framework of our MetaDT. Given an episode, we first construct a
class hierarchy as inputs of DTINet. Second, a task-specific IDDTree is
inferred by finetuning the DTINet with $M$-step gradient updates on support
set $\mathcal{S}$. Finally, the IDDTree is evaluated on query set
$\mathcal{Q}$ and the evaluated results are used to train our DTINet in meta-
training. In meta-test, the evaluated results are used to assign labels.

###  2.2 Zero-Shot Learning

Zero-Shot Learning (ZSL) is a challenging machine learning task, which targets
at recognizing novel classes without any labeled examples by resorting to some
semantic knowledge summarized from human’s past experiences [42, 14]. The main
idea is regarding class-level semantic knowledge as the auxiliary information,
and then learning a map function between semantic knowledge and class
representations for novel class prediction. The semantic knowledge can be
class attributions [42], class names [46], class descriptions [12], or class
hierarchies [17]. Our work differs from these models in two aspects: 1) our
MetaDT focuses on interpretable FSL, where few labeled samples should be fully
exploited; 2) resorting to the semantic knowledge of class hierarchy, we focus
on learning an interpretable decision tree, _i.e._ , learning a map from class
hierarchy to decision trees instead of learning class representations.

###  2.3 Interpretable Decision Models

Decision model interpretability is a very popular research topic in machine
learning, which focuses on clearing model decision process to ensure decision
reliability [7]. In earlier studies [33, 20], the decision tree is a very
popular model on a wide variety of tasks, as its attractive interpretability.
After that, considering the high accuracy of neural networks, some works [41,
18, 51, 19, 49] further combine neural network and decision tree to improve
accuracy and interpretability jointly. Recently, several FSL studies also
attempt to improve FSL interpretability [6, 48, 43]. For example, in [48], Xue
_et al._ propose a region comparison network to highlight the region
similarity between samples for the interpretable FSL. Wang _et al._ [43]
propose an attention-based explainable FSL model by highlighting the
discriminative patterns of each image. Almost all methods improve FSL
interpretability from the perspective of heatmap visualization, which does not
directly reveal why FSL models make such decision. Different from these
existing methods, we focus on the interpretability of FSL decision process and
propose a novel decision tree-based meta-learning framework (_i.e._ , MetaDT),
which directly reveals the reason for the classification decision of each
sample.

##  3 Problem Definition

For the FSL, we are given two data sets: 1) a training set $\mathcal{S}$
(called support set) that contains $N$ novel classes $\mathcal{C}_{novel}$,
with $K$ labeled samples per class where $K$ is very small (_e.g._ , $K=1$ or
5); and 2) an auxiliary data set $\mathcal{D}_{base}$ that consists of
abundant labeled images from base classes $\mathcal{C}_{base}$. Here, the sets
of class $\mathcal{C}_{base}$ and $\mathcal{C}_{novel}$ are disjoint, _i.e._ ,
$\mathcal{C}_{base}\cap\mathcal{C}_{novel}=\emptyset$. Based on the support
set $\mathcal{S}$ and the auxiliary data set $\mathcal{D}_{base}$, our goal is
to learn a good classifier for a test set $\mathcal{Q}$ (called query set)
that consists of unlabeled novel class samples. This is a common setting in
FSL studies, which is called $N$-way $K$-shot tasks (_e.g._ , 5-way 1-shot
tasks, or 5-way 5-shot tasks).

In this paper, we focus on the interpretable FSL. Our goal is to learn a
decision-interpretable classifier for query set $\mathcal{Q}$, _i.e._ , the
learned classifier not only can perform novel class prediction but also can
provide a clear explanation for making the class prediction. To this end, in
this paper, we introduce a tree-like class hierarchy (i.e., the semantic
hierarchy relations between classes) from external knowledge graphs (e.g.,
WordNet [26]) as FSL priors, and then carefully design a meta-learner
leveraging the priors to effectively infer an interpretable decision tree for
recognizing novel classes.

##  4 MetaDT Framework

In this section, we introduce the technical details of the proposed MetaDT
framework. As shown in Figure 2, the framework consists of three key modules:
a CNN-based feature extractor $f_{\theta_{f}}()$ with parameters $\theta_{f}$,
a decision tree inference network (DTINet) $f_{\theta_{g}}()$ with parameters
$\theta_{g}$, and an interpretable and differentiable decision tree (IDDTree)
$f_{\theta_{d}}()$ with parameters $\theta_{d}$. Here, the feature extractor
$f_{\theta_{f}}()$ aims to provide a good $d_{f}$-dimensional representation
for each image, which is obtained by following the representation learning
method [30]. The DTINet $f_{\theta_{g}}()$ is a meta-learner, which accounts
for learning to quickly infer a task-specific decision tree $f_{\theta_{d}}()$
(_i.e._ , IDDTree) by using the priors of class hierarchy and few labeled
samples. The IDDTree $f_{\theta_{d}}()$ is a classifier where its parameters
$\theta_{d}$ is not trainable but infered by the DTINet $f_{\theta_{g}}()$,
which aims to perform class prediction for each support/query sample.

Figure 3: The intuition for fast adaptation of decision tree.

###  4.1 Overall Workflow

The main details of the IDDTree $f_{\theta_{d}}()$ and the DTINet
$f_{\theta_{g}}()$ will be introduced in Sections 4.2 and 4.3, respectively.
Here, we mainly present the workflow depicted in Figure 2, including meta-
training and meta-test phases.

Meta-Training. The key challenge of our MetaDT is how to train our meta-
learner (_i.e._ , DTINet $f_{\theta_{g}}()$) by leveraging the priors of class
hierarchy to quickly infer an interpretable decision tree (_i.e._ , IDDTree
$f_{\theta_{d}}()$) with very few labeled samples. To address the challenge,
we employ a two-loop optimization mechanism [13] (_i.e._ , an inner-loop
optimization and an outer-loop optimization) to learn the DTINet
$f_{\theta_{g}}()$. As shown in Figure 3, our intuition is treating the
parameters $\theta_{g}$ of our meta-learner $f_{\theta_{g}}()$ as the initial
parameters of inner-loop optimization, and then attempting to learn good
(_i.e._ , transferable) initial parameters $\theta_{g}$ by using an outer-loop
optimization. By doing so, the initial model $f_{\theta_{g}}()$ can quickly
adapt to novel classes with gradient updates upon few labeled samples
$(x,y)\in\mathcal{S}$.

Specifically, as shown in Figure 2, following the episodic training manner
[40], we mimic the test setting and construct a number of $N$-way $K$-shot
tasks (called episodes) from base classes. Then, we train our meta-learner
$f_{\theta_{g}}()$ to quickly infer a task-specific IDDTree for each episode
by minimizing the cross entropy loss on query set $\mathcal{Q}$. That is,

$\displaystyle\min\limits_{\theta_{g}}\
\mathbb{E}_{(\mathcal{S},\mathcal{Q})\in\mathbb{T}}L_{\theta_{g}}(\mathcal{Q}),$
(1) $\displaystyle L_{\theta_{g}}(\mathcal{Q})=\frac{1}{|\mathcal{Q}|}$
$\displaystyle\sum_{(x,y)\in\mathcal{Q}}CE(P(k|x,\mathcal{S},\theta_{g}),y),$

where $\mathbb{T}$ is the set of constructed episodes and $CE()$ denotes the
cross entropy loss function. The above training process can be regarded as an
outer-loop optimization for learning good initial parameters $\theta_{g}$.
Next, we introduce how to calculate the class probability
$P(k|x,\mathcal{S},\theta_{g})$ that each query sample $x\in\mathcal{Q}$
belongs to class $k$ in an inner-loop optimization manner, given initial
parameters $\theta_{g}$ and support set $\mathcal{S}$. It includes the
following three stages:

_Stage 1. Class Hierarchy Construction._ We first construct a tree-like class
hierarchy as FSL priors. The class hierarchy reflects the semantic relations
between classes, _i.e._ , what kinds of superclasses these few-shot classes
share. For example, the classes “pomeranian” and “bichon frise” share the same
superclass “dog”. We note that such type of class hierarchy can be easily
obtained from some external knowledge graphs, _e.g._ , WordNet [26]. Then, we
encode the tree-like class hierarchy by using an undirected graph
$\mathcal{G}$, where its nodes denote all semantic classes (_i.e._ , all few-
shot classes and their superclasses) and its edges represent the hierarchy
relations between all classes. Let $\mathcal{V}$ and $\mathcal{E}$ denote the
set of nodes and edges respectively, _i.e._ ,
$\mathcal{G}=(\mathcal{V},\mathcal{E})$. In the graph $\mathcal{G}$, each node
is represented by a $d_{s}$-dimensional semantic vector $h_{i}$, _i.e._ , the
mean of all $d_{s}$-dimensional Glove-based word embeddings [28] of their
class names, denoted by $\mathcal{H}=\\{h_{i}\\}_{i=0}^{F-1}$ where $F$ is the
number of graph nodes. The edge set $\mathcal{E}$ is represented as a
adjacency matrix $A\in\mathcal{R}^{F\times F}$ where $A_{i,j}=1$ if the node
$i$ is associated with the node $j$, otherwise $A_{i,j}=0$.

_Stage 2. Fast Adaptation._ In this stage, we regard the class hierarchy
$\mathcal{G}$ as inputs and the IDDTree $f_{\theta_{d}}()$ as predicted
targets of our initial DTINet $f_{\theta_{g}}()$. Then, we evaluate the
predicted IDDTree $f_{\theta_{d}}()$ on the support set $\mathcal{S}$ and
leverage the evaluated classification loss $L_{\theta_{g}}(\mathcal{S})$ to
quickly finetune the initial DTINet $f_{\theta_{g}}()$ with $M$-step gradient
updates. As a result, a task-specific DTINet $f_{\theta^{\prime}_{g}}()$ with
parameters $\theta^{\prime}_{g}$ can be obtained for each few-shot task. For
example, when using one gradient updates (_i.e._ , $M$=1), the process of such
fast adaptation can be expressed as:

$\displaystyle\theta_{g}^{{}^{\prime}}$
$\displaystyle=\theta_{g}-\alpha_{inner}\triangledown
L_{\theta_{g}}(\mathcal{S}),$ (2)

where $\alpha_{inner}$ is the learning rate of gradient updates. The above
process can be regarded as an inner-loop optimization for learning good task-
specific parameters $\theta^{\prime}_{g}$. Next, we elaborate on the
calculation details for the classification loss $L_{\theta_{g}}(\mathcal{S})$
on the support set $\mathcal{S}$.

Speficifally, we first extract the features of each support sample
$(x,y)\in\mathcal{S}$ via the feature extractor $f_{\theta_{f}}()$. Then, the
class probability $P(k|x,\theta_{g})$ that each sample $(x,y)\in\mathcal{S}$
belongs to class $k$ can be calculated by using the IDDTree $f_{\theta_{d}}()$
infered by our meta-learner $f_{\theta_{g}}()$ (Please refers to Section 4.2
for its calculation details). That is,

$\displaystyle\theta_{d}=f_{\theta_{g}}(\mathcal{G}),$ (3) $\displaystyle
P(k|x,\theta_{g})$ $\displaystyle=f_{\theta_{d}}(f_{\theta_{f}}(x)),\
(x,y)\in\mathcal{S}.$

Finally, we calculate the classification loss
$L_{\theta_{g}}(f_{\theta_{g}},\mathcal{S})$ by using a cross entropy loss on
support set $\mathcal{S}$:

$\displaystyle
L_{\theta_{g}}(\mathcal{S})=\frac{1}{|\mathcal{S}|}\sum_{(x,y)\in\mathcal{S}}CE(P(k|x,$
$\displaystyle\theta_{g}),y).$ (4)

_Stage 3. Class Probability Prediction._ Based on the above task-specific
DTINet $f_{\theta_{g}^{{}^{\prime}}}()$, we can obtain a task-specific IDDTree
$f_{\theta^{\prime}_{d}}()$. Then, the class probability
$P(k|x,\mathcal{S},\theta_{g})$ that each query sample $x\in\mathcal{Q}$
belongs to class $k$ can be calculated in a sequential decision manner (Please
refers to Section 4.2 for its details). That is,

$\displaystyle\theta^{\prime}_{d}=f_{\theta_{g}^{{}^{\prime}}}(\mathcal{G}),$
(5) $\displaystyle P(k|x,$
$\displaystyle\mathcal{S},\theta_{g})=f_{\theta^{\prime}_{d}}(x),x\in\mathcal{Q}.$

Meta-Test. The workflow of meta-test is similar to meta-training. As shown in
Figure 2, the difference is that we remove the training step of Eq. 1 and
directly evaluate the class probability $P(k|x,\mathcal{S},\theta_{g})$ of
each query sample $x\in\mathcal{Q}$ by following Eqs. 2 $\sim$ 5. Finally, we
perform the novel class prediction $\hat{y}$ to the novel class with highest
probability:

$\hat{y}=\mathop{\arg\max}_{k\in[0,N-1]}P(k|x,\mathcal{S},\theta_{g}).$ (6)

###  4.2 Interpretable and Differentiable Decision Tree

In the subsection, we introduce the technical details of our IDDTree
$f_{\theta_{d}}()$, including the following two parts.

How to design the IDDTree $f_{\theta_{d}}()$? As shown in Figure 4, for
interpretability, our main idea is treating the hierarchy relations in the
tree-like class hierarchy $\mathcal{G}$ as explicit decision rules, and then
performing class predictions by following the rules. For differentiability,
our notion is assigning each tree node $i$ of the class hierarchy
$\mathcal{G}$ with a $d_{f}$-dimensional weight vector
$\theta^{i}_{d}\in\mathcal{R}^{d_{f}}$ (_i.e._ ,
$\theta_{d}=\\{\theta^{i}_{d}\\}_{i=0}^{F-1}$ where $F$ is the number of tree
nodes), and then performing the class prediction in a sequential and soft
probability inference manner from the root of decision rules to its leafs.

How to evaluate the class probability? Intuitively, the weight vector
$\theta^{i}_{d}$ can be regarded as the prototype or class center of each
class $i\in[0,F-1]$ in the feature space. Thus, given a support/query image
$x\in\mathcal{S}/\mathcal{Q}$ and its features $f_{\theta_{f}}(x)$, we can
evaluate the class probability $P(k|x,\theta_{d})$ that the sample $x$ belongs
to few-shot class $k\in[0,N-1]$ in a metric-based decision manner from the
root of decision tree to its leaf nodes. It includes the following two steps:

_Step 1._ For each non-leaf node $i$ (_i.e._ , superclasses), we need to make
decisions for its all child nodes denoted by $\mathcal{C}_{child}^{i}$, _i.e._
, evaluating the conditional probability $P(j|i,x,\theta_{d})$ that the sample
$x$ belongs to child class $j\in\mathcal{C}_{child}^{i}$. We evaluate the
conditional probability $P(j|i,x,\theta_{d})$ by computing the cosine
similarity between the sample $x$ and the weights $\theta^{j}_{d}$ of child
class $j\in\mathcal{C}_{child}^{i}$. That is,

$P(j|i,x,\theta_{d})=\frac{e^{<f_{\theta_{f}}(x),\ \theta^{j}_{d}>\ \cdot\
\gamma}}{\sum_{c\in\mathcal{C}_{child}^{i}}e^{<f_{\theta_{f}}(x),\
\theta^{c}_{d}>\ \cdot\ \gamma}},$ (7)

where $<\cdot>$ denotes the cosine similarity of two vectors and $\gamma$ is a
scale parameter. Following [8], $\gamma=10$ is used.

_Step 2._ For each leaf node (_i.e._ , few-shot class $k$), given its
traversal path $\tau_{k}$ from the root and the conditional probability
$P(j|i,x,\theta_{d})$ of each node $i\in\tau_{k}$ traversing its child node
$j$ in the path $\tau_{k}$, _i.e._ ,
$j\in\tau_{k}\cap\mathcal{C}_{child}^{i}$, the class probability
$P(k|x,\theta_{d})$ that each support/query sample $x$ belongs to the class
$k$ can be calculated in a cumulative product manner along the traversal path
$\tau_{k}$. That is,

$P(k|x,\theta_{d})=\prod\limits_{i\in\tau_{k}}P(j|i,x,\theta_{d}),j\in\tau_{k}\cap\mathcal{C}_{child}^{i}.$
(8)

For clarity, we give an example to show the above calculation process. As
shown in Figure 4, given a sample $x$, 1) we compute all conditional
probability of non-leaf nodes in Eq. 7, _e.g._ , $P(5|6,x,\theta_{d})=0.7$;
and then 2) the class probability is obtained by using Eq. 8, _e.g._ ,
$P(3|x,\theta_{d})=P(6|x,\theta_{d})P(5|6,x,\theta_{d})P(3|5,x,\theta_{d})=1\cdot
0.7\cdot 0.6=0.42$.

Figure 4: Details of DTINet and IDDTree. $\mathcal{H}$ and $A$ is the sementic
vectors and adjacency matrix of class hierarchy, respectively. $\theta_{d}$ is
the outputs where $\theta^{i}_{d}$ is correspondings to $h_{i}$,
$i=0,1,...,6$.

###  4.3 Decision Tree Inference Network

In the IDDTree $f_{\theta_{d}}()$, its key challenge is how to quickly infer
its parameters $\theta_{d}=\\{\theta^{i}_{d}\\}_{i=0}^{F-1}$ when only few
labeled samples are available. To address the challenge, we regard the class
hierarchy $\mathcal{G}$ as inputs and then design a graph convolution-based
inference network $f_{\theta_{g}}()$ to learn the map from class hierarchy
$\mathcal{G}$ to the parameters $\theta_{d}$. The intuition behind this design
is fully leveraging human’s prior knowledge to enable fast adaptation of
decision trees.

The overall structure is shown in Figure 4, which consists of an input layer,
a hidden layer, and an output layer. The input and hidden layers aim to obtain
a good representation for each graph node by exploiting abundant semantic
information and hierarchy relations between classes. Then, the parameters
$\theta_{d}$ of IDDTree $f_{\theta_{d}}()$ are predicted by the final output
layer. The overall process can be defined as:

$\displaystyle\theta_{d}=f_{\theta_{g}}(\mathcal{H},A)=\hat{A}\delta(\hat{A}\delta(\hat{A}\mathcal{H}W^{(0)})W^{(1)})W^{(2)}$
(9)

where $W^{(0)}$, $W^{(1)}$, and $W^{(2)}$ denote a trainable weight matrix for
input, hidden and output layers, respectively, _i.e._ ,
$\theta_{g}=\\{W^{(0)},W^{(1)},W^{(2)}\\}$; $\hat{A}$ is the normalized $A$,
_i.e._ , $\hat{A}=D^{-\frac{1}{2}}AD^{\frac{1}{2}}$; $D$ is the degree matrix,
_i.e._ , $D_{ii}=\sum_{j}A_{ij}$; and $\delta()$ denotes the ReLU activation
function.

Table 1: Comparison with state-of-the-art methods on miniImagenet and CIFAR-
FS. The best results are highlighted in bold.

Method | Using Knowledge | Backbone | miniImagenet | CIFAR-FS
---|---|---|---|---
5-way 1-shot | 5-way 5-shot | 5-way 1-shot | 5-way 5-shot
CGCS [15] | No | ResNet12 | $67.02\pm 0.20\%$ | $82.32\pm 0.14\%$ | $71.66\pm 0.23\%$ | $85.50\pm 0.15\%$
RCNet [48] | No | ResNet12 | $57.40\pm 0.86\%$ | $75.19\pm 0.64\%$ | $69.02\pm 0.92\%$ | $82.96\pm 0.67\%$
ALFA [3] | No | ResNet12 | $59.74\pm 0.49\%$ | $77.96\pm 0.41\%$ | - | -
MeTAL [2] | No | ResNet12 | $59.64\pm 0.38\%$ | $76.20\pm 0.19\%$ | $67.97\pm 0.47\%$ | $82.17\pm 0.38\%$
CRF-GNN [37] | No | ConvNet256 | 57.89 $\pm$ 0.87$\%$ | 73.58 $\pm$ 0.87$\%$ | 76.45 $\pm$ 0.99$\%$ | 88.42 $\pm$ 0.23$\%$
Neg-Cosine [24] | No | ResNet12 | $63.85\pm 0.81\%$ | 81.57 $\pm$ 0.56% | - | -
RFS [38] | No | ResNet12 | $62.02\pm 0.63\%$ | 79.64 $\pm$ 0.44% | $71.51\pm 0.80\%$ | $86.00\pm 0.50\%$
InvEq [30] | No | ResNet12 | $66.82\pm 0.80\%$ | $84.35\pm 0.51\%$ | $76.83\pm 0.82\%$ | $89.26\pm 0.58\%$
TriNet [10] | Yes | ResNet18 | $58.12\pm 1.37\%$ | $76.92\pm 0.69\%$ | - | -
AM3-PNet [46] | Yes | ResNet12 | $65.21\pm 0.30\%$ | $75.20\pm 0.27\%$ | - | -
AM3-TRAML [22] | Yes | ResNet12 | 67.10 $\pm$ 0.52$\%$ | 79.54 $\pm$ 0.60% | - | -
FSLKT [27] | Yes | ConvNet128 | $64.42\pm 0.72\%$ | $74.16\pm 0.56\%$ | - | -
MetaDT | Yes | ResNet12 | 69.08 $\pm$ 0.73$\%$ | 83.40 $\pm$ 0.51$\%$ | 79.03 $\pm$ 0.75$\%$ | 88.50 $\pm$ 0.58$\%$
MetaDT + Cosine Classifier | Yes | ResNet12 |  70.45 $\pm$ 0.81$\%$ |  84.84 $\pm$ 0.52$\%$ |  80.14 $\pm$ 0.78$\%$ |  89.84 $\pm$ 0.56$\%$

##  5 Enhancement: MetaDT + Cosine Classifier

Till now, we have introduced all details of our MetaDT. Its advantage is that
the decision process of prediction is interpretable resorting to the priors of
class hierarchy. In this section, we introduce how to fuse our MetaDT and the
existing cosine classifier [9] for better performance.

Intuitively, the two methods are complementary to each other: 1) when the
labeled samples are very scarce (_e.g._ , $K$=1), our MetaDT is more reliable
because it fully exploits the priors of class hierarchy which effectively
alleviates the data sparsity issue; and 2) as more and more labeled samples
become available, the data sparsity issue gradually disappears and the
existing cosine classifier becomes more effective. Thus, we propose to fuse
the two methods in class probability via a convex combination manner. That is,

$\displaystyle P_{fused}(k|x)=\lambda
P(k|x,\mathcal{S},\theta_{g})+(1-\lambda)P_{cos}(k|x,\mathcal{S}),$ (10)

where $P(k|x,\mathcal{S},\theta_{g})$, $P_{cos}(k|x,\mathcal{S})$, and
$P_{fused}(k|x)$ denote the class probability of our MetaDT, the cosine
classifier, and after fusion, respectively; $\lambda\in[0,1]$ is a weight
parameter. Finally, we perform the class prediction of query sample
$x\in\mathcal{Q}$ by using fused probability $P_{fused}(k|x)$. The fusion
strategy is only used in meta-test. In real applications, the “MetaDT + Cosine
Classifier” can be used when we merely go for better performance; otherwise
our MetaDT would be a good choice for FSL, which not only delivers promising
performance but also has clear interpretability.

##  6 Experiments

###  6.1 Datasets and Settings

miniImagenet. The dataset is a subset from ImageNet, containing 100 classes.
Following [40], we split it into 64, 16, and 20 classes for training,
validation, and test, respectively. Besides, the tree-like class hierarchy is
constructed from WordNet by using the relation of “hypernyms()”.

CIFAR-FS. The dataset is built from CIFAR100, including 100 classes. Following
[5], we split the data set into 64 classes for training, 16 classes for
validation, and 20 classes for test, respectively. Similarly, its tree-like
class hierarchy is also acquired from WordNet via its class names.

tieredImagenet. The dataset is a more challenging FSL dataset constructed from
ImageNet, including 608 classes. Following [29], we partition the dataset into
34 high-level classes, and then split it into 20 classes for training, 6
classes for validation, and 8 classes for test. Similarly, its class hierarchy
is also extracted from WordNet.

###  6.2 Implementation Details

Architecture. We employ the ResNet12 trained following previous work [30] as
our feature extractor. For DTINet, we use a three-layer graph convolution with
a 300-dimensional input, a 1024-dimensional input layer, a 2048-dimensional
hidden layer, and a 640-dimensional output layer to infer the IDDTree. In each
layer, ReLU is used as the activation function except for the output layer,
and a dropout layer with probability of 0.5 is used for better generalization.

Training and Test Details. In meta-training phase, we train our meta-learner
with 20 epochs by using an Adam optimizer with a weight decay of 0.0005 and a
learning rate of 0.0001. The update steps $M$ and learning rate
$\alpha_{inner}$ of inner-loop optimization is set to 25 and 0.05,
respectively. In meta-test phase, the learning rate $\alpha_{inner}$ remains
unchanged, and the update step $M$ is changed to 50, 70, and 150 on
miniImagenet, CIFAR-FS, and tieredImagenet datasets, respectively. For the
weight parameter $\lambda$, we set it to 0.8 and 0.1 for 1-shot and 5-shot
tasks, respectively.

Evaluation. We evaluate our methods on 600 5-way 1-shot/5-shot tasks randomly
sampled from the test set. Lastly, the mean accuracy together with the 95%
confidence interval is reported as the evaluation results.

###  6.3 Experimental Results

Table 2: Comparison with state-of-the-art methods on tieredImagenet. Here,
“CC” denotes “Cosine Classifier”.

Method | Backbobe | tieredImagenet
---|---|---
5-way 1-shot | 5-way 5-shot
CGCS [15] | ResNet12 | $71.66\pm 0.23\%$ | $85.50\pm 0.15\%$
ALFA [3] | ResNet12 | $64.62\pm 0.49\%$ | $82.48\pm 0.38\%$
MeTAL [2] | ResNet12 | $63.89\pm 0.43\%$ | $80.14\pm 0.40\%$
CRF-GNN [37] | ConvNet256 | 58.45 $\pm$ 0.59$\%$ | 74.58 $\pm$ 0.84$\%$
RFS [38] | ResNet12 | $69.74\pm 0.72\%$ | 84.41 $\pm$ 0.55%
InvEq [30] | ResNet12 | $71.87\pm 0.89\%$ | $86.82\pm 0.58\%$
AM3-PNet [46] | ResNet12 | $67.23\pm 0.34\%$ | $78.95\pm 0.22\%$
MetaDT | ResNet12 | 70.56 $\pm$ 0.90$\%$ | 85.17 $\pm$ 0.56$\%$
MetaDT + CC | ResNet12 |  72.69 $\pm$ 0.90$\%$ |  87.10 $\pm$ 0.56$\%$

Table 3: Percentage of seen and unseen superclasses.

Types | miniImagenet | CIFAR-FS | tieredImagenet
---|---|---|---
Seen Superclasses | 97.30% | 85% | 49.60%
Unseen Superclasses | 2.70% | 15.00% | 50.40%

Results on miniImagenet and CIFAR-FS. Table 1 shows the experimental results
of various methods on miniImagenet and CIFAR-FS. From these results, we
observe that our “MetaDT + Cosine Classifier” exceeds these state-of-the-art
methods by around 1% $\sim$ 4%, which shows the superiority of our method on
FSL. In addition, it can be found that our MetaDT achieves superior
performance over existing methods by around 2% $\sim$ 3% on 1-shot tasks, as
well as comparable performance on 5-shot tasks. Note that our MetaDT is
decision-interpretable. Specifically, compared with these methods without
exploiting knowledge, our MetaDT additionally explores the priors of class
hierarchy, and mainly focuses on leveraging these priors to quickly infer a
decision tree. The results validate the effectiveness of our decision tree
classifier. It is worth noting that RCNet [48] is also developed for
interpretable FSL, which is a strong competitor. Our MetaDT significantly
outperforms RCNet by a large margin by 6% $\sim$ 12%. This further shows the
superiority of our MetaDT for interpretable FSL.

As for the FSL methods using knowledge, they also explore the semantic
knowledge as priors for improving FSL but mainly use the nearest neighbor
classifier. Different from these methods, our MetaDT leverages the priors of
class hierarchy to quickly infer a task-specific decision tree for
interpretable FSL. The results demonstrate the effectiveness of our MetaDT.
Note that our MetaDT exceeds FSLKT around 6% $\sim$ 10%, which also utilizes
the priors of class hierarchy. Different from FSLKT, our MetaDT regards the
class hierarchy as a decision tree, so that more superclasses can be exploited
for novel class prediction. The results show the superiority of our MetaDT to
leverage class hierarchy.

Results on tieredImagenet. In Table 2, we report the classification results of
our methods and various baseline methods on tieredImagenet. We find that our
“MetaDT + Cosine Classifier” achieves superior performance over state-of-the-
art methods. In addition, our MetaDT achieves competitive performance with
some state-of-the-art methods. It is worth noting that the performance
improvement of our MetaDT on tieredImagenet is limited, which is inconsistent
with the one of miniImagenet and CIFAR-FS. This may be reasonable because the
tieredImagenet dataset is split by following the high-level semantic classes.
As a result, some superclasses are unseen in our meta-training phase, which
impedes the knowledge transfer of our MetaDT.

To illustrate this point, we calculate the percentage of seen and unseen
superclasses on miniImagenet, CIFAR-FS, and tieredImagenet, in Table 3. From
the results, we indeed find that the percentage of unseen superclasses of
tieredImagenet is much larger than the one of miniImagenet and CIFAR-FS. This
means the meta knowledge learned from the base classes is difficult to
transfer to the novel classes. For the issue, we will leave it to future work.

###  6.4 Ablation Study

We conduct various experiments on miniImagenet to investigate the impacts of
different components.

Table 4: Ablation study of different components for our MetaDT.

| Settings | 5-way 1-shot | 5-way 5-shot
---|---|---|---
(i) | MetaDT | $69.08\pm 0.73\%$ | $83.40\pm 0.51\%$
(ii) | w/o Class Semantic | $67.23\pm 0.83\%$ | $81.21\pm 0.54\%$
(iii) | w/o Graph Convolution | $66.79\pm 0.73\%$ | $82.18\pm 0.52\%$
(iv) | w/o DTINet | $67.14\pm 0.73\%$ | $81.12\pm 0.54\%$
(v) | w/o Fast Adaptation | $57.04\pm 0.86\%$ | $57.07\pm 0.83\%$

Table 5: Ablation study of the proposed fusion strategy.

| Settings | 5-way 1-shot | 5-way 5-shot
---|---|---|---
(i) | MetaDT | $69.08\pm 0.73\%$ | $83.40\pm 0.51\%$
Cosine Classifier | $66.96\pm 0.81\%$ | $84.53\pm 0.52\%$
(ii) | MetaDT + Cosine Classifier | $70.45\pm 0.81\%$ | $84.84\pm 0.52\%$

Figure 5: Analysis of 5-way $K$-shot tasks with different $K$ values.

How do different components affect our MetaDT? We conduct various ablation
studies to analyze the contribution of different components, including class
semantic, class hierarchy, graph convolution, DTINet, and fast adaptation.
Specifically, (i) we evaluate our MetaDT on 5-way 1/5-shot tasks; (ii) we
remove the semantic vectors $\mathcal{H}$ and replace it by using one-hot
vectors on (i); (iii) we replace the three-layer graph convolution by using a
three-layer fully-connected network on (i); (iv) we remove the DTINet on (i)
and estimate the parameters of IDDTree in an average manner on mean-based
prototypes by following the class hierarchy relations; and (v) we remove the
fast adaptation of our meta-learner on (i). These results are shown in Table
4. From these results, we can see that the classification performance
decreases by around 2% $\sim$ 16% when removing these components respectively.
This suggests that employing these four key components is beneficial for our
MetaDT.

Is the fusion of our MetaDT and cosine classifier effective? In Table 5, we
analyze the effectiveness of the fusion strategy described in Section 5.
Specifically, (i) we evaluate our MetaDT and cosine classifier, respectively;
(ii) following Eq. 10, we fuse our MetaDT with cosine classifier. From the
results, we find that the performance of our “MetaDT + Cosine Classifier”
exceeds our MetaDT and the cosine classifier by around 1% $\sim$ 4%. This
shows the superiority of the proposed fusion strategy on improving FSL
performance.

(a) A case of right decision.

(b) A case of wrong decision.

Figure 6: Decision interpretability analysis of our MetaDT.

How does our method perform under different numbers of support samples? In
Figure 5, we count the performance of our MetaDT, cosine classifier, and our
“MetaDT + Cosine Classifier” on different numbers of support samples. We find
that 1) our MetaDT is more reliable on 1/2-shot tasks while the cosine
classifier is more effective on 3/4/5-shot tasks; 2) our “MetaDT + Cosine
Classifier” performs best. This verifies that our MetaDT and the cosine
classifier indeed remedy the shortcomings of each other.

###  6.5 Interpretability and Visualization Analysis

Is the decision of our MetaDT interpretable? To show the decision
interpretability of our MetaDT, we conduct two 5-way 1-shot case studies on
miniImagenet, including a right and a wrong decision case. Specifically, we
randomly select a 5-way 1-shot task from test set. Then we construct and learn
a four-layer decision tree by using only one labeled sample. After that, we
randomly select an image that is rightly predicted and an image that is
wrongly predicted as an example to illustrate the interpretable decision
process, which are shown in Figure 6. From Figure 6(a), we can see that our
decision tree makes right decision in entire sequential decision path for the
image of “african hunting dog”, where all decisions have real meaning and fit
our intuition. Besides, we note that such interpretable decison process can
also help us trace and locate the root reason of making such wrong decisions.
As shown in Figure 6(b), the reason causing the image of “malamute” is
misidentified is that the image is misclassified as “african hunting dog” when
performing the decision of “canine”.

(a) The root nodes.

(b) The intermediate nodes.

(c) The leaf nodes.

(d) Structure of decision trees.

Figure 7: Visualization of the weights of decison trees.

How does our MetaDT work? In Figures 7(a), 7(b), and 7(c), we visualize the
weights of decision tree on a 5-way 1-shot task of miniImagenet. Here, the
weights of decision tree before and after fast adaptation are marked by the
squares and stars, respectively. Besides, the structure of decison tree is
shown in Figure 7(d). We find that the final weights (_i.e._ , final
prototypes) marked by stars are much closer to the centers of few-shot
classes/superclass after fast adaptation. This visualization indicates that
our MetaDT effectively learns the class hierarchy and infers the task-specific
decision tree.

##  7 Conclusion

In this paper, we propose a novel decision tree-based meta-learning framework
for interpretable few-shot learning, called MetaDT. In particular, we replace
the black-box FSL classifier with an interpretable decision tree. In addition,
a novel decision tree inference network and a two-loop optimization mechanism
is designed, respectively, for a fast adaptation of decision tree. Extensive
experiments and interpretability analyses show our MetaDT is effective and
provides interpretability for novel class predictions.

## References

  * [1] Han Altae-Tran, Bharath Ramsundar, Aneesh S Pappu, and Vijay Pande.  Low data drug discovery with one-shot learning.  ACS central science, 3(4):283–293, 2017.
  * [2] Sungyong Baik, Janghoon Choi, Heewon Kim, Dohee Cho, Jaesik Min, and Kyoung Mu Lee.  Meta-learning with task-adaptive loss function for few-shot learning.  In ICCV, pages 9465–9474, 2021.
  * [3] Sungyong Baik, Myungsub Choi, Janghoon Choi, Heewon Kim, and Kyoung Mu Lee.  Meta-learning with adaptive hyperparameters.  In NeurIPS, 2020.
  * [4] Antonio Jesús Banegas-Luna, Jorge Peña-García, Adrian Iftene, Fiorella Guadagni, Patrizia Ferroni, Noemi Scarpato, Fabio Massimo Zanzotto, Andrés Bueno-Crespo, and Horacio Pérez-Sánchez.  Towards the interpretability of machine learning predictions for medical applications targeting personalised therapies: A cancer case survey.  International Journal of Molecular Sciences, 22(9):4394, 2021.
  * [5] Luca Bertinetto, João F. Henriques, Philip H. S. Torr, and Andrea Vedaldi.  Meta-learning with differentiable closed-form solvers.  In ICLR, 2019.
  * [6] Kaidi Cao, Maria Brbic, and Jure Leskovec.  Concept learners for few-shot learning.  In ICLR, 2021.
  * [7] Diogo V Carvalho, Eduardo M Pereira, and Jaime S Cardoso.  Machine learning interpretability: A survey on methods and metrics.  Electronics, 8(8):832, 2019.
  * [8] Wei-Yu Chen, Yen-Cheng Liu, Zsolt Kira, Yu-Chiang Frank Wang, and Jia-Bin Huang.  A closer look at few-shot classification.  In ICLR, 2019.
  * [9] Yinbo Chen, Xiaolong Wang, Zhuang Liu, Huijuan Xu, Trevor Darrell, et al.  A new meta-baseline for few-shot learning.  In ICML, 2020.
  * [10] Zitian Chen, Yanwei Fu, Yinda Zhang, Yu-Gang Jiang, Xiangyang Xue, and Leonid Sigal.  Multi-level semantic feature augmentation for one-shot learning.  IEEE Trans. Image Process., 28(9):4594–4605, 2019.
  * [11] Zhengyu Chen, Jixie Ge, Heshen Zhan, Siteng Huang, and Donglin Wang.  Pareto self-supervised training for few-shot learning.  In CVPR, pages 13663–13672, 2021.
  * [12] Mohamed Elhoseiny and Mohamed Elfeki.  Creativity inspired zero-shot learning.  In ICCV, pages 5784–5793, 2019.
  * [13] Chelsea Finn, Pieter Abbeel, Sergey Levine, et al.  Model-agnostic meta-learning for fast adaptation of deep networks.  In ICML, pages 1126–1135, 2017.
  * [14] Andrea Frome, Gregory S. Corrado, Jonathon Shlens, Samy Bengio, Jeffrey Dean, Marc’Aurelio Ranzato, and Tomas Mikolov.  Devise: A deep visual-semantic embedding model.  In NeurIPS, pages 2121–2129, 2013.
  * [15] Zhi Gao, Yuwei Wu, Yunde Jia, and Mehrtash Harandi.  Curvature generation in curved spaces for few-shot learning.  In ICCV, pages 8691–8700, 2021.
  * [16] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun.  Deep residual learning for image recognition.  In CVPR, pages 770–778, 2016.
  * [17] Michael Kampffmeyer, Yinbo Chen, Xiaodan Liang, Hao Wang, Yujia Zhang, and Eric P Xing.  Rethinking knowledge graph propagation for zero-shot learning.  In CVPR, pages 11487–11496, 2019.
  * [18] Peter Kontschieder, Madalina Fiterau, Antonio Criminisi, and Samuel Rota Bulo.  Deep neural decision forests.  In ICCV, pages 1467–1475, 2015.
  * [19] Peter Kontschieder, Madalina Fiterau, Antonio Criminisi, and Samuel Rota Bulò.  Deep neural decision forests.  In IJCAI, pages 4190–4194, 2016.
  * [20] D Lavanya and K Usha Rani.  Ensemble decision tree classifier for breast cancer data.  International Journal of Information Technology Convergence and Services, 2(1):17–24, 2012.
  * [21] Kwonjoon Lee, Subhransu Maji, Avinash Ravichandran, and Stefano Soatto.  Meta-learning with differentiable convex optimization.  In CVPR, pages 10657–10665, 2019.
  * [22] Aoxue Li, Weiran Huang, Xu Lan, Jiashi Feng, Zhenguo Li, and Liwei Wang.  Boosting few-shot learning with adaptive margin loss.  In CVPR, pages 12576–12584, 2020.
  * [23] Wenbin Li, Jinglin Xu, Jing Huo, Lei Wang, Yang Gao, and Jiebo Luo.  Distribution consistency based covariance metric networks for few-shot learning.  In AAAI, pages 8642–8649, 2019.
  * [24] Bin Liu, Yue Cao, Yutong Lin, Qi Li, Zheng Zhang, Mingsheng Long, and Han Hu.  Negative margin matters: Understanding margin in few-shot classification.  In ECCV, pages 438–455, 2020.
  * [25] Puneet Mangla, Mayank Singh, Abhishek Sinha, Nupur Kumari, Vineeth N. Balasubramanian, and Balaji Krishnamurthy.  Charting the right manifold: Manifold mixup for few-shot learning.  In WACV, pages 2207–2216, 2020.
  * [26] George A Miller.  WordNet: An electronic lexical database.  MIT press, 1998.
  * [27] Zhimao Peng, Zechao Li, Junge Zhang, Yan Li, Guo-Jun Qi, and Jinhui Tang.  Few-shot image recognition with knowledge transfer.  In ICCV, pages 441–449, 2019.
  * [28] Jeffrey Pennington, Richard Socher, and Christopher Manning.  Glove: Global vectors for word representation.  In EMNLP, pages 1532–1543, 2014.
  * [29] Mengye Ren, Eleni Triantafillou, Sachin Ravi, Jake Snell, Kevin Swersky, Joshua B. Tenenbaum, Hugo Larochelle, and Richard S. Zemel.  Meta-learning for semi-supervised few-shot classification.  In ICLR, 2019.
  * [30] Mamshad Nayeem Rizve, Salman Khan, Fahad Shahbaz Khan, and Mubarak Shah.  Exploring complementary strengths of invariant and equivariant representations for few-shot learning.  In CVPR, pages 10836–10846, 2021.
  * [31] Pau Rodríguez, Issam Laradji, Alexandre Drouin, and Alexandre Lacoste.  Embedding propagation: Smoother manifold for few-shot classification.  In ECCV, 2020.
  * [32] Andrei A Rusu, Dushyant Rao, Jakub Sygnowski, Oriol Vinyals, Razvan Pascanu, Simon Osindero, and Raia Hadsell.  Meta-learning with latent embedding optimization.  In ICLR, 2018.
  * [33] S Rasoul Safavian and David Landgrebe.  A survey of decision tree classifier methodology.  IEEE transactions on systems, man, and cybernetics, 21(3):660–674, 1991.
  * [34] Victor Garcia Satorras and Joan Bruna Estrach.  Few-shot learning with graph neural networks.  In ICLR, 2018.
  * [35] Eli Schwartz, Leonid Karlinsky, Rogério Schmidt Feris, Raja Giryes, and Alexander M. Bronstein.  Baby steps towards few-shot learning with multiple semantics.  CoRR, abs/1906.01905, 2019.
  * [36] Jake Snell, Kevin Swersky, Richard Zemel, et al.  Prototypical networks for few-shot learning.  In NeurIPS, pages 4077–4087, 2017.
  * [37] Shixiang Tang, Dapeng Chen, Lei Bai, Kaijian Liu, Yixiao Ge, and Wanli Ouyang.  Mutual crf-gnn for few-shot learning.  In CVPR, pages 2329–2339, 2021.
  * [38] Yonglong Tian, Yue Wang, Dilip Krishnan, Joshua B Tenenbaum, and Phillip Isola.  Rethinking few-shot image classification: a good embedding is all you need?  In ECCV, pages 266–282, 2020.
  * [39] Manasi Vartak, Arvind Thiagarajan, Conrado Miranda, Jeshua Bratman, and Hugo Larochelle.  A meta-learning perspective on cold-start recommendations for items.  In NeurIPS, pages 6904–6914, 2017.
  * [40] Oriol Vinyals, Charles Blundell, Timothy Lillicrap, Daan Wierstra, et al.  Matching networks for one shot learning.  In NeurIPS, pages 3630–3638, 2016.
  * [41] Alvin Wan, Lisa Dunlap, Daniel Ho, Jihan Yin, Scott Lee, Henry Jin, Suzanne Petryk, Sarah Adel Bargal, and Joseph E. Gonzalez.  Nbdt: Neural-backed decision trees.  In ICLR, 2021.
  * [42] Ziyu Wan, Dongdong Chen, Yan Li, Xingguang Yan, Junge Zhang, Yizhou Yu, and Jing Liao.  Transductive zero-shot learning with visual structure constraint.  In NeurIPS, pages 9972–9982, 2019.
  * [43] Bowen Wang, Liangzhi Li, Manisha Verma, Yuta Nakashima, Ryo Kawasaki, and Hajime Nagahara.  Mtunet: Few-shot image classification with visual explanations.  In CVPR Workshops, pages 2294–2298, 2021.
  * [44] Yaqing Wang, Quanming Yao, James T. Kwok, and Lionel M. Ni.  Generalizing from a few examples: A survey on few-shot learning.  ACM Comput. Surv., 53(3):63:1–63:34, 2020.
  * [45] Davis Wertheimer, Luming Tang, and Bharath Hariharan.  Few-shot classification with feature map reconstruction networks.  In CVPR, pages 8012–8021, 2021.
  * [46] Chen Xing, Negar Rostamzadeh, Boris N Oreshkin, and Pedro O Pinheiro.  Adaptive cross-modal few-shot learning.  In NeurIPS, pages 4848–4858, 2019.
  * [47] Chengming Xu, Yanwei Fu, Chen Liu, Chengjie Wang, Jilin Li, Feiyue Huang, Li Zhang, and Xiangyang Xue.  Learning dynamic alignment via meta-filter for few-shot learning.  In CVPR, pages 5182–5191, 2021.
  * [48] Zhiyu Xue, Lixin Duan, Wen Li, Lin Chen, and Jiebo Luo.  Region comparison network for interpretable few-shot image classification.  arXiv preprint arXiv:2009.03558, 2020.
  * [49] Bin-Bin Yang, Song-Qing Shen, and Wei Gao.  Weighted oblique decision trees.  In AAAI, pages 5621–5627, 2019.
  * [50] Ling Yang, Liangliang Li, Zilun Zhang, Xinyu Zhou, Erjin Zhou, and Yu Liu.  DPGN: distribution propagation graph network for few-shot learning.  In CVPR, pages 13390–13399, 2020.
  * [51] Yongxin Yang, Irene Garcia Morillo, and Timothy M Hospedales.  Deep neural decision trees.  arXiv preprint arXiv:1806.06988, 2018.
  * [52] Han-Jia Ye, Hexiang Hu, De-Chuan Zhan, and Fei Sha.  Few-shot learning via embedding adaptation with set-to-set functions.  In CVPR, pages 8805–8814, 2020.
  * [53] Baoquan Zhang, Xutao Li, Yunming Ye, Shanshan Feng, and Rui Ye.  Metanode: Prototype optimization as a neural ode for few-shot learning.  arXiv preprint arXiv:2103.14341, 2021.
  * [54] Baoquan Zhang, Xutao Li, Yunming Ye, Zhichao Huang, and Lisai Zhang.  Prototype completion with primitive knowledge for few-shot learning.  In CVPR, pages 3754–3762, 2021.
  * [55] Chi Zhang, Yujun Cai, Guosheng Lin, and Chunhua Shen.  Deepemd: Few-shot image classification with differentiable earth mover’s distance and structured classifiers.  In CVPR, pages 12203–12213, 2020.
  * [56] Jian Zhang, Chenglong Zhao, Bingbing Ni, Minghao Xu, and Xiaokang Yang.  Variational few-shot learning.  In ICCV, pages 1685–1694, 2019.