# Spin-selective magneto-conductivity in WSe2

En-Min Shih1αβ Qianhui Shi1γ Daniel Rhodes2 Bumho Kim2 Kenji Watanabe3 Takashi
Taniguchi4 Kun Yang5 James Hone2 Cory R. Dean1 1Department of Physics,
Columbia University, New York, NY, USA  2Department of Mechanical Engineering,
Columbia University, New York, NY, USA  3Research Center for Electronic and
Optical Materials, National Institute for Materials Science, 1-1 Namiki,
Tsukuba 305-0044, Japan  4Research Center for Materials Nanoarchitectonics,
National Institute for Materials Science, 1-1 Namiki, Tsukuba 305-0044, Japan
5Department of Physics and National High Magnetic Field Laboratory, Florida
State University, Tallahassee, Florida 32306, USA  αPresent address: Physical
Measurement Laboratory, National Institute of Standards and Technology,
Gaithersburg, MD 20899, USA  βPresent address: Department of Chemistry and
Biochemistry, University of Maryland, College Park, MD 10 20742, USA  γPresent
address: Department of Physics and Astronomy, University of California, Los
Angeles, CA, USA

Material systems that exhibit tunable spin-selective conductivity are key
components of spintronic technologies. Here we demonstrate a novel type of
spin-selective transport, based on the unusual Landau level (LL) sequence
observed in bilayer WSe2 under large applied magnetic fields. We find that the
conductivity depends strongly on the relative iso-spin ordering between
conducting electrons in a partially filled LL and the localized electrons of
lower energy filled LLs, with conductivity observed to be almost completely
suppressed when the spin-ratio and field-tuned Coulomb energy exceed a
critical threshold. Switching between “on/off” states is achievable through
either modulation of the external magnetic or electric fields, with many-body
interaction driving a collective switching mechanism. In contrast to
magnetoresistive heterostructures, this system achieves electrically tunable
spin filtering within a single material, driven by interaction between free
and localized spins residing in energy-separated spin/valley polarized bands.
Similar spin-selective conductivity may be realizable in multi-flat band
systems at zero magnetic field.

Spin-dependent transport effects, such as giant magnetoresistance (GMR), play
a fundamental role in spintronics. In GMR structuresBinasch  _et al._ (1989);
Fert (2008); Telford  _et al._ (2020), electrons in a non-magnetic conduction
layer scatter off moments in the surrounding magnetic layers due to exchange
interaction. The scattering rate is typically enhanced when the conducting
electron and the moment have different spins, so that a change in the relative
polarization under applied fields causes a large change in resistanceFert and
Campbell (1968). This effect is exploited in functional devices such as spin
valves and magnetic field sensors, and plays an important role in current data
storage technologies Chappert  _et al._ (2007).

Evidence of a homologous effect has been observed in the quantum Hall regime
of certain large-mass two-dimensional quantum wells, but remains far less
explored. In both AlAs and ZnO, a combination of perpendicular and parallel
magnetic fields can induce a spin splitting that exceeds the cyclotron gap,
$E_{z}/E_{cylc.}>1$. This results in spin-polarized bands, with multiple
lower-energy, filled, Landau levels (LLs) polarized with one spin and the LL
at the Fermi energy carrying a different spin. It appears that the conducting
electrons interact with the localized spins of the filled states such that the
longitudinal conductivity is diminished (enhanced) when the Fermi energy is in
a spin minority (majority) LLVakili  _et al._ (2005); Maryenko  _et al._
(2015). However, the observed contrast between spin-minority and majority
conductivity has been comparatively modest and the precise mechanism has
remained unresolved. Nonetheless, these results hint at the possibility of
realizing a novel kind of spin-selective transport within a single material
with energy-separated spin-polarized bands, rather than a heterostructure with
spatially separated magnetic domains.

Here we investigate hole-transport in high mobility bilayer WSe2 in the
quantum Hall regime. WSe2 is a van der Waals semiconductor that, like
graphene, can be exfoliated to the atomically thin limit. Unlike graphene,
WSe2 possesses an exceptionally large spin suceptibilityXu  _et al._ (2017)
that leads to an atypical quantum Hall scenario where the Zeeman energy
exceeds the cyclotron energy under purely perpendicular fieldsFallahazad  _et
al._ (2016); Xu  _et al._ (2017); Gustafsson  _et al._ (2018); Shi  _et al._
(2020, 2021). Together with spin-valley locking, this gives rise again to a
highly polarized LL sequence, where a large number of the lowest energy bands
all share the same iso-spin polarization Gustafsson  _et al._ (2018); Shi  _et
al._ (2020, 2021) (Fig. 1a, b). We find that WSe2 exhibits a similar spin-
dependent conductance as in polarized AlAs and ZnO, but with a much more
dramatic effect. The conductance within partially filled majority- and
minority-spin LLs shows opposite temperature dependence, with the minority-
spin LL conductance approaching zero at low temperature and high magnetic
field. This observation identifies a regime of essentially perfect spin
selectivity arising from a complete localization behaviour for the minority
carriers.

Figure 1:  Device geometry and spin-dependent magneto-transport. (a) Schematic
of single layer WSe2 bands under magnetic field. The Landau level (LL)
sequences in different valley and carrier bands are labeled. (b) Hole filling
of LLs at filling fraction $\nu=6.8$, at which the first minority spin LL is
being filled. (c) Schematic of the dual-gated bilayer WSe2 device. (d) Two
measurement geometries. Top panel: two-probe measurement in Corbino
configuration. Bottom panel: four-probe measurement in van der Pauw
configuration. (e) Rxx and Rxy versus magnetic field in van der Pauw
configuration at $n=-3.64\times 10^{12}$ cm-2 with only top layer of bilayer
WSe2 populated. (f) Mapping of Rxx versus magnetic field and density. The
first few LL filling factors are labeled. (g) Schematic of (f) with the
orbital number and spin state of valence LL labeled.  Figure 2: Electronic
transport signatures of exotic localization in minority spin LL. (a) Two-probe
conductance versus filling factor in Corbino configuration at B = 15 T at
different temperature. The base temperature (black line) is 0.8 K. (b)
Normalized two-probe conductance versus temperature at different half-fillings
at B = 15T. The conductance is normalized with conductance at 12K. (c)
$R_{xx}$and $R_{xy}$versus filling factor at B = 24 T and B = 28 T. At
$\nu=5.5$, the valence LL is in spin-majority state and spin-minority state at
B = 24 T and B = 28T, respectively. (d)(e) Cartoon illustrations of the spin-
dependent transport mechanism. In (d) the valence LL is in spin-majority state
and the mobile charge can flow through the immobile same-spin charge
background freely, while in (e) the valence LL is in spin-minority state and
the mobile charge is auto-localized due to dressing the distorted immobile
opposite spin charges.

We suggest that the spin dependent localization may be understood from a
polaron-like model, in which a minority spin carrier displaces the background
incompressible majority spin carriers and becomes self-localized. By studying
this effect in blayer WSe2, we show that the spin-dependent transport can be
generalized to real spin or valley pseudospin. Finally, we demonstrate that
switching between the high/low conductance states can be initiated within a
single, partially filled, LL when Coulomb interaction drives a collective spin
flip. These results provide a prototypical demonstration of electric-field
tunable spin-dependent transport in an all-electron system, where Coulomb
interactions and exchange effects play central roles.

We measured magneto-transport in dual-gated bilayer WSe2 fabricated with pre-
patterned Pt contact elelctrodesFallahazad  _et al._ (2016) (Fig. 1c). The
bilayer WSe2 was exfoliated from a flux-grown single crystal, with charged
defect density lower than $10^{11}$ cm-2Edelberg  _et al._ (2019). Using the
van der Waals stacking approachWang  _et al._ (2013) together with standard
nanofabrication technqiues, we implemented a novel device geometry that
supports measurements of both edge and bulk transport in the same device (Fig.
1d). Six contacts arranged in a circular pattern enable four-terminal
longitudinal ($R_{xx}$) and Hall resistance ($R_{xy}$) measurements in a“van
der Pauw geometry”(Fig. 1d bottom). Additionally, a center contact enables
measurement of the two-terminal conductance between the center and edge
contacts probes the bulk response in the Corbino geometry (Fig. 1d top). Dual
gates provide independent control of the carrier density of the two layers Shi
_et al._ (2021); Zeng  _et al._ (2013); Xu  _et al._ (2014); Fallahazad  _et
al._ (2016); Pisoni  _et al._ (2019), allowing us to study the device response
with either a single layer or both layers populated. We restrict our
measurement here to the hole band response only.

We first focus on the regime with only one layer populated, where the device
effectively behaves like an isolated monolayer. Figure 1e shows $R_{xx}$ and
$R_{xy}$, measured in the van der Pauw geometry as a function of applied
perpendicular field $B$, with hole density tuned to
n${}_{\mathrm{hole}}=3.64\times 10^{12}$ cm-2, and transverse displacement
field, $D=1.14$ $V/$nm. The Hall mobility, measured in the low field regime,
is $\mu_{H}\approx 30,000$ cm2/Vs, which is the highest reported for any
semiconductor TMDLin  _et al._ (2019); Movva  _et al._ (2015). Quantum
oscillations appear below 2 T, giving an approximate estimate for the Landau
level broadening of 5.92 K, which is less than has been reported for high
mobility grapheneZeng  _et al._ (2019). In the high field regime, fully
quantized Hall plateaus concomitant with zero longitudinal resistance are
observed at integer fillings, further confirming the high quality of the
device.

The most dramatic feature seen in the magnetoresistance (Fig. 1e) is a an
alternating high/low longitudinal resistivity between neighbouring LLs.
Previous studies of WSe2 established the LL spin splitting to be density- and
field-dependent, giving rise to the approximate ladder of states shown in Fig.
1aGustafsson  _et al._ (2018); Shi  _et al._ (2020). At magnetic fields on the
order 15 T, the lowest 6 LLs are fully iso-spin polarized, with higher
fillings corresponding to alternating majority/minority iso-spin. We find that
the resistance modulation correlates with the relative spin order at the Fermi
energy with high (low) $R_{xx}$ coinciding with the spin majority (minority)
LLs. For example, the peak resistance at $\nu=6.8$, which corresponds to 6
filled polarized levels and one partial filled anti-poplarized (Fig. 1b), has
resistance less than 1/5 the value at partial filling of neighbouring, spin-
majority, LLs. The general influence of the spin state on transport is
unambiguously demonstrated in the plot of $R_{xx}$ versus field and density
shown in Fig. 1f. The $R_{xx}$ value periodically switches between high value
(red) and low value (blue), matching the spin transition map shown in Fig. 1g.

In the high-field, low-density regime, the longitudinal conductivity,
$\sigma_{xx}$, is approximately proportional to the longitidunal resistivity,
$\rho_{xx}$, such that low resistance indicates low conductance. The Corbino
geometry is used to probe the bulk conductance directly, i.e. independent of
any potential edge effects. Figure 2a shows the two-terminal conductance in
the Corbino geometry, acquired at $B=15$T and varying temperature. At T = 0.8K
(black line), majority-spin LLs exhibit high conductance and minority spins
exhibit low conductance, consistent with the van der Pauw resistivity
measurements. The temperature dependence likewise shows two behaviours (Fig.
2b). For majority-spin fillings, the conductance exhibits metallic-like
response, i.e. increases with decreasing temperature, as is typical for normal
QH systems at partial filling. For the minority spins, the conductance
decreases with decreasing temperature, indicating an insulating-like bulk
response consistent with localization of the carriers.

The localization strength varies with both total filling fraction and magnetic
field. In the Corbino measurement at $B=15$ T (Fig. 2a), the spin-minority
conductance is minimal at filling fraction 6.5, corresponding to a single
anti-polarized LL in a background of fullly polarized LLs, and then increases
monotonically with increasing LL index. Figure 2c shows the effect of varying
the magnetic field, measured in the van der Pauw geometry. As the magnetic
field is increased from 24 T, filling fraction at 5.5 undergoes a transition
from spin majority to spin minority, evidenced by transition from large
$R_{xx}$(dashed black line) to small $R_{xx}$(solid black line). By around 28
T, $R_{xx}$completely vanishes across the full LL, within measurement
resolution. Simultaneously, the $\nu=5$ Hall plateau extends across this same
filling range. This remarkable observation suggests that the spin-minority LL
is fully localized within the entire LL.

Figure 3: Flavor-dependent transport in the bilayer region. (a) Mapping of Rxx
versus total filling factor and displacement field. (b) Schematic diagram of
(a) with filling factors of the two layer labeled. At partial filling, the
state is color coded with the p value. (c)(d)(e) Examples of p value at
different LL configurations. (c) The valence LL species occupies all the
filled LLs, $p=1$. (d) No valence LL species in the filled LLs, $p=0$. (e)
Four valence LL species among total five filled LLs, $p=0.8$. (f)
$R_{xx}$plotted with displacement field at $\nu_{tot}=5.5$. At half-filling of
every valence LL, the p value and the specie of valence LL are labeled. (g)
Normalzed $R_{xx}$versus p value at different $\nu_{tot}$. The $R_{xx}$is
normalied to the value at $p=1$.  Figure 4: Minority-spin to majority-spin
transition within valence LL. (a) Mapping of conductance versus magnetic field
and filling factor around the transition. (b) Cartoon illustration of the
transition in (a). (c) Mapping of conductance versus temperature and filling
factor along B = 13.4 T (white dashed line in (a)). (d) The linecuts of
conductance versus temperature along $\nu=6.5$ (red dashed line in (c)) and
$\nu=6.77$ (blue dashed line in (c)). (e) Conductance versus magnetic field
(or filling fraction) along the the white dashed line shown in the inset.
Inset: mapping of conductance versus magnetic field and back-gate voltage.

Both the filling fraction and magnetic field dependence of the minority
carrier localization can be understood from the spin-dependent exchange
interaction between the mobile and localized carriers. Mobile carriers
interact weakly (strongly) with localized carriers that share the same
(opposite) iso-spin polarization. The effect on transport is illustrated
schematically in Fig. 2d, e. Mobile carriers in a partially-filled spin-
majority LL scatter only weakly from the localized carriers (Fig. 2d). For
spin-minority carriers, the strong Coulomb interaction can distort the
immobile background charges, such that the free carrier becomes effectively
more massive and less conductive (Fig. 2e). This is analogous to a polaron,
but in an all electron system with the immobile carriers in the filled LLs
playing the equivalent role of a crystal lattice. Since in the QHE regime the
Coulomb energy scales with the magnetic field, $E_{c}=e^{2}/\epsilon
l_{B}\propto\sqrt{B_{\perp}}$, the minority-carrier should become increasingly
localized with increasing field, consistent with our observation. On the other
hand, as filling fraction is increased at fixed magnetic field, the net
polarization of the carriers reduces towards zero, reducing the difference
between majority and minority spin mobilities, also consistent with
observation. The picture identified here may also explain why the observed
effect is magnified in WSe2 compared with previous studies. The influence of
the filled LLs can be characterized by the LL mixing parameterSodemann and
MacDonald (2013); Peterson and Nayak (2013); Simon and Rezayi (2013),
$\kappa=E_{Coulomb}/E_{cycl.}$, i.e. the ratio of the Coulomb energy, to the
cyclotron gap. For comparison we note that $\kappa=\alpha/\sqrt{B}$, with
$\alpha=2.6,16.7,22.5,31$ in GaAs, ZnO, AlAs and WSe2, respectively. The
unusually large $\kappa$, together with the large ratio of $E_{z}/E_{cycl.}$
creates a unique scenario in WSe2 with simultaneously larger polarization and
larger mixing than in previously studied systems.

Both the real spin and the valley pseudospin determine the exchange
interaction. In bilayer WSe2, the strong spin-orbit coupling and the stacking
order result in coupled spin, valley and layer degrees of freedom. A total of
four flavors are relevant: $(\uparrow,K)$, $(\downarrow,K^{\prime})$ localized
in the top layer, and $(\uparrow,K^{\prime})$, $(\downarrow,K)$ localized in
the bottom layer Shi  _et al._ (2021). Therefore, the valley degree of freedom
can be controlled by a displacement field, $D$, which moves carriers from one
layer to another. In Fig. 3a we plot $R_{xx}$measured at 15 T versus
$D=(V_{T}-V_{B})/2\epsilon_{0}$ and the total filling factor
$\nu_{tot}=hC(V_{T}+V_{B})/e^{2}B$, where $V_{T}$ and $V_{B}$ are top gate and
bottom gate voltage bias, and $C$ is the geometric capacitance.

The resistance shows a checkerboard-like pattern as charge is transferred
between different LLs localized in the two layers Shi  _et al._ (2021). For
$\nu<6$, all LLs have the same spin, but charges in the top layer reside in
the K valley and those in the bottom layer are in the K’ valley.

Examples of the change in LL configuration versus $D$ for fixed filling
fraction, $\nu=5.5$, are shown in Fig. 3c-e. At large negative $D$, the free
and localized electrons share the same spin and valley. As $|D|$ is decreased
from the fully valley-polarized regime, the Fermi level switches between
valley K and K’.

Figure 3f shows a plot of $R_{xx}$versus $D$ at $\nu=5.5$, corresponding to a
linecut along the dashed line in Fig. 3a. $R_{xx}$oscillates as the Fermi
level switches between majority (large $R_{xx}$) and minority (small $R_{xx}$)
valley spin. The resistance is approximately symmetric about $D=0$, confirming
that the value of $R_{xx}$is dominated by the relative net iso-spin order,
rather than other features such as the spin or valley flavour. To quantify how
$R_{xx}$varies with the relative iso-spins we introduce the parameter,
$p=N_{F}/N_{tot}$, where $N_{tot}$ is the total number of filled LLs, and
$N_{F}$ is the number of filled LLs that have the same flavor as the LL at the
Fermi energy. We label the peaks and minima in Fig. 3f by these $p$ values and
see that indeed $R_{xx}$is maximized (minimized) when $p$ is large (small).

Figure 3g plots the evolution of the normalized longitudinal resistance
$R_{xx}/R_{xx}(p=1)$ versus $p$ for three filling factors 3.5, 4.5 and 5.5.
$R_{xx}/R_{xx}(p=1)$ shows an approximately linear increase with $p$
confirming the longitudinal resistance is proportional to the net population
difference between the free carrier flavor and the localized carrier flavors.
In Fig. 3b, we colour distinct regions in the $D-\nu$ phase space by the
magnitude of $p$. For $\nu>6$, $p$ is calculated as the population of the
valence flavor among all four flavors. The consistency of the polarization
color map with the $R_{xx}$color map in Fig. 3a again indicates correlation
between $p$ and the resistance. These observations are consistent with our
picture that localization of the mobile carriers at the Fermi energy is driven
by interaction with incompressible carriers of a different flavor: the greater
the population of different-flavored incompressible carriers, the lower the
conductance.

Finally, we discuss the switching between majority/minority flavors within a
single partially filled LL. Figure 4a plots the conductance versus magnetic
field and filling factor around an iso-spin transition due to a LL crossing.
As before, high (low) conductance correlates with iso-spin majority (minority)
free carriers. In detail we see that the conductance transition from high to
low value onsets at partial fillings across a well defined and narrow
boundary. This suggests a collective transition that reorders the spin at the
Fermi level (Fig. 4b)Hunt  _et al._ (2017); Shkolnikov  _et al._ (2005);
Pisoni  _et al._ (2018). Figure 4c plots the conductance at $B$ = 13.4 T
versus the filling factor (white dashed-line in Fig. 4a) and temperature. The
temperature dependence of the conductance at two filling factors ($\nu=6.55$
and $6.77$) are compared in Fig. 4d. The temperature dependence is effectively
identical down to approximately 5 K, below which the trends suddenly diverges
with $\nu=6.55$ dropping sharply towards zero (consistent with minority
carrier localization) while $\nu=6.77$ shows metallic like response. The
sudden change in beahviour below an apparent critical temperature further
supports the notion of a many-body driven spin-transition when the temperature
drops below a characteristic interaction energy scale.

The collective flip of many spins with increasing filling factor is evocative
of LL levitation driven by Coulomb interactionsHaldane and Yang (1997). With
increasing partial filling factor in a minority-spin LL, the strong Coulomb
interaction between the minority-spin carriers at the Fermi level and the
majority-spin carriers in the background (and the urge to optimally screen
this interactions) effectively pushes the valence LL up. This LL levitation
with increasing partial filling explains the missing extended-states at high
magnetic field (Fig. 2c).

Finally, when the filling factor is tuned by sweeping the magnetic field at a
fixed density, a large conductance spike is observed at the transition. This
typically suggests the formation of domain walls Jungwirth and MacDonald
(2001) between the two different spin states indicating a first-order
transition (Fig. 4e).

In summary, we have shown strong spin-dependent transport in bilayer WSe2. We
demonstrate that spin/valley-polarized band arrangement in energy space can
give strong spin/valley-selective transport, achieving perfect selectivity in
the extreme limit. The ability to engineer polarized flat bands in van der
Waals heterstructures, such as through moire patterning, might enable a
similar electric-field driven iso-spin selectivity under zero magnetic fields.
This type of bandstructure driven GMR could pave the way for next-generation
spintronic devices.

We thank Luis Balicas, William Coniglio and Bobby Pullum for help with
experiments at the National High Magnetic Field Lab. This research is
primarily supported by US Department of Energy (DE-SC0016703). Synthesis of
WSe2 (D.R.,B.K.,K.B.) was supported by the Columbia University Materials
Science and Engineering Research Center (MRSEC), through NSF grants
DMR-1420634 and DMR-2011738. The work of KY was supported by the National
Science Foundation Grant No. DMR-1932796. A portion of this work was performed
at the National High Magnetic Field Laboratory, which is supported by National
Science Foundation Cooperative Agreement No. DMR-1644779 and the State of
Florida. K.W. and T.T. acknowledge support from the JSPS KAKENHI (Grant
Numbers 20H00354, 21H05233 and 23H02052) and World Premier International
Research Center Initiative (WPI), MEXT, Japan.

## References

  * Binasch  _et al._ (1989) G. Binasch, P. Grünberg, F. Saurenbach, and W. Zinn, Phys. Rev. B 39, 4828 (1989).
  * Fert (2008) A. Fert, Rev. Mod. Phys. 80, 1517 (2008).
  * Telford  _et al._ (2020) E. J. Telford, A. H. Dismukes, K. Lee, M. Cheng, A. Wieteska, A. K. Bartholomew, Y.-S. Chen, X. Xu, A. N. Pasupathy, X. Zhu, C. R. Dean, and X. Roy, Advanced Materials 32, 2003240 (2020).
  * Fert and Campbell (1968) A. Fert and I. A. Campbell, Phys. Rev. Lett. 21, 1190 (1968).
  * Chappert  _et al._ (2007) C. Chappert, A. Fert, and F. N. Van Dau, Nature Materials 6, 813 (2007).
  * Vakili  _et al._ (2005) K. Vakili, Y. Shkolnikov, E. Tutuc, N. Bishop, E. De Poortere, and M. Shayegan, Physical review letters 94, 176402 (2005).
  * Maryenko  _et al._ (2015) D. Maryenko, J. Falson, M. Bahramy, I. Dmitriev, Y. Kozuka, A. Tsukazaki, and M. Kawasaki, Physical review letters 115, 197601 (2015).
  * Xu  _et al._ (2017) S. Xu, J. Shen, G. Long, Z. Wu, Z.-q. Bao, C.-C. Liu, X. Xiao, T. Han, J. Lin, Y. Wu, _et al._ , Physical Review Letters 118, 067702 (2017).
  * Fallahazad  _et al._ (2016) B. Fallahazad, H. C. Movva, K. Kim, S. Larentis, T. Taniguchi, K. Watanabe, S. K. Banerjee, and E. Tutuc, Physical review letters 116, 086601 (2016).
  * Gustafsson  _et al._ (2018) M. V. Gustafsson, M. Yankowitz, C. Forsythe, D. Rhodes, K. Watanabe, T. Taniguchi, J. Hone, X. Zhu, and C. R. Dean, Nature materials 17, 411 (2018).
  * Shi  _et al._ (2020) Q. Shi, E.-M. Shih, M. V. Gustafsson, D. A. Rhodes, B. Kim, K. Watanabe, T. Taniguchi, Z. Papić, J. Hone, and C. R. Dean, Nature Nanotechnology , 1 (2020).
  * Shi  _et al._ (2021) Q. Shi, E.-M. Shih, D. Rhodes, B. Kim, K. Barmak, K. Watanabe, T. Taniguchi, Z. Papić, D. A. Abanin, J. Hone, _et al._ , arXiv preprint arXiv:2108.10477 (2021).
  * Edelberg  _et al._ (2019) D. Edelberg, D. Rhodes, A. Kerelsky, B. Kim, J. Wang, A. Zangiabadi, C. Kim, A. Abhinandan, J. Ardelean, M. Scully, _et al._ , Nano letters 19, 4371 (2019).
  * Wang  _et al._ (2013) L. Wang, I. Meric, P. Y. Huang, Q. Gao, Y. Gao, H. Tran, T. Taniguchi, K. Watanabe, L. M. Campos, D. A. Muller, J. Guo, P. Kim, J. Hone, K. L. Shepard, and C. R. Dean, Science 342, 614 (2013).
  * Zeng  _et al._ (2013) H. Zeng, G.-B. Liu, J. Dai, Y. Yan, B. Zhu, R. He, L. Xie, S. Xu, X. Chen, W. Yao, _et al._ , Scientific reports 3, 1 (2013).
  * Xu  _et al._ (2014) X. Xu, W. Yao, D. Xiao, and T. F. Heinz, Nature Physics 10, 343 (2014).
  * Pisoni  _et al._ (2019) R. Pisoni, T. Davatz, K. Watanabe, T. Taniguchi, T. Ihn, and K. Ensslin, Physical review letters 123, 117702 (2019).
  * Lin  _et al._ (2019) J. Lin, T. Han, B. A. Piot, Z. Wu, S. Xu, G. Long, L. An, P. Cheung, P.-P. Zheng, P. Plochocka, _et al._ , Nano letters 19, 1736 (2019).
  * Movva  _et al._ (2015) H. C. Movva, A. Rai, S. Kang, K. Kim, B. Fallahazad, T. Taniguchi, K. Watanabe, E. Tutuc, and S. K. Banerjee, ACS nano 9, 10402 (2015).
  * Zeng  _et al._ (2019) Y. Zeng, J. Li, S. Dietrich, O. Ghosh, K. Watanabe, T. Taniguchi, J. Hone, and C. Dean, Physical review letters 122, 137701 (2019).
  * Sodemann and MacDonald (2013) I. Sodemann and A. MacDonald, Physical Review B 87, 245425 (2013).
  * Peterson and Nayak (2013) M. R. Peterson and C. Nayak, Physical Review B 87, 245129 (2013).
  * Simon and Rezayi (2013) S. H. Simon and E. H. Rezayi, Physical Review B 87, 155426 (2013).
  * Hunt  _et al._ (2017) B. Hunt, J. Li, A. Zibrov, L. Wang, T. Taniguchi, K. Watanabe, J. Hone, C. Dean, M. Zaletel, R. Ashoori, _et al._ , Nature communications 8, 948 (2017).
  * Shkolnikov  _et al._ (2005) Y. Shkolnikov, S. Misra, N. Bishop, E. De Poortere, and M. Shayegan, Physical review letters 95, 066809 (2005).
  * Pisoni  _et al._ (2018) R. Pisoni, A. Kormányos, M. Brooks, Z. Lei, P. Back, M. Eich, H. Overweg, Y. Lee, P. Rickhaus, K. Watanabe, _et al._ , Physical review letters 121, 247701 (2018).
  * Haldane and Yang (1997) F. Haldane and K. Yang, Physical review letters 78, 298 (1997).
  * Jungwirth and MacDonald (2001) T. Jungwirth and A. MacDonald, Physical review letters 87, 216801 (2001).
  * Padmanabhan  _et al._ (2010) M. Padmanabhan, T. Gokmen, and M. Shayegan, Physical Review B 81, 113301 (2010).
  * Maryenko  _et al._ (2012) D. Maryenko, J. Falson, Y. Kozuka, A. Tsukazaki, M. Onoda, H. Aoki, and M. Kawasaki, Physical review letters 108, 186803 (2012).
  * Young  _et al._ (2012) A. F. Young, C. R. Dean, L. Wang, H. Ren, P. Cadden-Zimansky, K. Watanabe, T. Taniguchi, J. Hone, K. L. Shepard, and P. Kim, Nature Physics 8, 550 (2012).
  * Aleiner and Glazman (1995) I. Aleiner and L. Glazman, Physical Review B 52, 11296 (1995).

##  I Supplementary Materials

Spin-selective magneto-conductivity in WSe2
E. Shih, Q. Shi, D. Rhodes, B. Kim, K. Watanabe, T. Taniguchi, K. Yang, J.
Hone, C. R. Dean

This PDF file includes:
Supplementary Text
Materials and Methods

##  II S1. Device Fabrication and Geometry

The cross-section of bilayer WSe2 device described in the main text is shown
in Fig. S1a. The full stack is made of two pieces: the bottom stack (Fig. S1a
blue dashed block and photo in Fig. S1c) is made of BN encapsulated graphite
followed by deposition of prepatterned contact Cr/Pt (2nm/20nm), and the top
stack (Fig. S1a red dashed block and photo in Fig. S1b) is made by picking
layers in order of BN/Graphite/BN/2L WSe2. The top stack is then put on the
bottom stack with 2L WSe2 contacted with the Pt (Fig. S1d). Next, the partial
etching step shapes the top-gate which defined the channel area (Fig. S1e).
Then, another BN is put on top of stack to avoid shorting the top-gate to the
later evaporated leads (Fig. S1f). In this design, leads can be placed across
the channel and make Corbino geometry possible. Finally, holes are etched to
the Pt prepattern (Fig. S1g) followed by deposition of channel leads and
contact-gate (Fig. S1h). The device is designed so that the contact-gate can
turn on the contact region (with large negative voltage), and top/bottom gate
voltage can be positive or negative; therefore the full density and
displacement field phase space can be mapped out. In device a (Fig. S1h),
because the contact gate short all the outer leads together, we have to use
negative topgate voltage to turn on contact region, which leaves top-layer
always on. In device b (Fig. S2), the contact channel doesn’t short the leads;
therefore, we can map out the full phase space as shown in main text Fig. 3.

Figure S1:  a, Full stack schematic. Optical image of b, Top stack, c, Bottom
stack, d, Combined top and bottom stack, e, Etch-shaped top gate, f, Another
BN put on, g, Etching to the Pt prepattern h, Evaporated electrodes of
contact-gate and channel leads. Finished device a.  Figure S2:  Device b with
three (top, bottom, contact) gates labeled.

##  III S2. Landau Level Structure in Bilayer WSe2

The four iso-spin flavors of bilayer WSe2 are protected by spin-orbit coupling
and inversion symmetry of the crystal (more discussion see SI of Ref. Shi  _et
al._ (2021)). Under magnetic field, each flavor band split into Landau levels
and their relative ordering depends on Zeeman energy (Ez) and electric field
energy across the layers (EL), as schematically shown in Fig. S3.

Figure S3:  Schematic of bilayer WSe2 bands under magnetic field. The Landau
levels of different iso-spin flavors are labeled.

##  IV S3. Comparing Different 2D Charge Carrier Systems

| GaAs (electron) | GaAs (hole) | AlAs | ZnO | Graphene | WSe2
---|---|---|---|---|---|---
$m^{*}/m_{0}$ | 0.069 | 0.38 | 0.46 | 0.29 |  | 0.45
$\epsilon/\epsilon_{0}$ | 13 | 13 | 10 | 8.5 | 6-7 | 7
$g^{*}$ | 0.44 |  | 4-6 | 4 | 4-8 | 9-20
$E_{c}(=e^{2}/\epsilon l_{B}$) (K) | 50 $\sqrt{B_{\perp}}$ | 50 $\sqrt{B_{\perp}}$ | 64.7 $\sqrt{B_{\perp}}$ | 75 $\sqrt{B_{\perp}}$ | 108$\sqrt{B_{\perp}}$ | 93 $\sqrt{B_{\perp}}$
$E_{cycl}(=\hbar\omega)$ (K) | 19.4 B | 3.5 B | 2.9 B | 4.6 B | 216 B | 2.96 B
$\kappa(=E_{c}/E_{cycl})$ | 2.6/$\sqrt{B_{\perp}}$ | 14.6/$\sqrt{B_{\perp}}$ | 22.5/$\sqrt{B_{\perp}}$ | 16.6/$\sqrt{B_{\perp}}$ | 0.5-2.2 | 31/$\sqrt{B_{\perp}}$
$E_{z}/E_{c}(=g^{*}m^{*})$ | 0.03 |  | 1.84-2.76 | 1.16 | 0.01-0.04 | 4-9
Table S1: Comparison of characteristics of 2D charge carrier systems,
including GaAs, AlAsSodemann and MacDonald (2013); Padmanabhan  _et al._
(2010), ZnOSodemann and MacDonald (2013); Maryenko  _et al._ (2012),
grapheneYoung  _et al._ (2012), and WSe2Fallahazad  _et al._ (2016)(B is
measured in Tesla).

##  V S4. Landau Level Levitation

The screening capability depends on the partial filling factor in this
strongly-interacting system. The polarizability $\Pi$ of carriers in the
filled LLs peaks near a momentum q$\approx 1/l$, where $l$ is the magnetic
lengthAleiner and Glazman (1995). Therefore, the interaction between minority
carriers and the backround majority carriers is optimally screened when the
minority-spin charges are localized at a length scale $l$. As the filling of
minority spin carriers increases, their wavefunctions overlap and the
potential from them becomes more uniform and no longer localizes around $l$.
Therefore, to minimize the interaction energy, the minority-spin carriers
further localize themselves by mixing with higher LLs. The LL mixing in turn
pushes the extended states of the valence LL higher in energy Haldane and Yang
(1997); this process stops only when the majority spin electrons start to
enter the valence LL. This phenomenon can explain our observation in main text
Fig.2(c) that conduction due to extended states is missing in a large range of
filling factor $5<\nu<6$, with $R_{xx}$remaining at zero and $R_{xy}$at the
plateau value.

##  VI S5. Electrical characterization

Figure S4:  (a) Comparison between 2-probe resistance $R_{2p}$, 4-probe
resistance $R_{4p}$ and contact resistance $R_{c}$ versus charge density. The
contact resistance is derived by $R_{c}=R_{2p}-(L_{tot}/L_{in})R_{4p}$, where
$L_{tot}$ and $L_{in}$ are the total and inner channel lengths, respectively,
and $L_{tot}/L_{in}\approx 2$. (b) Hall mobility versus charge density.
$\mu_{H}=(L/W)dR_{xy}/dB/R_{xx}$, where L and W are the channel aspect
ratio$\approx 1$, $dR_{xy}/dB$ is calculated at low field.


# Self-Taught Cross-Domain Few-Shot Learning with Weakly Supervised Object
Localization and Task-Decomposition

Xiyao Liu, Zhong Ji*, Yanwei Pang, Zhongfei Zhang Manuscript received xxx xx,
2021; revised xxx xx, 2021. This work was supported by the National Natural
Science Foundation of China (NSFC) under Grants 61771329 and 61632018.X. Liu,
Z. Ji*(corresponding author), and Y. Pang are with the School of Electrical
and Information Engineering, Tianjin University, Tianjin 300072, China
(e-mails<EMAIL_ADDRESS><EMAIL_ADDRESS>pyw@tju.edu.cn).Z. Zhang is
with the Computer Science Department, Binghamton University, NY 13902, USA
(e-mail: zhongfei@cs.binghamton.edu).

###### Abstract

The domain shift between the source and target domain is the main challenge in
Cross-Domain Few-Shot Learning (CD-FSL). However, the target domain is
absolutely unknown during the training on the source domain, which results in
lacking directed guidance for target tasks. We observe that since there are
similar backgrounds in target domains, it can apply self-labeled samples as
prior tasks to transfer knowledge onto target tasks. To this end, we propose a
task-expansion-decomposition framework for CD-FSL, called Self-Taught (ST)
approach, which alleviates the problem of non-target guidance by constructing
task-oriented metric spaces. Specifically, Weakly Supervised Object
Localization (WSOL) and self-supervised technologies are employed to enrich
task-oriented samples by exchanging and rotating the discriminative regions,
which generates a more abundant task set. Then these tasks are decomposed into
several tasks to finish the task of few-shot recognition and rotation
classification. It helps to transfer the source knowledge onto the target
tasks and focus on discriminative regions. We conduct extensive experiments
under the cross-domain setting including 8 target domains: CUB, Cars, Places,
Plantae, CropDieases, EuroSAT, ISIC, and ChestX. Experimental results
demonstrate that the proposed ST approach is applicable to various metric-
based models, and provides promising improvements in CD-FSL.

###### Index Terms:

few-shot learning, cross-domain, meta-learning, weakly supervised object
localization.

##  I Introduction

Few-Shot Learning (FSL) is to recognize novel visual concepts with several
examples based on the acquired knowledge [1, 2, 3]. It is also taken as a
practice paradigm for meta-learning [4, 5], which learns the method to
transfer old knowledge on novel tasks. However, vanilla FSL methods are
limited by that old and novel tasks are from the same domain. Obviously, this
setting is far from the demands of real-world applications. Thus, Cross-Domain
Few-Shot Learning (CD-FSL) is proposed, in which acquired knowledge and novel
tasks come from the source domain and the target domain respectively.
Accordingly, the domain shift from the source domain to the target one is the
main challenge in CD-FSL.

Figure 1: The illustration of Cross-Domain Few-Shot Learning. (a) Domain A and
B have different distributions of embedding space. (b) Many studies aim at
learning a shared space to align domain distributions. (c) Our approach
applies self-labeled instances to construct target-oriented metric spaces.

There are different embedding distributions in the source and target domain,
as shown in Fig. 1. The domain shift results in that the knowledge of domain A
can not work well on domain B. A typical method to address this problem is
aligning the two distributions by adversarial training [6], domain alignment
[7] and knowledge distillation [8]. They usually choose some samples from the
target domain as an auxiliary set during training. Obviously, these approaches
violate the CD-FSL setting, on which the target domain is absolutely unknown
during the training on the source domain. Therefore, FT [9] explores to
realize feature transformation without access to the target domain, which
employs affine transforms to simulate various feature distributions under
different domains. This transformation is not directed to the target domain,
which performs unstably on the different target domains. How to employ limited
labeled samples to provide a directive metric space for the target source is a
research focus in CD-FSL.

We observe that directly measuring the distance of the target and the source
domain is difficult by limited labeled target samples. By contrast, the tasks
of the same domain are highly related, so that these tasks can be decomposed
to guide the metric space. This guidance can be formed by employing the
labeled samples and self-labeled samples to construct multiple task-oriented
metric spaces, which updates parameters to adapt for the target domain
quickly, as shown in Fig.1 (c).

To construct task-oriented and robust metric spaces, it is important to
generate diverse and real samples. The current setting of CD-FSL is
transferring universal knowledge (miniImageNet) to fine knowledge. Since the
target domains are usually fine-grained datasets, the test samples often have
similar backgrounds, as shown in Fig. 2. The share of backgrounds could
provide task-oriented and abundant metric spaces for task-decomposition. Thus,
we introduce Weakly Supervised Object Localization (WSOL) to find the object
regions.

Figure 2: The examples of the target domains. There are similar and
exchangeable backgrounds on the fine-grained datasets (a) CUB and (b) Cars.

WSOL aims at obtaining the localization of objects only with the image-level
labels, which has broad applications in video understanding [10], medical
imaging [11] and remote sensing imagery analysis [12]. Most WSOL methods
extract activated regions of feature maps, which is called Class Activation
Map (CAM) [13]. These CAM-based approaches have achieved promising results on
WSOL. Considering the capability of coverage, we apply the simplest CAM
approach to find the discriminative regions and exchange the regions to
realize task expansion.

To this end, we design inner tasks nesting in few-shot tasks (outer tasks), of
which inner tasks learn task-oriented metric spaces by iterations. The inner
tasks contain 2 stages, i.e., task-expansion and task-decomposition, which are
formed by a task-expansion-decomposition framework. We name it Self-Taught
(ST) approach. The illustration of ST is shown in Fig. 3. Specifically, in the
Task-Expansion Module (TE-Module), weakly supervised object localization and
self-supervised technologies are employed to enrich task-related samples by
exchanging and rotating the discriminative regions. The TE-Module generates a
more abundant task set. Then the Task-Composition Module (TD-Module) samples
tasks from the set produced by TE-Module to finish the inner task, which
contains the task of few-shot recognition and rotation classification. After
several iterations of inner tasks, the parameters of networks are updated to
the optimal metric space for the outer task, i.e., the actual few-shot task.
Our ST approach is capable of fully utilizing limited labeled samples to
construct target-oriented metric spaces by the task-expansion-decomposition
framework.

The highlight of our work is three-fold:

1\. We propose a task-expansion-decomposition framework, called Self-Taught
(ST), for cross-domain few-shot learning, which alleviates the problem of
undirected guidance caused by the unknown target domain. Our ST approach can
work on many mainstream metric-based FSL methods, which is a plug-and-play
framework.

2\. We incorporate weakly supervised object localization and self-supervised
technologies to construct target-oriented metric spaces, which aims at
focusing on discriminative region representations. To the best of our
knowledge, this is the first work to introduce weakly supervised object
localization to cross-domain few-shot learning.

3\. Extensive experiments on eight target benchmark datasets clearly
demonstrate the effectiveness of our ST approach.

##  II Related work

Figure 3: An overview of the proposed ST approach. The few-shot task is
denoted as outer task, and its support set inputs inner task iterations.
Firstly, WSOL generates the object locations from the feature maps of labeled
samples. The task-expansion network applies the locations and original images
to augment tasks by rotations, exchanges and random convolution filters. Then,
task-decomposition network samples inner tasks to update the metric-based few-
shot method $f_{\phi}$. When finishing the inner tasks, the outer task is
trained to update the metric-based few-shot method $f_{\mu}$.

###  II-A Few-Shot Learning

The task of Few-Shot Learning (FSL) is proposed to recognize new categories
only from few samples for scarce annotations. The current mainstream FSL
methods are built on the meta-learning framework, which transfers prior
knowledge to novel tasks. FSL can be mainly categorized into 3 approaches:
optimization-based, metric-based, and generation-based methods.

Optimization-based approaches are designed to generalize to novel tasks by a
few steps of updates. MAML [14] employs a second-order optimizing strategy
with the meta-learning framework by carrying out a small number of update
steps. Then Nichol $et~{}al.$ [15] simplified the implementation of MAML only
with the first-order derivation and presented Reptile to move meta-learner
parameters towards base-learner parameters. Follow this line, many variants
[16, 17, 18, 19] have made incremental changes for MAML. For example, LEO [16]
applies a low-dimensional latent space to update parameters, which is more
practical in low-data scenarios. Since the shared initialization may lead to
conflict over tasks, Baik $et\ al$. [17] proposed a task-and-layer-wise
attenuation to forget the prior information selectively.

Metric-based approaches aim at constructing an embedding space to measure
similarities between the labeled and unlabeled images. This line of approaches
mainly depends on designing a suitable embedding space and a distance
measurement method, which is simple and effective. Matching Network [1]
combines attention mechanism with LSTM to obtain the probability distribution
belonging to the categories in the support set. Besides, it also proposes the
episode training paradigm, which is widely employed in FSL. Relation Network
[3] employs deep convolution networks to replace the artificial distance
function, which achieves promising performance. Snell $et~{}al.$ [2] assumed
that there are class prototypes in the embedding space for each class. They
designed Prototypical Network, which averages the samples in the support set
as class prototypes. The above three approaches have established the
foundation of metric-based FSL. Many studies conduct incremental improvements
based on them, such as [20, 21, 22, 23, 24].

The main idea of the generation-based FSL methods improves the
generalizability of the classifier by producing diverse samples. Generative
Adversarial Network (GAN) is a typical generator in FSL. For example, Wang
$et\ al$. [25] applied GAN to generate samples, which improves performance by
about $6\%$. And Li $et\ al$. [26] employed cWGAN to ensure the diversity of
generated data. Besides, some studies employ auxiliary approaches to directly
generate image-level samples. For example, Zhang $et\ al$. [27] introduced
saliency-based masks to exchange foregrounds and backgrounds of different
images to augment samples.

###  II-B Cross-Domain Few-Shot Learning

Facing realistic FSL application, it is almost impossible that a large number
of labeled samples and unlabeled test samples come from the same domain. It
usually occurs that there are abundant auxiliary samples in related domains.
Therefore, Cross-Domain Few-Shot Learning (CD-FSL) is proposed, where labeled
and unlabeled samples belong to the source domain and the target domain
respectively. The domain shift problem caused by different distributions is a
challenge for CD-FSL.

A typical method to address the domain shift problem is aligning the two
distributions by adversarial training [6], domain alignment [7] and knowledge
distillation [8]. Besides, Chen $et\ al$. [28] conducted experiments employing
several popular vanilla FSL algorithms on the setting of CD-FSL, which
demonstrates that the pre-trained and fine-tuned approaches could improve the
performance on the unseen domain. Although these approaches achieve promising
results, they applied labeled or unlabeled samples from the target domain
before testing. Therefore, they break the CD-FSL setting that the target
domain is absolutely unknown during the training.

By contrast, some work explores to improve the knowledge transferability
without access to the target domain. For example, FT [9] employs the affine
transformation to simulate various feature distributions under different
domains. Since this transformation is not directed to the target domain, it
performs unstably on the different target domains. LRP [29] applies the model-
agnostic explanation-guided training strategy to improve the feature
representations. ATA [30] considers the task distribution space under the
worst-case problem by adversarial task augmentation, which strengthens the
robustness of the inductive bias. Different from them, we focus on how to
employ limited labeled samples to construct metric spaces pointing to the
target source. We propose a task-expansion-decomposition framework for CD-FSL,
which obtains optimal parameters by target-oriented inner tasks.

###  II-C Weakly Supervised Object Localization

Weakly Supervised Object Localization (WSOL) obtains the localization of
objects only with the image-level labels. Most WSOL methods extract activated
regions of feature maps, which is called Class Activation Map (CAM) [13]. It
employs a global average pooling (GAP) layer to preserve the spatial
information of the feature maps and extracts the spatial region of the feature
responses from the last convolutional feature map. Later, Grad-CAM [31] is
proposed to simplify CAM, which extracts the heatmap from any convolutional
feature map.

Although CAM-based methods are simple and effective, they probably focus on
the broad regions of objects without the marginal parts. Thus, an erasing-
based strategy is designed to preserve more relevant parts of objects.
According to the erasing regions, they are categorized into two types: random
erasing [32, 33] and network-guided erasing methods [34]. Apart from the above
erasing methods, SPG [35] and I2C [36] apply the constraint of pixel-level
correlations to improve the quality of localization maps.

In recent years, weakly supervised object localization techniques have been
employed in video understanding [10], medical imaging [11] and remote sensing
imagery analysis [12]. In our work, we apply weakly supervised object
localization as prior knowledge to generate task-oriented samples and focus on
the discriminative regions, which provides task transferring space in cross-
domain few-shot learning.

##  III Self-Taught Approach

###  III-A Problem Formulation

We follow the common episodic paradigm in [1, 2, 3], which is trained on the
$N$-way $K$-shot setting. There are $K$ labeled samples and a number of
unlabeled samples from $N$ categories composing of a few-shot task. The
support set consists of $N\times K$ labeled samples, denoted as
$S=\\{x_{i}^{s},y_{i}^{s}\\}_{i=1}^{N\times K}$. And the query set is
represented as $Q=\\{x_{i}^{q},y_{i}^{q}\\}_{i=1}^{M}$, where $M$ is the total
number of unlabeled samples. In vanilla FSL, the train set and test set belong
to the same domain. While in CD-FSL, the train set and test set are sampled
from different domains.

Algorithm 1 The training algorithm of the Self-Taught approach on $N$-way
$K$-shot.

0: The train set $\bm{D_{train}}$, the task settings including the number of
category $K$, the number of support examples per class $N$ and the total
number of query examples $M$.

0: Learned model.

1: Randomly initialize parameters

2: while training do

3: $\bm{T}=TaskSample(\bm{D_{train}},N,K,M)$

4: for $t$ in $\left\\{\bm{T}\right\\}$ do

5:
$\\{(x_{i}^{s},y_{i}^{s})\\}^{NK}\in\bm{S},\\{(x_{i}^{q},y_{i}^{q})\\}^{M}\in\bm{Q}$

6: $Loc_{i}=WSOL(x_{i}^{s})$

7: $Back_{i}=x_{i}^{s}-Loc_{i}$

8: $\hat{x_{i}^{s}},\hat{y_{i}^{\omega}}=rot(Loc_{i})\oplus Back_{j}$

9: $\\{\hat{x_{i}^{s}},y_{i}^{s},\hat{y_{i}^{\omega}}\\}\in\bm{S^{\prime}}$

10: $\bm{T^{inner}}=TaskSample(\bm{S^{\prime}},N,K,M^{\prime})$

11: for $t^{inner}$ in $\left\\{\bm{T^{inner}}\right\\}$ do

12:
$\\{(x_{i}^{\widetilde{s}},y_{i}^{\widetilde{s}},y_{i}^{\widetilde{s}\omega})\\}^{NK}\in\bm{\widetilde{S}},\\{(x_{i}^{\widetilde{q}},y_{i}^{\widetilde{q}})\\}^{M^{\prime}}\in\bm{\widetilde{Q}}$

13: $y_{i}^{\omega}=E(f_{\phi}(x_{i}^{\widetilde{s}}))$

14: $c_{k}={\dfrac{1}{N}\sum\limits f_{\phi}(\hat{x_{i}^{s}})}$

15: for $(x_{i}^{\widetilde{q}},y_{i}^{\widetilde{q}})$ in
$\bm{\widetilde{Q}}$ do

16: $Loss(t^{inner}|\theta^{\prime})=-\dfrac{1}{NK}\sum\limits
y_{i}^{\widetilde{s}\omega}log(y_{i}^{\omega})+\dfrac{1}{M^{\prime}}[log\sum\limits_{w^{\prime}}exp(-d(f_{\phi}(x_{i}^{\widetilde{s}}),c_{k^{\prime}}))+d(f_{\phi}(x_{i}^{\widetilde{s}}),c_{k})]$

17: end for

18: end for

19: ${c_{k}}={\dfrac{1}{N}\sum\limits f_{\mu}(x_{i}^{s})}$

20: for $(x_{i}^{q},y_{i}^{q})$ in $\bm{Q}$ do

21:
$Loss(t|\theta)=\dfrac{1}{M}[log\sum\limits_{w^{\prime}}exp(-d(f_{\mu}(x_{i}^{q}),c_{k^{\prime}}))+d(f_{\mu}(x_{i}^{q}),c_{k})]$

22: end for

23: end for

24: end while

###  III-B Overview

In this work, we propose a Self-Taught (ST) approach for Cross-Domain Few-Shot
Learning (CD-FSL) employing Weakly Supervised Object Localization (WSOL) and
task-decomposition, which is shown in Fig. 3. Our ST approach consists of a
Task-Expansion Module (TE-Module) and a Task-Decomposition Module (TD-Module).
The TE-Module applies WSOL and self-supervised technologies to generate novel
samples, which enables networks focus on discriminative regions and further
provides task-oriented spaces for task-decomposition. Then the TD-Module
samples few-shot tasks from novel labeled set to update network parameters.
This task-expansion-decomposition paradigm learns to adjust parameters
according to novel tasks from different domains.

The training procedure is shown in Alg. 1. There are 2 types of tasks in our
proposed ST approach. The support set inputs inner tasks to update the inner
network, and the traditional few-shot task is trained as the outer task. There
are the same metric-based few-shot methods in the outer task and inner task
but in different parameter update ways. Our ST approach can work on metric-
based few-shot methods, i.e., MatchingNet [1], ProtoNet [2], RelationNet [3].
Although graph-based methods are superior in performance, they can not deal
with the varying number of query samples by the same network. Therefore, our
ST approach is mainly designed on the metric-based approaches. We introduce
our method by taking ProtoNet as an example, where $f_{\phi}$ and $f_{\mu}$
are the inner network and outer network with the same architecture and initial
parameters.

###  III-C Task-Expansion Module

The Task-Expansion Module (TE-Module) employs WSOL and self-supervised
technologies to generate novel labeled samples and learn more discriminative
features. Different from other self-supervised approaches, we apply WSOL to
obtain the attentive locations of images, which utilizes the focus region to
exchange and rotate:

$Loc_{i}=WSOL(x_{i}^{s}),$ (1)

where $x_{i}^{s}$ is the $i$-th sample in support set and $Loc_{i}$ is the
object region of $x_{i}^{s}$. We apply a simple WSOL approach [13] to obtain
the location of objects, which is shown in Fig. 4. Firstly, feature maps are
extracted from the images in the support set by the feature extractor in
metric-based methods, which can be denoted as:

$p_{i}=f_{\phi}(x_{i}^{s}),$ (2)

where $f_{\phi}$ is the CNN encoder of ProtoNet in the inner task, and $p_{i}$
is the feature map of $x_{i}^{s}$. Then we employ Class Activation Map (CAM)
[13] to highlight the discriminative regions, which is represented as:

$h_{i}=CAM(p_{i}),$ (3)

where $h_{i}$ is the heatmap of $p_{i}$. To obtain bounding boxes from
$h_{i}$, we employ an auto-thresholding technique to segment the heatmap. We
turn the heatmap to a binary image by the threshold. Besides, we take the
square bounding box that covers the largest connected component in the binary
image:

$Loc_{i}=Seg(h_{i}).$ (4)

Accroding to the object location, the image can be divided into two parts:

$Back_{i}=x_{i}^{s}-Loc_{i},$ (5)

where $Back_{i}$ is the background of $x_{i}^{s}$. Our ST approach requires
generating novel task-oriented samples for task-decomposition. Thus, we design
2 types of generating methods: 1) Rotating the object localization. Different
from the rotation of the whole image, the network can focus on the object
location, especially for small object boxes. 2) Exchanging and randomly
rotating foregrounds and backgrounds. Since there are similar backgrounds in
fine-grained datasets, the exchanges of inter-class and intra-class provide
diverse task-oriented samples. They are defined as:

$\hat{x_{i}^{s}},\hat{y_{i}^{\omega}}=\begin{cases}rot(Loc_{i})\oplus
Back_{i},&i=j\\\ rot(Loc_{i})\oplus Back_{j},&i\neq j\end{cases}$ (6)

where $rot(\ )$ rotates the $Loc_{i}$ by the random angle from
$[0\degree,90\degree,180\degree,270\degree]$, $\hat{y_{i}^{\omega}}$ is the
angle label from $[0,1,2,3]$, $\oplus$ represents the image stitching and
random convolutions, $\hat{x_{i}^{s}}$ is the novel sample. Random
convolutions aim at smoothing images and adding noise. Thus, the expanded
support set $S^{\prime}=\\{\hat{x_{i}^{s}},y_{i}^{s},\hat{y_{i}^{\omega}}\\}$
contains more samples.

Figure 4: An illustration of the WSOL method. It extracts feature maps and
employs CAM to obtain heatmaps. Then the auto-thresholding technique is
employed to binarize the heatmap and the max bounding box is found to cover
the object regions.

###  III-D Task-Decomposition Module

Task-Decomposition Module (TD-Module) samples $\alpha$ iterations of $N$-way
$K$-shot task from $S^{\prime}$, including the support set
$\widetilde{S}=\\{(x_{i}^{\widetilde{s}},y_{i}^{\widetilde{s}},y_{i}^{\widetilde{s}\omega})\\}^{NK}$
and the query set
$\widetilde{Q}=\\{(x_{i}^{\widetilde{q}},y_{i}^{\widetilde{q}})\\}^{M^{\prime}}$.

The inner task is a multi-task framework, which includes the rotation
classification and few-shot recognition. For the rotation classification, a
nonlinear network $E$ embeds features to its rotation angle label as:

$y_{i}^{\omega}=E(f_{\phi}(x_{i}^{\widetilde{s}})),$ (7)

where $y_{i}^{\omega}$ is the predicted rotation label. We apply a cross-
entropy loss to measure the rotation angle error, which is defined as:

$Loss_{rot}=-\dfrac{1}{NK}\sum\limits
y_{i}^{\widetilde{s}\omega}log(y_{i}^{\omega}).$ (8)

For the inner few-shot recognition, the class prototypes are calculated by:

$c_{k}={\dfrac{1}{K}\sum\limits f_{\phi}(\hat{x_{i}^{s}})},$ (9)

where $c_{k}$ is the class prototypes of the $k$-th category. The encoder
$f_{\phi}$ gets as input an sample $x_{i}^{s}$ with its few-shot label
$y_{i}^{s}$, and yields as output a probability distribution over all possible
classification scores:

${p(y=j|x_{i}^{\widetilde{q}})}=\dfrac{exp(-d(x_{i}^{\widetilde{q}},c_{j}))}{\sum_{K}exp(-d(x_{i}^{\widetilde{q}},c_{k}))},$
(10)

where $d(\ )$ is Euclidean Distance. The few-shot classification loss is:

$Loss_{TD}=\dfrac{1}{M^{\prime}}[log\sum\limits_{w^{\prime}}exp(-d(f_{\phi}(x_{i}^{\widetilde{s}}),c_{k^{\prime}}))+d(f_{\phi}(x_{i}^{\widetilde{s}}),c_{k})].$
(11)

The inner loss consists of rotation angle prediction loss and few-shot
classification loss, which is denoted as:

$Loss(t^{inner}|\theta^{\prime})=Loss_{TD}+\lambda Loss_{rot},$ (12)

where $\lambda$ is a hyper-paremeter to balance the two tasks, and
$\theta^{\prime}$ is parameters of the inner network.

After $\alpha$ iterations of inner tasks, the outer network $f_{\mu}$ is
updated on $\theta^{\prime}$. Similarly, the class prototypes and the
classification scores are obtained by:

${c_{k}}={\dfrac{1}{k}\sum\limits f_{\mu}(x_{i}^{s})},$ (13)

${p(y=j|x_{i}^{q})}=\dfrac{exp(-d(x_{i}^{q},c_{j}))}{\sum_{K}exp(-d(x_{i}^{q},c_{k}))}.$
(14)

The outer network $f_{\mu}$ applies a cross-entropy loss to measure the
classification error, which is denoted as:

$Loss_{cls}=\dfrac{1}{M}[log\sum\limits_{w^{\prime}}exp(-d(f_{\mu}(x_{i}^{q}),c_{k^{\prime}}))+d(f_{\mu}(x_{i}^{q}),c_{k})].$
(15)

##  IV Experimental Setup

We conduct experiments to prove the effectiveness of our proposed ST approach
on three popular metric-based methods: MatchingNet [1], ProtoNet [2], and
RelationNet [3]. Under cross-domain settings, the source domain is
miniImageNet [37], and the target domains contain 8 fine-grained datasets.

TABLE I: Summarization of the datasets. Dataset | Train Set | Validation Set | Test Set
---|---|---|---
miniImageNet | 64 | 16 | 20
CUB | 100 | 50 | 50
Cars | 98 | 49 | 49
Places | 183 | 91 | 91
Plantae | 100 | 50 | 50
CropDieases | 38 | - | 38
EuroSAT | 10 | - | 10
ISIC | 7 | - | 7
ChestX | 14 | - | 14
TABLE II: Cross-domain few-shot classification accuracy on test splits of CUB,
Cars, Places and Plantae datasets with $\pm$ 95% confidence intervals.

| CUB | Cars | Places | Plantea
---|---|---|---|---
| 5-way 1-shot | 5-way 5-shot | 5-way 1-shot | 5-way 5-shot | 5-way 1-shot | 5-way 5-shot | 5-way 1-shot | 5-way 5-shot
MatchingNet [1] | 41.46% $\pm$ 0.4% | 61.21% $\pm$ 0.4% | 31.82% $\pm$ 0.3% | 48.11% $\pm$ 0.4% | 48.83% $\pm$ 0.5% | 66.29% $\pm$ 0.4% | 36.56% $\pm$ 0.3% | 53.81% $\pm$ 0.3%
+FT [9] | 43.59% $\pm$ 0.4% | 61.65% $\pm$ 0.4% | 35.03% $\pm$ 0.3% | 47.72% $\pm$ 0.4% | 53.00% $\pm$ 0.5% | 66.01% $\pm$ 0.4% | 36.28% $\pm$ 0.3% | 51.12% $\pm$ 0.3%
+ATA [30] | 41.59% $\pm$ 0.4% | 59.33% $\pm$ 0.4% | 35.14% $\pm$ 0.3% | 48.78% $\pm$ 0.4% | 51.86% $\pm$ 0.5% | 66.31% $\pm$ 0.4% | 37.02% $\pm$ 0.3% | 51.56% $\pm$ 0.3%
+ST (Ours) | 46.22% $\pm$ 0.4% | 73.87% $\pm$ 0.4% | 35.29% $\pm$ 0.3% | 60.29% $\pm$ 0.4% | 52.75% $\pm$ 0.5% | 69.56% $\pm$ 0.4% | 37.25% $\pm$ 0.3% | 61.30% $\pm$ 0.3%
ProtoNet [2] | 41.42% $\pm$ 0.4% | 62.69% $\pm$ 0.4% | 31.26% $\pm$ 0.3% | 48.35% $\pm$ 0.4% | 48.44% $\pm$ 0.5% | 70.47% $\pm$ 0.4% | 32.17% $\pm$ 0.3% | 53.13% $\pm$ 0.3%
+FT [9] | 41.74% $\pm$ 0.4% | 63.84% $\pm$ 0.4% | 31.47% $\pm$ 0.3% | 48.98% $\pm$ 0.3% | 48.41% $\pm$ 0.5% | 69.20% $\pm$ 0.4% | 33.02% $\pm$ 0.3% | 52.31% $\pm$ 0.3%
+ATA [30] | 39.85% $\pm$ 0.4% | 64.55% $\pm$ 0.4% | 32.21% $\pm$ 0.3% | 50.44% $\pm$ 0.4% | 49.15% $\pm$ 0.5% | 69.94% $\pm$ 0.4% | 31.72% $\pm$ 0.3% | 53.14% $\pm$ 0.3%
+ST (Ours) | 43.97% $\pm$ 0.4% | 74.62% $\pm$ 0.4% | 33.74% $\pm$ 0.3% | 59.38% $\pm$ 0.4% | 50.31% $\pm$ 0.5% | 73.71% $\pm$ 0.4% | 33.96% $\pm$ 0.3% | 62.72% $\pm$ 0.3%
RelationNet [3] | 41.27% $\pm$ 0.4% | 56.77% $\pm$ 0.4% | 30.09% $\pm$ 0.3% | 40.46% $\pm$ 0.4% | 48.16% $\pm$ 0.5% | 64.25% $\pm$ 0.4% | 31.23% $\pm$ 0.3% | 42.71% $\pm$ 0.3%
+FT [9] | 43.33% $\pm$ 0.4% | 59.77% $\pm$ 0.4% | 30.45% $\pm$ 0.3% | 40.18% $\pm$ 0.4% | 49.92% $\pm$ 0.5% | 65.55% $\pm$ 0.4% | 32.57% $\pm$ 0.3% | 44.29% $\pm$ 0.3%
+LRP [29] | 41.57% $\pm$ 0.4% | 57.70% $\pm$ 0.4% | 30.48% $\pm$ 0.3% | 41.21% $\pm$ 0.4% | 48.47% $\pm$ 0.5% | 65.35% $\pm$ 0.4% | 32.11% $\pm$ 0.3% | 43.70% $\pm$ 0.3%
+ATA [30] | 43.02% $\pm$ 0.4% | 59.36% $\pm$ 0.4% | 31.79% $\pm$ 0.3% | 42.95% $\pm$ 0.4% | 51.16% $\pm$ 0.5% | 66.90% $\pm$ 0.4% | 33.72% $\pm$ 0.3% | 45.32% $\pm$ 0.3%
+ST (Ours) | 43.10% $\pm$ 0.4% | 62.94% $\pm$ 0.4% | 32.34% $\pm$ 0.3% | 43.26% $\pm$ 0.4% | 50.53% $\pm$ 0.5% | 66.74% $\pm$ 0.4% | 33.19% $\pm$ 0.3% | 46.92% $\pm$ 0.3%

TABLE III: Cross-domain few-shot classification accuracy on test splits of
CropDieases, EuroSAT, ISIC and ChestX datasets with $\pm$ 95% confidence
intervals.

| CropDieases | EuroSAT | ISIC | ChestX
---|---|---|---|---
| 5-way 1-shot | 5-way 5-shot | 5-way 1-shot | 5-way 5-shot | 5-way 1-shot | 5-way 5-shot | 5-way 1-shot | 5-way 5-shot
MatchingNet [1] | 61.49% $\pm$ 0.4% | 85.12% $\pm$ 0.4% | 58.30% $\pm$ 0.3% | 77.54% $\pm$ 0.4% | 32.79% $\pm$ 0.3% | 45.12% $\pm$ 0.3% | 22.25% $\pm$ 0.2% | 25.52% $\pm$ 0.2%
+FT [9] | 64.95% $\pm$ 0.5% | 83.39% $\pm$ 0.4% | 57.56% $\pm$ 0.3% | 72.49% $\pm$ 0.3% | 32.73% $\pm$ 0.3% | 43.44% $\pm$ 0.3% | 22.76% $\pm$ 0.2% | 25.13% $\pm$ 0.2%
+ATA [30] | 66.09% $\pm$ 0.4% | 86.36% $\pm$ 0.4% | 59.65% $\pm$ 0.3% | 76.96% $\pm$ 0.4% | 33.34% $\pm$ 0.3% | 42.53% $\pm$ 0.3% | 22.74% $\pm$ 0.2% | 25.11% $\pm$ 0.2%
+ST (Ours) | 66.65% $\pm$ 0.4% | 90.86% $\pm$ 0.4% | 64.96% $\pm$ 0.3% | 86.83% $\pm$ 0.4% | 34.12% $\pm$ 0.3% | 59.99% $\pm$ 0.3% | 22.85% $\pm$ 0.2% | 27.01% $\pm$ 0.2%
ProtoNet [2] | 60.23% $\pm$ 0.4% | 86.83% $\pm$ 0.4% | 56.41% $\pm$ 0.3% | 78.67% $\pm$ 0.4% | 31.21% $\pm$ 0.3% | 45.89% $\pm$ 0.3% | 22.10% $\pm$ 0.2% | 26.24% $\pm$ 0.2%
+FT [9] | 60.68% $\pm$ 0.4% | 85.95% $\pm$ 0.4% | 56.02% $\pm$ 0.3% | 78.49% $\pm$ 0.4% | 29.80% $\pm$ 0.3% | 44.91% $\pm$ 0.3% | 21.90% $\pm$ 0.2% | 26.14% $\pm$ 0.2%
+ATA [30] | 61.25% $\pm$ 0.4% | 88.29% $\pm$ 0.4% | 55.45% $\pm$ 0.3% | 77.93% $\pm$ 0.4% | 30.85% $\pm$ 0.3% | 45.32% $\pm$ 0.3% | 22.26% $\pm$ 0.2% | 26.09% $\pm$ 0.2%
+ST (Ours) | 63.53% $\pm$ 0.4% | 92.40% $\pm$ 0.4% | 62.62% $\pm$ 0.3% | 88.35% $\pm$ 0.4% | 31.92% $\pm$ 0.2% | 61.55% $\pm$ 0.3% | 22.46% $\pm$ 0.2% | 28.40% $\pm$ 0.2%
RelationNet [3] | 53.58% $\pm$ 0.4% | 72.86% $\pm$ 0.4% | 49.08% $\pm$ 0.4% | 65.56% $\pm$ 0.4% | 30.53% $\pm$ 0.3% | 38.60% $\pm$ 0.3% | 21.95% $\pm$ 0.2% | 24.07% $\pm$ 0.2%
+FT [9] | 57.57% $\pm$ 0.5% | 75.78% $\pm$ 0.4% | 53.53% $\pm$ 0.4% | 69.13% $\pm$ 0.4% | 30.38% $\pm$ 0.3% | 38.68% $\pm$ 0.3% | 21.79% $\pm$ 0.2% | 23.95% $\pm$ 0.2%
+LRP [29] | 55.01% $\pm$ 0.4% | 74.21% $\pm$ 0.4% | 50.99% $\pm$ 0.4% | 67.54% $\pm$ 0.4% | 31.16% $\pm$ 0.3% | 39.97% $\pm$ 0.3% | 22.11% $\pm$ 0.2% | 24.28% $\pm$ 0.2%
+ATA [30] | 61.17% $\pm$ 0.4% | 78.20% $\pm$ 0.4% | 55.69% $\pm$ 0.4% | 71.02% $\pm$ 0.4% | 31.13% $\pm$ 0.3% | 40.38% $\pm$ 0.3% | 22.14% $\pm$ 0.2% | 24.43% $\pm$ 0.2%
+ST (Ours) | 63.29% $\pm$ 0.4% | 78.62% $\pm$ 0.4% | 57.36% $\pm$ 0.30% | 75.84% $\pm$ 0.4% | 32.09% $\pm$ 0.3% | 44.42% $\pm$ 0.3% | 22.28% $\pm$ 0.2% | 24.79% $\pm$ 0.2%

###  IV-A Datasets

There are 9 datasets involving in our experiment, including 1 source domain
and 8 target domains. Their information is shown in Table I.

miniImageNet. miniImageNet is proposed by [37], which is a subset of ImageNet
[38]. It contains 100 classes with 600 images per class, which covers animals,
food, tools, furniture and etc. Due to its universality, it is usually
employed as the source domain in CD-FSL. In our work, we follow the split in
[37], in which we apply train and validation set for training.

CUB. CUB-200 (CUB) [39] is a benchmark dataset for fine-grained bird
recognition, including 200 classes with a total of 11,788 images. We take its
test set as one of the target domains, following the split in [40].

Cars. Stanford Cars (Cars) [41] is proposed for fine-grained car
classification. It contains 16,185 images of 196 classes of cars, whose labels
are typically at the level of Make, Model, and Year.

Places. Places365-Standard (Places) [42] is a benchmark dataset for the scene
understanding task, which totally contains 1,803,460 images within 365 scene
categories. The number of images per class varies from 3,068 to 5,000.

Plantae. This dataset is extracted from [43], which contains 200 species of
plants. Since the natural world is heavily imbalanced, the number of each
class is different but reaches at least 100 samples.

CropDieases. This dataset [44] is a public dataset of 54,306 diseased and
healthy plant samples, labeling 14 crop species and 26 diseases. The version
in CD-FSL is reorganized with the format “crop species_disease”, which
contains 38 classes.

EuroSAT. It is a novel dataset based on Sentinel-2 satellite images [45],
which covers 13 spectral bands with a total of 27,000 labeled and geo-
referenced images. And it contains 10 classes, including annual crop,
permanent crop, pastures, highway, residential buildings, industrial
buildings, and etc.

ISIC. ISIC2018 (ISIC) [46] consists of 10,015 dermatoscopic images and is
publicly available through the ISIC archive. This benchmark dataset is
employed for comparisons with machine learning and human experts.

ChestX. ChestX-ray8 (ChestX) [47] is a new chest X-ray database, which
comprises 108,948 frontal view X-ray images of 32,717 unique patients with 14
types of disease pattern.

Note that the splits of CropDieases, EuroSAT, ISIC, and ChestX we process are
exactly the same as [30, 48].

###  IV-B Experimental Settings

As for the visual representations, we apply the pretrained ResNet-$10$ [49] to
extract the visual features. The hyperparameter $\lambda$ is set to $0.1$. For
the outer task, we employ Adam optimizer with initial learning rate with
$0.0001$. And for the inner task, we apply SGD optimizer with learning rate
with $0.001$. $\lambda$ of the inner loss is 0.1. We utilize the most popular
settings in few-shot learning: $5$-way $1$-shot and $5$-way $5$-shot settings.
The number of query set is 15 per class.

###  IV-C Competitors

Several FSL models are chosen as competitors:

(1) Metric-based approaches, including Matching Networks [1], Prototypical
Networks [2], and Relation Networks [3].

(2) Cross-domain few-shot approaches, including FT [9], ATA [30] and LRP [29].
They can work on the above metric-based approaches. While LRP only implements
on RelationNet due to its requiring of classifier parameters.

To fairly compare with the approaches, we utilize ResNet-10 [49] as embedding
networks for all the methods. The recognition accuracies for CD-FSL are
computed by averaging over $2000$ randomly generated episodes.

TABLE IV: Ablation studies of the ST approach on CUB and Places datasets. WSOL | Whole |  | CUB | Places
---|---|---|---|---
Rot | Exc-Rot | Rot | TD | 5-way 1-shot | 5-way 5-shot | 5-way 1-shot | 5-way 5-shot
|  |  |  | 41.42% $\pm$ 0.4% | 62.69% $\pm$ 0.4% | 48.44% $\pm$ 0.5% | 70.47% $\pm$ 0.4%
$\checkmark$ |  |  |  | 42.78% $\pm$ 0.4% | 66.77% $\pm$ 0.4% | 49.72% $\pm$ 0.5% | 71.49% $\pm$ 0.4%
| $\checkmark$ |  |  | 42.96% $\pm$ 0.4% | 67.35% $\pm$ 0.4% | 49.67% $\pm$ 0.5% | 71.56% $\pm$ 0.4%
|  | $\checkmark$ |  | 42.76% $\pm$ 0.4% | 65.89% $\pm$ 0.4% | 49.69% $\pm$ 0.5% | 71.53% $\pm$ 0.4%
|  |  | $\checkmark$ | 43.16% $\pm$ 0.4% | 70.46% $\pm$ 0.4% | 50.07% $\pm$ 0.5% | 72.79% $\pm$ 0.4%
$\checkmark$ |  |  | $\checkmark$ | 43.34% $\pm$ 0.4% | 72.21% $\pm$ 0.4% | 50.19% $\pm$ 0.5% | 73.32% $\pm$ 0.4%
| $\checkmark$ |  | $\checkmark$ | 43.49% $\pm$ 0.4% | 73.65% $\pm$ 0.4% | 50.23% $\pm$ 0.5% | 73.39% $\pm$ 0.4%
|  | $\checkmark$ | $\checkmark$ | 43.33% $\pm$ 0.4% | 71.58% $\pm$ 0.4% | 50.20% $\pm$ 0.5% | 73.25% $\pm$ 0.4%
$\checkmark$ | $\checkmark$ |  | $\checkmark$ | 43.97% $\pm$ 0.4% | 74.62% $\pm$ 0.4% | 50.31% $\pm$ 0.5% | 73.71% $\pm$ 0.4%

###  IV-D Comparison with State-of-the-Art Methods

We conduct experiments on the most common setting in FSL, $5$-way $1$-shot and
$5$-way $5$-shot on all datasets. We apply the train and validation set of
miniImageNet for training and then test the model on eight target domains.

The results on CUB, Cars, Places, and Plantae are shown in Table II. It can be
observed that the proposed ST approach achieves significant improvements over
most approaches on both $5$-way $1$-shot and $5$-way $5$-shot settings.
Specifically, the proposed model achieves accuracies of $73.87\%$ (MatchingNet
+ ST) and $74.62\%$ (ProtoNet + ST) on CUB, $60.29\%$ (MatchingNet + ST) and
$59.38\%$ (ProtoNet + ST) on Cars under $5$-way $5$-shot setting,
outperforming other approaches in a large margin. We notice that ATA
demonstrates competitive performance on RelationNet, which obtains the
accuracy of $51.16\%$ and $69.90\%$ on Places under both settings. While the
performance of our ST approach slightly declines on Places dataset, we infer
that the WSOL loses its superior for scene classification. Since there is no
specific object location in the scene, it turns into a normal self-supervised
approach. We employ ablation studies to prove this inference, which is shown
in Section IV.E.

Comparing with the above datasets, Cropdieases, EuroSAT, ISIC, ChestX datasets
are more natural and realistic. The results are shown in Table III. It is
observed that the proposed ST approach clearly beats all competitive
approaches on all datasets. Besides, we observe that the most significant
gains appear on $5$-way $5$-shot by ProtoNet + ST. It achieves accuracy gains
of $5.56\%$, $9.68\%$, $15.66\%$, and $2.16\%$ respectively against ProtoNet
on the four datasets. Especially, the ChestX dataset is the hardest dataset
for CD-FSL due to the huge difference between the source domain (miniImageNet)
and the target domain, on which other approaches are almost ineffective. By
contrast, our ST approach obtains satisfactory performance of $27.01\%$
(MatchingNet + ST), $28.40\%$ (ProtoNet + ST), and $24.79\%$ (RelationNet +
ST) on ChestX. It demonstrates that our task-expansion-decomposition framework
is capable of narrowing domain-shift. Our ST approach employs weakly
supervised object localization and self-supervised approaches to generate
task-oriented samples and then implements inner tasks to learn a task-specific
metric space. Besides, it utilizes a way of exchanging and rotating to make
the network focus on discriminative regions and learn more robust visual
information. Therefore, ST approach has a clear improvement against the
competitors.

(a) The impacts of different iterations of the inner task on $5$-way $1$-shot.
(b) The impacts of different iterations of the inner task on $5$-way $5$-shot.

Figure 5: The influences of varied iterations of the inner task $\alpha$ on
all datasets.

###  IV-E Ablation Studies

In order to evaluate the effectiveness of each component, we conduct
experiments as ablation studies. We choose two datasets, i.e., CUB and Places,
according to that whether there exist concrete objects in the images.

The components are designed as follows:

WSOL Rot: Rotating the region obtained by WSOL to generate samples.

WSOL Exc-Rot: Exchanging and rotating the region obtained by WSOL to generate
samples.

Whole Rot: Rotating the whole images to generate samples.

TD: Task-decomposition.

Figure 6: Examples of weakly supervised object location results on (a) CUB,
(b) Cars and (c) Plantae datasets.  Figure 7: The t-SNE visualization of the
embeddings learned by ProtoNet, ProtoNet+ST with 2 iterations, ProtoNet+ST
with 6 iterations on CropDieases.

The experimental results of different components are shown in Table IV. We
have the following observations: 1) TD has significant impacts on results,
which brings about $1.74\%$ and $7.77\%$ performance gains on CUB and $1.63\%$
and $2.05\%$ performance gains on Places under $5$-way $1$-shot and $5$-way
$5$-shot settings respectively. 2) On CUB, WSOL Exc-Rot performs better than
the other two approaches (WSOL Rot and Whole Rot) at most $0.2\%$ and $2.07\%$
on $5$-way $1$-shot and $5$-way $5$-shot. 3) On Places, three generation
approaches, WSOL Exc-Rot, WSOL Rot, and Whole Rot have similar performance. It
proves that WSOL does not work well on the scene classification without
obvious objects. It also can be observed that any one generation approach + TD
is capable of bringing performance gains on both datasets. The both WSOL
approaches + TD achieve the best results, which makes up the proposed ST
approach.

###  IV-F The impact of iterations $\alpha$

The quantitative analysis for iterations of the inner task $\alpha$ is
illustrated in Fig. 5 on the eight target domains under both $5$-way $1$-shot
and $5$-way $5$-shot settings. The hyper-parameter $\alpha$ is applied for
adjusting the strength of task decomposition. We can observe that there are
similar trends on both settings. Specifically, on $5$-way $1$-shot, the
performance gradually grows with an earlier increase of $\alpha$, which
illustrates that the iterations of the inner task $\alpha$ make contributions
to cross-domain few-shot recognition. However, the peak performance of the
different target domains is obtained by different $\alpha$, probably from 4 to
6. Considering the trade-off between cost and performance, we choose
$\alpha=4$ as the iterations of the inner task on $5$-way $1$-shot. Then, the
performance begins to decrease slightly. We infer that excessive iterations
hurt the generalization performance for the next task. Similarly, for the
$5$-way $5$-shot setting, the optimal value of $\alpha$ is 6$\sim$8\. Due to
the same reason, we select $\alpha=6$.

###  IV-G Qualitative Results

####  IV-G1 WSOL on unseen classes

We visualize WSOL results on CUB, Cars, and Plantae datasets to evaluate the
impacts on unseen classes, which is shown in Fig. 6. It can be observed that
although the feature extractor does not have access to the target domains, it
still performs robustly. When the main object appears in the image, WSOL gives
the approximate location boxes on three datasets. Since the target domains are
all fine-grained datasets, they have similar backgrounds. We design the Exc-
Rot based on WSOL to generate task-oriented samples by exchanging backgrounds
and objects between inner-class and intra-class. Experimental results have
proved that WSOL plays an important role in our approach.

####  IV-G2 Visualization Analysis

To prove the effectiveness of our method, we visualize the embedding space on
the CropDieases dataset by the t-SNE approach, which is shown in Fig. 7. It
can be observed that the class prototypes of ProtoNet obtained by averaging
labeled samples deviate from some query samples, which extremely hurts the
classification performance for CD-FSL. We choose iterations of the inner task
$\alpha=2$ and $\alpha=6$ to verify our ST approach. Obviously, the embeddings
obtained from ProtoNet+ST with 2 iterations are more separable than ProtoNet.
Moreover, ProtoNet+ST with 6 iterations further improve the performance, where
the shapes of class clusters are similar and balanced. The t-SNE visualization
demonstrates our ST approach is capable of addressing the cross-domain few-
shot task.

##  V Conclusion

This paper has introduced a Self-Taught (ST) approach to alleviate the problem
of non-directed guidance for target tasks in cross-domain few-shot learning.
Our ST approach can nest in various metric-based approaches and is a task-
expansion-decomposition framework. It first employs weakly supervised object
localization and self-supervision technologies to generate fine-grained level
samples. Then they decompose into subtasks to finish the task of few-shot
recognition and rotation classification. It not only constructs task-oriented
metric spaces but also focuses on the discriminative regions. The experimental
results show the impressive effectiveness of our ST approach on eight target
domains.

## References

  * [1] O. Vinyals, C. Blundell, T. P. Lillicrap, K. Kavukcuoglu, and D. Wierstra, “Matching networks for one shot learning,” in Advances in Neural Information Processing Systems, pp. 3637–3645, 2016.
  * [2] J. Snell, K. Swersky, and R. S. Zemel, “Prototypical networks for few-shot learning,” in Advances in Neural Information Processing Systems, pp. 4077–4087, 2017.
  * [3] F. Sung, Y. Yang, L. Zhang, T. Xiang, P. H. S. Torr, and T. M. Hospedales, “Learning to compare: Relation network for few-shot learning,” in International Conference on Computer Vision and Pattern Recognition, pp. 1199–1208, 2018.
  * [4] A. Phaphuangwittayakul, Y. Guo, and F. Ying, “Fast adaptive meta-learning for few-shot image generation,” IEEE Transactions on Multimedia, Online, pp. 1–14, 2021.
  * [5] H. Huang, J. Zhang, J. Zhang, J. Xu, and Q. Wu, “Low-rank pairwise alignment bilinear network for few-shot fine-grained image classification,” IEEE Transactions on Multimedia, vol. 23, no. 2, pp. 1666–1680, 2021.
  * [6] Z. Luo, Y. Zou, J. Hoffman, and L. Fei-Fei, “Label efficient learning of transferable representations across domains and tasks,” in Advances in Neural Information Processing Systems, pp. 165–177, 2017.
  * [7] L. Jiang, C. Zhong, K. Wu, Z. Gang, and C. Zhang, “Boosting few-shot image recognition via domain alignment prototypical networks,” in International Conference on Tools with Artificial Intelligence, pp. 260–264, 2018.
  * [8] A. Islam, C. F. Chen, R. Panda, L. Karlinsky, and R. J. Radke, “Dynamic distillation network for cross-domain few-shot recognition with unlabeled data,” arXiv preprint arXiv:2106.07807, 2021.
  * [9] H. Y. Tseng, H. Y. Lee, J. B. Huang, and M. H. Yang, “Cross-domain few-shot classification via learned feature-wise transformation,” in International Conference on Learning Representations, pp. 1–14, 2020.
  * [10] D. Zhang, J. Han, L. Yang, and D. Xu, “Spftn: A joint learning framework for localizing and segmenting objects in weakly labeled videos,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 42, no. 2, pp. 475–489, 2020.
  * [11] M. Liu, J. Zhang, C. Lian, and D. Shen, “Weakly supervised deep learning for brain disease prognosis using mri and incomplete clinical scores,” IEEE Transactions on Cybernetics, vol. 50, no. 7, pp. 3381–3392, 2020.
  * [12] D. Zhang, J. Han, G. Cheng, Z. Liu, S. Bu, and L. Guo, “Weakly supervised learning for target detection in remote sensing images,” IEEE Geoscience and Remote Sensing Letters, vol. 12, no. 4, pp. 701–705, 2015.
  * [13] B. Zhou, A. Khosla, A. Lapedriza, A. Oliva, and A. Torralba, “Learning deep features for discriminative localization,” in International Conference on Computer Vision and Pattern Recognition, pp. 2921–2929, 2016.
  * [14] C. Finn, P. Abbeel, and S. Levine, “Model-agnostic meta-learning for fast adaptation of deep networks,” in International Conference on Machine Learning, pp. 1126–1135, 2017.
  * [15] A. Nichol and J. Schulman, “Reptile: a scalable metalearning algorithm,” arXiv preprint arXiv:1803.02999, 2018.
  * [16] A. A. Rusu, D. Rao, J. Sygnowski, O. Vinyals, and R. Hadsell, “Meta-learning with latent embedding optimization,” in International Conference on Learning Representations, pp. 1–17, 2019.
  * [17] S. Baik, S. Hong, and K. M. Lee, “Learning to forget for meta-learning,” in IEEE Conference on Computer Vision and Pattern Recognition, pp. 2376–2384, 2020.
  * [18] E. Grant, C. Finn, S. Levine, T. Darrell, and T. Griffiths, “Recasting gradient-based meta-learning as hierarchical bayes,” in International Conference on Learning Representations, pp. 1–13, 2018.
  * [19] M. A. Jamal and G.-J. Qi, “Task agnostic meta-learning for few-shot learning,” in IEEE Conference on Computer Vision and Pattern Recognition, pp. 11711–11719, 2019.
  * [20] B. N. Oreshkin, P. R. Lopez, and A. Lacoste, “TADAM: Task dependent adaptive metric for improved few-shot learning,” in Advances in Neural Information Processing Systems, pp. 721–731, 2018.
  * [21] Z. Ji, X. Liu, Y. Pang, W. Ouyang, and X. Li, “Few-shot human-object interaction recognition with semantic-guided attentive prototypes network,” IEEE Transactions on Image Processing, vol. 30, no. 1, pp. 1648–1661, 2021\.
  * [22] Q. Cai, Y. Pan, T. Yao, C. Yan, and T. Mei, “Memory matching networks for one-shot image recognition,” in IEEE Conference on Computer Vision and Pattern Recognition, pp. 4080–4088, 2018.
  * [23] W. Li, L. Wang, J. Xu, J. Huo, Y. Gao, and J. Luo, “Revisiting local descriptor based image-to-class measure for few-shot learning,” in IEEE Conference on Computer Vision and Pattern Recognition, pp. 7253–7260, 2019.
  * [24] Y. Zhu, W. Min, and S. Jiang, “Attribute-guided feature learning for few-shot image recognition,” IEEE Transactions on Multimedia, vol. 23, no. 1, pp. 1200–1209, 2021.
  * [25] Y.-X. Wang, R. Girshick, M. Hebert, and B. Hariharan, “Low-shot learning from imaginary data,” in IEEE Conference on Computer Vision and Pattern Recognition, pp. 7278–7286, 2018.
  * [26] K. Li, Y. Zhang, K. Li, and Y. Fu, “Adversarial feature hallucination networks for few-shot learning,” in IEEE Conference on Computer Vision and Pattern Recognition, pp. 13467–13476, 2020.
  * [27] H. Zhang, J. Zhang, and P. Koniusz, “Few-shot learning via saliency-guided hallucination of samples,” in IEEE Conference on Computer Vision and Pattern Recognition, pp. 2765–2774, 2019.
  * [28] W. Chen, Y. Liu, Z. Kira, Y. F. Wang, and J. Huang, “A closer look at few-shot classification,” in International Conference on Learning Representations, pp. 1–16, 2019.
  * [29] J. Sun, S. Lapuschkin, W. Samek, Y. Zhao, N.-M. Cheung, and A. Binder, “Explanation-guided training for cross-domain few-shot classification,” in International Conference of Pattern Recognition, pp. 7609–7616, 2020.
  * [30] H. Wang and Z. H. Deng, “Cross-domain few-shot classification via adversarial task augmentation,” in International Joint Conference on Artificial Intelligence, pp. 1–7, 2021.
  * [31] R. R. Selvaraju, M. Cogswell, A. Das, R. Vedantam, D. Parikh, and D. Batra, “Grad-cam: Visual explanations from deep networks via gradient-based localization,” International Journal of Computer Vision, vol. 128, no. 2, pp. 336–359, 2020.
  * [32] K. K. Singh and Y. J. Lee, “Hide-and-seek: Forcing a network to be meticulous for weakly-supervised object and action localization,” in International Conference on Computer Vision, pp. 3544–3553, 2017.
  * [33] S. Yun, D. Han, S. Chun, S. J. Oh, Y. Yoo, and J. Choe, “Cutmix: Regularization strategy to train strong classifiers with localizable features,” in International Conference on Computer Vision, pp. 6022–6031, 2019.
  * [34] J. Mai, M. Yang, and W. Luo, “Erasing integrated learning: A simple yet effective approach for weakly supervised object localization,” in IEEE Conference on Computer Vision and Pattern Recognition, pp. 8763–8772, 2020.
  * [35] X. Zhang, Y. Wei, G. Kang, Y. Yi, and T. Huang, “Self-produced guidance for weakly-supervised object localization,” in European Conference on Computer Vision, pp. 597–613, 2018.
  * [36] X. Zhang, Y. Wei, and Y. Yang, “Inter-image communication for weakly supervised localization,” in European Conference on Computer Vision, pp. 1–17, 2020.
  * [37] S. Ravi and H. Larochelle, “Optimization as a model for few-shot learning,” in International Conference on Learning Representations, pp. 1–11, 2017\.
  * [38] D. Jia, D. Wei, R. Socher, L. J. Li, L. Kai, and F. F. Li, “Imagenet: A large-scale hierarchical image database,” in International Conference on Computer Vision and Pattern Recognition, pp. 248–255, 2009.
  * [39] C. Wah, S. Branson, P. Welinder, P. Perona, and S. Belongie, “The caltech-ucsd birds-200-2011 dataset,” Technical Report CNS-TR-2011-001, California Institute of Technology, 2011.
  * [40] N. Hilliard, L. Phillips, S. Howland, A. Yankov, and N. O. Hodas, “Few-shot learning with metric-agnostic conditional embeddings,” arXiv preprint arXiv:1802.04376, 2018.
  * [41] J. Krause, M. Stark, J. Deng, and F. F. Li, “3D object representations for fine-grained categorization,” in IEEE International Conference on Computer Vision Workshops, pp. 554–561, 2014.
  * [42] B. Zhou, A. Lapedriza, A. Khosla, A. Oliva, and A. Torralba, “Places: A 10 million image database for scene recognition,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 40, no. 6, pp. 1452–1464, 2018\.
  * [43] G. Van Horn, O. Mac Aodha, Y. Song, Y. Cui, C. Sun, A. Shepard, H. Adam, P. Perona, and S. Belongie, “The inaturalist species classification and detection dataset,” in IEEE Conference on Computer Vision and Pattern Recognition, pp. 8769–8778, 2018.
  * [44] S. P. Mohanty, D. P. Hughes, and M. Salathe, “Using deep learning for image-based plant disease detection,” Frontiers in Plant Science, vol. 7, pp. 1–10, 2016.
  * [45] P. Helber, B. Bischke, A. Dengel, and D. Borth, “Eurosat: A novel dataset and deep learning benchmark for land use and land cover classification,” IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, vol. 12, no. 7, pp. 2217–2226, 2019.
  * [46] P. Tschandl, C. Rosendahl, and H. Kittler, “The ham10000 dataset, a large collection of multi-source dermatoscopic images of common pigmented skin lesions,” Scientific Data, vol. 5, no. 161, pp. 1–9, 2018.
  * [47] X. Wang, Y. Peng, L. Lu, Z. Lu, M. Bagheri, and R. M. Summers, “Chestx-ray8: Hospital-scale chest x-ray database and benchmarks on weakly-supervised classification and localization of common thorax diseases,” in IEEE Conference on Computer Vision and Pattern Recognition, pp. 3462–3471, 2017.
  * [48] Y. Guo, N. C. Codella, L. Karlinsky, J. V. Codella, and R. Feris, “A broader study of cross-domain few-shot learning,” in European Conference on Computer Vision, pp. 124–121, 2020.
  * [49] K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in International Conference on Computer Vision and Pattern Recognition, pp. 770–778, 2016.

|  Xiyao Liu received the B.S. degree in telecommunication engineering from
Tianjin University, Tianjin, China, in 2015. She is currently pursuing a Ph.D.
degree in the School of Electrical and Information Engineering, Tianjin
University. Her research interests include few-shot learning, human-object
interaction, and computer vision.
---|---
|  Zhong Ji received the Ph.D. degree in signal and information processing
from Tianjin University, Tianjin, China, in 2008. He is currently a Professor
with the School of Electrical and Information Engineering, Tianjin University.
He has authored over 100 scientific papers. His current research interests
include multimedia understanding, zero/few-shot leanring, cross-modal
analysis, and video summarization.
---|---
|  Yanwei Pang received the Ph.D. degree in electronic engineering from the
University of Science and Technology of China, Hefei, China, in 2004. He is
currently a Professor with the School of Electrical and Information
Engineering, Tianjin University, Tianjin, China. He has authored over 120
scientific papers. His current research interests include object detection and
recognition, vision in bad weather, and computer vision.
---|---
|  Zhongfei Zhang received the BS degree in electronics engineering (with
honors), the MS degree in information science, both from Zhejiang University,
Hangzhou, China, and the PhD degree in computer science from the University of
Massachusetts at Amherst. He is currently a professor at the Computer Science
Department and directs the Multimedia Research Laboratory in the State
University of New York (SUNY) at Binghamton, USA. He has published more than
200 peer-reviewed academic papers.
---|---