††thanks: Contributed equally to this work††thanks: Contributed equally to
this work††thanks: Authors to whom correspondence should be addressed:
<EMAIL_ADDRESS><EMAIL_ADDRESS>Authors to whom correspondence
should be addressed<EMAIL_ADDRESS><EMAIL_ADDRESS>
# Asymmetric Chiral Coupling in a Topological Resonator

Shushu Shi  Xin Xie  Sai Yan  Beijing National Laboratory for Condensed Matter
Physics, Institute of Physics, Chinese Academy of Sciences, Beijing 100190,
China  CAS Center for Excellence in Topological Quantum Computation and School
of Physical Sciences, University of Chinese Academy of Sciences, Beijing
100049, China  Jingnan Yang  State Key Laboratory for Mesoscopic Physics and
Frontiers Science Center for Nano-optoelectronics, School of Physics, Peking
University, 100871 Beijing, China  Jianchen Dang  Shan Xiao  Longlong Yang
Danjie Dai  Beijing National Laboratory for Condensed Matter Physics,
Institute of Physics, Chinese Academy of Sciences, Beijing 100190, China  CAS
Center for Excellence in Topological Quantum Computation and School of
Physical Sciences, University of Chinese Academy of Sciences, Beijing 100049,
China  Bowen Fu  State Key Laboratory for Mesoscopic Physics and Frontiers
Science Center for Nano-optoelectronics, School of Physics, Peking University,
100871 Beijing, China  Yu Yuan  Rui Zhu  Beijing National Laboratory for
Condensed Matter Physics, Institute of Physics, Chinese Academy of Sciences,
Beijing 100190, China  CAS Center for Excellence in Topological Quantum
Computation and School of Physical Sciences, University of Chinese Academy of
Sciences, Beijing 100049, China  Xiangbin Su  Hanqing Liu  State Key
Laboratory of Superlattices and Microstructures, Institute of Semiconductors
Chinese Academy of Sciences, Beijing 100083, China  Zhanchun Zuo  Beijing
National Laboratory for Condensed Matter Physics, Institute of Physics,
Chinese Academy of Sciences, Beijing 100190, China  CAS Center for Excellence
in Topological Quantum Computation and School of Physical Sciences, University
of Chinese Academy of Sciences, Beijing 100049, China  Can Wang  Beijing
National Laboratory for Condensed Matter Physics, Institute of Physics,
Chinese Academy of Sciences, Beijing 100190, China  CAS Center for Excellence
in Topological Quantum Computation and School of Physical Sciences, University
of Chinese Academy of Sciences, Beijing 100049, China  Songshan Lake Materials
Laboratory, Dongguan, Guangdong 523808, China  Haiqiao Ni  State Key
Laboratory of Superlattices and Microstructures, Institute of Semiconductors
Chinese Academy of Sciences, Beijing 100083, China  Zhichuan Niu  State Key
Laboratory of Superlattices and Microstructures, Institute of Semiconductors
Chinese Academy of Sciences, Beijing 100083, China  Qihuang Gong  State Key
Laboratory for Mesoscopic Physics and Frontiers Science Center for Nano-
optoelectronics, School of Physics, Peking University, 100871 Beijing, China
Xiulai Xu  State Key Laboratory for Mesoscopic Physics and Frontiers Science
Center for Nano-optoelectronics, School of Physics, Peking University, 100871
Beijing, China

###### Abstract

Chiral light-matter interactions supported by topological edge modes at the
interface of valley photonic crystals provide a robust method to implement the
unidirectional spin transfer. The valley topological photonic crystals possess
a pair of counterpropagating edge modes. The edge modes are robust against the
sharp bend of $60^{\circ}$ and $120^{\circ}$, which can form a resonator with
whispering gallery modes. Here, we demonstrate the asymmetric emission of
chiral coupling from single quantum dots in a topological resonator by tuning
the coupling between a quantum emitter and a resonator mode. Under a magnetic
field in Faraday configuration, the exciton state from a single quantum dot
splits into two exciton spin states with opposite circularly polarized
emissions due to Zeeman effect. Two branches of the quantum dot emissions
couple to a resonator mode in different degrees, resulting in an asymmetric
chiral emission. Without the demanding of site-control of quantum emitters for
chiral quantum optics, an extra degree of freedom to tune the chiral contrast
with a topological resonator could be useful for the development of on-chip
integrated photonic circuits.

The integrated optical nanostructures with the ability to manipulate light-
matter interaction are important building blocks toward realizing scalable
optical quantum networks for quantum-information processing [1, 2, 3]. In
particular, nanophotonic waveguides with the chiral interface can not only
confine photons transversely but also enable the transfer of matter-qubit
information when embedded with quantum emitters [4], such as quantum dots
(QDs) with spin states as stationary qubits [5]. Transverse optical
confinement in waveguides results in longitudinal and transverse components of
electromagnetic fields [6, 7], which leads to polarization-momentum locking.
Consequently, when quantum emitters with polarization-dependent transitions
couple with the transverse spin angular momentum with the same polarization
carried by the waveguide’s electromagnetic fields, the spontaneous emissions
radiate unidirectionally as the signature of chirality [8]. The chiral
coupling has been demonstrated in many structures [4, 9], including nanofibres
[10, 11, 12], dielectric nanobeam waveguides [13, 14, 15, 16, 17, 18, 19], and
photonic crystal waveguides [20, 21, 22]. However, the backscattering loss
during the propagation of light in those devices is inevitable, which limits
their performance in chiral optical circuits.

The recent emergence of topological photonics provides a robust approach to
manipulate chiral light-matter interactions [23, 24]. The propagation of light
in the waveguide formed by topological photonic crystals is immune to
deformations and disorders [25, 26, 27, 28]. The chiral edge states at the
interface of two different topological photonic crystals with the same band
gaps have been utilized to realize chiral coupling with handed circular
dipoles [29, 30, 31]. These edge states in topological waveguides protect the
transmission of photons from scattering around sharp bends and defects, which
allows the design of nanophotonic resonators possessing chiral spin-photon
interfaces [32, 33, 34, 35, 36, 37]. The Purcell-enhanced strong coupling
between a quantum emitter and a topological resonator has been demonstrated
with attractive prospects [36], such as designing photonic integrated circuits
for quantum-information processing and studying quantum many-body dynamics
[38]. However, the realization of spin-momentum locking effect in the
topological resonator interface mainly depends on the position of quantum
emitters inside the resonator [36], which is challenging to be implemented in
photonic integrated circuits.

Here, we propose an extra degree of freedom to tune the asymmetric emission of
chiral coupling in a topological resonator through the magnetic field. The
resonator is formed by interfacing two topologically distinct valley photonic
crystals (VPCs) with sharp bends of $60^{\circ}$. Single QDs are embedded in
the resonator to excite the resonator modes and generate exciton states with
opposite circular polarizations when applied with a magnetic field in parallel
with QDs’ growth direction. Different unidirectional emissions are achieved as
QD exciton states emissions selectively coupled with the counterpropagating
edge modes in the interface of the resonator. The chiral coupling is
experimentally demonstrated to establish the chirality nature of the
resonator. Then, an unequal contrast of the unidirectional emission
intensities of opposite directions is simultaneously observed when two
branches of QD’s emissions are spectral-overlap with a resonator mode in
different degrees. The different coupling strength between QD’s emissions and
a resonator mode can be used to tune the chiral contrast, which is a effective
method for tuning chirality in solid systems instead of changing the position
of the quantum dots.

Figure 1: (a) Schematic of the topological chiral interface. (b) Schematic of
QD level structure under a magnetic field in Faraday configuration. (c)
Dispersion curves for the edge states formed at the bearded interfaces.

Figure 1(a) shows the scanning electron microscope (SEM) image of a valley-
Hall topological interface between two topologically distinct VPCs colored in
green and blue, respectively. The VPCs are formed by a honeycomb lattice of
two inverted equilateral triangular airholes with different side lengths of
$L_{1}$ and $L_{2}$. As $L_{1}\neq L_{2}$, the spatial inversion symmetry of
the VPCs breaks, which causes the formation of a topological band gap. The two
types of VPCs share the same side lengths of $L_{1}$ and $L_{2}$, resulting in
the same band gap. However, the triangular airholes of two VPCs are inverted
from each other. The signs of nonzero valley Chern numbers at the $K$ and
$K\textquoteright$ points in two VPCs are opposite, which differ in the
topological properties of two VPCs. Therefore, topological edge states are
formed at the interface of the two VPCs. The demonstration of chirality in the
topological interface requires a pair of circularly polarized dipoles to be
selectively coupled to the counterpropagating edge states. QDs can be used as
spin-polarized dipole sources and easily integrated in different systems.
Figure 1(b) depicts the QD transition under a magnetic field with a Faraday
configuration, as the exciton states degenerate due to Zeeman splitting [39,
40, 41]. Two exciton states are formed with opposite circular polarized
emissions [42] of $\sigma^{+}$ and $\sigma^{-}$.

Figure 1(c) shows the dispersion curves of edge states for the interface with
simulation parameters of lattice constant a = 255 nm, the side lengths of the
two triangular holes $L_{1}=1.3a/\sqrt{3}$ and $L_{2}=0.7a/\sqrt{3}$. The
simulation is performed by the three-dimensional finite-difference time-domain
(FDTD) method. The blue line in Fig. 1(c) corresponds to the topological edge
state, while the red line indicates the trivial state. The transmission of
chiral coupling in the wavelength range of the blue line is topologically
protected [28]. It is noted that the QD ensemble inhomogeneous emission in our
experiment corresponds to the fast-light regime at the bearded interface with
a wavelength around 940 nm, as shown in Fig. 1(c).

Figure 2: (a) False-color SEM image of a Vally-Hall topological resonator
(blue triangle) coupled to a topological waveguide (the interface between
green and blue squares) with two grating couplers (yellow). The white circle
indicates the excitation spot on the tip of the resonator. (b) Simulated mode
distribution of a topological resonator with the averaged FSR of about 11 nm.
(c) PL spectrum from a topological resonator embedded with QDs. The gray
regions correspond to the distribution of resonator modes. The averaged FSR
($\sim$ 11 nm) from the experimentally measured spectrum matches well with the
simulated result. (d) PL spectra of a resonator mode with a Q factor of 2600
under low excitation power (blue line) and high excitation power (red line) at
zero magnetic field with two QDs nearby. The half-height width of the mode is
0.358 nm.

Because of their robustness against sharp bends of $60^{\circ}$ and
$120^{\circ}$, the topological edge states can be used to generate a
topological resonator, in which the topologically protected chiral coupling
with enhanced light–matter interaction is expected [36]. The SEM image of a
topological resonator is shown in the inset of Fig. 2(a). The topological
resonator is coupled with a topological waveguide, which enables the
investigation of chiral coupling between QDs and the edge states inside the
resonator. The topological waveguide is terminated by two grating couplers to
probe the chiral coupling through a 0.8 numerical aperture objective lens. For
fabrication, a single layer of InGaAs QDs is grown in a 150 nm GaAs membrane
on a sacrificial layer of AlGaAs. The resonator pattern is transferred on the
slab through electron beam lithography and followed by inductively coupled
plasma etching. Then, hydrofluoric acid etching is used to remove the exposed
AlGaAs sacrificial layer, leaving a suspended membrane with designed
structures. The experimental measurements are implemented on a three-axis
piezoelectric positioner in a helium bath cryostat under 4.2 K.

The quantum dots on the tip (colored in white circle) of the resonator is
nonresonantly excited by a continuous-wave laser of 532 nm. The PL spectrum in
Fig. 2(c) collected from the same spot shows the modes of the topological
resonator under high excitation power with a free-spectral range (FSR) of
about 11 nm. The sharp peaks in Fig. 2(c) are single QDs with half-height
widths of around 0.15 nm. Figure 2(b) shows the simulated multiple modes of a
topological resonator with the same lattice constant a and the same number of
unit cells on the resonator edge as the sample in Fig. 2(a). The average FSR
obtained by Lorentz fitting the simulated modes in Fig. 2(b) is about 11 nm.
The experimental average FSR matches well with the simulation result. We chose
the resonator with a zigzag interface to simulate the distribution of the
resonator modes because the exist of slow light modes in the simulation of the
resonator with a bearded interface makes it difficult to extract FSR of the
whispering gallery modes. The FSR of the whispering gallery modes mainly
depends on resonator side length [32], instead of the types of the interface
of the resonator. Figure 2(d) depicts the PL spectrum (red line) under high
excitation power, which reveals the resonator mode, along with the PL spectrum
(blue line) under low excitation power, which indicates quantum dot emissions
close to the resonator mode with a Q factor of 2600. The emission energy of
quantum dots can be tuned by a magnetic field [43] to get into resonance with
the resonator mode.

Figure 3: The PL spectra as a function of the magnetic field collected from
the excitation spot of (a) QD1 (around 952 nm) and (d) QD2 (around 940.4 nm).
The transmission PL spectra collected from the right grating coupler and left
grating coupler from (b)-(c) QD1 and (e)-(f) QD2, respectively. Figure 4: The
chiral contrast of (a) QD1, (c) QD2 and (e) QD3 measured from the right (red
squares) or left (blue circles) grating coupler as a function of the magnetic
field. The contrasts between the corresponding chiral contrasts collected from
two grating couplers for (b) QD1, (d) QD2 and (f) QD3 under each magnetic
field. The blue region indicates the range of the contrasts in (f) which are
under 0.2. The intensity variation of the two spin states from (g) QD1 and
QD3, (h) QD2 as a function of the magnetic field during central collection.

The PL spectra of QD1 and QD2 shown by yellow circles in Fig. 2(c) near the
resonator modes are measured as a function of the magnetic field. Under the
magnetic field in Faraday configuration, the QD transition occurs due to
Zeeman splitting, forming a pair of circularly polarized emissions with
opposite spins, which can be spectrally resolved as two branches. The signals
of QD1 (around 952 nm) and QD2 (around 940.4 nm) are respectively collected
from the excitation spot, left grating coupler and right grating coupler,
which are shown in Fig. 3. Both branches of QD1 (around 952 nm) are observed
in Fig. 3(a). Only the single branch of QD1 with a circular polarization of
$\sigma^{-}$ is observed in Fig. 3(b), while the $\sigma^{+}$ branch of QD1 is
observed in Fig. 3(c). The transmission spectra exhibit directionality, which
is a signature of chiral coupling [19]. As the topological resonator’s
chirality is demonstrated, an abnormal transmission result is observed
simultaneously in the same spectrum. Figure 3(d) depicts the two branches of
QD2 (around 940.4 nm) with measured from the excitation spot. However, the
unidirectionality of transmission is more obvious in Fig. 3(e) collected from
the right grating coupler comparing to that in Fig. 3(f) collected from the
left grating coupler, which indicates the asymmetric chiral coupling for
different transmission directions.

To investigate the observed asymmetric chiral coupling in the topological
resonator, we calculate the chiral contrasts from each grating coupler as a
function of the magnetic field. The chiral contrast C is defined as
$C=|I_{\sigma^{+}}-I_{\sigma^{-}}|/(I_{\sigma^{+}}+I_{\sigma^{-}})$, where
$I_{\sigma^{-}}(I_{\sigma^{+}})$ denotes the intensity of the upper (lower)
branch. The QD3 (around 958 nm) which is away from the resonator modes is also
measured for comparison. Figures 4 show the chiral contrasts with the error
bar extracted from PL spectra of QD1, QD2 and QD3 collected from left and
right grating couplers, respectively. It is noted that the chiral contrast
depends on the position of the quantum emitter along the interface of the
resonator and the different positions of QD1 and QD2 cause the reversed
directional emissions [36]. The chiral contrasts of QD3 measured from left and
right grating coupler are almost equal to each other in Fig. 4(e). However, in
Fig. 4(a), the difference between the chiral contrasts of QD1 becomes larger
under high magnetic field. Moreover, the chiral contrasts of QD2 collected
from left and right grating coupler are different under each magnetic field,
as shown in Fig. 4(c). The distinction in chiral contrasts indicates that one
branch of QD emissions is more affected by the coupling with a resonator mode.

To further quantify the asymmetric chiral coupling, we define the contrast of
two chiral contrasts from two grating couplers with opposite transmission
directions as $C_{C}=|C_{R}-C_{L}|/(C_{R}+C_{L})$, where $C_{R}(C_{L})$
denotes the chiral contrat obtained in the spectrum from right (left) grating
coupler. The $C_{C}$ of QD3 (around 958 nm) under each magnetic field in Fig.
4(f) are ranging from 0 to 0.2 (colored in blue), which can be attributed to
the fabrication defects of grating couplers and the measurement error. The
$C_{C}$ of QD1 increases under high magnetic field in Fig. 4(b). Figure 4(g)
shows that the intensities of two spin states of QD1 is enhanced under high
magnetic field during central collection, while the intensities of two spin
states of QD3 are not affected by the resonator mode. And all the $C_{C}$ of
QD2 in Fig. 4(d) are far away from the range (colored in blue) and increase to
the maximum at 5.5 T. The rise of $C_{C}$ is attributed to that the two
branches of QD emissions are enhanced by the resonator mode in different
degrees, which can be demonstrated by the different intensity enhancement of
two spin states of QD2 shown in Fig. 4(h). By introducing the coupling between
two spin states of a single QD and a resonator mode, a rise of chiral contrast
for one direction and a decrease of that for the other direction are achieved.

In conclusion, we demonstrate the asymmetric emission of chiral coupling in a
topological resonator. A waveguide is coupled with the topological resonator
to read out the chiral coupling inside the resonator. The asymmetric chiral
coupling is demonstrated by calculating the contrast between chiral contrasts
collected from two grating couplers under each magnetic field. When two
branches of a QD emissions coupled with a resonator mode in different degrees,
the magnetic field modulated asymmetric chiral contrast is observed. It offers
an extra degree of freedom to tune the chiral contrast besides changing the
position of quantum emitters along the interface of topological resonators. In
addition, it offers the ability to enhance the light-matter interaction
simultaneously. Such a tunable in-plane asymmetric emission of chiral coupling
is beneficial for the development of complex optical circuits on a chip.

###### Acknowledgements.

This work was supported by the National Key Research and Development Program
of China (Grant No. 2021YFA1400700), the National Natural Science Foundation
of China (Grants Nos. 62025507, 11934019, 92250301, 11721404, 62175254 and
12204020), the Strategic Priority Research Program (Grant No. XDB28000000) of
the Chinese Academy of Sciences.

## References

  * Lodahl, Mahmoodian, and Stobbe [2015] P. Lodahl, S. Mahmoodian, and S. Stobbe, “Interfacing single photons and single quantum dots with photonic nanostructures,” Rev. Mod. Phys. 87, 347–400 (2015).
  * Reiserer and Rempe [2015] A. Reiserer and G. Rempe, “Cavity-based quantum networks with single atoms and optical photons,” Rev. Mod. Phys. 87, 1379–1418 (2015).
  * Dietrich  _et al._ [2016] C. P. Dietrich, A. Fiore, M. G. Thompson, M. Kamp, and S. Hofling, “Gaas integrated quantum photonics: Towards compact and multi-functional quantum photonic integrated circuits,” Laser Photonics Rev. 10, 870–894 (2016).
  * Lodahl  _et al._ [2017] P. Lodahl, S. Mahmoodian, S. Stobbe, A. Rauschenbeutel, P. Schneeweiss, J. Volz, H. Pichler, and P. Zoller, “Chiral quantum optics,” Nature 541, 473–480 (2017).
  * Arakawa and Holmes [2020] Y. Arakawa and M. J. Holmes, “Progress in quantum-dot single photon sources for quantum information technologies: A broad spectrum overview,” Appl. Phys. Rev. 7, 021309 (2020).
  * Bliokh and Nori [2015] K. Y. Bliokh and F. Nori, “Transverse and longitudinal angular momenta of light,” Phys. Rep. 592, 1–38 (2015).
  * Aiello  _et al._ [2015] A. Aiello, P. Banzer, M. Neugebauer, and G. Leuchs, “From transverse angular momentum to photonic wheels,” Nat. Photonics 9, 789–795 (2015).
  * Lang  _et al._ [2022] B. Lang, D. P. McCutcheon, E. Harbord, A. B. Young, and R. Oulton, “Perfect chirality with imperfect polarization,” Phys. Rev. Lett. 128, 073602 (2022).
  * Bliokh  _et al._ [2015] K. Y. Bliokh, F. J. Rodríguez-Fortuño, F. Nori, and A. V. Zayats, “Spin–orbit interactions of light,” Nat. Photonics 9, 796–808 (2015).
  * Petersen, Volz, and Rauschenbeutel [2014] J. Petersen, J. Volz, and A. Rauschenbeutel, “Chiral nanophotonic waveguide interface based on spin-orbit interaction of light,” Science 346, 67–71 (2014).
  * Mitsch  _et al._ [2014] R. Mitsch, C. Sayrin, B. Albrecht, P. Schneeweiss, and A. Rauschenbeutel, “Quantum state-controlled directional spontaneous emission of photons into a nanophotonic waveguide,” Nat. Commun. 5, 5713 (2014).
  * Sayrin  _et al._ [2015] C. Sayrin, C. Junge, R. Mitsch, B. Albrecht, D. O’Shea, P. Schneeweiss, J. Volz, and A. Rauschenbeutel, “Nanophotonic optical isolator controlled by the internal state of cold atoms,” Phys. Rev. X 5, 041036 (2015).
  * Coles  _et al._ [2016] R. Coles, D. Price, J. Dixon, B. Royall, E. Clarke, P. Kok, M. Skolnick, A. Fox, and M. Makhonin, “Chirality of nanophotonic waveguide with embedded quantum emitter for unidirectional spin transfer,” Nat. Commun. 7, 11183 (2016).
  * Coles  _et al._ [2017] R. J. Coles, D. M. Price, B. Royall, E. Clarke, M. S. Skolnick, A. M. Fox, and M. Makhonin, “Path-dependent initialization of a single quantum dot exciton spin in a nanophotonic waveguide,” Phys. Rev. B 95, 121401 (2017).
  * Luxmoore  _et al._ [2013a] I. Luxmoore, N. A. Wasley, A. J. Ramsay, A. Thijssen, R. Oulton, M. Hugues, A. Fox, and M. Skolnick, “Optical control of the emission direction of a quantum dot,” Appl. Phys. Lett. 103, 241102 (2013a).
  * Luxmoore  _et al._ [2013b] I. Luxmoore, N. Wasley, A. Ramsay, A. Thijssen, R. Oulton, M. Hugues, S. Kasture, V. Achanta, A. Fox, and M. Skolnick, “Interfacing spins in an ingaas quantum dot to a semiconductor waveguide circuit using emitted photons,” Phys. Rev. Lett. 110, 037402 (2013b).
  * Javadi  _et al._ [2018] A. Javadi, D. Ding, M. H. Appel, S. Mahmoodian, M. C. Löbl, I. Söllner, R. Schott, C. Papon, T. Pregnolato, S. Stobbe, _et al._ , “Spin–photon interface and spin-controlled photon switching in a nanobeam waveguide,” Nat. Nanotechnol. 13, 398–403 (2018).
  * Xiao  _et al._ [2021a] S. Xiao, S. Wu, X. Xie, J. Yang, W. Wei, S. Shi, F. Song, S. Sun, J. Dang, L. Yang, _et al._ , “Position-dependent chiral coupling between single quantum dots and cross waveguides,” Appl. Phys. Lett. 118, 091106 (2021a).
  * Xiao  _et al._ [2021b] S. Xiao, S. Wu, X. Xie, J. Yang, W. Wei, S. Shi, F. Song, J. Dang, S. Sun, L. Yang, _et al._ , “Chiral photonic circuits for deterministic spin transfer,” Laser Photonics Rev. 15, 2100009 (2021b).
  * Söllner  _et al._ [2015] I. Söllner, S. Mahmoodian, S. L. Hansen, L. Midolo, A. Javadi, G. Kiršanskė, T. Pregnolato, H. El-Ella, E. H. Lee, J. D. Song, _et al._ , “Deterministic photon–emitter coupling in chiral photonic circuits,” Nat. Nanotechnol. 10, 775–778 (2015).
  * Young  _et al._ [2015] A. B. Young, A. Thijssen, D. M. Beggs, P. Androvitsaneas, L. Kuipers, J. G. Rarity, S. Hughes, and R. Oulton, “Polarization engineering in photonic crystal waveguides for spin-photon entanglers,” Phys. Rev. Lett. 115, 153901 (2015).
  * Mahmoodian, Lodahl, and Sørensen [2016] S. Mahmoodian, P. Lodahl, and A. S. Sørensen, “Quantum networks with chiral-light–matter interaction in waveguides,” Phys. Rev. Lett. 117, 240501 (2016).
  * Iwamoto, Ota, and Arakawa [2021] S. Iwamoto, Y. Ota, and Y. Arakawa, “Recent progress in topological waveguides and nanocavities in a semiconductor photonic crystal platform,” Opt Mater Express. 11, 319–337 (2021).
  * Ozawa  _et al._ [2019] T. Ozawa, H. M. Price, A. Amo, N. Goldman, M. Hafezi, L. Lu, M. C. Rechtsman, D. Schuster, J. Simon, O. Zilberberg, _et al._ , “Topological photonics,” Rev. Mod. Phys. 91, 015006 (2019).
  * He  _et al._ [2019] X. He, E. Liang, J. Yuan, H. Qiu, X. Chen, F. Zhao, and J. Dong, “A silicon-on-insulator slab for topological valley transport,” Nat. Commun. 10, 872 (2019).
  * Shalaev  _et al._ [2019] M. I. Shalaev, W. Walasik, A. Tsukernik, Y. Xu, and N. M. Litchinitser, “Robust topologically protected transport in photonic crystals at telecommunication wavelengths,” Nat. Nanotechnol. 14, 31–34 (2019).
  * Yoshimi  _et al._ [2021] H. Yoshimi, T. Yamaguchi, R. Katsumi, Y. Ota, Y. Arakawa, and S. Iwamoto, “Experimental demonstration of topological slow light waveguides in valley photonic crystals,” Opt. Express 29, 13441–13450 (2021).
  * Yoshimi  _et al._ [2020] H. Yoshimi, T. Yamaguchi, Y. Ota, Y. Arakawa, and S. Iwamoto, “Slow light waveguides in topological valley photonic crystals,” Opt. Lett. 45, 2648–2651 (2020).
  * Barik  _et al._ [2018] S. Barik, A. Karasahin, C. Flower, T. Cai, H. Miyake, W. DeGottardi, M. Hafezi, and E. Waks, “A topological quantum optics interface,” Science 359, 666–668 (2018).
  * Barik  _et al._ [2016] S. Barik, H. Miyake, W. DeGottardi, E. Waks, and M. Hafezi, “Two-dimensionally confined topological edge states in photonic crystals,” New J. Phys. 18, 113013 (2016).
  * Parappurath  _et al._ [2020] N. Parappurath, F. Alpeggiani, L. Kuipers, and E. Verhagen, “Direct observation of topological edge states in silicon photonic crystals: Spin, dispersion, and chiral routing,” Sci. Adv. 6, eaaw4137 (2020).
  * Mehrabad  _et al._ [2020] M. J. Mehrabad, A. P. Foster, R. Dost, E. Clarke, P. K. Patil, A. M. Fox, M. S. Skolnick, and L. R. Wilson, “Chiral topological photonics with an embedded quantum emitter,” Optica 7, 1690–1696 (2020).
  * Jalali Mehrabad  _et al._ [2020] M. Jalali Mehrabad, A. Foster, R. Dost, E. Clarke, P. Patil, I. Farrer, J. Heffernan, M. Skolnick, and L. Wilson, “A semiconductor topological photonic ring resonator,” Appl. Phys. Lett. 116, 061102 (2020).
  * Yang and Hang [2018] Y. Yang and Z. H. Hang, “Topological whispering gallery modes in two-dimensional photonic crystal cavities,” Opt. Express 26, 21235–21241 (2018).
  * Ma, Xi, and Sun [2019] J. Ma, X. Xi, and X. Sun, “Topological photonic integrated circuits based on valley kink states,” Laser Photonics Rev. 13, 1900087 (2019).
  * Barik  _et al._ [2020] S. Barik, A. Karasahin, S. Mittal, E. Waks, and M. Hafezi, “Chiral quantum optics using a topological resonator,” Phys. Rev. B 101, 205303 (2020).
  * Xie  _et al._ [2021] X. Xie, S. Yan, J. Dang, J. Yang, S. Xiao, Y. Wang, S. Shi, L. Yang, D. Dai, Y. Yuan, _et al._ , “Topological cavity based on slow-light topological edge mode for broadband purcell enhancement,” Phys. Rev. Appl. 16, 014036 (2021).
  * Pichler  _et al._ [2015] H. Pichler, T. Ramos, A. J. Daley, and P. Zoller, “Quantum optics of chiral spin networks,” Phys. Rev. A 91, 042116 (2015).
  * Kuther  _et al._ [1998] A. Kuther, M. Bayer, A. Forchel, A. Gorbunov, V. Timofeev, F. Schäfer, and J. Reithmaier, “Zeeman splitting of excitons and biexcitons in single ${\mathrm{In}}_{0.60}{\mathrm{Ga}}_{0.40}\mathrm{A}\mathrm{s}/\mathrm{G}\mathrm{a}\mathrm{A}\mathrm{s}$ self-assembled quantum dots,” Phys. Rev. B 58, R7508 (1998).
  * Wu  _et al._ [2020] S. Wu, K. Peng, X. Xie, J. Yang, S. Xiao, F. Song, J. Dang, S. Sun, L. Yang, Y. Wang, S. Shi, J. He, Z. Zuo, and X. Xu, “Electron and hole g tensors of neutral and charged excitons in single quantum dots by high-resolution photocurrent spectroscopy,” Phys. Rev. Applied 14, 014049 (2020).
  * Peng  _et al._ [2017] K. Peng, S. Wu, J. Tang, F. Song, C. Qian, S. Sun, S. Xiao, M. Wang, H. Ali, D. A. Williams, and X. Xu, “Probing the dark-exciton states of a single quantum dot using photocurrent spectroscopy in a magnetic field,” Phys. Rev. Applied 8, 064018 (2017).
  * Bayer  _et al._ [2002] M. Bayer, G. Ortner, O. Stern, A. Kuther, A. Gorbunov, A. Forchel, P. Hawrylak, S. Fafard, K. Hinzer, T. Reinecke, _et al._ , “Fine structure of neutral and charged excitons in self-assembled In(Ga)As/(Al)GaAs quantum dots,” Phys. Rev. B 65, 195315 (2002).
  * Kim  _et al._ [2011] H. Kim, T. C. Shen, D. Sridharan, G. S. Solomon, and E. Waks, “Magnetic field tuning of a quantum dot strongly coupled to a photonic crystal cavity,” Appl. Phys. Lett. 98, 091102 (2011).


# Alleviating the Effect of Data Imbalance on Adversarial Training

Guanlin Li
Nanyang Technological University, S-Lab
<EMAIL_ADDRESS>Guowen Xu
City University of Hong Kong
<EMAIL_ADDRESS>Tianwei Zhang
Nanyang Technological University
<EMAIL_ADDRESS>
###### Abstract

In this paper, we study adversarial training on datasets that obey the long-
tailed distribution, which is practical but rarely explored in previous works.
Compared with conventional adversarial training on balanced datasets, this
process falls into the dilemma of generating uneven adversarial examples (AEs)
and an unbalanced feature embedding space, causing the resulting model to
exhibit low robustness and accuracy on tail data. To combat that, we
theoretically analyze the lower bound of the robust risk to train a model on a
long-tailed dataset to obtain the key challenges in addressing the
aforementioned dilemmas. Based on it, we propose a new adversarial training
framework – Re-balancing Adversarial Training (REAT). This framework consists
of two components: (1) a new training strategy inspired by the effective
number to guide the model to generate more balanced and informative AEs; (2) a
carefully constructed penalty function to force a satisfactory feature space.
Evaluation results on different datasets and model structures prove that REAT
can effectively enhance the model’s robustness and preserve the model’s clean
accuracy. The code can be found in https://github.com/GuanlinLee/REAT.

##  1 Introduction

Adversarial attacks [7, 21] have become a serious threat to deep learning
models, where the adversary crafts adversarial examples (AEs) by adding
imperceptible perturbations to a clean input, to deceive the model into making
wrong predictions. To mitigate adversarial attacks, a prominent way is
adversarial training [21], which generates AEs and incorporates them into the
training set to improve the model’s adversarial robustness.

However, existing efforts of adversarial training mainly focus on balanced
datasets, while ignoring more realistic datasets obeying long-tailed
distributions [19, 3, 6]. Informally, training data subject to a long-tailed
distribution has the property that the vast majority of the data belong to a
minority of total classes (i.e., “head” classes), while the remaining data
belong to other classes (“body” and “tail” classes) [29]. This distinct nature
yields new problems in adversarial training (see Section 2.2 for detailed
explanations). First, it is difficult to produce uniform and balanced
adversarial examples (AEs): AEs are mainly misclassified by the model into the
head classes with overwhelming probabilities regardless of the labels of their
corresponding clean samples. Second, the excessive dominance of head classes
in the feature embedding space further compresses the feature space of tail
classes. The mutual entanglement of the above two problems leads to the
underfitting of tail classes in both adversarial robustness and clean
accuracy, thus leading to unsatisfactory training performance.

To address these challenges, [31] proposed RoBal, the first work (and the only
work, to our best knowledge) towards adversarial training on datasets with
long-tailed distributions. It is essentially a two-stage re-balancing
adversarial training method. The first stage lies in the training process,
where a new class-aware margin loss function is designed to make the model pay
equal attention to data from head classes and tail classes. The second stage
focuses on the inference process, where a pre-defined bias is added to the
predicted logits vectors, thereby improving the prediction accuracy of samples
from the tail classes. Moreover, RoBal constructs a new normalized cosine
classification layer, to further improve models’ accuracy and adversarial
robustness.

While RoBal shows impressive results on a variety of datasets, it still has
several limitations. First, the robustness of RoBal benefits mainly from
gradient obfuscation (specifically, gradient vanishing) [1] in the proposed
new scale-invariant classification layer. This can be easily compromised by
simply multiplying the logits by a constant, as the constant can increase the
absolute value of gradients against gradient vanishing and correct the sign of
gradients during AE generation (see Tables 3 and 4). Second, the designed
class-aware margin loss ignores samples from body classes and exclusively
focuses on head and tail classes, which inevitably reduces the overall model
accuracy. Details can be found in Sections 2.2 and 5.2.

To advance the practicality of adversarial training on long-tailed datasets,
we design a new framework: Re-balancing adversarial training (REAT), which
demonstrates higher clean accuracy and adversarial robustness compared to
RoBal. Our insights come from the revisit of two key components in adversarial
training: AE generation and feature embedding. Particularly, for AE
generation, we force the generated AEs to be misclassified into each class as
uniformly as possible, so that the information of the tail classes is
sufficiently learned during adversarial training to improve the robustness.
Our implementation is inspired by the effective number [6] in long-tailed
recognition, which was proposed to increase the marginal benefits from data of
tail classes. We generalize its definition to the AE generation process and
propose a new Re-Balanced Loss (RBL) function. RBL dynamically adjusts the
weights assigned to each class, which significantly improves the effectiveness
of the original balanced loss. For feature embedding, it is challenging to
balance the volume of each class’s feature space, especially if the size of
each class varies. To address this issue, we propose a Tail-sample-mining-
based feature margin regularization (TAIL) approach. TAIL treats the samples
from tail classes as hard samples and optimizes feature embedding
distributions of tail classes and others. To better fit the unbalanced data
distribution, we propose a joint weight to increase the contribution of tail
features in the entire feature embedding space. Visualization results can be
found in the supplementary materials. We conduct comprehensive experiments on
CIFAR-10-LT, CIFAR-100-LT, and Tiny-Imagenet datasets to demonstrate the
superiority of REAT over existing methods. For instance, REAT achieves 67.33%
clean accuracy and 32.08% robust accuracy under AutoAttack, which are 1.25%
and 0.94% higher than RoBal.

We want to emphasize that REAT is not a simple combination of previous works.
First, our method is based on the theoretical analysis. Furthermore, its
superiority lies in the newly proposed modifications over previous methods, to
adapt to the long-tailed adversarial training scenario. We validate that
simply integrating existing methods cannot achieve satisfactory results. For
instance, without generalizing the definition of effective number from normal
training to AE generation, the original term will result in much lower
performance (“ENR” in Table 2).

##  2 Background and Motivation

###  2.1 Long-tailed Recognition

Data in the wild usually obey a long-tailed distribution [19, 3, 6], where
most samples belong to a small part of classes. Models trained on long-tailed
datasets usually give higher confidence to the samples from head classes,
which harms the generalizability for the samples from the body or tail
classes. It is challenging to solve such overconfidence issues under the long-
tailed scenarios [14, 9, 2]. Several approaches have been proposed to achieve
long-tailed recognition. For instance, (1) the re-sampling methods [20, 8, 24]
generate balanced data distributions by sampling data with different
frequencies in the training set. (2) The cost-sensitive learning methods [11,
6, 19] modify the training loss with additional weights to balance the
gradients from each class. (3) The training phase decoupling methods [15, 16]
first train a feature extractor on re-sampled balanced data, and then train a
classifier on the original dataset. (4) The classifier designing methods [15,
31] modify the classification layer with prior knowledge to better fit the
unbalanced data. Morel details about related works are in the supplementary
materials.

###  2.2 Long-tailed Adversarial Training

Adversarial training is a promising solution to enhance the model’s robustness
against AEs. Previous works mainly consider the balanced datasets. Discussions
of these works can be found in the supplementary materials. When the training
data become unbalanced, training a robust model becomes more challenging. As
mentioned in Section 2.1, in long-tailed recognition, most data come from the
head classes while the data of the tail classes are relatively scarce. This
causes two consequences: unbalance in the output probability space and
unbalance in the feature embedding space, which are detailed as follows.

First, we need to generate AEs on-the-fly during adversarial training. The
unbalanced output probability space caused by long-tailed datasets can lead to
unbalanced AEs, which cause the produced model to show unbalanced robustness
across different classes. Figure 2 shows such an example. We adopt PGD-based
adversarial training to train a ResNet-18 model and measure the distribution
of the model’s predictions for the generated AEs during the training process.
Figure 1(a) shows the case of a balanced training set (CIFAR-10). We observe
that the predictions of the AEs are uniformly distributed among all the
classes. In contrast, Figure 1(b) shows the case of an unbalanced training set
(CIFAR-10-LT). Due to the long-tailed distribution, most AEs are labeled as
head classes. This indicates that the final model has lower accuracy and
robustness for tail classes, making it more vulnerable to adversarial attacks
[5].

(a) Balanced dataset

(b) Unbalanced dataset

Figure 1: Prediction distributions of AEs. Clean label distributions are shown
by gray bars.

(c) Balanced dataset

(d) Unbalanced dataset

Figure 2: Visualization of feature maps of AEs from models.

Second, in an unbalanced training set, the head classes can dominate the
feature embedding space of the model, which can reduce the area of tail
features. As a result, the performance and generalizability of the model for
tail classes will be decreased. In contrast, a model trained on balanced data
will give an even feature space for each class. Figure 2 compares the feature
maps of AEs in these two scenarios, where we train ResNet-18 models with PGD-
based adversarial training on balanced and unbalanced CIFAR-10111For better
readability, we only show four classes (two head classes “airplane” (blue) and
“automobile” (orange), and two tail classes “ship” (green) and “truck”
(red).). The complete feature maps for 10 classes can be found in the
supplementary materials.. The long-tailed scenario has larger differences
between head and tail features compared to the balanced scenario.

A straightforward way is to directly adopt existing solutions introduced in
Section 2.1 (e.g., [19, 3, 6, 24]) for adversarial training, which can produce
more balanced AE prediction distributions and feature embedding space.
However, they can only partially address the overconfidence and
underconfidence issues in model prediction, due to the lack of tail samples
and AEs predicted as tail classes (see Section 5.2 and the supplementary
materials). RoBal [31] is the first methodology dedicated to adversarial
training with long-tailed datasets. It introduces a new loss function to
promote the model to learn features from head classes and tail classes
equally. It further replaces the traditional classification layer with a
cosine classifier, where both weights and features are normalized and the
outputs are multiplied by a temperature factor. In the inference phase, RoBal
adjusts the output logits with a prior distribution, which is aligned with the
label distribution. However, in our experiments, we find RoBal ignores the
features from the body classes, which can harm the clean accuracy and
robustness. Furthermore, RoBal can be easily defeated by a simple adaptive
attack, which multiplies the output logits by a factor when generating AEs
(see Section 5.2). This motivates us to explore a better solution for long-
tailed adversarial training.

##  3 Theoretical Analysis

Besides providing empirical studies in Section 2.2, we provide theoretical
analysis to give a lower bound for robust risk, aiming to support the most
vulnerable parts in models trained on a long-tailed dataset found in Section
2.2. To theoretically study the difficulties of adversarially training a model
on unbalanced datasets, we first consider the robust risk
$\mathcal{R}_{\mathrm{rob}}(\theta)$, where $\theta$ stands for the parameters
of a function $f$, which can be a deep learning model. Let
$\mathcal{X}_{i}\subset\mathbb{R}^{d}$ be the input space for class $i$, and
$\mathcal{Y}=\\{1,\dots,C\\}$ be the class set.

We consider a following dataset
$\mathcal{D}=\\{(x,i)|x\in\mathcal{X}_{i},i\in\mathcal{Y}\\}$, that for class
$i$, we only have $n_{i}$ data in it. Then, we define that
$p_{\theta}(\cdot|x)=\mathrm{softmax}(f_{\theta}(x))\in\mathbb{R}^{C}$ and
$F_{\theta}(x)=\mathrm{argmax}[f_{\theta}(x)]\in\mathcal{Y}$. Given a
perturbed set
$\mathcal{B}_{p}(x_{i},\epsilon)=\\{x_{i}^{\prime}\in\mathcal{X}_{i}|\|x_{i}-x_{i}^{\prime}\|_{p}\leq\epsilon\\}$,
we have
$\mathcal{R}_{\mathrm{rob}}(\theta)=\mathbb{E}_{\mathcal{D}}[\max_{x_{i}^{\prime}\in\mathcal{B}_{p}(x_{i},\epsilon)}\mathbbm{1}\\{F_{\theta}(x_{i}^{\prime})\neq
i\\}]$.

Assume that for each class $i$, its $p_{\theta}(\cdot|x_{i})$ is independent
to other class $j\neq i$. We have
$\mathcal{R}_{\mathrm{rob}}(\theta)=\sum_{i=1}^{C}\frac{1}{n_{i}}\sum_{j=1}^{n_{i}}\max_{x^{\prime}\in\mathcal{B}_{p}(x_{i}^{j},\epsilon)}\mathbbm{1}\\{F_{\theta}(x^{\prime})\neq
i\\}p(i)$. Let
$\mathcal{R}^{i}_{\mathrm{rob}}(\theta)=\sum_{j=1}^{n_{i}}\max_{x^{\prime}\in\mathcal{B}_{p}(x_{i}^{j},\epsilon)}\mathbbm{1}\\{F_{\theta}(x^{\prime})\neq
i\\}p(i)$, which can be further decomposed of two terms, i.e., the natural
risk and the boundary risk [35], i.e.,

$\displaystyle\mathcal{R}^{i}_{\mathrm{rob}}(\theta)=\mathcal{R}^{i}_{\mathrm{nat}}(\theta)+\mathcal{R}^{i}_{\mathrm{bdy}}(\theta)=\sum_{j=1}^{n_{i}}\mathbbm{1}\\{F_{\theta}(x_{i}^{j})\neq
i\\}p(i)$
$\displaystyle+\sum_{j=1}^{n_{i}}\mathbbm{1}\\{\exists{x^{\prime}\in\mathcal{B}_{p}(x_{i}^{j},\epsilon)},F_{\theta}(x^{\prime})\neq
i\\}p(i).$

We notice that for an unbalanced dataset,
$p(i)=\frac{n_{i}}{\sum_{i=1}^{C}n_{i}}$. Therefore, we have the following
lower bound for the robust risk $\mathcal{R}_{\mathrm{rob}}(\theta)$.

###### Theorem 1

Given a function $f_{\theta}$ and dataset $\mathcal{D}$, which we defined
above, let

$\displaystyle
g(x)\in\mathrm{argmax}_{x^{\prime}\in\mathcal{B}_{p}(x,\epsilon)}\mathbbm{1}\\{F_{\theta}(x)\neq
F_{\theta}(x^{\prime})\\}$

Assume that a larger $n_{i}$ will decrease the
$\mathcal{R}^{i}_{\mathrm{rob}}(\theta)$, and vice versa. Then, we have

$\displaystyle\mathcal{R}_{\mathrm{rob}}(\theta)\geq\sum_{i=1}^{C}\frac{1}{C}($
$\displaystyle\sum_{j=1}^{n_{i}}\mathbbm{1}\\{F_{\theta}(x_{i}^{j})\neq i\\}$
$\displaystyle+\sum_{j=1}^{n_{i}}\mathbbm{1}\\{F_{\theta}(g(x_{i}^{j}))\neq
F_{\theta}(x_{i}^{j})\\})$

The proof can be found in the supplementary materials. In Theorem 1,
$\mathbbm{1}\\{F_{\theta}(x_{i}^{j})\neq i\\}$ is the error for clean data
$x_{i}^{j}$, and $\mathbbm{1}\\{F_{\theta}(g(x_{i}^{j}))\neq
F_{\theta}(x_{i}^{j})\\}$ is a general robust error, which means that the
prediction of the perturbed data disagrees with the prediction of the clean
data. Specifically, $\mathbbm{1}\\{F_{\theta}(x_{i}^{j})\neq i\\}$ reflects
the representation entangling to some degree. And $g(x)$ can represent the
predicted label distribution of adversarial examples. These two terms are
aligned with the findings in Section 2.2. Therefore, the theoretical analysis
supports the motivation of balancing the label distribution of adversarial
examples and aligning feature representations.

##  4 Methodology

We introduce REAT, a new framework for adversarial training on unbalanced
datasets. REAT includes two innovations to address the two issues disclosed in
Section 2.2 and 3. Specifically, to balance the AE distribution and encourage
the model to learn more information from tail samples, we modify the objective
function in the AE generation process with weights calculated based on the
effective number [6]. To balance the feature embedding space, we propose a
loss term to increase the area of features from tail classes.

Preliminaries. We first give formal definitions of a long-tailed dataset.
Consider a dataset containing $C$ classes with $N_{i}$ samples in each class
$i$. We assume the classes are sorted in the descending order based on the
number of samples in each class, i.e., $N_{i}\geq N_{i+1}$. The unbalanced
ratio is defined as $\mathrm{UR}=\frac{N_{1}}{N_{C}}$ [3]. Following previous
works [29, 6], a long-tailed dataset can be divided into three parts: (1) $i$
is a head class (HC) if $1\leq i\leq\lfloor\frac{C}{3}\rfloor$, where $\lfloor
x\rfloor$ is a floor function; (2) $i$ is a tail class (TC) if
$\lceil\frac{2C}{3}\rceil\leq i\leq C$, where $\lceil x\rceil$ is a ceiling
function; (3) The rest are body classes (BC). Below, we describe the detailed
mechanisms of REAT.

###  4.1 Re-balancing AEs

For adversarial training, it is desirable that the objective function could
encourage AEs that are classified into rarely-seen classes while punishing AEs
that are classified into abundant classes. To realize this in the long-tailed
scenario, we borrow the idea of the effective number from [6] and generalize
it to adversarial training. The effective number is mainly used to measure the
data overlap of each class. For class $i$ containing $N_{i}$ data, its
effective number is defined as $E_{N_{i}}=\frac{1-\beta^{N_{i}}}{1-\beta}$,
where $\beta=\frac{\sum N_{i}-1}{\sum N_{i}}$. Given the effective numbers
$E_{N_{i}}$ and $E_{N_{j}}$, if $E_{N_{i}}>E_{N_{j}}$, the marginal benefit
obtained from increasing the number of training samples in class $i$ is less
than increasing the same number of training samples in class $j$ [6]. This
implies that we can adopt the effective number as a guide to balance the
distribution of AEs generated during training.

Motivation. We explain the motivation of using the effective number in AE
generation as follows. At a high level, the generation of AEs can be viewed as
a data sampling process, i.e., AEs are essentially sampled from the neighbors
of their corresponding clean samples. Therefore, we can calculate the
effective number between AEs generated in two consecutive epochs, and use it
as the basis to assign dynamic weights to each class in the loss function,
inducing the model to produce as many less overlapped AEs as possible in
consecutive epochs. This implicitly generates more AEs that are classified
into tail classes and makes the model extract more marginal benefits from
samples of tail classes, thus achieving our purpose.

Technical design. For simplicity, we assume that the predicted label
distributions (i.e., labels assigned by the model $M$ for AEs) in two
successive training epochs will stay stable and not change too much, which has
been proven in [36]. Then, in epoch $k-1$, we count the number of AEs that are
classified into each class, denoted as
$\mathbf{n}=[n_{1},n_{2},\dots,n_{C}]$.222For the first training epoch, we
directly use the number of clean data $N_{i}$ in each class as the prior
distribution.. As a result, generating AEs in epoch $k$ can be approximated as
sampling new AEs after sampling $n_{i}$ data for each class $i$. Therefore, we
can compute the effective number of class $i$ as
$E_{n_{i}}=\frac{1-\beta_{i}^{n_{i}}}{1-\beta_{i}}$, where
$\beta_{i}=\frac{N_{i}-1}{N_{i}}$. Note that our $\beta$ is class-related to
assign finer convergence parameters for each class, which is different from
the calculation in [6]. We will experimentally prove that this adaptive
effective number can better improve the model robustness in Section 5.1.

Based on the property that the effective number of each class is inversely
proportional to the marginal benefit of the new samples of this class, we
construct a new indicator variable weight $w_{i}$ for the marginal benefit,
which is inversely proportional to $E_{n_{i}}$. This weight can be used to
correct the loss in the AE generation process. Specifically, following the
class-balanced softmax cross-entropy loss proposed in [6], we compute the
weight $w_{i}$ for class $i$ as follows:

$\displaystyle w_{i}=\frac{C}{E_{n_{i}}\sum_{j=1}^{C}\frac{1}{E_{n_{j}}}}$ (1)

With the weight $w_{i}$ for each class $i$, we design a new Re-Balancing Loss
(RBL) function as below:

$\displaystyle\mathrm{\textit{RBL}}=-w_{i}*\log\frac{e^{z_{i}}}{\sum_{j}e^{z_{j}}}$
(2)

where $\log\frac{e^{z_{i}}}{\sum_{j}e^{z_{j}}}$ is the original loss function
adopted to generate AEs. Our goal is to maximize RBL to generate AEs for
adversarial training.

Analysis. We analyze why RBL can help generate balanced AEs from unbalanced
data samples. First, we show the effective number enjoys the asymptotic
properties: (1) when $n_{i}\to 0$, we have $E_{n_{i}}\to 0$ and $w_{i}\to C$;
(2) when $n_{i}\to\infty$, we have $E_{n_{i}}\to\frac{1}{1-\beta_{i}}$ and
$w_{i}\to 0$, as there exists $E_{n_{j}}\to 0,i\neq j$. Based on the
asymptotic properties, if there are many AEs assigned to the label of class
$i$ in epoch $k-1$, then in epoch $k$, the increased effective number
$E_{n_{i}}$ results in a smaller $w_{i}$. As a consequence, RBL will induce
AEs generated in this round with minimized data overlap compared to AEs of the
previous round, which implicitly generates more AEs that are classified into
other classes. Our experiments in Section 5.2 indicate that combined with
long-tailed recognition losses, RBL can better balance the AE generation
process and increase the number of AEs predicted into tail classes by
$5\times$. Figure 3 compares the distances of AEs between two consecutive
training epochs without and with RBL. The models (ResNet-18) are trained on
CIFAR-10-LT with the unbalanced ratio UR=50. A smaller distance indicates a
larger overlap of the two AEs and less marginal benefit the model can obtain
from the process. We observe that RBL is able to increase the distances of AEs
from tail classes, and generate more informative AEs to enhance the model’s
robustness.

###  4.2 Tail Feature Alignment

(a) w/o. RBL

(b) w/. RBL

Figure 3: Distributions of Euclidean distances of AEs generated from the same
clean data in consecutive training epochs.

From Figure 2, we find the feature space of tail classes is smaller than that
of head classes, making the model lean to classify the input into head
classes. So, it is important to expand the feature space for tail classes to
balance the feature representation. To achieve this goal, we first define a
probabilistic feature embedding space as
$\mathbf{f}^{p}=[\frac{e^{f_{1}}}{\sum_{j}e^{f_{j}}},\frac{e^{f_{2}}}{\sum_{j}e^{f_{j}}},\dots,\frac{e^{f_{K}}}{\sum_{j}e^{f_{j}}}]=[f_{1}^{p},f_{2}^{p},\dots,f_{K}^{p}]$,
where $f_{i}$ is the $i$-th feature before the final classification layer, and
$K$ is the feature dimension. The motivation of using a probabilistic feature
embedding space is to overcome the scale changes in feature representations
caused by the unbalanced data distribution [31]. For each class $i$, we assume
the probabilistic feature is sampled from a distribution
$\mathcal{D}_{i}^{f}$. As a result, given any two classes $i\in TC$ and $j\in
HC\cup BC$, our goal is to maximize the difference between
$\mathcal{D}_{i}^{f}$ and $\mathcal{D}_{j}^{f}$, thereby rebalancing the
distributions of different classes in the feature space and making them more
divisible.

Algorithm 1 TAIL

1: Input: probabilistic feature batch $\mathbf{F}^{p}$, label $\mathbf{y}$,
class weights $\mathbf{\Omega}$, tail classes TC, batch size $B$

2: $R\leftarrow 0,S\leftarrow 0$

3: for $i=1\to B$ do

4: if $y_{i}\in$ TC then

5: $S=S+1$

6: Update $R$ following Equation 3

7: end if

8: end for

9: if $S=0$ then

10: return 0

11: else

12: return $\frac{R}{S}$

13: end if

We design a Tail-sample-mining-based feature margin regularization (TAIL)
approach to achieve this goal. Algorithm 1 describes its detailed mechanism.
Specifically, let
$\mathbf{F}^{p}=[\mathbf{f}_{1}^{p},\mathbf{f}_{2}^{p},\dots,\mathbf{f}_{B}^{p}]$
denote all probabilistic features of a batch containing $B$ samples, and
$\mathbf{y}=[y_{1},y_{2},\dots,y_{B}]$ denote the labels of the corresponding
feature representations. The class weights
$\mathbf{\Omega}=[\omega_{1},\omega_{2},\dots,\omega_{C}]$ are calculated
based on the smoothed inverse class frequency [22, 23], i.e.,
$\omega_{i}=\sqrt{\frac{\sum_{j}N_{j}}{N_{i}}}$, implying tail classes have
larger class weights than head classes. The core component of TAIL is the
computation of the regularization term $R$, which is updated for each
$y_{i}\in TC$ using the following equation:

$\displaystyle
R=R-\frac{1}{B}\sum_{j=1}^{B}(-1)^{\mathds{1}(y_{i}=y_{j})}(\omega_{i}+\omega_{j})\sum_{k=1}^{K}f^{p}_{j,k}\log\frac{f^{p}_{j,k}}{f^{p}_{i,k}}$
(3)

where $\mathds{1}(y_{i}=y_{j})$ is the indicator function (outputting 1 if
$y_{i}=y_{j}$, and 0 otherwise), and the initial value of $R$ is 0. In
Equation 3, we first compute the feature distribution differences using the
Kullback–Leibler divergence (KLD):
$\sum_{k=1}^{K}f^{p}_{j,k}\log\frac{f^{p}_{j,k}}{f^{p}_{i,k}}$, where
$f^{p}_{i,k}$ is the value of the $k$-th dimension in the probabilistic
feature $\mathbf{f}^{p}_{i}$ for the $i$-th sample. A larger KLD value means a
larger difference between the distributions of the feature embeddings of the
$i$-th and $j$-th samples. Hence, with the property of $R$, for each batch, we
can maximize the distributional differences between
$\mathcal{D}_{i}^{f},i\in{TC}$ and $\mathcal{D}_{j}^{f},j\neq i,j\in[C]$, and
minimize the distributional gap for samples from the same tail class. To
further enhance the influence of the regularization term among tail classes,
we assign a joint weight $(\omega_{i}+\omega_{j})$ to the feature pair
$(\mathbf{f}^{p}_{i},\mathbf{f}^{p}_{j})$. $\omega_{i}$ for tail samples is
bigger than that for head samples. To increase the distinction between pairs
of tail classes and non-tail classes and pairs of two tail classes, the joint
weight further strengthens the effect of the regularization for pairs of two
tail classes, thus improving the performance. Finally, we adopt the average
distance inside the batch: $\frac{R}{S}$.

Note that our regularization term TAIL is general and can be used with any
other long-tailed recognition loss function $L_{\mathrm{lt}}$ in the following
form:

$\displaystyle L=L_{\mathrm{lt}}+\mathrm{\textit{TAIL}}$

To summarize, the training pipeline of our REAT uses RBL to generate
adversarial examples, from which it trains the robust model with the loss
function $L$.

Losses Method Clean Accuracy PGD-20 PGD-100 CW-100 AA FL PGD-AT 53.58(0.81)
30.88(0.24) 30.85(0.25) 28.48(0.59) 27.00(0.60) REAT 55.22(1.42) 31.14(0.34)
31.08(0.33) 28.71(0.53) 27.23(0.56) EN PGD-AT 55.26(0.38) 31.82(0.36)
31.75(0.40) 29.91(0.27) 28.26(0.22) REAT 55.25(0.87) 32.20(0.25) 32.14(0.23)
30.12(0.33) 28.69(0.48) LDAM PGD-AT 52.74(0.71) 31.31(0.25) 31.24(0.23)
29.41(0.42) 28.03(0.37) REAT 53.47(1.04) 31.52(0.27) 31.52(0.27) 29.63(0.25)
28.20(0.21) BSL PGD-AT 66.99(0.17) 35.23(0.45) 35.01(0.43) 33.17(0.37)
31.15(0.49) REAT 67.33(0.45) 36.20(0.06) 36.02(0.09) 33.98(0.23) 32.08(0.12)

Table 1: Results on CIFAR-10-LT (UR=50) with different long-tailed recognition
losses. For this and the following tables, standard errors are shown inside
(). Best results are in bold.

Rebalancing Method Clean Accuracy PGD-20 PGD-100 CW-100 AA – PGD-AT
66.99(0.17) 35.23(0.45) 35.01(0.43) 33.17(0.37) 31.15(0.49) RW 66.82(0.40)
35.80(0.05) 35.65(0.08) 33.29(0.32) 31.40(0.31) RWS 67.28(0.63) 35.83(0.35)
35.70(0.37) 33.38(0.50) 31.50(0.67) ENR 66.53(0.91) 35.26(0.14) 35.08(0.12)
32.95(0.27) 31.05(0.13) BRW 67.98(0.09) 34.65(0.30) 34.47(0.32) 33.55(0.33)
31.36(0.40) RBL w/o TAIL 67.46(0.65) 35.59(0.18) 35.48(0.19) 33.51(0.39)
31.68(0.33) RBL w TAIL 67.33(0.45) 36.20(0.06) 36.02(0.09) 33.98(0.23)
32.08(0.12)

Table 2: Comparisons between different AE generation re-balancing strategies.
BSL loss is adopted.

##  5 Experiments

Datasets and Models. We evaluate REAT on CIFAR-10-LT and CIFAR-100-LT, which
are the mainstream datasets for evaluating long-tailed recognition tasks [6,
3, 24, 31]. Furthermore, we evaluate our method on a large dataset from the
real world, Tiny-Imagenet [18], in the supplementary materials. To generate
the unbalanced dataset, we follow the approach in [3] to set the unbalanced
ratio (UR) as {10, 20, 50, 100} for CIFAR-10-LT and {10, 20, 50} for
CIFAR-100-LT. We choose ResNet-18 (ResNet) [10] and WideResNet-28-10 (WRN)
[34] as the target models.

Baselines. We consider two baselines. The first one is to simply combine
existing adversarial training methods with various long-tailed recognition
losses. Our experiments and analysis in the supplementary materials show that
some adversarial training methods cannot converge well with the long-tailed
recognition loss, such as TRADES [35], AWP [31] and MART [28]. So we choose
the most effective one: PGD-AT [21]. The second baseline is RoBal [31].

Implementation. In our experiments, the number of training epochs is 80. The
learning rate is 0.1 at the beginning and decayed in epochs 60 and 75 with a
factor of 0.1. The weight decay is 0.0005. We adopt SGD to optimize the model
parameters with a batch size of 128. We save the model with the highest
robustness on the test set. For adversarial training, we adopt
$l_{\infty}$-norm PGD [21], with a maximum perturbation size $\epsilon=8/255$
for 10 iterations, and step length $\alpha=2/255$ in each iteration. For each
configuration, we report the mean and standard error under three repetitive
experiments with different random seeds. Training with REAT is efficient and
does not incur significant costs, as demonstrated in the supplementary
materials.

Attacks. We mainly consider the $l_{\infty}$-norm attacks to evaluate the
model’s robustness. The results under the $l_{2}$-norm attacks can be found in
the supplementary materials. We choose four representative attacks: PGD attack
[21] with the cross-entropy loss under 20 and 100 steps (PGD-20 and PGD-100),
PGD attack with the C&W loss [4] under 100 steps (CW-100), and AutoAttack [5]
(AA).

###  5.1 Ablation Studies

Impact of Long-tailed Recognition Losses. REAT is general and can be combined
with different long-tailed recognition losses. We select four state-of-the-art
losses and add each one with TAIL to evaluate REAT: focal loss (FL) [19],
effective number loss (EN) [6], label-distribution-aware margin loss (LDAM)
[3], and balanced softmax loss (BSL) [24]. For comparisons, we also choose
PDG-AT and replace the original cross-entropy loss with the above long-tailed
recognition loss for model parameter optimization.

Table 1 shows the comparison results with ResNet-18 and CIFAR-10-LT (UR=50).
We obtain two observations. (1) BSL loss can significantly outperform other
long-tailed recognition losses for clean accuracy as well as adversarial
robust accuracy against different attacks. So in the rest of our paper, we
mainly adopt this choice for evaluations. (2) REAT achieves better adversarial
robustness than PGD-AT for whatever loss function is adopted to train the
model. Furthermore, the clean accuracy is improved in most cases when REAT is
used. Therefore, we conclude that REAT has strong generalization and
applicability to different recognition losses.

Impact of AE Generation Re-balancing Losses. We then compare the effectiveness
of our RBL with various rebalancing methods adopted in the AE generation
process. We replace the cross-entropy with four SOTA rebalancing strategies:
(1) ReWeight (RW) [12, 29]; (2) ReWeight Smooth (RWS) [22, 23]; (3) Effective
Number Reweight (ENR) [6]; (4) Balanced Softmax ReWeight (BRW) [24].

Table 2 shows the comparison results on CIFAR-10-LT (UR=50) with the ResNet-18
model structure. We observe that the considered strategies can indeed increase
the clean accuracy and robustness of the final models by re-balancing the
generated AEs. Particularly, our RBL outperforms other approaches, giving
better robustness under different attacks. Furthermore, our dynamic effective
number achieves better results than the original effective number
implementation (ENR), in which the model adopts labels of the clean data to
balance the AE generation. This is because our adaptive effective number
allows the samples to equally learn features of both head and tail classes,
and makes the model obtain more marginal benefit from the AEs. By combining
RBL and TAIL, REAT achieves the best results under various attacks, which
proves the effectiveness of the AE re-balancing and feature distribution
alignment strategy.

###  5.2 Evaluation under Various Settings

UR Method Clean Accuracy PGD-20 PGD-100 CW-100 AA 10 RoBal 75.33(0.39)
45.98(0.39) Adaptive: 41.25 45.97(0.39) Adaptive: 41.13 41.02(0.02)
39.30(0.10) REAT 75.20(0.03) 42.97(0.17) 42.76(0.19) 41.52(0.22) 39.25(0.21)
20 RoBal 71.92(0.62) 43.23(0.25) Adaptive: 38.45 43.19(0.22) Adaptive: 38.20
38.23(0.07) 36.17(0.28) REAT 72.73(0.50) 40.57(0.15) 40.41(0.12) 38.55(0.29)
36.53(0.21) 50 RoBal 66.08(0.69) 38.46(0.18) Adaptive: 33.54 38.44(0.11)
Adaptive: 33.20 33.90(1.72) 31.14(0.44) REAT 67.33(0.45) 36.20(0.06)
36.02(0.09) 33.98(0.23) 32.08(0.12) 100 RoBal 60.11(0.62) 36.08(0.18)
Adaptive: 30.55 36.05(0.20) Adaptive: 30.36 30.57(0.77) 28.64(0.28) REAT
63.92(0.68) 32.84(0.07) 32.69(0.15) 30.73(0.38) 28.90(0.33)

Table 3: Results on CIFAR-10-LT with different values of UR. Red numbers
represent the results under our adaptive attack.

UR Method Clean Accuracy PGD-20 PGD-100 CW-100 AA 10 RoBal 43.47(0.31)
20.55(0.20) Adaptive: 18.49 20.49(0.20) Adaptive: 18.31 18.12(0.23)
16.86(0.10) REAT 45.94(0.15) 19.26(0.18) 19.16(0.18) 17.99(0.09) 16.58(0.06)
20 RoBal 39.58(0.40) 17.73(0.11) Adaptive: 16.00 17.71(0.08) Adaptive: 15.93
15.58(0.11) 14.55(0.10) REAT 41.98(0.21) 16.84(0.10) 16.72(0.12) 15.77(0.23)
14.45(0.20) 50 RoBal 34.24(0.54) 14.77(0.09) Adaptive: 13.22 14.77(0.10)
Adaptive: 13.18 12.77(0.10) 12.02(0.08) REAT 37.43(0.37) 14.25(0.22)
14.18(0.26) 13.38(0.15) 12.32(0.17)

Table 4: Results on CIFAR-100-LT with different URs. Red numbers represent the
results under our adaptive attack.

Method Clean Accuracy PGD-20 PGD-100 CW-100 AA ResNet RoBal 66.08(0.69)
38.46(0.18) Adaptive: 33.54 38.44(0.11) Adaptive: 33.20 33.90(1.72)
31.14(0.44) REAT 67.33(0.45) 36.20(0.06) 36.02(0.09) 33.98(0.23) 32.08(0.12)
WRN RoBal 69.33(0.11) 39.97(0.30) Adaptive: 34.58 39.98(0.32) Adaptive: 34.26
34.83(0.21) 33.09(0.41) REAT 72.58(0.31) 36.53(0.31) 36.35(0.32) 35.30(0.37)
33.37(0.37)

Table 5: Results on CIFAR-10-LT (UR=50) with different model structures. Red
numbers represent the results under our adaptive attack.

(a) Acc. of attack steps.

(b) Acc. of each class.

Figure 4: More comparisons results between REAT and RoBal for clean accuracy
and adversarial robustness.

(c) RoBal

(d) REAT

Figure 5: Distributions of model predictions for AEs during training. Clean
label distributions are shown by gray bars.

(a) RoBal

(b) REAT

Figure 6: The distribution of model predictions for AEs in Epoch {10, 30, 60}
on CIFAR-10-LT (UR=50).

To the best of our knowledge, RoBal [31] is the only work specifically
focusing on adversarial training on unbalanced datasets. As analyzed in
Section 2.2, there are several limitations in RoBal. Besides, we find that the
scale-invariant classification layer in RoBal can cause gradient vanishing
when generating AEs with the cross-entropy loss. It is because the normalized
weights of the classification layer and the normalized features greatly reduce
the scale of the gradients, making it fail to generate powerful AEs. We
propose a simple adaptive attack to break the gradient vanishing and
invalidate RoBal: the adversary can multiply the output logits by a factor (10
in all cases) when generating AEs, and then use these AEs to attack RoBal.
This can significantly decrease the adversarial robustness of the trained
model.

We perform experiments to compare RoBal and REAT from different perspectives,
as shown in Tables 3 and 4. We adopt the CIFAR-10-LT and CIFAR-100-LT with
ResNet-18 settings, respectively. In Table 5, we compare the results under
different model structures on CIFAR-10. More results with different
configurations can be found in the supplementary materials. First, for PGD-
based attacks, we show that the model robustness partially originates from the
gradient vanishing, and our adaptive attack can successfully break this
effort. CW attack and AA can break the gradient obfuscation in the
classification layer to some degree, due to the different loss functions in
the AE generation process. Therefore, we do not design adaptive attacks for
them. Second, comparing the results of RoBal and REAT under different values
of UR, REAT can achieve higher clean accuracy and robustness, especially with
larger UR. Third, for different model architectures, we prove that REAT
outperforms RoBal consistently. This indicates REAT is a better training
strategy for highly unbalanced datasets.

Furthermore, Figure 4(a) illustrates the robust accuracy of ResNet-18 on
CIFAR-10-LT (UR=50) under different numbers of PGD attack steps. It proves
that our REAT outperforms RoBal under all attack budgets. Figure 4(b) plots
the accuracy of ResNet-18 on CIFAR-10-LT (UR=50) under the PGD-20 attack for
each class. Our REAT can achieve higher clean accuracy and robust accuracy on
“body” classes, which will be explained below. More results can be found in
the supplementary materials.

Interpretation. We perform an in-depth analysis of the comparisons between
RoBal and REAT. We show the distributions of the predicted labels of AEs
during adversarial training for these two approaches in Figure 5 and 6. We
choose the configurations of CIFAR-10-LT (UR=50) and ResNet-18. Results under
other configurations can be found in the supplementary materials. For RoBal,
we observe that there are fewer AEs classified into body classes and more AEs
classified into tail classes, indicating that RoBal makes the model pay more
attention to head and tail classes while overlooking the body classes. In
contrast, REAT treats the body and tail classes more equally, and this is one
reason to achieve better performance on the “body” classes. Furthermore, we
plot the feature embedding space with the t-SNE tool in the supplementary
materials for feature-level comparison.

Summary. First, from the results, our solution brings more benefit than RoBal,
as shown in Table 3. RoBal can sometimes hurt the robustness and clean
accuracy. Second, we prove the robustness from RoBal partially depends on the
gradient obfuscation and can be defeated by an adaptive attack. Third, through
the results of PGD-AT with BSL loss, RoBal, and REAT, we find that compared
with improving the robustness, it is easier to enhance the clean accuracy,
which is consistent with the conclusion from [31]. To explore the reason
behind this phenomenon, we analyze the difficulty and challenges of
adversarial training on long-tailed datasets in the supplementary materials.
All in all, improving robustness requires more data, and the number of data in
the tail classes is not enough to train a model with high robustness, which is
a big challenge in adversarial training. How to further improve it is our
future work.

##  6 Conclusion

In this paper, we propose REAT, a new long-tailed adversarial training
framework to improve the training performance on unbalanced datasets. Our
method is inspired by our theoretical analysis of the lower bound of robust
risk when training on a long-tailed dataset. Our analysis shows that the label
distribution of adversarial examples and the feature representations can be
key points in reducing robust risk. Specifically, we present two novel
components, RBL for promoting the model to generate balanced AEs, and a
regularization term TAIL for forcing the model to assign larger feature spaces
for tail classes. With these techniques, REAT helps models achieve state-of-
the-art results and outperforms existing solutions on different datasets and
model structures. There still exists a robustness gap between the ideal result
obtained in the balanced setting and our approach. In the future, we aim to
keep reducing this gap with more advanced solutions, e.g., new robust network
structures or training loss functions.

## References

  * Athalye et al. [2018] Anish Athalye, Nicholas Carlini, and David A. Wagner.  Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examples.  In _Proc. of the ICML_ , pages 274–283, 2018.
  * Buda et al. [2018] Mateusz Buda, Atsuto Maki, and Maciej A Mazurowski.  A systematic study of the class imbalance problem in convolutional neural networks.  _Neural networks_ , 106:249–259, 2018.
  * Cao et al. [2019] Kaidi Cao, Colin Wei, Adrien Gaidon, Nikos Aréchiga, and Tengyu Ma.  Learning Imbalanced Datasets with Label-Distribution-Aware Margin Loss.  In _Proc. of the NeurIPS_ , pages 1565–1576, 2019.
  * Carlini and Wagner [2017] Nicholas Carlini and David Wagner.  Towards Evaluating the Robustness of Neural Networks.  In _Proc. of the SP_ , pages 39–57, 2017.
  * Croce and Hein [2020] Francesco Croce and Matthias Hein.  Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks.  In _Proc. of the ICML_ , pages 2206–2216, 2020.
  * Cui et al. [2019] Yin Cui, Menglin Jia, Tsung-Yi Lin, Yang Song, and Serge J. Belongie.  Class-Balanced Loss Based on Effective Number of Samples.  In _Proc. of the CVPR_ , pages 9268–9277, 2019.
  * Goodfellow et al. [2015] Ian J. Goodfellow, Jonathon Shlens, and Christian Szegedy.  Explaining and Harnessing Adversarial Examples.  In _Proc. of the ICLR_ , 2015.
  * Han et al. [2005] Hui Han, Wenyuan Wang, and Binghuan Mao.  Borderline-SMOTE: A New Over-Sampling Method in Imbalanced Data Sets Learning.  In _Proc. of the ICIC_ , pages 878–887, 2005.
  * He and Garcia [2009] Haibo He and Edwardo A Garcia.  Learning from imbalanced data.  _IEEE Transactions on knowledge and data engineering_ , 21(9):1263–1284, 2009.
  * He et al. [2016] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun.  Deep Residual Learning for Image Recognition.  In _Proc. of the CVPR_ , pages 770–778, 2016.
  * Hong et al. [2021] Youngkyu Hong, Seungju Han, Kwanghee Choi, Seokjun Seo, Beomsu Kim, and Buru Chang.  Disentangling Label Distribution for Long-Tailed Visual Recognition.  In _Proc. of the CVPR_ , pages 6626–6636, 2021.
  * Huang et al. [2016] Chen Huang, Yining Li, Chen Change Loy, and Xiaoou Tang.  Learning Deep Representation for Imbalanced Classification.  In _Proc. of the CVPR_ , pages 5375–5384, 2016.
  * Huang et al. [2020] Lang Huang, Chao Zhang, and Hongyang Zhang.  Self-Adaptive Training: beyond Empirical Risk Minimization.  In _Proc. of the NeurIPS_ , 2020.
  * Japkowicz and Stephen [2002] Nathalie Japkowicz and Shaju Stephen.  The class imbalance problem: A systematic study.  _Intelligent data analysis_ , 6(5):429–449, 2002.
  * Kang et al. [2020] Bingyi Kang, Saining Xie, Marcus Rohrbach, Zhicheng Yan, Albert Gordo, Jiashi Feng, and Yannis Kalantidis.  Decoupling Representation and Classifier for Long-Tailed Recognition.  In _Proc. of the ICLR_ , 2020.
  * Kang et al. [2021] Bingyi Kang, Yu Li, Sa Xie, Zehuan Yuan, and Jiashi Feng.  Exploring Balanced Feature Spaces for Representation Learning.  In _Proc. of the ICLR_ , 2021.
  * Khan et al. [2019] Salman H. Khan, Munawar Hayat, Syed Waqas Zamir, Jianbing Shen, and Ling Shao.  Striking the Right Balance With Uncertainty.  In _Proc. of the CVPR_ , pages 103–112, 2019.
  * Le and Yang [2015] Ya Le and Xuan Yang.  Tiny imagenet visual recognition challenge.  _CS 231N_ , 7(7):3, 2015.
  * Lin et al. [2017] Tsung-Yi Lin, Priya Goyal, Ross B. Girshick, Kaiming He, and Piotr Dollár.  Focal Loss for Dense Object Detection.  In _Proc. of the ICCV_ , pages 2999–3007, 2017.
  * Liu et al. [2009] Xu-Ying Liu, Jianxin Wu, and Zhi-Hua Zhou.  Exploratory Undersampling for Class-Imbalance Learning.  _IEEE Transactions on Systems, Man, and Cybernetics, Part B_ , 39(2):539–550, 2009.
  * Madry et al. [2018] Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu.  Towards Deep Learning Models Resistant to Adversarial Attacks.  In _Proc. of the ICLR_ , 2018.
  * Mahajan et al. [2018] Dhruv Mahajan, Ross B. Girshick, Vignesh Ramanathan, Kaiming He, Manohar Paluri, Yixuan Li, Ashwin Bharambe, and Laurens van der Maaten.  Exploring the Limits of Weakly Supervised Pretraining.  In _Proc. of the ECCV_ , pages 185–201, 2018.
  * Mikolov et al. [2013] Tomás Mikolov, Ilya Sutskever, Kai Chen, Gregory S. Corrado, and Jeffrey Dean.  Distributed Representations of Words and Phrases and their Compositionality.  In _Proc. of the NeurIPS_ , pages 3111–3119, 2013.
  * Ren et al. [2020] Jiawei Ren, Cunjun Yu, Shunan Sheng, Xiao Ma, Haiyu Zhao, Shuai Yi, and Hongsheng Li.  Balanced Meta-Softmax for Long-Tailed Visual Recognition.  In _Proc. of the NeurIPS_ , 2020.
  * Rice et al. [2020] Leslie Rice, Eric Wong, and J. Zico Kolter.  Overfitting in adversarially robust deep learning.  In _Proc. of the ICML_ , pages 8093–8104, 2020.
  * Tang et al. [2020] Kaihua Tang, Jianqiang Huang, and Hanwang Zhang.  Long-Tailed Classification by Keeping the Good and Removing the Bad Momentum Causal Effect.  In _Proc. of the NeurIPS_ , 2020.
  * Wang et al. [2020a] Tao Wang, Yu Li, Bingyi Kang, Junnan Li, Jun Hao Liew, Sheng Tang, Steven C. H. Hoi, and Jiashi Feng.  The Devil Is in Classification: A Simple Framework for Long-Tail Instance Segmentation.  In _Proc. of the ECCV_ , pages 728–744, 2020a.
  * Wang et al. [2020b] Yisen Wang, Difan Zou, Jinfeng Yi, James Bailey, Xingjun Ma, and Quanquan Gu.  Improving Adversarial Robustness Requires Revisiting Misclassified Examples.  In _Proc. of the ICLR_ , 2020b.
  * Wang et al. [2017] Yu-Xiong Wang, Deva Ramanan, and Martial Hebert.  Learning to Model the Tail.  In _Proc. of the NeurIPS_ , pages 7029–7039, 2017.
  * Wu et al. [2020a] Dongxian Wu, Shu-Tao Xia, and Yisen Wang.  Adversarial Weight Perturbation Helps Robust Generalization.  In _Proc. of the NeurIPS_ , 2020a.
  * Wu et al. [2021] Tong Wu, Ziwei Liu, Qingqiu Huang, Yu Wang, and Dahua Lin.  Adversarial Robustness Under Long-Tailed Distribution.  In _Proc. of the CVPR_ , pages 8659–8668, 2021.
  * Wu et al. [2020b] Tz-Ying Wu, Pedro Morgado, Pei Wang, Chih-Hui Ho, and Nuno Vasconcelos.  Solving Long-Tailed Recognition with Deep Realistic Taxonomic Classifier.  In _Proc. of the ECCV_ , pages 171–189, 2020b.
  * Yin et al. [2019] Xi Yin, Xiang Yu, Kihyuk Sohn, Xiaoming Liu, and Manmohan Chandraker.  Feature Transfer Learning for Face Recognition With Under-Represented Data.  In _Proc. of the CVPR_ , pages 5704–5713, 2019.
  * Zagoruyko and Komodakis [2016] Sergey Zagoruyko and Nikos Komodakis.  Wide Residual Networks.  In _Proc. of the BMVC_ , 2016.
  * Zhang et al. [2019] Hongyang Zhang, Yaodong Yu, Jiantao Jiao, Eric P. Xing, Laurent El Ghaoui, and Michael I. Jordan.  Theoretically Principled Trade-off between Robustness and Accuracy.  In _Proc. of the ICML_ , pages 7472–7482, 2019.
  * Zheng et al. [2020] Haizhong Zheng, Ziqi Zhang, Juncheng Gu, Honglak Lee, and Atul Prakash.  Efficient adversarial training with transferable adversarial examples.  In _Proc. of the CVPR_ , pages 1178–1187, 2020.

##  Appendix A Related Works

###  A.1 Long-tailed Recognition

Long-tailed learning means training a machine learning model on a dataset that
follows a long-tailed distribution. It has been applied to various scenarios
including classification tasks [24], object detection tasks [19] and
segmentation tasks [27]. To alleviate the uneven distribution of data in the
dataset, i.e., the majority of the data belong to the head classes, while the
data belonging to the tail classes are insufficient, many methods have been
proposed, which can be roughly divided into four categories:re-sampling, cost-
sensitive learning, training phase decoupling and classifier designing.

The re-sampling methods can be divided into four classes, i.e., random under-
sampling head classes [20], random over-sampling tail classes [8], class-
balanced re-sampling [24] and scheme-oriented sampling [12]. These methods
solve the unbalance problem by using sampling strategies to generate desired
balanced distributions.

The cost-sensitive learning methods have two types of applications, i.e.,
class-level re-weighting [11, 6, 19] and class-level re-margining [3, 17, 31].
It assigns different weights to each class or adjust the minimal margin
between the features and the classifier to balance the learning difficulties,
achieving better performance under unbalanced data distributions.

The training phase decoupling is used to improve both the feature extractor
and classifier. [15] find that training the feature extractor with instance-
balanced re-sampling strategy and re-adjusting the classifier can
significantly improve the accuracy in long-tailed recognition. [16] further
observe that a balanced feature space benefits the long-tailed recognition.

The classifier designing aims to address the biases that the weight norms for
head classes are larger than them of tail classes [33] in the traditional
layers under long-tailed datasets. [15] propose a normalized classification
layer to re-balance the weight norms for all classes. [31] also adopt a
normalized classifier to defend against adversarial attacks. [32] propose a
hierarchical classifier mapping the images into a class taxonomic tree
structure. [26] propose a classifier with causal inference to better stabilize
the gradients. Note that modifying the classifier can indeed improve the
performance of models on unbalanced data. However, we argue that it may
introduce gradient obfuscation resulting in adaptive adversarial attacks. For
more details, please refer to Section 5 in the main paper.

###  A.2 Adversarial Training

Adversarial training [21, 35, 28, 25] is widely studied to defend against
adversarial attacks. Its basic idea is to generate on-the-fly AEs to augment
the training set. It can be formulated as the following min-max problem [21]:

$\displaystyle\min_{\theta}\max_{x^{*}}\ell(x^{*},y;\theta)$

where $x^{*}$ is the training sample generated from a clean one $x$ to
maximize the loss function $\ell(\cdot)$, $y$ is the ground-truth label,
$\theta$ is the model parameters. The first phase (maximization optimization)
is to generate samples maximizing the loss function. The second stage
(minimization optimization) is to optimize the model parameter $\theta$ to
minimize the loss function under samples generated in phase one.

In previous works, there are three main research topics in adversarial
training, i.e., improving the model robustness [21, 28], reducing the gap
between clean accuracy and robustness [35, 30] and addressing overfitting
challenges [25, 13].

In this paper, we focus on adversarial training on datasets with long-tailed
distributions. To our best knowledge, [31] present the first work dedicated to
improving the accuracy as well as robustness to tail class during adversarial
training. They design a new loss function and cosine classifier to achieve
this. However, we experimentally demonstrate the unsatisfactory security and
performance of this work in Section 5.2, which motivates us to design more
secure and satisfactory adversarial training methods tailored to datasets that
obey long-tailed distributions.

##  Appendix B Proof

###### Theorem 2

Given a function $f_{\theta}$ and dataset $\mathcal{D}$, which we defined
above, let

$\displaystyle
g(x)\in\mathrm{argmax}_{x^{\prime}\in\mathcal{B}_{p}(x,\epsilon)}\mathbbm{1}\\{F_{\theta}(x)\neq
F_{\theta}(x^{\prime})\\}$

Assume that a larger $n_{i}$ will decrease the
$\mathcal{R}^{i}_{\mathrm{rob}}(\theta)$, and vice versa. Then, we have

$\displaystyle\mathcal{R}_{\mathrm{rob}}(\theta)\geq\sum_{i=1}^{C}\frac{1}{C}(\sum_{j=1}^{n_{i}}\mathbbm{1}\\{F_{\theta}(x_{i}^{j})\neq
i\\}$
$\displaystyle+\sum_{j=1}^{n_{i}}\mathbbm{1}\\{F_{\theta}(g(x_{i}^{j}))\neq
F_{\theta}(x_{i}^{j})\\})$

###### Proof 1

We have

$\displaystyle\mathcal{R}^{i}_{\mathrm{bdy}}(\theta)=$
$\displaystyle\sum_{j=1}^{n_{i}}\mathbbm{1}\\{\exists{x^{\prime}\in\mathcal{B}_{p}(x_{i}^{j},\epsilon)},F_{\theta}(x^{\prime})\neq
i\\}p(i)$ $\displaystyle\geq$
$\displaystyle\sum_{j=1}^{n_{i}}(\mathbbm{1}\\{F_{\theta}(g(x_{i}^{j}))\neq
F_{\theta}(x_{i}^{j}),F_{\theta}(x_{i}^{j})=i\\}$
$\displaystyle+\mathbbm{1}\\{F_{\theta}(g(x_{i}^{j}))\neq
F_{\theta}(x_{i}^{j}),F_{\theta}(x_{i}^{j})\neq i\\})p(i)$ $\displaystyle=$
$\displaystyle\sum_{j=1}^{n_{i}}(\mathbbm{1}\\{F_{\theta}(g(x_{i}^{j}))\neq
F_{\theta}(x_{i}^{j})\\}\mathbbm{1}\\{F_{\theta}(x_{i}^{j})=i\\}$
$\displaystyle+\mathbbm{1}\\{F_{\theta}(g(x_{i}^{j}))\neq
F_{\theta}(x_{i}^{j})\\}\mathbbm{1}\\{F_{\theta}(x_{i}^{j})\neq i\\})p(i)$

Let $\mathcal{A}=\mathbbm{1}\\{F_{\theta}(x_{i}^{j})\neq i\\}$ and
$\mathcal{B}=\mathbbm{1}\\{F_{\theta}(g(x_{i}^{j}))\neq
F_{\theta}(x_{i}^{j})\\}$, we have

$\displaystyle\mathcal{R}_{\mathrm{rob}}(\theta)\geq$
$\displaystyle\sum_{i=1}^{C}p(i)\left(\sum_{j=1}^{n_{i}}\mathcal{A}+\mathcal{B}(1-\mathcal{A})+\mathcal{B}\mathcal{A}\right)$
$\displaystyle=$
$\displaystyle\sum_{i=1}^{C}p(i)\left(\sum_{j=1}^{n_{i}}\mathcal{A}+\mathcal{B}\right)$

Based on the assumption and Chebyshev inequality, we have

$\displaystyle\mathcal{R}_{\mathrm{rob}}(\theta)\geq$
$\displaystyle\sum_{i=1}^{C}\frac{1}{C}\left(\sum_{j=1}^{n_{i}}\mathcal{A}+\mathcal{B}\right)$
$\displaystyle=$
$\displaystyle\sum_{i=1}^{C}\frac{1}{C}(\sum_{j=1}^{n_{i}}\mathbbm{1}\\{F_{\theta}(x_{i}^{j})\neq
i\\}$ $\displaystyle+$
$\displaystyle\sum_{j=1}^{n_{i}}\mathbbm{1}\\{F_{\theta}(g(x_{i}^{j}))\neq
F_{\theta}(x_{i}^{j})\\})$

##  Appendix C Evaluation on Real-World Dataset

We consider a real-world dataset, Tiny-Imagenet, to evaluate our method.
Because RoBal does not provide the hyperparameters for this dataset, we
compare REAT with PGD-AT method. We randomly downsample the training set, to
obtain a long-tailed subset. In detail, the number of data in each class
decreases linearly. In our subset, the UR is 100, which means the class with
the most data contains 500 images, and the class with the fewest data contains
5 images. We train a ResNet-18 on this subset with PGD-AT and REAT,
respectively. The results can be found in Table 6. From the results, we can
find that our method performs better than the baseline method with about 2% 3%
improvement on clean accuracy and robust accuracy. Therefore, our approach is
a better choice for solving real-world long-tail challenges.

Model Clean Accuracy PGD-20 PGD-100 CW-100 AA PGD-AT 40.29 30.66 30.65 29.10
28.73 REAT 43.09 33.00 33.00 31.16 30.74

Table 6: Evaluation on the real-world dataset, Tiny-Imagenet. The attack
budget is $\epsilon=2/255$.

##  Appendix D Adversarial Training on Unbalanced Dataset

Method Clean Accuracy PGD-20 PGD-100 CW-100 AA PGD-AT 51.28(1.34) 29.57(0.17)
29.47(0.15) 29.05(0.07) 27.71(0.15) TRADES 45.55(0.89) 28.24(0.21) 28.21(0.20)
27.29(0.17) 26.78(0.10) PGD-AWP 36.45(3.40) 26.52(1.64) 26.47(1.62)
26.04(1.54) 25.23(1.47) TRADES-AWP 41.30(0.47) 27.04(0.08) 27.00(0.07)
25.65(0.08) 25.37(0.07) MART 41.76(0.63) 29.18(0.12) 29.14(0.13) 27.04(0.06)
26.06(0.03)

Table 7: Results on CIFAR-10-LT (UR=50) with different training strategies.

To explore the effectiveness of adversarial training strategies proposed on
balanced datasets, we compare recent adversarial training methods in Table 7.
The results indicate that improving robustness on a balanced dataset is non-
trivial, but these improvements cannot be expressed under an unbalanced
dataset. Furthermore, we find that the simplest and the most straightforward
method, PGD-AT, obtains the best results. On the other hand, methods adopting
clean samples to train models, like TRADES and MART, will achieve lower clean
accuracy, as the unbalanced data will harm the model’s accuracy on the
balanced test set.

In Figure 7, the t-SNE results prove that each class is assigned an area of a
similar size in the feature space when the model is trained on balanced data.
But, if the model is trained on unbalanced data, the areas for head classes
expand and encroach areas that should belong to tail classes, causing the area
of tail features to shrink, which represents the unbalanced feature embedding
space. As a result, the performance and generalizability for tail classes
decrease.

To alleviate the unbalance problem, we replace the cross-entropy loss in
TRADES and MART with Balanced Softmax Loss (BSL). However, in our experiments,
we find that BSL will make the model not converge. The reason can be that the
gradient directions of BSL and KL divergence are contradicted. So, in our
paper, we mainly consider enhancing the PGD-AT method to better fit the
unbalanced datasets.

(a) Balanced dataset

(b) Unbalanced dataset

Figure 7: AE’s feature t-SNE results for ResNet-18 trained with PGD-AT on
balanced and unbalanced CIFAR-10.

##  Appendix E Studying Data Hunger and Data Unbalance

UR Method Clean Accuracy PGD-20 PGD-100 CW-100 AA 10 PGD-AT (BS) 77.12(0.73)
44.73(0.20) 44.49(0.20) 43.81(0.26) 41.50(0.19) PGD-AT 75.27(0.32) 42.66(0.20)
42.36(0.20) 41.18(0.21) 38.81(0.10) REAT 75.20(0.03) 42.97(0.17) 42.76(0.19)
41.52(0.22) 39.25(0.21) 20 PGD-AT (BS) 75.61(0.10) 43.37(0.15) 43.22(0.17)
42.12(0.12) 40.01(0.03) PGD-AT 72.31(0.24) 39.79(0.31) 39.61(0.30) 38.42(0.06)
36.18(0.03) REAT 72.73(0.50) 40.57(0.15) 40.41(0.12) 38.55(0.29) 36.53(0.21)
50 PGD-AT (BS) 72.98(0.74) 41.14(0.26) 40.89(0.30) 39.92(0.49) 37.75(0.40)
PGD-AT 66.99(0.17) 35.23(0.45) 35.01(0.43) 33.17(0.37) 31.15(0.49) REAT
67.33(0.45) 36.20(0.06) 36.02(0.09) 33.98(0.23) 32.08(0.12) 100 PGD-AT (BS)
72.83(0.53) 40.25(0.35) 40.10(0.45) 39.29(0.19) 37.24(0.18) PGD-AT 62.70(0.52)
32.91(0.17) 32.73(0.19) 30.45(0.15) 28.60(0.21) REAT 63.92(0.68) 32.84(0.07)
32.69(0.15) 30.73(0.38) 28.90(0.33)

Table 8: Results on CIFAR-10 and CIFAR-10-LT with different URs. BSL loss is
adopted for PGD-AT and REAT.

UR Method Clean Accuracy PGD-20 PGD-100 CW-100 AA 10 PGD-AT (BS) 48.32(0.50)
20.08(0.24) 19.95(0.25) 18.88(0.28) 17.44(0.21) PGD-AT 45.96(0.49) 18.85(0.19)
18.73(0.17) 17.70(0.13) 16.21(0.13) REAT 45.94(0.15) 19.26(0.18) 19.16(0.18)
17.99(0.09) 16.58(0.06) 20 PGD-AT (BS) 45.14(0.28) 17.95(0.19) 17.82(0.17)
17.15(0.19) 15.80(0.23) PGD-AT 42.45(0.53) 16.36(0.13) 16.24(0.14) 15.47(0.17)
14.17(0.09) REAT 41.98(0.21) 16.84(0.10) 16.72(0.12) 15.77(0.23) 14.45(0.20)
50 PGD-AT (BS) 42.86(0.37) 16.52(0.26) 16.38(0.24) 15.86(0.14) 14.54(0.05)
PGD-AT 37.70(0.12) 13.95(0.07) 13.86(0.05) 13.17(0.11) 12.10(0.02) REAT
37.43(0.37) 14.25(0.22) 14.18(0.26) 13.38(0.15) 12.32(0.17)

Table 9: Results on CIFAR-100 and CIFAR-100-LT with different URs. BSL loss is
adopted for PGD-AT and REAT.

Method Clean Accuracy PGD-20 PGD-100 CW-100 AA ResNet PGD-AT (BS) 72.98(0.74)
41.14(0.26) 40.89(0.30) 39.92(0.49) 37.75(0.40) PGD-AT 66.99(0.17) 35.23(0.45)
35.01(0.43) 33.17(0.37 31.15(0.49) REAT 67.33(0.45) 36.20(0.06) 36.02(0.09)
33.98(0.23) 32.08(0.12) WRN PGD-AT (BS) 78.65(0.15) 42.42(0.33) 42.05(0.32)
42.21(0.08) 39.86(0.37) PGD-AT 72.38(0.30) 35.93(0.10) 35.64(0.04) 34.93(0.14)
32.84(0.19) REAT 72.58(0.31) 36.53(0.31) 36.35(0.32) 35.30(0.37) 33.37(0.37)

Table 10: Results on CIFAR-10 and CIFAR-10-LT (UR=50) with different model
structures. BSL loss is adopted for PGD-AT and REAT.

Method Clean Accuracy PGD-20 PGD-100 CW-100 AA ResNet PGD-AT (BS) 48.32(0.50)
20.08(0.24) 19.95(0.25) 18.88(0.28) 17.44(0.21) PGD-AT 45.96(0.49) 18.85(0.19)
18.73(0.17) 17.70(0.13) 16.21(0.13) REAT 45.94(0.15) 19.26(0.18) 19.16(0.18)
17.99(0.09) 16.58(0.06) WRN PGD-AT (BS) 52.33(0.42) 21.95(0.10) 21.77(0.14)
21.41(0.20) 19.58(0.17) PGD-AT 50.07(0.25) 20.79(0.39) 20.69(0.38) 20.17(0.27)
18.32(0.28) REAT 49.99(0.18) 20.85(0.16) 20.71(0.20) 20.18(0.09) 18.35(0.17)

Table 11: Results on CIFAR-100 and CIFAR-100-LT (UR=10) with different model
structures. BSL loss is adopted for PGD-AT and REAT.

In this part, we further examine the effects of the data hunger and the data
unbalance on the model robustness, which is explored to construct an
experimental upper bound on the robustness of the long-tailed adversarial
training methods. To be specific, the data hunger raises from the insufficient
data from the body classes and tail classes, which is one of the impacts of
the long-tailed datasets. And another one is the data unbalance. To
exclusively study the data hunger in a balanced dataset, for a given
unbalanced ratio, we sample the same number of samples as the long-tail
dataset but form them into balanced small (BS) datasets. We then train models
on this dataset with PGD-AT to learn an experimental upper bound, which is
represented as “PGD-AT (BS)” in our experiments. When we train models with
PGD-AT (BS) the loss function used to optimize models is Cross-Entropy loss.
On the other hand, when we train models on unbalanced datasets, the basic loss
function used to optimize models is BSL.

Comparing the results of models trained under balanced datasets and unbalanced
datasets in Tables 8– 11, it is clear that the models train on unbalanced
datasets suffer from a bigger reduction when the number of training samples
decreases, which means that the data unbalance harms the model’s robustness in
a larger degree than the data hunger. Training models on unbalanced data is
more challenging than training models on small but balanced data under
adversarial scenarios for different model structures and datasets. It is
reasonable, because in the long-tailed datasets, there are fewer data in the
tail classes, making the model unable to learn much information for such
classes. Furthermore, compared with training a model on CIFAR-10, when
training a model on a more complex dataset, such as CIFAR-100, the performance
decrease is less than expected, which will be studied in our future work. On
the other hand, the experimental results of PGD-AT (BS) can be seen as upper
bounds for the models trained on same-size unbalanced datasets.

##  Appendix F Results under $l_{2}$-norm Attacks

UR Method Clean Accuracy PGD-20 PGD-100 CW 10 PGD-AT 75.27(0.32) 30.92(0.47)
28.82(0.63) 70.76(0.20) RoBal 75.33(0.39) 33.15/28.26(0.62) 31.43/26.18(0.80)
71.36(0.39) REAT 75.20(0.03) 30.82(0.60) 28.71(0.74) 71.02(0.18) 20 PGD-AT
72.31(0.24) 29.23(0.50) 27.24(0.55) 67.87(0.26) RoBal 71.92(0.62)
31.79/27.13(0.37) 30.34/25.38(0.44) 67.95(0.58) REAT 72.73(0.50) 29.27(0.58)
27.44(0.59) 68.13(0.43) 50 PGD-AT 66.99(0.17) 26.75(0.31) 25.20(0.31)
62.54(0.14) RoBal 66.08(0.69) 29.22/24.17(0.68) 27.97/22.83(0.60) 62.03(0.65)
REAT 67.33(0.45) 27.45(0.30) 25.91(0.40) 63.08(0.23) 100 PGD-AT 62.70(0.52)
24.91(0.46) 23.49(0.42) 58.06(0.40) RoBal 60.11(0.62) 27.77/23.33(0.46)
26.48/21.91(0.43) 56.34(0.28) REAT 63.92(0.68) 24.63(0.21) 23.24(0.21)
59.17(0.49)

Table 12: Results on CIFAR-10-LT with different URs under $l_{2}$-norm
attacks. Red numbers represent the results under our adaptive attack. BSL loss
is adopted for PGD-AT and REAT.

In Table 12, we show the results of models under $l_{2}$-norm attacks. For the
PGD attacks, the max perturbation size is $\epsilon=1.0$, and the step length
is $\alpha=0.2$. We consider the 20-step attack, PGD-20, and the 100-step
attack, PGD-100. For the C&W attack, we follow its official implementation.
The results confirm that our REAT can improve the model’s robustness under
different threat models. On the other hand, the gradient obfuscation is more
serious under $l_{2}$-norm attacks, so our adaptive attacks achieve better
results.

##  Appendix G Comparing with RoBal

Method Clean Accuracy PGD-20 PGD-100 CW-100 AA ResNet RoBal 43.47(0.31)
20.55(0.20) Adaptive: 18.49 20.49(0.20) Adaptive: 18.31 18.12(0.23)
16.86(0.10) REAT 45.94(0.15) 19.26(0.18) 19.16(0.18) 17.99(0.09) 16.58(0.06)
WRN RoBal 48.84(0.24) 21.45(0.18) Adaptive: 19.29 21.44(0.21) Adaptive: 19.19
19.71(0.09) 18.21(0.02) REAT 49.99(0.18) 20.85(0.16) 20.71(0.20) 20.18(0.09)
18.35(0.17)

Table 13: Results on CIFAR-100-LT (UR=10) with different model structures. Red
numbers represent the results under our adaptive attack. BSL loss is adopted
for REAT. Figure 8: Accuracy of Attack Steps on CIFAR-100-LT (UR=10). Figure
9: Accuracy of Each Class on CIFAR-100-LT (UR=10).

(a) PGD-AT: balanced

(b) PGD-AT (BSL)

(c) RoBal

(d) REAT (BSL)

Figure 10: AE’s feature map results with different strategies. (a) is trained
with the balanced dataset (CIFAR-10) while the rest three are trained with the
unbalanced dataset (CIFAR-10-LT, UR=50).

Varying Datasets. Similar to our main paper, we illustrate the robust accuracy
under the different numbers of PGD attack steps of ResNet-18 on CIFAR-100-LT
(UR=10) in Figure 8 in this section. The results prove that our REAT
outperforms RoBal under all attack budgets. In Figure 9, we plot the accuracy
of ResNet-18 on CIFAR-100-LT (UR=10) under the PGD-20 attack for each class.
The results indicate that our REAT can achieve higher clean accuracy and
robust accuracy on “body” classes, which is consistent with the conclusion in
the main paper.

Varying Model Structures. To show the superiority of REAT on different model
structures, we compare the results of RoBal and REAT on ResNet-18 and
WideResNet-28-10, respectively. The results in Table 13 prove that models
trained with REAT lead models trained with RoBal on both clean accuracy and
robustness, which means REAT is a better training strategy for different model
structures.

Interpretation. We choose the configurations of CIFAR-10-LT (UR=50) and
ResNet-18. Then, we plot the feature embedding space with the t-SNE tool for
models trained with different strategies in Figure 10. We first generate AEs
with the PGD-20 attack on the test set and use t-SNE to plot the feature
distribution for AEs. ResNet-18 is adopted as the model architecture. Figure
10(a) is the feature result for PGD-AT over the balanced dataset CIFAR-10. We
observe that samples from different classes are not quite overlapped with each
other in the feature space, making them easier to be classified. In contrast,
Figures 10(b) and 10(c) show the results for PGD-AT (BSL loss) and RoBal over
the unbalanced dataset CIFAR-10. We observe that there are more samples from
different classes entangled together in their feature embeddings, which can
harm the model’s robustness. Figure 10(d) shows the results of our REAT under
the same unbalanced setting. We can see the feature space is more similar to
the one obtained from the balanced dataset (Figure 10(a)). This explains the
effectiveness of REAT in enhancing the model robustness and clean accuracy
from the feature perspective.

##  Appendix H AE Prediction Distribution

(a) RoBal

(b) REAT

Figure 11: The distribution of model predictions for AEs during the training
process on CIFAR-100-LT (UR=10). Clean label distributions are shown by gray
bars.

In Figure 11, we compare the AE distribution on CIFAR-100-LT (UR=10), when
training models with RoBal and REAT, respectively. The results prove that
RoBal will cause unbalanced AE distribution when the number of classes
increases. There are more samples predicted as tail classes by the model.
However, our REAT can keep the balanced AE distribution and obtain better
results.

##  Appendix I Training Cost Overhead of REAT

We compare the training time overhead of REAT compared with PGD-AT (BSL is
adopted) method and RoBal on one single V100 GPU card. The results are shown
in Table 14. When we train a ResNet-18 on CIFAR-10-LT (UR=50), the training
time overhead for one epoch is about 8 seconds. When we train a ResNet-18 on
CIFAR-10-LT (UR=100), the training time overhead for one epoch is about 5
seconds. So, our REAT is efficient on long-tailed datasets and does not
increase too much training time.

Dataset UR Time Cost (Secs) PGD-AT RoBal REAT CIFAR-10-LT 50 46 50 54 100 42
45 47 CIFAR-100-LT 20 53 58 75 50 42 45 52

Table 14: Time cost (seconds) for one training epoch on CIFAR-10-LT and
CIFAR-100-LT. The model is ResNet-18. BSL loss is adopted for PGD-AT and REAT.