aainstitutetext:  School of Physics, Nankai University, Tianjin 300071,
Chinabbinstitutetext:  Deutsches Elektronen-Synchrotron DESY, Notkestr. 85,
22607 Hamburg, Germany

# Searching for heavy neutral lepton and lepton number violation through VBS
at high-energy muon colliders

Tong Li<EMAIL_ADDRESS>a,b Chang-Yuan Yao<EMAIL_ADDRESS>a Man
Yuan<EMAIL_ADDRESS>
###### Abstract

High-energy muon collider can play as an emitter of electroweak gauge bosons
and thus leads to substantial vector boson scattering (VBS) processes. In this
work, we investigate the production of heavy neutral lepton (HNL) $N$ and
lepton number violation (LNV) signature through VBS at high-energy muon
colliders. VBS induces LNV processes
$W^{\pm}Z/\gamma\to\ell^{\pm}N\to\ell^{\pm}\ell^{\pm}W^{\mp}\to\ell^{\pm}\ell^{\pm}q\bar{q}^{\prime}$
with an on-shell HNL $N$ at $\mu^{+}\mu^{-}$ colliders. In analogy to
neutrinoless double-beta decay with the HNL in t-channel, the LNV signature
$W^{+}W^{+}\to\ell^{+}\ell^{+}$ can also happen via VBS at same-sign muon
collider. They provide clean and robust LNV signatures to tell the nature of
Majorana HNLs and thus have more advantageous benefits than direct $\mu\mu$
annihilation. We analyze the potential of searching for Majorana HNL and
obtain the exclusion limits on mixing $V_{\ell N}$. Based on this same-sign
lepton signature, we also obtain the sensitivity of muon collider to the
Weinberg operator.

††preprint: DESY-23-092

##  1 Introduction

Neutrino oscillation experiments provide clear and compelling evidence that
neutrinos have non-zero, but very small masses. The Standard Model (SM)
includes only left-handed neutrino fields in the lepton doublets, and
therefore predicts neutrino masses of exactly zero. Although one can certainly
introduce right-handed (RH) neutrino field, the Yukawa coupling has to be
tuned to a very small constant $y_{\nu}\lesssim 10^{-13}$ to accommodate the
observed neutrino mass. A more economical way to generate neutrino mass in the
SM content is through the so-called “Weinberg operator” Weinberg:1979sa

$\displaystyle\ell_{L}\ell_{L}HH\;,$ (1)

where $\ell_{L}$ and $H$ stand for the SM left-handed lepton doublet and the
Higgs doublet, respectively. The price we pay here is to introduce a high-
dimensional (dimension-5) operator and the violation of global lepton number
symmetry. The minimal ultraviolet (UV) realization of this dimension-5
operator is the Type I Seesaw mechanism Minkowski:1977sc ; Yanagida:1979as ;
GellMann:1980vs ; Glashow:1979nm ; Mohapatra:1979ia ; Shrock:1980ct ;
Schechter:1980gr . In the minimal Type I Seesaw, the “sterile neutrinos” have
the nature of Majorana fermions as they transform as singlet under the SM
gauge group. The RH neutrinos possess a Majorana mass term $M_{R}$ and
interact with SM leptons through a Yukawa coupling $y_{\nu}$. The light
neutrino masses are given by $m_{\nu}\sim y_{\nu}^{2}v^{2}/M_{R}$ with $v$
being the vacuum expectation value (vev) of SM Higgs. The heavy Majorana
neutrinos can also arise in the Type III Seesaw mechanism with triplet
fermionic multiplet Foot:1988aq . The eigenstates of RH neutrinos are usually
referred as heavy neutral leptons (HNLs) which will be denoted as $N$ below.
After mass mixing, one obtains an important mixing matrix $V_{\ell N}$
transiting heavy neutrinos to charged leptons in the mixed mass-flavor basis
Atre:2009rg .

If $y_{\nu}$ is close to the electron Yukawa coupling $y_{e}\sim 10^{-6}$ in
Type I Seesaw, the heavy Majorana neutrino receives a mass at TeV level and
can be searched through lepton number violation (LNV) signature at high-energy
colliders. This low-scale neutral lepton and the mixing matrix $V_{\ell N}$
have been searched at colliders. See Ref. Cai:2017mow  and references therein
for a review of LNV searches. The mass of HNL has been excluded up to about 1
TeV depending on the value of $|V_{\ell N}|^{2}\lesssim 0.1$ CMS:2018jxx ;
CMS:2018iaf ; CMS:2022rqc ; ATLAS:2023tkz . The future FCC-hh can push the
exclusion limit up to a few TeV Antusch:2016ejd ; Pascoli:2018heg ;
Liu:2023gpt . One needs a machine with higher energy and integrated luminosity
to further explore the measurement of $V_{\ell N}$ for the high mass region of
HNL $N$. High-energy muon colliders have recently received much attention in
the community MICE:2019jkl ; Delahaye:2019omf ; Bartosik:2020xwr ; Han:2020uid
; Yin:2020afe ; AlAli:2021let ; Bose:2022obr ; Maltoni:2022bqs ;
Narain:2022qud ; Li:2022kkc ; Li:2023ksw ; Chowdhury:2023imd . The high-energy
options of muon collider can have multi-TeV center-of-mass (c.m.) energies and
high integrated luminosity scaling with energy quadratically Delahaye:2019omf

$\mathcal{L}=\left(\frac{\sqrt{s}}{10\ {\rm TeV}}\right)^{2}\times
10~{}\textrm{ab}^{-1}\;.$ (2)

It thus provides an excellent opportunity to produce and discover HNLs in Type
I and III Seesaw mechanisms.

Recently, there arose quite a few studies of searching for HNL at
$\mu^{+}\mu^{-}$ colliders Mekala:2023diu ; Kwok:2023dck ; Li:2023tbx . They
proposed that an HNL $N_{\ell}$ can be produced together with a light “active”
neutrino $\nu_{\ell}$ from $\mu^{+}\mu^{-}$ collision with c.m. energy
$\sqrt{s}=3$ or 10 TeV. The HNL then decays into a charged lepton $\ell^{\pm}$
and a $W$ gauge boson with hadronic decay. The production mode then becomes

$\displaystyle\mu^{+}\mu^{-}\to
N_{\ell}\bar{\nu}_{\ell}\to\ell^{\pm}\bar{\nu}_{\ell}q\bar{q}^{\prime}\;.$ (3)

Despite of large production cross section of $\sigma(\mu^{+}\mu^{-}\to
N_{\mu}\bar{\nu}_{\mu})\sim\mathcal{O}(10)$ pb through a $W$ exchange in
t-channel Li:2023tbx , this signal channel cannot tell whether the HNL is
Majorana fermion because of the missing neutrino in final states. One has to
define a forward-backward symmetry in $N_{\ell}$’s decay pattern Li:2023tbx
or other kinematic quantity Kwok:2023dck  to distinguish Majorana and Dirac
neutrinos. An alternative approach to search for Majorana neutrino is to
consider the inverse $0\nu\beta\beta$-like channel $\mu^{+}\mu^{+}\to
W^{+}W^{+}$ Yang:2023ojm ; Jiang:2023mte  which however relies on a same-sign
muon collider.

In this work, we propose a clear way to search for heavy Majorana neutrino
through LNV signature at $\mu^{+}\mu^{-}$ colliders. At high-energy muon
colliders, the initial muon beams substantially emit electroweak (EW) gauge
bosons under an approximately unbroken SM gauge symmetry. The gauge bosons are
associated with very energetic muons or muon-neutrinos in the forward region
with respect to the beam. The EW gauge bosons behave like initial state
partons and lead to vector boson scattering (VBS) processes Costantini:2020stv
; AlAli:2021let ; Han:2021udl ; BuarqueFranzosi:2021wrv ; Ruiz:2021tdt . The
VBS becomes an increasingly important mode as colliding energies go higher,
due to the logarithmic enhancement from gauge boson radiation. Thus, a LNV
signal can be produced through VBS

$\displaystyle
W^{\pm}Z/\gamma\to\ell^{\pm}N\to\ell^{\pm}\ell^{\pm}W^{\mp}\to\ell^{\pm}\ell^{\pm}q\bar{q}^{\prime}\;.$
(4)

This signal is composed of same-sign charged leptons in final states and can
thus clearly tell the Majorana nature of heavy neutrinos. It is exactly the
smoking-gun signature searching for Majorana neutrino via Drell-Yan production
$pp\to W^{\ast}\to\ell^{\pm}N$ Han:2006ip ; Atre:2009rg  or $W\gamma$ VBS
Dev:2013wba ; Alva:2014gxa  at LHC. It can serve as a robust channel to
identify LNV and Majorana neutrino suppose a discovery of HNL through the
channel in Eq. (3). We will utilize the leading-order framework of electroweak
parton distribution functions (EW PDFs) Han:2021udl ; Ruiz:2021tdt  to
calculate the HNL production through VBS. The detector simulation of LNV
signal and SM backgrounds will be performed to analyze the search potentials
on heavy Majorana neutrino and obtain the exclusion limit on mixing $V_{\ell
N}$.

In analogy to neutrinoless double-beta ($0\nu\beta\beta$) decay, the VBS
process can also induce LNV signature with the HNL in t-channel Dicus:1991fk ;
Han:2006ip ; Atre:2009rg ; Fuks:2020att ; Fuks:2020zbm ; CMS:2022rqc ;
Schubert:2022lcp . This VBS process happens at same-sign muon collider (e.g.,
$\mu^{+}\mu^{+}$ collider) and leads to the signal of same-sign charged
leptons

$\displaystyle W^{+}W^{+}\to\ell^{+}\ell^{+}\;.$ (5)

This process provides a clean LNV signal compared with the inverse process
$\mu^{+}\mu^{+}\to W^{+}W^{+}$ as there are no missing neutrinos and the
suppression from two $W$ bosons’ decay branching fractions. We also analyze
this LNV channel and explore the exclusion limit on mixing $V_{\ell N}$. It
can help to probe all charged lepton flavors and corresponding mixing elements
rather than only $V_{\mu N}$ through $\mu^{+}\mu^{+}\to W^{+}W^{+}$. In the
absence of HNL, it was shown that this LNV process can also help to probe the
Weinberg operator Fuks:2020zbm . We also evaluate the sensitivity of muon
collider to the scale of Weinberg operator.

This paper is organized as follows. In Sec. 2, we first outline the HNLs in
canonical Type I and Type III Seesaw models. Then we simulate the production
of HNL and LNV signature via VBF processes at high-energy $\mu^{+}\mu^{-}$
colliders in Sec. 3. The results of projected sensitivity bounds for $V_{\ell
N}$ are also given. In Sec. 4, we also analyze the LNV signature with the HNL
in t-channel through VBS at same-sign muon collider. Finally, in Sec. 5 we
summarize our conclusions.

##  2 Heavy neutral leptons in Seesaw mechanisms

The HNLs can be realized in canonical Type I and Type III Seesaw mechanisms.
The neutrino Yukawa interactions in Type I Seesaw are

$\displaystyle-\mathcal{L}_{Y}^{I}=Y^{D}_{\nu}\bar{\ell}_{L}\tilde{H}N_{R}+{\frac{1}{2}}\overline{(N^{c})_{L}}M_{R}N_{R}+\text{h.c.},$
(6)

where $\tilde{H}=i\sigma_{2}H^{\ast}$, $N_{R}$ denotes RH neutrino, and
$M_{R}$ is the RH neutrino mass matrix. Once $H$ receives the SM vev $\langle
H\rangle=v/\sqrt{2}$, the Dirac mass term becomes
$m_{D}=Y^{D}_{\nu}v/\sqrt{2}$ and the diagonalization of the neutrino mass
matrix is

$\displaystyle\mathbb{N}^{\dagger}\left(\begin{array}[]{cc}0&m_{D}\\\
m_{D}^{T}&M_{R}\\\ \end{array}\right)\mathbb{N}^{\ast}$ $\displaystyle=$
$\displaystyle\left(\begin{array}[]{cc}m_{\nu}^{\rm diag}&0\\\ 0&M_{N}^{\rm
diag}\\\ \end{array}\right),$ (11)

with the transformation of mass eigenstates as

$\displaystyle\left(\begin{array}[]{c}\nu_{L}\\\ (N^{c})_{L}\\\
\end{array}\right)=\mathbb{N}\left(\begin{array}[]{c}\nu_{L}\\\ (N^{c})_{L}\\\
\end{array}\right)_{mass},\ \ \ \mathbb{N}=\left(\begin{array}[]{cc}U&V\\\
X&Y\\\ \end{array}\right)\;.$ (18)

The mass eigensvalues of physical neutrinos are $m_{\nu}^{\rm diag}={\rm
diag}(m_{1},m_{2},m_{3})$ and $M_{N}^{\rm diag}={\rm diag}(M_{1},\cdots)$.
With another matrix $E$ diagonalizing the charged lepton mass matrix, we have

$\displaystyle E^{\dagger}U\equiv U_{\rm PMNS}\;,~{}~{}~{}E^{\dagger}V\equiv
V_{\ell N}\;,$ (19)

where $U_{\rm PMNS}$ is the approximate Pontecorvo-Maki-Nakagawa-Sakata (PMNS)
neutrino mass mixing matrix. In particular, the matrix $V_{\ell N}$ transits
heavy neutrinos to charged leptons Atre:2009rg . The active neutrino states
are decomposed into a general number of massive eigenstates

$\displaystyle\nu_{\ell}=\sum_{m=1}^{3}(U_{\rm PMNS})_{\ell
m}\nu_{m}+\sum_{m^{\prime}=1}(V_{\ell N})_{\ell
m^{\prime}}N_{m^{\prime}}^{c}\;,$ (20)

where we assume that the HNL mass eigenstates are denoted as $N_{m^{\prime}}$.

From Eq. (11), one can derive a relationship between diagonalized neutrino
mass matrices and mixing matrices

$\displaystyle U^{\ast}_{\rm PMNS}m_{\nu}^{\rm diag}U^{\dagger}_{\rm
PMNS}+V^{\ast}_{\ell N}M_{N}^{\rm diag}V^{\dagger}_{\ell N}=0\ .$ (21)

In the mixed mass-flavor basis, the matrix $V_{\ell N}$ determines the
interactions between HNL and SM EW gauge bosons such as

$\displaystyle\mathcal{L}_{\rm Type-I}$ $\displaystyle\supset$
$\displaystyle-{\frac{g}{\sqrt{2}}}W_{\mu}^{-}\sum_{\ell=e}^{\tau}\Big{(}\sum_{m=1}^{3}\bar{\ell}(U_{\rm
PMNS})_{\ell m}\gamma^{\mu}P_{L}\nu_{m}+\sum_{m^{\prime}=1}\bar{\ell}(V_{\ell
N})_{\ell m^{\prime}}\gamma^{\mu}P_{L}N^{c}_{m^{\prime}}\Big{)}+\text{h.c.}$
$\displaystyle-\frac{g}{2\cos\theta_{W}}Z_{\mu}\sum_{\ell=e}^{\tau}\Big{(}\sum_{m=1}^{3}\bar{\nu}_{\ell}(U_{\rm
PMNS})_{\ell
m}\gamma^{\mu}P_{L}\nu_{m}+\sum_{m^{\prime}=1}\bar{\nu}_{\ell}(V_{\ell
N})_{\ell
m^{\prime}}\gamma^{\mu}P_{L}N^{c}_{m^{\prime}}\Big{)}+\text{h.c.}\;.$

The partial width of one HNL $N$ decay into charged lepton becomes

$\displaystyle\Gamma(N\to\ell^{\pm}W^{\mp})$ $\displaystyle\equiv$
$\displaystyle\Gamma(N\to\ell^{+}W^{-})=\Gamma(N\to\ell^{-}W^{+})$ (23)
$\displaystyle=$ $\displaystyle\frac{G_{F}}{8\sqrt{2}\pi}|V_{\ell
N}|^{2}m_{N}(m_{N}^{2}+2m_{W}^{2})\left(1-\frac{m_{W}^{2}}{m_{N}^{2}}\right)^{2},$

and an asymptotic behavior holds when $m_{N}\gg m_{W},m_{Z},m_{h}$.

$\displaystyle\Gamma(N\to\sum_{\ell}\ell^{\pm}W^{\mp})\approx\Gamma(N\to\sum_{\nu}\nu
Z+\bar{\nu}Z)\approx\Gamma(N\to\sum_{\nu}\nu h+\bar{\nu}h)\;,$ (24)

where $\ell=e,\mu,\tau$.

The Type III Seesaw introduces SU$(2)_{L}$ triplet leptons $\Sigma_{L}$ with
zero hypercharge in addition to the SM fields Foot:1988aq

$\displaystyle\Sigma_{L}=\left(\begin{array}[]{cc}\Sigma_{L}^{0}/\sqrt{2}&\Sigma^{+}_{L}\\\
\Sigma_{L}^{-}&-\Sigma_{L}^{0}/\sqrt{2}\\\ \end{array}\right)\ .$ (27)

The Type III Seesaw Lagrangian is given by

$\mathcal{L}_{\rm Type-III}={\rm
Tr}\left[\overline{\Sigma_{L}}i\cancel{D}\Sigma_{L}\right]-\left(\frac{1}{2}{\rm
Tr}\left[\overline{\Sigma_{L}^{c}}M_{\Sigma}\Sigma_{L}\right]+Y_{\Sigma}\overline{\ell_{L}}\tilde{H}\Sigma_{L}^{c}+\text{h.c.}\right)\;.$
(28)

The Yukawa term in Eq. (28) induces both the neutrino mass mixing and a mass
mixing between the charged SM leptons and the charged triplet leptons

$\displaystyle\left(\begin{array}[]{cc}\overline{\nu_{L}^{c}}&\overline{\Sigma_{L}^{0c}}\\\
\end{array}\right)\left(\begin{array}[]{cc}0&Y_{\Sigma}^{T}v_{0}/2\sqrt{2}\\\
Y_{\Sigma}v_{0}/2\sqrt{2}&M_{\Sigma}/2\\\
\end{array}\right)\left(\begin{array}[]{c}\nu_{L}\\\ \Sigma_{L}^{0}\\\
\end{array}\right)+\left(\begin{array}[]{cc}\overline{\ell_{R}}&\overline{\Sigma_{L}^{+c}}\\\
\end{array}\right)\left(\begin{array}[]{cc}m_{\ell}&0\\\
Y_{\Sigma}v_{0}&M_{\Sigma}\\\
\end{array}\right)\left(\begin{array}[]{c}\ell_{L}\\\ \Sigma_{L}^{-}\\\
\end{array}\right)+\text{h.c.}\;.$ (39)

After diagonalizing mass matrices by introducing unitary matrices, one obtains
mass eigenvalues for neutrinos and charged leptons. The mixing between the SM
charged leptons and triplet leptons follows the same relation in Eq. (21). The
interactions between HNL and SM EW gauge bosons are also the same as those in
Type I Seesaw.

In this section, we outline the theoretical motivations of the mixing matrix
$V_{\ell N}$ between heavy neutrinos and charged leptons, i.e., Type I and
Type III Seesaw models. In fact, the mixing parameter would scale like
$V_{\ell N}^{2}\sim m_{\nu}/m_{N}$ and should be very small. We must state
that we will assume the mixing angle $V_{\ell N}$ and the HNL mass $m_{N}$ as
independent parameters in the following collider studies.

##  3 Heavy neutral lepton and lepton number violation through VBS at
$\mu^{+}\mu^{-}$ collider

In this section, we investigate the search for Majorana HNL through VBS
processes at future high-energy muon colliders. We simulate the LNV signatures
from the decay of on-shell Majorana neutral lepton and obtain the sensitivity
to the mixing parameters between HNL and SM charged leptons.

###  3.1 The VBS production of HNL and LNV signal

We propose to produce an HNL accompanied by a SM charged lepton
$\ell_{1}^{\pm}N$. Due to charge conservation, this combination can only be
produced through VBS processes at muon colliders. After considering the decay
of HNL $N$, a LNV signature with $\Delta L=2$ can emerge

$V_{i}~{}V_{j}\to\ell_{1}^{\pm}N\to\ell_{1}^{\pm}\ell_{2}^{\pm}q~{}\bar{q}^{\prime}\;,$
(41)

where $\ell_{1},\ell_{2}=e,\mu,\tau$ and $V_{i},V_{j}$ denote SM EW gauge
bosons. The decay channel of Majorana HNL is considered to be
$N\to\ell_{2}^{\pm}W^{\mp}$ and the $W$ boson has subsequent decay into
$q~{}\bar{q}^{\prime}$. The Feynman diagrams of above LNV production are shown
in Fig. 1. The panels (a) and (b) illustrate t-channel processes via
exchanging a SM lepton and the panel (c) represents an s-channel process
mediated by a virtual $W$ boson.

(a)

(b)

(c)

Figure 1: The Feynman diagrams of processes
$V_{i}~{}V_{j}\to\ell^{+}_{1}N\to\ell^{+}_{1}\ell^{+}_{2}q~{}\bar{q}^{\prime}$
at the muon collider.

For the initial gauge boson partons $V_{i}$ and $V_{j}$, the VBS production
cross section can be factorized as the product of the parton luminosity
$d\mathcal{L}_{ij}/d\tau$ and the partonic cross section

$\displaystyle\sigma(\ell^{+}\ell^{-}\to F+X)$ $\displaystyle=$
$\displaystyle\int^{1}_{\tau_{0}}d\tau\sum_{ij}\frac{d\mathcal{L}_{ij}}{d\tau}\sigma(V_{i}V_{j}\to
F)\;,$ $\displaystyle\frac{d\mathcal{L}_{ij}}{d\tau}$ $\displaystyle=$
$\displaystyle\frac{1}{1+\delta_{ij}}\int^{1}_{\tau}\frac{d\xi}{\xi}\Big{[}f_{i}(\xi,Q^{2})f_{j}(\frac{\tau}{\xi},Q^{2})+(i\leftrightarrow
j)\Big{]}\;,$ (42)

where $F(X)$ denotes an exclusive final state (the underlying remnants),
$f_{i(j)}(\xi,Q^{2})$ is the EW PDF for vector $V_{i(j)}$ with
$Q=\sqrt{\hat{s}}/2$ being the factorization scale, $\tau_{0}=m_{F}^{2}/s$ and
$\tau=\hat{s}/s$. By summing over all gauge boson initial states, one can
obtain the total cross section of the “inclusive” production processes. The
scattering amplitude of the LNV process in Eq. (41) is proportional to
$|V_{\ell_{1}N}V_{\ell_{2}N}|$ and the corresponding cross sections can be
expressed as

$\displaystyle\sigma~{}(V_{i}~{}V_{j}\to\ell_{1}^{\pm}N\to\ell_{1}^{\pm}\ell_{2}^{\pm}q\bar{q}^{\prime})$
$\displaystyle\approx\sigma~{}(V_{i}~{}V_{j}\to\ell_{1}^{\pm}N)\times{\rm
BR}(N\to\ell_{2}^{\pm}q\bar{q}^{\prime})\times(2-\delta_{\ell_{1}\ell_{2}})$
$\displaystyle\equiv\frac{|V_{\ell_{1}N}V_{\ell_{2}N}|^{2}}{\sum_{\ell=e,\mu,\tau}|V_{\ell
N}|^{2}}\times\sigma_{0}\times(2-\delta_{\ell_{1}\ell_{2}})\;,$ (43)

where $\sigma_{0}$ represents the part of the cross section that is
independent of the mixing parameters. We then define a new parameter which
includes all the information of the mixing parameters Atre:2009rg

$S_{\ell_{1}\ell_{2}}\equiv\frac{|V_{\ell_{1}N}V_{\ell_{2}N}|^{2}}{\sum_{\ell=e,\mu,\tau}|V_{\ell
N}|^{2}}\;.$ (44)

The parameter-independent cross section
$\sigma_{0}=\sigma/S_{\ell_{1}\ell_{2}}$ for $\ell_{1}=\ell_{2}$ are shown in
Fig. 2. The cross section of $\ell_{1}\neq\ell_{2}$ case is two times that
shown here because of the exchange $\ell_{1}\leftrightarrow\ell_{2}$ in Eq.
(43). If including the charge conjugation, another factor of two should be
multiplied. For the following simulation of LNV signatures, we choose three
cases of lepton flavor combinations in the final states with
$\ell_{1},\ell_{2}=e,\mu$

$S_{\mu\mu}=\frac{|V_{\mu N}|^{4}}{\sum_{\ell=e,\mu,\tau}|V_{\ell
N}|^{2}},~{}S_{ee}=\frac{|V_{eN}|^{4}}{\sum_{\ell=e,\mu,\tau}|V_{\ell
N}|^{2}},~{}S_{e\mu}=\frac{|V_{eN}V_{\mu
N}|^{2}}{\sum_{\ell=e,\mu,\tau}|V_{\ell N}|^{2}}\;.$ (45)

Next we consider the production of one single HNL denoted as $N$.

Figure 2: The parameter-independent cross section
$\sigma_{0}=\sigma/S_{\ell_{1}\ell_{2}}$ for the VBS process
$V_{i}~{}V_{j}\to\ell_{1}^{\pm}N\to\ell_{1}^{\pm}\ell_{2}^{\pm}q~{}\bar{q}^{\prime}$
with $\ell_{1}=\ell_{2}$, as a function of the HNL mass $m_{N}$ (left) or c.m.
energy $\sqrt{s}$ (right) at muon colliders. The benchmark c.m. energy of the
muon collider are $\sqrt{s}=3,~{}10,~{}20$ and 30 TeV, and the benchmark HNL
masses are $m_{N}=0.5,~{}1,~{}3$ and 5 TeV.

###  3.2 The search for LNV signal at $\mu^{+}\mu^{-}$ collider

We employ the Universal FeynRules Output (UFO) Degrande:2011ua  files of the
model “HeavyN” Degrande:2016aje ; Pascoli:2018heg  and pass the model files to
MadGraph5_aMC@NLO Alwall:2014hca  to generate the above LNV signal events at
parton-level. The latest version of MadGraph5 has included the leading order
framework of EW PDFs for the computation of the VBS processes at muon
colliders Ruiz:2021tdt . We then use PYTHIA 8 Sjostrand:2014zea ;
Bierlich:2022pfr  for the parton shower of parton-level events. The detector
card of the muon collider in Delphes 3 deFavereau:2013fsa  is utilized to
include the detector effects.

Our LNV signal is composed of two same-sign charged leptons plus jet(s), as
shown in Fig. 1. We consider the following SM backgrounds

$\displaystyle{\rm B_{1}}:~{}~{}V~{}V\to
W^{\pm}W^{\pm}W^{\mp}\to\ell^{\pm}\ell^{\pm}\mathop{\nu_{\ell}}\limits^{(-)}\mathop{\nu_{\ell}}\limits^{(-)}q~{}\bar{q}^{\prime}\;,$
$\displaystyle{\rm B_{2}}:~{}~{}V~{}V\to t~{}\overline{t}~{}W^{\pm}\to
b~{}W^{+}\overline{b}~{}W^{-}W^{\pm}\to
b~{}\bar{b}~{}\ell^{\pm}\ell^{\pm}\mathop{\nu_{\ell}}\limits^{(-)}\mathop{\nu_{\ell}}\limits^{(-)}+X\;,$
(46)

where $X$ denotes the decay products of the opposite-sign $W$ boson in ${\rm
B_{2}}$ background, i.e., $X=\ell^{\mp}\mathop{\nu_{\ell}}\limits^{(-)}$ or
$q\bar{q}^{\prime}$. For ${\rm B_{1}}$ background, the intermediate states are
three $W$ bosons which are produced through VBS process. The two same-sign $W$
bosons leptonically decay into a pair of same-sign leptons, and the other $W$
decays into dijet. Thus, the final states of ${\rm B_{1}}$ consist of two
same-sign charged leptons, two jets and missing energy. In general,
$b~{}(\bar{b})$ quark can be mistakenly identified as a light jet in the
detector of the collider. Thus we also consider the ${\rm B_{2}}$ background
in which a pair of $t~{}\bar{t}$ quarks associated with a $W$ boson are
produced by VBS process. The $t~{}(\bar{t})$ quark decays into the
$b~{}(\bar{b})$ quark and the $W^{\pm}$ boson. The same-sign $W$ bosons again
decay into the lepton pairs and the other $W$ with opposite charge may decay
into all possible final states. The events with
$X=\ell^{\mp}\mathop{\nu_{\ell}}\limits^{(-)}$ can be efficiently reduced by
imposing a soft $p_{T}$ cut on the opposite-sign lepton. Here we include them
for a conservative background estimate.

We take the case of $\ell_{1}=\ell_{2}=\mu$ in Eq. (41) to illustrate the
kinematic observables of our signal and backgrounds in Fig. 3 and Fig. 4. The
benchmark masses of HNL are considered to be $m_{N}=200,~{}1000,~{}5000$ and
$9000$ GeV for $\sqrt{s}=30$ TeV. From the decay of such heavy Majorana
neutrino, the $W$ boson could be highly boosted and the produced dijet can be
regarded as a single fat-jet $J$ Das:2017gke ; Bhardwaj:2018lma ; Das:2018usr
in the detector of muon collider. The fat-jet is reconstructed via the
“Valencia” algorithm with $R=1.2$. We use $\mu_{1}$ to denote the muon
produced in association with the HNL, and $\mu_{2}$ and $J$ (or
$q\bar{q}^{\prime}$) are the decay products of HNL. We will explain how we
distinguish them in our analysis below. The signal and backgrounds exhibit
quite different kinematic distributions. We employ a series of selection cuts
based on the distributions to suppress the SM backgrounds and enhance the
significance.

Figure 3: The normalized distributions of the transverse momentum
$p_{T}(\mu_{1})$ (top) and $p_{T}(\mu_{2})$ (bottom) are presented for the
signal and the backgrounds for $\sqrt{s}=$ 30 TeV. The benchmark masses of HNL
from left to right are $m_{N}=200,~{}1000,~{}5000$ and $9000$ GeV.

Figure 4: The normalized distributions of $p_{T}(J)$ (top), missing energy
$\cancel{E}_{T}$ (middle), and the invariant mass $m_{N}$ of HNL (bottom) are
presented for the signal and the backgrounds for $\sqrt{s}=$ 30 TeV. The
benchmark masses of HNL from left to right are $m_{N}=200,~{}1000,~{}5000$ and
$9000$ GeV.

  * •

We first select the generated events by requiring the number of same-sign
muons and fat-jet, and identify the fat-jet as a $W$ boson

$\displaystyle N_{\mu}\geq 2,~{}N_{J}\geq 1\;;~{}~{}65~{}{\rm GeV}\leq
m_{J}\leq 95~{}{\rm GeV}\;.$ (47)

If no fat-jet is found in the event, we also request two leading ordinary jets
to satisfy the invariant mass window $65~{}{\rm GeV}\leq m_{j_{1}j_{2}}\leq
95~{}{\rm GeV}$. But we find that taking into account these events would not
change our results much. Thus, we only deal with the events with at least one
fat-jet in the following analysis.

  * •

We employ some basic cuts for the muons in final states

$\displaystyle p_{T}(\mu_{1})>50~{}{\rm
GeV},~{}~{}p_{T}(\mu_{2})>m_{N}/4;~{}~{}|\eta(\mu_{1})|,|\eta(\mu_{2})|<2.5;~{}~{}\Delta
R_{\mu_{1}\mu_{2}}>0.5\;.$ (48)

The transverse momentum cut for $\mu_{1}$ is essential to avoid possible
collinear divergence. As $\mu_{2}$ comes from the decay of $N$, it could be
very energetic depending on $m_{N}$ as shown in Fig. 3. We thus tighten up the
kinematic cut for the transverse momentum of $\mu_{2}$. It is worth mentioning
that for the two muons in the final states, we separately combine them with
$J$. The muon that generates an invariant mass closer to $m_{N}$ is considered
as $\mu_{2}$.

  * •

For the final fat-jet $J$ coming from the decay of $N$, its $p_{T}$
distribution is shown in the upper panels of Fig. 4. We employ the following
kinematic cuts

$\displaystyle p_{T}(J)>m_{N}/4,~{}~{}|\eta(J)|<2.5\;.$ (49)

  * •

The presence of neutrinos in the final states of the backgrounds leads to
missing energy, as shown in Eq. (46). For our signal without neutrinos, due to
the effect of the energy resolution of detector, there still exists a
distribution of small missing energy as shown in the middle panels of Fig. 4.
As $m_{N}$ increases, this effect becomes progressively more significant.
Thus, we only employ the missing energy cut for $m_{N}\leq 1000$ GeV

$\displaystyle\cancel{E}_{T}<30~{}{\rm GeV}~{}~{}~{}{\rm when}~{}~{}m_{N}\leq
1000~{}{\rm GeV}\;.$ (50)

  * •

We reconstruct Majorana HNL with $\mu_{2}$ and $J$ according to the invariant
mass $m_{\mu_{2}J}$ shown in the lower panels of Fig. 4. Based on the signal
distribution of the $m_{\mu_{2}J}$, we employ an asymmetric mass window cut

$\displaystyle 0.8\times m_{N}<~{}m_{\mu_{2}J}~{}<1.1\times m_{N}\;.$ (51)

The background events can be efficiently suppressed after employing the above
cuts. For illustration, we take $\sqrt{s}=30$ TeV for muon collider to present
the signal and background cross sections after these cuts, as shown in Table
1.

sig. and bkgs |  $\sigma/S_{\mu\mu}$ or |  $N_{\mu}$, $N_{J}$, $m_{J}$ | $\mu_{1},~{}\mu_{2}$ |  fat-jet $J$ | $\cancel{E}_{T}$ | $m_{\mu_{2}J}$
---|---|---|---|---|---|---
$\sigma_{\rm B_{i}}$ [fb] |  in Eq. (47) |  in Eq. (48) |  in Eq. (49) |  in Eq. (50) |  in Eq. (51)
$m_{N}=200$ GeV | 7.67 | 1.17 | 0.90 | 0.89 | 0.80 | 0.80
$\rm B_{1}$ | 0.65 | 0.23 | 0.13 | 0.13 | 0.0081 |  8.6$\times 10^{-4}$
$\rm B_{2}$ | 0.094 | 0.0054 | 0.0020 | 0.0020 |  2.0$\times 10^{-4}$ |  4.0$\times 10^{-5}$
$m_{N}=1000$ GeV | 9.05 | 4.54 | 4.11 | 3.79 | 2.80 | 2.79
$\rm B_{1}$ | 0.65 | 0.23 | 0.048 | 0.036 | 0.0017 |  5.4$\times 10^{-4}$
$\rm B_{2}$ | 0.094 | 0.0054 |  4.4$\times 10^{-4}$ |  2.6$\times 10^{-4}$ |  2.0$\times 10^{-5}$ |  9.9$\times 10^{-6}$
$m_{N}=5000$ GeV | 4.56 | 2.70 | 2.41 | 2.19 | – | 2.17
$\rm B_{1}$ | 0.65 | 0.23 | 0.0066 | 0.0035 | – | 0.0012
$\rm B_{2}$ | 0.094 | 0.0054 |  1.2$\times 10^{-5}$ | 0.0 | – | 0.0
$m_{N}=9000$ GeV | 2.11 | 1.17 | 1.03 | 0.95 | – | 0.94
$\rm B_{1}$ | 0.65 | 0.23 | 0.0020 |  8.8$\times 10^{-4}$ | – |  3.2$\times 10^{-4}$
$\rm B_{2}$ | 0.094 | 0.0054 |  5.1$\times 10^{-6}$ | 0.0 | – | 0.0
Table 1: The representative signal cross section $\sigma/S_{\mu\mu}$ (in blue)
and those for SM backgrounds (in black) at muon collider after selection cuts.
For illustration, four benchmarks $m_{N}=200,~{}1000,~{}5000$ and $9000$ GeV
are considered at muon collider with $\sqrt{s}=30$ TeV.

After implementing the above analysis, we evaluate the sensitivity to the
mixing constant $S_{\ell_{1}\ell_{2}}$ for Majorana HNL at muon colliders. For
the significance, we use the following formula

$\mathcal{S}=\frac{N_{\rm S}}{\sqrt{N_{\rm S}+N_{\rm B}}}\;,$ (52)

where $N_{\rm S}$ and $N_{\rm B}=N_{\rm B_{1}}+N_{\rm B_{2}}$ are the event
numbers of signal and backgrounds, respectively. They are given by

$\displaystyle N_{\rm S}=\sigma_{0}~{}S_{\ell_{1}\ell_{2}}\times\epsilon_{\rm
S}\times\mathcal{L}\;,~{}N_{\rm B_{1}}=\sigma_{{\rm
B}_{1}}\times\epsilon_{{\rm B}_{1}}\times\mathcal{L}\;,~{}N_{\rm
B_{2}}=\sigma_{{\rm B}_{2}}\times\epsilon_{{\rm B}_{2}}\times\mathcal{L}\;,$
(53)

where $\epsilon_{\rm S,B_{1(2)}}$ represent the efficiencies of the above cuts
and $\mathcal{L}$ denotes the integrated luminosity. From Eq. (2), we can get
the corresponding integrated luminosity of the muon collider for different
c.m. energies

$\displaystyle\sqrt{s}$ $\displaystyle=~{}3,~{}10,~{}~{}30~{}\left[{\rm
TeV}\right]\;,$ $\displaystyle\mathcal{L}$
$\displaystyle=~{}1,~{}10,~{}~{}90~{}\left[{\rm ab}^{-1}\right]\;.$ (54)

We obtain the 2$\sigma$ exclusion limits to $S_{\ell_{1},\ell_{2}}$ with
$\ell_{1},\ell_{2}=e,\mu$ versus $m_{N}$ at muon colliders with $\sqrt{s}=3$
(red), 10 (green) and 30 (blue) TeV, as shown in solid lines in Fig. 5.

Figure 5: The 2$\sigma$ exclusion limits for $|V_{\mu N}|^{2}$ (upper panel),
$|V_{eN}|^{2}$ (lower-left panel) and $S_{e\mu}$ or $|V_{eN}V_{\mu N}|$
(lower-right panel) as a function of $m_{N}$ at muon colliders with
$\sqrt{s}=3$ TeV (red), 10 TeV (green) and 30 TeV (blue). The solid and dash-
dotted lines correspond to $V_{i}V_{j}\to\ell_{1}^{\pm}\ell_{2}^{\pm}+{\rm
jet(s)}$ and $W^{+}W^{+}\to\ell_{1}^{+}\ell_{2}^{+}$ channels, respectively.
The frame label of the lower-right panel is $S_{e\mu}$ for
$\ell_{1}^{\pm}\ell_{2}^{\pm}+{\rm jet(s)}$ channel or $|V_{eN}V_{\mu N}|$ for
$\ell_{1}^{+}\ell_{2}^{+}$. The grey area shows the region excluded by a
global scan Chrzaszcz:2019inj . The recent results from two groups’
simulations of $\mu^{+}\mu^{-}\to N\nu$ are also added for comparison,
including the dotted lines Kwok:2023dck  and the dashed lines Li:2023tbx  for
$\sqrt{s}=3$ TeV (red) and 10 TeV (green).

The quantity $S_{\ell\ell}$ is equal to the commonly used $|V_{\ell N}|^{2}$
under the assumption of only one non-vanishing flavor element for the matrix
$V_{\ell N}$. Under this assumption, we use $|V_{\mu N}|^{2}$ or
$|V_{eN}|^{2}$ as the frame labels of the y-axis in the upper and lower-left
panels of Fig. 5. One can see that, for $m_{N}$ well below $\sqrt{s}$, the
value of $|V_{\mu N}|^{2}$ can be probed as low as $4\times 10^{-3}$ ($3\times
10^{-4}$) [$5\times 10^{-5}$] for $\sqrt{s}=3~{}(10)~{}[30]$ TeV. This
potential is worse than that through $\mu^{+}\mu^{-}\to
N_{\mu}\bar{\nu}_{\mu}$ channel Kwok:2023dck ; Li:2023tbx , because the latter
happens via a t-channel exchange of a $W$ boson with huge production cross
section. The projected sensitivity bound of $|V_{eN}|^{2}$ is similar to that
of $|V_{\mu N}|^{2}$. The difference comes from the efficiency of identifying
$e$ or $\mu$ from the collider simulation. However, this $|V_{eN}|^{2}$
exclusion limit is stronger than that through $\mu^{+}\mu^{-}\to
N_{e}\bar{\nu}_{e}$ channel for $\sqrt{s}=10$ TeV. This is because the
electron HNL can only be produced from $\mu^{+}\mu^{-}$ annihilation via
s-channel and its production cross section is exceeded by VBS production at
high energies. Moreover, this LNV process through VBS can provide a clean
signal of $e^{\pm}\mu^{\pm}$ and an exclusion limit on quantity $S_{e\mu}$ as
shown in the lower-right panel of Fig. 5.

##  4 Lepton number violation through VBS at $\mu^{+}\mu^{+}$ collider

###  4.1 The probe of heavy neutral lepton

In this section we consider the LNV signature through VBS at same-sign muon
collider

$\displaystyle W^{+}W^{+}\to\ell^{+}\ell^{+}\;.$ (55)

This VBS process is mediated by either light neutrino mass eigenstates
$\nu_{m}$ or the HNL $N_{m^{\prime}}$ in t-channel and is analogous to
neutrinoless double-$\beta$ decay. Their amplitudes are proportional to
$m_{\nu_{m}}$ and $m_{N_{m^{\prime}}}$, respectively. We thus only consider
the process mediated by one HNL $N$ in t-channel as shown by the Feynman
diagram in Fig. 6.

Figure 6: The representative Feynman diagram of process
$W^{+}W^{+}\to\ell_{1}^{+}\ell_{2}^{+}$ at the $\mu^{+}\mu^{+}$ collider.
There is another u-channel diagram by exchanging the initial $W$ bosons.

Figure 7: The parameter-independent cross section
$\sigma_{0}=\sigma/|V_{\ell_{1}N}V_{\ell_{2}N}|^{2}$ for the VBS process
$W^{+}W^{+}\to\ell_{1}^{+}\ell_{2}^{+}$ with $\ell_{1}=\ell_{2}$, as a
function of the HNL mass $m_{N}$ (left) and c.m. energy $\sqrt{s}$ (right) at
muon colliders. The benchmark c.m. energy of the muon collider are
$\sqrt{s}=3,~{}10,~{}20$ and 30 TeV, the benchmark HNL masses are
$m_{N}=0.5,~{}1,~{}3$ and 5 TeV.

The cross section of this VBS process can be expressed as

$\sigma~{}(W^{+}W^{+}\to\ell_{1}^{+}\ell_{2}^{+})\equiv|V_{\ell_{1}N}V_{\ell_{2}N}|^{2}\times\sigma_{0}\;,$
(56)

where $\sigma_{0}$ is the “bare” cross section independent of the mixing
parameters. The values of cross section $\sigma_{0}$ for $\ell_{1}=\ell_{2}$
are shown in Fig. 7 which are one-half of those for $\ell_{1}\neq\ell_{2}$
because of the average of identical final states. This LNV signal is purely
composed of two same-sign charged leptons. We consider three primary SM
backgrounds

$\displaystyle{\rm B_{1}}:~{}~{}\mu^{+}\mu^{+}\to\mu^{+}\mu^{+}\;,$
$\displaystyle{\rm
B_{2}}:~{}~{}\mu^{+}\mu^{+}\to\mu^{+}\mu^{+}\nu~{}\bar{\nu}\;,$
$\displaystyle{\rm B_{3}}:~{}~{}\mu^{+}\mu^{+}\to
W^{+}W^{+}\bar{\nu}_{\mu}~{}\bar{\nu}_{\mu}\to\ell^{+}\ell^{+}\nu_{\ell}~{}\nu_{\ell}~{}\bar{\nu}_{\mu}~{}\bar{\nu}_{\mu}\;,$

where $\nu~{}(\bar{\nu})$ donates the summation of neutrinos (anti-neutrinos)
with three flavors. The backgrounds ${\rm B_{1}}$ and ${\rm B_{2}}$ only
appear for the situation with $\ell_{1}=\ell_{2}=\mu$, and we also take this
case as an example to illustrate the distributions of signal and backgrounds
in Fig. 8. Corresponding to other flavor combinations of charged leptons in
our signal, only background ${\rm B_{3}}$ applies. According to the different
kinematic distributions of signal and backgrounds, we adopt a series of
selection cuts to suppress the backgrounds and enhance the significance.

Figure 8: The normalized distributions of the transverse momentum $p_{T}^{\rm
max}(\mu)$ (row 1), $\cos\theta_{\ell\ell}$ (row 2), missing energy
$\cancel{E}_{T}$ (row 3) and the invariant mass $m_{\ell\ell}$ (row 4) for the
signal and the backgrounds. The benchmark masses of HNL are
$m_{N}=200,~{}1000,~{}5000$ and $9000$ GeV from left to right for $\sqrt{s}=$
30 TeV.

  * •

Firstly, we require the number of same-sign muons in the generated events, and
employ some basic cuts for the muons in final states

$N_{\mu}\geq 2,~{}~{}p_{T}(\mu_{1},\mu_{2})>50~{}{\rm
GeV},~{}~{}|\eta(\mu_{1})|,|\eta(\mu_{2})|<2.5,~{}~{}\Delta
R_{\mu_{1}\mu_{2}}>0.5\;.$ (57)

  * •

The leading transverse momentum $p_{T}^{\rm max}(\mu)$ of the final muons is
shown in the first row of Fig. 8. We employ some relatively conservative
kinematic cuts

$p_{T}^{\rm max}(\mu)~{}>~{}100~{}(200)~{}[500]~{}{\rm GeV}\;,$ (58)

for $\sqrt{s}=3~{}(10)~{}[30]$ TeV. They help to reduce the background ${\rm
B_{3}}$ in which the charged leptons are from two $W$ bosons’ leptonic decay.

  * •

The cosine of the angle between two muons in final states
$\cos\theta_{\ell\ell}$ exhibits very different behaviors between signal and
SM backgrounds, as shown in the second row of Fig. 8. Thus, we employ the
following cut

$\cos\theta_{\ell\ell}<0.2\;.$ (59)

  * •

For backgrounds ${\rm B_{2}}$ and ${\rm B_{3}}$, the presence of neutrinos
leads to missing energy, as shown in the third row of Fig. 8. We can employ
the cut for the missing energy to reduce the backgrounds 111The cut on the
$p_{T}$ of the sub-leading charged lepton could also help to reduce the
backgrounds at the LHC ATLAS:2023tkz . We find that it is worse than the
missing energy cut here.

$\cancel{E}_{T}<20~{}(50)~{}[150]~{}{\rm GeV}\;,$ (60)

for $\sqrt{s}=3~{}(10)~{}[30]$ TeV.

  * •

The final muons in background ${\rm B_{1}}$ carry all the collision energy.
Its invariant mass leads to a significantly different distribution from the
signal and other backgrounds, as shown in the last row of Fig. 8. According to
this feature, we can employ the following invariant mass cut to extremely
reduce the background ${\rm B_{1}}$

$\displaystyle m_{\ell\ell}~{}$
$\displaystyle<~{}2000~{}(5000)~{}[15000]~{}{\rm GeV}\;,$ (61)

for $\sqrt{s}=3~{}(10)~{}[30]$ TeV.

We take the case of $\sqrt{s}=30$ TeV for muon collider to illustrate the
cross sections of the signal and backgrounds after the above series of cuts,
as shown in Table 2.

sig. and bkgs |  $\sigma/|V_{\mu N}|^{4}$ or |  $N_{\mu}$ and basic cuts | $p_{T}^{\rm max}(\mu)$ | ${\rm cos}~{}\theta_{\ell\ell}$ | $\cancel{E}_{T}$ | $m_{\ell\ell}$
---|---|---|---|---|---|---
$\sigma_{\rm B_{i}}$ [pb] |  in Eq. (57) |  in Eq. (58) |  in Eq. (59) |  in Eq. (60) |  in Eq. (61)
$m_{N}=200$ GeV | 0.072 | 0.070 | 0.041 | 0.030 | 0.027 | 0.026
$m_{N}=1000$ GeV | 0.64 | 0.62 | 0.57 | 0.44 | 0.36 | 0.36
$m_{N}=5000$ GeV | 1.99 | 1.95 | 1.94 | 1.77 | 1.01 | 0.97
$m_{N}=9000$ GeV | 1.84 | 1.84 | 1.83 | 1.75 | 0.83 | 0.76
$\rm B_{1}$ | 0.055 | 0.051 | 0.051 | 0.051 | 0.011 | 0.0
$\rm B_{2}$ | 0.0046 | 0.0042 | 0.0042 | 0.0024 |  4.5$\times 10^{-5}$ |  1.3$\times 10^{-5}$
$\rm B_{3}$ | 0.011 | 0.0018 |  4.1$\times 10^{-4}$ |  2.8$\times 10^{-4}$ |  9.9$\times 10^{-6}$ |  9.9$\times 10^{-6}$
Table 2: The representative signal cross section $\sigma/|V_{\mu N}|^{4}$ (in
blue) and those for SM backgrounds (in black) at muon collider after selection
cuts. For illustration, four benchmarks $m_{N}=200,~{}1000,~{}5000$ and $9000$
GeV are considered at muon collider with $\sqrt{s}=30$ TeV.

We adopt the same significance formula in Eq. (52) here with $N_{\rm B}=N_{\rm
B_{1}}+N_{\rm B_{2}}+N_{\rm B_{3}}$. The $N_{\rm S}$ and $N_{{\rm B}_{i}}$
($i=1,2,3$) are given by

$\displaystyle N_{\rm
S}=\sigma_{0}~{}|V_{\ell_{1}N}V_{\ell_{2}N}|^{2}\times\epsilon_{\rm
S}\times\mathcal{L}\;,~{}N_{\rm B_{i}}=\sigma_{{\rm
B}_{i}}\times\epsilon_{{\rm B}_{i}}\times\mathcal{L}\;.$ (62)

The corresponding integrated luminosity of the muon collider for different
c.m. energies are shown in Eq. (54). We obtain the sensitivity of muon
colliders to the mixing parameter $|V_{\ell_{1}N}V_{\ell_{2}N}|$ for Majorana
HNL. The $2\sigma$ exclusion limits to $|V_{\mu N}|^{2}$, $|V_{eN}|^{2}$ and
$|V_{eN}V_{\mu N}|$ versus the mass of Majorana HNL are shown in dash-dotted
lines in Fig. 5. In contrast to the $\ell_{1}^{\pm}\ell_{2}^{\pm}+{\rm
jet(s)}$ signature losing sensitivity near the energy threshold, this
t-channel LNV signal can probe much heavier HNL.

###  4.2 The probe of Weinberg operator

Finally, we comment on the probe of dimension-5 Weinberg operator in the
absence of HNL. The above signature of same-sign charged leptons can also be
used to probe the Weinberg operator Fuks:2020zbm ; ATLAS:2023tkz

$\displaystyle{C_{5}^{\ell\ell^{\prime}}\over\Lambda}H\cdot\bar{\ell}_{L}^{c}\ell^{\prime}_{L}\cdot
H\;,$ (63)

where $\ell_{L}^{T}=(\nu_{\ell},\ell)$. After EW symmetry breaking and the SM
Higgs gains its vev $v$, this Weinberg operator generates a Majorana neutrino
mass in the flavor basis

$\displaystyle
m_{\ell\ell^{\prime}}={C_{5}^{\ell\ell^{\prime}}v^{2}\over\Lambda}\,,$ (64)

which appears as the mass of the intermediate fermion in the same-sign $W$
scattering. It was shown in Ref. Fuks:2020zbm  that the parton-level cross
section of this same-sign $W$ scattering process in this model is

$\displaystyle\sigma(W^{+}W^{+}\to\ell\ell^{\prime})\approx{2-\delta_{\ell\ell^{\prime}}\over
18\pi}\Big{|}{C_{5}^{\ell\ell^{\prime}}\over\Lambda}\Big{|}^{2}\;.$ (65)

The cross section is proportional to an overall scaling factor of
$|C_{5}^{\ell\ell^{\prime}}/\Lambda|^{2}$.

Next we follow the approach in Ref. Fuks:2020zbm  to simulate the probe of
Weinberg operator for $\ell=\ell^{\prime}=\mu$ at high-energy muon colliders.
It was verified that a reasonable simulation using the UFO for Weinberg
operator (called “SMWeinberg”) has to be performed with
$\Lambda/|C_{5}^{\ell\ell^{\prime}}|\geq 200$ TeV Fuks:2020zbm . We adopt the
same selection strategies mentioned in last subsection and obtain the number
of signal events denoted by $N_{\rm S}^{200}$ for a benchmark with
$\Lambda=200$ TeV and $C_{5}^{\mu\mu}=1$. According to Eq. (65), the number of
signal events is given by

$\displaystyle N_{\rm S}=N_{\rm S}^{200}|C_{5}^{\mu\mu}|^{2}\Big{(}{200~{}{\rm
TeV}\over\Lambda}\Big{)}^{2}\;.$ (66)

By requiring $2\sigma$ significance, we can obtain the sensitivity to the
scale $\Lambda/|C_{5}^{\mu\mu}|$. We find that the sensitivity bound of muon
collider to the scale is

$\displaystyle\Lambda/|C_{5}^{\mu\mu}|\lesssim 9.3~{}(23.0)~{}[47.7]~{}{\rm
TeV}\;,$ (67)

for $\sqrt{s}=3~{}(10)~{}[30]$ TeV. This is equivalent to the limit of
effective $\mu\mu$ Majorana mass as

$\displaystyle m_{\mu\mu}\gtrsim 6.5~{}(2.6)~{}[1.3]~{}{\rm GeV}\;.$ (68)

##  5 Conclusion

In this work, we propose the search for Majorana HNLs and LNV signatures
through VBS processes at high-energy muon colliders. At high energies, the
muon beams radiate substantial EW gauge bosons and can play as “gauge boson
colliders”. The VBS processes lead to clear LNV signatures so as to tell the
nature of Majorana HNLs and thus provide more advantageous benefits than
direct $\mu\mu$ annihilation. We use the method of EW PDFs to calculate the
VBS production of HNL at both $\mu^{+}\mu^{-}$ and $\mu^{+}\mu^{+}$ colliders,
and analyze the produced LNV signatures. At $\mu^{+}\mu^{-}$ collider, the
scattering of gauge bosons $W^{\pm}Z/\gamma$ induces the associated production
of HNL $N$ and charged lepton $\ell^{\pm}$. At same-sign muon collider, the
$W^{+}W^{+}$ scattering can result in a $0\nu\beta\beta$-like LNV process
$W^{+}W^{+}\to\ell^{+}\ell^{+}$ with the HNL in t-channel.

We perform the detector simulation of LNV signal and SM backgrounds. The
search potentials on heavy Majorana neutrino are analyzed and the exclusion
limits on the mixing of HNL $N$ and charged lepton $V_{\ell N}$ are obtained.
We find the following conclusions:

  * •

The VBS processes can provide smoking-gun LNV signatures
$\ell_{1}^{\pm}\ell_{2}^{\pm}+{\rm jet(s)}$ and $\ell_{1}^{+}\ell_{2}^{+}$ at
$\mu^{+}\mu^{-}$ collider and same-sign muon collider, respectively.

  * •

The probing potential of $|V_{\mu N}|^{2}$ through
$W^{\pm}Z/\gamma\to\mu^{\pm}\mu^{\pm}+{\rm jet(s)}$ signature is worse than
that from annihilation channel $\mu^{+}\mu^{-}\to N_{\mu}\bar{\nu}_{\mu}$.
However, the exclusion limit on $|V_{eN}|^{2}$ from VBS channel is stronger
than that through the $\mu^{+}\mu^{-}$ annihilation for $\sqrt{s}=10$ TeV or
above.

  * •

At same-sign muon collider, compared with the $\ell^{\pm}\ell^{\pm}+{\rm
jet(s)}$ signature, the $0\nu\beta\beta$-like LNV signal can help to probe
much heavier HNL. The mixings other than $|V_{\mu N}|$ can be probed through
VBS as well, which is contrary to the process $\mu^{+}\mu^{+}\to W^{+}W^{+}$.
Based on this signature, we also estimate the sensitivity of muon collider to
the scale of Weinberg operator.

  * •

The above LNV signatures can both be employed to look for the combinations of
different charged lepton flavors and probe the mixing
$|V_{\ell_{1}N}V_{\ell_{2}N}|$ with $\ell_{1}\neq\ell_{2}$.

###### Acknowledgements.

We would like to thank Richard Ruiz for useful discussion. T.L. is supported
by the National Natural Science Foundation of China (Grants No. 11975129,
12035008) and “the Fundamental Research Funds for the Central Universities”,
Nankai University (Grant No. 63196013). C.Y.Y. is supported in part by the
Grants No. NSFC-11975130, No. NSFC-12035008, No. NSFC-12047533, the Helmholtz-
OCPC International Postdoctoral Exchange Fellowship Program, the National Key
Research and Development Program of China under Grant No. 2017YFA0402200, the
China Postdoctoral Science Foundation under Grant No. 2018M641621, and the
Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) under
Germany’s Excellence Strategy — EXC 2121 “Quantum Universe” — 390833306.

## References

  * (1) S. Weinberg, _Baryon and Lepton Nonconserving Processes_ , _Phys. Rev. Lett._ 43 (1979) 1566–1570.
  * (2) P. Minkowski, _$\mu\to e\gamma$ at a Rate of One Out of $10^{9}$ Muon Decays?_, _Phys. Lett. B_ 67 (1977) 421–428.
  * (3) T. Yanagida, _Horizontal gauge symmetry and masses of neutrinos_ , _Conf. Proc. C_ 7902131 (1979) 95–99.
  * (4) M. Gell-Mann, P. Ramond and R. Slansky, _Complex Spinors and Unified Theories_ , _Conf. Proc. C_ 790927 (1979) 315–321, [1306.4669].
  * (5) S. L. Glashow, _The Future of Elementary Particle Physics_ , _NATO Sci. Ser. B_ 61 (1980) 687.
  * (6) R. N. Mohapatra and G. Senjanovic, _Neutrino Mass and Spontaneous Parity Nonconservation_ , _Phys. Rev. Lett._ 44 (1980) 912.
  * (7) R. E. Shrock, _General Theory of Weak Leptonic and Semileptonic Decays. 1\. Leptonic Pseudoscalar Meson Decays, with Associated Tests For, and Bounds on, Neutrino Masses and Lepton Mixing_ , _Phys. Rev. D_ 24 (1981) 1232.
  * (8) J. Schechter and J. W. F. Valle, _Neutrino Masses in SU(2) x U(1) Theories_ , _Phys. Rev. D_ 22 (1980) 2227.
  * (9) R. Foot, H. Lew, X. G. He and G. C. Joshi, _Seesaw Neutrino Masses Induced by a Triplet of Leptons_ , _Z. Phys. C_ 44 (1989) 441.
  * (10) A. Atre, T. Han, S. Pascoli and B. Zhang, _The Search for Heavy Majorana Neutrinos_ , _JHEP_ 05 (2009) 030, [0901.3589].
  * (11) Y. Cai, T. Han, T. Li and R. Ruiz, _Lepton Number Violation: Seesaw Models and Their Collider Tests_ , _Front. in Phys._ 6 (2018) 40, [1711.02180].
  * (12) CMS collaboration, A. M. Sirunyan et al., _Search for heavy Majorana neutrinos in same-sign dilepton channels in proton-proton collisions at $\sqrt{s}=13$ TeV_, _JHEP_ 01 (2019) 122, [1806.10905].
  * (13) CMS collaboration, A. M. Sirunyan et al., _Search for heavy neutral leptons in events with three charged leptons in proton-proton collisions at $\sqrt{s}=$ 13 TeV_, _Phys. Rev. Lett._ 120 (2018) 221801, [1802.02965].
  * (14) CMS collaboration, _Probing heavy Majorana neutrinos and the Weinberg operator through vector boson fusion processes in proton-proton collisions at $\sqrt{s}$ = 13 TeV_, 2206.08956.
  * (15) ATLAS collaboration, G. Aad et al., _Search for Majorana neutrinos in same-sign $WW$ scattering events from $pp$ collisions at $\sqrt{s}=13$ TeV_, 2305.14931.
  * (16) S. Antusch, E. Cazzato and O. Fischer, _Sterile neutrino searches at future $e^{-}e^{+}$, $pp$, and $e^{-}p$ colliders_, _Int. J. Mod. Phys. A_ 32 (2017) 1750078, [1612.02728].
  * (17) S. Pascoli, R. Ruiz and C. Weiland, _Heavy neutrinos with dynamic jet vetoes: multilepton searches at $\sqrt{s}=14$ , 27, and 100 TeV_, _JHEP_ 06 (2019) 049, [1812.08750].
  * (18) W. Liu, J. Li, Z. Chen and H. Sun, _Probing Heavy Neutrinos at the LHC from Fat-jet using Machine Learning_ , 2303.15920.
  * (19) MICE collaboration, M. Bogomilov et al., _Demonstration of cooling by the Muon Ionization Cooling Experiment_ , _Nature_ 578 (2020) 53–59, [1907.08562].
  * (20) J. P. Delahaye, M. Diemoz, K. Long, B. Mansoulié, N. Pastrone, L. Rivkin et al., _Muon Colliders_ , 1901.06150.
  * (21) N. Bartosik et al., _Detector and Physics Performance at a Muon Collider_ , _JINST_ 15 (2020) P05001, [2001.04431].
  * (22) T. Han, Y. Ma and K. Xie, _High energy leptonic collisions and electroweak parton distribution functions_ , _Phys. Rev. D_ 103 (2021) L031301, [2007.14300].
  * (23) W. Yin and M. Yamaguchi, _Muon g-2 at a multi-TeV muon collider_ , _Phys. Rev. D_ 106 (2022) 033007, [2012.03928].
  * (24) H. Al Ali et al., _The muon Smasher’s guide_ , _Rept. Prog. Phys._ 85 (2022) 084201, [2103.14043].
  * (25) T. Bose et al., _Report of the Topical Group on Physics Beyond the Standard Model at Energy Frontier for Snowmass 2021_ , 2209.13128.
  * (26) F. Maltoni et al., _TF07 Snowmass Report: Theory of Collider Phenomena_ , 2210.02591.
  * (27) M. Narain et al., _The Future of US Particle Physics - The Snowmass 2021 Energy Frontier Report_ , 2211.11084.
  * (28) T. Li, H. Qin, C.-Y. Yao and M. Yuan, _Probing heavy triplet leptons of the type-III seesaw mechanism at future muon colliders_ , _Phys. Rev. D_ 106 (2022) 035021, [2205.04214].
  * (29) T. Li, C.-Y. Yao and M. Yuan, _Revealing the origin of neutrino masses through the Type II Seesaw mechanism at high-energy muon colliders_ , _JHEP_ 03 (2023) 137, [2301.07274].
  * (30) T. A. Chowdhury, A. Jueid, S. Nasri and S. Saad, _Probing Zee-Babu states at Muon Colliders_ , 2306.01255.
  * (31) K. Mękała, J. Reuter and A. F. Żarnecki, _Optimal search reach for heavy neutral leptons at a muon collider_ , _Phys. Lett. B_ 841 (2023) 137945, [2301.02602].
  * (32) T. H. Kwok, L. Li, T. Liu and A. Rock, _Searching for Heavy Neutral Leptons at A Future Muon Collider_ , 2301.05177.
  * (33) P. Li, Z. Liu and K.-F. Lyu, _Heavy neutral leptons at muon colliders_ , _JHEP_ 03 (2023) 231, [2301.07117].
  * (34) J.-L. Yang, C.-H. Chang and T.-F. Feng, _The leptonic di-flavor and di-number violation processes at high energy $\mu^{\pm}\mu^{\pm}$ colliders_, 2302.13247.
  * (35) R. Jiang, T. Yang, S. Qian, Y. Ban, J. Li, Z. You et al., _Searching for Majorana Neutrinos at a Same-Sign Muon Collider_ , 2304.04483.
  * (36) A. Costantini, F. De Lillo, F. Maltoni, L. Mantani, O. Mattelaer, R. Ruiz et al., _Vector boson fusion at multi-TeV muon colliders_ , _JHEP_ 09 (2020) 080, [2005.10289].
  * (37) T. Han, S. Li, S. Su, W. Su and Y. Wu, _Heavy Higgs bosons in 2HDM at a muon collider_ , _Phys. Rev. D_ 104 (2021) 055029, [2102.08386].
  * (38) D. Buarque Franzosi et al., _Vector boson scattering processes: Status and prospects_ , _Rev. Phys._ 8 (2022) 100071, [2106.01393].
  * (39) R. Ruiz, A. Costantini, F. Maltoni and O. Mattelaer, _The Effective Vector Boson Approximation in high-energy muon collisions_ , _JHEP_ 06 (2022) 114, [2111.02442].
  * (40) T. Han and B. Zhang, _Signatures for Majorana neutrinos at hadron colliders_ , _Phys. Rev. Lett._ 97 (2006) 171804, [hep-ph/0604064].
  * (41) P. S. B. Dev, A. Pilaftsis and U.-k. Yang, _New Production Mechanism for Heavy Neutrinos at the LHC_ , _Phys. Rev. Lett._ 112 (2014) 081801, [1308.2209].
  * (42) D. Alva, T. Han and R. Ruiz, _Heavy Majorana neutrinos from $W\gamma$ fusion at hadron colliders_, _JHEP_ 02 (2015) 072, [1411.7305].
  * (43) D. A. Dicus, D. D. Karatas and P. Roy, _Lepton nonconservation at supercollider energies_ , _Phys. Rev. D_ 44 (1991) 2033–2037.
  * (44) B. Fuks, J. Neundorf, K. Peters, R. Ruiz and M. Saimpert, _Majorana neutrinos in same-sign $W^{\pm}W^{\pm}$ scattering at the LHC: Breaking the TeV barrier_, _Phys. Rev. D_ 103 (2021) 055005, [2011.02547].
  * (45) B. Fuks, J. Neundorf, K. Peters, R. Ruiz and M. Saimpert, _Probing the Weinberg operator at colliders_ , _Phys. Rev. D_ 103 (2021) 115014, [2012.09882].
  * (46) J. L. Schubert and O. Ruchayskiy, _Neutrinoless double-beta decay at colliders: interference between Majorana states_ , 2210.11294.
  * (47) C. Degrande, C. Duhr, B. Fuks, D. Grellscheid, O. Mattelaer and T. Reiter, _UFO - The Universal FeynRules Output_ , _Comput. Phys. Commun._ 183 (2012) 1201–1214, [1108.2040].
  * (48) C. Degrande, O. Mattelaer, R. Ruiz and J. Turner, _Fully-Automated Precision Predictions for Heavy Neutrino Production Mechanisms at Hadron Colliders_ , _Phys. Rev. D_ 94 (2016) 053002, [1602.06957].
  * (49) J. Alwall, R. Frederix, S. Frixione, V. Hirschi, F. Maltoni, O. Mattelaer et al., _The automated computation of tree-level and next-to-leading order differential cross sections, and their matching to parton shower simulations_ , _JHEP_ 07 (2014) 079, [1405.0301].
  * (50) T. Sjöstrand, S. Ask, J. R. Christiansen, R. Corke, N. Desai, P. Ilten et al., _An introduction to PYTHIA 8.2_ , _Comput. Phys. Commun._ 191 (2015) 159–177, [1410.3012].
  * (51) C. Bierlich et al., _A comprehensive guide to the physics and usage of PYTHIA 8.3_ , 2203.11601.
  * (52) DELPHES 3 collaboration, J. de Favereau, C. Delaere, P. Demin, A. Giammanco, V. Lemaître, A. Mertens et al., _DELPHES 3, A modular framework for fast simulation of a generic collider experiment_ , _JHEP_ 02 (2014) 057, [1307.6346].
  * (53) A. Das, P. Konar and A. Thalapillil, _Jet substructure shedding light on heavy Majorana neutrinos at the LHC_ , _JHEP_ 02 (2018) 083, [1709.09712].
  * (54) A. Bhardwaj, A. Das, P. Konar and A. Thalapillil, _Looking for Minimal Inverse Seesaw scenarios at the LHC with Jet Substructure Techniques_ , _J. Phys. G_ 47 (2020) 075002, [1801.00797].
  * (55) A. Das, S. Jana, S. Mandal and S. Nandi, _Probing right handed neutrinos at the LHeC and lepton colliders using fat jet signatures_ , _Phys. Rev. D_ 99 (2019) 055030, [1811.04291].
  * (56) M. Chrzaszcz, M. Drewes, T. E. Gonzalo, J. Harz, S. Krishnamurthy and C. Weniger, _A frequentist analysis of three right-handed neutrinos with GAMBIT_ , _Eur. Phys. J. C_ 80 (2020) 569, [1908.02302].


# Cross-level distillation and feature denoising for cross-domain few-shot
classification

Hao ZHENG11footnotemark: 1
Tokyo Institute of Technology
<EMAIL_ADDRESS>
&Runqi Wang11footnotemark: 1
Huawei Noah’s Ark Lab
<EMAIL_ADDRESS>
&Jianzhuang Liu
Huawei Noah’s Ark Lab
<EMAIL_ADDRESS>
& Asako Kanezaki22footnotemark: 2
Tokyo Institute of Technology
<EMAIL_ADDRESS>Co-first author.Corresponding author.

###### Abstract

The conventional few-shot classification aims at learning a model on a large
labeled base dataset and rapidly adapting to a target dataset that is from the
same distribution as the base dataset. However, in practice, the base and the
target datasets of few-shot classification are usually from different domains,
which is the problem of cross-domain few-shot classification. We tackle this
problem by making a small proportion of unlabeled images in the target domain
accessible in the training stage. In this setup, even though the base data are
sufficient and labeled, the large domain shift still makes transferring the
knowledge from the base dataset difficult. We meticulously design a cross-
level knowledge distillation method, which can strengthen the ability of the
model to extract more discriminative features in the target dataset by guiding
the network’s shallow layers to learn higher-level information. Furthermore,
in order to alleviate the overfitting in the evaluation stage, we propose a
feature denoising operation which can reduce the feature redundancy and
mitigate overfitting. Our approach can surpass the previous state-of-the-art
method, Dynamic-Distillation, by 5.44$\%$ on 1-shot and 1.37$\%$ on 5-shot
classification tasks on average in the BSCD-FSL benchmark. The implementation
code will be available at https://github.com/jarucezh/cldfd.

##  1 Introduction

Deep learning has achieved great success on image recognition tasks with the
help of a large number of labeled images. However, it is the exact opposite of
the human perception mechanism which can recognize a new category by learning
only a few samples. Besides, a large amount of annotations is costly and
unavailable for some scenarios. It is more valuable to study few-shot
classification which trains a classification model on a base dataset and
rapidly adapts it to the target dataset. However, due to the constraint that
the base data and the target data need to be consistent in their
distributions, the conventional few-shot classification may not cater to the
demands in some practical scenarios. For example, it may fail in scenarios
where the training domain is natural images, but the evaluation domain is
satellite images. Considering this domain shift in practical applications, we
focus on cross-domain few-shot classification (CD-FSC) in this paper. Previous
methods, such as (Mangla et al., 2020; Adler et al., 2020; Tseng et al.,
2020), can handle this problem with small domain gaps. However, the CD-FSC
problem with a large domain gap is still a challenge.

BSCD-FSC (Guo et al., 2020) is a suitable benchmark for studying this problem,
where the base dataset has natural images and the target datasets contain
satellite images, crop disease images, skin disease images and X-ray images of
sundry lung diseases. On this benchmark, previous methods following the
traditional CD-FSC protocol train their models on the base dataset and
evaluate them on the target dataset, but their performances are far from
satisfactory. STARTUP (Phoo & Hariharan, 2021) and Dynamic-Distillation (Islam
et al., 2021a) introduce a more realistic setup that makes a small portion of
the unlabeled target images accessible during the training phase. These target
images bring a prior to the model and dramatically promote the model’s
performance on the target datasets. Inspired by that, we follow their setup to
explore the CD-FSC problem with a large domain shift.

Figure 1: Our CD-FSC framework. The first phase is pre-training, which trains
the teacher network on the labeled base dataset by optimizing the cross-
entropy loss. The second phase trains the student network using our proposed
cross-level distillation (CLD). The third phase fine-tunes a linear classifier
on a few labeled images in the target domain, and feature denoising (FD) is
conducted to remove the noise in the final feature vectors. The final phase
classifies images in the target domain.

In this work, we propose a cross-level distillation (CLD), which can
effectively transfer the knowledge from the base dataset and improve the
performance of the student network on the target domain. Besides, we propose
feature denoising (FD) to remove the noise in the features during the fine-
tuning stage. Our CD-FSC framework is given in Figure 1.

The detail of CLD is shown in Figure 2, which distills a teacher’s deeper
layers to a student’s shallower layers, where the student and the teacher
share the same structure. Unlike the distillation methods in STARTUP and
Dynamic-Distillation, which only distill the teacher’s last layer to the
student’s last layer, our CLD leads the shallow layers of the student to mimic
the features generated from the deeper levels of the teacher so that the
student can learn more deeper semantic information and extract more
discriminative features on the target dataset. Additionally, since the teacher
networks in STARTUP and Dynamic-Distillation are pre-trained on the base
dataset only, the teacher’s observation of the target data is biased. In order
to calibrate the bias, we design an iterative process by building another
network which shares the same structure and parameters with the historical
student network, named old student network. In each training iteration, the
features from the teacher and the old student in the same layers are
dynamically fused to guide the corresponding layers of the student. The latter
the training iteration, the fewer fusion features from the teacher network,
and the more from the old student network. Due to the target data used in
training is unlabeled, the self-supervised loss is introduced to excavate the
target domain information further.

The self-supervised loss not only supports the network in mining valuable
information on the target domain, but also brings a phenomenon where the final
feature vector for classification has a small number of dominant (strongly
activated) elements with the others are close to zero (Hua et al., 2021;
Kalibhat et al., 2022). We find that during the fine-tuning phase in Figure 1,
these small activated elements are redundant and considered as noise. Our FD
operation keeps the top $h$ largest elements and sets the others to zero. It
is experimentally verified that FD can greatly improve the model’s
performance.

Above all, our main contributions are summarized below:

  * •

We propose a cross-level distillation (CLD) framework, which can well transfer
the knowledge of the teacher trained on the base dataset to the student. We
also use an old student network mechanism is also necessary to calibrate the
teacher’s bias learned from the base data.

  * •

Considering the noisy feature activations, we design a feature denoising (FD)
operation that can significantly improve the performance of our model.

  * •

Extensive experiments are conducted to verify that our proposed CLD and FD can
achieve state-of-the-art results on the BSCD-FSL benchmark with large domain
gaps.

##  2 Related Work

Cross-domain few-shot classification. The cross-domain few-shot classification
is firstly defined by (Chen et al., 2018), which trains a model on the base
dataset and evaluates it on the target dataset in a different domain from the
base dataset. LFT (Tseng et al., 2020) simulates the domain shift from the
base dataset to the target dataset by meta-learning and inserts linear layers
into the network to align the features from the different domains. Meta-
FDMixup (Fu et al., 2021) uses several labeled images from the target dataset
for domain-shared feature disentanglement and feeds the domain-shared features
to the classifier. FLUTE (Triantafillou et al., 2021) learns the universal
templates of features across multi-source domains to improve the
transferability of the model. However, all these methods concentrate on the
CD-FSC problem with small domain shifts. Some methods handle CD-FSC with large
domain gaps, in which the target datasets have obvious dissimilarity from the
base dataset on perspective distortion, semantics, and/or color depth. For
example, ConfeSS (Das et al., 2021) extracts useful feature components from
labeled target images. ATA (Wang & Deng, 2021) does not require any prior of
the target dataset and proposes a plug-and-play inductive bias-adaptive task
augmentation module. CI (Luo et al., 2022) trains an encoder on the base
dataset and converts the features of the target data with a transformation
function in the evaluation stage. UniSiam (Lu et al., 2022) adopts a self-
supervised approach to address the CD-FSC problem. Among the methods dealing
with large domain shifts, STARTUP (Phoo & Hariharan, 2021) is a strong
baseline, which uses a few unlabeled target images in the training stage. It
firstly trains a teacher network on the base dataset in a supervised fashion
and transfers the teacher’s knowledge to the student by knowledge distillation
(KD). It jointly optimizes the cross-entropy loss with the base dataset,
contrastive loss of the unlabeled target images and the KD loss to upgrade the
student network. Dynamic-Distillation (Islam et al., 2021a) also uses a small
number of unlabeled images and KD. The main difference between Dynamic-
Distillation and STARTUP is that the former upgrades the pre-trained teacher
dynamically by exponential moving averages, while the latter fixes the
teacher. In our work, we follow their data setup allowing a small proportion
of the unlabeled target images to be seen during the training phase. Different
from the two methods that perform KD at the last layers of the teacher and the
student, our KD is carried out at cross levels. Besides, our denoising
operation further improves the performance.

Self-supervised learning. Self-supervised learning is widely used in the
scenarios where labels are not available for training. It defines a “pretext
task” to pre-train a network. For example, (Gidaris et al., 2018) pre-trains
the model by predicting the rotation angle of the image. One popular method is
contrastive learning, such as SimCLR (Chen et al., 2020) which pulls different
augmented versions of the same image closer and pushes the versions from
different images away. Beyond contrastive learning, BYOL (Grill et al., 2020)
and SimSiam (Chen & He, 2021) rely only positive pairs.

Knowledge distillation. (Hinton et al., 2015) firstly propose knowledge
distillation by guiding a compact network (student) to mimic the output of a
large network (teacher). Since features in the intermediate layers are
informative, some previous methods distill the teacher’s intermediate features
(Romero et al., 2014) or attention maps of the features (Zagoruyko &
Komodakis, 2017). Besides, self-distillation methods, such as BYOT (Zhang et
al., 2019), distill the last layer of the network to its own shallower layers.
BYOT’s teacher and student are the same network.

In our framework (Figure 2), in addition to the KD in the intermediate layers,
we design an old student that shares the same structure as the student but
with different parameters. The introduction of this old student not only
alleviates the teacher’s bias learned from the base dataset, but also has the
effect of assembling multiple historic students during training.

Figure 2: Framework of CLD for knowledge distillation (KD). The teacher
network is pre-trained on the labeled base dataset with the cross-entropy loss
and is fixed during KD. When training the student network, the target image
$\mathbf{x}$ is augmented into $\mathbf{x}_{1}$, $\mathbf{x}_{2}$ and
$\mathbf{x}_{3}$. Then ${\mathbf{x}_{1},\mathbf{x}_{2}}$ is fed to both the
student and the teacher, and $\mathbf{x}_{3}$ is fed to the old student. At
the $i$-th iteration, the parameters of the old student are a copy of those of
the student at $(i-\tau)$-th iteration. The feature $\mathbf{s}_{l}^{i}$ of
the student is firstly projected by $\omega_{l}$ for dimensionality alignment,
where $l$ is the block index. Then we fuse the features $\mathbf{t}_{l+1}^{i}$
and $\mathbf{o}_{l+1}^{i}$ obtaining $\mathbf{u}_{l+1}^{i}$, which are from
the $(l+1)$-th block of the teacher and the old student, respectively. The KD
is conducted by forcing $\omega_{l}(\mathbf{s}_{l}^{i})$ to mimic
$\mathbf{u}_{l+1}^{i}$. Additionally, the self-supervised loss
$\mathcal{L}_{ss}$ is introduced on the student network.

##  3 Methodology

###  3.1 Preliminary

During the training period, we follow the setting in STARTUP (Phoo &
Hariharan, 2021) and Dynamic-Distillation (Islam et al., 2021a), where a
labeled base dataset $\mathcal{D}_{\rm B}$ and a few unlabeled target images
sampled from the target dataset $\mathcal{D}_{\rm T}$ are available. In the
testing stage, the support set $\mathcal{D}_{\rm S}$ comprises $N$ classes,
and $K$ samples are randomly selected from each class in $\mathcal{D}_{\rm
T}$, which is the so-called $N$-way $K$-shot task. The support set
$\mathcal{D}_{\rm S}$ is for fine-tuning a new classifier with the frozen
encoder (the student network in this work). The images in the query set
$\mathcal{D}_{\rm Q}$ are randomly picked from the selected $N$ classes for
evaluating the classification accuracy. The support set $\mathcal{D}_{\rm S}$
and the query set $\mathcal{D}_{\rm Q}$ have no overlap.

###  3.2 Cross-level distillation

The proposed cross-level distillation (CLD) framework is shown in Figure 2.
The teacher network $f_{\rm t}$ is pre-trained on $\mathcal{D}_{\rm B}$ with
the cross-entropy loss. The student network $f_{\rm s}$ is expected to inherit
the knowledge of the teacher and extract discriminative features of
$\mathcal{D}_{\rm T}$. However, the teacher’s observation of the target data
is biased since it is pre-trained on the base dataset only. In the $i$-th
training iteration, if the features extracted by the student $f_{\rm s}^{i}$
directly mimic the features of the teacher, the teacher’s bias will be
transferred to the student. To reduce the bias, we introduce an _old student
network_ $f_{\rm o}^{i}$, which is a copy of $f_{\rm s}^{i-\tau}$, where the
hyper-parameter $\tau$ denotes the training iteration interval between $f_{\rm
s}^{i-\tau}$ and $f_{\rm s}^{i}$.

To simplify the KD complexity, we divide each backbone of $f_{\rm s}$, $f_{\rm
o}$, and $f_{\rm t}$ into $L$ residual blocks. Let $\mathbf{s}_{l}^{i}$,
$\mathbf{o}_{l}^{i}$, and $\mathbf{t}_{l}^{i}$ be the features obtained by the
student, the old student, and the teacher in the $l$-th block at the $i$-th
iteration. The fusion between $\mathbf{t}_{l}^{i}$ and $\mathbf{o}_{l}^{i}$ is
defined as:

$\mathbf{u}_{l}^{i}=\alpha^{i}\mathbf{o}_{l}^{i}+(1-\alpha^{i})\mathbf{t}_{l}^{i},$
(1)

where $\alpha^{i}=\frac{i}{T}$ is a dynamic weight with $T$ being the total
number of training iterations. The KD loss $\mathcal{L}_{\rm{dis}}^{i}$ in the
$i$-th iteration is defined as:

$\mathcal{L}_{\rm{dis}}^{i}=\begin{cases}\sum_{l=1}^{L}\|\omega_{l}(\mathbf{s}_{l}^{i})-\mathbf{t}_{l+1}^{i}\|_{2}^{2}&\text{if}\
i\leq\tau\\\ \\\
\sum_{l=1}^{L}\|\omega_{l}(\mathbf{s}_{l}^{i})-\mathbf{u}_{l+1}^{i}\|_{2}^{2}&\text{otherwise,}\end{cases}$
(2)

where $\omega_{l}(\cdot)$ is a projector of the $l$-th student’s block for
feature dimensionality alignment. It is comprised of a convolutional layer, a
batch normalization operator and ReLU activation. Note that the KD in Equation
2 is from the $(l+1)$-th block of the teacher to the $l$-th block of the
student. In our experiments, we find that this style is better than others
(see Section 4.3).

The total loss $\mathcal{L}$ for training the student network is:

$\mathcal{L}=\mathcal{L}_{\rm{ss}}+\lambda\mathcal{L}_{\rm{dis}},$ (3)

where $\mathcal{L}_{\rm{ss}}$ is a self-supervised loss, and $\lambda$ is a
weight coefficient of loss function for balancing the two losses.

For $\mathcal{L}_{\rm{ss}}$, off-the-shelf self-supervised losses like SimCLR
(Chen et al., 2020) or BYOL (Grill et al., 2020) can be used. The contrastive
loss in SimCLR is:

$\mathcal{L}_{\rm{simclr}}=-\frac{1}{\left|\mathbf{B}\right|}\sum_{m,n\in\mathbf{B}}\log\frac{\exp\left(\operatorname{sim}\left(\mathbf{z}_{m},\mathbf{z}_{n}\right)/\gamma\right)}{\sum_{q=1,q\neq
m}^{2\left|\mathbf{B}\right|}\exp\left(\operatorname{sim}\left(\mathbf{z}_{m},\mathbf{z}_{q}\right)/\gamma\right)},$
(4)

where $\mathbf{z}_{m}$, $\mathbf{z}_{n}$ and $\mathbf{z}_{q}$ are the
projected embeddings of different augmentations, $(m,n)$ is a positive pair
from the same image, $\operatorname{sim}(\cdot)$ is a similarity function,
$\mathbf{B}$ is a mini-batch of unlabeled target images, and $\gamma$ is a
temperature coefficient. The self-supervised loss in BYOL is:

$\mathcal{L}_{\rm{byol}}=\sum_{m\in\mathbf{B}}2-2\cdot\frac{\operatorname{sim}(p(\mathbf{z}_{m}),\mathbf{z}_{m}^{\prime})}{\left\|p(\mathbf{z}_{m})\right\|_{2}\cdot\left\|\mathbf{z}_{m}{\prime}\right\|_{2}},$
(5)

where $\mathbf{z}_{m}$ and $\mathbf{z}_{m}^{\prime}$ are embeddings of the
online network and the target network, respectively, and $p(\cdot)$ is a
linear predictor.

Note that the last convolution block of the student network is not involved in
KD and is trained by minimizing $\mathcal{L}_{\rm{ss}}$ only. The reason is
that the last block mainly discovers semantic information that is highly
domain-specific. Therefore, we constrain it on the target data rather than
letting it learn from the teacher that is pre-trained on the base data.

###  3.3 Feature Denoising

The self-supervised loss brings a phenomenon where the final feature vector
for classification has a small number of strongly activated elements while the
others are close to zero (Hua et al., 2021; Kalibhat et al., 2022). These
elements of small magnitudes are regarded as noise, which may cause
overfitting. We propose a feature denoising (FD) operation to remove their
contribution during the fine-tuning phase (see Figure 1). FD keeps the largest
$h$ elements of the feature from the student network and zeros the other
elements. The FD operation is only performed on the feature $\mathbf{s}_{L}$
in the last layer of the student network. Specifically, let
$\mathbf{s}_{L}=[s_{(L,1)},s_{(L,2)},\dots,s_{(L,D_{L})}]$ and
$\widetilde{\mathbf{s}}_{L}=[\widetilde{s}_{(L,1)},\widetilde{s}_{(L,2)},\dots,\widetilde{s}_{(L,D_{L})}]$
be the features before and after the FD operation, respectively, where $D_{L}$
is the feature’s dimensionality. Then the FD operation is defined as:

$\widetilde{s}_{L,d}=\begin{cases}({s_{L,d}})^{\beta}&\text{ if }\
{s_{L,d}}\in top_{h}(\mathbf{s}_{L}),\ d=1,2,...,D_{L}\\\ 0&\text{
otherwise},\end{cases}$ (6)

where $\beta$ is a hyper-parameter which makes the non-zero elements more
distinguishable and $top_{h}(\cdot)$ is the operator selecting the largest $h$
elements of the feature. Finally, $\widetilde{\mathbf{s}}_{M}$ is fed to the
classifier for fine-tuning.

##  4 Experiments

###  4.1 Datasets and Implementation

Datasets. We evaluate the proposed CLD and FD for the CD-FSC problem on the
BSCD-FSL benchmark (Guo et al., 2020) with large domain gaps. The miniImageNet
dataset (Vinyals et al., 2016) serves as the base dataset $\mathcal{D}_{B}$
which has sufficient labeled images. EuroSAT (Helber et al., 2019),
CropDisease (Mohanty et al., 2016), ISIC (Codella et al., 2019) and ChestX
(Wang et al., 2017) in BSCD-FSL are the unlabeled target datasets
$\mathcal{D}_{T}$. We follow the training protocol in STARTUP (Phoo &
Hariharan, 2021) and Dynamic-Distillation (Islam et al., 2021a), which allows
the whole labeled training set of miniImageNet and a small proportion (20$\%$)
of unlabeled target images available during the training period. The remaining
80$\%$ of target images are utilized for fine-tuning and evaluating by
building $5$-way $K$-shot tasks, $K\in\\{1,5\\}$.

Implementation details. For a fair comparison, all the methods use ResNet-10
(Guo et al., 2020) as the backbone. Our model is optimized by SGD with the
momentum 0.9, weight decay 1e-4, and batch size 32 for 600 epochs. The
learning rate is 0.1 at the beginning and decays by $0.1$ after the $300$th
epoch and $500$th epoch. The hyper-parameters $\lambda$ in Equation 3, and $h$
and $\beta$ in Equation 6 are set to $2$, $64$, and $0.4$, respectively. For
fine-tuning and evaluating the student network, we randomly sample $600$
episodes of $5$-way $K$-shot tasks. The performance is represented by the
average classification accuracy over the 600 episodes within the $95\%$
confidence interval. In each episode, the parameters of the student are
frozen. An additional linear classifier is fine-tuned by minimizing the cross-
entropy loss. The above mentioned hyperparameters are determined based on the
EuroSAT dataset only, and then they are used for evaluation on all the
datasets, showing the generalization ability of our method.

Table 1: The averaged 5-way 1-shot and 5-shot averaged accuracy and 95%
confidence interval among 600 episodes are given. The reported results of
SimCLR (Base) and previous state-of-the-art Dynamic-Distillation are from
(Islam et al., 2021a). The results of CI are from (Luo et al., 2022). The
results of ATA are from (Wang & Deng, 2021). The performance of ConFeSS refers
to (Das et al., 2021) and they do not give the confidence interval. The
champion results are marked in bold.

| EuroSAT | CropDisease | ISIC | ChestX
---|---|---|---|---
| 1-shot | 5-shot | 1-shot | 5-shot | 1-shot | 5-shot | 1-shot | 5-shot
Transfer | 58.42$\pm$0.94 | 75.31$\pm$0.71 | 68.45$\pm$0.87 | 89.12$\pm$0.52 | 32.82$\pm$0.60 | 47.13$\pm$0.58 | 22.48$\pm$0.41 | 26.65$\pm$0.43
SimCLR (Base) | 58.28$\pm$0.90 | 80.83$\pm$0.64 | 68.26$\pm$0.86 | 83.44$\pm$0.61 | 32.15$\pm$0.59 | 45.90$\pm$0.58 | 22.37$\pm$0.42 | 26.63$\pm$0.46
CI | 58.82$\pm$0.92 | 76.26$\pm$0.70 | 61.58$\pm$0.88 | 89.25$\pm$0.51 | 32.43$\pm$0.56 | 44.04$\pm$0.55 | 23.23$\pm$0.41 | 27.20$\pm$0.44
Transfer+SimCLR | 65.92$\pm$0.88 | 81.83$\pm$0.59 | 81.39$\pm$0.80 | 94.85$\pm$0.38 | 33.86$\pm$0.61 | 47.25$\pm$0.59 | 22.91$\pm$0.45 | 27.03$\pm$0.43
STARTUP | 65.24$\pm$0.88 | 81.60$\pm$0.59 | 78.18$\pm$0.83 | 92.86$\pm$0.44 | 34.15$\pm$0.62 | 46.42$\pm$0.58 | 22.86$\pm$0.43 | 26.98$\pm$0.43
ATA | 65.94$\pm$0.50 | 79.47$\pm$0.30 | 77.82$\pm$0.50 | 88.15$\pm$0.50 | 34.70$\pm$0.40 | 45.83$\pm$0.30 | 21.67$\pm$0.20 | 23.60$\pm$0.20
BYOL | 64.38$\pm$0.93 | 82.44$\pm$0.56 | 67.15$\pm$1.17 | 89.40$\pm$0.63 | 29.21$\pm$0.51 | 41.44$\pm$0.51 | 21.66$\pm$0.31 | 25.99$\pm$0.43
ConFeSS | $-$ | 84.65 | $-$ | 88.88 | $-$ | 48.85 | $-$ | 27.09
Dynamic-Distillation | 73.14$\pm$0.84 | 89.07$\pm$0.47 | 82.14$\pm$0.78 | 95.54$\pm$0.38 | 34.66$\pm$0.58 | 49.36$\pm$0.59 | 23.38$\pm$0.43 | 28.31$\pm$0.46
SimCLR | 75.10$\pm$0.85 | 90.17$\pm$0.43 | 88.54$\pm$0.80 | 96.09$\pm$0.39 | 35.71$\pm$0.66 | 48.84$\pm$0.61 | 22.00$\pm$0.42 | 24.84$\pm$0.41
BYOL+CLD+FD (ours) | 72.78$\pm$0.84 | 88.50$\pm$0.45 | 86.94$\pm$0.73 | 95.85$\pm$0.38 | 37.70$\pm$0.67 | 51.19$\pm$0.63 | 24.34$\pm$0.44 | 30.15$\pm$0.44
SimCLR+CLD+FD (ours) | 82.52$\pm$0.76 | 92.89$\pm$0.34 | 90.48$\pm$0.72 | 96.58$\pm$0.39 | 39.70$\pm$0.69 | 52.29$\pm$0.62 | 22.39$\pm$0.44 | 25.98$\pm$0.43

Figure 3: t-SNE results of (a) STARTUP and (b) SimCLR+CLD+FD at different
blocks on the EuroSAT dataset. It is obvious that our method can extract more
discriminative features.

###  4.2 Main Results

We select several basic and competitive methods for comparison in Table 1. The
basic model is Transfer that trains the encoder on the labeled base dataset
with the cross-entropy loss. SimCLR (Base) (Chen et al., 2020) trains the
model on the base dataset (miniImageNet) with the contrastive loss of SimCLR.
CI (Luo et al., 2022) trains the encoder on the base dataset and converts the
features of the target dataset with a transformation function in the
evaluation stage. Transfer+SimCLR is proposed in (Islam et al., 2021b) and
exhibits good transferability, which simultaneously optimizes the cross-
entropy loss on the base dataset and the contrastive loss of SimCLR on the
target dataset. STARTUP (Phoo & Hariharan, 2021) trains the model with three
loss functions: cross-entropy loss on the labeled base dataset, and
contrastive loss and KD loss on the unlabeled target domain. ATA (Wang & Deng,
2021) designs a plug-and-play inductive bias-adaptive task augmentation
module. BYOL (Grill et al., 2020) is trained on the target dataset. ConfeSS
(Das et al., 2021) uses labeled target images to find out the useful
components of the target image features. Dynamic-Distillation (Islam et al.,
2021a) designs a distillation loss that draws support from the dynamically
updated teacher network. SimCLR trains the encoder on unlabeled target data
only and shows good generalization ability. For our method, we build two
models BYOL+CLD+FD and SimCLR+CLD+FD, which use Equations 5 and 4 for self-
supervised losses, respectively.

Figure 4: Different single-block distillation structures. B2-to-B1 means that
the second block of the teacher is distilled to the first block of the
student. B3-to-B2 and B4-to-B3 are similar. Table 2: Results of different
single-block distillation stuctures on EuroSAT (5-way 1-shot). B2-to-B1,
B3-to-B2, and B4-to-B3 are defined in Figure 4.

SimCLR | B2-to-B1 | B3-to-B2 | B4-to-B3 | CLD
---|---|---|---|---
75.10$\pm$0.85 | 78.54$\pm$0.80 | 78.38$\pm$0.77 | 77.92$\pm$0.83 | 79.38$\pm$0.82

Table 1 gives the comparisons among our models and the baselines. All the best
results on the four datasets are obtained by either of our two models. In
particular on EuroSAT, our SimCLR+CLD+FD can bring 9.38% and 7.42% gains over
Dynamic-Distillation and SimCLR on the $5$-way $1$-shot task, respectively. On
average, on the four datasets, SimCLD+CLD+FD outperforms Dynamic-Distillation
significantly (58.77% vs. 53.33%) for 1-shot; 66.94% vs. 65.57% for 5-shot.
Besides, although SimCLR+CLD+FD and BYOL+CLD+FD are based on SimCLR and BYOL,
respectively, the performances of SimCLR and BYOL are improved greatly.
Finally, we visualize the feature distributions of STARTUP and our
SimCLR+CLD+FD in Figure 3. It can be seen that SimCLR+CLD+FD exhibits better
clustering results at all the blocks, especially at the last one.

###  4.3 Ablation Study on Cross-level Distillation

Figure 5: Different multi-blocks distillation structures. (a) and (b) are two
common KD methods. (c) is our cross-level KD. Table 3: Results of different
multi-blocks KD structures.

| EuroSAT | CropDisease | ISIC | ChestX
---|---|---|---|---
| 1-shot | 5-shot | 1-shot | 5-shot | 1-shot | 5-shot | 1-shot | 5-shot
SimCLR | 75.10$\pm$0.85 | 90.17$\pm$0.43 | 88.54$\pm$0.80 | 96.09$\pm$0.39 | 35.71$\pm$0.66 | 48.84$\pm$0.61 | 22.00$\pm$0.42 | 24.84$\pm$0.41
same-level distillation | 75.48$\pm$0.85 | 90.71$\pm$0.40 | 88.14$\pm$0.80 | 95.76$\pm$0.44 | 36.56$\pm$0.65 | 50.51$\pm$0.58 | 22.12$\pm$0.42 | 25.20$\pm$0.42
one-to-all distillation | 77.70$\pm$0.83 | 91.80$\pm$0.38 | 89.11$\pm$0.79 | 95.83$\pm$0.44 | 36.69$\pm$0.64 | 50.64$\pm$0.62 | 22.10$\pm$0.43 | 24.77$\pm$0.43
CLD | 79.38$\pm$0.82 | 92.78$\pm$0.35 | 89.43$\pm$0.78 | 96.08$\pm$0.43 | 37.82$\pm$0.68 | 52.28$\pm$0.61 | 22.08$\pm$0.43 | 25.08$\pm$0.42

Figure 6: Rand Index values in the residual blocks of the methods in Table 3.

For simplicity, the combination (according to Equation 1) of the teacher and
the old student in Figure 2 is denoted as the united teacher in Figure 4 and
Figure 5. In these experiments, $\mathcal{L}_{\rm{simclr}}$ is used as
$\mathcal{L}_{\rm{ss}}$ and FD is not employed. Table 2 gives the results of
SimCLR, different single-block distillation structures, and CLD on EuroSAT
(5-way 1-shot). We can notice that all the single-block distillation
structures perform better than SimCLR, and CLD can outperform all the single-
block distillation structures. Table 3 gives the results of SimCLR, different
multi-blocks distillation structures and CLD. Compared with SimCLR without KD,
all the multi-blocks KD structures improve the model’s overall performance.
Second, the one-to-all KD outperforms the same-level KD in most cases.
Finally, our CLD performs best on average among the three multi-blocks KD
structures.

We further explore the reason that CLD can exceed other methods. Rand Index
(Rand, 1971) is a metric to reflect the quality of feature clustering. The
bigger Rand Index is, the better clustering the method provides. Figure 6 is
comprised of the Rand Index values of the features in each residual block on
the EuroSAT test dataset. Our CLD increases more than all the other methods in
each block. It means that our CLD can pick up more discriminative information
at each level so that the model gradually hunts the useful features as the
network deepens.

Next, we examine the usefulness of the old student and the effect of the
training iteration interval $\tau$ (Section 3.2). Table 4 shows the
experimental results of different settings. First, CLD without the old student
outperforms SimCLR. Second, using the old student is better than not using it.
Considering the performance and the memory requirement, we choose $\tau=1$ in
all the other experiments on the four datasets.

Table 4: Effects of the old student and the training iteration interval
$\tau$. All the methods in the table use $\mathcal{L}_{\rm{simclr}}$ as
$\mathcal{L}_{\rm{ss}}$.

| EuroSAT | CropDisease | ISIC | ChestX
---|---|---|---|---
| 1-shot | 5-shot | 1-shot | 5-shot | 1-shot | 5-shot | 1-shot | 5-shot
SimCLR | 75.10$\pm$0.85 | 90.17$\pm$0.43 | 88.54$\pm$0.80 | 96.09$\pm$0.39 | 35.71$\pm$0.66 | 48.84$\pm$0.61 | 22.00$\pm$0.42 | 24.84$\pm$0.41
CLD (w/o old student) | 76.85$\pm$0.83 | 91.58$\pm$0.40 | 88.72$\pm$0.78 | 95.85$\pm$0.43 | 37.04$\pm$0.65 | 50.90$\pm$0.60 | 21.40$\pm$0.32 | 23.91$\pm$0.39
CLD ($\tau=0$) | 78.56$\pm$0.82 | 92.32$\pm$0.35 | 89.11$\pm$0.81 | 95.69$\pm$0.45 | 37.82$\pm$0.69 | 51.17$\pm$0.62 | 22.03$\pm$0.43 | 25.12$\pm$0.44
CLD ($\tau=1$) | 79.38$\pm$0.82 | 92.78$\pm$0.35 | 89.43$\pm$0.78 | 96.08$\pm$0.43 | 37.82$\pm$0.68 | 52.28$\pm$0.61 | 22.08$\pm$0.43 | 25.08$\pm$0.42
CLD ($\tau=10$) | 78.18$\pm$0.81 | 92.17$\pm$0.36 | 89.24$\pm$0.80 | 95.83$\pm$0.44 | 38.02$\pm$0.68 | 51.77$\pm$0.62 | 22.02$\pm$0.43 | 25.04$\pm$0.43
CLD ($\tau=100$) | 78.79$\pm$0.80 | 92.44$\pm$0.37 | 89.56$\pm$0.81 | 96.06$\pm$0.42 | 36.81$\pm$0.65 | 50.60$\pm$0.62 | 22.29$\pm$0.43 | 25.17$\pm$0.43

Figure 7: (a) Accuracy _vs._ $h$ when $\beta=0.4$. (b) Accuracy _vs._ $\beta$
when $h=64$. Table 5: Comparison of using FD or not on EuroSAT (5-way 1-shot).

| SimCLR | BYOL | SimCLR+CLD | BYOL+CLD | one-to-all distillation | Transfer+SimCLR | STARTUP
---|---|---|---|---|---|---|---
w/o FD | 75.10$\pm$0.85 | 64.38$\pm$0.93 | 79.38$\pm$0.82 | 69.47$\pm$0.84 | 77.70$\pm$0.83 | 65.92$\pm$0.88 | 65.24$\pm$0.88
w/ FD | 78.02$\pm$0.77 | 69.80$\pm$0.85 | 82.52$\pm$0.76 | 72.78$\pm$0.84 | 80.62$\pm$0.77 | 63.97$\pm$0.85 | 60.87$\pm$0.88
$\Delta$ | +2.92 | +5.42 | +3.20 | +3.31 | +2.92 | -1.95 | -4.37

###  4.4 Ablation Study on Feature Denoising

On EuroSAT, the model SimCLR+CLD+FD is used to find optimal $h$ and $\beta$ on
the 5-way 1-shot tasks. As shown in Figure 7, $h=64$ and $\beta=0.4$ are
chosen for the best accuracy.

With $h=64$ and $\beta=0.4$ in FD, we compare using FD or not in Table 5. We
can see that the first five models are improved with FD, while the last two
are not. The different positions, which the auxiliary loss (cross-entropy
and/or KD) is applied to in the two groups, are shown in Figures 9(a) and
9(b), respectively. The reason why FD does not help in Figure 9(b) is that the
loss $\mathcal{L}_{\rm{aux}}$ used in the final layer reduces the noisy
elements of the final feature vector for classification.

Figure 8: $\mathcal{L}_{\rm{ss}}$ is the self-supervised loss and
$\mathcal{L}_{\rm{aux}}$ includes other losses such as the cross-entropy and
KD losses. (a) $\mathcal{L}_{\rm{aux}}$ is applied to intermediate layers. (b)
$\mathcal{L}_{\rm{aux}}$ is applied to the final layer.

Figure 9: Results of SimCLR+CLD+FD with the different $\lambda$ values on the
EuroSAT dataset (5-way 1-shot).

###  4.5 Ablation Study on Weight Coefficient of Loss Function

We give the results of SimCLR+CLD+FD with the different $\lambda$ values in
Equation 3 on the EuroSAT dataset (5-way 1-shot), as shown in Figure 9. We can
see that the best result is obtained with $\lambda=2$. So we set $\lambda=2$
in all the related experiments.

##  5 conclusion

In this work, we handle the CD-FSC problems with large domain gaps between the
base dataset and the target datasets. We propose the cross-level distillation
KD for better transferring the knowledge of the base dataset to the student.
We also present the feature denoising operation for mitigating the
overfitting. Our method improves the performance of a strong baseline Dynamic-
Distillation by 5.44$\%$ on 1-shot and 1.37$\%$ on 5-shot classification tasks
on average in the BSCD-FSL benchmark, establishing new state-of-the-art
results.

## Acknowledgements

We would also like to express our sincere gratitude to Mr. Haichen ZHENG, Mrs.
Jing LIN for their unwavering support and confidence in our work.

## References

  * Adler et al. (2020) Thomas Adler, Johannes Brandstetter, Michael Widrich, Andreas Mayr, David Kreil, Michael Kopp, Günter Klambauer, and Sepp Hochreiter.  Cross-domain few-shot learning by representation fusion.  _arXiv:2010.06498_ , 2020.
  * Chen et al. (2020) Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton.  A simple framework for contrastive learning of visual representations.  In _ICML_ , 2020.
  * Chen et al. (2018) Wei-Yu Chen, Yen-Cheng Liu, Zsolt Kira, Yu-Chiang Frank Wang, and Jia-Bin Huang.  A closer look at few-shot classification.  In _ICLR_ , 2018.
  * Chen & He (2021) Xinlei Chen and Kaiming He.  Exploring simple siamese representation learning.  In _CVPR_ , 2021.
  * Codella et al. (2019) Noel Codella, Veronica Rotemberg, Philipp Tschandl, M Emre Celebi, Stephen Dusza, David Gutman, Brian Helba, Aadi Kalloo, Konstantinos Liopyris, Michael Marchetti, et al.  Skin lesion analysis toward melanoma detection 2018: A challenge hosted by the international skin imaging collaboration (isic).  _arXiv:1902.03368_ , 2019.
  * Das et al. (2021) Debasmit Das, Sungrack Yun, and Fatih Porikli.  Confess: A framework for single source cross-domain few-shot learning.  In _ICLR_ , 2021.
  * Fu et al. (2021) Yuqian Fu, Yanwei Fu, and Yu-Gang Jiang.  Meta-fdmixup: Cross-domain few-shot learning guided by labeled target data.  In _ACM MM_ , 2021.
  * Gidaris et al. (2018) Spyros Gidaris, Praveer Singh, and Nikos Komodakis.  Unsupervised representation learning by predicting image rotations.  In _ICLR_ , 2018.
  * Grill et al. (2020) Jean-Bastien Grill, Florian Strub, Florent Altché, Corentin Tallec, Pierre Richemond, Elena Buchatskaya, Carl Doersch, Bernardo Avila Pires, Zhaohan Guo, Mohammad Gheshlaghi Azar, et al.  Bootstrap your own latent-a new approach to self-supervised learning.  In _NeurIPS_ , 2020.
  * Guo et al. (2020) Yunhui Guo, Noel C Codella, Leonid Karlinsky, James V Codella, John R Smith, Kate Saenko, Tajana Rosing, and Rogerio Feris.  A broader study of cross-domain few-shot learning.  In _ECCV_ , 2020.
  * Helber et al. (2019) Patrick Helber, Benjamin Bischke, Andreas Dengel, and Damian Borth.  Eurosat: A novel dataset and deep learning benchmark for land use and land cover classification.  _IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing_ , pp. 2217–2226, 2019.
  * Hinton et al. (2015) Geoffrey Hinton, Oriol Vinyals, Jeff Dean, et al.  Distilling the knowledge in a neural network.  _NeurIPS Workshop_ , 2015.
  * Hua et al. (2021) Tianyu Hua, Wenxiao Wang, Zihui Xue, Sucheng Ren, Yue Wang, and Hang Zhao.  On feature decorrelation in self-supervised learning.  In _ICCV_ , 2021.
  * Islam et al. (2021a) Ashraful Islam, Chun-Fu Richard Chen, Rameswar Panda, Leonid Karlinsky, Rogerio Feris, and Richard J Radke.  Dynamic distillation network for cross-domain few-shot recognition with unlabeled data.  In _NeurIPS_ , 2021a.
  * Islam et al. (2021b) Ashraful Islam, Chun-Fu Richard Chen, Rameswar Panda, Leonid Karlinsky, Richard Radke, and Rogerio Feris.  A broad study on the transferability of visual representations with contrastive learning.  In _ICCV_ , 2021b.
  * Kalibhat et al. (2022) Neha Mukund Kalibhat, Kanika Narang, Hamed Firooz, Maziar Sanjabi, and Soheil Feizi.  Towards better understanding of self-supervised representations.  In _ICML Workshop_ , 2022.
  * Lu et al. (2022) Yuning Lu, Liangjian Wen, Jianzhuang Liu, Yajing Liu, and Xinmei Tian.  Self-supervision can be a good few-shot learner.  In _ECCV_ , 2022.
  * Luo et al. (2022) Xu Luo, Jing Xu, and Zenglin Xu.  Channel importance matters in few-shot image classification.  In _ICML_ , 2022.
  * Mangla et al. (2020) Puneet Mangla, Nupur Kumari, Abhishek Sinha, Mayank Singh, Balaji Krishnamurthy, and Vineeth N Balasubramanian.  Charting the right manifold: Manifold mixup for few-shot learning.  In _WACV_ , 2020.
  * Mohanty et al. (2016) Sharada P Mohanty, David P Hughes, and Marcel Salathé.  Using deep learning for image-based plant disease detection.  _Frontiers in plant science_ , pp. 1419–1432, 2016.
  * Phoo & Hariharan (2021) Cheng Perng Phoo and Bharath Hariharan.  Self-training for few-shot transfer across extreme task differences.  In _ICLR_ , 2021.
  * Rand (1971) William M Rand.  Objective criteria for the evaluation of clustering methods.  _Journal of the American Statistical association_ , pp. 846–850, 1971.
  * Romero et al. (2014) Adriana Romero, Nicolas Ballas, Samira Ebrahimi Kahou, Antoine Chassang, Carlo Gatta, and Yoshua Bengio.  Fitnets: Hints for thin deep nets.  _arXiv:1412.6550_ , 2014.
  * Triantafillou et al. (2021) Eleni Triantafillou, Hugo Larochelle, Richard Zemel, and Vincent Dumoulin.  Learning a universal template for few-shot dataset generalization.  In _ICML_ , 2021.
  * Tseng et al. (2020) Hung-Yu Tseng, Hsin-Ying Lee, Jia-Bin Huang, and Ming-Hsuan Yang.  Cross-domain few-shot classification via learned feature-wise transformation.  In _ICLR_ , 2020.
  * Vinyals et al. (2016) Oriol Vinyals, Charles Blundell, Timothy Lillicrap, Daan Wierstra, et al.  Matching networks for one shot learning.  In _NeurIPS_ , 2016.
  * Wang & Deng (2021) Haoqing Wang and Zhi-Hong Deng.  Cross-domain few-shot classification via adversarial task augmentation.  In _IJCAI_ , 2021.
  * Wang et al. (2017) Xiaosong Wang, Yifan Peng, Le Lu, Zhiyong Lu, Mohammadhadi Bagheri, and Ronald M Summers.  Chestx-ray8: Hospital-scale chest x-ray database and benchmarks on weakly-supervised classification and localization of common thorax diseases.  In _CVPR_ , 2017.
  * Zagoruyko & Komodakis (2017) Sergey Zagoruyko and Nikos Komodakis.  Paying more attention to attention: Improving the performance of convolutional neural networks via attention transfer.  In _ICLR_ , 2017.
  * Zhang et al. (2019) Linfeng Zhang, Jiebo Song, Anni Gao, Jingwei Chen, Chenglong Bao, and Kaisheng Ma.  Be your own teacher: Improve the performance of convolutional neural networks via self distillation.  In _ICCV_ , 2019.

##  Appendix A Appendix

###  A.1 Results of the different batch sizes

We present the results of the SimCLR+CLD+FD trained with different batch sizes
on the EuroSAT dataset (5-way 1-shot) in Figure 10. We can see that the
optimal batch size is 32, so we utilize this value in all the experiments of
our method.

Figure 10: Results of the SimCLR+CLD+FD with the different batch sizes on the
EuroSAT dataset (5-way 1-shot).

###  A.2 Results of the different percentages of unlabeled images

Figure 11 shows that as more unlabeled target images become available, the
accuracy of the model will gradually converge to a plateau.

Figure 11: Results of the SimCLR+CLD+FD with the different percentages on the
EuroSAT dataset (5-way 1-shot).

###  A.3 Results with self-supervised pre-trained teacher

Table 6: Results with self-supervised & supervised pre-trained teacher.

| EuroSAT | CropDisease | ISIC | ChestX
---|---|---|---|---
| 1-shot | 5-shot | 1-shot | 5-shot | 1-shot | 5-shot | 1-shot | 5-shot
SimCLR+CLR+FD (self-supervised) | 80.30$\pm$0.72 | 91.63$\pm$0.36 | 89.94$\pm$0.72 | 96.50$\pm$0.35 | 37.42$\pm$0.46 | 49.36$\pm$0.64 | 22.58$\pm$0.43 | 25.96$\pm$0.43
SimCLR+CLR+FD (supervised) | 82.52$\pm$0.76 | 92.89$\pm$0.34 | 90.48$\pm$0.72 | 96.58$\pm$0.39 | 39.70$\pm$0.69 | 52.29$\pm$0.62 | 22.39$\pm$0.44 | 25.98$\pm$0.43

On the EuroSAT dataset (5-way 1-shot), we give the results of the
SimCLR+FD+CLD where the teacher is pre-trained in a self-supervised scheme.
The performances of the SimCLR+CLD+FD are shown in Table 6.


# Addendum to “Structure of seeds in generalized cluster algebras”

Tomoki Nakanishi

###### Abstract.

This is Addendum to “Structure of seeds in generalized cluster algebras”,
Pacific J. Math. 277 (2015), 201–218. We extend the class of generalized
cluster algebras studied therein to embrace examples in some applications.

###### Key words and phrases:

cluster algebra, generalized cluster algebra

###### 2020 Mathematics Subject Classification:

13F60

The definition of the class of generalized cluster algebras in [Nak15] turned
out to be too restrictive for some applications (e.g., [GM23]). However, it is
easily extended as follows. Let $\mathbb{P}$ be the semifield in Section 2A.
In (2.1) we replace each data $z_{i,s}\in\mathbb{P}$ with
$z_{i,s}\in\mathbb{Z}_{\geq 0}\mathbb{P}$. For $z_{i,s}\neq 0$, let
$z_{i,s}=\sum_{j}m_{j}p_{j}$ ($m_{j}\in\mathbb{Z}_{>0}$,
$p_{j}\in\mathbb{P}$), where the multiplication by $m_{j}$ is in
$\mathbb{Z}_{\geq 0}\mathbb{P}$. Then, we set
$\tilde{z}_{i,s}=\bigoplus_{j}m_{j}p_{j}\in\mathbb{P}$, where the
multiplication by $m_{j}$ is in the semifield $\mathbb{P}$. For $z_{i,s}=0$,
we formally set $\tilde{z}_{i,s}=0$, though this is not an element in
$\mathbb{P}$. Then, we replace the formulas (2.5) and (2.6) with

$\displaystyle{y}^{\prime}_{i}$
$\displaystyle=\begin{cases}\displaystyle{y}_{k}^{-1}&i=k\\\
\displaystyle{y}_{i}\Biggl{(}{y}_{k}^{[\varepsilon{b}_{ki}]_{+}}\Biggr{)}^{d_{k}}\Biggl{(}\bigoplus_{s=0}^{d_{k}}\tilde{z}_{k,s}{y}_{k}^{\varepsilon
s}\Biggr{)}^{-{b}_{ki}}&i\neq k,\\\ \end{cases}$ $\displaystyle
x^{\prime}_{i}$ $\displaystyle=\begin{cases}\displaystyle
x_{k}^{-1}\Biggl{(}\prod_{j=1}^{n}x_{j}^{[-\varepsilon{b}_{jk}]_{+}}\Biggr{)}^{d_{k}}\frac{\displaystyle\sum_{s=0}^{d_{k}}z_{k,s}\hat{y}_{k}^{\varepsilon
s}}{\displaystyle\bigoplus_{s=0}^{d_{k}}\tilde{z}_{k,s}{y}_{k}^{\varepsilon
s}}&i=k\\\ x_{i}&i\neq k,\\\ \end{cases}$

where, if $\tilde{z}_{k,s}=0$ appears in the sum in $\mathbb{P}$, we simply
ignore it.

If we set $\mathbb{P}$ to be the trivial semifield, it embraces the examples
in [GM23]. If there is some $z_{i,s}\not\in\mathbb{P}$, it is no longer a
generalized cluster algebra in the sense of [CS14]; accordingly, the
correspondence given in Section 2A after Proposition 2.5 does not hold.
However, all the other propositions and theorems remain valid with the obvious
modification, namely, replacing
$F_{i}^{t}|_{\mathbb{P}}(\mathbf{y},\mathbf{z})$ with
$F_{i}^{t}|_{\mathbb{P}}(\mathbf{y},\tilde{\mathbf{z}})$ in Theorems 3.22 and
3.23. We thank Yasuaki Gyoda and Kodai Matsushita for discussion.

## References

  * [CS14] L. Chekhov and M. Shapiro, _Teichmüller spaces of Riemann surfaces with orbifold points of arbitrary order and cluster variables_ , Int. Math. Res. Notices 2014 (2014), 2746–2772; arXiv:1111.3963 [math–ph].
  * [GM23] Y. Gyoda and K. Matsushita, _Generalization of Markov Diophantine equation via generalized cluster algebra_ , Electron. J. Combin. 30 (2023), P4.10; arXiv: arXiv:2201.10919 [math.NT].
  * [Nak15] T. Nakanishi, _Structure of seeds in generalized cluster algebras_ , Pacific J. Math. 277 (2015), 201–218; arXiv:1409.5967 [math.RA].