comments comments


# Evidence for the production of three massive vector bosons with the ATLAS
detector

On behalf of the ATLAS Collaboration
Physikalisches Institut, Universität Bonn, Nussallee 12, 53115 Bonn, Germany.
E-mail Supported by the European Research Council grant ERC–CoG–617185 and by
the German Federal Ministry of Education and Research (FSP-103)

###### Abstract:

The search for the production of three massive vector bosons in proton–proton
collisions, performed using data at $\sqrt{s}=13\,{\mathrm{\
Te\kern-1.00006ptV}}$ recorded with the ATLAS detector at the Large Hadron
Collider in the years 2015–2017, corresponding to an integrated luminosity of
$79.8$ fb-1, is presented. Events with two same-sign leptons $\ell$ (electrons
or muons) and at least two reconstructed jets are selected to search for
$WWW\to\ell\nu\ell\nu qq$. Events with three leptons without any same-flavour
opposite-sign lepton pairs are used to search for
$WWW\to\ell\nu\ell\nu\ell\nu$, while events with three leptons and at least
one same-flavour opposite-sign lepton pair and one or more reconstructed jets
are used to search for $WWZ\to\ell\nu qq\ell\ell$. Finally, events with four
leptons are analysed to search for $WWZ\to\ell\nu\ell\nu\ell\ell$ and $WZZ\to
qq\ell\ell\ell\ell$. Evidence for the joint production of three massive vector
bosons is observed with a significance of 4.1 standard deviations, where the
expectation is 3.1 standard deviations.

##  1 Introduction

The joint production of three vector bosons is a rare process in the Standard
Model (SM). Studies of triboson production can test the non-Abelian gauge
structure of the SM theory and any deviations from the SM prediction would
provide hints of new physics at higher energy scales. Triboson production has
been studied at the Large Hadron Collider (LHC) using proton–proton ($pp$)
collision data taken at $\sqrt{s}=8\,{\mathrm{\ Te\kern-1.00006ptV}}$ for
processes involving at least one photon, and for the $WWW$ process [1].

Here, the first evidence for the joint production of three massive vector
bosons in $pp$ collisions using the dataset collected with the ATLAS detector
[2] between 2015 and 2017 at $\sqrt{s}=13\,{\mathrm{\ Te\kern-1.00006ptV}}$
[3], for a total integrated luminosity of $79.8\,\textrm{fb}^{-1}$, is
presented. At leading order (LO) in QCD, the production of three massive
vector bosons ($VVV$, with $V=W,Z$) can proceed via the radiation of each
vector boson from a fermion, from an associated boson production with an
intermediate boson ($W$, $Z/\gamma^{*}$ or $H$) decaying into two vector
bosons, or from a quartic gauge coupling vertex. Representative Feynman
diagrams are shown in Figure 1.

Figure 1: Representative Feynman diagrams at LO for the production of three
massive vector bosons, including diagrams sensitive to triple and quartic
gauge couplings.

##  2 Analysis strategy

Two dedicated searches are performed, one for the $W^{\pm}W^{\pm}W^{\mp}$
(denoted as $WWW$) process and one for the $W^{\pm}W^{\mp}Z$ (denoted as
$WWZ$) and $W^{\pm}ZZ$ (denoted as $WZZ$) processes. To search for the $WWW$
process, events with two same-sign leptons with at least two jets resulting
from $WWW\to\ell\nu\ell\nu qq$ ($\ell=e,\mu$, including $\tau\to\ell\nu\nu$)
or three leptons resulting from $WWW\to\ell\nu\ell\nu\ell\nu$ are considered
and are hereafter referred to as the $\ell\nu\ell\nu qq$ and
$\ell\nu\ell\nu\ell\nu$ channels, respectively. To search for the $WWZ$ and
$WZZ$ (denoted as $WVZ$) processes, events with three or four leptons
resulting from $WVZ\to\ell\nu qq\ell\ell$, $WWZ\to\ell\nu\ell\nu\ell\ell$, and
$WZZ\to qq\ell\ell\ell\ell$ are used. Selection criteria are chosen in order
to ensure there is no overlap between different channels. A discriminant that
separates the $WWW$ or $WVZ$ signal from the background is defined in each
channel. The discriminants are combined using a binned maximum-likelihood fit,
which allows the signal yield and the background normalisations to be
extracted. The combined observable is the signal strength parameter $\mu$
defined as the ratio of the measured $WVV$ cross section to its SM
expectation, where one common ratio is assumed for $WWW$ and $WVZ$.

Signal and background processes were simulated with several Monte Carlo (MC)
event generators. Triboson signal events [4] were generated using Sherpa,
where all three bosons are on-mass-shell, using a factorised approach [5].
Events with an off-mass-shell boson through $WH\rightarrow WVV^{*}$ and
$ZH\rightarrow ZVV^{*}$ were generated using Powheg-Box for the $WWW$
analysis, while for the $WVZ$ analysis Pythia was used. Both on-mass-shell and
off-mass-shell processes were generated at next-to-leading order (NLO) QCD
accuracy and are included in the signal definition.

Diboson ($WW$, $WZ$, $ZZ$), $W/Z+\gamma$ and single boson ($W/Z$+jets)
production, as well as electroweak production of $W^{\pm}W^{\pm}+2$ jets,
$WZ+2$ jets, and $ZZ+2$ jets, were modelled using Sherpa. In order to improve
the agreement between the simulated and observed jet multiplicity
distributions for the $WZ\rightarrow\ell\nu\ell\ell$ and
$ZZ\rightarrow\ell\ell\ell\ell$ events, a jet-multiplicity based reweighting
was applied to the simulated $WZ$ and $ZZ$ samples. Top-quark pair events
($t\bar{t}$) and other background processes containing top quarks were
generated using Powheg-Box or MadGraph5_aMC@NLO.

Leptons are required to pass certain identification quality requirements and
to be isolated from other particles in both the calorimeters and the inner
detector. The lepton isolation cone size is at most $\Delta R=0.2$, except for
the muon isolation in the inner detector, where it is at most $\Delta R=0.3$.
The requirements are more restrictive in the $WWW$ analysis because a larger
contamination is expected from jets misidentified as leptons or leptons from
hadron decays (including $b$\- and $c$-hadron decays), referred to as “non-
prompt” leptons in the following.

A dedicated boosted decision tree (BDT), termed “non-prompt lepton BDT” [6],
is used to reject leptons likely to originate from heavy-flavour decays. In
addition, electrons have to pass the “charge misidentification suppression
BDT” [7] to reject electrons likely to have the electric charge wrongly
measured.

##  3 Analysis targeting $WWW$

The experimental signature of the $\ell\nu\ell\nu qq$ process is the presence
of two same-sign leptons, $E_{\mathrm{T}}^{\mathrm{miss}}$, and two jets. The
signature of the $\ell\nu\ell\nu\ell\nu$ process is the presence of three
leptons and $E_{\mathrm{T}}^{\mathrm{miss}}$. To reduce the background
contributions from processes that have more than two (three) leptons in the
$\ell\nu\ell\nu qq$ ($\ell\nu\ell\nu\ell\nu$) channel a “veto lepton”
definition is introduced. Compared with the nominal lepton selection criteria,
the veto lepton $p_{\rm T}$ threshold is lowered, and the isolation, non-
prompt lepton BDT, charge misidentification suppression BDT, and impact
parameter requirements are removed.

To select $\ell\nu\ell\nu qq$ candidates, events are required to have exactly
two nominal leptons and the same electric charge, at least two jets, and no
identified $b$-jets. Four regions are considered, based on the lepton flavour,
namely $ee$, $e\mu$, $\mu e$, and $\mu\mu$. The invariant mass of the dilepton
system is required to be in the range $40<m_{\ell\ell}<400\,{\mathrm{\
Ge\kern-1.00006ptV}}$. The upper mass limit reduces the contribution from the
$WZ$+jets process. A dijet system, formed by the two jets with the largest
$p_{\rm T}$, is required to have $m_{jj}<300\,{\mathrm{\ Ge\kern-1.00006ptV}}$
and $|\Delta\eta_{jj}|<1.5$. The cuts applied on the dijet system mainly
reduce the contributions from the same-sign $WW$ vector boson scattering
process. Additionally, in the $ee$ final state,
$E_{\mathrm{T}}^{\mathrm{miss}}$ is required to be above $55\,{\mathrm{\
Ge\kern-1.00006ptV}}$ and $m_{\ell\ell}$ must satisfy
$m_{\ell\ell}<80\,{\mathrm{\ Ge\kern-1.00006ptV}}$ or
$m_{\ell\ell}>100\,{\mathrm{\ Ge\kern-1.00006ptV}}$, to reduce contamination
from $Z\to ee$ where the charge of one electron is misidentified. To select
$\ell\nu\ell\nu\ell\nu$ candidates, events are required to have exactly three
nominal leptons and no identified $b$-jets. To reduce the contribution from
the $WZ\to\ell\nu\ell\ell$ process, events are required to have no same-
flavour opposite-sign (SFOS) lepton pairs.

A major background originates from the $WZ$+jets $\to\ell\nu\ell\ell$+jets
process, contributing to the $\ell\nu\ell\nu qq$ channel when one lepton is
not reconstructed or identified, or to the $\ell\nu\ell\nu\ell\nu$ channel
when a $Z$ boson decays into a pair of $\tau$ leptons both of which decay to
an electron or muon. Simulation is used to estimate this background. Data and
simulation agree in a dedicated validation region. Contributions from SM
processes that produce at least one non-prompt lepton are estimated using a
data-driven method as described in Ref. [8], where data control regions are
scaled by a “fake factor”, which is derived from two $t\bar{t}$-enriched
regions selected with two or three leptons (no SFOS lepton pairs) and exactly
one $b$-jet. Events resulting from the $V\gamma jj$ production can pass the
$ee$, $e\mu$ and $\mu e$ signal selection criteria if the photon is
misreconstructed as an electron. This contribution (referred to as “$\gamma$
conv.”) is evaluated using a data-driven method similar to the non-prompt
lepton background evaluation by introducing “photon-like” electrons. The
charge misidentification background originates from processes that produce
oppositely-charged prompt leptons, where one lepton’s charge is misidentified
and results in final states with two same-sign leptons. The background is
estimated using a data-driven technique.

##  4 Analysis targeting $WWZ$ and $WZZ$

The experimental signature of the $WVZ\to\ell\nu qq\ell\ell$,
$WWZ\to\ell\nu\ell\nu\ell\ell$, and $WZZ\to qq\ell\ell\ell\ell$ processes is
the presence of three or four charged leptons. In order to increase the signal
acceptance, “loose” leptons are defined in addition to nominal leptons, the
latter being a subset of the former. Loose leptons have both the isolation and
non-prompt lepton BDT requirements removed. In addition, for loose electrons
the charge misidentification suppression BDT requirement is removed. Six
regions are defined with either three or four loose leptons, sensitive to
triboson final states containing $Z$ bosons. Among all possible SFOS lepton
pairs, the one with $m_{\ell\ell}$ closest to the $Z$ boson mass is defined as
the best $Z$ candidate. In all regions, the presence of such a best $Z$
candidate with $|m_{\ell\ell}-91.2\,{\mathrm{\
Ge\kern-1.00006ptV}}|<10\,{\mathrm{\ Ge\kern-1.00006ptV}}$, is required.
Furthermore, any SFOS lepton pair combination is required to have a minimum
invariant mass of $m_{\ell\ell}>12\,{\mathrm{\ Ge\kern-1.00006ptV}}$. Events
with $b$-tagged jets are vetoed.

For the three-lepton channel, the lepton which is not part of the best $Z$
candidate is required to be a nominal lepton. The scalar sum of the transverse
momenta of all leptons and jets ($H_{\mathrm{T}}$) is required to be larger
than $200\,\,{\mathrm{\ Ge\kern-1.00006ptV}}$. This significantly reduces the
contribution of the $Z\to\ell\ell$ processes with one additional non-prompt
lepton. Three regions are defined according to the number of jets in the
event: one jet (3$\ell$-1j), two jets (3$\ell$-2j), and at least three jets
(3$\ell$-3j). For the four-lepton channel, the third and fourth leading
leptons are required to be nominal leptons. The two leptons which are not part
of the best $Z$ candidate definition are required to have opposite charges.
These “other leptons” are used to define three regions, depending on whether
they are different-flavour (4$\ell$-DF), or same-flavour and their mass lies
within a window of $10\,{\mathrm{\ Ge\kern-1.00006ptV}}$ around the $Z$ boson
mass (4$\ell$-SF-Z) or their mass is outside this window (4$\ell$-SF-noZ).

In each of the six regions the distribution of a dedicated boosted-decision-
tree discriminant, separating the $WVZ$ signal from the dominant diboson
background, is fed as input to the binned maximum-likelihood fit to extract
the signal. For the three-lepton channels, 13, 15, and 12 input variables are
used for the 3$\ell$-1j, 3$\ell$-2j, and 3$\ell$-3j final states,
respectively, while for the four-lepton channels, six input variables are used
for each of the 4$\ell$-DF, 4$\ell$-SF-Z and 4$\ell$-SF-noZ final states.

Due to the required presence of nominal leptons in the three- and four-lepton
channels, backgrounds with a $Z$ boson and non-prompt leptons are reduced. The
remaining backgrounds are dominated by processes with prompt leptons and thus
all backgrounds are estimated using simulation. The $WZ+$jets and $Z$+jets
backgrounds are validated in a region defined in the same way as the
3$\ell$-1j region, with the exception that no requirement on $H_{\mathrm{T}}$
is applied, the third-highest-$p_{\rm T}$ lepton is required to have a small
transverse momentum ($10\,{\mathrm{\ Ge\kern-1.00006ptV}}<p_{\rm
T}<15\,{\mathrm{\ Ge\kern-1.00006ptV}}$), and the invariant mass of the three
leptons has to be smaller than $150\,{\mathrm{\ Ge\kern-1.00006ptV}}$. Data
and expectation agree in a dedicated validation region. The $t\bar{t}Z$
background is determined in a region defined like the 3$\ell$-3j region with
the exception that no requirement on $H_{\mathrm{T}}$ is applied, and at least
four jets are required, of which at least two are $b$-tagged. This region is
included as a single-bin control region (CR) in the fit model.

##  5 Signal extraction and combination

The $WWW$, $WWZ$ and $WZZ$ regions are combined using the profile likelihood
method based on a simultaneous fit to distributions in the signal and
background control regions. A total of eleven signal regions are considered:
four regions ($ee$, $e\mu$, $\mu e$, and $\mu\mu$) for the $\ell\nu\ell\nu qq$
channel, one region ($\mu ee$ and $e\mu\mu$ combined) for the
$\ell\nu\ell\nu\ell\nu$ channel, three regions (3$\ell$-1j, 3$\ell$-2j, and
3$\ell$-3j) for the $WVZ$ three-lepton channel, and three regions (4$\ell$-DF,
4$\ell$-SF-Z, and 4$\ell$-SF-noZ) for the $WVZ$ four-lepton channel. One
control region is considered: the $t\bar{t}Z$ control region described in
Section 4. The distributions used in the fit are the $m_{jj}$ distributions
for the $\ell\nu\ell\nu qq$ channel and the BDT distributions for the $WVZ$
three-lepton and four-lepton channels. The number of selected events in the
$\ell\nu\ell\nu\ell\nu$ channel and the $t\bar{t}Z$ control region are each
included as a single bin in the fit. In total, 186 bins are used in the
combined fit. Correlations of systematic uncertainties arising from common
sources are maintained across processes and channels.

The simultaneous fit model has the power to constrain the normalisations of
the dominant backgrounds from the $WZ$ and $ZZ$ processes at the $\sim$5%
level. The contribution to the $WVV$ signal from $VH$ associated production is
$\sim$40% in the $WWW$ fiducial regions and $\sim$30% in the $WVZ$ fiducial
regions. Figure 2 shows the comparison between data and post-fit prediction of
the combined $m_{jj}$ distribution for the $\ell\nu\ell\nu qq$ channel, the
number of selected events for the $\ell\nu\ell\nu\ell\nu$ channel, and the BDT
output distributions in the 3$\ell$-2j and 4$\ell$-DF regions for the $WVZ$
analysis. The 3$\ell$-2j and 4$\ell$-DF regions are chosen since they have the
best sensitivity among the three-lepton and four-lepton channels. Data and
predictions agree in all distributions.

(a)  (b)

Figure 2:  Post-fit distribution of (a) $m_{jj}$ for the $WWW\to\ell\nu\ell\nu
qq$ analysis ($ee$, $e\mu$, $\mu e$, $\mu\mu$ combined) and (b) the BDT
response in the 4$\ell$-DF channel for the $WVZ$ analysis [3]. The uncertainty
band includes both statistical and systematic uncertainties as obtained by the
fit.

The overall observed (expected) significance for $WVV$ production is found to
be 4.1$\sigma$ (3.1$\sigma$), constituting evidence for the production of
three massive vector bosons. The combined best-fit signal strength for the
$WVV$ process, obtained by the fit to the eleven signal regions and one
control region, is $\mu_{WVV}=1.40^{+0.39}_{-0.37}$ with respect to the SM
prediction. The statistical uncertainty in the measured signal strength is
${}^{+0.25}_{-0.24}$ and the systematic uncertainty is ${}^{+0.30}_{-0.27}$.
The largest systematic uncertainties come from uncertainties related to data-
driven background evaluations affecting the $WWW$ channels, from theoretical
uncertainties related to renormalisation and factorisation scale variations,
and from experimental uncertainties.

Fits are also performed separately in the $WWW$ and the $WVZ$ channels. where
the other signal strength is fixed to its SM expectation. For the fits of the
$WWW$ channels, an additional $WZ$ control region is used in the fit, helping
to constrain the overall normalisation of the $WZ$+jets background, which in
the combined fit is constrained by the $WVZ$ three-lepton signal regions. The
observed (expected) significance is 3.2$\sigma$ (2.4$\sigma$) for $WWW$
production and 3.2$\sigma$ (2.0$\sigma$) for $WVZ$ production.

Decay channel | Significance
---|---
Observed | Expected
$WWW$ combined | 3.2$\sigma$ | 2.4$\sigma$
$WWW\to\ell\nu\ell\nu qq$ | 4.0$\sigma$ | 1.7$\sigma$
$WWW\to\ell\nu\ell\nu\ell\nu$ | 1.0$\sigma$ | 2.0$\sigma$
$WVZ$ combined | 3.2$\sigma$ | 2.0$\sigma$
$WVZ\to\ell\nu qq\ell\ell$ | 0.5$\sigma$ | 1.0$\sigma$
$WVZ\to\ell\nu\ell\nu\ell\ell/qq\ell\ell\ell\ell$ | 3.5$\sigma$ | 1.8$\sigma$
$WVV$ combined | 4.1$\sigma$ | 3.1$\sigma$
Table 1: Observed and expected significances with respect to the SM
background-only hypothesis for the four $WVV$ channels entering the fit.

Table 1 and Figure 3(a) summarise the observed and expected significances with
respect to the background-only hypothesis and the observed best-fit values of
the signal strength for the individual and combined fits. The measured signal
strengths from the individual fits are converted to inclusive cross-section
measurements using the signal samples and the central values of the
theoretical predictions. All uncertainties determined in the fit are included
in the conversion, except for the normalisation uncertainty in the signal
prediction. The results are:
$\sigma_{WWW}=0.65^{+0.16}_{-0.15}\,\textrm{(stat.)}\,^{+0.16}_{-0.14}\,\textrm{(syst.)}$
pb and $\sigma_{WWZ}=0.55\pm
0.14\,\textrm{(stat.)}\,^{+0.15}_{-0.13}\,\textrm{(syst.)}$ pb. For the
$\sigma_{WWZ}$ extraction, the $WZZ$ normalisation is fixed to the SM
expectation. The cross section of the latter is not reported, since there is
not enough sensitivity to this channel to quote a separate cross-section
value.

(a)  (b)

Figure 3: (a) Extracted signal strengths $\mu$ for the four analysis regions
and for the combination. (b) Event yields as a function of
$\log_{10}\left(S/B\right)$ for data, background $B$ and the signal $S$.
Events in all eleven signal regions are included. The background and signal
yields are shown after the global signal-plus-background fit. The hatched band
corresponds to the systematic uncertainties, and the statistical uncertainties
are represented by the error bars on the data points. The lower panel shows
the ratio of the data to the expected background estimated from the fit,
compared to the expected distribution including the signal (red line) [3].

Figure 3(b) shows the data, background and signal yields, where the
discriminant bins in all signal regions are combined into bins of
$\log_{10}(S/B)$, $S$ being the expected signal yield and $B$ the background
yield. The background and signal yields are shown after the global signal-
plus-background fit to the data.

##  6 Summary

The search for the joint production of three massive vector bosons ($W$ or
$Z$) in proton–proton collisions using 79.8 fb-1 of data at
$\sqrt{s}=13\,{\mathrm{\ Te\kern-1.00006ptV}}$ collected by the ATLAS detector
at the LHC, was presented. Events with two, three or four reconstructed
electrons and muons were analysed. First evidence for the production of three
massive vector bosons has been observed with a combined significance of 4.1
standard deviations, where the expectation is 3.1 standard deviations. The
measured production cross sections are $\sigma_{WWW}=0.65^{+0.23}_{-0.21}$ pb,
and $\sigma_{WWZ}=0.55^{+0.21}_{-0.19}$ pb, in agreement with the Standard
Model predictions.

## Acknowledgements

This work was partially funded by the European Research Council under the
European Union’s Seventh Framework Programme ERC Consolidator Grant Agreement
n. 617185 (TopCoup) and by the German Federal Ministry of Education and
Research (BMBF) in FSP-103 under grant n. 05H15PDCAA.

## References

  * [1] ATLAS Collaboration, _Search for triboson $W^{\pm}W^{\pm}W^{\mp}$ production in $pp$ collisions at $\sqrt{s}=$ 8 TeV with the ATLAS detector_, [rgb]0.,0.7,0.Eur. Phys. J. C 77 (2017) 141, arXiv:[rgb]0.,0.7,0.1610.05088 [hep-ex].
  * [2] ATLAS Collaboration, _The ATLAS Experiment at the CERN Large Hadron Collider_ , [rgb]0.,0.7,0.JINST 3 (2008) S08003.
  * [3] ATLAS Collaboration, _Evidence for the production of three massive vector bosons with the ATLAS detector_ , accepted for publication in Phys. Lett. B, arXiv:[rgb]0.,0.7,0.1903.10415 [hep-ex].
  * [4] ATLAS Collaboration, _Multi-Boson Simulation for 13 TeV ATLAS Analyses_ , ATLAS-PHYS-PUB-2017-005, [rgb]0.,0.7,0. https://cds.cern.ch/record/2261933.
  * [5] ATLAS Collaboration, _Multi-Boson Simulation for 13 TeV ATLAS Analyses_ , ATLAS-PHYS-PUB-2016-002, [rgb]0.,0.7,0. https://cds.cern.ch/record/2119986.
  * [6] ATLAS Collaboration, _Evidence for the associated production of the Higgs boson and a top quark pair with the ATLAS detector_ , [rgb]0.,0.7,0.Phys. Rev. D 97 (2018) 072003, arXiv:[rgb]0.,0.7,0.1712.08891 [hep-ex].
  * [7] ATLAS Collaboration, _Electron reconstruction and identification in the ATLAS experiment using the 2015 and 2016 LHC proton–proton collision data at $\sqrt{s}=$ 13 TeV_, [rgb]0.,0.7,0.Eur. Phys. J. C 79 (2019) 639, arXiv:[rgb]0.,0.7,0.1902.04655 [hep-ex].
  * [8] ATLAS Collaboration, _Measurement of $W^{\pm}W^{\pm}$ vector-boson scattering and limits on anomalous quartic gauge couplings with the ATLAS detector_, [rgb]0.,0.7,0.Phys. Rev. D 96 (2017) 012007, arXiv:[rgb]0.,0.7,0.1611.02428 [hep-ex].


# Self-Supervised Learning with Masked Image Modeling for Teeth Numbering,
Detection of Dental Restorations, and Instance Segmentation in Dental
Panoramic Radiographs

Amani Almalki Longin Jan Latecki
Department of Computer and Information Sciences, Temple University,
Philadelphia, USA
<EMAIL_ADDRESS>
###### Abstract

The computer-assisted radiologic informative report is currently emerging in
dental practice to facilitate dental care and reduce time consumption in
manual panoramic radiographic interpretation. However, the amount of dental
radiographs for training is very limited, particularly from the point of view
of deep learning. This study aims to utilize recent self-supervised learning
methods like SimMIM and UM-MAE to increase the model efficiency and
understanding of the limited number of dental radiographs. We use the Swin
Transformer for teeth numbering, detection of dental restorations, and
instance segmentation tasks. To the best of our knowledge, this is the first
study that applied self-supervised learning methods to Swin Transformer on
dental panoramic radiographs. Our results show that the SimMIM method obtained
the highest performance of 90.4% and 88.9% on detecting teeth and dental
restorations and instance segmentation, respectively, increasing the average
precision by 13.4 and 12.8 over the random initialization baseline. Moreover,
we augment and correct the existing dataset of panoramic radiographs. The code
and the dataset are available at https://github.com/AmaniHAlmalki/DentalMIM.

##  1 Introduction

The need for computer-assisted decisions is rising to facilitate diagnosis and
treatment planning for dental care providers. Dental imaging is a valuable
diagnostic tool for diagnosis and treatment plans, which is not possible
solely through clinical exams and patient history [33]. A dental panoramic
X-ray is a comprehensive tool that screens the teeth, surrounding alveolar
bone and upper and lower jaws [28].

Moreover, dental restoration is a biocompatible synthetic material used to
restore missing tooth structures. The missing tooth structure can be restored
with full and partial coverage depending on the extension and intensity of the
missing structure to restore the tooth’s coronal (top) part. Furthermore, root
canal filling is a restorative procedure used to fill the space inside the
tooth structure (root portion) with biocompatible restorative materials.
Various dental restorative materials are available in the dental world; each
has its indication, advantages, disadvantages, and clinician preferences. Most
dental restorative materials appear radiopaque in the x-ray, and they can be
identified by dental care providers [1, 24].

However, manual intervention for teeth numbering and identification of tooth
restorations is time-consuming and may overlook significant data. Thus, the
interest in computer vision and computer science for automated processes was
aroused. Few studies have attempted to apply computer vision algorithms in
dental radiograph analysis. They include convolutional neural networks (CNNs)
for teeth numbering and instance segmentation [21], two-stage network [36],
Faster R-CNN [6, 15, 32, 30, 35], PANet [29], Mask R-CNN [14, 17, 18, 8], and
U-Net network [27, 26, 16]. Recently, CNNs have enormous emerging applications
in analyzing medical images with the advent of computation hardware/algorithm
and expansion in the amount of data [21]. However, CNNs are limited in overall
capability because of inherent inductive biases [11].

In this study, we propose to use a recently introduced Swin Transformer [22]
to analyze dental panoramic radiographs. However, Swin Transformer requires
large data for training, but there is only a very limited number of available
dental radiographs. To alleviate this problem, we propose to use self-
supervised learning. To the best of our knowledge, this is the first study
that applied self-supervised learning methods to Swin Transformer on dental
panoramic radiographs.

Recently, the self-supervised learning methods, SimMIM [34], UM-MAE [19], BEiT
[2], MAE [13], SplitMask [12], MoCo v3 [9], and DINO [5], are effective in
pre-training Transformers [11, 22] for learning visual representation.
However, only UM-MAE and SimMIM pre-training methods are enabled for Pyramid-
based ViTs with locality (Swin Transformer). Generally, the Masked Image
Modeling (MIM) methods mask some image patches before they are fed into the
transformer to predict the original patches in the masked area. This feature
of aggregating information from the context helps many vision tasks. Although
both UM-MAE and SimMIM provide a simple and efficient pre-training strategy
for the Swin transformer encoder [22], the process of the input to the encoder
is dissimilar. MAE discards the masked tokens and inputs only visible patches
to the lightweight decoder. However, MAE also breaks the two-dimensional
structure of the input image. Therefore, it is not applicable to the Swin
transformer without the Uniform Masking (UM) introduced in [19] to bridge the
gap between the MAE and Swin transformer. SimMIM includes the masked tokens in
the encoder and uses them as a direct prediction mechanism. Using the randomly
masked patches for SimMIM is a reasonable reconstruction target, and a
lightweight prediction head is sufficient for pre-training. In addition, the
location of the patches is essential in dental radiographs for a predictable
outcome. SimMIM maintains the location of the patches known to both encoder
and decoder, while MAE drops the location information, which may induce
inaccuracy, as we demonstrate in this paper.

As there is no standard dental image dataset for pre-training (unlike ImageNet
for natural images), SimMIM and UM-MAE are trained on the same dataset as the
downstream tasks (excluding the test dataset). We conduct experiments on
dental image tasks, including teeth numbering, detection of dental
restorations, and instance segmentation on the dental panoramic X-rays dataset
[29]. For these tasks, we use the base Swin Transformer (Swin-B) [22] as the
backbone of Cascade Mask R-CNN [4]. We compare four Swin Transformer
initializations, including SimMIM and UM-MAE, supervised initialization, and
random initialization baseline. Our results show that SimMIM self pre-training
can significantly improve object detection and instance segmentation
performance on dental images.

Although previous studies have investigated teeth segmentation, we still
address many gaps in this work. First, there is no comprehensive instance
segmentation data set for teeth numbering. Previous work on the matter [29]
used modified versions of binary semantic segmentation masks, which leads to a
lack of instance overlapping and low-resolution outputs, resulting in
inaccurate predictions, especially on the boundaries of the teeth. Second,
there is a considerable amount of systematic errors because of the absence of
dental expert supervision. Third, no prior work has simultaneously considered
dental restoration segmentation besides tooth segmentation. The inclusion of
teeth restorations increases the complexity of the computer vision problem
because of class quantity and class imbalance.

To solve the data set issues, we augment and correct the existing dataset
introduced in [29]. In addition to correcting the manual segmentation errors
under expert supervision, we further expand the dataset by developing
annotations for dental restorations, including direct restorations, indirect
restorations, and root canal therapy. The labeling procedure resulted in a
unique high quality, augmented dataset. Our data is available, upon request,
under the name TNDRS (Teeth Numbering, Detection of Restorations, and
Segmentation) annotations.

Our main contributions are twofold:

  * •

We utilize self-supervised learning with SimMIM and UM-MAE to alleviate the
problem of small data for panoramic radiographs.

  * •

The corrected dataset leads to a significant increase in performance, while
added labeling of dental restorations extends the horizon of possible dental
applications.

##  2 Teeth numbering

In dentistry, various dental numbering systems are available for teeth
numbering for adults and children. These numbering systems are universally
accepted for better communication between dental care providers. The Universal
Numbering System, Palmer Notation Numbering System, and Federation Dentiaure
International numbering system (FDI) are the most commonly used system across
the globe among dental professionals. The FDI system is the most widely used
international system. In this system, every single tooth is assigned two-digit
numbers; the first digit number represents each quadrant. The maxillary right
and left quadrants are identified by the numbers 1 and 2, while the mandibular
left and right quadrants are the numbers 3 and 4, respectively. The second
digit numbers represent each tooth based on its location in the jaw from the
middle. The central incisor is assigned to number 1, whereas the third molar
is set to number 8 [32, 31].

##  3 Methods

The methods include two stages: the MIM pre-training and the downstream tasks,
as illustrated in Fig. 1.

Figure 1: Pipeline for teeth detection, detection of dental restorations, and
instance segmentation with MIM Self Pre-training. (a) A Swin Transformer is
first pre-trained by MIM methods on the target dataset. (b) The pre-trained
Swin Transformer is used as the backbone in Cascade Mask R-CNN with FPN for
the detection and segmentation tasks.

In the first stage, Swin Transformer is pre-trained with MIM self-supervised
learning methods as the encoder. SimMIM divides the image into patches,
replacing some random patches with mask tokens. Then, these patches, along
with mask tokens, are input to the Swin encoder. Hence the positional encoding
of both visible and masked patches is preserved, while UM-MAE drops those mask
positions entirely. UM-MAE samples three random patches from each two-by-two
grid, dropping 25% of the entire image. Then it randomly masks 25% of the
already sampled areas as shared learnable tokens. Finally, the sampled patches
and the masked tokens are reorganized as a compact two-dimensional input under
a quarter of the original image resolution to feed via the Swin encoder.

Then a decoder is appended to reconstruct the original patches at the masked
area for both methods. In the second stage, the pre-trained Swin weights are
transferred to initialize the detection and segmentation encoder. The features
of the Swin Transformer backbone are fed to the neck (FPN [20]) and detection
head (Cascade Mask R-CNN) for bounding box regression and classification as
illustrated in Fig. 2. We select the Cascade Mask R-CNN [4] framework due to
its ubiquitous presence in object detection and instance segmentation
research. Then, the whole network is fine-tuned to perform the detection and
segmentation tasks.

Figure 2: Illustration of the architecture for object detection.

We use the base Swin Transformer backbone (Swin-B) and compare the
effectiveness of four configurations as follows:

Random. The network is trained from scratch with randomly initialized weights,
and no self-supervised methods are used. The Swin backbone configuration
follows the code of [22], and the Cascade Mask R-CNN configuration uses the
defaults in MMDetection [7].

Supervised. The Swin backbone is pre-trained for supervised object detection
and instance segmentation using ImageNet-1K [10] images with their labels. We
use the weights from [22] for Swin-B. Swin-B was pre-trained for 300 epochs.

SimMIM. We use the Swin-B weights pre-trained on self-supervised ImageNet-1K
from [34]. This model was pre-trained for 100 epochs.

UM-MAE. Since ImageNet-1K pre-trained weights are not available; we use the
official UM-MAE code release [19] to train Swin-B ourselves for 800 epochs
(the default training length used in [19]) on unsupervised ImageNet-1K.

##  4 Experiments

###  4.1 Dataset augmentation and correction

TNDRS dental panoramic radiographs dataset. Detection, Numbering, and
Segmentation (DNS) [29] is a dental panoramic X-rays dataset consisting of 543
annotated images with ground truth segmentation labels, including numbering
information based on the FDI teeth numbering system. The image size is
1991x1127 pixels. The dataset annotations have some limitations as follows: 1)
lack of instance overlapping; 2) some systematic errors because of the absence
of dental expert supervision; 3) no segmentation of dental restorations. To
overcome these issues, we modify and correct teeth instance segmentation and
overlapping in all images. In addition, we contribute to further expanding the
dataset by developing segmentation for dental restorations, including direct
restorations, indirect restorations, and root canal therapy. This process was
under a supervision of a dentist using the COCO-Annotator tool [3]. We
attended weekly meetings where related issues, such as numbering, dental
restorations, and segmentation questions, were discussed. In the end, the
annotations were reviewed to assure quality and avoid systematic and random
errors. Fig. 3 shows a sample comparing the old and new versions of the
dataset annotations, highlighting both the instance overlapping (blue arrow)
and the correction of systematic errors (green arrow). Fig. 4 presents samples
of segmentation of dental restorations.

Figure 3: Comparison between the old and new dataset annotations. (a) Dataset
old annotations. (b) Dataset new annotations. The blue arrow donates the
inclusion of instance overlapping, while the green arrow indicates the
correction of systematic errors, for example, unsegmented molar roots. Figure
4: Samples of segmentation of dental restorations. Red arrows show an example
of a) indirect restoration, b) direct restoration, and c) root canal therapy.

We believe this is the most inclusive dataset for segmenting teeth and dental
restorations in dental panoramic radiographs. We are providing our data, upon
request, under the name TNDRS (Teeth Numbering, Detection of Restorations, and
Segmentation) annotations.

###  4.2 Evaluation metric

For all our experiments, we split the data into five folds, each containing
approximately 20% of the images. One of these folds is fixed as the test
dataset (consisting of 111 images), and the other four folds (consisting of
108 images each) compose the training and validation datasets in a cross-
validation manner. This process is repeated five times. The evaluation metric
we adopt is the Average Precision for object detection and instance
segmentation models.

###  4.3 Implementation details

Our experiments are implemented based on the PyTorch [25] framework and
trained with NVIDIA Tesla Volta V100 GPUs. In all experiments, the batch size
equals the total number of the training sample, which is 432. The input images
are all resized to 800×600 pixels. We utilize the AdamW [23] optimizer in all
experiments.

Data augmentation. We apply noise addition and horizontal flipping, which
changes teeth numbers to their equivalent new values (left teeth numbers
turned into the right numbers and vice-versa).

SimMIM pre-training. The base learning rate is set to 8e-4, weight decay is
0.05, $\beta 1$ = 0.9, $\beta 2$ = 0.999, with a cosine learning rate
scheduler with warm-up for 10 epochs. We use a random MIM with a patch size of
16×16 and a mask ratio of 20%. We employ a linear prediction head with a
target image size of 800×600 and use L1 loss to compute the loss for masked
pixel prediction.

UM-MAE pre-training. The base learning rate is set to 1.5e-4, weight decay is
0.05, $\beta 1$ = 0.9, $\beta 2$ = 0.95, with a cosine decay learning rate
scheduler with warm-up for 10 epochs. We use a random MIM with a patch size of
16×16 and a mask ratio of 25%. We employ a linear prediction head with a
target image size of 800×600 and adopt mean squared error (MSE) to compute the
loss for masked pixel prediction.

Task fine-tuning. For downstream tasks, we utilize single-scale training. The
initial learning rate is 0.0001, and the weight decay is 0.05.

##  5 Results and analysis

SimMIM and UM-MAE reconstruction. The reconstruction results of SimMIM and UM-
MAE are shown in Fig. 5. The five columns show the original images, the UM-MAE
masked images, the UM-MAE reconstructed images, the SimMIM masked images, and
the SimMIM reconstructed images. The results show that both MIM methods can
restore lost information from the random context. It is worth noting that the
ultimate goal of the MIM is to benefit the downstream tasks instead of
generating high-quality reconstructions.

Figure 5: SimMIM and UM-MAE reconstruction results. The first column is the
original image, and the second and fourth columns are the masked image where
the masked region is denoted by gray patches. The third and fifth columns are
the reconstruction of MIM from the unmasked patches.

###  5.1 Quantitative results

Comparing initializations. Table 1 shows the results of teeth detection and
instance segmentation only and compares them to the previously published
article from Silva _et al_. [29]. We present TNDRS fine-tuning results using
the pre-trained models and random configurations described in Section 3. We
make several observations.

(1) All four Swin Transformer initializations surpass the CNN-based SOTA of
PANet with ResNet-50 backbone using ImageNet pre-training from Silva _et al_.
[29].

(2) Fine-tuning from supervised IN-1K pre-training yields 3.4 higher
${AP}^{box}$ than training from scratch (79.1 vs. 75.7) and 3.5 higher
${AP}^{mask}$ (78.3 vs. 74.8).

(3) UM-MAE substantially outperforms supervised initialization by 5.4
${AP}^{box}$ (84.5 vs. 79.1), and 4.9 ${AP}^{mask}$ (83.2 vs. 78.3).

(4) SimMIM outperforms UM-MAE by 1.6 ${AP}^{box}$ (86.1 vs. 84.5), and 1.4
${AP}^{mask}$ (84.6 vs. 83.2).

Initialization | Backbone | Pre-training Data | ${AP}^{box}$ | ${AP}^{mask}$
---|---|---|---|---
PANet[29] | ResNet-50 | IN-1K w/ Labels | 75.4 | 73.9
Random | Swin-B | None | 75.7 | 74.8
Supervised | Swin-B | IN-1K w/ Labels | 79.1 | 78.3
UM-MAE | Swin-B | IN-1K | 84.5 | 83.2
SimMIM | Swin-B | IN-1K | 86.1 | 84.6
Table 1: Results of teeth detection and instance segmentation only.

Table 2 compares the four Swin Transformer initializations after data
augmentation of dental restorations. Our results prove that the SimMIM method
achieved the highest performance of 90.4% and 88.9% on detecting teeth and
dental restorations and instance segmentation, respectively.

Initialization | Backbone | Pre-training Data | ${AP}^{box}$ | ${AP}^{mask}$
---|---|---|---|---
Random | Swin-B | None | 77.0 | 76.1
Supervised | Swin-B | IN-1K w/ Labels | 80.3 | 79.2
UM-MAE | Swin-B | IN-1K | 88.3 | 85.7
SimMIM | Swin-B | IN-1K | 90.4 | 88.9
Table 2: Results after augmenting dental restorations.

Parameter setting. In Table 3, we conduct experiments on teeth detection and
instance segmentation tasks with different SimMIM pre-training epochs and mask
ratios. First, the performance of SimMIM does not benefit from longer
training. Second, unlike the high mask ratio [34] adopted in natural images,
the downstream tasks show different preferences for the mask ratio. Both tasks
are consistently improved with a decrease in mask ratio from 60% to 10%. The
reason why this decrease facilitates the training may be attributed to the
fact that the relevant features are small on panoramic X-rays.

Mask ratio | Pre-training Epochs | ${AP}^{box}$ | ${AP}^{mask}$
---|---|---|---
60% | 100 | 84.3 | 83.2
50% | 100 | 84.7 | 83.6
50% | 800 | 83.1 | 83.0
40% | 100 | 85.5 | 83.9
30% | 100 | 85.9 | 84.1
20% | 100 | 86.1 | 84.6
10% | 100 | 85.8 | 84.3
Table 3: The influence of Mask Ratios on teeth detection and instance
segmentation tasks.

Dataset correction. After we correct teeth segmentation on DNS discussed in
Section 4.1, teeth detection and instance segmentation performance are
remarkably improved by 5.9 ${AP}^{box}$ and 6.4 ${AP}^{mask}$ as shown in
Table 4.

DNS Annotations | ${AP}^{box}$ | ${AP}^{mask}$
---|---|---
Before Correction | 80.2 | 78.2
After Correction | 86.1 | 84.6
Table 4: Correction of teeth segmentation.

###  5.2 Qualitative results

In Fig. 6, the displayed results for four different images demonstrate
qualitative samples of improved performance when Swin Transformer is pre-
trained with SimMIM for teeth detection and segmentation only. These
improvements in detection and segmentation agree with the quantitative results
in Section 5.1.

Figure 6: Qualitative results of teeth detection and instance segmentation
only. Note that teeth detection and instance segmentation are missing (white
arrows) when created by the baseline Swin Transformer approach compared to the
segmentation produced by Swin Transformer pre-trained with SimMIM architecture
(orange arrows).

Fig. 7 displays qualitative results after augmenting dental restorations when
Swin Transformer is pre-trained with SimMIM.

Figure 7: Qualitative results of detecting teeth and dental restorations and
instance segmentation using SimMIM.

###  5.3 Pre-training time and memory consumption

Comparing UM-MAE to the SimMIM framework, the core advantage of UM-MAE is the
memory and runtime efficiency. In Table 5, we show their clear comparisons
based on Swin-B. It is observed that UM-MAE speeds up by about 2× and reduces
the memory by at least 2× against SimMIM, where their performances under the
downstream tasks show the opposite.

Method | Time | Memory
---|---|---
SimMIM | 24.6 h | 18.7 GB
UM-MAE | 12.5 h | 6.7 GB
Table 5: The comparison of pre-training time and memory consumption.

##  6 Conclusions

Two self-supervised learning methods were applied to Swin Transformer on
dental panoramic radiographs: SimMIM and UM-MAE. The results of the masking-
based method, SimMIM, obtained superior performance than UM-MAE, supervised
and random initialization for detection of teeth, dental restorations, and
instance segmentation. Based on this experiment, we can conclude that
adjusting parameters, including mask ratio and pre-training epochs, is useful
when applying SimMIM pre-training to the dental imaging domain for reliable
outcomes. In addition, correcting the dataset annotations lead to further
improvements that significantly surpass the available state-of-the-art
results. Our plan for future work is to examine the efficacy of SimMIM pre-
training in prognosis and outcome prediction tasks.

##  7 Acknowledgments

We would like to express our deepest thanks to Dr. Abdulrahman Almalki, a
dental expert from the University of Pennsylvania, for his valuable
discussions related to dentistry. This work was in part supported by the NSF
Grant IIS-1814745. The calculations were carried out on HPC resources at
Temple University supported in part by the NSF through grant number 1625061
and by the U.S. ARL under contract number W911NF-16-2-0189.

## References

  * [1] Ragda Abdalla-Aslan, Talia Yeshua, Daniel Kabla, Isaac Leichter, and Chen Nadler.  An artificial intelligence system using machine-learning for automatic detection and classification of dental restorations in panoramic radiography.  Oral Surgery, Oral Medicine, Oral Pathology and Oral Radiology, 130(5):593–602, 2020.
  * [2] Hangbo Bao, Li Dong, and Furu Wei.  Beit: Bert pre-training of image transformers.  arXiv preprint arXiv:2106.08254, 2021.
  * [3] Justin Brooks.  COCO Annotator.  https://github.com/jsbroks/coco-annotator/, 2019.
  * [4] Zhaowei Cai and Nuno Vasconcelos.  Cascade r-cnn: high quality object detection and instance segmentation.  IEEE transactions on pattern analysis and machine intelligence, 43(5):1483–1498, 2019.
  * [5] Mathilde Caron, Hugo Touvron, Ishan Misra, Hervé Jégou, Julien Mairal, Piotr Bojanowski, and Armand Joulin.  Emerging properties in self-supervised vision transformers.  In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 9650–9660, 2021.
  * [6] Hu Chen, Kailai Zhang, Peijun Lyu, Hong Li, Ludan Zhang, Ji Wu, and Chin-Hui Lee.  A deep learning approach to automatic teeth detection and numbering based on object detection in dental periapical films.  Scientific reports, 9(1):1–11, 2019.
  * [7] Kai Chen, Jiaqi Wang, Jiangmiao Pang, Yuhang Cao, Yu Xiong, Xiaoxiao Li, Shuyang Sun, Wansen Feng, Ziwei Liu, Jiarui Xu, et al.  Mmdetection: Open mmlab detection toolbox and benchmark.  arXiv preprint arXiv:1906.07155, 2019.
  * [8] Liang-Chieh Chen, George Papandreou, Iasonas Kokkinos, Kevin Murphy, and Alan L Yuille.  Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs.  IEEE transactions on pattern analysis and machine intelligence, 40(4):834–848, 2017.
  * [9] Xinlei Chen, Saining Xie, and Kaiming He.  An empirical study of training self-supervised vision transformers.  In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 9640–9649, 2021.
  * [10] Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei.  Imagenet: A large-scale hierarchical image database.  In 2009 IEEE conference on computer vision and pattern recognition, pages 248–255. Ieee, 2009.
  * [11] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al.  An image is worth 16x16 words: Transformers for image recognition at scale.  arXiv preprint arXiv:2010.11929, 2020.
  * [12] Alaaeldin El-Nouby, Gautier Izacard, Hugo Touvron, Ivan Laptev, Hervé Jegou, and Edouard Grave.  Are large-scale datasets necessary for self-supervised pre-training?  arXiv preprint arXiv:2112.10740, 2021.
  * [13] Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Dollár, and Ross Girshick.  Masked autoencoders are scalable vision learners.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 16000–16009, 2022.
  * [14] Gil Jader, Jefferson Fontineli, Marco Ruiz, Kalyf Abdalla, Matheus Pithon, and Luciano Oliveira.  Deep instance segmentation of teeth in panoramic x-ray images.  In 2018 31st SIBGRAPI Conference on Graphics, Patterns and Images (SIBGRAPI), pages 400–407. IEEE, 2018.
  * [15] Changgyun Kim, Donghyun Kim, HoGul Jeong, Suk-Ja Yoon, and Sekyoung Youm.  Automatic tooth detection and numbering using a combination of a cnn and heuristic algorithm.  Applied Sciences, 10(16):5624, 2020.
  * [16] Thorbjørn Louring Koch, Mathias Perslev, Christian Igel, and Sami Sebastian Brandt.  Accurate segmentation of dental panoramic radiographs with u-nets.  In 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019), pages 15–19. IEEE, 2019.
  * [17] Jeong-Hee Lee, Sang-Sun Han, Young Hyun Kim, Chena Lee, and Inhyeok Kim.  Application of a fully deep convolutional neural network to the automation of tooth segmentation on panoramic radiographs.  Oral surgery, oral medicine, oral pathology and oral radiology, 129(6):635–642, 2020.
  * [18] André Ferreira Leite, Adriaan Van Gerven, Holger Willems, Thomas Beznik, Pierre Lahoud, Hugo Gaêta-Araujo, Myrthel Vranckx, and Reinhilde Jacobs.  Artificial intelligence-driven novel tool for tooth detection and segmentation on panoramic radiographs.  Clinical oral investigations, 25(4):2257–2267, 2021.
  * [19] Xiang Li, Wenhai Wang, Lingfeng Yang, and Jian Yang.  Uniform masking: Enabling mae pre-training for pyramid-based vision transformers with locality.  arXiv preprint arXiv:2205.10063, 2022.
  * [20] Tsung-Yi Lin, Piotr Dollár, Ross Girshick, Kaiming He, Bharath Hariharan, and Serge Belongie.  Feature pyramid networks for object detection.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 2117–2125, 2017.
  * [21] Geert Litjens, Thijs Kooi, Babak Ehteshami Bejnordi, Arnaud Arindra Adiyoso Setio, Francesco Ciompi, Mohsen Ghafoorian, Jeroen Awm Van Der Laak, Bram Van Ginneken, and Clara I Sánchez.  A survey on deep learning in medical image analysis.  Medical image analysis, 42:60–88, 2017.
  * [22] Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo.  Swin transformer: Hierarchical vision transformer using shifted windows.  In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 10012–10022, 2021.
  * [23] Ilya Loshchilov and Frank Hutter.  Decoupled weight decay regularization.  arXiv preprint arXiv:1711.05101, 2017.
  * [24] Björn Molander.  Panoramic radiography in dental diagnostics.  Swedish Dental journal. Supplement, 119:1–26, 1996.
  * [25] Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al.  Pytorch: An imperative style, high-performance deep learning library.  Advances in neural information processing systems, 32, 2019.
  * [26] Olaf Ronneberger, Philipp Fischer, and Thomas Brox.  Dental x-ray image segmentation using a u-shaped deep convolutional network.  In International Symposium on Biomedical Imaging, volume 1, pages 1–13, 2015.
  * [27] Olaf Ronneberger, Philipp Fischer, and Thomas Brox.  U-net: Convolutional networks for biomedical image segmentation.  In International Conference on Medical image computing and computer-assisted intervention, pages 234–241. Springer, 2015.
  * [28] VE Rushton and K Horner.  The use of panoramic radiology in dental practice.  Journal of dentistry, 24(3):185–201, 1996.
  * [29] Bernardo Silva, Laís Pinheiro, Luciano Oliveira, and Matheus Pithon.  A study on tooth segmentation and numbering using end-to-end deep neural networks.  In 2020 33rd SIBGRAPI Conference on Graphics, Patterns and Images (SIBGRAPI), pages 164–171. IEEE, 2020.
  * [30] Karen Simonyan and Andrew Zisserman.  Very deep convolutional networks for large-scale image recognition.  arXiv preprint arXiv:1409.1556, 2014.
  * [31] DB Smith.  The numbering of teeth.  New Zealand School Dental Service gazette, 37(4):56, 1976.
  * [32] Dmitry V Tuzoff, Lyudmila N Tuzova, Michael M Bornstein, Alexey S Krasnov, Max A Kharchenko, Sergey I Nikolenko, Mikhail M Sveshnikov, and Georgiy B Bednenko.  Tooth detection and numbering in panoramic radiographs using convolutional neural networks.  Dentomaxillofacial Radiology, 48(4):20180051, 2019.
  * [33] Stuart C White, Edward W Heslop, Lars G Hollender, Kristine M Mosier, Axel Ruprecht, Michael K Shrout, et al.  Parameters of radiologic care: An official report of the american academy of oral and maxillofacial radiology.  Oral Surgery, Oral Medicine, Oral Pathology, Oral Radiology, and Endodontology, 91(5):498–511, 2001.
  * [34] Zhenda Xie, Zheng Zhang, Yue Cao, Yutong Lin, Jianmin Bao, Zhuliang Yao, Qi Dai, and Han Hu.  Simmim: A simple framework for masked image modeling.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9653–9663, 2022.
  * [35] Kailai Zhang, Ji Wu, Hu Chen, and Peijun Lyu.  An effective teeth recognition method using label tree with cascade network structure.  Computerized Medical Imaging and Graphics, 68:61–70, 2018.
  * [36] Yue Zhao, Pengcheng Li, Chenqiang Gao, Yang Liu, Qiaoyi Chen, Feng Yang, and Deyu Meng.  Tsasnet: Tooth segmentation on dental panoramic x-ray images by two-stage attention segmentation network.  Knowledge-Based Systems, 206:106338, 2020.


# Third-order Electrical Conductivity of the Charge-ordered Organic Salt
$\alpha$-(BEDT-TTF)2I3

Mayu Ishii  Ryuji Okazaki<EMAIL_ADDRESS>Masafumi Tamura  Department of
Physics, Faculty of Science and Technology, Tokyo University of Science, Noda
278-8510, Japan

###### Abstract

We performed third-order electrical conductivity measurements on the organic
conductor $\alpha$-(BEDT-TTF)2I3 using an ac bridge technique sensitive to
nonlinear signals. Third-order conductance $G_{3}$ is clearly observed even at
low electric fields, and interestingly, $G_{3}$ is critically enhanced above
the charge-order transition temperature $T_{\rm CO}=136$ K. The observed
frequency dependence of $G_{3}$ is incompatible with a percolation model, in
which a Joule heating in a random resistor network is relevant to the
nonlinear conduction. We instead argue the nonlinearity of the relaxation time
according to a phenomenological model on the mobility in materials with large
dielectric constants, and find that the third-order conductance $G_{3}$
corresponds to the third-order electric susceptibility $\chi_{3}$. Since the
nonlinear susceptibility is known as a probe for higher-order multipole
ordering, the present observation of the divergent behavior of $G_{3}$ above
$T_{\rm CO}$ reveals an underlying quadrupole instability at the charge-order
transition of the organic system.

Electronic crystallization on a lattice, known as charge ordering, is an
emergent phenomenon driven by strong electron-electron interactions [1, 2, 3,
4]. It is well materialized in a class of organic molecular conductors, and a
quasi-two-dimensional (quasi-2D) charge transfer salt $\alpha$-(ET)2I3 [ET
being bis(ethylenedithio)-tetrathiafulvalene (BEDT-TTF)] is known as a quite
essential material to show charge ordering. $\alpha$-(ET)2I3 exhibits a metal-
insulator transition at $T_{\rm CO}=136$ K [5], below which a stripe-type
charge disproportionation (charge ordering) has been observed by various
experimental methods including nuclear magnetic resonance [6, 7], optical
measurements [8, 9, 10, 11, 12, 13], and synchrotron radiation x-ray
diffraction [14]. An electronic ferroelectricity without inversion symmetry is
detected by second harmonic generation technique [15] and the polarization
switching is also confirmed [16]. Interestingly, this charge-ordered state is
highly sensitive to the external perturbations to realize massless Dirac
fermions in hydrostatic pressure and superconducting state under uniaxial
pressure [17, 18, 19, 20].

Among the various intriguing properties of $\alpha$-(ET)2I3, the detailed
response function for the electric field, which may act as a conjugate field
to the electronic crystal, is of fundamental interest. Indeed, charge ordering
in $\alpha$-(ET)2I3 shows nontrivial behavior for applied electric field [21,
22, 23, 24, 25, 26, 27, 28, 29, 30, 31], and moreover, a novel current-induced
thermoelectric phenomenon has been also suggested recently [32, 33]. Dressel
et al. first reported a nonlinear electrical conductivity above the threshold
fields [21], reminiscent of sliding motion of charge density wave (CDW).
Voltage oscillation and Shapiro steps have been subsequently observed above
the threshold fields [22, 23], indicating a collective excitation of charge
ordering similar to CDW [24]. In addition, nonequilibrium hot-electron state
is recently proposed by simultaneous optical and transport measurements at
high fields [25], while a scanning Raman microspectroscopy experiment has
clearly shown that application of such a high electric field triggers a
filamentary-like spatially-inhomogeneous state [26], which often becomes an
extrinsic origin of nonlinear conduction [34]. At low fields where such a
spatial inhomogeneity is irrelevant, a nonlinear conduction is also examined
and the observed power-law current($I$)-voltage($V$) curves ($I\propto
V^{\alpha}$) are explained by thermal excitation of electron-hole pairs from
2D logarithmic potential [27]. In particular, the power-law exponent $\alpha$
in low-field range abruptly changes from $\alpha=1$ to 3 below $T_{\rm
KT}\approx 35$ K, suggesting an occurrence of a Kosterlitz-Thouless transition
at $T_{\rm KT}$ [28]. In contrast, above $T_{\rm KT}$, an Ohmic $I$-$V$ curve
($\alpha=1$) is obtained at low fields even in the charge-ordered phase; a
crucial issue how the nonlinear response emerges with the formation of the
charge ordering at $T_{\rm CO}$ has not been discussed.

The aim of this paper is to clarify such an unexplored relation in
$\alpha$-(ET)2I3 by means of the third-order electrical conductivity
measurements. This harmonic method enables us to directly measure the
nonlinear conductivity in low-field range by detecting $3\omega$ component for
the applied voltage with a fundamental frequency $\omega$, in contrast to
conventional pulsed technique in which the self-heating effect is unavoidable
at high fields. The third-order conductance $G_{3}$ has been observed in a
whole temperature range across $T_{\rm CO}$ and largely enhanced at $T_{\rm
CO}$. To explain this nonlinearity, we refer to the mobility of the polar
materials recently investigated, and find that the third-order conductance
$G_{3}$ approximately corresponds to the third-order electric susceptibility
$\chi_{3}$, which expresses a quadrupole fluctuation. Our present results thus
imply a quadrupole nature in the charge-ordered organic salt $\alpha$-(ET)2I3.

Figure 1:  (Color online) (a) In-phase components of third-harmonic current
$I_{3}$ as a function of the applied voltage to the sample $V_{\rm s}$
measured at several temperatures. The dashed lines represent the $V_{\rm
s}^{3}$ dependence. The inset depicts the bridge circuit to exclude the linear
voltage into the lock-in amplifier. (b) The frequency dependence of the third-
harmonic voltage $V_{3}$ measured at $T=136$ K with the sample voltage of
$V_{\rm s}\approx 6$ mV. The dashed lines represent the $f^{-1/2}$ dependence,
which is expected in the case that the self-heating effect is dominant.

Single crystals of $\alpha$-(ET)2I3 were prepared by an electrochemical
method. Two gold wires were attached by using a gold paste to reduce the
contact resistance [35] and the voltage was applied along the in-plane
direction. The rate of temperature change is lower than 0.3 K/min to prevent
the damage to the sample. As schematically shown in the inset of Fig. 1(a),
third-order electrical conductivity measurements were performed by using a
bridge circuit to exclude the extrinsic third-order terms coming from
nonlinearities of voltage source and electrical circuit inside the lock-in
amplifier [36, 37, 38]. When a sample exhibits nonlinear conductivity,
electrical current $I$ flowing in the sample under the applied voltage $V_{\rm
s}$, is expressed by a Taylor series as

$\displaystyle I\simeq I_{1}+I_{3}=G_{1}V_{\rm s}+G_{3}V_{\rm s}^{3},$ (1)

where $I_{1}$ ($G_{1}$) and $I_{3}$ ($G_{3}$) are linear and third-order
current (conductance), respectively [38], and the higher-order terms are
assumed to be negligible. Note that the inversion symmetry is broken below
$T_{\rm CO}$ in $\alpha$-(ET)2I3 but we have not applied a magnetic field in
this experiment, so the second-order term $G_{2}V_{\rm s}^{2}$ is not
considered here [39]. Thus, the sample can be regarded as a parallel circuit
consisting of a linear resistance $R_{1}=G_{1}^{-1}$ and a nonlinear current
source $I_{3}$.

In the bridge circuit, the input voltage to the lock-in amplifier $V_{\rm in}$
is then given as

$\displaystyle V_{\rm in}=\frac{R_{1}R^{\prime\prime}-R_{\rm
v}R^{\prime}}{(R_{1}+R^{\prime})(R_{\rm v}+R^{\prime\prime})}V_{\rm
source}-\frac{I_{3}}{1/R_{1}+1/R^{\prime}},$ (2)

where $R_{\rm v}$, $R^{\prime}$, and $R^{\prime\prime}$ are resistances
constituting the bridge circuit and $V_{\rm source}$ is the source voltage
[38]. The voltage signal measured by a lock-in amplifier $V_{\rm m}$ must
contain small but finite nonlinear components as $V_{\rm m}\simeq V_{\rm
in}\left\\{1+\beta(V_{\rm in})\right\\},$ where non-dimensional $\beta(V_{\rm
in})$ is a series of $V_{\rm in}$ to express the nonlinearity in the lock-in
amplifier, order of which is typically $10^{-4}$ [38]. Here, in the balanced
condition of $R_{1}R^{\prime\prime}=R_{\rm v}R^{\prime}$, the first term in
the right-hand side of Eq. (2) becomes zero, and then the nonlinearity of the
voltage source can be excluded. In addition, in the balanced condition, one
can also eliminate the nonlinearity of the lock-in amplifier since the third-
order term, $-I_{3}/(1/R_{1}+1/R^{\prime})$, becomes the leading term.
Moreover, in the measured frequency range, the frequency dependence of the
linear conductivity is negligible, i.e. $G_{1}(\omega)=G_{1}(3\omega)$ [16],
so that the balanced condition is kept both in $\omega$ and $3\omega$. Note
that this harmonic technique is similar to the $3\omega$ method for thermal
conductivity measurements [40, 41] and the ac method for the specific heat
measurements [42, 43], in which the harmonic signals from the heater and the
thermometer are respectively measured, while the present method detects the
third-harmonic voltage of the sample directly.

In this experiments, we applied the ac voltage $V_{\rm
source}(t)=V_{0}\sin\omega t$ to the circuit with the frequency of
$f=\omega/2\pi=89$ Hz, and measured the fundamental voltage $V_{1}$ and the
third-harmonic voltage $V_{3}$ simultaneously as $V_{\rm m}=V_{1}\sin\omega
t+V_{3}\sin 3\omega t$ by using two lock-in amplifiers to detect $1\omega$ and
$3\omega$ signals. We then balance the circuit by adjusting variable
resistance $R_{v}$ to achieve $V_{1}=0$, and obtain $I_{3}$ by using measured
$V_{3}$ as

$\displaystyle I_{3}=-(1/R_{1}+1/R^{\prime})V_{3}=G_{3}V_{\rm s}^{3},$ (3)

where $V_{\rm s}\simeq V_{0}R_{1}/(R_{1}+R^{\prime})$ is the applied voltage
to the sample [38]. It is worth to note that the measured voltage ratio
$V_{3}/V_{1}$ is as small as the order of $10^{-6}$, which demonstrates the
necessity of the bridge technique to exclude extrinsic nonlinearities as
mentioned above. We also utilized a four-terminal double-bridge method [37]
for the low-resistance range and the results were similar to that of the two-
terminal bridge method.

Figure 2:  (Color online) (a) Temperature variations of the linear conductance
$G_{1}$ (squares) and the in-phase component of the third-harmonic conductance
$G_{3}$ (circles). Solid curve is the linear conductance measured with a
constant current of $I=$ 10 nA. The inset shows temperature dependence of
$dG_{1}/dT$. (b,c) In-phase and out-of-phase components of the third-harmonic
conductance $G_{3}$ as a function of temperature. Dashed line represents the
charge order transition temperature $T_{\rm CO}=$136 K.

Figure 1(a) depicts the in-phase component of the third-harmonic current
$I_{3}$ as a function of $V_{\rm s}$ obtained at several temperatures.
Hereafter all the current and voltage data are expressed by rms. Although the
signals are comparable to the noise level at the low-current range below
$10^{-12}$ A, which is close to the noise level in the earlier report [28],
$I_{3}$ is certainly proportional to $V_{\rm s}^{3}$, and the slope gives the
third-order conductance $G_{3}$ according to Eq. (3). We emphasize that the
electric field corresponding the present voltage is lower than 10 V/cm at
$T=100$ K, which is fairly low compared with the earlier reports [22, 23, 25].

Figure 2(a) compares temperature dependence of the fundamental and third-order
conductances $G_{1}$ and $G_{3}$, both of which are the in-phase components.
Note that, although we utilize the two-terminal method owing to the usage of
bridge circuit, the obtained temperature variation of
$\sigma_{1}=G_{1}\times(L/S)$, where $L\approx 4\times 10^{-2}$ cm and
$S\approx 1\times 10^{-5}$ cm2 are the length between the terminals and the
cross-sectional area respectively, is close to that measured by the four-
terminal method, implying that the effect of the contact resistance formed by
the gold paint is negligible. Figures 2(b) and 2(c) represent the temperature
variations of the in-phase and out-of-phase components of third-order
conductance $G_{3}$, respectively. Interestingly, $G_{3}$ is observed even
above $T_{\rm CO}=136$ K, and compared with temperature dependence of $G_{1}$,
$G_{3}(T)$ is notably enhanced near $T_{\rm CO}$, as is discussed in the
following section.

Table 1: A simple sort of origins for the nonlinear conduction in low field range. $\sigma(E)$ |  $\sigma[T(E)]$ |  $\sigma[T_{\rm p}(E)]$ |  self-heating effect [37]
---|---|---|---
$\sigma[T_{\rm e}(E)]$ | hot electron [47]
$\sigma(E)$ | $en(E)\mu$ |  field-dependent energy gap [27, 28]
$en\mu(E)$ | field-dependent mobility

Let us consider the origin of the observed third-order nonlinearity. Although
there are various causes of the nonlinear conduction, as categorized in Table
I, one may first sort it whether the conductivity $\sigma$ varies when the
temperature changes. Note that the applied electric field to the sample is
lower than the threshold field [22, 23], above which charge ordering shows a
collective sliding mode as is similar to that of CDW [44]. It should also be
emphasized that there are few studies on the third-harmonic conductivity [46,
45] and the interpretation remains unclear. In the simplest case, the phonon
(lattice) temperature $T_{\rm p}$ increases to $T_{\rm p}+\Delta T_{\rm p}$ by
the applied power of $G_{1}V^{2}$, and then the conductivity seems to be
varied, which is well known as the extrinsic self-heating effect. In this
case, the conductance is expressed as $G(V)\simeq G_{1}+(\partial
G_{1}/\partial T_{\rm p})\Delta T_{\rm p}$ and the temperature increase is
given as $\Delta T_{\rm p}=R_{\rm th}G_{1}V^{2}$, where $R_{\rm th}$ is the
thermal resistance between the sample and the thermal bath, leading to the
third-order conductance of $G_{3}=R_{\rm th}G_{1}(\partial G_{1}/\partial
T_{\rm p})$ [37]. This heating effect is unlikely in the present case, because
the sign of observed $G_{3}$ is positive while $\partial G_{1}/\partial T_{\rm
p}$ changes its sign at around 200 K as seen in the inset of Fig. 2(a). Note
that this nonlinearity due to the heating effect is of importance to detect
higher-order moment of the current distribution in the percolation network
consisting of the metallic and insulating domains in the sample. In this case,
a peculiar frequency dependence of $V_{3}\propto f^{-1/2}$ is expected [37].
On the other hand, the observed $V_{3}$ shows almost no frequency dependence
as shown in Fig. 1(b), again indicating that the heating effect is unlikely in
the present case.

We then discuss the hot-electron model. In this scenario, an electron
temperature $T_{\rm e}$, which is larger than $T_{\rm p}$, is enhanced by
applying electric field to vary the conductivity. This model is suggested to
explain the high-field nonlinearity in this material [25]. In this case, the
electron and phonon systems are connected with the thermal resistance $R_{\rm
ep}$ and the characteristic time scale $\tau_{\rm e}$ is given as $\tau_{\rm
e}=R_{\rm ep}C_{\rm e}$, where $C_{\rm e}$ is the electronic specific heat
[47]. Now the order of $\tau_{\rm e}$ is generally as small as $10^{-9}$ s,
and thus previous hot-electron phenomena have been investigated by using a
pulsed technique with a short pulse width of $10^{-9}$ s [47]. In contrast,
the period of the applied voltage in the present measurement of $f^{-1}\sim
10^{-2}$ s is much longer than $\tau_{\rm e}$. Therefore, the electron and
lattice systems are in equilibrium ($T_{\rm e}\simeq T_{\rm p}$) in the
present time scale, and thus hot-electron scenario is excluded due to the same
reason to the self-heating case as discussed above.

Next we argue a non-thermal origin for nonlinear conduction, which is not
induced through the temperature change in field. If we adapt a simple Drude
formula of $\sigma=en\mu$ for the measured temperature range, where $n$ and
$\mu$ are the carrier density and the mobility, respectively, one may consider
the electric-field dependence of $n$ and/or $\mu$ for the nonlinearity of
$\sigma$. According to Refs. 27, 28, the charge gap $\Delta$ is effectively
reduced by the field $E$ as $\Delta(E)=U_{0}\ln(\lambda/a)-e\lambda E$, where
$\lambda$ is a cutoff length of the potential and $a$ is the minimum length
scale taken as the size of the ET molecule, resulting in increase of the
activation-type conductivity $\sigma(E)=\sigma_{0}\exp[-\Delta(E)/2k_{\rm
B}T]$ due to the field-induced increase of the carrier density $n(E)$.
However, this scenario is based on the 2D logarithmic potential developed well
below $T_{\rm CO}$, in contrast to the present results that the nonlinearity
appears critically above $T_{\rm CO}$.

Figure 3:  (Color online) Temperature dependence of the ratio of the in-phase
components of $G_{3}$ to $G_{1}$. Dashed line represents the charge order
transition temperature $T_{\rm CO}=$136 K.

We then consider possible field-dependent mobility $\mu(E)$. Recently, it has
been suggested that the mobility in the large-permittivity compound is
expressed as $\mu\propto\varepsilon^{1/2}$ ($\varepsilon$ is the permittivity)
[48, 49]. Here, mean free path $l$ in large-permittivity materials may be
proportional to the Thomas-Fermi screening length $r_{\rm TR}$, since it gives
a characteristic length scale for the potential modulation which acts as the
scattering centers [48]. In such a case, since $r_{\rm TR}\propto\sqrt{a_{\rm
B}^{*}}$, where $a_{\rm B}^{*}$ is the effective Bohr radius given as $a_{\rm
B}^{*}=(\varepsilon/\varepsilon_{0})(m/m^{*})a_{\rm B}$ ($\varepsilon_{0}$
being the vacuum permittivity, $m$ the electron mass, $m^{*}$ the effective
mass, $a_{\rm B}$ the Bohr radius), the mobility $\mu$ is given as $\mu\propto
l\propto\varepsilon^{1/2}$. Here, the permittivity is given as
$\varepsilon=\varepsilon_{0}(1+\chi)$ and the electric susceptibility $\chi$
is generally expanded as $\chi\simeq\chi_{1}+\chi_{3}E^{2}$, where $\chi_{1}$
and $\chi_{3}$ are linear and third-order electric susceptibilities,
respectively. Up to the $E^{2}$ term, we then obtain a field-dependent
mobility as

$\displaystyle\mu(E)\propto(1+\chi)^{1/2}\simeq(1+\chi_{1})^{1/2}\left(1+\frac{1}{2}\frac{\chi_{3}}{1+\chi_{1}}E^{2}\right).$
(4)

Therefore, by comparing it with
$\sigma=en\mu\simeq\sigma_{1}[1+(\sigma_{3}/\sigma_{1})E^{2}]$, the
conductance ratio is given as

$\displaystyle\frac{G_{3}}{G_{1}}\propto\frac{\sigma_{3}}{\sigma_{1}}\propto\frac{1}{2}\frac{\chi_{3}}{1+\chi_{1}}.$
(5)

In Fig. 3, we plot temperature dependence of $G_{3}/G_{1}$. Interestingly, it
exhibits a peak structure near $T_{\rm CO}$, indicating an enhancement of the
third-order electric susceptibility $\chi_{3}$ at the charge-order transition.
Note that the linear susceptibility $\chi_{1}$ in the denominator is difficult
to contribute to the enhancement of $G_{3}/G_{1}$, because $\chi_{1}$ is
usually enhanced near the charge-order transition as seen in the one-
dimensional organic salts [50] while it is very difficult to measure
$\chi_{1}$ above $T_{\rm CO}$ in $\alpha$-(ET)2I3 owing to the high
conductivity [51]. We also mention that the observed enhancement of the out-
of-phase component of $G_{3}$ at $T_{\rm CO}$ [Fig. 2(c)] may support this
picture because the imaginary part of the complex conductivity
$\tilde{\sigma}=\sigma+i\omega\chi\varepsilon_{0}$ generally expresses the
electric susceptibility.

In general, third-order electric susceptibility thermodynamically represents
the fluctuation of the quadrupole moments [52]. In the magnetic analogy, the
higher-order magnetic susceptibility is known as the probe for multipole
ordering [53, 54]. Therefore, the present results imply a quadrupole
instability hidden in the charge-ordered salt $\alpha$-(ET)2I3, which may be
natural in a sense that it is also an even-ordered multipole expansion. It
should be noted that a signature of charge order is observed with an
inhomogeneity even in the high-temperature metallic phase [11, 55]. Also note
that such a quadrupole nature is mentioned in $\kappa$-type ET salt [45] and
manganese oxide [46], although these nonlinear experiments were performed with
a standard lock-in technique without bridge circuit in contrast to the present
study. Now the quadrupole nature in $\alpha$-(ET)2I3 may be attributed to two
possible origins: Firstly, the charge-density distribution should be
asymmetric in the ET molecule, leading to the quadrupole moment. This is
supported by the fact that the framework of ET molecule is sensitive to the
amount of charge [56]. Note that the precise determination of the molecular
structure using the synchrotron radiation is recently developed [57, 58].
Secondly, the charge distribution inside the unit cell becomes more asymmetric
by the formation of charge ordering at $T_{\rm CO}$. This is also deduced from
the temperature dependence of the in-plane anisotropy of the resistivity [59,
60]. At present, it is difficult to clarify the most appropriate
interpretation; both origins seem to contribute to the quadrupole instability.

To summarize, we have measured the third-order conductivity of
$\alpha$-(ET)2I3 over a broad temperature range across the charge ordering
transition temperature $T_{\rm CO}=136$ K. We find the enhancement of the
third-order conductance at $T_{\rm CO}$, which may attribute to the possible
quadrupole instability hidden in the charge-ordered organic salt.

###### Acknowledgements.

We thank M. D. Anoop, N. Kikugawa, Y. Maeno, I. Terasaki, K. Ueda, and S.
Yonezawa for discussion. This work was supported by JSPS KAKENHI Grants No.
17H06136.

## References

  * [1] H. Seo, J. Merino, H. Yoshioka, and M. Ogata, J. Phys. Soc. Jpn. 75, 051009 (2006).
  * [2] C. Hotta, Crystals 2, 1155 (2012).
  * [3] R. T. Clay and S. Mazumdar, Phys. Rep. 788, 1 (2019).
  * [4] M. Dressel and S. Tomić, Adv. Phys. 69, 1 (2020).
  * [5] K. Bender, I. Hennig, D. Schweitzer, K. Dietz, H. Endres, and H. J. Keller, Mol. Cryst. Liq. Cryst. 108, 359 (1984).
  * [6] Y. Takano, K. Hiraki, H. Yamamoto, T. Nakamura, and T. Takahashi, J. Phys. Chem. Solids 62, 393 (2001).
  * [7] T. Takahashi, Y. Nogami, and K. Yakushi, J. Phys. Soc. Jpn. 75, 051008 (2006).
  * [8] M. Meneghetti, R. Bozio, and C. Pecile, Journal de Physique 47, 1377 (1986).
  * [9] J. Moldenhauer, C. Horn, K. Pokhodnia, D. Schweitzer, I. Heinen, and H. Keller, Synth. Met. 60, 31 (1993).
  * [10] R. Wojciechowski, K. Yamamoto, K. Yakushi, M. Inokuchi, and A. Kawamoto, Phys. Rev. B 67, 224105 (2003).
  * [11] Y. Yue, K. Yamamoto, M. Uruichi, C. Nakano, K. Yakushi, S. Yamada, T. Hiejima, and A. Kawamoto, Phys. Rev. B 82, 075134 (2010).
  * [12] T. Ivek, B. Korin-Hamzić, O. Milat, S. Tomić, C. Clauss, N. Drichko, D. Schweitzer, and M. Dressel, Phys. Rev. B 83, 165128 (2011).
  * [13] K. Yakushi, Crystals 2, 1291 (2012).
  * [14] T. Kakiuchi, Y. Wakabayashi, H. Sawa, T. Takahashi, and T. Nakamura, J. Phys. Soc. Jpn. 76, 113702 (2007).
  * [15] K. Yamamoto, S. Iwai, S. Boyko, A. Kashiwazaki, F. Hiramatsu, C. Okabe, N. Nishi, and K. Yakushi, J. Phys. Soc. Jpn. 77, 074709 (2008).
  * [16] P. Lunkenheimer, B. Hartmann, M. Lang, J. Müller, D. Schweitzer, S. Krohns, and A. Loidl, Phys. Rev. B 91, 245132 (2015).
  * [17] N. Tajima, A. Ebina-Tajima, M. Tamura, Y. Nishio, and K. Kajita, J. Phys. Soc. Jpn. 71, 1832 (2002).
  * [18] N. Tajima, S. Sugawara, M. Tamura, R. Kato, Y. Nishio, and K. Kajita, Eur. Phys. Lett. 80, 47002 (2007).
  * [19] A. Kobayashi, S. Katayama, Y. Suzumura, and H. Fukuyama, J. Phys. Soc. Jpn. 76, 034711 (2007).
  * [20] N. Tajima, S. Sugawara, R. Kato, Y. Nishio, and K. Kajita, Phys. Rev. Lett. 102, 176403 (2009).
  * [21] M. Dressel, G. Grüner, J. Pouget, A. Breining, and D. Schweitzer, Synth. Met. 70, 929 (1995).
  * [22] K. Tamura, T. Ozawa, Y. Bando, T. Kawamoto, and T. Mori, J. Appl. Phys. 107, 103716 (2010).
  * [23] F. Itose, T. Kawamoto, and T. Mori, J. Appl. Phys. 113, 213702 (2013).
  * [24] T. Ivek, I. Kovačević, M. Pinterić, B. Korin-Hamzić, S. Tomić, T. Knoblauch, D. Schweitzer, and M. Dressel, Phys. Rev. B 86, 245125 (2012).
  * [25] T. Peterseim, T. Ivek, D. Schweitzer, and M. Dressel, Phys. Rev. B 93, 245133 (2016).
  * [26] A. Ito, Y. Nakamura, A. Nakamura, and H. Kishida, Phys. Rev. Lett. 111, 197801 (2013).
  * [27] K. Kodama, M. Kimata, Y. Takahide, N. Kurita, A. Harada, H. Satsukawa, T. Terashima, S. Uji, K. Yamamoto, and K. Yakush, J. Phys. Soc. Jpn. 81, 044703 (2012).
  * [28] S. Uji, K. Kodama, K. Sugii, Y. Takahide, T. Terashima, N. Kurita, S. Tsuchiya, M. Kohno, M. Kimata, K. Yamamoto, and K. Yakushi, Phys. Rev. Lett. 110, 196602 (2013).
  * [29] T. Iimori, T. Naito, and N. Ohta, J. Am. Chem. Soc. 129, 3486 (2007).
  * [30] T. Iimori and N. Ohta, J. Phys. Chem. C 118, 7251 (2014).
  * [31] T. Ishikawa, Y. Sagae, Y. Naitoh, Y. Kawakami, H. Itoh, K. Yamamoto, K. Yakushi, H. Kishida, T. Sasaki, S. Ishihara, Y. Tanaka, K. Yonemitsu, and S. Iwai, Nat. Commun. 5, 5528 (2014).
  * [32] T. Osada and A. Kiswandhi, J. Phys. Soc. Jpn. 90, 053704 (2021).
  * [33] A. Kiswandhi and T. Osada, J. Phys.: Condens. Matter 34, 105602 (2022).
  * [34] R. Okazaki, Y. Nishina, Y. Yasui, F. Nakamura, T. Suzuki, and I. Terasaki, J. Phys. Soc. Jpn. 82, 103702 (2013).
  * [35] N. Tajima, M. Tamura, Y. Nishio, K. Kajita, and Y. Iye, J. Phys. Soc. Jpn. 69, 543 (2000).
  * [36] T. Guldbrandsen, N. I. Meyer, and J. Schjaer-Jakobsen, Rev. Sci. Instrum. 36, 743 (1965).
  * [37] M. A. Dubson, Y. C. Hui, M. B. Weissman, and J. C. Garland, Phys. Rev. B 39, 6807 (1989).
  * [38] C. Thibierge, D. L’Hôte, F. Ladieu, and R. Tourbot, Rev. Sci. Instrum. 79, 103905 (2008).
  * [39] Y. Tokura and N. Nagaosa, Nat. Commun. 9, 3740 (2018).
  * [40] D. G. Cahill and R. O. Pohl, Phys. Rev. B 35, 4067 (1987).
  * [41] I. K. Moon, Y. H. Jeong, and S. I. Kwun, Rev. Sci. Instrum. 67, 29 (1996).
  * [42] P. F. Sullivan and G. Seidel, Phys. Rev. 173, 679 (1968).
  * [43] D. H. Jung, T. W. Kwon, D. J. Bae, I. K. Moon, and Y. H. Jeong, Meas. Sci. Technol. 3, 475 (1992).
  * [44] G. Grüner, Rev. Mod. Phys. 60, 1129 (1988); Rev. Mod. Phys. 66, 1 (1994).
  * [45] R. Rommel, B. Hartmann, J. Brandenburg, J. A. Schlueter, and J. Müller, Phys. Status Solidi B 250, 568 (2013).
  * [46] V. Moshnyaga, K. Gehrke, O. I. Lebedev, L. Sudheendra, A. Belenchuk, S. Raabe, O. Shapoval, J. Verbeeck, G. Van Tendeloo, and K. Samwer, Phys. Rev. B 79, 134413 (2009).
  * [47] E. J. Ryder, Phys. Rev. 90, 766 (1953).
  * [48] K. Behnia, J. Phys.: Condens. Matter 27, 375501 (2015).
  * [49] J. Wang, L. Yang, C. W. Rischau, Z. Xu, Z. Ren, T. Lorenz, J. Hemberger, X. Lin, and K. Behnia, npj Quant. Mat. 4, 61 (2019).
  * [50] F. Nad and P. Monceau, J. Phys. Soc. Jpn. 75, 051005 (2006).
  * [51] P. Lunkenheimer and A. Loidl, J. Phys.: Condens. Mat. 27, 373001 (2015).
  * [52] R. Martoňák and E. Tosatti, Phys. Rev. B 54, 15714 (1996).
  * [53] P. Morin and D. Schmitt, Phys. Rev. B 23, 5936 (1981).
  * [54] A. P. Ramirez, P. Coleman, P. Chandra, E. Brück, A. A. Menovsky, Z. Fisk, and E. Bucher, Phys. Rev. Lett. 68, 2680 (1992).
  * [55] K. Katono, T. Taniguchi, K. Ichimura, Y. Kawashima, S. Tanda, and K. Yamamoto, Phys. Rev. B 91, 125110 (2015).
  * [56] T. C. Umland, S. Allie, T. Kuhlmann, and P. Coppens, J. Chem. Phys. 92, 6456 (1988).
  * [57] S. Kitou, T. Fujii, T. Kawamoto, N. Katayama, S. Maki, E. Nishibori, K. Sugimoto, M. Takata, T. Nakamura, and H. Sawa, Phys. Rev. Lett. 119, 065701 (2017).
  * [58] S. Kitou, Y. Hosogi, R. Kitaura, T. Naito, T. Nakamura, and H. Sawa, Crystals 10, 998 (2020).
  * [59] T. Ivek, B. Korin-Hamzić, O. Milat, and S. Tomić, C. Clauss, N. Drichko, D. Schweitzer, and M. Dressel, Phys. Rev. Lett. 104, 206406 (2010).
  * [60] T. Ivek, M. Čulo, M. Kuveždić, E. Tutiš, M. Basletić, B. Mihaljević, E. Tafra, S. Tomić, A. Löhle, M. Dressel, D. Schweitzer, and B. Korin-Hamzić, Phys. Rev. B 96, 075141 (2017).