# Generalized Quantifiers as a Source of Error
in Multilingual NLU Benchmarks

Ruixiang Cui, Daniel Hershcovich, Anders Søgaard
University of Copenhagen
{rc, dh<EMAIL_ADDRESS>
###### Abstract

Logical approaches to representing language have developed and evaluated
computational models of quantifier words since the 19th century, but today’s
NLU models still struggle to capture their semantics. We rely on Generalized
Quantifier Theory for language-independent representations of the semantics of
quantifier words, to quantify their contribution to the errors of NLU models.
We find that quantifiers are pervasive in NLU benchmarks, and their occurrence
at test time is associated with performance drops. Multilingual models also
exhibit unsatisfying quantifier reasoning abilities, but not necessarily worse
for non-English languages. To facilitate directly-targeted probing, we present
an adversarial generalized quantifier NLI task (GQNLI) and show that pre-
trained language models have a clear lack of robustness in generalized
quantifier reasoning.

##  1 Introduction

Quantifier words—such as each or most or more than three—have been extensively
studied, both in logic and in linguistics (Westerståhl, 1989; Peters and
Westerståhl, 2006), going all the way back to Frege (1879). In this paper, we
examine the extent to which they present a challenge to modern NLU systems.
Our analysis is motivated by three observations:

Quantifier words are abstract Unlike nouns, verbs and adjectives, quantifier
words do not have referents out in the world. Rather, quantifier words specify
relationships between sets of entities, events and properties. To provide
intuitions about the semantics of quantifier words, and to be able to refer to
quantifiers in a language-independent way, we rely on the notion of
generalized quantifiers Mostowski (1957), as described in §2.

Quantifier words vary across languages Quantifier word inventories differ
across languages. Often what is considered rough translation equivalents also
differ in syntax, fine-grained semantics or pragmatics. Stateva et al. (2019)
show, e.g., that perceptions of the numerical bounds of existential
quantifiers differ across speakers of English, French, Slovenian, and German.
Other papers showing discrepancies between quantifier systems include
comparisons of Salish to English Matthewson (2001), Adyghe to English
Nikolaeva (2012), or of Dutch, Hebrew and Bengali Gil (1982). The cross-
linguistic differences in how generalized quantifiers are expressed motivates
a cross-lingual error analysis, since quantifiers may contribute more to error
when processing some languages rather than others.

QA_English |  Context: A piece of paper was later found on which he had written his last statements in two languages, Latin and German. Only one statement was in Latin and the rest in German.
---|---
Question: In what language were most statements written? Answer: German
Predicted Answer: Latin and German
NLI_Spanish |  Premise: Más de tres personas resultaron heridas en un accidente de dos vehículos el lunes por la noche. (translation: More than three people were injured in a two-vehicle crash Monday evening.)
Hypothesis: Había 4 personas involucradas. (translation: There were 4 people
involved. Label: Neutral Predicted Label: Entailment
Table 1: Examples of quantifiers (marked in bold texts) in NLP tasks, with RoBERTa’s prediction for QA and XLM-R’s prediction for NLI after fine-tuning. Generalized Quantifiers |  Logical Denotation |  Example
---|---|---
some(A)(B) = 1 |  $A\cap B\neq\varnothing$ |  This process is known to increase security in several ways.
all(A)(B) = 1 |  $A\subseteq B$ |  Everyone agreed the food was terrible.
more than k the(A)(B) = 1 |  $|A\cap B|>k$ |  They do let them go more than twice a week.
less than k the(A)(B) = 1 |  $|A\cap B|<k$ |  San Augustin Acolman has less than 1,000 residents.
k (A)(B) = 1 |  $|A\cap B|=k$ |  Please donate 100 million to the School of Nursing.
between p and k the(A)(B) = 1 |  $p<|A\cap B|<k$ |  The USA added ten states to its nation between 1800 and 1850.
the p/k (A)(B) = 1 |  $|A\cap B|=p\cdot(|A|/k)$ |  Captain Blood has 20/20 vision.
the k% (A)(B) = 1 |  $|A\cap B|=k\cdot(|A|/100)$ |  The lending fund is always guaranteed 9% interest.
most (A)(B) = 1 |  $|A\cap B|>|A\backslash B|$ |  Most ZIP Codes cover roughly ten thousand addresses.
few (A)(B) = 1 |  $|A\cap B|<|A\backslash B|$ |  Only a few teenagers were still listening to Rock ’n’ Roll.
each other (A)(B) = 1 |  $\forall a\in(A\cap B)\exists b\in(A\cap B)(a\neq b)$ |  All of these trails are located within the a one hour drive of each other.
Table 2: The categorization set of quantifiers for task analysis. The first
six are Aristotelian/counting quantifiers and the following four are
proportional quantifiers. The last one is a Ramsey quantifier (Schmerl and
Simpson, 1982). For each quantifier, its logical denotation is listed in the
second column. The third conlumn contains English examples with quantifiers
taken from XNLI.

Quantifier words are important Quantifier words are extremely important for
tasks that require inference, including natural language inference, question
answering, fact-checking, etc. Datasets have, for example, been developed for
numerical reasoning in English Dua et al. (2019). Several researchers have
identified quantifier words as important sources of errors for natural
language processing systems Joshi et al. (2020); see Table 1 for examples of
such errors. Unfortunately, most efforts have concentrated on subsets of
quantifier words and on English.

Contributions We analyze how quantifiers are represented in NLU benchmarks,
and how their occurrence at test time contributes to errors by neural language
models (LMs). We derive a linguistically motivated 11-way categorization set
for generalized quantifiers and look into their distribution in three steps:
(a) monolingual NLI; (b) cross-lingual NLI; (c) cross-lingual question
answering. We also propose GQNLI111https://github.com/ruixiangcui/GQNLI, an
adversarial generalized quantifier NLI challenge dataset. Our work shows that
(i) generalized quantifiers are pervasive and cause overall performance drops
in NLU benchmarks; (ii) the contribution of quantifier words to system error
varies across languages; and (iii) generalized quantifiers are particularly
difficult for LMs in interaction with negation and subsumption.

##  2 Background

Generalized quantifiers (GQs) are developed upon first-order predicate logic,
denoting relations between sets (Mostowski, 1957). Given a universe E, a
quantifier Q would be treated as a mapping $Q_{E}$ from the Cartesian product
of powersets $\mathcal{P}(E)\times\mathcal{P}(E)$ to the set {false,true} or,
as a binary relation on subsets of E (Dvořák and Holčapek, 2015). GQs are
generalizations of the $\forall$,$\exists$ quantifiers from first-order
predicate logic (Mostowski, 1957; Lindström, 1966; Montague, 1973; Bach et
al., 1995; Keenan and Paperno, 2012). A generalized quantifier is, abstractly,
a relation between sets. Generalized quantifier theory, while developed by
logicians, is used by formal linguists to analyze the meaning of quantifier
words in combination with referential expressions (Barwise and Cooper, 1981;
Higginbotham and May, 1981).

Most human languages contain ways of expressing generalized quantifiers, and
their semantics exhibit striking similarities across languages Matthewson
(2004); Fintel and Matthewson (2008); Steinert-Threlkeld (2019). At the same
time, generalized quantifiers can be instantiated very differently across
languages due to pragmatic considerations (Grice, 1989) or cognitive economy
and cost-benefit optimisation in the exchange of information (Levinson et al.,
2000; Steinert-Threlkeld, 2021; Uegaki, 2022). Quantifier words also exhibit
syntactic differences, e.g., with some languages having specialized words to
express quantity, while others rely on metaphorical usage of common nouns
(Katsos et al., 2012). In English, most is a determiner, but Spanish and
French express the same concept through common nouns, la mayoría and la
majorité. The relative stability of the core semantics of quantifiers makes a
cross-linguistic comparison possible, but the syntactic and pragmatic
variation associated with the expression of generalized quantifiers poses a
challenge for multilingual NLU. We consult quantifier taxonomy studies (Keenan
and Westerståhl, 1997; Peters and Westerståhl, 2006; Szymanik and Thorne,
2015; Szymanik, 2016) and derive a categorization set for quantifier analysis
in NLU benchmarks. In Table 2, we list the 11-way quantifier categorization
set and their logical denotation based on set theory.

While other foci of formal linguistics have attracted the attention of NLP
researchers—including coreference (Ogrodniczuk et al., 2019, 2020), negation
(Hossain et al., 2020; Hartmann et al., 2021), and consistency (Li et al.,
2019; Ribeiro et al., 2019; Asai and Hajishirzi, 2020; Geva et al.,
2022)—there has been little work on generalized quantifiers as a source of
error in NLU, let alone in multilingual NLU. It remains an open problem
whether LMs represent the semantics of quantifiers words adequately, or if
they provide a basis for resolving scopal ambiguities.222Note that generalized
quantifiers are not always explicit in discourse. The sentence inadequate
sleep causes obesity should be interpreted as Most of those who do not sleep
adequately, gain weight (Zadeh, 1983). Such implicit quantifiers related to
pragmatic variation are important for language understanding, but will be
ignored in this work.

##  3 NLU Benchmarks

We conduct an error analysis focusing on the role of generalized quantifiers
in two NLU tasks, Natural Language Inference (NLI) and Question Answering
(QA), which generally require understanding of quantifiers. For each type of
task, both monolingual and cross-lingual evaluation are conducted. We focus on
generalized quantifiers in the hypotheses in NLI examples—and on generalized
quantifiers in the question fields in question answering.

Quantifier | English | Cross-lingual
---|---|---
MNLI_m |  MNLI_mm |  SNLI |  ANLI_R1 |  ANLI_R2 |  ANLI_R3 |  XNLI
some | 171 | 132 | 191 | 5 | 1 | 17 | 115
all | 255 | 239 | 65 | 15 | 8 | 29 | 166
$>k$ | 14 | 23 | 8 | 10 | 16 | 14 | 16
$<k$ | 3 | 3 | 0 | 6 | 7 | 5 | 1
$k$ | 266 | 269 | 988 | 55 | 62 | 48 | 159
between | 2 | 3 | 0 | 3 | 2 | 0 | 1
$p/k$ | 1 | 5 | 1 | 1 | 1 | 0 | 2
$k\%$ | 10 | 7 | 0 | 0 | 0 | 1 | 5
most | 35 | 39 | 1 | 0 | 2 | 1 | 9
few | 14 | 15 | 11 | 0 | 0 | 6 | 11
each other | 4 | 3 | 35 | 0 | 0 | 2 | 5
Total | 775 | 738 | 1300 | 95 | 99 | 124 | 499
Frequency | 7.9% | 7.5% | 13.2% | 9.5% | 9.9% | 12.4% | 10.0%

Table 3: Quantifier distribution in four NLI tasks, among which three are
monolingual English and one is cross-lingual. The table show statistics of the
test set, if not available, dev set, of the target task. All but the last rows
show the occurrence time of the type of quantifier in the first column. The
last row represents the distribution rate of any quantifier in the dataset.

To this end, we identify quantifiers by the lemma and the universal dependency
relation (Nivre et al., 2020) of a quantifier after preprocessing the
sentences using Stanza (Qi et al., 2020). Take the sentence “The Yiddish
culture has survived for more than a thousand years.”, we annotate it as
“The/det Yiddish/amod culture/nsubj have/aux survive/root for/case more/advmod
than/fixed a/det thousand/nummod year/obl ./punct”. By matching the regex
pattern of the quantifier “more than k”, in this case “((more|great)\/advmod
than\/(fixed|case)|at\/case least\/nmod) .+\/nummod .+\/(nsubj|obj|obl)”, we
approximate the surface form of the type “more than k”.Through matching
quantifier patterns, we are able to find entries in which quantifiers are
instantiated. See Appendix A for the list of regex patterns we write to
identify GQs. In Table 3 and Table 6, we present the statistics of the
quantifier distributions in NLI and QA tasks, respectively. As can be seen,
quantifiers are indeed widespread in NLU tasks, accounting for roughly 10% in
NLI tasks and 5% in QA tasks. We will further discuss the statistics and
experiments in the following section.

##  4 Quantifiers in English NLI Benchmarks

Figure 1: Relative distribution of quantifiers in NLI and QA tasks ranked by
semantic complexity. The bars show the relative frequency of such quantifier
and the lines indicate the cumulative frequency for a task.

Quantifier | BERT | RoBERTa
---|---|---
M_m |  M_mm |  SNLI |  A_R1 |  A_R2 |  A_R3 |  weig. |  M_m |  M_mm |  SNLI |  A_R1 |  A_R2 |  A_R3 |  weig.
some | 82.5 | 84.1 | 86.9 | 100 | 0 | 47.1 | 83.4 | 83 | 84.8 | 86.9 | 100 | 100 | 41.1 | 83.7
all | 85.9 | 88.3 | 89.2 | 46.7 | 37.5 | 34.5 | 83.2 | 85.9 | 92.1 | 90.8 | 66.7 | 37.5 | 34.5 | 85.3
$>k$ | 85.7 | 100 | 87.5 | 70 | 43.8 | 42.9 | 73 | 85.7 | 91.3 | 87.5 | 80 | 37.5 | 28.5 | 68.2
$<k$ | 100 | 100 |  | 33.3 | 57.1 | 80 | 66.7 | 100 | 100 |  | 83.3 | 85.7 | 100 | 91.7
$k$ | 87.2 | 81.8 | 92.4 | 43.6 | 43.5 | 33.3 | 84.8 | 88.3 | 88.8 | 92.9 | 56.3 | 61.3 | 43.8 | 87.8
between | 100 | 100 |  | 66.7 | 50 |  | 80 | 100 | 66.7 |  | 66.7 | 50 |  | 70
$p/k$ | 100 | 60 | 100 | 100 | 100 |  | 77.8 | 100 | 80 | 100 | 100 | 0 |  | 77.8
$k\%$ | 90 | 100 |  |  |  | 100 | 94.4 | 70 | 85.7 |  |  |  | 0 | 72.2
most | 74.3 | 79.5 | 0 |  | 50 | 0 | 74.4 | 77 | 87.2 | 100 |  | 59 | 0 | 80.9
few | 78.6 | 73.3 | 90.9 |  |  | 33.3 | 73.9 | 85.7 | 80 | 90.9 |  |  | 33.3 | 78.3
each other | 75 | 100 | 85.7 |  |  | 50 | 84.1 | 50 | 100 | 88.6 |  |  | 50 | 84.1
all GQs | 85 | 84.8 | 91.2 | 50.5 | 44.4 | 39 | 83.3 | 85.4 | 88.8 | 91.7 | 65.3 | 56.5 | 40.3 | 85.5
full | 86.5 | 86.1 | 91.3 | 58.6 | 48 | 43.2 | 84.4 | 89.5 | 89.4 | 92.3 | 71.7 | 49.6 | 49 | 87.3

Table 4: BERT and RoBERTa performance on NLI tasks. The weig. column
represents the percentage of all true predictions in six subtasks over total
instances. The penultimate row stands for the overall performance when
quantifiers exist in a dataset. The last row reports the overall performance
in a dataset. Number marked in bold signifies a lower score than the overall
performance.

NLI is commonly framed as a three-way classification task with labels
entailment, contradiction and neutral Bowman et al. (2015a). While SOTA models
exhibit low error rates on NLI benchmarks, it is unclear when they succeed or
fail in their underlying reasoning. We are interested in whether generalized
quantifers challenge modern NLI models. In our error analysis, we initially
focus on three English NLI datasets, MultiNLI (MNLI; Williams et al., 2018),
SNLI (Bowman et al., 2015a) and ANLI (Nie et al., 2020) as testbeds.

Table 3 presents statistics of quantifier distribution in these datasets,
where we observe that, across, about 10% of all hypotheses contain quantifier
words, indicating the pervasiveness of quantification. We also plot the
frequency of quantifiers in NLI in Figure 1 and find the quantifier word
distribution follows Zipf’s law (Zipf, 1949). Note the top three most common
quantifiers account for more than 90% of all.

### Experiments and Results

In order to investigate whether NLU systems can solve quantifiers in NLI, we
experiment with two pretrained LMs: BERT333wwm_cased_L-24_H-1024_A-16 (Devlin
et al., 2019) and RoBERTa444roberta-large (Liu et al., 2019). We use the
codebase by Nie et al. (2020). The training data combines SNLI, MNLI, FEVER-
NLI (Nie et al., 2019) and ANLI.

In Table 4, we report the test set performance on SNLI and ANLI, and the dev
set performance on MLNI matched and mismatched sections. We can observe that
SOTA models suffer from performance drops across almost all quantification
phenomena in every task. When it comes to performance over all quantifiers,
the improvement from RoBERTa to BERT (2.2%) is less prominent than that over
full datasets (2.9%), suggesting RoBERTa is particularly challenged.

Taking a closer look at error by category, proportional quantifiers seem
harder to solve than Aristotelian/counting quantifiers. Except for k%, all
proportional quantifiers—p/k, most, and few—are about 10% lower than the five
counting quantifiers (except less than k) with BERT; and about 5% lower with
RoBERTa. RoBERTa is not generally superior to BERT; e.g., for k%, BERT
outperforms it by 22%. We show a pairwise analysis of how GQs affect
performance when they appear in both the premises and hypotheses in the
Appendix LABEL:app:pairwise. Generally, our results attest to the difficulty
of resolving GQs in NLI benchmarks.

##  5 Quantifiers in Cross-lingual NLU Benchmarks

Quantifiers are acquired in similar orders across languages Katsos et al.
(2016), although languages express quantifiers in different ways. For example,
there are eight different universal quantifiers with different level of
distributivity in Malagasy (Matthewson, 2008). This poses challenges to
training multilingual LMs and transfer learning. We are interested in whether
quantifiers are universally and evenly challenging for all languages.

Quantifier | mBERT | XLM
---|---|---
en | zh | es | ar | vi | de | weig. | en | zh | es | ar | vi | de | weig.
some | 85.2 | 69.6 | 80 | 63.5 | 67.8 | 74.8 | 73.4 | 85.2 | 70.3 | 79.1 | 71.3 | 73.9 | 69.6 | 69.6
all | 80.1 | 65.7 | 72.8 | 69.3 | 63.9 | 74.1 | 70.9 | 82.5 | 62.7 | 74.1 | 67.5 | 71.7 | 73.5 | 72
$>k$ | 87.5 | 50 | 68.8 | 43.8 | 56.2 | 62.5 | 61.6 | 81.2 | 62.5 | 56.2 | 62.5 | 50 | 75 | 75
$<k$ | 100 | 100 | 100 | 100 | 100 | 100 | 100 | 100 | 100 | 100 | 100 | 100 | 100 | 100
$k$ | 86.2 | 69.1 | 80.5 | 71.7 | 76.7 | 82.4 | 77.7 | 83 | 66.7 | 78.6 | 71.7 | 74.2 | 81.1 | 75.8
between | 100 | 100 | 100 | 100 | 100 | 100 | 100 | 100 | 100 | 100 | 100 | 100 | 100 | 100
$p/k$ | 100 | 50 | 100 | 100 | 100 | 100 | 91.7 | 100 | 0 | 100 | 100 | 50 | 50 | 66.7
$k\%$ | 100 | 100 | 80 | 100 | 100 | 100 | 96.7 | 80 | 80 | 80 | 100 | 100 | 80 | 86.7
most | 55.6 | 55.6 | 66.7 | 66.7 | 33.3 | 66.7 | 57.4 | 55.6 | 33.3 | 66.7 | 55.6 | 44.4 | 77.8 | 55.6
few | 72.7 | 54.5 | 72.7 | 63.6 | 45.5 | 72.7 | 63.6 | 63.6 | 36.4 | 54.5 | 63.6 | 54.5 | 72.7 | 57.5
each other | 60 | 60 | 60 | 60 | 80 | 80 | 66.7 | 80 | 20 | 60 | 20 | 40 | 60 | 46.7
all GQs | 83 | 67.1 | 76.7 | 68.1 | 68.3 | 76.9 | 73.3 | 82.4 | 64.2 | 75.7 | 69.3 | 71.4 | 74.8 | 73
comp. | 82.6 | 88.9 | 74.7 | 65.6 | 70.7 | 71.4 | 72.4 | 83.1 | 64.8 | 76.3 | 66.9 | 71.6 | 71.3 | 72.3

Table 5: Results of mBERT and XLM performance on XNLI tasks decomposed by
quantifier categories.

### Quantifiers in Cross-lingual NLI

We choose XNLI (Conneau et al., 2018), a manual translation of the development
and test set of MNLI into 15 languages, for this multilingual error analysis.
We should clarify that for XNLI, the authors annotate entailment labels for
the English data only and apply them to the other languages. We do not assume
label changes due to translation in this study, but it is worth investigate in
the future. We choose five languages belonging to different language families,
namely Arabic, Chinese, German, Spanish and Vietnamese as targets. The last
column in Table 3 shows the numbers of quantifiers in XNLI. The distribution
rate is 10%. Note that the universal quantifier is the most common quantifier
in XNLI.

We fine-tune mBERT555multi_cased_L-12_H-768_A-12 (Devlin et al., 2019) and
XLM666xlm-mlm-100-1280 (Lample and Conneau, 2019) on the MNLI training set and
evaluate them on XNLI. We report the results in Table 5. We find that
performance varies across languages. For Chinese and Vietnamese, we see
significant drops in performance for examples with GQs, whereas for Arabic and
German, we see improvements. The results per quantifier are more homogeneous,
however.

Similar to our results for English, we can see that the lowest accuracies in
XNLI are with proportional quantifiers, such as most and few. But the gap in
non-English languages is wider for these two categories, especially for
Chinese, the difference reaches 30%. Other hard quantifiers include all, $>k$,
$<k$, and each other.

### Quantifiers in Cross-lingual QA

Quantifier | MLQA | XQuAD
---|---|---
en | zh | es | ar | vi | de | …
some | 66 | 39 | 41 | 44 | 37 | 33 | 12
all | 31 | 14 | 26 | 21 | 19 | 16 | 7
$<k$ | 1 | 0 | 0 | 0 | 1 | 0 | 0
$k$ | 322 | 168 | 166 | 195 | 204 | 149 | 32
between | 4 | 2 | 2 | 2 | 3 | 0 | 3
$p/k$ | 1 | 1 | 1 | 0 | 0 | 0 | 0
$k\%$ | 1 | 1 | 0 | 1 | 0 | 0 | 0
most | 27 | 19 | 11 | 30 | 17 | 9 | 5
Total | 453 | 244 | 247 | 293 | 281 | 207 | 59
Frequency | 3.9% | 4.7% | 4.7% | 5.4% | 5.1% | 4.5% | 5.0%

Table 6: Quantifier distribution in two multilingual QA tasks, MLQA and XQuAD.
We choose six common languages apprearing in both tasks to facilitate
comparisons. XQuAD is strictly parellel while MLQA is not, hence only the
latter has statistics by languges. Categories that no entry exists are
omitted.

Quantifier | XQuAD | MLQA
---|---|---
en | zh | es | ar | vi | de | weighted | en | zh | es | ar | vi | de | weighted
EM | F1 | EM | F1 | EM | F1 | EM | F1 | EM | F1 | EM | F1 | EM | F1 | EM | F1 | EM | F1 | EM | F1 | EM | F1 | EM | F1 | EM | F1 | EM | F1
some | 75 | 84.2 | 50 | 55.5 | 58.3 | 76.1 | 50 | 50 | 16.6 | 42.4 | 33.3 | 43.8 | 47.2 | 58.7 | 59 | 80 | 28.2 | 52.1 | 34.1 | 59.2 | 36.3 | 54.9 | 5.4 | 24 | 33.3 | 58.4 | 32.7 | 54.8
all | 28.5 | 62.2 | 14.2 | 35.2 | 28.5 | 82 | 42.8 | 52.3 | 14.2 | 29.4 | 28.5 | 56 | 26.1 | 52.9 | 67.7 | 79.8 | 14.2 | 46.4 | 38.4 | 62.8 | 33.3 | 57.9 | 10.5 | 30.1 | 31.2 | 51.6 | 32.6 | 54.8
$<k$ |  |  |  |  |  |  |  |  |  |  |  |  |  |  | 0 | 0 |  |  |  |  |  |  | 0 | 13.3 |  |  | 0 | 6.7
$k$ | 78.1 | 90.1 | 68.7 | 80.4 | 56.2 | 72.1 | 40.6 | 64.3 | 12.5 | 35.7 | 56.2 | 77.1 | 52.1 | 70 | 74.9 | 79.4 | 47 | 63.4 | 41.5 | 65.9 | 27.6 | 50.3 | 6.3 | 23.7 | 38.2 | 53 | 39.3 | 56
between | 100 | 100 | 33.3 | 72.2 | 66.6 | 93.3 | 100 | 100 | 0 | 19 | 0 | 56.5 | 50 | 73.5 | 50 | 88.5 | 50 | 83.3 | 0 | 26.6 | 0 | 68.7 | 0 | 26.6 |  |  | 20 | 58.7
$p/k$ |  |  |  |  |  |  |  |  |  |  |  |  |  |  | 100 | 100 | 0 | 0 | 0 | 0 |  |  |  |  |  |  | 33.3 | 33.3
$k\%$ |  |  |  |  |  |  |  |  |  |  |  |  |  |  | 100 | 100 | 0 | 26.6 |  |  | 0 | 23.7 |  |  |  |  | 33.3 | 50.1
most | 40 | 53.3 | 40 | 40 | 0 | 10 | 0 | 26.6 | 0 | 0 | 20 | 49.3 | 16.7 | 29.9 | 55.5 | 76 | 47.3 | 62.1 | 45.4 | 61.7 | 30 | 46.8 | 5.8 | 15.7 | 33.3 | 40.7 | 36.2 | 50.3
all GQs | 70 | 83.2 | 55 | 66.7 | 50 | 70.3 | 41.6 | 58.2 | 11.6 | 32.5 | 43.3 | 65 | 45.3 | 62.7 | 63.5 | 79.2 | 41.8 | 60.3 | 39.6 | 63.7 | 29.3 | 51.3 | 6.4 | 23.6 | 36.1 | 53.2 | 36.1 | 55.2
comp. | 71.8 | 83.7 | 48 | 59.1 | 56 | 74.5 | 40.8 | 57.9 | 13.9 | 32.4 | 50.7 | 67.2 | 46.9 | 62.5 | 67.2 | 80.6 | 37.5 | 57.9 | 47.3 | 66 | 30 | 48.4 | 11.2 | 28 | 40.8 | 56 | 39 | 56.2

Table 7: Results of mBERT performance on XQA tasks decomposed by quantifier
categories.

Cross-lingual question answering (XQA) is another important NLU task that
evaluates the cross-lingual transferability of LMs. We evaluate the effect of
quantifiers on system errors across two XQA datasets, namely XQuAD (Artetxe et
al., 2020) and MLQA (Lewis et al., 2020). As demonstrated in Figure 1,
quantifier word distributions in XQA tasks also follow Zipf’s law, as in NLI
tasks, but k is more frequent (perhaps because of a traditional emphasis on
numerical reasoning), and we see less variance across languages. This is
probably because question answering is targeting quantification less directly.
To evaluate cross-lingual QA performance on GQs, we fine-tune mBERT and
XLM-R777xlm-roberta-large (Conneau et al., 2020) using Hu et al. (2020)’s
architecture. We present results for mBERT in Table 7; for XLM-R results,
please refer to Appendix LABEL:app:xqa_result.

Just as with XNLI, LMs suffer from performance drops across all languages for
almost all GQ phenomena with significant, cross-lingual variation. The most
distinguished is that Exact Match (EM) suffers from a greater deterioration
than F1 scores for all languages. For example, the weighted EM difference for
mBERT on MLQA is 2.9% while the weighted F1 is 1%. As one example in Table 1,
we observe that the plausible answers selected by models, while being
incorrect, result in a sharper decrease of EMs comparing to F1s. Questions
containing GQs also tend to have less verbal answers comparing to those
without GQs, and therefore require higher precision.

Regarding cross-lingual comparisons, Chinese and Arabic are the two languages
that do not have lower performance over GQs compared to the performance over
the complete dataset. Despite the overall trends, subtle differences from XNLI
performance still exist. For example, XLM-R is worse than mBERT on quantifier
reasoning on XQuAD Chinese, especially at proportional quantifiers, but this
is not the case on MLQA Chinese.

##  6 GQNLI

Quantifier | some | all | $>k$ | $<k$ | k | between | $p/k$ | $k\%$ | most | few | each other | Overall
---|---|---|---|---|---|---|---|---|---|---|---|---
# Occurrence | 27 | 51 | 51 | 33 | 170 | 21 | 24 | 45 | 18 | 9 | 36 | 485
Model | Training Data | % Performance
BERT | S,M,F,ANLI | 40.7 | 41.2 | 33.3 | 30.3 | 30.6 | 14.3 | 37.5 | 22.2 | 61.1 | 22.2 | 41.7 | 30
ELECTRA | S,M,F,ANLI | 37.0 | 17.6 | 54.9 | 27.3 | 38.2 | 14.3 | 62.5 | 31.1 | 61.1 | 0.0 | 16.7 | 38.0
SBERT | S,M,F,ANLI | 66.7 | 43.1 | 47.1 | 24.2 | 32.4 | 14.3 | 25.0 | 31.1 | 77.8 | 66.7 | 36.1 | 39.3
RoBERTa | MNLI | 55.6 | 25.5 | 17.6 | 27.3 | 24.7 | 23.8 | 45.8 | 17.8 | 33.3 | 33.3 | 11.1 | 28.2
S,M,F,ANLI | 63.0 | 41.2 | 41.2 | 27.3 | 34.1 | 28.6 | 75.0 | 33.3 | 50.0 | 33.3 | 38.9 | 39.3
ALBERT | S,M,F,ANLI | 70.4 | 45.1 | 35.3 | 33.3 | 36.5 | 19.0 | 37.5 | 37.8 | 50.0 | 11.1 | 36.1 | 41.7
BART | MNLI | 40.7 | 21.6 | 60.8 | 36.4 | 50.6 | 66.7 | 37.5 | 46.7 | 27.8 | 33.3 | 22.2 | 41.3
S,M,F,ANLI | 59.3 | 51.0 | 35.3 | 30.3 | 35.3 | 19.0 | 66.7 | 20.0 | 50.0 | 66.7 | 47.2 | 42.7
DeBERTa-v3 | MNLI | 48.1 | 37.3 | 33.3 | 33.3 | 35.9 | 33.3 | 41.7 | 33.3 | 33.3 | 33.3 | 41.7 | 34.7
M,F,ANLI | 81.5 | 54.9 | 49.0 | 33.3 | 44.7 | 28.6 | 50.0 | 48.9 | 66.7 | 55.6 | 44.4 | 48.0
M,F,Ling,DocNLI | 77.8 | 70.6 | 49.0 | 54.5 | 44.7 | 4.8 | 33.3 | 42.2 | 50.0 | 66.7 | 58.3 | 45.0

Table 8: GQNLI statstics and seven types of models’ performance with different
combinations of training data. The second row shows the occurrence time of the
type of GQ in GQNLI. The following rows show models’ performance on the
dataset. We tested most competitive models fine-tuned for NLI available on
Hugging Face. All but ALBERT (xxlarge) and DeBERTa-v3 (base) are size large.
S, M, F, Ling, A, DocNLI refer to SNLI, MNLI, Fever-NLI, LingNLI (Parrish et
al., 2021), ANLI and DocNLI (Yin et al., 2021), respectively. Numbers in bold
represent the highest accuracy in one category. Due to space limitation we
provide the link to each model in the Appendix LABEL:app:gqnli_hugging.

We have seen how quantifiers present challenges to NLI and QA models. Using an
approach similar to ANLI (Nie et al., 2020) and DynaBench (Kiela et al.,
2021), we use model difficulty (RoBERTa’s) as a heuristic to select hard
examples for a challenge dataset that can hopefully be used to evaluate any
future progress on this. We propose GQNLI, a generalized quantifier NLI
challenge dataset, consisting of 30 premises and 300 hypotheses. The average
sentence lengths of hypothesis and premises are 15.97 and 7.35, respectively.
Both numbers are comparable to those of MNLI, but lower than ANLI’s (Williams
et al., 2020). It should be noted that GQNLI is designed for evaluating future
models; obviously not for benchmarking RoBERTa.

### Dataset Creation

Firstly, we manually create 100 premise-hypothesis pairs, in which various
types of GQs appear. For each premise and hypothesis, the number of GQs varies
from one to three. To choose the premises, we randomly sampled 100 premises
with GQs from SNLI and ANLI test sets, respectively, and selected 10 premises
in total, that we consider are semantically adequate for adding GQs and making
simple hypotheses.

To construct the hypotheses, we rely on RoBERTa fine-tuned on MNLI and
manually select examples about which the model is unsure or incorrect. To
focus on GQs, we keep the challenge examples otherwise simple Ribeiro et al.
(2020), and avoid lexical variations in the hypotheses. Hard examples were
found to be characterized by (i) mixing generalized quantifiers with other
logical operators, such as subsumption or negation, and (ii) combining
multiple different generalized quantifiers. We discuss these observations in
Section 7.

Two of the authors annotated the examples. The inter-annotator agreement
(Fleiss’ kappa) was 0.895, substantially higher than ANLI’s (0.672–0.740). It
is worth noting that the level of semantic or pragmatic interpretation
difference of GQs is reflected in the measurement.

We augmented the examples by substituting non-quantifier words (e.g.,
replacing “dogs” with “cats”) while keeping the labels, to exclude the effect
of specific lexical items. The resulting labels are uniformly distributed.
Table 8 presents GQNLI statistics. Since the dataset is curated to probe the
ability to reason with quantifiers, the distribution of generalized
quantifiers does not follow Zipf’s law; see §4. A list of GQNLI examples per
category is shown in Appendix LABEL:app:gqnli_example.

### Experiments and Results

We evaluate seven types of models on GQNLI, fine-tuned with different
combinations of NLI datasets. As data creation only relied on RoBERTa and
MNLI, nothing prevents that models with different architectures and training
data will perform well. They do not, however. The results are shown in Table
8.

We see that all models have great difficulty with GQNLI. With more training
data, models improve, but the best performance is 48%, less than 15 points
above chance level. In general, the counting quantifiers, especially the
existential and universal quantifiers, are easier than proportional
quantifiers. Particularly, most models struggle with less than k and between.
This is in some contrast with the NLU tasks studied above, where these
quantifiers were among the easiest.

We also observe unstable GQ reasoning ability in simple word substitution
cases. For instance, it happens for DeBERTa fine-tuned with M, F, Ling, DocNLI
that it predicted correctly the contradiction relation between “There are six
children standing on top of a yellow mountain. Two thirds wear red tops and
one third wear green.” and “Between 80% and 90% children do not wear red
tops.”, but incorrectly when “red” is substituted with “beige” and “green”
with “cyan”. We are yet to study what kind of cues lead to the instability.
Our experiments suggest a lack of testing proportionality reasoning and
robustness in existing benchmarks.

##  7 Discussion

### Negation

The interaction between negation words and quantifiers increases semantic
complexity (Partee, 1970; Horn, 2010). We investigate whether this holds for
NLI tasks, using negation cue detection to find all cases where a negation
word and a quantifier appear in the hypotheses.

We break down the performances on negation of the seven models in Appendix
LABEL:app:negation. As indicated, LMs overall have polarized results for
negation cases comparing to the entire dataset. We can see a majority of the
models even predicted opposite labels for some GQ categories, with 0%
accuracy. BART is no longer the second best model, replaced by RoBERTa. The
improvement by training with more data is overall consistent for reasoning
over GQs with negation.

For a cross-lingual investigation of the interaction of GQs and negation, we
find that in XNLI, the number of cases combining both phenomena is
insufficient: we identified four such cases, involving only the quantifiers
“all” and “more than.” For English, mBERT predicted two cases successfully.
For Chinese, German, Vietnamese and Arabic, one is correct. For Spanish, all
are wrongly predicted.

It is evident that NLU models suffer from reasoning difficulties in certain
cases when negation interacts with GQs, especially in cross-lingual
evaluation. In future work, we are interested in expanding GQNLI to more
instances and more languages to facilitate qualitative investigations.

### Subsumption

In generalized term subsumption languages (TSLs; Yen, 1991; Ali and Shapiro,
1993), a term $a$ subsumes another term $b$ if and only if the extension of
$a$ is a superset of the extension of $b$ . Rather than surface number
comparison, subsumption reasoning requires knowledge of the relations between
supersets and subsets. For example, to decide whether “There are six dogs.
Three brown dogs, a black dog and a white dog run along the green grass”
entails “One dog sits”, LMs should be aware that “six dogs” is a superset of
the extension of the “brown dogs”, “black dog” and “white dog”. Another
example in GQNLI is to infer whether “There are twelve singers on a stage,
less than half from Argentina and one from Cape Verde” entails “Several
singers do not come from Chile”.

We annotate 63 cases out of the first 100 in GQNLI requiring subsumption
reasoning. We show the statistics and results regarding subsumption in
Appendix LABEL:app:Subsumption. It can be seen that more training data leads
to higher accuracies. Especially, DeBERTa fine-tuned with DocNLI, which
unifies the two classes “neutral” and “contradict” into a new class “not
entail”, has a significant improvement on subsumption cases with neutral
label. The training bias give an advantage to the model on the subsumption
subset, half cases of which are labelled neutral. But such bias has a negative
effect on non-subsumption cases; the accuracy drops by 20.2% comparing to the
model without training with DocNLI. It is worth investigating whether DocNLI
is truly helping subsumption reasoning in future work. Subsumption is a key
concept in the study of knowledge representation (Woods, 1991), but is
neglected in current NLP research. The fact that LMs struggle to perform
subsumption reasoning asserts the necessity to explicit tackle the problem.

##  8 Related Work

We examine the sensitivity of NLU models to generalized quantifiers. These
models are designed to induce correlations from large volumes of data, not to
reason symbolically with logical quantifiers. Such models have, nevertheless,
been probed for logical knowledge.

Mul and Zuidema (2019), for example, show neural networks encode fragments of
first-order logic and exhibit zero-shot generalization ability. Evans et al.
(2018) present a neural architecture that improves performance on
propositional logical inference. Bowman et al. (2015b) also suggest neural
networks learn semantic representations for logical inference in natural
languages. However, on the same task, Veldhoen and Zuidema (2017) find neural
networks fail to do so on a more stringent test. Geiger et al. (2019) also
show that neural networks fail to exhibit robust logical inference. Srivastava
et al. (2018) use semantic parsers to encode quantifiers and improve zero-shot
learning in classification tasks. Haruta et al. (2020) present a system that
computes logical inference over GQs and see improvements on two specialized
datasets, FraCaS (Cooper et al., 1994) and MED (Yanaka et al., 2019). None of
these papers explicitly discussed generalized quantifiers, and all were
limited to studying the ability of neural networks to capture the logical
semantics of English.

Many studies have instead focused on LMs’ ability to capture negation
(Gururangan et al., 2018; Naik et al., 2018; Hossain et al., 2020; Ettinger,
2020; Hartmann et al., 2021) or coreference (Ye et al., 2020; Varkel and
Globerson, 2020; Abdou et al., 2020). Others have focused on LMs’ ability to
reason with numbers (Johnson et al., 2020). DROP (Dua et al., 2019), for
example, is a question answering dataset designed specifically to probe LMs’
ability to count, add and subtract for answering factoid questions. Models
have also been tailored for numerical reasoning Geva et al. (2020); Zhang et
al. (2020). Cobbe et al. (2021) proposes to use a verification task during
pretraining of LMs to improve their ability to solve math word problems.
Others have studied monotonicity inference (Hu et al., 2019; Yanaka et al.,
2019, 2020), and Fang and Lou (2021) recently focused on the two quantifier
words part and whole in an error analysis for named entity recognition.

Many NLU benchmarks contain quantifier words, but their influence on
performance has not been studied systematically. One exception to this is that
generalized quantifiers have been used to generate adversarial examples in the
context of numerical reasoning (Naik et al., 2018; Nie et al., 2020). TaxiNLI
(Joshi et al., 2020), which categorizes 15 types of reasoning abilities, is a
dataset drawn from MNLI. In their taxonomy, the Quantifier category only
refers to universal and existential quantifiers, not to generalized
quantifiers, and ditto for Kim et al. (2019). All of the above focused on
English, but in an extension to TaxiNLI, K et al. (2021) incorporated
quantifiers into the Logic class and found a large cross-lingual transfer gap
on LMs.

##  9 Conclusion

Quantifiers lie in the intersection of logic, linguistics and NLP research. It
is essential for NLU systems to learn quantifier reasoning. We examined
generalized quantifiers in multilingual NLU tasks with regards to their
expressiveness and logical reasoning requirement. Our survey and experiments
indicate quantifiers are neglected to a degree and cause significant
performance drops for neural LMs. To better understand LMs’ reasoning
abilities, we release GQNLI, a novel generalized quantifier NLI challenge
dataset. With the pervasiveness of generalized quantifiers, we stress that
more efforts are necessary to investigate: (1) when and why models
systematically fail when quantifiers interact with other operators; (2) how to
improve cross-lingual transferability of quantifiers; (3) how we can exploit
the theoretical results about generalized quantifiers from logic and
linguistic studies, so as to improve the logical inference ability of neural
LMs.

## Acknowledgements

We would like to thank Miryam de Lhoneux, Constanza Fierro, Desmond Elliott
and the anonymous reviewers for their valuable feedback.

## References

  * Abdou et al. (2020) Mostafa Abdou, Vinit Ravishankar, Maria Barrett, Yonatan Belinkov, Desmond Elliott, and Anders Søgaard. 2020.  The sensitivity of language models and humans to Winograd schema perturbations.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 7590–7604, Online. Association for Computational Linguistics.
  * Ali and Shapiro (1993) Syed S Ali and Stuart C Shapiro. 1993.  Natural language processing using a propositional semantic network with structured variables.  _Minds and machines_ , 3(4):421–451.
  * Artetxe et al. (2020) Mikel Artetxe, Sebastian Ruder, and Dani Yogatama. 2020.  On the cross-lingual transferability of monolingual representations.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 4623–4637, Online. Association for Computational Linguistics.
  * Asai and Hajishirzi (2020) Akari Asai and Hannaneh Hajishirzi. 2020.  Logic-guided data augmentation and regularization for consistent question answering.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 5642–5650, Online. Association for Computational Linguistics.
  * Bach et al. (1995) Emmon W. Bach, Eloise Jelinek, Angelika Kratzer, and Barbara H. Partee. 1995.  Quantification in natural languages.
  * Barwise and Cooper (1981) Jon Barwise and Robin Cooper. 1981.  Generalized quantifiers and natural language.  In _Philosophy, language, and artificial intelligence_ , pages 241–301. Springer.
  * Bowman et al. (2015a) Samuel R. Bowman, Gabor Angeli, Christopher Potts, and Christopher D. Manning. 2015a.  A large annotated corpus for learning natural language inference.  In _Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing_ , pages 632–642, Lisbon, Portugal. Association for Computational Linguistics.
  * Bowman et al. (2015b) Samuel R. Bowman, Christopher Potts, and Christopher D. Manning. 2015b.  Recursive neural networks can learn logical semantics.  In _Proceedings of the 3rd Workshop on Continuous Vector Space Models and their Compositionality_ , pages 12–21, Beijing, China. Association for Computational Linguistics.
  * Cobbe et al. (2021) Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. 2021.  Training verifiers to solve math word problems.  _ArXiv_ , abs/2110.14168.
  * Conneau et al. (2020) Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzmán, Edouard Grave, Myle Ott, Luke Zettlemoyer, and Veselin Stoyanov. 2020.  Unsupervised cross-lingual representation learning at scale.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 8440–8451, Online. Association for Computational Linguistics.
  * Conneau et al. (2018) Alexis Conneau, Ruty Rinott, Guillaume Lample, Adina Williams, Samuel Bowman, Holger Schwenk, and Veselin Stoyanov. 2018.  XNLI: Evaluating cross-lingual sentence representations.  In _Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing_ , pages 2475–2485, Brussels, Belgium. Association for Computational Linguistics.
  * Cooper et al. (1994) Robin Cooper, Richard Crouch, Jan Van Eijck, Chris Fox, Josef Van Genabith, Jan Jaspers, Hans Kamp, Manfred Pinkal, Massimo Poesio, Stephen Pulman, et al. 1994\.  Fracas: A framework for computational semantics.  _Deliverable D6_.
  * Devlin et al. (2019) Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019.  BERT: Pre-training of deep bidirectional transformers for language understanding.  In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)_ , pages 4171–4186, Minneapolis, Minnesota. Association for Computational Linguistics.
  * Dua et al. (2019) Dheeru Dua, Yizhong Wang, Pradeep Dasigi, Gabriel Stanovsky, Sameer Singh, and Matt Gardner. 2019.  DROP: A reading comprehension benchmark requiring discrete reasoning over paragraphs.  In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)_ , pages 2368–2378, Minneapolis, Minnesota. Association for Computational Linguistics.
  * Dvořák and Holčapek (2015) Antonín Dvořák and Michal Holčapek. 2015.  Type 1 , 1 fuzzy quantifiers determined by fuzzy measures on residuated lattices. part iii. extension, conservativity and extensionality.  _Fuzzy Sets Syst._ , 271(C):133–155.
  * Ettinger (2020) Allyson Ettinger. 2020.  What BERT is not: Lessons from a new suite of psycholinguistic diagnostics for language models.  _Transactions of the Association for Computational Linguistics_ , 8:34–48.
  * Evans et al. (2018) Richard Evans, David Saxton, David Amos, Pushmeet Kohli, and Edward Grefenstette. 2018.  Can neural networks understand logical entailment?  _arXiv preprint arXiv:1802.08535_.
  * Fang and Lou (2021) Lei Fang and Jian-Guang Lou. 2021.  Part & whole extraction: Towards a deep understanding of quantitative facts for percentages in text.  _ArXiv_ , abs/2110.13505.
  * Fintel and Matthewson (2008) Kai Fintel and Lisa Matthewson. 2008.  Universals in semantics.
  * Frege (1879) Gottlob Frege. 1879.  Begriffsschrift. eine der arithmetischen nachgebildete formelsprache des reinen denkens.
  * Geiger et al. (2019) Atticus Geiger, Ignacio Cases, Lauri Karttunen, and Christopher Potts. 2019.  Posing fair generalization tasks for natural language inference.  In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)_ , pages 4485–4495, Hong Kong, China. Association for Computational Linguistics.
  * Geva et al. (2020) Mor Geva, Ankit Gupta, and Jonathan Berant. 2020.  Injecting numerical reasoning skills into language models.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 946–958, Online. Association for Computational Linguistics.
  * Geva et al. (2022) Mor Geva, Tomer Wolfson, and Jonathan Berant. 2022.  Break, Perturb, Build: Automatic Perturbation of Reasoning Paths Through Question Decomposition.  _Transactions of the Association for Computational Linguistics_ , 10:111–126.
  * Gil (1982) David Gil. 1982.  Quantifier scope, linguistic variation, and natural language semantics.  _Linguistics and Philosophy_ , 5(4):421–472.
  * Grice (1989) Paul Grice. 1989.  _Studies in the Way of Words_.  Harvard University Press.
  * Gururangan et al. (2018) Suchin Gururangan, Swabha Swayamdipta, Omer Levy, Roy Schwartz, Samuel Bowman, and Noah A. Smith. 2018.  Annotation artifacts in natural language inference data.  In _Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers)_ , pages 107–112, New Orleans, Louisiana. Association for Computational Linguistics.
  * Hartmann et al. (2021) Mareike Hartmann, Miryam de Lhoneux, Daniel Hershcovich, Yova Kementchedjhieva, Lukas Nielsen, Chen Qiu, and Anders Søgaard. 2021.  A multilingual benchmark for probing negation-awareness with minimal pairs.  In _Proceedings of the 25th Conference on Computational Natural Language Learning_ , pages 244–257, Online. Association for Computational Linguistics.
  * Haruta et al. (2020) Izumi Haruta, Koji Mineshima, and Daisuke Bekki. 2020.  Logical inferences with comparatives and generalized quantifiers.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics: Student Research Workshop_ , pages 263–270, Online. Association for Computational Linguistics.
  * Higginbotham and May (1981) James Higginbotham and Robert May. 1981.  Questions, quantifiers and crossing.
  * Horn (2010) Laurence R. Horn. 2010.  The expression of negation.
  * Hossain et al. (2020) Md Mosharaf Hossain, Venelin Kovatchev, Pranoy Dutta, Tiffany Kao, Elizabeth Wei, and Eduardo Blanco. 2020.  An analysis of natural language inference benchmarks through the lens of negation.  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , pages 9106–9118, Online. Association for Computational Linguistics.
  * Hu et al. (2019) Hai Hu, Qi Chen, and Larry Moss. 2019.  Natural language inference with monotonicity.  In _Proceedings of the 13th International Conference on Computational Semantics - Short Papers_ , pages 8–15, Gothenburg, Sweden. Association for Computational Linguistics.
  * Hu et al. (2020) Junjie Hu, Sebastian Ruder, Aditya Siddhant, Graham Neubig, Orhan Firat, and Melvin Johnson. 2020.  Xtreme: A massively multilingual multi-task benchmark for evaluating cross-lingual generalisation.  In _International Conference on Machine Learning_ , pages 4411–4421. PMLR.
  * Johnson et al. (2020) Devin Johnson, Denise Mak, Andrew Barker, and Lexi Loessberg-Zahl. 2020.  Probing for multilingual numerical understanding in transformer-based language models.  In _Proceedings of the Third BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP_ , pages 184–192, Online. Association for Computational Linguistics.
  * Joshi et al. (2020) Pratik Joshi, Somak Aditya, Aalok Sathe, and Monojit Choudhury. 2020.  TaxiNLI: Taking a ride up the NLU hill.  In _Proceedings of the 24th Conference on Computational Natural Language Learning_ , pages 41–55, Online. Association for Computational Linguistics.
  * K et al. (2021) Karthikeyan K, Aalok Sathe, Somak Aditya, and Monojit Choudhury. 2021.  Analyzing the effects of reasoning types on cross-lingual transfer performance.  In _Proceedings of the 1st Workshop on Multilingual Representation Learning_ , pages 86–95, Punta Cana, Dominican Republic. Association for Computational Linguistics.
  * Katsos et al. (2016) Napoleon Katsos, Chris Cummins, Maria-José Ezeizabarrena, Anna Gavarró, Jelena Kuvač Kraljević, Gordana Hrzica, Kleanthes K Grohmann, Athina Skordi, Kristine Jensen De López, Lone Sundahl, et al. 2016.  Cross-linguistic patterns in the acquisition of quantifiers.  _Proceedings of the National Academy of Sciences_ , 113:9244 – 9249\.
  * Katsos et al. (2012) Napoleon Katsos, Maria-Jose Ezeizabarrena, Anna Gavarro, Jelena Kuvac Kraljevic, Gordana Hrzica, Kleanthes K Grohmann, Athina Skordi, Kristine Jensen de Lopez, Lone Sundahl, Angeliek Van Hout, et al. 2012.  The acquisition of quantification across languages: some predictions.
  * Keenan and Paperno (2012) Edward L. Keenan and Denis Paperno. 2012.  Handbook of quantifiers in natural language.
  * Keenan and Westerståhl (1997) Edward L. Keenan and Dag Westerståhl. 1997.  Generalized quantifiers in linguistics and logic.  In _Handbook of Logic and Language_.
  * Kiela et al. (2021) Douwe Kiela, Max Bartolo, Yixin Nie, Divyansh Kaushik, Atticus Geiger, Zhengxuan Wu, Bertie Vidgen, Grusha Prasad, Amanpreet Singh, Pratik Ringshia, Zhiyi Ma, Tristan Thrush, Sebastian Riedel, Zeerak Waseem, Pontus Stenetorp, Robin Jia, Mohit Bansal, Christopher Potts, and Adina Williams. 2021.  Dynabench: Rethinking benchmarking in NLP.  In _Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_ , pages 4110–4124, Online. Association for Computational Linguistics.
  * Kim et al. (2019) Najoung Kim, Roma Patel, Adam Poliak, Patrick Xia, Alex Wang, Tom McCoy, Ian Tenney, Alexis Ross, Tal Linzen, Benjamin Van Durme, Samuel R. Bowman, and Ellie Pavlick. 2019.  Probing what different NLP tasks teach machines about function word comprehension.  In _Proceedings of the Eighth Joint Conference on Lexical and Computational Semantics (*SEM 2019)_ , pages 235–249, Minneapolis, Minnesota. Association for Computational Linguistics.
  * Lample and Conneau (2019) Guillaume Lample and Alexis Conneau. 2019.  Cross-lingual language model pretraining.  In _NeurIPS_.
  * Levinson et al. (2000) Stephen C Levinson, C Stephen, and Stephen C Levinson. 2000.  _Presumptive meanings: The theory of generalized conversational implicature_.  MIT press.
  * Lewis et al. (2020) Patrick Lewis, Barlas Oguz, Ruty Rinott, Sebastian Riedel, and Holger Schwenk. 2020\.  MLQA: Evaluating cross-lingual extractive question answering.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 7315–7330, Online. Association for Computational Linguistics.
  * Li et al. (2019) Tao Li, Vivek Gupta, Maitrey Mehta, and Vivek Srikumar. 2019.  A logic-driven framework for consistency of neural models.  In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)_ , pages 3924–3935, Hong Kong, China. Association for Computational Linguistics.
  * Lindström (1966) Per Lindström. 1966.  First order predicate logic with generalized quantifiers.  _Theoria_ , 32(3):186–195.
  * Liu et al. (2019) Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019.  Roberta: A robustly optimized bert pretraining approach.  _ArXiv_ , abs/1907.11692.
  * Matthewson (2001) Lisa Matthewson. 2001.  Quantification and the nature of crosslinguistic variation.  _Natural Language Semantics_ , 9(2):145–189.
  * Matthewson (2004) Lisa Matthewson. 2004.  On the methodology of semantic fieldwork.  _International journal of American linguistics_ , 70(4):369–415.
  * Matthewson (2008) Lisa Matthewson. 2008.  Quantification: A cross-linguistic perspective.
  * Montague (1973) Richard Montague. 1973.  The proper treatment of quantification in ordinary english.  In _Approaches to natural language_ , pages 221–242. Springer.
  * Mostowski (1957) Andrzej Mostowski. 1957.  On a generalization of quantifiers.  _Fundamenta Mathematicae_ , 44(1):12–36.
  * Mul and Zuidema (2019) Mathijs Mul and Willem Zuidema. 2019.  Siamese recurrent networks learn first-order logic reasoning and exhibit zero-shot compositional generalization.
  * Naik et al. (2018) Aakanksha Naik, Abhilasha Ravichander, Norman Sadeh, Carolyn Rose, and Graham Neubig. 2018.  Stress test evaluation for natural language inference.  In _Proceedings of the 27th International Conference on Computational Linguistics_ , pages 2340–2353, Santa Fe, New Mexico, USA. Association for Computational Linguistics.
  * Nie et al. (2019) Yixin Nie, Haonan Chen, and Mohit Bansal. 2019.  Combining fact extraction and verification with neural semantic matching networks.  In _Association for the Advancement of Artificial Intelligence (AAAI)_.
  * Nie et al. (2020) Yixin Nie, Adina Williams, Emily Dinan, Mohit Bansal, Jason Weston, and Douwe Kiela. 2020.  Adversarial NLI: A new benchmark for natural language understanding.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 4885–4901, Online. Association for Computational Linguistics.
  * Nikolaeva (2012) Liudmila Nikolaeva. 2012.  _Quantifiers in Adyghe_, pages 21–82. Springer Netherlands, Dordrecht.
  * Nivre et al. (2020) Joakim Nivre, Marie-Catherine de Marneffe, Filip Ginter, Jan Hajič, Christopher D. Manning, Sampo Pyysalo, Sebastian Schuster, Francis Tyers, and Daniel Zeman. 2020.  Universal Dependencies v2: An evergrowing multilingual treebank collection.  In _Proceedings of the 12th Language Resources and Evaluation Conference_ , pages 4034–4043, Marseille, France. European Language Resources Association.
  * Ogrodniczuk et al. (2020) Maciej Ogrodniczuk, Vincent Ng, Yulia Grishina, and Sameer Pradhan, editors. 2020\.  _Proceedings of the Third Workshop on Computational Models of Reference, Anaphora and Coreference_. Association for Computational Linguistics, Barcelona, Spain (online).
  * Ogrodniczuk et al. (2019) Maciej Ogrodniczuk, Sameer Pradhan, Yulia Grishina, and Vincent Ng, editors. 2019\.  _Proceedings of the Second Workshop on Computational Models of Reference, Anaphora and Coreference_. Association for Computational Linguistics, Minneapolis, USA.
  * Parrish et al. (2021) Alicia Parrish, William Huang, Omar Agha, Soo-Hwan Lee, Nikita Nangia, Alexia Warstadt, Karmanya Aggarwal, Emily Allaway, Tal Linzen, and Samuel R. Bowman. 2021\.  Does putting a linguist in the loop improve NLU data collection? In _Findings of the Association for Computational Linguistics: EMNLP 2021_ , pages 4886–4901, Punta Cana, Dominican Republic. Association for Computational Linguistics.
  * Partee (1970) Barbara Hall Partee. 1970.  Negation, conjunction, and quantifiers: Syntax vs. semantics.  _Foundations of Language_ , 6(2):153–165.
  * Peters and Westerståhl (2006) Stanley Peters and Dag Westerståhl. 2006.  _Quantifiers in language and logic_.  Oxford University Press.
  * Qi et al. (2020) Peng Qi, Yuhao Zhang, Yuhui Zhang, Jason Bolton, and Christopher D. Manning. 2020\.  Stanza: A python natural language processing toolkit for many human languages.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics: System Demonstrations_ , pages 101–108, Online. Association for Computational Linguistics.
  * Ribeiro et al. (2019) Marco Tulio Ribeiro, Carlos Guestrin, and Sameer Singh. 2019.  Are red roses red? evaluating consistency of question-answering models.  In _Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics_ , pages 6174–6184, Florence, Italy. Association for Computational Linguistics.
  * Ribeiro et al. (2020) Marco Tulio Ribeiro, Tongshuang Wu, Carlos Guestrin, and Sameer Singh. 2020.  Beyond accuracy: Behavioral testing of NLP models with CheckList.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 4902–4912, Online. Association for Computational Linguistics.
  * Schmerl and Simpson (1982) James H. Schmerl and Stephen G. Simpson. 1982.  On the role of ramsey quantifiers in first order arithmetic.  _J. Symb. Log._ , 47:423–435.
  * Srivastava et al. (2018) Shashank Srivastava, Igor Labutov, and Tom Mitchell. 2018.  Zero-shot learning of classifiers from natural language quantification.  In _Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pages 306–316, Melbourne, Australia. Association for Computational Linguistics.
  * Stateva et al. (2019) Penka Stateva, Arthur Stepanov, Viviane Déprez, Ludivine Emma Dupuy, and Anne Colette Reboul. 2019.  Cross-linguistic variation in the meaning of quantifiers: Implications for pragmatic enrichment.  _Frontiers in Psychology_ , 10:957.
  * Steinert-Threlkeld (2019) Shane Steinert-Threlkeld. 2019.  Learnability and semantic universals.  _Semantics and Pragmatics_ , 12:4.
  * Steinert-Threlkeld (2021) Shane Steinert-Threlkeld. 2021.  Quantifiers in natural language: Efficient communication and degrees of semantic universals.  _Entropy_ , 23(10).
  * Szymanik (2016) Jakub Szymanik. 2016.  _Cognitive Processing of Quantifiers_, pages 51–83. Springer International Publishing, Cham.
  * Szymanik and Thorne (2015) Jakub Szymanik and Camilo Thorne. 2015.  Semantic complexity of quantifiers and their distribution in corpora.  In _Proceedings of the 11th International Conference on Computational Semantics_ , pages 64–69, London, UK. Association for Computational Linguistics.
  * Uegaki (2022) Wataru Uegaki. 2022.  The Informativeness/Complexity Trade-Off in the Domain of Boolean Connectives.  _Linguistic Inquiry_ , pages 1–39.
  * Varkel and Globerson (2020) Yuval Varkel and Amir Globerson. 2020.  Pre-training mention representations in coreference models.  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , pages 8534–8540, Online. Association for Computational Linguistics.
  * Veldhoen and Zuidema (2017) Sara Veldhoen and Willem Zuidema. 2017.  Can neural networks learn logical reasoning?  _CLASP Papers in Computational Linguistics_ , page 34.
  * Westerståhl (1989) Dag Westerståhl. 1989.  Quantifiers in formal and natural languages.  In _Handbook of philosophical logic_ , pages 1–131. Springer.
  * Williams et al. (2018) Adina Williams, Nikita Nangia, and Samuel Bowman. 2018.  A broad-coverage challenge corpus for sentence understanding through inference.  In _Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers)_ , pages 1112–1122, New Orleans, Louisiana. Association for Computational Linguistics.
  * Williams et al. (2020) Adina Williams, Tristan Thrush, and Douwe Kiela. 2020.  Anlizing the adversarial natural language inference dataset.
  * Woods (1991) William A. Woods. 1991.  Understanding subsumption and taxonomy: A framework for progress.  In _Principles of Semantic Networks_.
  * Yanaka et al. (2020) Hitomi Yanaka, Koji Mineshima, Daisuke Bekki, and Kentaro Inui. 2020.  Do neural models learn systematicity of monotonicity inference in natural language? In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 6105–6117, Online. Association for Computational Linguistics.
  * Yanaka et al. (2019) Hitomi Yanaka, Koji Mineshima, Daisuke Bekki, Kentaro Inui, Satoshi Sekine, Lasha Abzianidze, and Johan Bos. 2019.  Can neural networks understand monotonicity reasoning? In _Proceedings of the 2019 ACL Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP_ , pages 31–40, Florence, Italy. Association for Computational Linguistics.
  * Ye et al. (2020) Deming Ye, Yankai Lin, Jiaju Du, Zhenghao Liu, Peng Li, Maosong Sun, and Zhiyuan Liu. 2020.  Coreferential Reasoning Learning for Language Representation.  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , pages 7170–7186, Online. Association for Computational Linguistics.
  * Yen (1991) John Yen. 1991.  Generalizing term subsumption languages to fuzzy logic.  In _IJCAI_.
  * Yin et al. (2021) Wenpeng Yin, Dragomir Radev, and Caiming Xiong. 2021.  DocNLI: A large-scale dataset for document-level natural language inference.  In _Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021_ , pages 4913–4922, Online. Association for Computational Linguistics.
  * Zadeh (1983) Lotfi A Zadeh. 1983.  A computational approach to fuzzy quantifiers in natural languages.  In _Computational linguistics_ , pages 149–184. Elsevier.
  * Zhang et al. (2020) Xikun Zhang, Deepak Ramachandran, Ian Tenney, Yanai Elazar, and Dan Roth. 2020.  Do language embeddings capture scales? In _Findings of the Association for Computational Linguistics: EMNLP 2020_ , pages 4889–4896, Online. Association for Computational Linguistics.
  * Zipf (1949) George Kingsley Zipf. 1949.  Human behavior and the principle of least effort.

## Appendices

##  Appendix A Regular Expressions for Generalized Quantifiers

Generalized Quantifiers |  Regular Expressions
---|---
some(A)(B) = 1 |  ⬇ (some|several|much|many)\/det .*\/(nsubj|obj|obl)|(some|several|much|many)\/nsubj|(some|several|much|many)\/amod \w+\/nsubj:pass\end{lstlisting}  \\\  \textbf{all}(A)(B) = 1 & \begin{lstlisting}[breaklines]  (every|all|each)\/det .*\/(nsubj|obj|obl)|all\/det:predet .*\/(nsubj|obj|obl) |everything|everyone|everybody
more than k the(A)(B) = 1 |


# Graphene whisperitronics: transducing whispering gallery modes into
electronic transport

Boris Brun1∗ Viet-Hung Nguyen1 Nicolas Moreau1 Sowmya Somanchi2,3
Kenji Watanabe4 Takashi Taniguchi5 Jean-Christophe Charlier1 Christoph
Stampfer2,3 & Benoit Hackens1∗ 1IMCN/NAPS & MODL, Université catholique de
Louvain (UCLouvain),
B-1348 Louvain-la-Neuve, Belgium
2 JARA-FIT and 2nd Institute of Physics - RWTH Aachen University, Germany
3 Peter Grünberg Institute (PGI-9), Forschungszentrum Jülich, 52425 Jülich,
Germany
4 Research Center for Functional Materials, National Institute for Materials
Science,
1-1 Namiki, Tsukuba 305-0044, Japan.
5 International Center for Materials Nanoarchitectonics, National Institute
for Materials Science,
1-1 Namiki, Tsukuba 305-0044, Japan
∗ To whom correspondence should be addressed;
E-mails: boris.brun<EMAIL_ADDRESS>
###### Abstract

When confined in circular cavities, graphene relativistic charge carriers
occupy whispering gallery modes (WGM) in analogy to classical acoustic and
optical fields. The rich geometrical patterns of the WGM decorating the local
density of states offer promising perspectives to devise new disruptive
quantum devices. However, exploiting these highly sensitive resonances
requires the transduction of the WGMs to the outside world through source and
drain electrodes, a yet unreported configuration. Here we create a circular
p-n island in a graphene device using a polarized scanning gate microscope
tip, and probe the resulting WGMs signatures in in-plane electronic transport
through the p-n island. Combining tight-binding simulations and exact solution
of the Dirac equation, we assign the measured device conductance features to
WGMs, and demonstrate mode selectivity by displacing the p-n island with
respect to a constriction. This work therefore constitutes a proof of concept
for graphene whisperitronics devices.

Keywords: Graphene, scanning gate microscopy, quantum transport, whispering
gallery modes

### Introduction

Whispering gallery modes (WGMs) denote the guided waves circulating along a
concave surface, the seminal example of this phenomenon occurring in the
whispering galleries of St Paul’s cathedral with acoustic waves. This
originally macroscopic-scale phenomenon found rich applications in
nanotechnology with the ability to design resonating galleries, e.g. for
electromagnetic waves, with nanometer-scale precision 1, 2. Taking advantage
of the high quality factor and the reduced size of optical and mechanical
whispering nano-resonators, a new class of ultra-sensitive detectors has
recently been developed, stretching the limits of physical and biological
quantities detection down to a single molecule or virus 3, 4. In the nano-
electronic world, WGMs have proven successful to achieve distant coupling
between GaAs quantum dots 5. Recently, electronic WGMs have been reported in
graphene 6, offering promising perspectives to develop a new class of devices,
taking advantage of the sensitivity of these geometrical resonances combined
with the high technological integrability of graphene.

Graphene charge carriers behave as relativistic massless particles 7,
therefore sharing part of their fundamental properties with photons.
Consequently, graphene has early been proposed as an ideal platform to realize
electron-optics experiments, in which p-n junctions play the role of lenses,
prisms or fibers, and are used to guide, refract and transmit electrons and
holes taking advantage of Klein tunneling 8, 9, 10, 11, 12, 13, 14. As a
seminal example of Dirac fermions optics, it has early been proposed 15 and
observed 16 that Fabry-Pérot (FP) interferences can arise between two facing
p-n junctions, acting as semi-reflecting mirrors. When considering a circular
p-n junction, FP interferences patterns then correspond to concentric rings,
indeed visible in simulated wavefunctions 6. But FP interferences are only a
sub-class of a larger set of WGMs : at larger angular momentum $m$, a manifold
of patterns mixing radial and concentric symmetries emerge due to circular
geometric symmetry, enriching the existing palette of Dirac fermions optics.

Figure 1:  Whisperitronics, transducing graphene whispering gallery modes to
external contacts: a, Scheme of the electrostatic potential landscape: a
biased AFM tip induces a Lorentzian potential whose maximum is $\varphi_{max}$
and half-width at half maximum is $R_{tip}$, centered on a graphene
constriction doped at a Fermi energy $E_{F}$. Electronic transport is measured
through the WGMs induced in the p-n island,the p region is below the tip. One
WGM wave-function is plotted at the center with red and blue colors. b, Real
part of the wave-function $\Re(\psi_{m}(x,y))$ corresponding to the angular
momentum $m$ = 3/2, at different resonant energies, calculated by solving the
radial Dirac equation in presence of a Lorentzian potential, in an isotropic
plane, without constriction. c, $\Re(\psi_{m}(x,y))$ for $m$ = 7/2, showing
higher angular symmetry patterns. d, LDOS in a p-n island, placed at the
center of a 240 nm-wide graphene constriction, as a function of energy $E$ in
units of the hopping parameter t and $x$ position along red dashed line in the
inset. Inset: LDOS map for the third resonant mode. Scale bar: 200 nm.

Up to now, evidencing graphene p-n islands WGMs has relied on the use of a
scanning tunneling microscope (STM)6, 17, probing the local density of states
(LDOS) through tunneling between the island and a metallic tip. However, the
practical integration of WGMs-supporting p-n islands in future electronic
devices will require to incorporate them in electron-optics setups, and
combine them with other Dirac fermions optics tools12, 13, 14. This
transduction of the WGM devices to the outside world requires driving current
between in-plane electrical contacts, a configuration in which WGMs have not
been demonstrated in graphene to date.

Here we demonstrate the detection and manipulation of WGMs in in-plane
transport through a circular encapsulated graphene p-n junction, the essential
ingredient to realize a new class of whisperitronics devices. The p-n junction
is created using the polarized tip of a scanning gate microscope (SGM) 18, 19,
which can be moved in the vicinity of a constriction (Fig. 1a). This
demonstrates a surprising robustness of WGMs with respect to geometrical
pinching of the circular island inside the constriction. Ultimately, WGMs
selectivity is revealed by displacing laterally the p-n island with respect to
the constriction, offering a very high degree of tunability. Consequently, our
work paves the way towards controllable whisperitronics, merging these freshly
revealed WGMs 6 with the existing graphene electron-optics palette.

### Results

Charge carriers in a graphene p-n island occupy a rich set of states 6, 20,
21, 22, labeled by different angular momentum m, each of them exhibiting
several resonant energies as exemplified in Fig. 1b and 1c. To sense these
different WGMs using in-plane electronic transport, the easiest way is to
connect the resonator to two separate graphene regions by pinching the p-n
island in a narrow channel, e.g. an etched constriction, as schemed in Figs.
1a and 2a. Surprisingly, the WGMs are really robust, and survive when placing
the p-n island in such a constriction. This remarkable fact is demonstrated in
Fig. 1d, where we plot the LDOS profile along transport axis in a circular p-n
junction placed at the center of a 240 nm-wide constriction. Despite the
constriction, the pattern is strikingly similar to the unperturbed p-n island
case (see section 3 of the supplementary materials). Noteworthy, the resonant
states yield LDOS patterns that extend in the low density region (indicated by
white arrows Fig. 1d), as shown experimentally in STM in Refs.17, 23, and
supported by recent theoretical explorations 24. The assignment of different m
values to these LDOS extensions is made possible by explicit comparison of the
tight-binding simulations shown in Fig. 1d (obtained using the Kwant code 25
with proper scaling 26, as well as a home-made recursive green function
code27), with the exact solution of the Dirac equation 28 (see section 2-3 of
the supplementary materials).

Figure 2:  Transport through WGMs in a constriction. a, Scheme of the
experiment: a polarized AFM tip is placed above a constriction defined in
encapsulated graphene. The tip potential is discussed in section 5 of the
supplementary materials. b, Calculated resistance as a function of the sheet
carrier density $n_{bulk}$ and the maximum density change $\Delta n_{max}$,
exhibiting oscillations in the N-P-N configuration. The value of $\Delta
n_{max}$ = $-6.5\times 10^{15}\rm m^{-2}$ used in Figs. 1d and 2d is indicated
by a blue dashed line as well as the 6 first resistance minima in this
configuration that are labeled by colored dots (red to blue), reported in
Figs. 1d and 2d for comparison. N’ and P’ denote the carrier nature in the
tip-pertubed region when it is similar to the sheet carrier type. Blue line
indicates the N-N’-N/N-P-N limit $n_{bulk}$ = $-\Delta n_{max}$. Note that it
does not coincide with the resistance maximum. c, Calculated resistance versus
$n_{bulk}$, for $\Delta n_{max}$ = $-6.5\times 10^{15}\rm m^{-2}$. d,
Resistance as a function of $V_{tip}$ and $V_{bg}$, recorded by placing the
tip above the constriction at a distance $Z_{tip}$ = 90 nm from the graphene
plane. Couples of $V_{tip}$ and $V_{bg}$ values for which the SGM images in
Fig. 3a-c are recorded are indicated by dots. e, Constriction resistance as a
function of $n_{bulk}$, for a neutral tip ($\Delta n_{max}$ = 0, green dashed
line in c). f, Constriction resistance as a function of $n_{bulk}$, for
$V_{tip}$ = -11 V ($\Delta n_{max}$ = $-6.5\times 10^{15}\rm m^{-2}$) and
$Z_{tip}$ = 90 nm, corresponding to blue dashed line in c. Colored dots
correspond to minima in the simulated data (in Fig. 2d).

The total resistance of the WGM resonator exhibits strong oscillations versus
carrier density, as shown by Fig. 2b and 2d. Remarkably, the resistance minima
correspond to the extended WGMs, as indicated by the color dots showing the
resonant energies of WGMs in Fig. 1d, translated in carriers density and
reported on the calculated resistance plots of Fig. 2b and 2d. This is
surprising, since WGMs arise due to internal reflections of the Dirac fermions
on the border of the resonator. On the other hand, they yield a non-zero LDOS
in the low-density region at the edge, which favors a coupling with the Dirac
continuum of states outside the resonator. These competing effects counter-
intuitively result in a low total resistance. Finally, we show in Fig. 2b the
evolution of the resistance oscillations with $n_{bulk}$ and $\Delta n_{max}$,
that correspond to the evolution of the different WGMs with energy and
potential strength. Noticeably, these oscillations are not equally spaced in
$n_{bulk}$, and their spacing increases as $\Delta n_{max}$ is made more
negative. The location of minima also strongly depends on the geometry of the
junction, as discussed in section S6 of the supplementary materials. This
configuration of a circular Dirac fermion resonator embedded in a constriction
is therefore expected to exhibit strong signatures of WGMs in electrical
transport.

We now provide an experimental demonstration of this theoretical prediction,
by generating a tunable p-n junction in a graphene device 19 using the
polarized metallic tip of an atomic force microscope 18. Scanning gate
microscopy (SGM) consists in scanning a polarized tip in the vicinity of a
mesoscopic device, while recording its conductance. It has initially been
developed to study mesoscopic phenomena in III-V semiconductor
heterostructures 29, 30, in which an insulating layer prevents tunneling from
a STM tip to the surface, and has been successfully been extended to the study
of transport in graphene devices 31, 32, 33. The sample (see methods),
consists of a h-BN-encapsulated monolayer graphene flake, in which a 250 nm-
wide constriction is defined by etching 34, consistent with the geometry
defined in the simulation. We first place the tip 90 nm above the constriction
center, as sketched in Fig. 2a, and record the device resistance $R$ while
varying the tip and backgate voltages $V_{tip}$ and $V_{bg}$, the experimental
knobs linearly related to $\Delta n_{max}$ and $n_{bulk}$ through the
relations $\Delta n_{max}=C_{tip}(V_{tip}-V_{tip}^{0})$ and
$n_{bulk}=C_{bg}(V_{bg}-V_{bg}^{0})$, where $C_{tip}$ is the tip-graphene
capacitance, $C_{bg}$ the back-gate capacitance, $V_{tip}^{0}$ and
$V_{bg}^{0}$ are the charge neutrality points when sweeping the tip and back-
gate voltages respectively. Details about the determination of the
capacitances and neutrality voltages are provided in supplementary materials.
The result, measured at low temperature (4.2 K), is shown in Fig. 2c, where
different regions can be identified (NN’N, NPN, or PP’P), depending on the
type of charge carriers below the tip, and in the regions away from the
constriction. When the tip voltage compensates its work-function, i.e. $\Delta
n_{max}=0$, one recovers an unpertubed behavior of the sample and the
resistance shows a single maximum at the Dirac point, as shown in Fig. 2e.
Conversely, quasi-periodic oscillations are visible in the n-p-n
configuration, similar to the corresponding simulation of Fig. 2b. This
corresponds to a bulk occupied by electrons and holes in the tip-perturbed
region. For comparison, we plot in Fig. 2f the resistance as a function of
$n_{bulk}$ for a tip voltage of -11 V, corresponding to $\Delta n_{max}$ =
$-6.5\times 10^{15}\rm m^{-2}$, similar to the calculation presented in Fig.
2d. The qualitative agreement in the evolution of the oscillations in Fig. 2d
and 2f in the range between 2$\times 10^{15}$ m-2 and 5$\times 10^{15}$ m-2,
and in particular in the positions of the seven minima marked by colored dots
in Fig. 2d, represent a first hint that the experimental resistance
oscillations correspond to the internal WGMs of the p-n island.

However, the patterns of Figs. 2b and 2c are reminiscent of FP oscillations
observed in straight gate-defined n-p-n junctions 16, 35. In a recent work 36,
we showed that when the tip is centered above the constriction, these tip-
induced oscillations exhibit similarities with FP oscillations. In particular
their contrast can be controlled by the potential smoothness, that governs the
quasi-confinement of Dirac fermions. Indeed, FP oscillations share a common
ground with WGMs’, both arising due to coherent superposition of waves in a
confined geometry. However, as indicated above, FP are actually a subclass of
WGMs in circular or elliptic cavities, corresponding to the fundamental
angular momentum m. The main question at this point is therefore whether
signatures of higher order symmetry WGMs can be clearly evidenced and
controlled when charge carriers are transmitted through the circular cavity,
making new quantum whisperitronic applications possible.

Figure 3:  Experiment: Scanning gate microscopy images and spectroscopy. a-c,
Up: SGM images, i.e. device resistance as a function of $X_{tip}$ and
$Y_{tip}$, for $V_{bg}$ = +1.5 V, $Z_{tip}$ = 90 nm and $V_{tip}$ = -6 V (a),
-8 V (b) and -10 V (c). Scale bar: 200 nm. Resistance colorscale has been
adapted for each image. Down: derivative of the same SGM images versus
$Y_{tip}$ (vertical direction). d-f, Simulated differentiated SGM maps, for 3
different tip-induced density changes corresponding to the experimental
conditions of a-c respectively. g, Measured resistance as a function of
$V_{bg}$ and tip position along the blue dash-dotted line in Fig. 3a, for a
tip voltage $V_{tip}$ = -14 V and a tip-to-graphene distance $Z_{tip}$ = 110
nm.

To reveal the wealth of oscillations arising from internal resonances of the
tip-induced p-n island, one needs to off-center the tip with respect to the
constriction. Intuitively, gradually overlapping the circular cavity with the
etched triangle-shaped edges of the constriction will strongly alter high
angular momentum radial resonances as it will break circular symmetry. We
first scan the polarized tip around the constriction, and map out resistance
as a function of tip position for different $V_{tip}$ and $V_{bg}$, yielding
the SGM maps shown in Fig. 3a to 3c (top panels). We showed in Ref. 19 that
the resistance maps obtained in this type of experiment reflect the average
Dirac fermions flow through the p-n island. Here we go one step further with
higher resolution SGM maps and spectroscopies, showing that displacing the p-n
island with respect to the constriction allows to probe its internal LDOS
resonances. In the bottom panels of Figs. 3a to 3c, we plot the derivative of
the resistance versus y-direction (transverse to transport axis) for each of
the top panels maps. The differentiated maps better highlight oscillations
discussed in the remaining part of the paper. These maps indeed exhibit arcs
centered on the constriction apexes, that are limited to the constriction
area. Such arcs are fundamentally different from the usually observed
concentric circles, ubiquitous in SGM of quantum dots 37, 38, 39 or disordered
systems40, 32, 41. The latter circles correspond to Coulomb blockade peaks
forming when the confined resonant modes associated with local cavities
induced e.g. by disorder are raised locally to the Fermi level by the gating
action of the tip. Contrary to Coulomb blockade-related circles, the arcs
visible in Figs. 3b-c can be reproduced by single particle tight-binding
simulations, as shown in Fig. 3d-f where we present the simulated SGM maps
differentiated versus $Y_{tip}$. This therefore rules out a possible Coulombic
origin.

To clarify the origin of SGM arcs, we scan the tip along a line transverse to
transport axis (blue dash-dotted line in Fig. 3a) while varying continuously
$V_{bg}$, and plot the resulting resistance map in Fig. 3g. The overall
envelope delineating the high resistance region in the lower part of Fig. 3g
(corresponding to n-p-n junction) reflects the tip-induced potential, and can
be fitted to evaluate it accurately (see Refs.19, 36 and section 6 of the
supplementary materials). Within this high resistance region, local resistance
maxima undergo the same Lorentzian evolution, with a lateral offset with
respect to the envelope, and lead to higher resistance spots at their crossing
points. These successive maxima correspond to the interference observed in
Fig. 2f and the SGM arcs in Fig. 3b-c. Figure 3g reveals that these modes are
brought to lower bulk energy (i.e. higher energy in the island) when
displacing laterally the tip with respect to the constriction center, i.e.
when the p-n island overlaps with the constriction side.

Figure 4:  Selective transmission of WGMs by lateral displacement: a, LDOS
calculated for the same tip potential as in Fig. 2, for different tip
positions along y-axis as marked in (c). Scale bar: 200 nm. b Same data
calculated for a lower bulk energy (i.e. higher energy -hence smaller Fermi
wavelength- in the resonator). Corresponding energies and tip positions are
indicated by colored dots in c. c Calculated resistance as a function of
energy and lateral p-n island displacement $Y_{c}$, for the same potential as
in Figs. 1d and 2d and 4a-b.

To understand this observation, we perform additional tight-binding
simulations and show the evolution of the LDOS in the p-n island when
displacing it along the axis transverse to the constriction axis in Fig. 4a
and 4b, for two different energies. From these maps, the role of the
constriction on the resonator WGMs is clarified: bringing the tip-induced p-n
junction above the etched region reduces the available area for the carriers
in the p-n island, and consequently promotes lower angular momentum modes in
the resonator. Indeed, depending on the starting eigen-mode, a well-defined
number of local LDOS minima are clearly visible as soon as the p-n island is
off-centered by a few tens of nanometers, as visible in second panels of Figs.
4a and 4b. This number of “holes” is then reduced one by one as the p-n island
is brought away from the constriction center (see supplementary movies SM
1-2). This illustrates that WGMs are morphology-dependent resonances, that can
be modified by changing the resonator geometry.

Finally, we present in Fig. 4c the calculated resistance as a function of
density and tip position along y-axis $Y_{c}$. Though richer details are
present compared to the experimental data of Fig. 3g, the average evolution of
the calculated resistance is consistent with the measurement, indicating that
the different resonant modes energies are brought to lower bulk density (hence
higher holes density in the p-n island) when the tip-induced potential
overlaps with the etched region.

### Discussion

Our results demonstrate two different ways to selectively address the
different electrical WGMs of the Dirac fermion resonator:
(i) Playing independently on $V_{tip}$ and $V_{bg}$ allows to clearly select
the different WGMs, with strong fingerprints in the current through the p-n
island. This contrasts with STM experiments, where the tip bias not only
changes the resonator shape but also affects the injection energy of charge
carriers, leading to a spurious replication of the WGMs at the tip bias energy
6. Here the injection energy can be tuned completely independently since the
tip and the graphene plane are only capacitively connected, so that the
intrinsic behavior of WGMs can be more readily accessed.
(ii) Displacing the p-n island laterally with respect to the constriction also
allows to selectively address the different WGMs. This is a promising
configuration since it is analogous to the configuration used in high-quality
factor opto-mechanical resonators, where the readout line is side-coupled to
the WGMs of the resonator and allows for ultra-fast and sensitive readout of
the resonator-state 42.

We would like to emphasize that in the studied geometry, both radial (“Fabry-
Pérot” like) and higher angular momentum WGM resonances contribute to
transport. Engineering further the p-n junction potential and increasing the
quality and geometry of the constriction may allow in the future to more
accurately select the different modes contributions to transport. Additional
possibilities for coupling WGMs and a transport channel, as well as for
addressing individual WGMs, can also be envisioned. For example, in analogy
with the geometry of some optical WGM devices, we anticipate that lateral
tunnel-coupling of a propagating one-dimensional graphene channel with a
circular resonating cavity would provide an even more sensitive and tunable
platform to probe individual WGMs. It would indeed combine a less invasive
approach with the extreme tunability of coupling offered by the tunnel
barrier. Finally, the high quality factor required to improve the sensitivity
of such devices can readily be engineered by tuning the potential smoothness,
as reported in Ref.36. Opening the way towards such perspectives, this work
bridges the recently discovered WGMs of graphene p-n islands with the field of
Dirac fermion optics, heralding the advent of relativistic whisperitronics, a
promising field for the engineering of disruptive quantum devices.

## References

  * Vahala 2003 Vahala, K. J. Optical microcavities. _Nature_ 2003, _424_ , 839–846
  * Foreman et al. 2015 Foreman, M. R.; Swaim, J. D.; Vollmer, F. Whispering gallery mode sensors. _Adv. Opt. Photon._ 2015, _7_ , 168–240
  * Vollmer and Arnold 2008 Vollmer, F.; Arnold, S. Whispering-gallery-mode biosensing: label-free detection down to single molecules. _Nature Methods_ 2008, _5_ , 591–596
  * He et al. 2011 He, L.; Özdemir, S. K.; Zhu, J.; Kim, W.; Yang, L. Detecting single viruses and nanoparticles using whispering gallery microlasers. _Nature Nanotechnology_ 2011, _6_ , 428–432
  * Nicolí et al. 2018 Nicolí, G.; Ferguson, M. S.; Rössler, C.; Wolfertz, A.; Blatter, G.; Ihn, T.; Ensslin, K.; Reichl, C.; Wegscheider, W.; Zilberberg, O. Cavity-Mediated Coherent Coupling between Distant Quantum Dots. _Phys. Rev. Lett._ 2018, _120_ , 236801
  * Zhao et al. 2015 Zhao, Y.; Wyrick, J.; Natterer, F. D.; Rodriguez-Nieva, J. F.; Lewandowski, C.; Watanabe, K.; Taniguchi, T.; Levitov, L. S.; Zhitenev, N. B.; Stroscio, J. A. Creating and probing electron whispering-gallery modes in graphene. _Science_ 2015, _348_ , 672–675
  * Novoselov et al. 2005 Novoselov, K. S.; Geim, A. K.; Morozov, S. V.; Jiang, D.; Katsnelson, M. I.; Grigorieva, I. V.; Dubonos, S. V.; Firsov, A. A. Two-dimensional gas of massless Dirac fermions in graphene. _Nature_ 2005, _438_ , 197
  * Klein 1929 Klein, O. Die Reflexion von Elektronen an einem Potentialsprung nach der relativistischen Dynamik von Dirac. _Zeitschrift für Physik_ 1929, _53_ , 157–165
  * Cheianov et al. 2007 Cheianov, V. V.; Fal’ko, V.; Altshuler, B. L. The Focusing of Electron Flow and a Veselago Lens in Graphene p-n Junctions. _Science_ 2007, _315_ , 1252–1255
  * Williams et al. 2011 Williams, J. R.; Low, T.; Lundstrom, M. S.; Marcus, C. M. Gate-controlled guiding of electrons in graphene. _Nature Nanotechnology_ 2011, _6_ , 222–
  * Rickhaus et al. 2015 Rickhaus, P.; Liu, M.-H.; Makk, P.; Maurand, R.; Hess, S.; Zihlmann, S.; Weiss, M.; Richter, K.; Schönenberger, C. Guiding of Electrons in a Few-Mode Ballistic Graphene Channel. _Nano Letters_ 2015, _15_ , 5819–5825
  * Liu et al. 2017 Liu, M.-H.; Gorini, C.; Richter, K. Creating and Steering Highly Directional Electron Beams in Graphene. _Phys. Rev. Lett._ 2017, _118_ , 066801
  * Barnard et al. 2017 Barnard, A. W.; Hughes, A.; Sharpe, A. L.; Watanabe, K.; Taniguchi, T.; Goldhaber-Gordon, D. Absorptive pinhole collimators for ballistic Dirac fermions in graphene. _Nature Communications_ 2017, _8_ , 15418
  * Bøggild et al. 2017 Bøggild, P.; Caridad, J. M.; Stampfer, C.; Calogero, G.; Papior, N. R.; Brandbyge, M. A two-dimensional Dirac fermion microscope. _Nature Communications_ 2017, _8_ , 15783
  * Shytov et al. 2008 Shytov, A. V.; Rudner, M. S.; Levitov, L. S. Klein Backscattering and Fabry-Pérot Interference in Graphene Heterojunctions. _Phys. Rev. Lett._ 2008, _101_ , 156804
  * Young and Kim 2009 Young, A. F.; Kim, P. Quantum interference and Klein tunnelling in graphene heterojunctions. _Nature Physics_ 2009, _5_ , 222
  * Ghahari et al. 2017 Ghahari, F.; Walkup, D.; Gutiérrez, C.; Rodriguez-Nieva, J. F.; Zhao, Y.; Wyrick, J.; Natterer, F. D.; Cullen, W. G.; Watanabe, K.; Taniguchi, T.; Levitov, L. S.; Zhitenev, N. B.; Stroscio, J. A. An on/off Berry phase switch in circular graphene resonators. _Science_ 2017, _356_ , 845
  * Eriksson et al. 1996 Eriksson, M. A.; Beck, R. G.; Topinka, M.; Katine, J. A.; Westervelt, R. M.; Campman, K. L.; Gossard, A. C. Cryogenic scanning probe characterization of semiconductor nanostructures. _Applied Physics Letters_ 1996, _69_ , 671–673
  * Brun et al. 2019 Brun, B.; Moreau, N.; Somanchi, S.; Nguyen, V.-H.; Watanabe, K.; Taniguchi, T.; Charlier, J.-C.; Stampfer, C.; Hackens, B. Imaging Dirac fermions flow through a circular Veselago lens. _Phys. Rev. B_ 2019, _100_ , 041401
  * Gutiérrez et al. 2016 Gutiérrez, C.; Brown, L.; Kim, C.-J.; Park, J.; Pasupathy, A. N. Klein tunnelling and electron trapping in nanometre-scale graphene quantum dots. _Nature Physics_ 2016, _12_ , 1069
  * Lee et al. 2016 Lee, J.; Wong, D.; Velasco Jr, J.; Rodriguez-Nieva, J. F.; Kahn, S.; Tsai, H.-Z.; Taniguchi, T.; Watanabe, K.; Zettl, A.; Wang, F.; Levitov, L. S.; Crommie, M. F. Imaging electrostatically confined Dirac fermions in graphene quantum dots. _Nature Physics_ 2016, _12_ , 1032
  * Jiang et al. 2017 Jiang, Y.; Mao, J.; Moldovan, D.; Masir, M. R.; Li, G.; Watanabe, K.; Taniguchi, T.; Peeters, F. M.; Andrei, E. Y. Tuning a circular p-n junction in graphene from quantum confinement to optical guiding. _Nature Nanotechnology_ 2017, _12_ , 1045
  * Gutiérrez et al. 2018 Gutiérrez, C.; Walkup, D.; Ghahari, F.; Lewandowski, C.; Rodriguez-Nieva, J. F.; Watanabe, K.; Taniguchi, T.; Levitov, L. S.; Zhitenev, N. B.; Stroscio, J. A. Interaction-driven quantum Hall wedding cake-like structures in graphene quantum dots. _Science_ 2018, _361_ , 789
  * Le and Nguyen 2020 Le, T. L.; Nguyen, V. L. Quantitative study of electronic whispering gallery modes in electrostatic-potential induced circular graphene junctions. _Journal of Physics: Condensed Matter_ 2020, _32_ , 255502
  * Groth et al. 2014 Groth, C. W.; Wimmer, M.; Akhmerov, A. R.; Waintal, X. Kwant: a software package for quantum transport. _New Journal of Physics_ 2014, _16_ , 063065
  * Liu et al. 2015 Liu, M.-H.; Rickhaus, P.; Makk, P.; Tóvári, E.; Maurand, R.; Tkatschenko, F.; Weiss, M.; Schönenberger, C.; Richter, K. Scalable Tight-Binding Model for Graphene. _Phys. Rev. Lett._ 2015, _114_ , 036601
  * Nguyen et al. 2010 Nguyen, V. H.; Bournel, A.; Dollfus, P. The conduction gap in double gate bilayer graphene structures. _Journal of Physics: Condensed Matter_ 2010, _22_ , 115304
  * Nguyen and Charlier 2018 Nguyen, V. H.; Charlier, J.-C. Klein tunneling and electron optics in Dirac-Weyl fermion systems with tilted energy dispersion. _Phys. Rev. B_ 2018, _97_ , 235113
  * Topinka et al. 2000 Topinka, M. A.; LeRoy, B. J.; Shaw, S. E. J.; Heller, E. J.; Westervelt, R. M.; Maranowski, K. D.; Gossard, A. C. Imaging Coherent Electron Flow from a Quantum Point Contact. _Science_ 2000, _289_ , 2323–2326
  * Topinka et al. 2001 Topinka, M. A.; LeRoy, B. J.; Westervelt, R. M.; Shaw, S. E. J.; Fleischmann, R.; Heller, E. J.; Maranowski, K. D.; Gossard, A. C. Coherent branched flow in a two-dimensional electron gas. _Nature_ 2001, _410_ , 183–186
  * Connolly et al. 2011 Connolly, M. R.; Chiu, K. L.; Lombardo, A.; Fasoli, A.; Ferrari, A. C.; Anderson, D.; Jones, G. A. C.; Smith, C. G. Tilted potential induced coupling of localized states in a graphene nanoconstriction. _Phys. Rev. B_ 2011, _83_ , 115441
  * Pascher et al. 2012 Pascher, N.; Bischoff, D.; Ihn, T.; Ensslin, K. Scanning gate microscopy on a graphene nanoribbon. _Applied Physics Letters_ 2012, _101_ , 063101
  * Garcia et al. 2013 Garcia, A. G. F.; König, M.; Goldhaber-Gordon, D.; Todd, K. Scanning gate microscopy of localized states in wide graphene constrictions. _Phys. Rev. B_ 2013, _87_ , 085446
  * Terrés et al. 2016 Terrés, B.; Chizhova, L. A.; Libisch, F.; Peiro, J.; Jörger, D.; Engels, S.; Girschik, A.; Watanabe, K.; Taniguchi, T.; Rotkin, S. V.; Burgdörfer, J.; Stampfer, C. Size quantization of Dirac fermions in graphene constrictions. _Nature Communications_ 2016, _7_ , 11528
  * Rickhaus et al. 2013 Rickhaus, P.; Maurand, R.; Liu, M.-H.; Weiss, M.; Richter, K.; Schönenberger, C. Ballistic interferences in suspended graphene. _Nature Communications_ 2013, _4_ , 2342
  * Brun et al. 2020 Brun, B.; Moreau, N.; Somanchi, S.; Nguyen, V.-H.; Mreńca-Kolasińska, A.; Watanabe, K.; Taniguchi, T.; Charlier, J.-C.; Stampfer, C.; Hackens, B. Optimizing Dirac fermions quasi-confinement by potential smoothness engineering. _2D Materials_ 2020, _7_ , 025037
  * Pioda et al. 2004 Pioda, A.; Kičin, S.; Ihn, T.; Sigrist, M.; Fuhrer, A.; Ensslin, K.; Weichselbaum, A.; Ulloa, S. E.; Reinwald, M.; Wegscheider, W. Spatially Resolved Manipulation of Single Electrons in Quantum Dots Using a Scanned Probe. _Phys. Rev. Lett._ 2004, _93_ , 216801
  * Schnez et al. 2010 Schnez, S.; Güttinger, J.; Huefner, M.; Stampfer, C.; Ensslin, K.; Ihn, T. Imaging localized states in graphene nanostructures. _Phys. Rev. B_ 2010, _82_ , 165445
  * Walkup et al. 2020 Walkup, D.; Ghahari, F.; Gutiérrez, C.; Watanabe, K.; Taniguchi, T.; Zhitenev, N. B.; Stroscio, J. A. Tuning single-electron charging and interactions between compressible Landau level islands in graphene. _Phys. Rev. B_ 2020, _101_ , 035428
  * Bleszynski et al. 2007 Bleszynski, A. C.; Zwanenburg, F. A.; Westervelt, R. M.; Roest, A. L.; Bakkers, E. P. A. M.; Kouwenhoven, L. P. Scanned Probe Imaging of Quantum Dots inside InAs Nanowires. _Nano Letters_ 2007, _7_ , 2559–2562
  * Liu et al. 2015 Liu, P.; Martins, F.; Hackens, B.; Desplanque, L.; Wallart, X.; Pala, M. G.; Huant, S.; Bayot, V.; Sellier, H. Formation of quantum dots in the potential fluctuations of InGaAs heterostructures probed by scanning gate microscopy. _Phys. Rev. B_ 2015, _91_ , 075313
  * Rosenblum et al. 2015 Rosenblum, S.; Lovsky, Y.; Arazi, L.; Vollmer, F.; Dayan, B. Cavity ring-up spectroscopy for ultrafast sensing with optical microresonators. _Nature Communications_ 2015, _6_ , 6788

## Acknowledgments

We thank Prof. J.A. Stroscio for enthusiastic support and valuable inputs. B.B
thanks C. Groth for valuable help regarding Kwant simulations. Computational
resources have been provided by the Consortium des Équipements de Calcul
Intensif (CÉCI), funded by the Fonds de la Recherche Scientifique de Belgique
(F.R.S.-FNRS) under Grant No. 2.5020.11 and by the Walloon Region. The present
research was funded by the Fédération Wallonie-Bruxelles through the ARC
Grants No. 16/21-077 and No. 21/26-116 and from the European Union’s Horizon
2020 Research and Innovation program (Core 3 No. 881603). B.B. (research
assistant), N.M. (FRIA fellowship), B.H. (research associate and PDR No.
T.0105.21), V.-H.N. and J.-C.C. (PDR No. T.1077.15 and ERA-Net No.
R.50.07.18.F) acknowledge financial support from the F.R.S.-FNRS of Belgium.
Support by the Helmholtz Nanoelectronic Facility (HNF), the EU ITN SPINOGRAPH
and the DFG (SPP-1459) is gratefully acknowledged. This work was also
supported by the FLAG-ERA grant TATTOOS, by the Deutsche
Forschungsgemeinschaft (DFG, German Research Foundation) - 437214324. K.W. and
T.T. acknowledge support from the Elemental Strategy Initiative conducted by
the MEXT, Japan ,Grant Number JPMXP0112101001, JSPS KAKENHI Grant Numbers
JP20H00354 and the CREST(JPMJCR15F3), JST.

#### Supporting informations:

The Supporting Information is available free of charge on the ACS Publications
website at DOI: XXXX. Supporting Information include figures S1 to S6 and a
text detailing sample and measurement methods, individual contributions of the
WGMs to the total LDOS, labelling the WGMs in transport, scaling of the tight-
binding model, tip-induced potential, simulation of SGM images and effect of
npn junction shape on WGM resonances. Two Supplementary Movies (SM1 and SM2)
show the evolution of LDOS for the 4th and 6th resonant mode when displacing
the tip perpendicular to transport axis.

#### Figures.