# Two-Stage Cryogenic HEMT Based Amplifier For Low Temperature Detectors

J. Anczarski<EMAIL_ADDRESS>Stanford University, Stanford, CA 94305,
USA  SLAC National Accelerator Laboratory, Menlo Park, CA, 94025, USA  Kavli
Institute for Particle Astrophysics and Cosmology, Stanford University,
Stanford, CA, 94035, USA  M. Dubovskov  Santa Clara University, Santa Clara,
CA 95053, USA  C. W. Fink  Los Alamos National Laboratory, Los Alamos, NM
87545, USA  S. Kevane  Stanford University, Stanford, CA 94305, USA  SLAC
National Accelerator Laboratory, Menlo Park, CA, 94025, USA  Kavli Institute
for Particle Astrophysics and Cosmology, Stanford University, Stanford, CA,
94035, USA  N. A. Kurinsky  SLAC National Accelerator Laboratory, Menlo Park,
CA, 94025, USA  Kavli Institute for Particle Astrophysics and Cosmology,
Stanford University, Stanford, CA, 94035, USA  S. J. Meijer  Los Alamos
National Laboratory, Los Alamos, NM 87545, USA  A. Phipps  California State
University, East Bay, Hayward CA 94542, USA  F. Ronning  Los Alamos National
Laboratory, Los Alamos, NM 87545, USA  I. Rydstrom  Santa Clara University,
Santa Clara, CA 95053, USA  A. Simchony  Stanford University, Stanford, CA
94305, USA  SLAC National Accelerator Laboratory, Menlo Park, CA, 94025, USA
Kavli Institute for Particle Astrophysics and Cosmology, Stanford University,
Stanford, CA, 94035, USA  Z. Smith  Stanford University, Stanford, CA 94305,
USA  SLAC National Accelerator Laboratory, Menlo Park, CA, 94025, USA  Kavli
Institute for Particle Astrophysics and Cosmology, Stanford University,
Stanford, CA, 94035, USA  S. M. Thomas  Los Alamos National Laboratory, Los
Alamos, NM 87545, USA  S. L. Watkins  Los Alamos National Laboratory, Los
Alamos, NM 87545, USA  B. A. Young  Santa Clara University, Santa Clara, CA
95053, USA

###### Abstract

To search for dark matter candidates with masses below $\mathcal{O}$(MeV), the
SPLENDOR (Search for Particles of Light dark mattEr with Narrow-gap
semiconDuctORs) experiment is developing novel narrow-bandgap semiconductors
with electronic bandgaps on the order of 1-100 meV. In order to detect the
charge signal produced by scattering or absorption events, SPLENDOR has
designed a two-stage cryogenic HEMT-based amplifier with an estimated charge
resolution approaching the single-electron level. A low-capacitance ($\sim$1.6
pF) HEMT is used as a buffer stage at $T=10\,\mathrm{mK}$ to mitigate effects
of stray capacitance at the input. The buffered signal is then amplified by a
higher-capacitance ($\sim$200 pF) HEMT amplifier stage at $T=4\,\mathrm{K}$.
Importantly, the design of this amplifier makes it usable with any insulating
material - allowing for rapid prototyping of a variety of novel detector
materials. We present the two-stage cryogenic amplifier design, preliminary
voltage noise performance, and estimated charge resolution of 7.2 electrons.

##  I Introduction

Understanding the particle nature of dark matter, which makes up approximately
85% of the matter content in the universe, remains one of the biggest open
questions in the fields of particle physics and cosmology. After decades of
null results in searches for weakly interacting massive dark matter candidates
[1, 2, 3, 4], experimental and theoretical efforts have shifted towards a
broad range of masses, including lighter mass dark matter candidates with
masses below $\mathcal{O}$(MeV) [5]. These light mass dark matter particles
present a substantial detection challenge, as their relatively low kinetic
energy limits the energy deposited in a target to be sub-eV. The SPLENDOR
collaboration (Search for Particles of Light dark mattEr with Narrow-gap
semiconDuctORs) is developing novel narrow-bandgap single-crystal
semiconductors with electronic bandgaps on the order 1-100meV [6, 7].

When dark matter interacts with a semiconductor, electrons (holes) are excited
into the conduction (valence) band. This created charge can be drifted in an
electric field and the image current integrated to measure the induced signal.
To readout this charge signal, the SPLENDOR collaboration designed and built a
two-stage cryogenic charge amplifier utilizing high electron mobility
transistors (HEMTs) using similar HEMT transistors that have been used in
experiments such as SuperCDMS [8] and RICOCHET [9]. This amplifier input stage
operates at $\sim$10 mK to both reduce thermal noise and parasitic
capacitance. An important feature of this charge amplifier design is that its
performance is independent of the detector substrate material—allowing for
rapid prototyping of different semiconducting detector geometries and
materials.

Figure 1: The simplified amplifier topology. Charge produced within the
detector is integrated onto Q1, which acts as a voltage buffer. The signal is
amplified at the 4K gain stage, and undergoes a futher stage of room
temperature amplification before digitization. The accrued charge on the Q1
gate is passively reset via the feedback resistor $R_{\text{fb}}$

##  II Two-Stage HEMT Amplifier

The approximate resolution scaling for a charge-integrating amplifier depends
on the noise spectrum of the front end amplifier ($N_{V}$), the total
capacitance of the system, the charge collection efficiency
$\varepsilon_{\text{CCE}}$, and the inverse bandwidth $\tau$ as [10, 8]

$\sigma_{q}\propto\frac{N_{V}}{\varepsilon_{\text{CCE}}\tau}\left(C_{\text{in}}+C_{\text{det}}+C_{\text{par}}\right),$
(1)

where the total capacitance has been written as a sum of the input capacitance
of the amplifier $(C_{\text{in}}$), the capacitance of the detector
($C_{\text{det}}$), and the parasitic capacitance present at the amplifier
input ($C_{\text{par}}$). To optimize this charge resolution, we employ a
design to minimize each of these three components. We use a point contact
style geometry on $\mathcal{O}(1\,\mathrm{mm})$ thick crystals to achieve
$C_{\text{det}}\approx 2\,\mathrm{pF}$. To minimize $C_{\text{in}}$ we use a
front-end HEMT with a gate-to-source capacitance of $1.6\,\mathrm{pF}$. The
detector, HEMT, and coupling capacitor are all connected within a few mm of
each other to minimize $C_{\text{par}}$. Finally, the HEMTs are specifically
designed to reduce low frequency noise over the signal bandwidth ($\sim$10 Hz
– 1 MHz), minimizing $N_{V}$.

This is achieved by using a two-stage amplifier topology (Fig. 1) in which the
front-end stage is integrated into the detector housing held at
$10\,\mathrm{mK}$ and is configured as a “common-drain” voltage follower with
approximately unity gain. The buffer stage prevents signal degradation from
parasitic capacitance in the wiring to the warmer amplifier stages. This
design results in a total capacitance of $C_{\text{total}}\approx
5\,\mathrm{pF}$.

The 10 mK buffer stage capacitively couples to the detector and acts as a
transimpedance amplifier. The high-value feedback and bias resistors integrate
the drift current produced within the detector into a voltage signal at the
gate of Q1. A NbTi cable connects the buffer stage to the 4K gain stage. The
gain stage hosts the common-drain source resistor and couples the buffered
voltage signal to the gate of Q2. The higher capacitance Q2 is configured as a
common-source voltage amplifier, providing cryogenic gain. Via a low
resistance copper cable, the 4K gain stage connects to room temperature
electronics, which include a Stanford Research Systems SR 560 differential
amplifier, low noise DC power supplies, and a Moku:Pro for data acquisition
[11].

Both the base stage and gain stage utilize commercial low frequency, low noise
cryoHEMTs [12]. The gain stage uses a 200 pF HEMT with an estimated 30-50
gain, while the buffer stage utilizes a 1.6 pF HEMT with a gain of
approximately unity. Within the amplifier bandwidth, the total open loop gain
is given by

$G_{OL}=\left(\frac{g_{m1}R_{\text{follow}}}{1+g_{m1}R_{\text{follow}}}\right)\left(-g_{m2}R_{\text{gain}}\right)$
(2)

where $g_{m1}$ is the transconductance of Q1 and $g_{m2}$ is the
transconductance of Q2.

The base stage input HEMT (Q1) is coupled to the detector through a 100 pF
coupling capacitor, which is connected directly to the detector via a fuzz
button as shown in Fig. 2. We estimate that this compact geometry between the
detector and the gate of Q1 should make $C_{\text{par}}\sim$1 pF.

Figure 2: Top: Picture of the backside of the prototype detector housing with
silicon test devices installed. Bottom: Cutaway view of the side of the CAD
model of the detector housing. Shown is the detector assembly highlighting the
low capacitance connection between the detector and the front end HEMT of the
two stage amplifier.

##  III Voltage Noise Performance and Estimated Charge Resolution

For the preliminary testing discussed in this work, a 100 k$\Omega$ feedback
resistor was installed on the base temperature board and the amplifier was run
without a detector. In this configuration, the resistor Johnson noise at
$T=10\,\mathrm{mK}$ is negligible compared to the expected amplifier noise.
While this relatively low resistor value prevents the input network from
acting as an effective charge integrator, the standalone amplifier voltage
noise performance can be measured. The amplifier was installed in an Oxford
Proteox dilution refrigerator with a base temperature of 9.5 mK, including the
amplifier power dissipation.

The voltage gain of the amplifier was found to be 36 at 10 Hz (after removing
the SR560 contribution) by injecting a known amplitude sine wave into the Q1
gate via the feedback resistor. The gain at higher frequencies was not
measured due to the room temperature power supply filtering required for low
noise performance. In separate tests without power supply filtering, the gain
was found to be stable up to several hundred kHz with the bandwidth being
determined by time constant formed by the 4K gain resistor and output cable
capacitance.

The input-referred noise voltage performance of the amplifier is shown in Fig.
3. Operating parameters of Q1 and Q2 are summarized in Table 1. Also shown are
the expected noise contributions using a model based on Ref. [10]. The higher
observed noise may be due to the lower operating power of the Q1 HEMT (1.3
$\mu$W), which are typically biased at 100 $\mu$W.

Figure 3: The measured amplifier voltage noise spectrum, with the base temperature board at $T=10\,\mathrm{mK}$. The theoretical noise spectrum of the amplifier combines the HEMT white noise, current noise, and $1/f$ noise with 100k$\Omega$ feedback resistor. The high noise floor is dominated by room temperature electronics coupling down to the input of the amplifier. Table 1: The Q1 and Q2 gate-source capacitance, gate-drain capacitance, operating temperature, drain-source voltage, drain-source current, and power dissipation. HEMT | $C_{\text{gs}}\,\mathrm{[pF]}$ | $C_{\text{gd}}\,\mathrm{[pF]}$ | $T\,\mathrm{[K]}$ | $V_{\text{ds}}\,\mathrm{[mV]}$ |  $I_{\text{ds}}\,[\mu\mathrm{A}$] | $P\,[\mu\mathrm{W}]$
---|---|---|---|---|---|---
Q1 | 1.6 | 0.8 | 0.01 | 25 | 50 | 1.3
Q2 | 236 | 8.9 | 4 | 89 | 940 | 83.7

To estimate the expected charge resolution, the measured amplifier noise
spectrum is added in quadrature to the theoretical $T=10\,\mathrm{mK}$ Johnson
noise of a 100 M$\Omega$ resistor (the parallel combination of the bias and
feedback resistors). The total input capacitance
($C_{\text{in}}+C_{\text{det}}+C_{\text{par}}$) is assumed to be 5 pF. An
estimated charge resolution of 7.2 electrons is calculated from using the
combined noise spectrum and pulse template with the optimal filter formalism
[8].

##  IV Conclusion

We have developed a cryogenic split-stage amplifier for use with novel narrow
bandgap semiconductor ionization detectors developed by the SPLENDOR
experiment. We have shown that our low-capacitance HEMTs perform without issue
at $10\,\mathrm{mK}$ and with low power dissipation. Combining our preliminary
noise spectrum with an optimal filter calculation achieves a one-sigma
estimated charge resolution of 7.2 electrons. The unique geometry of this low-
noise charge amplifier allows it to be easily coupled to any insulating
detector material—making it possible to generically study novel material
substrates as detector prototypes with never-before-achieved charge
resolution.

##  V Acknowledgments

This work was supported by the U.S. Department of Energy through the Los
Alamos National Laboratory. Los Alamos National Laboratory is operated by
Triad National Security, LLC, for the National Nuclear Security Administration
of U.S. Department of Energy (Contract No. 89233218CNA000001). Research
presented in this article is supported by the Laboratory Directed Research and
Development program of Los Alamos National Laboratory under project numbers
20220135DR, 20220252ER, 20230777PRD1, 20230782PRD1, and is receiving support
from US DOE HEP Detector R&D KA-25 program. This work was also supported in
part by the US Department of Energy Early Career Research Program (ECRP) under
FWP 100872. J. Anczarski is supported by the Kavli Institute for Particle
Astrophysics and Cosmology Chabollah Fellowship. Support for A. T. J. Phipps
was provided by a faculty support grant from the Cal State East Bay Division
of Academic Affairs.

## References

  * Aprile  _et al._ [2018] E. Aprile _et al._ (XENON Collaboration), Dark Matter Search Results from a One Ton-Year Exposure of XENON1T, Phys. Rev. Lett. 121, 111302 (2018).
  * Akerib  _et al._ [2017] D. S. Akerib _et al._ (LUX Collaboration), Results from a Search for Dark Matter in the Complete LUX Exposure, Phys. Rev. Lett. 118, 021303 (2017).
  * Cui  _et al._ [2017] X. Cui _et al._ (PandaX-II Collaboration), Dark Matter Results from 54-Ton-Day Exposure of PandaX-II Experiment, Phys. Rev. Lett. 119, 181302 (2017).
  * Alkhatib  _et al._ [2021] I. Alkhatib _et al._ , Light Dark Matter Search with a High-Resolution Athermal Phonon Detector Operated above Ground, Phys. Rev. Lett. 127, 061801 (2021).
  * Battaglieri  _et al._ [2017] M. Battaglieri _et al._ , US Cosmic Visions: New Ideas in Dark Matter 2017: Community Report (2017), arXiv:1707.04591 [hep-ph] .
  * Rosa  _et al._ [2020] P. Rosa, Y. Xu, M. Rahn, J. Souza, S. Kushwaha, L. Veiga, A. Bombardi, S. Thomas, M. Janoschek, E. Bauer, _et al._ , Colossal magnetoresistance in a nonsymmorphic antiferromagnetic insulator, npj Quantum Mater. 5, 52 (2020).
  * Piva  _et al._ [2021] M. M. Piva, M. C. Rahn, S. M. Thomas, B. L. Scott, P. G. Pagliuso, J. D. Thompson, L. M. Schoop, F. Ronning, and P. F. S. Rosa, Robust Narrow-Gap Semiconducting Behavior in Square-Net La3Cd2As6, Chem. Mater. 33, 4122 (2021).
  * Phipps  _et al._ [2019] A. Phipps, A. Juillard, B. Sadoulet, B. Serfass, and Y. Jin, A HEMT-based cryogenic charge amplifier with sub-100 eVee ionization resolution for massive semiconductor dark matter detectors, Nucl. Instrum. Meth. Phys. Res. A 940, 181 (2019).
  * Augier  _et al._ [2023] C. Augier, G. Baulieu, V. Belov, L. Bergé, J. Billard, G. Bres, J. L. Bret, A. Broniatowski, M. Calvo, A. Cazes, D. Chaize, M. Chala, M. Chapellier, L. Chaplinsky, G. Chemin, R. Chen, J. Colas, E. Cudmore, M. D. Jesus, _et al._ , First demonstration of 30 eVee ionization energy resolution with Ricochet germanium cryogenic bolometers (2023), arXiv:2306.00166 [astro-ph.IM] .
  * Juillard  _et al._ [2019] A. Juillard, J. Billard, D. Chaize, J.-B. Filippini, D. Misiak, L. Vagneron, A. Cavanna, Q. Dong, Y. Jin, C. Ulysse, A. Bounab, X. de la Broise, C. Nones, and A. Phipps, Low-Noise HEMTs for Coherent Elastic Neutrino Scattering and Low-Mass Dark Matter Cryogenic Semiconductor Detectors, J. Low Temp. Phys. 199, 798 (2019).
  * Watkins [2023] S. L. Watkins, SPLENDAQ: A Detector-Agnostic Data Acquisition System for Small-Scale Physics Experiments (2023), arXiv:2310.01279 [physics.ins-det] .
  * Jin  _et al._ [2014] Y. Jin, Q. Dong, A. Cavanna, U. Gennser, L. Couraud, and C. Ulysse, in  _2014 12th IEEE International Conference on Solid-State and Integrated Circuit Technology (ICSICT)_ (2014) pp. 1–4.


# Searching for PETs: Using Distributional and Sentiment-Based Methods to Find
Potentially Euphemistic Terms

Patrick Lee  Martha Gavidia  Anna Feldman  Jing Peng
Montclair State University
Montclair, New Jersey
{leep6, gavidiam1 feldmana<EMAIL_ADDRESS>

###### Abstract

This paper presents a linguistically driven proof of concept for finding
_potentially euphemistic terms_ , or PETs. Acknowledging that PETs tend to be
commonly used expressions for a certain range of sensitive topics, we make use
of distributional similarities to select and filter phrase candidates from a
sentence and rank them using a set of simple sentiment-based metrics. We
present the results of our approach tested on a corpus of sentences containing
euphemisms, demonstrating its efficacy for detecting single and multi-word
PETs from a broad range of topics. We also discuss future potential for
sentiment-based methods on this task.

##  1 Introduction

Euphemisms are mild or indirect expressions used in place of harsher or more
offensive ones. They can be used to show politeness when discussing sensitive
or taboo topics Bakhriddionova (2021) such as saying _passed away_ instead of
_died_ , or as a way to make unpleasant or unappealing things sound better
Karam (2011), such as _ethnic cleansing_ instead of _genocide_. They can even
be used as a means to conceal the truth Rababah (2014); for example, saying
_enhanced interrogation techniques_ but meaning _torture_. Euphemisms pose a
challenge to natural language processing due to this figurative behavior, but
also because they can have a literal interpretation in certain contexts.
Furthermore, humans may not agree on what a euphemism is Gavidia et al.
(2022). Thus, we consider any words/phrases used in this nature to be a
_Potentially Euphemistic Term_ (PET).

In this paper, we present a proof of concept for finding PETs in an input
sentence, and apply it to a novel euphemism corpus Gavidia et al. (2022)111Our
code is available at https://github.com/marsgav/PETDetection.. We base our
approach on several linguistic intuitions: (1) PETs tend to be commonly used
expressions about a certain range of sensitive topics, (2) humans make a
conscious lexical choice to convey politeness and formality and (3) because of
their linguistic function, PETs should result in greater sentiment shifts when
replaced by their literal interpretations; we experiment with distributionally
similar alternatives as a source of such interpretations. Leveraging a variety
of existing tools (Gensim’s Phrases Rehurek and Sojka (2011), word2vec classes
Mikolov et al. (2013), and roBERTa Liu et al. (2019)), we implement a simple
algorithm to extract, filter, and rank PET candidates. Despite its simplicity,
our approach is able to identify the target euphemism as one of the top two
phrase candidates for 725 out of 1382 sentences in our test dataset. It also
shows promising results in identifying PETs that were not originally marked,
as well as for sentences outside our dataset. We believe our results and
subsequent discussion are an important baseline for using distributional and
sentiment-based methods for detecting euphemisms.

The structure of the paper is as follows: in Section 2, we discuss related
work surrounding euphemisms. Section 3 provides details on the text data used,
Section 4 describes our approach broken down into 4 stages: phrase extraction,
phrase filtering, phrase paraphrasing and phrase ranking. Section 5 includes
our results and a quantitative and qualitative analysis, and Section 6
concludes with future work.

##  2 Related Work

Computational approaches to processing euphemisms Felt and Riloff (2020); Zhu
et al. (2021); Zhu and Bhat (2021); Magu and Luo (2018); Kapron-King and Xu
(2021); Gavidia et al. (2022) have shown much promise, but the dynamic nature
of euphemisms remains an obstacle. A euphemism annotation task conducted by
Gavidia et al. (2022) shows that the inherent ambiguity of euphemisms leads to
low agreement in what qualifies as a euphemism. Through this task, the
researchers found that some euphemisms are used so often to discuss sensitive
topics (e.g., venereal disease as a euphemism for sexually transmitted
disease), that they become _commonly accepted terms_ , or CATs. Additionally,
they find that even when annotators agreed on the intended meaning of a
euphemism, e.g. slim as a euphemism for skinny, they still did not agree on
the label of euphemistic vs. non euphemistic. The nuance associated with
euphemisms still remains one of the biggest challenges.

Felt and Riloff (2020) were one of the first to tackle euphemisms from a
computational standpoint. They leverage sentiment analysis to recognize _x-
phemisms_ , which is the term they use to refer to both euphemisms and
dysphemisms. Whereas euphemisms are polite expressions to discuss sensitive
topics, dysphemisms are purposely direct, blunt and can be derogatory. They
find near-synonym pairs for three topics: lying, firing and stealing, and use
a weakly supervised bootstrapping algorithm for semantic lexicon induction
Thelen and Riloff (2002). They use lexical cues and sentiment analysis to
classify phrases as euphemistic, dysphemistic or neutral. Their approach is
interesting, as it is the first of its kind and their use of sentiment
analysis to identify euphemisms has inspired our work.

Zhu et al. (2021) approach the task of discovering euphemisms from the lens of
content moderation. Their goal was the detection of euphemisms used for formal
drug names on social media. They define two problems: the first is the
detection of euphemisms, and the second is identifying what the euphemisms
found actually refer to. However, their view on euphemisms is different from
ours, as they treat euphemisms simply as code words. This work is similar to
Magu and Luo (2018), who also explore euphemisms as code words in hate speech.
Zhu and Bhat (2021) and Zhu et al. (2021) both treat detection and
identification as a masked language problem where they use a masked language
model (MLM) as a filter to get rid of sentences that are not related to their
seedlist of euphemisms and then again to find euphemistic candidates. Like
Felt and Riloff (2020), Zhu et al. (2021) and Zhu and Bhat (2021) show
promise, though their narrow topic focus limit the kinds of euphemisms that
can be found.

Lastly, Kapron-King and Xu (2021) conduct a diachronic evaluation of euphemism
usage between genders. While this work is not aimed at finding euphemisms,
their work provides many of the PETs used in the creation of the Euphemism
Corpus Gavidia et al. (2022), which we use in this paper.

##  3 Data

Our work utilizes a Euphemism Corpus created by Gavidia et al. (2022) as our
test data. The raw text data for this corpus comes from The Corpus of Global
Web-Based English (GloWbE)Davies and Fuchs (2015). GloWbE contains text data
for 20 English speaking countries from websites, blogs and forums; this corpus
is compiled using just a portion of the US Dialect of English text.
The Euphemism Corpus contains 1,382 euphemistic sentences, each annotated with
one potentially euphemistic term per sentence. These potentially euphemistic
terms, or PETs Gavidia et al. (2022) are single and multi word expressions
that are used in a euphemistic sense.
Futhermore, we use the US Dialect of English portion of GloWbE to train a
Phrases model (gensim) Rehurek and Sojka (2011) to create word collocations
within our data which are then fed into a word2vec model to produce vector
representations for the words in our corpus. The following section explains
both of these aspects in further detail.

##  4 Our Approach

The algorithm developed for this experiment performs the following sub tasks
to identify a PET in a sentence: phrase extraction, phrase filtering, phrase
paraphrasing and phrase ranking. Simply put, the algorithm locates all of the
single and multi word expressions within a sentence and through the subsequent
tasks, determines which expressions may be a PET.

###  4.1 Phrase Extraction

We use the phrase (collocation) detection model, Phrases, in the Gensim
library Rehurek and Sojka (2011) to identify single and multi word expressions
within the US Dialect of English portion of GloWbE Davies and Fuchs (2015).
Phrases takes raw text as input and detects a bigram if a scoring function for
two words exceeds a certain threshold. It joins two unigrams into a single
token, separated by an underscore. We use Phrases to train our data twice in
order to create up to 3 word expressions to account for PETs like enhanced
interrogation techniques. Upon training, Phrases creates a Phraser object that
can be applied to new text data to identify bigram and trigram expressions. As
such, we use this Phraser object on the Euphemism Corpus, resulting in
identification of single and multiword expressions contained within it.

###  4.2 Phrase Filtering

The single and multiword expressions found with Phrases now need to be
topically filtered. This step is essential in identifying the phrases that are
related to a sensitive topic. We remove all stopwords, and then, leveraging
the embeddings created with word2vec, calculate the cosine similarity between
the phrases and a list of words representing sensitive topics Gavidia et al.
(2022). These sensitive topics include: death, sexual activity, employment,
bodily functions, politics, physical/mental attributes, and substances. We
notice that many of the PETs in the Euphemism Corpus have a summed cosine
similarity score above 1.5; therefore, we empirically set this as the
threshold. Every phrase with a similarity measure above this is referred to as
a quality phrase and moves on to the next task of paraphrasing.

###  4.3 Phrase Paraphrasing

The idea behind paraphrasing a PET is that, in theory, if we replace quality
phrases with "paraphrases" that are more literal, there should be a shift in
the sentiment of the sentence. Since using euphemisms can be seen as a
conscious lexical choice made to avoid awkward or uncomfortable situations,
when we choose to use a PET, our goal is to make our speech less negative,
more positive and less offensive. We test this by "paraphrasing" quality
phrases using the top 25 most similar words as output by word2vec (excluding
paraphrases which contain the quality phrase as a substring, as these are not
really distinct alternatives) and perform sentiment analysis to measure
negative, positive and offensive scoresLiu et al. (2019) before and after
replacement.
Using the distributionally similar words output by word2vec follows the
intuition that phrase semantics are determined by their context, and that
phrases which have the same mentions should have the same semantics Li et al.
(2022). We recognize that these are not official paraphrases; however, as seen
by the example below for the PET _intoxicated_ , word2vec produces good
results.

From this list, we see that "drunk" and "under the influence of alcohol" would
be considered literal interpretations of "intoxicated", and as such, would
serve as suitable replacements for the paraphrasing task.

###  4.4 Phrase Ranking

To measure differences in sentiment and offensiveness of the original
sentences before and after substituting with alternatives output by word2vec,
we use a roBERTa base model trained on tweets for sentiment analysis and
offensive language identification Liu et al. (2019). We chose RoBERTa’s
sentiment and offensiveness models because they have been shown to be useful
in distinguishing PETs from other phrases Gavidia et al. (2022). The specific
scores we utilize are negative, neutral, and positive sentiment scores, as
well as non-offensiveness and offensiveness scores. We calculate scores for
all replacements and aggregate them into a single score as a measure of which
PET had replacements that caused the greatest shift in sentiment. Reasoning
that alternatives to PETs are likely more polarized than alternatives to non-
PETs, we rank the quality phrases using this aggregate from highest to lowest.
The phrases with the top 2 highest scores in each sentence are deemed to be
PET candidates.

Empirically, we notice that both offensiveness scores tend to be particularly
useful for distinguishing euphemisms from polarized (but otherwise non-
euphemistic) terms, so we attribute more weight to them. We hypothesize that
both non-offensive and offensive scores are useful because terms that are
distributionally similar to PETs are likely to be either (1) similar, non-
offensive alternatives or (2) their offensive alternatives. See Appendix A for
an illustration of the paraphrasing stage, along with sample sentiment shifts.

##  5 Results and Discussion

This section provides our quantitative and qualitative analyses and a
discussion on the failures and limitations of our algorithm.

###  5.1 Quantitative Analysis

Table 1 summarizes the results from each step of our procedure. The second
column shows the number of total candidate phrases at every stage while the
last column shows how many test sentences, out of 1382, still retain the
target PETs in the list of candidates at that stage. Note the paraphrasing
stage shows no changes as this stage is not meant to reduce the list of PETs.

Stage |  # Candidates |  # Targets Retained
---|---|---
Phrase Extraction |  31348 |  1251
Phrase Filtering |  10503 |  1198
Phrase Paraphrasing |  10503 |  1198
Phrase Ranking |  2728 |  725
Table 1:  A summary of the subtasks in our algorithm, along with the number of
candidate phrases and PETs that were retained after each.

The algorithm correctly identifies the target PET in 725 sentences.
Additionally, through human evaluation, we find that it also identifies new
non-target PETs in the data. Out of the 725 PETs deemed to have been
successfully detected, 468 of them were ranked as the 1st place candidate,
while 257 were 2nd place. Overall, this gives us a success rate of about
52.5%. Since there was an average of 7.6 phrase candidates per sentence, we
calculate the chance of randomly selecting the target to be one of the top two
candidates to be 2 * (1/7.6) $\approx\textbf{26.3\%}$. The sizable improvement
over this baseline — which doesn’t include new, non-target PETs that were
detected — leads us to believe our results are significant.

###  5.2 Qualitative Analysis

Below, Table 1 includes an example of a correctly identified target PET as
well as a new PET that was not annotated for in our test data. While the
target PET mentally disabled is identified as the second top ranked phrase, we
deem the first ranked phrase, intoxicated person, to be a PET as well.

Sentence: in addition bats that are found in a room with a person who can not
reliably rule out physical contact for example a sleeping person a child a
mentally disabled person or an intoxicated person will need to be tested for
rabies
---
Target PET: mentally disabled
ExtractedPhrases: [’in’, ’addition’, ’bats’, ’that’, ’are’, ’found’, ’in’,
’a’, ’room’, ’with’, ’a’, ’person_who’, ’can_not’, ’reliably’, ’rule_out’,
’physical_contact’, ’for’, ’example’, ’a’, ’sleeping’, ’person’, ’a’, ’child’,
’a’, ’mentally_disabled’, ’person’, ’or’, ’an’, ’intoxicated_person’,
’will_need’, ’to’, ’be_tested’, ’for’, ’rabies’]
QualityPhrases: [’bats’, ’person_who’, ’can_not’, ’reliably’,
’physical_contact’, ’sleeping’, ’person’, ’child’, ’mentally_disabled’,
’intoxicated_person’, ’be_tested’, ’rabies’]
RankedPhrases: [(’intoxicated person’, 2.898948520421982), (’mentally
disabled’, 2.7745959013700485), (’rabies’, 2.036529041826725), (’physical
contact’, 1.7015496864914894), (’can not’, 1.6931570619344711), (’sleeping’,
1.267698973417282), (’person’, 1.171182319521904), (’person who’,
1.0447067320346832), (’bats’, 0.9130769670009613), (’be tested’,
0.864994041621685), (’reliably’, 0.8625116124749184), (’child’,
0.23687118291854858)]
Table 2: Example of target PET ’mentally disabled’ as second ranked phrase
with new PET ’intoxicated person’ ranked first.

We include additional examples of sentences in which the target PET was
correctly identified as a top two candidate phrase in Appendix B. Appendix C
also showcases more new PETs that were found - by human evaluation. We discuss
instances where our algorithm failed to detect a target PET in the following
section.

###  5.3 Failures

The output candidates may not include the target PET for a couple of reasons:
(1) it is not retained from the phrase detection or topic filtering stages, or
(2) it produces a low sentiment or offensiveness shift compared to other
candidates. Notably, for (1), we notice MWEs are sometimes not collocated
properly, either because they aren’t detected as a common collocation (e.g.,
’between’ and ’jobs’ are never joined into a single phrase) or because they
are collocated with other terms (e.g., ’almost_lost’ and ’my_lunch’ are
detected to be MWEs, but as a result, not ’lost_my_lunch’). For (2), we notice
that other candidates (polarized phrases or broad nouns in particular) simply
produce higher shifts in all or most sentiment categories compared to the
target PET. (See Appendix D for more examples.) As such, while simply
computing the increases in sentiment scores and prioritizing offensiveness
scores produces workable results for this proof of concept, there is a clear
need to experiment with better methods for utilizing sentiment; this is left
to future experimentation.

##  6 Conclusion and Future Work

Our work is a proof of concept for finding PETs in a given euphemistic
sentence. While our algorithm produces significant results, we recognize the
limitations of our work and propose the following ideas for advancement of
this specific task. Firstly, we rely on the Gensim library for identifying
multi-word expressions and obtaining word embeddings, but experimentation with
different parameters and techniques (e.g., using different phrase extraction
methods, different bigram scoring functions or contextualized word embeddings)
may yield better results. Secondly, a mechanism for filtering each candidate’s
alternatives could help reduce the number of semantically dissimilar
replacements during the paraphrasing stage. Next, while we only use aggregate
increases in sentiment and offensiveness scores for ranking candidates, a
variety of other methods (e.g., taking averages or maximums) and measures
(e.g., indirectness and vagueness) may be useful for distinguishing PETs.
Lastly, while the task of differentiating literal versus euphemistic usages of
PETs is not a focus on this paper, our algorithm shows some promise on the
issue (see Appendix E), and it is an important task that could use future
work; Appendix E also shows the performance of our algorithm on unseen data.

## Acknowledgements

This material is based upon work supported by the National Science Foundation
under Grant No. 1704113.

## References

  * Bakhriddionova (2021) Dildora Oktamovna Bakhriddionova. 2021.  The needs of using euphemisms.  _Mental Enlightenment Scientific-Methodological Journal_ , 2021(06):55–64.
  * Davies and Fuchs (2015) Mark Davies and Robert Fuchs. 2015.  Expanding horizons in the study of world englishes with the 1.9 billion word global web-based english corpus (glowbe).  _English World-Wide_ , 36(1):1–28.
  * Felt and Riloff (2020) Christian Felt and Ellen Riloff. 2020.  Recognizing euphemisms and dysphemisms using sentiment analysis.  In _Proceedings of the Second Workshop on Figurative Language Processing_ , pages 136–145.
  * Gavidia et al. (2022) Martha Gavidia, Patrick Lee, Anna Feldman, and Jing Peng. 2022.  Cats are fuzzy pets: A corpus and analysis of potentially euphemistic terms.  _arXiv preprint arXiv:2205.02728_.
  * Kapron-King and Xu (2021) Anna Kapron-King and Yang Xu. 2021.  A diachronic evaluation of gender asymmetry in euphemism.  In _Proceedings of the 2nd International Workshop on Computational Approaches to Historical Language Change 2021_ , pages 28–38, Online. Association for Computational Linguistics.
  * Karam (2011) Savo Karam. 2011.  Truths and euphemisms: How euphemisms are used in the political arena.  _3L: Language, Linguistics, Literature®_ , 17(1).
  * Li et al. (2022) Jiacheng Li, Jingbo Shang, and Julian McAuley. 2022.  Uctopic: Unsupervised contrastive learning for phrase representations and topic mining.  _arXiv preprint arXiv:2202.13469_.
  * Liu et al. (2019) Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019.  Roberta: A robustly optimized bert pretraining approach.  _arXiv preprint arXiv:1907.11692_.
  * Magu and Luo (2018) Rijul Magu and Jiebo Luo. 2018.  Determining code words in euphemistic hate speech using word embedding networks.  In _Proceedings of the 2nd workshop on abusive language online (ALW2)_ , pages 93–100.
  * Mikolov et al. (2013) Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013.  Efficient estimation of word representations in vector space.  _arXiv preprint arXiv:1301.3781_.
  * Rababah (2014) Hussein Abdo Rababah. 2014.  The translatability and use of x-phemism expressions (x-phemization): Euphemisms, dysphemisms and orthophemisms) in the medical discourse.  _Studies in Literature and Language_ , 9(3):229–240.
  * Rehurek and Sojka (2011) Radim Rehurek and Petr Sojka. 2011.  Gensim–python framework for vector space modelling.  _NLP Centre, Faculty of Informatics, Masaryk University, Brno, Czech Republic_ , 3(2).
  * Thelen and Riloff (2002) Michael Thelen and Ellen Riloff. 2002.  A bootstrapping method for learning semantic lexicons using extraction pattern contexts.  In _Proceedings of the 2002 conference on empirical methods in natural language processing (EMNLP 2002)_ , pages 214–221.
  * Zhu and Bhat (2021) Wanzheng Zhu and Suma Bhat. 2021.  Euphemistic phrase detection by masked language model.  In _Findings of the Association for Computational Linguistics: EMNLP 2021_ , pages 163–168, Punta Cana, Dominican Republic. Association for Computational Linguistics.
  * Zhu et al. (2021) Wanzheng Zhu, Hongyu Gong, Rohan Bansal, Zachary Weinberg, Nicolas Christin, Giulia Fanti, and Suma Bhat. 2021.  Self-supervised euphemism detection and identification for content moderation.  _arXiv preprint arXiv:2103.16808_.

##  Appendix A Example Sentiment Shifts when Replacing PET Candidates

Below we illustrate the paraphrasing stage for two sample PETs (only showing
the top 10 replacements for each). Each replacement is listed along with the
sentiment shifts it produces in the original sentence (of which, only the
increases are aggregated into a final score for the candidate). The five
numbers indicate, in order, the [negative, neutral, positive, non-offensive,
offensive] sentiment shifts.

Example 1
---
Original Sentence: the city has told quite a few mistruths in order to get
this city office approved
Target PET: mistruths
Top 10 Replacements:
half-truths [-0.04721798, 0.02886498, 0.018352773, -0.01894176, 0.018941715]
outright lies [0.5365411, -0.49552178, -0.041019425, -0.19229656, 0.1922966]
falsehoods [0.4495571, -0.4112787, -0.0382785, -0.117421865, 0.11742185]
untruths [0.15190402, -0.13002646, -0.021877721, -0.03839171, 0.03839159]
half truths [0.12055096, -0.11013514, -0.010415968, -0.029488266, 0.029488236]
blatant lies [0.58376455, -0.54074687, -0.04301778, -0.268206, 0.268206]
race-baiting [0.08099219, -0.06833702, -0.01265521, -0.05247575, 0.052475646]
spewing lies [0.5345895, -0.49333698, -0.041252725, -0.34204996, 0.34205]
lies and distortions [0.5648471, -0.5210455, -0.043801878, -0.15087801,
0.15087792]
fearmongering [0.12043443, -0.107453406, -0.012981113, -0.029256463,
0.029256403]
Comment: Note other potentially non-offensive alternatives like "half-truths"
and "untruths" (which sometimes result in greater shifts in non-offensiveness
than this example), and literal interpretations like "outright lies" and "lies
and distortions" (which result in significant offensiveness shifts).
Example 2
---
Original Sentence: after deadly ethnic riots rocked southern kyrgyzstan last
month one georgian minister claimed that russia has been behind the ethnic
cleansing of uzbeks
Target PET: ethnic cleansing
Top 10 Replacements:
genocide [0.022250175, -0.021986336, -0.00026384578, -0.017507195,
0.017507195]
collective punishment [-0.027520716, 0.026981518, 0.0005392225, 0.0048098564,
-0.004809916]
apartheid [-0.00591588, 0.005712375, 0.000203504, 0.011624634, -0.011624634]
massacres [0.0055012107, -0.0054178983, -8.32919e-05, -0.004523158,
0.004523158]
israeli occupation [-0.019839048, 0.019360632, 0.00047835405, 0.039074123,
-0.039074093]
islamic terrorism [-0.004287958, 0.0042657405, 2.238946e-05, -0.028740644,
0.028740555]
mass murder [0.021661818, -0.021403424, -0.00025822758, -0.08434576,
0.08434579]
sectarian conflict [-0.02048409, 0.020332336, 0.00015191245, 0.047309637,
-0.047309637]
islamization [-0.035422206, 0.03482026, 0.00060210144, 0.027191758,
-0.027191669]
foreign occupation [-0.018171906, 0.017809838, 0.0003620449, 0.04308176,
-0.0430817]
Comment: Note the literal interpretations "genocide", "massacres" and "mass
murder".

##  Appendix B Examples of Successfully Detected PETs

Below are examples where our algorithm successfully detected the target PET.
The output is as follows. We identify a Target PET along with the sentence it
belongs to. The first set of phrases, ExtractedPhrases, are those retrieved
through Phrases; after using word2vec to further filter phrases according to
our topics, we obtain our QualityPhrases; finally, we display our
RankedPhrases where our candidate PETs appeared in one of the top two
rankings.

Example 1
---
Target PET: psychiatric hospital
Sentence: you may believe that if you have signed yourself voluntarily into a
psychiatric hospital you can sign yourself out and leave when you decide to do
so
ExtractedPhrases: [’you_may’, ’believe_that’, ’if_you’, ’have’, ’signed’,
’yourself’, ’voluntarily’, ’into’, ’a’, ’psychiatric_hospital’, ’you_can’,
’sign’, ’yourself’, ’out’, ’and’, ’leave’, ’when_you’, ’decide’, ’to’, ’do’,
’so’]
QualityPhrases: [’believe_that’, ’if_you’, ’voluntarily’,
’psychiatric_hospital’, ’sign’, ’leave’, ’when_you’, ’decide’]
RankedPhrases: [(’psychiatric hospital’, 7.978855848312378), (’voluntarily’,
4.409763276576996), (’sign’, 2.7386385649442673), (’if you’,
2.3423103243112564), (’believe that’, 1.915013164281845), (’when you’,
1.7038534581661224), (’leave’, 1.6538356691598892), (’decide’,
1.548440158367157)]
Example 2
---
Target PET: armed conflict
Sentence: when this happens something of considerable legal significance does
occur the law of armed conflict begins to govern belligerent relations between
the states
ExtractedPhrases: [’when’, ’this_happens’, ’something’, ’of’, ’considerable’,
’legal_significance’, ’does_occur’, ’the’, ’law’, ’of’, ’armed_conflict’,
’begins’, ’to’, ’govern’, ’belligerent’, ’relations_between’, ’the’, ’states’]
QualityPhrases: [’this_happens’, ’considerable’, ’legal_significance’,
’does_occur’, ’law’, ’armed_conflict’, ’govern’, ’belligerent’,
’relations_between’]
RankedPhrases: [(’legal significance’, 3.8234215676784515), (’armed conflict’,
3.674671307206154), (’this happens’, 3.6536989957094193), (’belligerent’,
2.823164239525795), (’considerable’, 1.5059781521558762), (’govern’,
1.2904964834451675), (’does occur’, 1.1230540722608566), (’relations between’,
0.7008794546127319), (’law’, 0.5298605561256409)]
Example 3
---
Target PET: pro-life
Sentence: however i am also a person who respects life in all of its forms and
so i could also qualify as a pro-life person
ExtractedPhrases: [’however_i_am’, ’also’, ’a’, ’person_who’, ’respects’,
’life’, ’in’, ’all’, ’of’, ’its_forms’, ’and’, ’so’, ’i_could’, ’also’,
’qualify_as’, ’a’, ’pro-life’, ’person’]
QualityPhrases: [’person_who’, ’life’, ’its_forms’, ’qualify_as’, ’pro-life’,
’person’]
RankedPhrases: [(’pro-life’, 14.923447516746819), (’person’,
4.519588744267821), (’qualify as’, 2.345528486184776), (’life’,
1.7386144306510687), (’its forms’, 1.536714962683618), (’person who’,
1.4028910771012306)]

##  Appendix C Examples of New PETs Found

Since our algorithm works by placing a candidate PET in one of the top two
rankings, we evaluated the results and found that new PETs were found and
correctly placed in top ranking positions. One of the limitations of the
Euphemism Corpus is that it only includes one annotated PET per sentence, our
algorithm shows potential to expand upon the annotations in the corpus to
include the new PETs found. We underline the new PETs in the examples below as
well as provide our interpretations.

Example 1
---
Sentence: or acknowledge real-world trade-offs such as the strong likelihood
of amount of of civilian casualties if aq detainees were treated according to
either geneva convention or uc criminal law standards
RankedPhrases: [(’civilian casualties’, 3.9511645138263702), (’criminal law’,
2.082954853773117), (’trade-offs’, 2.0316174626350403), (’geneva convention’,
1.7544293403625488), (’acknowledge’, 1.5634678304195404), (’detainees were’,
1.2355359494686127), (’treated’, 1.2001541256904602), (’standards’,
0.8081734478473663)]
New PET: civilian casualties
Interpretation: the unintended deaths of civilians
Example 2
---
Sentence: pelosi says she was briefed by bush administration officials on the
legal justification for using waterboarding but that they never followed
through on promises to inform her when they actually began using enhanced
interrogation techniques
RankedPhrases: [(’using waterboarding’, 6.236076384782791), (’enhanced
interrogation techniques’, 3.640248477458954), (’she was’,
1.1687388718128204), (’legal justification’, 1.1285315454006195), (’when
they’, 0.9696991741657257)]
New PET: using waterboarding
Interpretation: a form of torture where a person is strapped down to a board
and water is poured over their face in a way that is similar to drowning
Example 3
---
Sentence: religious people often complain that secular therapists see their
faith as a problem or a symptom rather than as a conviction to be respected
and incorporated into the therapeutic dialogue a concern that is especially
pronounced among the elderly and twentysomethings
RankedPhrases: [(’secular therapists’, 1.9648141264915466), (’especially
pronounced’, 1.7061323672533035), (’among the elderly’, 1.6529535502195358),
(’their faith’, 1.3943422138690948), (’rather than’, 1.2891167849302292),
(’concern’, 1.2376690953969955), (’religious people’, 0.9915256798267365),
(’symptom’, 0.8965674340724945), (’therapeutic’, 0.8766119182109833),
(’twentysomethings’, 0.8552953451871872), (’be respected’,
0.5095183551311493), (’conviction’, 0.48858143389225006), (’dialogue’,
0.3565850257873535)]
New PET: secular therapists
Interpretation: a non-religious therapist who uses science based therapy
methods

##  Appendix D Examples of Failed Target PET Detection

The following examples show instances where our algorithm failed to correctly
detect the target PET. We include examples showing sentences in which our MWE
extraction method failed to initially recognize a PET as a phrase, and other
examples showing where different words, such as action words, had a higher
ranking.

Example 1
---
Target PET: comfort women
Sentence: and what about the’ comfort women industry in israel that uses
slavic women as sex slaves
RankedPhrases: [(’sex slaves’, 4.3283873945474625), (’slavic’,
3.69672554731369), (’women’, 1.4681523442268372), (’israel’,
1.3728241324424744), (’comfort’, 1.0837920159101486), (’industry’,
0.8285560309886932)]
Failure: The Target PET ’comfort women’ was never identified as a MWE, and
thus could not be detected. Additionally, polarized non-euphemisms like "sex
slaves" are ranked higher as well as neutral candidates such as "slavic" or
"women". This is likely the result of highly polarized alternatives that
produce a high score.
Example 2
---
Target PET: correctional facility
Sentence: very few correctional facilities have formal vocational education
programs that provide offenders with marketable skills and assistance in
employment planning
RankedPhrases:[(’offenders’, 3.9866801872849464), (’vocational education
programs’, 2.453631855547428), (’very few’, 2.0981270894408226),
(’correctional facilities’, 1.8522954508662224), (’marketable skills’,
1.2003385424613953), (’assistance’, 0.7983754873275757), (’employment’,
0.5764055326581001), (’formal’, 0.4696378782391548)]
Failure: Here, again the Target PET was identified as a phase however the
shift in sentiment was greater for the other phrases in the sentence and thus
it was not ranked in one of the top two spots.
Example 3
---
Target PET: pro-life
Sentence: finally i think many pro-life people are politically naive and are
too willing to accept empty promises
RankedPhrases: [(’politically naive’, 8.384997591376305), (’empty promises’,
4.581491872668266), (’pro-life’, 4.001500993967056), (’people’,
3.438477225601673), (’i think’, 1.7039387673139572)]
Failure: We count this example as a failure as our Target PET is in third
place; however, we believe both of the top two candidates to be PETs.
Interpretation:  politically naive: someone who has little knowledge and/or
experience with politics and empty promises: promises made that are never
intended to be carried out
Example 4
---
Target PET: expecting
Sentence: i had stopped searching while we were expecting our second child
because we were unable to travel if called upon to candidate
RankedPhrases: [(’unable to travel’, 7.015634283423424), (’searching’,
1.7277799248695374), (’second child’, 1.598520651459694), (’candidate’,
0.5451297163963318)]
Failure: The target PET is not a phrase candidate because it was incorrectly
filtered out at the topic filtering stage. This is likely the case because
"expecting" is an otherwise common word.

##  Appendix E New Applications

Below are a few examples where our algorithm shows promise for new
applications. We test our algorithm on sentences that are not in our corpus to
see if it is able to detect PETs in unseen data. An example is shown below:

Example 1
---
Sentence: i heard last week at her birthday party that she has a bun in the
oven he whispered as he ate a hot dog bun
RankedPhrases: [(’bun in the oven’, 5.764157593250275), (’hot dog bun’,
3.9777240827679634), (’he whispered’, 3.6007840037345886), (’she has’,
2.2385976165533066), (’party’, 1.9190692454576492), (’he ate’,
1.8731415495276451), (’her birthday’, 1.3221752345561981)]
New PET: bun in the oven
Interpretation: a baby in a belly; a pregnancy

Below, we show an example where our algorithm shows potential in
distinguishing euphemistic versus non-euphemistic usages of the same word.
First, we show the output for a non-euphemistic sentence containing a non-
euphemistic usage of the PET _dismissed_ :

Example 2
---
Sentence: the class is dismissed and we bow to each other expressing our
gratitude for the shared experience
RankedPhrases: [(’shared experience’, 3.9033331400714815), (’bow’,
3.663858987390995), (’each other’, 2.1924624936655164), (’dismissed’,
1.9963299129158258), (’expressing’, 1.848377185408026), (’class’,
0.9816022356972098)]
Interpretation: allowed to leave or disband

Now, we show the output for a sentence containing a euphemistic usage of
dismissed. Note how _dismissed_ is now detected as a euphemism, as well as its
higher sentiment score compared to the previous example.

Example 3
---
Sentence: at nichols college outside worcester massachusetts a non-tenured
professor who questioned the leadership of the college president was summarily
dismissed
RankedPhrases: [(’dismissed’, 5.921802910044789), (’was summarily’,
3.158419349696487), (’worcester massachusetts’, 1.4444764871150255),
(’college’, 1.196013430133462), (’non-tenured’, 1.1229130360297859),
(’president’, 1.0259317518211901), (’leadership’, 1.0157726714387536)]
Interpretation: forced to leave a position; fired


# Safe Imitation Learning on Real-Life Highway Data for Human-like Autonomous
Driving

Flavia Sofia Acerbo1, Mohsen Alirezaei2, Herman Van der Auweraer1, Tong Duy
Son1 1 Siemens Digital Industries Software, 3001 Leuven, Belgium
<EMAIL_ADDRESS>Siemens Industry Software and Services B.V., 5708
JZ Helmond, The Netherlands<EMAIL_ADDRESS>
###### Abstract

This paper presents a safe imitation learning approach for autonomous vehicle
driving, with attention on real-life human driving data and experimental
validation. In order to increase occupant’s acceptance and gain drivers’
trust, the autonomous driving function needs to provide a both safe and
comfortable behavior such as risk-free and naturalistic driving. Our goal is
to obtain such behavior via imitation learning of a planning policy from human
driving data. In particular, we propose to incorporate barrier functions and
smooth spline-based motion parametrization in the training loss function. The
advantage is twofold: improving safety of the learning algorithm, while
reducing the amount of needed training data. Moreover, the behavior is learned
from highway driving data, which is collected consistently by a human driver
and then processed towards a specific driving scenario. For development
validation, a digital twin of the real test vehicle, sensors, and traffic
scenarios are reconstructed toward high-fidelity and physics-based modeling
technologies. These models are imported to simulation tools and co-simulated
with the proposed algorithm for validation and further testing. Finally, we
present experimental results and analyses, and compare with the conventional
imitation learning technique (behavioral cloning) to justify the proposed
development.

##  I Introduction

Recently, comfort appears as a crucial challenge for automotive manufacturers
and suppliers in Advanced Driver Assistance Systems (ADAS) development. The
main objective is the design of ADAS functions that gain user acceptance and
improve driving experience, while still providing an appropriate trade-off
with safety requirements. The challenge includes not only algorithm
developments but also finding appropriate testing and validation technologies.

Since (good) human drivers are capable of operating vehicles in a comfortable
manner, a way to improve autonomous driving control systems would be to
recreate human-like driving policies in them [1]. In this paper, we attempt to
collect human driving data and mimic it with imitation learning. Although
recent studies showed the potential of end-to-end learning [2, 3, 4], mapping
images to controls, it is also known to have many disadvantages, such as
sample inefficiency [5] and the lack of safety guarantees and stability
properties, being prone to catastrophic errors even by low disturbances [6].
On the other hand, mid-to-mid approaches, combining learning and model-based
components can provide interpretability of the framework and the possibilities
to formally consider safety aspects [7, 8, 9].

In the proposed algorithm, we consider two main properties. First, safety
should always be prioritized over comfort. Hence, constraints to avoid hazards
on the passengers are explicitly treated and formally guaranteed. Secondly,
sample efficiency is preferable. In the view of a novel design process of ADAS
and Autonomous Vehicles (AV), involving continuous data collection, learning
and updates throughout the entire life cycle of the product, we recognize the
need of an agile learning, easy to integrate in an infinite development loop.
Accordingly, it is preferable that the designed learning modules are trained
with small amounts of data and in a short amount of time.

Recently, we have proposed [10] a novel methodology for imitation learning
combined with model-based information for training a neural network to output
planned trajectories mimicking demonstrated ones. The methodology proposed to
employ smooth spline-based motion planning in a loss-constrained neural
network, using the convex hull property of B-splines. The approach has shown
potential in increasing safety and sample efficiency of imitation learning
algorithms (e.g. DAgger [11]). In this paper, the proposed safe imitation
learning is validated on real-life data to learn a highway driving planning
policy from humans.

Hence, the designed ADAS policies need to be properly tested to compare the
learned behavior to the human collected one and validate their safety.
Previous works, including ours, validated their algorithms in the same
environment where the data was collected, e.g. generating simulated expert
data from an optimal planner [10, 8, 9]. However, in this case, human driving
data is collected in the real world but, for safety and economical reasons,
testing should be done in simulation. This induces the domain shift problem
[12], i.e. the challenge of transferring a policy learned into the real domain
to a simulated one. Indeed, following the learned trajectories in a simulated
setting can produce a completely different distribution of feedback states,
from vehicle and environment, and impair the performance evaluation of the
algorithms. In this paper, we also demonstrate a testing approach to mitigate
this challenge. It consists in the creation of a digital twin i.e., an
accurate reconstruction of the real-life framework, made of all its
components: vehicle dynamics, traffic scenario and perception.

The paper is organized as follows: Section II provides background on imitation
learning algorithms and on B-splines. Section III formulates the safe
imitation learning methodology. The description of the experimental setting
and the validation results are provided in Section IV.

##  II Background

###  II-A Imitation Learning

Imitation learning aims at learning how to mimic a certain behavior, called
expert policy, defined as $a^{*}_{t}=\pi^{*}(o_{t},\theta^{*})$, i.e. a
mapping between observations and actions. A dataset of expert demonstrations
is available $D^{*}=\\{(o^{*}_{1},a^{*}_{1}),...,(o^{*}_{N},a^{*}_{N})\\}$,
defining the set of visited states as $S^{*}$. Applying supervised learning on
$D^{*}$, a policy $\hat{\pi}$ is learned such that

$\hat{\pi}=\operatorname*{arg\,min}_{\pi\in\mathcal{C}}{\mathcal{L}}(\pi(\theta),\pi^{*}(\theta^{*})),$

where $\mathcal{L}$ is usually represented by the Mean Squared Error between
actions generated by the two policies and $\hat{\pi}$ is chosen to belong
inside a class of functions $\mathcal{C}$, and with $\theta$ parameters. This
is also known as behavioral cloning and the class $C$ is usually represented
by neural networks. The learning is not always accurate, and $\hat{\pi}$ will
present imperfections that will lead the system, when in a closed-loop
setting, to reach states not included in $S^{*}$, for which the policy
behavior becomes unpredictable. This is usually called compounding of errors
problem and it is due to the fact that the statistical i.i.d. assumption
between training and testing data is not valid for sequential predictions.
Common solutions to mitigate the unsafe consequences of this problem require
data augmentation. This can be done online by the expert, as in the DAgger
algorithm [11]. However, this is impossible with a human expert, as in our
case. It has also been proposed an entirely offline learning algorithm
enriched by synthesized perturbations in the training dataset [7], creating
artificial corrective trajectories from artificially perturbed states. The
effect of adding such demonstrations, enhanced by having a small dataset,
could be contamination in the driving pattern that we are trying to learn,
acting as misguiding outliers. Hence, in this paper, both solutions are
excluded. Instead, the integration of constraints and domain-aware information
inside the training loss function of the neural network is proposed, through
barrier function constraints.

###  II-B B-splines parametrization

Splines are piecewise polynomial function that can represent trajectories as
smooth, continuous functions by only a limited number of variables. The points
where the pieces meet are called knots, which are sorted in non-decreasing
order and not necessarily distinct. A spline can be expressed as a linear
combination of B-splines basis function $B_{i}(\tau)$ with spline coefficients
$\boldsymbol{\alpha}_{i}$ as

$s(\tau)=\sum_{i=1}^{n}\boldsymbol{\alpha}_{i}B_{i}(\tau).$ (1)

The number of coefficients $n$ depends on the basis function degree $d$ and
the number of spline knots $m$, that is, $n=m-d-1$. The coefficients
$\boldsymbol{\alpha}_{i}$ can be points in the $x,y$ plane and are also called
control points. A B-spline is always contained in the convex hull of its
control points. Consequently, a constraint on the values of the
$\boldsymbol{\alpha}_{i}$ coefficients would imply an equivalent constraint on
the amplitude of the spline function $s(\tau)$ for any $\tau$. Apart from
imposing position constraints over the entire time horizon, we could also
impose other kinematic limits such as velocity, acceleration and jerk
constraints in the similar manner.

##  III Safe Imitation Learning

In our proposed safe imitation learning approach [10], the policy network
makes use of B-splines coefficients, i.e. points in 2D plane, to parametrize
the output trajectory and enhances the safety of the learned policy with
domain-aware information, in the form of barrier-function constraints added to
the imitation loss, keeping its trajectories contained in a safe convex hull.
Considering an imitation loss as $\mathcal{L}=\left\lVert
a_{t}-a_{t}^{*}\right\rVert^{2}_{2}$, the constrained loss can be made by
adding a barrier function, yielding:
$\mathcal{L}_{safe}=\mathcal{L}+I(o_{t},a_{t},a_{t}^{*})$. The advantages of
using B-splines lie mainly in the convex hull property. Secondarily, using
coefficients as outputs allows a forced smooth trajectory, imposition of
constraints at each time instant just by limiting the coefficient values and
few output nodes. In this way, it is more scalable for longer horizons.

Eventually, our proposed learning method (as shown in Figure 1) can generate a
driving imitation policy to avoid unsafe behaviors due to the compounding
errors problem when tested in a closed-loop framework. The algorithms were
trained and validated in simulators only.

Figure 1: Safe Imitation Learning: the policy network outputs coefficients of
B-splines (points in the 2D plane), and the loss function used during training
is enhanced with barrier function constraints on the coefficient values,
constraining the trajectory inside a safe convex hull.

##  IV Experimental Validation on highway driving

The presented safe imitation learning algorithm is implemented and evaluated
on human driving data in a highway scenario. Firstly, the data has been
collected and processed, to be ready to use for imitation learning algorithms.
Then, the planning and control system, including a policy network trained with
safe imitation learning is implemented. Finally the designed system is
validated with respect to its similarity with human driving data, and compared
with a system trained with behavioral cloning, showing both ability in
replicating the human driving pattern and in staying within the safety
boundaries.

###  IV-A Data Collection and Processing

A Toyota Prius is equipped with different ADAS sensors and a user is asked to
drive it naturally on N74/N715 roads (see Figure 2).

Figure 2: On the left, the road where the data was collected. The road is 29.2
kilometers long and it took around 30min to drive along it. On the right, a
snapshot of how the road structure is like: highway-like, with low curvature
and clear lane markers.

####  IV-A1 Vehicle, Sensors and Data Logging Architecture

To collect the data, a Toyota Prius (Figure 3) has been used, retroﬁtted with
the necessary equipment for data collection. Among others, it encompasses a
Rapid Control Prototyping (RCP) system, ROS as a middleware for data logging,
a modem for wireless communications (according to then ITS G5 standard), and a
human-machine interface (HMI) computer platform to inform the driver via a
dashboard screen. In particular, the Prius Carlab is equipped with a MobilEye
front camera, normal GPS, accurate positioning OxTS system and 4 radars, of
which two of them are fitted to the front and the other ones to the rear of
the vehicle. Furthermore, the Carlab is equipped with an Ibeo laser scanning
system. The latter consists of 6 laser scanners, giving a 360 deg view around
the vehicle with an angular resolution of 0.8 deg in the vertically, 0.125 deg
horizontally, distance resolution of 4 cm, distance repetition accuracy of 10
cm, object classification (including velocity and direction information).
Finally, it uses XSens GPS and IMU to register ego vehicle position, motion
and dynamics via the vehicle CAN.

(a) The vehicle

(b) Close-up of the vehicle instrumentation in the trunk

(c) Sensors field of view

Figure 3: Data Collection: vehicle, instrumentation and sensors

####  IV-A2 Data Processing

Firstly, a filtering is done by automatically removing all lane changes
present in the data. Then, for each timestamp, the following entries are
obtained:

  * •

X-Y coordinates: global coordinates in a reference frame with an origin that
is zeroed every 5 minutes.

  * •

Ego vehicle longitudinal speed.

  * •

Coefficients of the fitting polynomials of the left and right lane boundaries,
as in $y=c_{0}+c_{1}x+c_{2}x^{2}+c_{3}x^{3}$, where $y$ and $x$ are the
lateral and longitudinal coordinates in the local vehicle reference frame.

  * •

Leading vehicle coordinates: x-y position of the leading vehicle in the local
vehicle reference frame, obtained from the radar.

An example of recorded trajectory extracted from these entries can be seen in
Figure 4.

The task of autonomously driving on a defined lane on the highway involves two
types of control: lateral control, which computes the steering wheel signal
necessary to keep the vehicle on the lane, and longitudinal control, which
computes the acceleration (throttle or brake) signal. Highway longitudinal
control is traditionally divided in velocity control, i.e. maintaining a
desired speed without a leading vehicle in front, and spacing control, i.e.
maintaining a desired distance from the leading vehicle. This driving control
can be implicitly linked to human driving behavior style, although the desired
speed and distance are not explicitly given by the driver. It is checked if
these tasks are equally represented in the data: there is a clear prevalence
of spacing control, i.e. most often, there is a leading vehicle that the
driver has to keep the distance to. For these reasons, we only consider
spacing control imitation learning and filter out the samples with no leading
vehicle.

$-2$$0$$2$$0$$20$$40$$60$y [m]x [m]Future Trajectory of the ego vehicleLane
LinesDetected leading vehicle

Figure 4: Example of trajectory states and sensors information that can be
extracted from the recorded human driving data.

The leading vehicle speed can be computed by numerically differentiating two
of its subsequent relative x-positions. However, we notice that many outliers
are present, showing unfeasible leading vehicle speed values. By looking at
the camera images recorded during those moments, these are happening in cut-
in/cut-out/merging scenarios. Indeed, when a vehicle cuts in front of the ego
vehicle, the radar will detect, in a single timestamp, an obstacle in front
which is much closer than the previous one. This results in the erratic
computation of an unfeasible deceleration of the leading vehicle.

After filtering such scenarios, the dataset is divided in different _car-
following maneuvers_ , in which the ego vehicle follows another one for the
entire maneuver. Hence, the selected ADAS task can be seen as a combination of
lane keeping and adaptive cruise control. From these maneuvers, experience
tuples made of (current state, future trajectory) are created, where the
future trajectory is obtained from the 20 samples ahead.

###  IV-B Planning and Control System

We propose to learn a planning policy, via our safe imitation learning
approach, to perform the task of highway driving extracted in Section IV-A2.
The policy takes the form of a shallow neural network, with manually chosen
features. The policy, to plan a proper trajectory, shall take into account the
direction of the road ahead and the detected distance and relative speed to
the vehicle in front. Therefore, the inputs are chosen as: the coefficients of
the second-order polynomial fitting the left and right lane lines, in the
local reference frame of the ego vehicle, $\boldsymbol{c}_{l}$ and
$\boldsymbol{c}_{r}$; the ego longitudinal vehicle speed, $v_{x}$; the leading
vehicle speed, $v_{lead}$ and the distance from the leading vehicle,
$d_{lead}$.

The network then outputs the planned trajectory
$\boldsymbol{s}(t)=(x(t),y(t))$, for 20s ahead, which is then tracked by a
low-level control. This is realized by a PID controller for the longitudinal
dynamics and a Pure Pursuit for the lateral dynamics. The computed steering
angle and acceleration signal, converted into throttle and brake controls, are
given to the vehicle plant. Specifically, a cubic spline ($d=3$) with 8 knots
is chosen (open uniform normalized knot vector $[0,0,0,0,1,1,1,1]$), resulting
in $n=4$ coefficients, each of them being a point in $\mathbb{R}^{2}$. Since
the trajectory always starts from the origin of the current local reference
frame, the coefficient multiplying the first B-spline is always null and can
be neglected. Therefore, the total number of network outputs is 6, i.e.
$\left(\alpha_{xi},\alpha_{yi}\right)_{i=1,..,3}$, regardless of the length of
the planned trajectory. The barrier functions can be written as a linear
combination of other functions $\sigma(z)$, in similar form to softplus
functions: $\sigma(z)=ln(1+e^{z})$, limiting the values of the coefficients
according to collision avoidance and lane boundaries constraints and, thanks
to the convex hull property, the overall trajectory will be contained inside
the safe area.

Here, the barrier function is selected as following:

$\displaystyle I(o_{t},a_{t})$
$\displaystyle=K\sum_{i=1}^{3}{\sigma\left(\alpha_{yi}-\left(c_{0l}+c_{1l}\alpha_{xi}+c_{2l}\alpha_{xi}^{2}-\frac{T}{2}\right)\right)}$
(2)
$\displaystyle+K\sum_{i=1}^{3}{\sigma\left(-\alpha_{yi}+\left(c_{0r}+c_{1r}\alpha_{xi}+c_{2r}\alpha_{xi}^{2}+\frac{T}{2}\right)\right)}$
(3)
$\displaystyle+K\sum_{i=1}^{3}{\sigma\left(\alpha_{xi}-d_{lead}(i)\right)},$
(4)

where $K=1000$. For the lane boundaries constraints [(2), (3)]: $T$ represents
the track, i.e. the width, of the vehicle and $c_{il}$ and $c_{ir}$ are
respectively the coefficients of the polynomials fitting the left and right
lane. For the collision constraints (4): the leading vehicle is modeled with
constant speed, such that $d_{lead}(i)=d_{lead}(t=0)+v_{lead}t(i)$, where
$t(i)$ is the time instant corresponding to the $i^{th}$ control point,
computed from the knot vector.

###  IV-C Closed-Loop Validation Methodology

Also on-policy performance should be tested, i.e. letting the network act on
the actual system, in closed-loop, and compare the resulting states with the
ones recorded during human driving.

This is a challenge that can be addressed through the use of a digital twin
[13], i.e. a simulated replica of the scenario and platform where the human
driving data is collected.

The digital twin that fits our use case is made of a virtual driving scenario,
which will give the feedback inputs to the planner, and of a high-fidelity
vehicle model, so that the provided control actions can generate realistic
vehicle dynamics behaviors. This combination, one focused on the vehicle
dynamics and one focused on the environment outside the vehicle, yields the
basis for the virtual validation of the learned planner [14].

####  IV-C1 Leave-One-Out and Scenario Creation in Simcenter Prescan©

From the maneuvers extracted from the dataset, a sample maneuver can be
extracted and excluded from the training data. For example, 50 samples of a 10
seconds car-following maneuver at 115km/h are left out. From the recorded
dataset, at each timestamp, the position of the lane boundaries and the speed
of the leading vehicle are known, as well as the initial states. This
information is enough to easily recreate the scenario. To do this, Simcenter
Prescan is employed: a traffic simulator which is also able to model ADAS
sensors at different fidelity levels.

Figure 5: Recreation of the traffic scenario in Simcenter Prescan©

####  IV-C2 Full Vehicle Modelling in Simcenter Amesim©

A vehicle dynamics model for ADAS validation is not trivial to obtain: while
Automotive OEMs have access to full or high-fidelity vehicle models, having
such information is usually a significant challenge for software suppliers who
need to develop and validate their ADAS solutions. In this project, a test-
driven full vehicle model identification is proposed. The approach maximizes
the use of vehicle driving data, while limiting as much as possible the need
for bench tests. The identified vehicle characteristics are then easily
plugged in a 15 degree-of-freedom model in Simcenter Amesim.

###  IV-D Validation Results

According to the presented strategy, the proposed planning and control system
is validated. Two planning policies are trained: one using our safe imitation
learning approach and one using standard behavioral cloning, as baseline. We
evaluate the two resulting systems for:

  * •

Human-like performance: we replicate a collected driving scenario in
simulation and compare the closed-loop simulated states with human real-life
ones.

  * •

Safety performance: we define multiple artificial test scenarios, in
simulation and with varying external parameters, and check the ability of the
two controllers to respect safety constraints.

##### Human-like performance

As shown in Figure 6, with behavioral cloning the compounding error problem
causes the vehicle to drift off the lane after a few seconds. This makes it
impossible to check the human-like performance of the policy. The descending
step in the lower right diagram (_Distance to the leading vehicle_) is due to
sensors not detecting anymore the leading vehicle. Conversely, by training the
network with the safe imitation learning method, the vehicle can respect the
lane boundaries also when deviating from the training features distribution.

From the figure, it is interesting to see how the planning network has an
_averaging effect_ on the capture of the driving pattern: indeed, its main
preferred behavior is replicated, but at the cost of losing the higher-
frequency dynamics characterizing the collected human driving. This is
expected, considering that the imitation loss considers a _mean_ of $L_{2}$
errors between different splines coefficients.

$0$$2$$4$$6$$8$$10$$0$$100$$200$$300$$400$X(m)X global coordinate

$0$$2$$4$$6$$8$$10$$0$$0.5$$1$$1.5$Y(m)Y global coordinate

$0$$2$$4$$6$$8$$10$$31$$31.5$$32$$32.5$$v_{x}$ (m/s)Longitudinal ego vehicle
speed

$0$$2$$4$$6$$8$$10$$57$$58$$59$$60$$61$time (s)$d_{lead}$ [m]Distance to the
leading vehicleBehavioral CloningSafe Imitation LearningHuman Driving

Figure 6: Human-like Perfomance Validation Results

##### Safety performance

Additionally, the two controllers are tested on other artificially created
scenarios for car following. A curved track of 500m radius and 1km length is
designed in Prescan, and a leading vehicle is added in front of the simulated
ego vehicle, with a constant random speed between 25 and 32 m/s. Both
algorithms are tested at the same time in closed-loop, and a flag is raised
whenever a safety constraint is violated, i.e. when the vehicle exits its own
lane or when it gets too close to the vehicle in front (time to collision
$<=1s$). Then, it is computed the percentage of completed path before the flag
is raised, and the results can be seen in Figure 7. In general, safe imitation
learning performs better, being able to complete paths, whereas behavioral
cloning could not. In some scenarios (4-6-7), where safe imitation learning
violated safety constraints, it can be seen that the leading vehicle was close
to its possible minimum speed (25m/s), hence provided a smaller time-to-
collision since the beginning of the path.

$1$$2$$3$$4$$5$$6$$7$$8$$9$$10$$0$$0.2$$0.4$$0.6$$0.8$$1$Test Scenario #Path
Completed [%]Safety PerformanceSafe Imitation LearningBehavioral Cloning
Figure 7: Safety Perfomance Validation Results: percentage of completed path
before a safety constraint (lane boundaries or collision) is violated

##  V Conclusions

This paper presented the validation on real-life data of a safe imitation
learning approach in the design of a human-like autonomous driving system for
highway driving, during maneuvers concerning lane keeping and car following.
For safety, sample efficiency and easy simulation-based testing, the system
has been designed with a motion planning neural network policy followed by a
model-based control. The planning policy has been learned to mimic the human
driving patterns but, based on previous studies, has also been informed about
safety constraints through barrier functions in the training loss, limiting
the control points of a B-spline convex hull. The use of real-life data
required a fine data processing, analytically discovering and removing the
scenarios that acted as outliers in the learning process, e.g. cut-ins/cut-
outs/merging. The system has been tested using a digital twin of a recorded
real life scenario, where it was possible to compare the simulated vehicle
states, controlled by our system, to the collected ones, controlled by the
human driver. The results show that, in the tested scenario, the policy
generates an averaged human-like driving behavior, presenting a similar
pattern when considering slow-changing dynamics. Moreover, it has been shown
that, without any possibility of online learning with interactive expert, our
safe imitation learning is able to stay within safety limit, whereas a common
behavioral cloning algorithm could not.

## Acknowledgment

This work has been conducted within the ID2CON (Integrated IDentification for
CONtrol) and the H2020-EU FOCETA (Foundations for Continuous Engineering of
Trustworthy Autonomy) projects. The authors would also like to thank Irfan M.
Badshah, Ludovico Ruga and Lorenzo Lugo, from Siemens, for their supports in
vehicle setting, data collection and full vehicle modelling.

## References

  * [1] M. Elbanhawi, M. Simic, and R. Jazar, “In the passenger seat: Investigating ride comfort measures in autonomous cars,” _IEEE Intelligent Transportation Systems Magazine_ , vol. 7, no. 3, pp. 4–17, 2015.
  * [2] M. Bojarski, D. Del Testa, D. Dworakowski, B. Firner, B. Flepp, P. Goyal, L. D. Jackel, M. Monfort, U. Muller, J. Zhang, X. Zhang, J. Zhao, and K. Zieba, “End to end learning for self-driving cars,” 2016.
  * [3] A. Amini, I. Gilitschenski, J. Phillips, J. Moseyko, R. Banerjee, S. Karaman, and D. Rus, “Learning robust control policies for end-to-end autonomous driving from data-driven simulation,” _IEEE Robotics and Automation Letters_ , vol. 5, no. 2, pp. 1143–1150, 2020.
  * [4] Y. Xiao, F. Codevilla, A. Gurram, O. Urfalioglu, and A. M. Lopez, “Multimodal end-to-end autonomous driving,” _IEEE Transactions on Intelligent Transportation Systems_ , p. 1–11, 2020. [Online]. Available: http://dx.doi.org/10.1109/TITS.2020.3013234
  * [5] S. Shwartz and A. Shashua, “On the sample complexity of end-to-end training vs. semantic abstraction training,” _CoRR_ , 2016.
  * [6] H. Wu and W. Ruan, “Adversarial driving: Attacking end-to-end autonomous driving systems,” 2021.
  * [7] M. Bansal, A. Krizhevsky, and A. Ogale, “Chauffeurnet: Learning to drive by imitating the best and synthesizing the worst,” 2018.
  * [8] H. Pulver, F. Eiras, L. Carozza, M. Hawasly, S. Albrecht, and S. Ramamoorthy, “Pilot: Efficient planning by imitation learning and optimisation for safe autonomous driving,” 2021.
  * [9] J. Chen, B. Yuan, and M. Tomizuka, “Deep imitation learning for autonomous driving in generic urban scenarios with enhanced safety,” 2019.
  * [10] F. Acerbo, H. Van der Auweraer, and S. Tong, “Safe and computational efficient imitation learning for autonomous vehicle driving,” in _2020 American Control Conference (ACC)_ , 2020, pp. 647–652.
  * [11] S. Ross, G. Gordon, and D. Bagnell, “A reduction of imitation learning and structured prediction to no-regret online learning,” in _Proceedings of the fourteenth international conference on artificial intelligence and statistics_ , 2011, pp. 627–635.
  * [12] K. Kelchtermans and T. Tuytelaars, “Doshico challenge: Domain shift in control prediction,” 2018.
  * [13] D. Hartmann and H. Van der Auweraer, “Digital twins,” vol. abs/2001.09747, 2020.
  * [14] S. Tong, A. Bhave, and H. Van der Auweraer, “Simulation-based testing framework for autonomous driving development,” in _2019 IEEE International Conference on Mechatronics (ICM)_ , vol. 1, 2019, pp. 576–583.