# Incorporating Physical Priors into Weakly-Supervised Anomaly Detection

Chi Lung Cheng<EMAIL_ADDRESS>Department of Physics, University of
Wisconsin, Madison, WI 53706, USA  Physics Division, Lawrence Berkeley
National Laboratory, Berkeley, CA 94720, USA  Gurpreet Singh
<EMAIL_ADDRESS>Physics Division, Lawrence Berkeley National Laboratory,
Berkeley, CA 94720, USA  Benjamin Nachman<EMAIL_ADDRESS>Physics Division,
Lawrence Berkeley National Laboratory, Berkeley, CA 94720, USA  Berkeley
Institute for Data Science, University of California, Berkeley, CA 94720, USA

###### Abstract

We propose a new machine-learning-based anomaly detection strategy for
comparing data with a background-only reference (a form of weak supervision).
The sensitivity of previous strategies degrades significantly when the signal
is too rare or there are many unhelpful features. Our Prior-Assisted Weak
Supervision (PAWS) method incorporates information from a class of signal
models in order to significantly enhance the search sensitivity of weakly
supervised approaches. As long as the true signal is in the pre-specified
class, PAWS matches the sensitivity of a dedicated, fully supervised method
without specifying the exact parameters ahead of time. On the benchmark LHC
Olympics anomaly detection dataset, our mix of semi-supervised and weakly
supervised learning is able to extend the sensitivity over previous methods by
a factor of 10 in cross section. Furthermore, if we add irrelevant (noise)
dimensions to the inputs, classical methods degrade by another factor of 10 in
cross section while PAWS remains insensitive to noise. This new approach could
be applied in a number of scenarios and pushes the frontier of sensitivity
between completely model-agnostic approaches and fully model-specific
searches.

##  I Introduction

Even if it were possible to enumerate all potential new physics scenarios
discoverable at the Large Hadron Collider (LHC), it would be impossible to
conduct a search for each one. Therefore, physics model agnostic anomaly
detection searches are an essential complement to the traditional physics
model specific searches. Modern machine learning has ignited a rapidly growing
literature on innovative anomaly detection strategies that are projected to
qualitatively extend the sensitivity of the LHC search program
Kasieczka:2021xcg ; Aarrestad:2021oeb ; Karagiorgi:2022qnh . A number of these
strategies have already been applied to data collaboration2020dijet ;
ATLAS:2023ixc ; ATLAS:2023azi ; CMS-PAS-EXO-22-026 .

One powerful set of anomaly detection strategies are based on classifiers
trained to distinguish data from a background-only reference Collins:2018epr ;
Collins:2019jip ; DAgnolo:2018cun ; DAgnolo:2019vbw ; Andreassen:2020nkr ;
Nachman:2020lpy ; 1815227 ; Hallin:2021wme ; dAgnolo:2021aun ; Golling:2022nkl
; Hallin:2022eoq ; Golling:2023yjq ; 1815227 ; Sengupta:2023xqy . If the
background only reference is statistically identical to the background in the
data, then the classifier will be able to pick up on any non-background
component in the data. This weakly supervised strategy is asymptotically
optimal Nachman:2020lpy  and various methods are distinguished by how they
construct the background reference, how they construct the classifier, and how
they determine the statistical compatibility of the data with the reference.

Until recently, most studies of the background reference strategy used low-
dimensional, high-level features. Additional features can degrade the
sensitivity of the search, especially when the new features have the same
spectrum for signal and background Finke:2023ltw ; Freytsis:2023cjr . The
authors of Ref. Buhmann:2023acn  have shown how to employ the background
reference strategy to high-dimensional, low-level features, but the
sensitivity degrades rapidly with low signal cross sections. Our hypothesis is
that existing methods use classifiers that are too flexible. In other words,
the extra machine learning network capacity can be deployed to model
unphysical features in the data, reducing sensitivity. Our solution is to
incorporate physical priors to reduce the space of functions.

To this end, we introduce a Prior-Assisted Weak Supervision (PAWS) strategy
for anomaly detection. Our approach starts with specifying a class of signal
models, parameterized by $\theta\in\mathbb{R}^{N}$. These parameters could be
masses, couplings, etc. As a first step, we use simulated signal samples to
pre-train neural network classifiers parameterized by $\theta$ Cranmer:2015bka
; Baldi:2016fzo  using features $x\in\mathbb{R}^{M}$. Then, we use the same
classifier to distinguish the data from the background-only reference. In the
second classification step, we freeze all of the network weights and promote
$\theta$ to be learnable. Whereas previous methods scale poorly with $M$, PAWS
is independent of $M$ and degrades instead with $N$. The tradeoff is that we
assume the anomaly is in our signal class (and can be reasonably simulated).
This is a strong constraint, but this still gives us access to large classes
of physics models that cannot be searched for individually.

There have been a number of studies that have investigated related aspects of
our approach. References Park:2020pak ; Khosa:2020qrz  use a set of signal
models to indirectly improve the performance to out-of-distribution signals. A
pair of recent studies showed that less expressive classifiers (boosted
decision trees) can help mitigate degradation due to irrelevant features
Finke:2023ltw ; Freytsis:2023cjr . References Beauchesne:2023vie ;
Heinrich:2024sbg ; Mikuni:2024qsr  use transfer learning from models trained
on similar tasks to inject some physics information through the initialization
of the classifier. Instead of pre-training on signal models, Ref. Das:2023bcj
recycles a background prediction to enhance the sensitivity to small signals.
Such an approach is complementary to our proposal. Another way of viewing our
approach is as a variation of neural simulation-based inference Cranmer_2020 ,
where the signal fraction $\mu$ and $\theta$ are inferred from the data. In
the anomaly detection case, we are (mostly) interested in knowing if the
classifier learns anything non-trivial, but a byproduct of the approach is
that there is also an estimate of the physics parameters.

This paper is organized as follows. Section II introduces the Large Hadron
Collider Olympics (LHCO) 2020 benchmark dataset Kasieczka:2021xcg ;
LHCOlympics , and its extension, that we use to develop PAWS. The machine
learning strategy is described in Sec. III. Numerical results are presented in
Sec. IV and the paper ends with conclusions and outlook in Sec. V.

##  II Dataset

We build on the LHC Olympics (LHCO) R&D dataset Kasieczka:2021xcg ;
LHCOlympics  and the extended background dataset from Ref. Hallin:2021wme  for
our studies. The original dataset consists of generic quark-gluon scattering
processes to produce dijet events as the Standard Model background and
$W^{\prime}\rightarrow XY$ with $X,Y\rightarrow q\bar{q}$ as signal. Both
processes are simulated using Pythia 8.219 Sjostrand:2006za ;
Sjostrand:2014zea  and Delphes 3.4.1 deFavereau:2013fsa ; Mertens:2015kba .
When $m_{W^{\prime}}\gg m_{X},m_{Y}$, the $X$ and $Y$ particle decay products
will be mostly captured inside a single jet each and their invariant mass will
be close to $m_{W^{\prime}}$. The original dataset had $m_{W^{\prime}}$ = 3.5
TeV, $m_{X}$ = 500 GeV, and $m_{Y}$ = 100 GeV. To train our parameterized
classifier and to demonstrate the performance on a variety of signal models,
we use the same simulator settings to generate an entire grid with $50$ GeV
$<m_{X},m_{Y}<600$ GeV in increments of 50 GeV. In addition to the two-prong
$X,Y\rightarrow qq$ decay, we also include the three-prong $Y\rightarrow qqq$
decay from Ref. LHCOlympics  to increase the dimensionality of our parameter
space. In total, each decay features 144 signal models, each with about 100k
events. The parameters of our signal model set are
$\theta=\\{m_{X},m_{Y},\alpha\equiv\text{BR}(Y\rightarrow qqq)\\}$.

The reconstructed particles of each event are clustered into jets using the
anti-$k_{T}$ algorithm Cacciari:2005hq ; Cacciari:2011ma ; Cacciari:2008gp
with $R=1.0$. All events are required to satisfy $p_{T}>$ 1.2 TeV. The two
highest $p_{T}$ jets $J_{1}$ and $J_{2}$ are sorted such that
$m_{J_{1}}>m_{J_{2}}$. Each event is characterized by the same six features
that have been widely used in LHCO studies:
$x=\\{m_{J_{1}},m_{J_{2}},\tau_{21}^{J_{1}},\tau_{21}^{J_{2}},\tau_{32}^{J_{1}},\tau_{32}^{J_{2}}\\}$,
where the $n$-subjettiness ratios are defined as
$\tau_{ij}\equiv\tau_{i}/\tau_{j}$ Thaler:2010tr ; Thaler:2011gf . These are
sensitive to the two- ($\tau_{21}$) or three- ($\tau_{32}$) prong nature of
the jets.

The signal is resonant in $m_{JJ}$ and a full analysis would scan over
$m_{JJ}$ to form signal regions and sideband regions. As in many previous
studies, for simplicity, we focus on the one signal region
$m_{JJ}\in[3.3,3.7]$ TeV that contains the signal. The sideband
$m\not\in[3.3,3.7]$ TeV would be used to both estimate the background-only
distribution of features $p_{\text{B}}(x)$ for training classifiers and to
estimate the total number of background events after applying a threshold in
the classifier. Previous studies have shown that deep learning is very
effective for estimating $p_{\text{B}}$ from data (e.g. Ref. Hallin:2021wme )
and so we assume it is already known; instead, we focus on how to use samples
from $p_{\text{B}}$ to achieve signal sensitivity. Future experimental studies
would synthesize all of these components into a full analysis. In practice,
our reference dataset is statistically identical to and has the same
size111Previous studies have shown that there may be advantages to
oversampling the reference Hallin:2021wme  \- this could be combined with our
approach in future studies. as the background in our actual ‘data’.

##  III Methods

Given features $x$ and parameters $\theta$, we start with a fully supervised
parameterized classifier $f_{\text{FS}}(x,\theta)$ trained with the binary
cross entropy (BCE) loss:

$\displaystyle-\sum_{(x,\theta)\in
B\times\Theta}\log(f_{\text{FS}}(x,\theta))-\sum_{(x,\theta)\in
S}\log(1-f_{\text{FS}}(x,\theta))\,,$ (1)

where $B$ are background events specified by their features and $S$ are signal
events specified by their features and their parameters. The signal dataset is
generated such that every event uses parameters in the set $\Theta$ and then
$X_{i}\sim p_{\text{S}}(\theta_{i})$ for $\theta_{i}\in\Theta$. The notation
$B\times\Theta$ in Eq. 1 represents the fact that the $\theta$ values used for
the background sample are independent from the features and distributed in the
same way as for the signal. This means that $\theta$ itself is not useful for
classification.

Asymptotically, the optimal $f_{\text{FS}}$ from Eq. 1 will approximate

$\displaystyle f_{\text{FS}}^{*}(x)$
$\displaystyle\approx\frac{p_{\text{S}}(x|\theta)p(\theta)}{p_{\text{S}}(x|\theta)p(\theta)+p_{\text{B}}(x)p(\theta)}$
(2)
$\displaystyle=\frac{p_{\text{S}}(x|\theta)}{p_{\text{S}}(x|\theta)+p_{\text{B}}(x)}\,.$
(3)

In particular, we can extract the likelihood ratio from the classifier:
$p_{\text{S}}(x|\theta)/p_{\text{B}}(x)\approx
f^{*}_{\text{FS}}(x)/(1-f^{*}_{\text{FS}}(x))\equiv\Lambda_{\text{FS}}(x|\theta)$.

Our weakly supervised classifiers are also trained using the BCE loss
Metodiev:2017vrx ; Collins:2018epr ; Collins:2019jip :

$\displaystyle-\sum_{(x)\in D}\log(f_{\text{WS}}(x))-\sum_{(x)\in
R}\log(1-f_{\text{WS}}(x))\,,$ (4)

where $D$ is the data with $p_{\text{D}}(x)=\mu
p_{S}(x|\theta)+(1-\mu)p_{B}(x)$ and $R$ is the reference with $p_{R}=p_{B}$.
Asymptotically, the optimal $f_{\text{WS}}$ from Eq. 4 will approximate

$\displaystyle f_{\text{WS}}^{*}(x)$
$\displaystyle\approx\frac{p_{D}(x)}{p_{D}(x)+p_{R}(x)}$ (5)
$\displaystyle=\frac{\mu\frac{p_{S}(x|\theta)}{p_{B}(x)}+(1-\mu)}{\mu\frac{p_{S}(x|\theta)}{p_{B}(x)}+(2-\mu)}\,.$
(6)

Previous approaches parameterized $f_{\text{WS}}$ as a generic neural network.
In our new approach PAWS, we insert $f^{*}_{\text{FS}}$ by replacing the
likelihood ratio in Eq. 6 with $\Lambda_{\text{FS}}$:

$\displaystyle\tilde{f}_{\text{WS}}(x|\mu,\theta)=\frac{\mu\Lambda_{\text{FS}}(x|\theta)+(1-\mu)}{\mu\Lambda_{\text{FS}}(x|\theta)+(2-\mu)}\,,$
(7)

with the functional form of $\Lambda_{\text{FS}}$ fixed and only $\theta$ and
$\mu$ are optimized. When $p(x|\theta)$ factorizes, it is possible to add
additional structure. We utilize this fact to consider both two-prong and
three-prong decays of the $Y$ particle. In particular, we train separate
supervised classifiers for each decay mode and then mix the likelihood ratios
according to the branching fraction $\alpha$.

Each model $f_{\text{FS}}$ and $f_{\text{WS}}$ is parameterized as a three-
layer fully-connected neural network with node configurations of (256, 128,
64). The model $\tilde{f}_{\text{WS}}$ is parameterized in terms of
$f_{\text{FS}}$ according to Eq. 7. The networks are implemented in TensorFlow
tensorflow2015-whitepaper  and optimized using Adam Kingma:2014vow . The data
are divided so that 50% are used for training, 25% are used for validation and
the rest for testing. In weakly supervised training, 50% of the background is
in $D$, 50% is in $R$ and the signals are included in $D$ according to $\mu$
and $\alpha$. In practice, a $k$-fold procedure would be used to ensure that
all data could participate in both training and testing Collins:2018epr ;
Collins:2019jip ; collaboration2020dijet . To improve robustness, we train 10
models with the same input data and different random network initializations.
In the case of $\tilde{f}_{\text{WS}}$, this corresponds to sampling uniformly
in the range of masses with fixed values of $\mu=\exp(-4)$ and
$\alpha=\exp(-1)$. Per event, the scores of the 5 models with the lowest
validation loss are averaged. As there is a local minimum when
$m_{X}\leftrightarrow m_{Y}$, we always check the symmetric point after
training and take the model with the lowest validation loss. We also found
that it was useful to regularize the network with gradient clipping and the
loss function with a penalty for $\theta$ that grows exponentially with the
distance away from physically valid regions. That is, we constrain $m_{X}$ and
$m_{Y}$ to lie within the range $\left[50,600\right]$ GeV, penalizing values
outside the boundary.

Our figure of merit for comparing different approaches is the Significance
Improvement Characteristic defined as
$\text{SIC}=\varepsilon_{S}/\sqrt{\varepsilon_{B}}$, where $\varepsilon_{S}$
and $\varepsilon_{B}$ are the signal and background efficiencies,
respectively. The SIC represents the multiplicative factor by which the signal
significance increases for a selection on the network corresponding to
$\varepsilon_{B}$. In practice, we do not know a priori what selection to
make, so we consider the value of the SIC at a fixed value of
$\varepsilon_{B}=0.1\%$, which we can determine using $R$. This is also more
numerically stable than the maximum of the SIC curve which has often been used
in the past. In order to quantify the uncertainty from the finite data
statistics, we compute the SIC for 10 random realizations of the training
inputs.

##  IV Results

We find that PAWS works well across the entire grid of $m_{X}$ and $m_{Y}$
values. To illustrate the performance in detail, we focus on two benchmark
points: $m_{X}$, $m_{Y}=(300,300)$ GeV and $(100,500)$ GeV, showcasing
scenarios of highly symmetric and asymmetric masses, respectively. The
asymmetric model is also the same as the one in the R&D dataset of the LHCO.
The branching ratio for the three-prong decay of the $Y$ particle is set to
$\alpha=0.5$ and we scan over $\mu$ to show our sensitivity to low levels of
signal.

In contrast to traditional weak supervision with fully flexible neural
networks, for PAWS, the learned parameters are physical and thus directly
interpretable. Figure 1 shows the weakly-supervised loss (Eq. 4) as a function
of $m_{X}$ and $m_{Y}$ for $\alpha$ and $\mu$ fixed to their correct values.
The $\mu$ values were chosen to approximately correspond to just before and
then just about where the classical weakly supervised approach becomes
sensitive. All four plots in Fig. 1 show clear minima at the correct values of
$m_{X}$ and $m_{Y}$. In the asymmetric case, there is also a (local) minimum
for the incorrect point with $m_{X}\leftrightarrow m_{Y}$. The classifier at
this point is still very effective because $m_{J_{1}}$ and $m_{J_{2}}$ have
nearly the same spectra as for the right mass point.

Figure 1: The weakly-supervised loss as a function of $m_{X}$ and $m_{Y}$ for
$\alpha$ and $\mu$ fixed to their correct values. The top (bottom) row shows
$mu=0.1\%$ ($\mu=0.3\%$) and the left (right) column corresponds to
$m_{X}=100$ GeV, $m_{Y}=500$ GeV ($m_{X}=m_{Y}=300$ GeV). The loss has been
transformed by subtracting $\log(2)$ and then multiplying by $10^{5}$.

When the correct mass points are chosen, PAWS performs at the same level as
the fully supervised model. This is shown in Fig. 2. The fact that our model
matches the dedicated fully supervised approach for $S/B\gtrsim 0.1\%$ also is
a validation that the parameterized model is effective. PAWS achieves
sensitivity to signals that are more than a factor of 10 weaker than for the
classical weakly-supervised approach. This sensitivity is reaching the limit –
for a signal fraction of 0.03%, the injected significance is only 0.13 and the
fully supervised SIC is about 20. This means that the fully supervised
approach would enhance the signal to only about 2. Therefore, no method can
achieve useful performance below 0.03%.

The lower panels of Fig. 2 also demonstrate that the physics parameters are
accurate. Due to the subtle differences between the two- and three-prong
models, we are not very sensitive to the exact value of $\alpha$. We also
consistently find that $\mu$ is over-estimated by about a factor of two. This
does not have a large impact on the discovery potential, but it does mean that
precise inference of the cross section may require additional considerations.

In order to show the robustness of our approach to the dimensionality of the
feature space, we add 10 dimensions of pure noise to the input Finke:2023ltw ;
Freytsis:2023cjr . Each of these dimensions is sampled from a standard Normal
distribution for both the signal and the background. Since our pre-trained
models only consider physical information, they effectively learn to ignore
these additional features. As a result, PAWS is unaffected by the noise. In
contrast, the traditional weakly supervised methods are strongly dependent on
the noise and their sensitivity degrades by about a factor of 10 in $\mu$.

Figure 2: Significance improvement at a fixed value of $\varepsilon_{B}=0.1\%$
(top) and predicted parameter values (bottom) with varying amounts of signal
injections for $(m_{X},m_{Y})=(300,300)$ GeV (left) and $(100,500)$ GeV
(right). The shaded uncertainty bands quantify the standard deviation (around
the median) from retraining the respective models with random realizations of
the inputs. The top axis indicates the starting significance of the signal
($S/\sqrt{B}$). The dashed lines from the top plots include scenarios where 10
dimensions of pure noise are added to the inputs. The dashed lines from the
bottom plots indicate the truth values of the parameters represented by the
inputs.

##  V Conclusions

We have introduced a Prior-Assisted Weak Supervision (PAWS) approach to
anomaly detection. Like previous weakly-supervised methods, PAWS is based on
training a classifier to distinguish data from a background-only reference
dataset. We impose structure on the function class of classifiers through a
pre-training step based on a set of signal models. This trades off breadth for
depth of sensitivity - we show that PAWS is able to improve the sensitivity of
previous methods by a factor of 10–100 in signal cross section. While we
require pre-specifying the signal class, it can (and in our case, does)
contain a large number of models that would be infeasible to probe one at a
time. One could scan, in steps consistent with the experimental resolution,
over the parameters of the models. In addition to the computational challenge,
this would also incur a large trial factor that PAWS does not pay.
Furthermore, the sensitivity of PAWS is at the limit of what is possible for
any method and so there is little to be lost compared to a dedicated approach.

There are a number of extensions of PAWS that would be exciting to explore in
future work. One direction would be to push the dimensionality of the
parameter space. In our numerical example, the parameter space was three-
dimensional ($m_{X},m_{Y}$, and $\alpha$) in addition to the signal fraction
$\mu$. Adding parameters would simply require adding features to the
classifiers, but it may be advisable to not scan over them uniformly when
building the parameterized classifier. Another direction would be to push the
dimensionality of the feature space. In our numerical example, the feature
space was six-dimensional. Preliminary studies suggest that the setup works
well when using the full phase space Buhmann:2023acn , but the performance is
so good that more data is required for training and testing. It would also be
interesting to explore the sensitivity to signals that are similar to the ones
used in the pre-training, but not exactly the same. We found empirically that
the model does not extrapolate well to $m_{X}$ and $m_{Y}$ outside of the
training dataset, but this is not conclusive because the jet substructure also
changes significantly (e.g. the decay products are no longer contained within
a single jet). While our example focused on resonant anomalies, we expect that
the PAWS strategy would work in any scenario where data is compared to a
background-only reference. Exploring various combinations of PAWS with other
weakly-supervised approaches for signal sensitivity and background estimation
could lead to further improvements. Ultimately, we expect that a variety of
strategies with complementary assumptions and strengths will be required to
achieve broad discovery potential at the LHC and beyond.

## Data and code availability

The original LHC Olympics datasets are available on Zenodo. Our additional
signal models, with the corresponding scripts to generate them, are also on
Zenodo. The code we used to train and evaluate our models is available at
https://github.com/hep-lbdl/PAWS.

## Acknowledgments

We thank V. Mikuni, R. Mastandrea, and D. Shih for helpful discussions and
feedback on the manuscript. CLC, GS, and BN are supported by the U.S.
Department of Energy (DOE), Office of Science under contract DE-
AC02-05CH11231. This work was supported in part by the U.S. Department of
Energy, Office of Science, Office of Workforce Development for Teachers and
Scientists (WDTS) under the Science Undergraduate Laboratory Internships
(SULI) program. This research used resources of the National Energy Research
Scientific Computing Center, a DOE Office of Science User Facility supported
by the Office of Science of the U.S. Department of Energy under Contract No.
DE-AC02-05CH11231 using NERSC award HEP-ERCAP0021099.

## References

  * (1) G. Kasieczka et al., The LHC Olympics 2020: A Community Challenge for Anomaly Detection in High Energy Physics, arXiv:2101.08320.
  * (2) T. Aarrestad et al., The Dark Machines Anomaly Score Challenge: Benchmark Data and Model Independent Event Classification for the Large Hadron Collider, arXiv:2105.14027.
  * (3) G. Karagiorgi, G. Kasieczka, S. Kravitz, B. Nachman, and D. Shih, Machine learning in the search for new fundamental physics, Nature Rev. Phys. 4 (2022), no. 6 399–412.
  * (4) ATLAS Collaboration, Dijet resonance search with weak supervision using 13 TeV pp collisions in the ATLAS detector, arXiv:2005.02983.
  * (5) ATLAS Collaboration, G. Aad et al., Search for New Phenomena in Two-Body Invariant Mass Distributions Using Unsupervised Machine Learning for Anomaly Detection at s=13 TeV with the ATLAS Detector, Phys. Rev. Lett. 132 (2024), no. 8 081801, [arXiv:2307.01612].
  * (6) ATLAS Collaboration, Anomaly detection search for new resonances decaying into a Higgs boson and a generic new particle $X$ in hadronic final states using $\sqrt{s}=13$ TeV $pp$ collisions with the ATLAS detector, arXiv:2306.03637.
  * (7) CMS Collaboration, Model-agnostic search for dijet resonances with anomalous jet substructure in proton-proton collisions at $\sqrt{s}$ = 13 TeV, tech. rep., CERN, Geneva, 2024.
  * (8) J. H. Collins, K. Howe, and B. Nachman, Anomaly Detection for Resonant New Physics with Machine Learning, Phys. Rev. Lett. 121 (2018), no. 24 241803, [arXiv:1805.02664].
  * (9) J. H. Collins, K. Howe, and B. Nachman, Extending the search for new resonances with machine learning, Phys. Rev. D99 (2019), no. 1 014038, [arXiv:1902.02634].
  * (10) R. T. D’Agnolo and A. Wulzer, Learning New Physics from a Machine, Phys. Rev. D99 (2019), no. 1 015014, [arXiv:1806.02350].
  * (11) R. T. D’Agnolo, G. Grosso, M. Pierini, A. Wulzer, and M. Zanetti, Learning Multivariate New Physics, arXiv:1912.12155.
  * (12) A. Andreassen, B. Nachman, and D. Shih, Simulation Assisted Likelihood-free Anomaly Detection, Phys. Rev. D 101 (2020), no. 9 095004, [arXiv:2001.05001].
  * (13) B. Nachman and D. Shih, Anomaly Detection with Density Estimation, Phys. Rev. D 101 (2020) 075042, [arXiv:2001.04990].
  * (14) K. Benkendorfer, L. L. Pottier, and B. Nachman, Simulation-Assisted Decorrelation for Resonant Anomaly Detection, arXiv:2009.02205.
  * (15) A. Hallin, J. Isaacson, G. Kasieczka, C. Krause, B. Nachman, T. Quadfasel, M. Schlaffer, D. Shih, and M. Sommerhalder, Classifying Anomalies THrough Outer Density Estimation (CATHODE), arXiv:2109.00546.
  * (16) R. T. d’Agnolo, G. Grosso, M. Pierini, A. Wulzer, and M. Zanetti, Learning New Physics from an Imperfect Machine, arXiv:2111.13633.
  * (17) T. Golling, S. Klein, R. Mastandrea, and B. Nachman, Flow-enhanced transportation for anomaly detection, Phys. Rev. D 107 (2023), no. 9 096025, [arXiv:2212.11285].
  * (18) A. Hallin, G. Kasieczka, T. Quadfasel, D. Shih, and M. Sommerhalder, Resonant anomaly detection without background sculpting, arXiv:2210.14924.
  * (19) T. Golling, G. Kasieczka, C. Krause, R. Mastandrea, B. Nachman, J. A. Raine, D. Sengupta, D. Shih, and M. Sommerhalder, The interplay of machine learning-based resonant anomaly detection methods, Eur. Phys. J. C 84 (2024), no. 3 241, [arXiv:2307.11157].
  * (20) D. Sengupta, S. Klein, J. A. Raine, and T. Golling, CURTAINs Flows For Flows: Constructing Unobserved Regions with Maximum Likelihood Estimation, arXiv:2305.04646.
  * (21) T. Finke, M. Hein, G. Kasieczka, M. Krämer, A. Mück, P. Prangchaikul, T. Quadfasel, D. Shih, and M. Sommerhalder, Back To The Roots: Tree-Based Algorithms for Weakly Supervised Anomaly Detection, arXiv:2309.13111.
  * (22) M. Freytsis, M. Perelstein, and Y. C. San, Anomaly Detection in Presence of Irrelevant Features, arXiv:2310.13057.
  * (23) E. Buhmann, C. Ewen, G. Kasieczka, V. Mikuni, B. Nachman, and D. Shih, Full Phase Space Resonant Anomaly Detection, arXiv:2310.06897.
  * (24) K. Cranmer, J. Pavez, and G. Louppe, Approximating Likelihood Ratios with Calibrated Discriminative Classifiers, arXiv:1506.02169.
  * (25) P. Baldi, K. Cranmer, T. Faucett, P. Sadowski, and D. Whiteson, Parameterized neural networks for high-energy physics, Eur. Phys. J. C76 (2016), no. 5 235, [arXiv:1601.07913].
  * (26) S. E. Park, D. Rankin, S.-M. Udrescu, M. Yunus, and P. Harris, Quasi Anomalous Knowledge: Searching for new physics with embedded knowledge, arXiv:2011.03550.
  * (27) C. K. Khosa and V. Sanz, Anomaly Awareness, arXiv:2007.14462.
  * (28) H. Beauchesne, Z.-E. Chen, and C.-W. Chiang, Improving the performance of weak supervision searches using transfer and meta-learning, JHEP 02 (2024) 138, [arXiv:2312.06152].
  * (29) L. Heinrich, T. Golling, M. Kagan, S. Klein, M. Leigh, M. Osadchy, and J. A. Raine, Masked Particle Modeling on Sets: Towards Self-Supervised High Energy Physics Foundation Models, arXiv:2401.13537.
  * (30) V. Mikuni and B. Nachman, OmniLearn: A Method to Simultaneously Facilitate All Jet Physics Tasks, arXiv:2404.16091.
  * (31) R. Das, G. Kasieczka, and D. Shih, Residual ANODE, arXiv:2312.11629.
  * (32) K. Cranmer, J. Brehmer, and G. Louppe, The frontier of simulation-based inference, Proceedings of the National Academy of Sciences 117 (May, 2020) 30055–30062.
  * (33) G. Kasieczka, B. Nachman, and D. Shih, Official Datasets for LHC Olympics 2020 Anomaly Detection Challenge (Version v6) [Data set]., 2019.  https://doi.org/10.5281/zenodo.4536624.
  * (34) T. Sjostrand, S. Mrenna, and P. Z. Skands, PYTHIA 6.4 Physics and Manual, JHEP 05 (2006) 026, [hep-ph/0603175].
  * (35) T. Sjöstrand, S. Ask, J. R. Christiansen, R. Corke, N. Desai, P. Ilten, S. Mrenna, S. Prestel, C. O. Rasmussen, and P. Z. Skands, An introduction to PYTHIA 8.2, Comput. Phys. Commun. 191 (2015) 159–177, [arXiv:1410.3012].
  * (36) DELPHES 3 Collaboration, J. de Favereau, C. Delaere, P. Demin, A. Giammanco, V. Lemaitre, A. Mertens, and M. Selvaggi, DELPHES 3, A modular framework for fast simulation of a generic collider experiment, JHEP 02 (2014) 057, [arXiv:1307.6346].
  * (37) A. Mertens, New features in Delphes 3, J. Phys. Conf. Ser. 608 (2015), no. 1 012045.
  * (38) M. Cacciari and G. P. Salam, Dispelling the $N^{3}$ myth for the $k_{t}$ jet-finder, Phys. Lett. B641 (2006) 57–61, [hep-ph/0512210].
  * (39) M. Cacciari, G. P. Salam, and G. Soyez, FastJet User Manual, Eur. Phys. J. C 72 (2012) 1896, [arXiv:1111.6097].
  * (40) M. Cacciari, G. P. Salam, and G. Soyez, The anti-$k_{t}$ jet clustering algorithm, JHEP 04 (2008) 063, [arXiv:0802.1189].
  * (41) J. Thaler and K. Van Tilburg, Identifying Boosted Objects with N-subjettiness, JHEP 03 (2011) 015, [arXiv:1011.2268].
  * (42) J. Thaler and K. Van Tilburg, Maximizing Boosted Top Identification by Minimizing N-subjettiness, JHEP 02 (2012) 093, [arXiv:1108.2701].
  * (43) E. M. Metodiev, B. Nachman, and J. Thaler, Classification without labels: Learning from mixed samples in high energy physics, JHEP 10 (2017) 174, [arXiv:1708.02949].
  * (44) M. Abadi, A. Agarwal, P. Barham, E. Brevdo, Z. Chen, C. Citro, G. S. Corrado, A. Davis, J. Dean, M. Devin, S. Ghemawat, I. Goodfellow, A. Harp, G. Irving, M. Isard, Y. Jia, R. Jozefowicz, L. Kaiser, M. Kudlur, J. Levenberg, D. Mané, R. Monga, S. Moore, D. Murray, C. Olah, M. Schuster, J. Shlens, B. Steiner, I. Sutskever, K. Talwar, P. Tucker, V. Vanhoucke, V. Vasudevan, F. Viégas, O. Vinyals, P. Warden, M. Wattenberg, M. Wicke, Y. Yu, and X. Zheng, TensorFlow: Large-scale machine learning on heterogeneous systems, 2015.  Software available from tensorflow.org.
  * (45) D. P. Kingma and J. Ba, Adam: A Method for Stochastic Optimization, 12, 2014.  arXiv:1412.6980.


# NDST: Neural Driving Style Transfer for Human-Like Vision-Based Autonomous
Driving

Donghyun Kim, Aws Khalil, Haewoon Nam, , and Jaerock Kwon A. Khalil and J.
Kwon are with the University of Michigan-Dearborn, Michigan, 48128 USA.D. Kim
and H. Nam are with the Department of Electrical and Electronic Engineering,
Hanyang University, Ansan, 15588, South Korea (e-mail: hnam@hanyang.ac.kr).

###### Abstract

Autonomous Vehicles (AV) and Advanced Driver Assistant Systems (ADAS)
prioritize safety over comfort. The intertwining factors of safety and comfort
emerge as pivotal elements in ensuring the effectiveness of Autonomous Driving
(AD). Users often experience discomfort when AV or ADAS drive the vehicle on
their behalf. Providing a personalized human-like AD experience, tailored to
match users’ unique driving styles while adhering to safety prerequisites,
presents a significant opportunity to boost the acceptance of AVs. This paper
proposes a novel approach, Neural Driving Style Transfer (NDST), inspired by
Neural Style Transfer (NST), to address this issue. NDST integrates a
Personalized Block (PB) into the conventional Baseline Driving Model (BDM),
allowing for the transfer of a user’s unique driving style while adhering to
safety parameters. The PB serves as a self-configuring system, learning and
adapting to an individual’s driving behavior without requiring modifications
to the BDM. This approach enables the personalization of AV models, aligning
the driving style more closely with user preferences while ensuring baseline
safety critical actuation. Two contrasting driving styles (Style A and Style
B) were used to validate the proposed NDST methodology, demonstrating its
efficacy in transferring personal driving styles to the AV system. Our work
highlights the potential of NDST to enhance user comfort in AVs by providing a
personalized and familiar driving experience. The findings affirm the
feasibility of integrating NDST into existing AV frameworks to bridge the gap
between safety and individualized driving styles, promoting wider acceptance
and improved user experiences.

###### Index Terms:

AV Personalization, Deep learning, Driving Style, Style Transfer

††publicationid: pubid: 0000–0000/00$00.00 © 2021 IEEE

##  I Introduction

In the last decade, Autonomous Vehicles (AV) and Advanced Driver Assistant
Systems (ADAS) have progressed significantly, moving closer to their goal of
providing a safer and more regulated mode of transportation. However, the
attainment of safety alone does not ensure the success of Autonomous Driving
(AD) technology [1]. Persistent reservations and concerns among the public [2]
highlight the critical need for AD technology to not only prioritize safety
but also deliver a comfortable and stress-free user experience.

The intertwining factors of safety and comfort emerge as pivotal elements in
ensuring the effectiveness of AD. Passenger discomfort arises when a driving
style significantly differs from what individuals are accustomed to [3]. This
driving style, including turning dynamics, speed modulation, acceleration, and
braking, is a key factor influencing the overall user acceptance of AD
technology. Therefore, adopting a human-like driving style is crucial for
improving both comfort and safety in the AD experience [4].

(a)

(b)

Figure 1:  (a) Neural Image Style Transfer (NST) [5]. NST extracts a style
from one image and applies the style to the content of another image. (b)
Neural Driving Style Transfer (NDST). NDST can produce personalized driving
results by applying a user’s driving style to an existing AD network.

Chandrayee Basu [6] delves into the investigation of preferred autonomous
driving styles, further underscoring this imperative. Basu’s research showed
that even though the majority of drivers could not accurately identify their
own driving style, there was a high correlation between the style they
considered their own and the style they preferred, indicating a preference for
styles similar to their own. These findings emphasize the need for a
personalized human-like AD style. Providing a personalized human-like AD
experience, tailored to match users’ unique driving styles while adhering to
safety prerequisites, presents a significant opportunity to boost the
acceptance of AV [7][8].

To achieve this, we propose a new method specifically designed for vision-
based AD. While existing studies [9, 10, 11, 12, 13, 14, 15, 16] typically
adopt a standardized driving model based on specific applications (e.g., urban
driving, racing, etc), our method, Neural Driving Style Transfer (NDST),
acknowledges the diversity in drivers’ habits influenced by factors like
personality and environment [17]. Inspired by Neural Image Style Transfer
(NST) [5] from deep neural networks-based computer vision communities, the
proposed method, NDST, leverages the NST concepts to integrate a user’s
driving style with the vehicle’s AD model, creating a personalized driving
experience while maintaining standard safety baseline.

The two main components of NDST are the Baseline Driving Model (BDM) and the
Personalized Block (PB). Fig. 1 illustrates the comparison between NST and
NDST. The BDM, serving as the standard AD model, is comparable to the image
content in NST, while the PB incorporates the user’s driving style. This
approach aims to bridge the gap between safety and personalization in AD.

To validate our methodology, we collected data from two drivers with distinct
driving styles labeled as; A and B. Style B exhibited faster acceleration and
deceleration compared to Style A. The main contributions of this paper are as
follows:

  1. 1.

This paper proposes a novel learning method, NDST, designed to provide a
human-like personalized AD experience that increases the acceptance of AV and
enhances the user experience.

  2. 2.

The PB is designed to be a self-configuring system, which can be seamlessly
integrated with the original AD system (i.e., the BDM) without requiring any
modifications. The PB is dependent on the output of the BDM to ensure that any
safety-related decisions made by the BDM will be reflected in the final output
of the PB. This increases comfort while maintaining the required baseline
safety.

  3. 3.

An extensive amount of simulations in various test tracks are performed to
evaluate the performance of the proposed method. It is observed that the
proposed network gives stable autonomous driving on the tracks with the
driving style of choice by imitating the driver’s acceleration and
deceleration patterns.

The remainder of this paper is organized as follows. Section II introduces
related research and Section III describes the proposed method. Section IV
shows the experimental results in the simulation environment, and analyzes the
results. Section V concludes the thesis and presents directions for future
work.

##  II Related Work

Various research efforts have been conducted with the aim to effectively
extract and analyze driving styles from drivers’ data.

Hironori Hiraishi [18] used the Neural Image Style Transfer (NST) method to
learn the driving style after forming the driving data as an image in RGB
format. However, research on how to decode the driving style image to control
the vehicle has not been conducted. Qi et al.[19] analyzed the driving style
based on the change of the temporal distance, which is the distance to the
vehicle in front. Based on their analysis, they developed a structure to
distinguish different driving styles and to build a connection between style
and human driving behavior. However, in their analysis, they do not consider
different factors, other than the temporal distance. Choi et al.[20] proposed
a DSA-GAN that recognizes the driver’s driving style and predicts the
trajectory by reflecting the style. They used Recurrence Plot and CNN network
to learn three driving styles, normal, aggressive, and careless, and predicted
their trajectories. In a similar work, Kim et al. [21] conducted a work to
improve prediction performance and create a personalized trajectory by
recognizing and reflecting on various driving styles to predict the trajectory
of an ego vehicle. Their proposed system uses a conditional variational
autoencoder (CVAE) structure, and studies have been conducted to learn and
predict the trajectory of three driving styles: normal, aggressive, and
distracted. However, in these studies, the predicted trajectories were not
used to drive the vehicle. This was presented as future work. Bingzhao Gao
[22] proposed a personalized adaptive cruise control (ACC) system based on
model predictive control (MPC). In this work, the driving style was defined as
conservative, moderate, and aggressive, and the driving style of the driver
was learned using the relative speed and headway of the vehicle in front. The
work assumed a situation in which a leading vehicle exists, it did not discuss
whether the autonomous vehicle continues to adapt its driving style in the
absence of a leading vehicle. Xu et al.[23] proposed a technique for creating
driver models that accommodate different driving styles by modeling actual
human driving behavior. The authors used old machine learning techniques to
examine the driving behaviors of participants who were driving on a closed
circuit after collecting their data. The efficiency of the produced driver
models was then assessed using a driving simulator. The driving style was
represented by the throttle, brake, and vehicle speed data. In another work,
Shi et al.[24] evaluated these driving styles to detect abnormal driving
behavior. Similarly, Kuderer et al.[25] worked on learning the driving style
from demonstration, in which they build an algorithm to learn the highway
driving style using recorded human driving data. However, these studies were
done based on driving data only without visual input as vision-based
autonomous driving was not yet popular (imitation learning or behavioral
cloning adopting a vision-based end-to-end technique did not acquire
popularity until DAVE-2 was developed [11]).

##  III Proposed Method

This section explains how to use the proposed method to transfer the driving
style of a driver. The proposed method is based on building a network that
learns the driving style of a driver and integrates it into the AV. The
driving data used to work the driving style is the acceleration and
deceleration data based on road type. However, other driving data can be added
as well, such as steering angles, vehicle speed, and time.

The proposed method uses driving data from the user to learn the
acceleration/deceleration patterns. To validate our approach, we conducted an
experiment using data from two drivers with distinct driving styles,
arbitrarily referred to as style A and style B. In practice, the method is
designed to accommodate any number of unique driving styles. In our
experimental setup, style B represents a driver who accelerates and
decelerates more quickly compared to the driver characterized by style A.

###  III-A Neural Driving Style Transfer (NDST) Network Architecture

Figure 2:  Structure of NDST Network. The NDST architecture comprises two main
components: the Baseline Driving Model (BDM), a standard autonomous vehicle
driving model based on NVIDIA’s PilotNet, and the Personalized Block (PB),
which customizes driving outputs according to individual driver styles. The
BDM, depicted in blue, integrates image and speed inputs to generate standard
vehicle actuations like steering, throttle, and braking through its five
convolutional and three fully-connected layers. The PB, shown in red, inputs
the BDM’s predicted actions, road features, current speed, and the difference
between target and current speeds, outputting personalized steering, throttle,
and braking values.

The NDST network architecture is depicted in Fig. 2. The NDST architecture
consists of two major blocks: a BDM block and a PB block. The reason behind
choosing this design is to make the proposed system a self-configuring system,
which makes it easy to integrate with the original vehicle system without the
need to modify it. Therefore, the proposed system, NDST, allows the
personalization feature to be easily enabled/disabled based on the user’s
preference.

####  III-A1 Baseline Driving Model (BDM)

The baseline driving model is a generic AV driving model that can drive the
vehicle. The BDM network architecture is based on the NVIDIA PilotNet
structure [26], and is shown in blue in Fig. 2. However, the network is
modified so that the image and current speed are used as inputs, and the
steering angle, throttle, and brake values are defined as outputs. The network
processes the first input (i.e., visual observation) using five convolutional
layers, and then the output of the fifth is flattened to form an image feature
vector. Then, the second input (current speed) is concatenated with the image
feature vector. After that, the concatenated vectors are fed to three fully
connected layers (Dense layers). The BDM block output is the output of its
last fully connected layer and is the prediction of three actuation values
(standard actuation in Fig. 2): Steering, Throttle, and Brake values.

####  III-A2 Personalized Block (PB)

The PB is a neural network for personalizing the output value of the BDM based
on the driving style of a particular driver, which is shown in Fig. 2 in red.
It is seen that the PB has four inputs. The first input is the predicted
action values of the BDM. The second input is the output of the third and last
convolution layer of the BDM after being flattened, which gives an indication
of the road conditions. The third input is the current speed data. The Fourth
input is the difference between the target speed of the vehicle, which is
defined by the user and the current speed. Each input goes into a fully
connected layer to form a vector. Then, all four vectors are concatenated and
fed to three fully connected layers. A dropout layer was needed after the
first fully connected layer for generalization purposes. The output of the
third layer is the final output of the PB which is a personalized predicted
actuation based on the style of a particular driver and contains three values:
steering angle, throttle, and brake.

###  III-B Training Method for AV Personalization

Figure 3:  Neural driving style transfer (NDST) training process for AVs
Personalization. The NDST training process begins with establishing a Baseline
Driving Model (BDM). Subsequently, the Personalized Block (PB) is trained
using data collected for each driver and the BDM, which is set to be non-
trainable, ensuring that the PB adjusts and reflects the unique driving style
of each driver.

The proposed method for the personalization of AVs is shown in Fig. 3. The
structure of the proposed system consists of the following steps.

Step 0: Learning the Baseline Driving Model

The proposed system essentially works under the presumption that the vehicle
is equipped with a baseline autonomy model, called the baseline driving model
(BDM), that is capable of driving the vehicle. The BDM does not represent the
driving preferences of any one driver, thus it is not biased toward the unique
driving style of one driver.

The dataset used to train such a model can be defined as

$D=\\{I_{t},A_{t}\\}^{N}_{t=1},$ (1)

where $D$ is dataset, $N$ is the total time steps, $I_{t}$ is the input data
at time $t$ and $A_{t}$ is the correlative action that controls the vehicle.
The input data $I_{t}$ can be defined as

$I_{t}=\left\\{O_{t},v_{t}^{c}\right\\},$ (2)

where $O_{t}$ is visual observation (e.g. images) and $v_{t}^{c}$ is the
vehicle’s current speed. The action that controls the vehicle $A_{t}$ can be
defined as

$A_{t}=\left\\{S_{t},T_{t},B_{t}\right\\},$ (3)

where $S_{t}$, $T_{t}$, and $B_{t}$ are the steering angle, throttle and brake
at time $t$, respectively. Therefore, total input data $I$ and action data $A$
can be defined as

$I=\left\\{I_{t}\right\\}_{t=1}^{N},\;A=\left\\{A_{t}\right\\}_{t=1}^{N}.$ (4)

$\tilde{A}=\pi^{BDM}\left(I\right)$ (5)

In (5), $\pi^{BDM}$ represents the BDM policy, which takes $I$ as an input,
and provides $\tilde{A}$ as an output, where $\tilde{A}$ is the predicted
actuation of the BDM. The BDM is trained to minimize the average prediction
error as

$\omega^{*}=\underset{\omega}{\arg\min}\sum_{t}^{N}L(\tilde{A}_{t},{A}_{t}),$
(6)

where $\omega^{*}$ represents the parameters of the BDM, and L is the loss per
sample.

Step 1: Collecting driving data of a driver

After building the BDM, we start collecting data from a driver whose driving
style we want to transfer. This is illustrated in step 1 in Fig. 3, which can
be expressed in the following as

$\begin{split}D^{n}=\\{I^{n},A^{n}\\},\end{split}$ (7)

where $n$ represents the driver’s number. ${D}^{n}$ is the entire dataset
collected by driver $n$. ${I}^{n}$ and ${A}^{n}$ mean the $I$ and $A$
collected by driver $n$, respectively.

Step 2: Training the Personalized Block (PB) for the driver

Finally, PB is trained using the learned BDM and driving data obtained through
Steps 0 and 1. One PB reflects one driver’s style. In order to train PB only,
the baseline driving model is frozen so that it is not trained. Input data are
then fed to the BDM. The BDM estimates the ’action’ after processing the input
data, as shown in (5). Note that the estimated action is the result of a
generic autonomous driving model (i.e., BDM), so non-specific driving style
has been applied.

To modify the resulting action into an action that reflects driver $n$’s
style, as shown in step 2 in Fig. 3, the PB would get four inputs for
training, that are depicted in Fig. 2. The first input ($\tilde{A}^{n}$) is
the predicted action values of the BDM when fed the input data of driver $n$
($I^{n}$) is represented by

$\tilde{A}^{n}=\pi^{BDM}\left(I^{n}\right).$ (8)

The second input ($\phi(O^{n})$) is obtained by combining the output of two
convolution layers when $O^{n}$ is input to BDM, which extracts features from
observation data ($O^{n}$). This data is used to give the PB spatial
information about the road. The third input is the current speed data
collected by driver $n$ in step 1 $\left(v^{c}\right)^{n}$. The last input is
the difference between the target speed of the vehicle ($V^{g}$) which is
defined by the user and the current speed $v^{c}$. The target speed is used by
PB to determine acceleration and deceleration. The action obtained through
driver $n$ ($A^{n}$) is used as the ground truth for learning. The final
output of the trained PB is a personalized predicted actuation of driver $n$
($\tilde{\tilde{{A}}}^{n}$) and can be defined as

$\tilde{\tilde{A}}^{n}=\pi^{PB}\left(\tilde{A}^{n},\phi\left(O^{n}\right),\left(v^{c}\right)^{n},V^{g}\right).$
(9)

The loss function used to learn PB can be defined as

$\chi^{*}=\underset{\chi}{\arg\min}\sum_{t}^{N}L(\tilde{\tilde{A}}_{t}^{n},{A}_{t}^{n}),$
(10)

where $\chi$ represents parameters of the PB, and $L$ is a loss per sample.

The proposed method for personalization is designed not to modify the original
system but to add to it, by plugging the PB block into the vehicle system.

###  III-C Simulation Environment

To implement the NDST, we used the OSCAR simulator [27] which was introduced
in [15], for its simplicity as it was built using ROS (Robotic Operating
System) [28] and Gazebo simulator [29] and because it was designed for vision-
based autonomous driving.

####  III-C1 Track

Figure 4:  The training track used to collect training data from two different
drivers.  Figure 5:  The test track used to test Neural Driving Style Transfer
(NDST) system.

Since the driving style we are trying to transfer is represented by the
steering angle, throttle (acceleration), and brake (deceleration), the
training track, shown in Fig. 4, was designed to cover three cases that
reflect the driving style: acceleration/deceleration to reach the target
speed, deceleration before entering the curved section, and acceleration after
exiting the curved section. To test the NDST model after training, a test
track was constructed to check the acceleration and deceleration of straight
and curved sections, and the corresponding track is shown in Fig. 5.

TABLE I: Total number of data collected for training. Train model | Driving style | Number of Data
---|---|---
Baseline Driving Model | Standard | 85910
Personalized Block 1 | Style A | 41321
Personalized Block 2 | Style B | 37673

(a) Style A Straight (b) Style A Left (c) Style A Right

(d) Style B Straight (e) Style B Left (f) Style B Right

Figure 6:  (a), (b), and (c) each represent the G-G diagram of the straight,
left-turn, and right-turn roads in Style A, respectively. Similarly, (d), (e),
and (f) each correspond to the G-G diagram of the straight, left-turn, and
right-turn roads in Style B, respectively.

(a)  (b)

Figure 7:  Training dataset visualization on the training track. (a) The
dataset was collected by driver 1 (driving in style A) on the training track.
(b) The dataset was collected by driver 2 (driving in style B) on the training
track.

####  III-C2 Dataset

Before training the NDST Network, the BDM should be trained to enable normal
driving without reflecting a specific driving style. Therefore, in order to
collect a dataset suitable for this purpose and eliminate any bias in the
data, the BDM was trained using multiple datasets with various acceleration,
deceleration, and speed data on the track. Afterward, training data was
collected to train two personalized blocks (PB1 and PB2), representing driving
data in two different styles; style A and style B. Table I shows the number of
data collected for training the BDM and the two personalized blocks. The total
number of training data was $85.9$k, $41.3$k, and $37.6$k, for BDM, PB1, and
PB2, respectively.

In style A, the rate of change in acceleration and deceleration was low
compared to style B. The G-G diagram visualizing the acceleration of style A
and style B data is shown in Fig. 6. On the straight road, the style A G-G
diagram can confirm that the range of x-axis acceleration is relatively
smaller than the style B G-G diagram, and the area of the G-G diagram measured
for the left and right turn roads is also smaller than style B. This shows
that style B exhibits more rapid acceleration and deceleration than style A in
terms of vehicle control. As shown in Fig. 7, both styles decelerate upon
entering the curve and accelerate upon exiting the curve, but the values of
the acceleration and deceleration are different for each style. Style B (Fig.
7b) had higher values of acceleration and deceleration compared to style A
(Fig. 7a), and acceleration and deceleration were controlled quickly and over
a shorter distance.

##  IV Experimental Results

In our experiments, we used a variety of plots to validate the performance of
the NDST method. These evaluations confirmed that self-configuring PB can
effectively transfer a driver’s driving style into a generic AV model for
personalization.

In the training phase, the NDST method was used to learn driving styles, style
A and style B. During this process, the PBs learned the distribution of
acceleration and deceleration control values associated with each driving
style.

(a) PB1 (transfer to Style A) (b) PB2 (transfer to Style B)

Figure 8:  Results from driving on the test track with trained PBs (a) A G-G
diagram of the acceleration/deceleration results from PB1 which transferred
Style A. (b) a G-G diagram of the acceleration/deceleration results from PB2
which transferred Style B.

The training results were evaluated by analyzing acceleration and deceleration
outputs obtained from test track drives using PB1 and PB2, respectively.
Subsequent verification confirmed the successful execution of the style
transfer. The result is seen in Fig. 8, where Fig. 8a is a G-G diagram of the
acceleration/deceleration results from PB1 which transferred style A, and Fig.
8b is a G-G diagram of the acceleration/deceleration results from PB2 which
transferred style B. The difference between the two G-G diagrams clearly shows
that the styles have been transferred successfully, as each PB drove in a
different style. PB1 does not have a large change in acceleration, so the area
displayed on the G-G diagram is small. In contrast, PB2 (which transfers style
B) has a large change in acceleration, almost three times the acceleration
value compared to style A transferred by PB1.

(a)  (b)

Figure 9:  Testing driving data. (a) Style A driving data. Trained NDST model
with PB1 driving on the test track. (b) Style B driving data. Trained NDST
model with PB2 driving on the test track.  Figure 10:
Acceleration/deceleration in curved sections (a) Results from PB1 sending
Style A. (b) Results from PB2 sending Style B.

(a)  (b)

Figure 11:  The distance needed for each model to reach a target speed of
$30m/s$ in a straight road section. (a) The driving data of the original
dataset was collected by driver 1 (top) and the PB1 (style A) driving data
(bottom). (b) The driving data of the original dataset was collected by driver
2 (top) and the PB2 (style B) driving data (bottom). The PB1 (style A) took
$2250m$ from the starting point to reach $30m/s$, but the PB2 (style B)
reached $30m/s$ in $500m$, showing rapid acceleration and deceleration.

Based on the results of driving on the test track, using a 3D plot, the
acceleration values for each road section were visualized. Fig. 9 is the
result of visualizing these acceleration values. Fig. 9a shows the result of
driving on the test track in style A using the trained PB1, and Fig. 9b shows
the result of driving on the same test track in style B using the trained PB2.
Since acceleration/deceleration values are higher in style B, we can see that
the acceleration/deceleration periods of time are shorter in comparison with
style A. This is obvious at the points of entering or exiting a curved road
section, as depicted in Fig. 10.

The target speed is set to $20$m/s and shows the vehicle accelerates from the
starting point to reach the target speed. It then shows the car slowing down
as it enters a curve and accelerating as it exits. While on the curve itself,
both PB1 and PB2 slowed down from $20$m/s to $15$m/s before entering the
curve, driving through the curve and maintaining $15$m/s. While reaching the
curve exit, we compared the acceleration and distance required by the two
models to reach the target speed of $20$m/s after escaping from the curve.
Although both PB1 and PB2 accelerate and decelerate to reach the target speed,
the acceleration, and deceleration applied to reach that target speed are
different. PB2 (style B) reached the target speed of $80$m, while PB1 traveled
$150$m to reach the target speed. Fig. 11 shows the velocity and control
values of each model when the target speed is set on the straight road
section. Fig. 11a shows the driving data of the original dataset collected by
driver 1 (top) and the PB1 (style A) driving data (bottom), and Fig. 11b shows
the driving data of the original dataset collected by driver 2 (top) and the
PB2 (style B) driving data (bottom). The PB1 (style A) took $2250$m from the
starting point to reach $30$m/s, but the PB2 (style B) reached $30$m/s in
$500$m, showing rapid acceleration and deceleration.

##  V Conclusion

This paper introduces a novel approach called Neural Driving Style Transfer
(NDST) that aims to personalize the driving experience of vision-based AV.
NDST can transfer a driver’s unique driving style to a vision-based AV,
allowing the vehicle to drive in a way that is more similar to the driver’s
preferences by integrating a standardized driving model with a Personalized
Block (PB) capable of acquiring causal associations between acceleration and
deceleration from each driver’s unique data. The proposed system, NDST, was
successfully validated through simulated driving results, underscoring its
ability to adapt an AV system to individual driving styles.

## Acknowledgment

## References

  * [1] J. Ling, J. Li, K. Tei, and S. Honiden, “Towards personalized autonomous driving: An emotion preference style adaptation framework,” in _Proc. IEEE. Int. Conf. Agents_ , 2021, pp. 47–52.
  * [2] N. Dillen, M. Ilievski, E. Law, L. E. Nacke, K. Czarnecki, and O. Schneider, “Keep calm and ride along: Passenger comfort and anxiety as physiological responses to autonomous driving styles,” in _Proc. CHI Conf. Hum. Factors Comput. Syst._ , Apr. 2020, pp. 1–13.
  * [3] S. Ittner, D. Mühlbacher, and T. H. Weisswange, “The discomfort of riding shotgun – why many people don’t like to be co-driver,” _Front. Psychol._ , vol. 11, 2020. [Online]. Available: https://www.frontiersin.org/articles/10.3389/fpsyg.2020.584309
  * [4] L. Vasile, B. Seitz, V. Staab, M. Liebherr, C. Däsch, and D. Schramm, “Influences of personal driving styles and experienced system characteristics on driving style preferences in automated driving,” _Applied Sciences_ , vol. 13, no. 15, p. 8855, 2023.
  * [5] L. A. Gatys, A. S. Ecker, and M. Bethge, “Image style transfer using convolutional neural networks,” in _Proc. IEEE Conf. Comput. Vis. Pattern Recognit._ , 2016, pp. 2414–2423.
  * [6] C. Basu, Q. Yang, D. Hungerman, M. Singhal, and A. D. Dragan, “Do you want your autonomous car to drive like you?” in _Proc. ACM/IEEE Int. Conf. Human-Robot Interact._ , 2017, pp. 417–425.
  * [7] Z. Ma and Y. Zhang, “Investigating the effects of automated driving styles and driver’s driving styles on driver trust, acceptance, and take over behaviors,” _Proc. Hum. Factors Ergonom. Soc. Annu. Meeting_ , vol. 64, no. 1, pp. 2001–2005, 2020.
  * [8] M. Natarajan, K. Akash, and T. Misu, “Toward adaptive driving styles for automated driving with users’ trust and preferences,” in _Proc. ACM/IEEE Int. Conf. Human-Robot Interact._ , 2022, pp. 940–944.
  * [9] D. Pomerleau, “Alvinn: An autonomous land vehicle in a neural network,” in _Proc. Adv. Neural Inf. Process. Syst._ , December 1989, pp. 305 – 313.
  * [10] Net-Scale, “Autonomous Off-Road Vehicle Control Using End-to-End Learning,” Jul. 2004. [Online]. Available: http://net-scale.com/doc/net-scale-dave-report.pdf
  * [11] Z. Chen and X. Huang, “End-to-end learning for lane keeping of self-driving cars,” in _Proc. IEEE Intell. Veh. Symp. (IV)_ , 2017, pp. 1856–1860.
  * [12] M. Bojarski _et al._ , “Explaining how a deep neural network trained with end-to-end learning steers a car,” _arXiv preprint arXiv:1704.07911_ , 2017. [Online]. Available: http://arxiv.org/abs/1704.07911
  * [13] Q. Wang, L. Chen, B. Tian, W. Tian, L. Li, and D. Cao, “End-to-end autonomous driving: An angle branched network approach,” _IEEE Trans. Veh. Technol._ , vol. 68, no. 12, pp. 11 599–11 610, 2019.
  * [14] T. Wu, A. Luo, R. Huang, H. Cheng, and Y. Zhao, “End-to-end driving model for steering control of autonomous vehicles with future spatiotemporal features,” in _Proc. IEEE/RSJ Int. Conf. Intell. Robots Syst._ , 2019, pp. 950–955.
  * [15] J. Kwon, A. Khalil, D. Kim, and H. Nam, “Incremental end-to-end learning for lateral control in autonomous driving,” _IEEE Access_ , vol. 10, pp. 33 771–33 786, 2022.
  * [16] T. Weiss and M. Behl, “Deepracing: Parameterized trajectories for autonomous racing,” _arXiv preprint arXiv:2005.05178_ , 2020.
  * [17] L. Eboli, G. Mazzulla, and G. Pungillo, “How drivers’ characteristics can affect driving style,” _Transp. Res. Proc._ , vol. 27, pp. 945–952, 2017.
  * [18] H. Hiraishi, “Cognitive driving data visualization and driving style transfer,” in _Proc. IEEE Int. Conf. Cogn. Inform. Cogn. Comput._ , 2021, pp. 92–97.
  * [19] G. Qi, Y. Du, J. Wu, N. Hounsell, and Y. Jia, “What is the appropriate temporal distance range for driving style analysis?” _IEEE Intell. Transp. Syst._ , vol. 17, no. 5, pp. 1393–1403, 2015.
  * [20] S. Choi, N. Kweon, C. Yang, D. Kim, H. Shon, J. Choi, and K. Huh, “Dsa-gan: Driving style attention generative adversarial network for vehicle trajectory prediction,” in _Proc. IEEE Int. Intell. Transp. Syst. Conf._ , 2021, pp. 1515–1520.
  * [21] D. Kim, H. Shon, N. Kweon, S. Choi, C. Yang, and K. Huh, “Driving style-based conditional variational autoencoder for prediction of ego vehicle trajectory,” _IEEE Access_ , vol. 9, pp. 169 348–169 356, 2021.
  * [22] B. Gao, K. Cai, T. Qu, Y. Hu, and H. Chen, “Personalized adaptive cruise control based on online driving style recognition technology and model predictive control,” _IEEE Trans. Veh. Technol._ , vol. 69, no. 11, pp. 12 482–12 496, 2020.
  * [23] L. Xu, J. Hu, H. Jiang, and W. Meng, “Establishing style-oriented driver models by imitating human driving behaviors,” _IEEE Intell. Transp. Syst._ , vol. 16, no. 5, pp. 2522–2530, 2015.
  * [24] B. Shi, L. Xu, J. Hu, Y. Tang, H. Jiang, W. Meng, and H. Liu, “Evaluating driving styles by normalizing driving behavior based on personalized driver modeling,” _IEEE Trans. Syst., Man, Cybern. Syst._ , vol. 45, no. 12, pp. 1502–1508, 2015.
  * [25] M. Kuderer, S. Gulati, and W. Burgard, “Learning driving styles for autonomous vehicles from demonstration,” in _Proc. IEEE Int. Conf. Robot. Autom._ , 2015, pp. 2641–2646.
  * [26] M. Bojarski, D. Del Testa, D. Dworakowski, B. Firner, B. Flepp, P. Goyal, L. D. Jackel, M. Monfort, U. Muller, J. Zhang _et al._ , “End to end learning for self-driving cars,” _arXiv preprint arXiv:1604.07316_ , 2016.
  * [27] J. Kwon, “OSCAR (Open-Source robotic Car Architecture for Research and education),” Feb. 2021, original-date: 2020-11-29T19:48:57Z. [Online]. Available: https://github.com/jrkwon/oscar
  * [28] M. Quigley, K. Conley, B. Gerkey, J. Faust, T. Foote, J. Leibs, R. Wheeler, A. Y. Ng _et al._ , “Ros: an open-source robot operating system,” in _Proc. ICRA Workshop Open Source Softw._ , vol. 3, 2009, p. 5.
  * [29] N. Koenig and A. Howard, “Design and use paradigms for gazebo, an open-source multi-robot simulator,” in _Proc. IEEE/RSJ Int. Conf. Intell. Robots Syst._ , vol. 3, 2004, pp. 2149–2154 vol.3.

|  Donghyun Kim received the B.S. degree in electronics and communication
engineering from Hanyang University, Ansan, South Korea, in 2018, where he is
currently pursuing the Ph.D. degree. His research interests include autonomous
vehicles, personalized autonomous driving, and artificial intelligence. He
received numerous scholarships, such as the Korean Government Scholarship for
his M.S. and Ph.D. degrees.
---|---
|  Aws Khalil earned a B.S. in Mechatronics Engineering from the University of
Jordan, Amman, Jordan in 2018. He received his M.S. degree from Budapest
University of Technology and Economics (BME) in Autonomous Vehicle Control
Engineering. He is currently a Ph.D. student in Electrical, Electronics, and
Computer Engineering at the University of Michigan - Dearborn. Mr. Khalil is
interested in autonomous vehicles, and machine learning. During his master’s
studies, he worked as a research assistant at BME Automated Drive Lab, and now
he works at the University of Michigan - Dearborn’s Bio-inspired Machine
Intelligence (BIMI) Laboratory.
---|---
|  Haewoon Nam (S’99–M’07–SM’10) received the B.S. degree from Hanyang
University, Seoul, South Korea, the M.S. degree from Seoul National
University, and the Ph.D. degree in electrical and computer engineering from
The University of Texas at Austin, Austin, TX, USA. From 1999 to 2002, he was
with Samsung Electronics, Suwon, South Korea, where he was engaged in the
design and development of code division multiple access and global system for
mobile communications (GSM)/general packet radio service baseband modem
processors. In the summer of 2003, he was with the IBM Thomas J. Watson
Research Center, Yorktown Heights, NY, USA, where he performed extensive radio
channel measurements and analysis at 60 GHz. In the fall of 2005, he was with
the Wireless Mobile System Group, Freescale Semiconductor, Austin, where he
was engaged in the design and test of the worldwide interoperability for
microwave access (WiMAX) medium access control layer. His industry experience
also includes work with the Samsung Advanced Institute of Technology, Kiheung,
South Korea, where he participated in the simulation of multi-input–multi-
output systems for the Third-Generation Partnership Project (3GPP) Long-Term
Evolution (LTE) standard. In October 2006, he joined the Mobile Devices
Technology Office, Motorola, Inc., Austin, where he was involved in algorithm
design and development for the 3GPP LTE mobile systems, including modeling of
3GPP LTE modem processor. Later in 2010, he was with Apple Inc., Cupertino,
CA, USA, where he worked on research and development of next-generation smart
mobile systems. Since March 2011, he has been with the Division of Electrical
Engineering, Hanyang University, Ansan, South Korea, where he is currently a
Professor. He received the Korean Government Overseas Scholarship for his
doctoral studies in the field of electrical engineering.
---|---
|  Jaerock Kwon (M’06–SM’20) received the B.S. and M.S. degrees from Hanyang
University, Seoul, South Korea, in 1992 and 1994, respectively, and the Ph.D.
degree in computer engineering from Texas A&M University, College Station,
USA, in 2009. Between 1994 and 2004, he worked at LG Electronics, SK Teletech,
and Qualcomm Internet Service. From 2009 to 2010, he was a Professor with the
Department of Electrical and Computer Engineering, Kettering University,
Flint, MI, USA. Since 2010, he has been a Professor with the Department of
Electrical and Computer Engineering, University of Michigan-Dearborn, MI, USA.
His research interests include mobile robotics, autonomous vehicles, and
artificial intelligence. Dr. Kwon’s awards and honors include the Outstanding
Researcher Award, Faculty Research Fellowship (Kettering University), and SK
Excellent Employee (SK Teletech). He serves as the President of the Korean
Computer Scientists and Engineers Association in America (KOCSEA), in 2020 and
2021.
---|---