# Enhancing SUMO simulator for simulation based testing and validation of
autonomous vehicles

Arpan Kusari1, Pei Li1, Hanzhi Yang2, Nikhil Punshi3, Mich Rasulis1, Scott
Bogard1 and David J. LeBlanc1 1 Arpan Kusari, Pei Li, Mich Rasulis, Scott
Bogard and David J. LeBlanc are with the University of Michigan Transportation
Research Institute, University of Michigan, Ann Arbor, MI 48109, USA kusari,
peiliesg, mich, sbogard<EMAIL_ADDRESS>Hanzhi Yang is with the Department
of Mechanical Engineering, University of Michigan, Ann Arbor, MI 48109, USA
<EMAIL_ADDRESS>Nikhil Punshi is with the Department of Aerospace
Engineering, University of Michigan, Ann Arbor, MI 48109, USA
<EMAIL_ADDRESS>
###### Abstract

Current autonomous vehicle (AV) simulators are built to provide large-scale
testing required to prove capabilities under varied conditions in controlled,
repeatable fashion. However, they have certain failings including the need for
user expertise and complex inconvenient tutorials for customized scenario
creation. Simulation of Urban Mobility (SUMO) simulator, which has been
presented as an open-source AV simulator, is used extensively but suffer from
similar issues which make it difficult for entry-level practitioners to
utilize the simulator without significant time investment. In that regard, we
provide two enhancements to SUMO simulator geared towards massively improving
user experience and providing real-life like variability for surrounding
traffic. Firstly, we calibrate a car-following model, Intelligent Driver Model
(IDM), for highway and urban naturalistic driving data and sample
automatically from the parameter distributions to create the background
vehicles. Secondly, we combine SUMO with OpenAI gym, creating a Python package
which can run simulations based on real world highway and urban layouts with
generic output observations and input actions that can be processed via any AV
pipeline. Our aim through these enhancements is to provide an easy-to-use
platform which can be readily used for AV testing and validation.

##  I INTRODUCTION

With the rise in research into autonomous vehicles (AVs), there has been an
increased focus on the effective testing and validation of AVs. Traditional
testing and validation for vehicles is performed via repeated closed track
testing and open road testing. However, researchers have shown that for AVs,
the vehicles would need to be driven hundreds of millions to hundreds of
billions of miles to demonstrate reliability which would make AV market
penetration improbable [1]. Large scale simulation based testing provides a
lucrative alternative for fast and efficient prototyping without the need for
a physical vehicle and actual testing in a real world [2]. There are a lot of
advantages offered by simulation based testing mostly in terms of
controllability, repeatability and scalability which can make it essential for
testing highly complex systems such as AVs. As a result, there are various
commercial and open-source driving simulation platforms that have been
developed for AV testing such as CARLA, Metamoto, Airsim etc. While these
simulation software contain sophisticated, intelligent behavior models for
vehicular traffic, bicyclists and pedestrians, there are two inherent problems
for those who want to utilize these for large scale AV testing and validation:

  * •

most of these simulation environments are set up for small scale, repeatable
scenarios often in urban layouts which constrain the number of scenarios that
can be reliably tested and

  * •

they often have a complicated interface which requires a lot of man-hours to
learn and run with the AV pipeline in a modular fashion.

Simulation of Urban Mobility (SUMO) has been proposed as an alternative for
testing autonomous vehicles in their simulation environment [3]. SUMO is
currently the most popular 2D microscopic traffic simulator which can simulate
multimodal traffic, import large scale real-world maps from mapping platforms
such as OpenStreetMap (OSM) and can be interacted with using an socket based
communication interface, TraCI [4]. The main advantages in utilizing SUMO for
large scale AV testing would be access to a large variety of Operational
Design Domains (ODDs) that can help AV developers to find edge cases in the
pipeline and native support for different car-following models which can
provide real-world like behavior of the background vehicles (BVs).

However, there are still some gaps in using SUMO simulator as a reliable
testing environment for AVs:

  * •

the BVs behave in a predetermined fashion which is inconsistent with the real-
world variability and

  * •

there is a steep learning curve in running SUMO and setting it up to interact
with the AV pipeline.

We propose two enhancements to the SUMO simulator intended to address these
two issues. The first enhancement is to learn the parameter distributions of
well-known car-following models from naturalistic driving databases and
randomly sampling from them to simulate BV behavior. The second enhancement is
in providing scenario abstraction by combining the strengths of SUMO simulator
with that of OpenAI gym, a very popular and highly accessible toolkit for
reinforcement learning research. Our focus on offering these enhancements is
on convenience and accessibility while offering real-world like variability.

There has been some work done in combining OpenAI Gym with driving simulators,
one specific popular instance being the highway-env [5] environment. Highway-
env is a Python based simulation environment which is built on top of OpenAI
gym and contains different environments such as multilane highway, merge,
roundabout etc. However, building a custom environment is difficult to do from
scratch and the simulation does not scale well with the increase in vehicles
and road network layout. Also the background vehicles are sampled from the
same parameter values and therefore, offer almost no variability.

The paper is set up as follows: Section II provides the background for
estimating parameters of car-following models using Bayesian parameter
estimation from naturalistic data and showcases results from in-depth analysis
of large amounts of data collected in highway and urban scenarios; Section III
explores in detail the combination of the SUMO simulator with the OpenAI gym
to create a scenario abstraction pipeline and finally, Section IV provides in-
depth discussion and some paths towards future work.

##  II BAYESIAN PARAMETER ESTIMATION OF CAR-FOLLOWING MODELS

Car-following models have been used extensively to model the behavior of human
drivers when following a preceding vehicle in different traffic conditions.
Car-following models have been shown to have variety of uses in different
fields of transportation such as traffic flow (macroscopic) [6] and vehicle
dynamics (microscopic) [7]. These models aim at revealing interactions between
an ego vehicle and its corresponding leading vehicle within the same lane,
thus, creating the longitudinal profile of the ego vehicle over time.
Individual values of the model parameters also reveal the driving
characteristics. With increasing amount of naturalistic data of individual
vehicles collected in different kinds of scenarios, there has been a larger
focus towards calibrating car-following models from high accuracy vehicle
trajectory data.

There has been a lot of research done on parameter estimation of car-following
models from naturalistic datasets under different scenario types. Earlier
research utilized macroscopic data to calibrate car-following models, which
made the models less precise. Helbing and Tilch proposed the General Force
model and calibrated it with Bosch GmbH car-following data collected using
radars [8]. Kesting and Treiber calibrated and compared intelligent driver
model (IDM) with Optimal Velocity model using the Bosch GmbH dataset using
genetic algorithm [9]. They were able to show that the errors between the
actual trajectory data and the predicted values were below 30% and the intra-
driver variability accounts for the larger amount of error than the inter-
driver variability. In the past decade, there has been an explosion of studies
for calibration of car-following models using different kinds of naturalistic
data and under different scenarios [10, 11, 12].

In order to simulate real-world driving, we have to sample different parameter
values from the distributions for each individual vehicle. We choose IDM as
the car-following model in this research and estimate parameters using
naturalistic driving data (NDD) from the Safety Pilot Model Deployment (SPMD)
program [13]. Since we require the entire distribution to sample from, we
employ Bayesian statistics in the form of Markov Chain Monte Carlo (MCMC)
methods. MCMC methods employ sequential sampling using Markov Chain starting
from a prior probability distribution and moving towards the desired
distribution. We utilize a particular MCMC method, Metropolis-Hastings (M-H)
algorithm, to determine parameter distributions for highway and urban
scenarios using SPMD data. We then sample from these distributions to create
background vehicles for the SUMO-Gym environment automatically. We describe
the process in the subsections below.

###  II-A Intelligent Driver Model

Car-following models have been developed from the 1950s onwards in order to
understand the driving patterns of individual drivers [14]. Numerous car-
following models have been developed in the last seven decades such as the
Gazis-Herman-Rothery (GHR) model [15], IDM [16], the optimal velocity model
[17], and the models proposed by [18], [19], and [20]. Currently, the one of
the most popular car-following models is the IDM based on the desired measures
model which assumes that each individual driver has a desired following
distance and following speed and the driver tries to minimize the gap between
the actual measures and the desired. The acceleration for a vehicle $n$ is
given as:

$a_{n}(t)=a_{max}^{(n)}\bigg{(}1-\bigg{(}\frac{v_{n}(t)}{v_{des}(t)}\bigg{)}^{\delta}-\bigg{(}\frac{d_{des}(t)}{d_{front}(t)}\bigg{)}^{2}\bigg{)}$
(1)

where $a_{max}^{(n)}$ is the maximum acceleration of the vehicle, $v_{n}(t)$
is the actual speed of the vehicle, $v_{des}(t)$ is the desired speed of the
vehicle, $d_{front}$ is the distance to the preceding vehicle and $d_{des}$ is
the desired distance to the preceding vehicle. The desired following distance
is dependent on various factors: speed of the vehicle ($v_{n}(t)$), speed
difference ($\Delta v$), maximum acceleration ($a_{max}^{(n)}$), comfortable
deceleration ($a_{comf}^{(n)}$), minimum time headway ($T$) and minimum
distance to preceding vehicle ($d_{min}$). Mathematically, it can represented
as:

$d_{des}=d_{min}+v_{n}(t)T+\frac{v_{n}(t)\Delta
v}{2\sqrt{a_{max}^{(n)}a_{comf}^{(n)}}}$ (2)

The calibration of this model requires estimation of six parameters:
$a_{max}^{(n)}$, $a_{comf}^{(n)}$, $v_{des}(t)$, $d_{min}$, $T$ and $\delta$,
all of them should be positive.

###  II-B Data collection and filtering

SPMD was a project undertaken by the U.S. Department of Transportation (USDOT)
to evaluate the effectiveness of vehicle-to-vehicle communication technology
(V2V) in safety applications. University of Michigan Transportation Reseearch
Institute (UMTRI) was the primary contractor and was in charge of device
fitting on to vehicles of participant drivers, data collection and data
management. The SPMD database is one of the largest databases in the world
that recorded naturalistic driving behaviors over 34.9 million travel miles
from 2842 equipped vehicles in Ann Arbor, Michigan. In the database, there are
98 sedans equipped with the data acquisition system (DAS). Since we only
calibrate the car-following model in this research, we need the longitudinal
data including the acceleration and speed of the ego-vehicle and the distance
and speed difference to the preceding vehicle. As the first filtering step, we
utilize the GPS data of fifty DAS-equipped vehicles to select portions of
trajectories in highways and urban settings. This yields a large collection of
trajectories in both bins which need to be further subsampled to feed into the
M-H model. We filter out any start-and-stop trajectories and portions where
lane changes occur from the trajectories. For the urban trajectories, we do
not consider the trajectory portion near the intersections.

###  II-C Bayesian Calibration

The calibration for a car-following model based on trajectory data is
essentially a non-linear optimization problem, in order to search for the best
values of model parameters that minimize the measure of distance between the
observed and the simulated ego-vehicle’s behavior [21]. The optimization
problem has the following form:

$\theta^{*}=arg\min_{\theta\in D}f\bigg{(}Y^{obs},Y(\theta)^{sim}\bigg{)}$ (3)

where $\theta$ is the vector of parameters, $D$ is the domain of possible
model parameters, $f$ is the distance function between observed ($Y^{obs}$)
and computed acceleration ($Y(\theta)^{sim}$).

From Equation 1, it is abundantly clear that it is a complex, nonstandard
multivariate distribution. For such distributions, a popular method of
calibration is utilizing Bayesian sequential sampling methods known as Markov
Chain Monte Carlo (MCMC) methods. In this research, we utilize the M-H
algorithm developed by Metropolis, Rosenbluth, Rosenbluth, Teller, and Teller
[22] and subsequently generalized by Hastings [23]. The algorithm has two main
ideas that combine together to provide a simple yet robust method -

  * •

Transition function - at each step, there needs to be an update of parameter
values i.e. given current parameter values, we need to calculate proposed
parameter values. This is accomplished by provision of transition function. We
utilize a very simple way of generating a new parameter values by adding a
random noise term. The new parameter values are then calculated as
$\theta_{prop}=\theta_{curr}+\xi$, where
$\xi\backsim\mathcal{N}(0,\Sigma_{prop})$. The transition function is known as
the random walk kernel and given as

$Q(\theta_{prop}|\theta_{curr})=\frac{1}{\sqrt{2\pi}}e^{-0.5(\theta_{prop}-\theta_{curr})^{2}}$
(4)

  * •

Acceptance of proposed values - Given the proposed values, we have to decide
whether to accept them as the new parameter values or sample again. This is
done by calculating the acceptance probability:

$\alpha=min\bigg{(}1,\frac{\Pi(\theta_{prop})Q(\theta_{prop}|\theta_{curr})}{\Pi(\theta_{curr})Q(\theta_{curr}|\theta_{prop})}\bigg{)}$
(5)

where $\Pi$ is the stationary distribution. The proposed parameter values are
accepted with probability $\alpha$. Otherwise the current parameter values are
retained.

###  II-D Simulation results

We collect simulation data for 50 vehicles with 200 observations each with all
vehicles running IDM with the following parameters: $a_{max}=3m/s^{2}$,
$a_{comf}=5m/s^{2}$, $v_{des}=35m/s$, $d_{min}=10m$, $T=2s$ and $\delta=4$.
Fig. 1 shows the results for the simulation using M-H algorithm. Fig. 1(a)
provides the initial estimate of acceleration for an individual vehicle and
the final estimate after running M-H algorithm for 100,000 iterations. We find
that the M-H algorithm can approximate the observed acceleration closely, with
the maximum deviation of 5%. The final distributions of each parameter is
given in Fig. 1(b). We find that the M-H algorithm can approximate the actual
parameters very closely with the means of the distributions lying closely to
the actual parameters. In order to understand the convergence of the Markov
chains, we plot the samples over the iterations as given in Fig. 1(c). The M-H
Markov chains are considered convergent if they samples look like random noise
about a straight line. Finally, the autocorrelation plot in Fig. 1(d) shows
values of the lag-k autocorrelation function (ACF) against increasing k
values. The autocorrelation is an important diagnostic for convergence of MCMC
chains and the lesser the correlation the better the convergence. Thus, we can
show that M-H algorithm works really well to approximate the original values
and can provide meaningful distributions.

Figure 1: IDM parameter calibration using simulation data - (a) observed vs
estimated acceleration; (b) parameter distributions; (c)parameter sampling
over iterations; (d) autocorrelation between parameters

###  II-E Real data results

We present the distribution results from running the M-H algorithm on the SPMD
highway and urban data in Fig. 2 and Fig. 3. It is evident from the plots,
that the desired speed in highway is an unimodal distribution centered around
30 m/s, whereas in the urban data, it is a bimodal distribution with two peaks
centered on 18 and 20 m/s. Another large difference is in the minimum
distance, where the ranges lie between 65-75 m in highway driving data while
they lie between 15-45 m in the urban data. We provide a comparison table
(Table I) between the mean values found by us to the ones found in the
literature. From the table, we see that while there is agreement between the
values of desired speed, acceleration and $\delta$, the values vary for other
parameter values.

TABLE I: Comparison of values between our method and given in [24] for highways Parameter values | Our method | Schakel et. al.[24]
---|---|---
$v_{des}[m/s]$ | 29.7 | 34.4
$T[s]$ | 2.0 | 1.2
$d_{min}[m]$ | 63.9 | 7.0
$a_{max}[m/s^{2}]$ | 1.2 | 1.0
$a_{comf}[m/s^{2}]$ | 2.0 | 1.67
$\delta$ | 4.0 | 4.0
Figure 2: Parameter distributions from SPMD highway data Figure 3: Parameter
distributions from SPMD urban data

###  II-F Assimilating into SUMO-Gym

Given the parameter distributions for highway and urban scenarios, we need to
sample parameters for individual vehicles from the output distributions. Since
most of these parameters have multimodal distributions, we sample directly
from the histograms. We sample by randomly selecting a bin according to its
probability mass and then sampling a value from an uniform distribution over
the chosen bin. These are the parameter values that we input into the SUMO-Gym
environment. We create routes using randomTrips.py script in SUMO which
generates a set of random trips for a given network. We use an automated
script to place a certain number of vehicles in each route with parameter
values sampled from the parameter distribution file as described above. The
script outputs a route file which is used by the SUMO-Gym environment.

##  III SCENARIO ABSTRACTION COMBINING SUMO AND OPENAI GYM

###  III-A Introduction to OpenAI Gym

Gym is an open source Python library developed by OpenAI [25]. It was
originally designed for developing and testing reinforcement learning
algorithms. Gym provides a convenient way to communicate between algorithms
and environments by using its standard API by creating a common interface for
all the simulation environments and abstracting the process of controlling the
agents in the simulation via easy-to-use commands. This paper aims to leverage
the power of OpenAI Gym to abstract the SUMO simulation environment and
propose a novel AV testing platform named SUMO-Gym. While SUMO users need to
create a road network file and a route file containing the dynamic traffic
elements for each new simulation, Similar to other environments of OpenAI Gym,
users of SUMO-Gym only need to prepare their AV pipeline without setting up
the simulation environment. Fig. 4 compares the process of preparing a SUMO
simulation from scratch to that using SUMO-Gym.

Figure 4: Comparison of developing a traffic simulation from scratch and using
SUMO-Gym

###  III-B Introduction to SUMO-Gym

SUMO-Gym is a portable, user-friendly AV simulation platform. Users of SUMO-
Gym can test their AV pipeline to generate actions compatible with the package
based on generic observations from SUMO-Gym. Fig. 6 shows the interactions
between different components of SUMO-Gym. The first part of SUMO-Gym is the
simulation scenario. SUMO-Gym provides multiple simulation scenarios which
cover different ODDs, including highways, major arterials, and local
collectors.

###  III-C Scenario abstraction with SUMO-Gym

SUMO-Gym lets users choose preset scenario types (such as highway and urban).
For each type of the scenarios, we create a library of road network layouts
and dynamic route files, and the subsequent configuration files. A random
configuration file of the scenario type is chosen and loaded by SUMO-Gym. Two
types of scenarios were included in SUMO-Gym, highway and urban scenario. For
example, the road network of highways around Ann Arbor was used to develop an
highway scenario while the traffic network of downtown Orlando was used to
develop an urban scenario. The layout along with the OSM map is shown in Fig.
5. In addition, SUMO-Gym has the ability to operate using customized traffic
scenarios based on users’ needs. In general, a simulation scenario has three
crucial files: network, demand, and configuration file. Specifically, the
network file defines the road network of the simulation. The demand file
consists of various traffic routes, which contain vehicles’ trajectory
information. Moreover, to make the demand file can reflect realistic driving
environment, parameters of the car-following model was calibrated using
naturalistic driving data. A configuration file is then used to provide
network and demand files as inputs to the SUMO simulation. The testing process
is then run based on the particular configuration file.

Figure 5: Example of an urban scenario layout

###  III-D Inheriting environment from Gym

The second component of SUMO-Gym is the environment, a Python class which
inherits from Gym and provides a convenient way to run SUMO simulation with
few generic commands. The environment contains multiple Python functions to
initialize, reset, render, start, step through, and stop a simulation. The
environment uses TraCI [26] to control and communicate with SUMO. The
environment provides observations currently in the form of an observation
matrix to users of SUMO-Gym. The observation matrix is a $V\times F$ array for
$V$ neighboring vehicles (leaders and followers to the ego vehicle in each
lane) and $F$ features: presence, x-position, y-position, velocity in x and
velocity in y. Users of SUMO-Gym can use their customized AV algorithms to
generate actions for the environment. Currently, we utilize continuous action
in the form of longitudinal and lateral acceleration at each step. Note that
the clipping of acceleration is to be performed by the user before inputting
into SUMO-Gym. We compute the change in x and y positions of the ego vehicle
in the road aligned coordinate system and transform them to SUMO’s global
coordinate system. The ego vehicle is then moved via the moveToXY function in
TraCI. During each step, we also check whether the ego vehicle has veered off
the road or is in a collision. In such cases, we stop the simulation and send
pertinent information back to the user.

Figure 6: Components of SUMO-Gym

Fig. 7 provides detailed information about the process diagram of SUMO-Gym.
Users of SUMO-Gym need to make the selection of scenarios and pass this
information to SUMO-Gym. SUMO-Gym first loads the scenario and start the
simulation as the initialization. Then, SUMO-Gym resets the simulation and
returns initial observations to users. Using actions provided by users’ AV
pipeline, SUMO-Gym will iteratively take a simulation step, which updates the
status of ego-vehicle and returns observations. In the current framework, we
have a placeholder for reward function. If so desired, the user can modify the
function and receive a scalar reward at each simulation step. In addition,
SUMO-Gym can render the simulation using SUMO-GUI, which helps users visually
observe the simulation.

Figure 7: SUMO-Gym diagram

##  IV DISCUSSION AND CONCLUSIONS

We present a software solution for AV based simulation and testing geared
towards user convenience and providing real world variability. We provide the
software as an open source package available as a docker container
https://github.com/arpan-kusari/Naturalistic-SUMO-Gym. The docker container
installs SUMO, OpenAI Gym and SUMO-Gym package. In addition to the SUMO-Gym
package, we also include a Python script with the ego vehicle running a simple
IDM model to test the package. By abstracting the entire simulation
environment, SUMO-Gym makes SUMO more user-friendly and lets users solely
focus on preparing their AV pipeline. Our hope in releasing this software is
to provide a platform which can accelerate the process of verification and
validation of autonomous vehicles.

In terms of future work, we would like to increase the number of scenarios in
each scenario type and increase functionality of the simulation environment.
Specifically, we would like to broaden the type of observations and actions
that are inputted and outputted by the simulation. Our other aim in releasing
this software is to solicit opinions from the community at large in terms of
features and functionalities that could be incorporated into the package.

## References

  * [1] N. Kalra and S. M. Paddock, “Driving to safety: How many miles of driving would it take to demonstrate autonomous vehicle reliability?” _Transportation Research Part A: Policy and Practice_ , vol. 94, pp. 182–193, 2016.
  * [2] E. Thorn, S. C. Kimmel, M. Chaka, B. A. Hamilton, _et al._ , “A framework for automated driving system testable cases and scenarios,” United States. Department of Transportation. National Highway Traffic Safety …, Tech. Rep., 2018.
  * [3] Q. Lu, T. Tettamanti, D. Hörcher, and I. Varga, “The impact of autonomous vehicles on urban traffic network capacity: an experimental analysis by microscopic traffic simulation,” _Transportation Letters_ , vol. 12, no. 8, pp. 540–549, 2020.
  * [4] D. Krajzewicz, J. Erdmann, M. Behrisch, and L. Bieker, “Recent development and applications of sumo-simulation of urban mobility,” _International journal on advances in systems and measurements_ , vol. 5, no. 3&4, 2012.
  * [5] E. Leurent, “An environment for autonomous driving decision-making,” https://github.com/eleurent/highway-env, 2018.
  * [6] R. W. Rothery, “Car following models,” _Trac Flow Theory_ , 1992.
  * [7] F. E. Gunawan, “Two-vehicle dynamics of the car-following models on realistic driving condition,” _Journal of Transportation Systems Engineering and Information Technology_ , vol. 12, no. 2, pp. 67–75, 2012.
  * [8] D. Helbing and B. Tilch, “Generalized force model of traffic dynamics,” _Physical review E_ , vol. 58, no. 1, p. 133, 1998.
  * [9] A. Kesting and M. Treiber, “Calibrating car-following models by using trajectory data: Methodological study,” _Transportation Research Record_ , vol. 2088, no. 1, pp. 148–156, 2008.
  * [10] M. M. Rahman, “Application of parameter estimation and calibration method for car-following models,” Ph.D. dissertation, Clemson University, 2013.
  * [11] M. Treiber and A. Kesting, “Microscopic calibration and validation of car-following models–a systematic approach,” _Procedia-Social and Behavioral Sciences_ , vol. 80, pp. 922–939, 2013.
  * [12] V. Papathanasopoulou and C. Antoniou, “Towards data-driven car-following models,” _Transportation Research Part C: Emerging Technologies_ , vol. 55, pp. 496–509, 2015.
  * [13] D. Bezzina and J. Sayer, “Safety pilot model deployment: Test conductor team report,” _Report No. DOT HS_ , vol. 812, no. 171, p. 18, 2014.
  * [14] M. Brackstone and M. McDonald, “Car-following: a historical review,” _Transportation Research Part F: Traffic Psychology and Behaviour_ , vol. 2, no. 4, pp. 181–196, 1999.
  * [15] D. C. Gazis, R. Herman, and R. W. Rothery, “Nonlinear follow-the-leader models of traffic flow,” _Operations research_ , vol. 9, no. 4, pp. 545–567, 1961\.
  * [16] M. Treiber, A. Hennecke, and D. Helbing, “Congested traffic states in empirical observations and microscopic simulations,” _Physical review E_ , vol. 62, no. 2, p. 1805, 2000.
  * [17] M. Bando, K. Hasebe, A. Nakayama, A. Shibata, and Y. Sugiyama, “Dynamical model of traffic congestion and numerical simulation,” _Physical review E_ , vol. 51, no. 2, p. 1035, 1995.
  * [18] W. Helly, “Simulation of bottlenecks in single-lane traffic flow,” 1959.
  * [19] P. G. Gipps, “A behavioural car-following model for computer simulation,” _Transportation Research Part B: Methodological_ , vol. 15, no. 2, pp. 105–111, 1981.
  * [20] R. Wiedemann, “Simulation des strassenverkehrsflusses.” 1974.
  * [21] V. Punzo, B. Ciuffo, and M. Montanino, “Can results of car-following model calibration based on trajectory data be trusted?” _Transportation research record_ , vol. 2315, no. 1, pp. 11–24, 2012.
  * [22] N. Metropolis, A. W. Rosenbluth, M. N. Rosenbluth, A. H. Teller, and E. Teller, “Equation of state calculations by fast computing machines,” _The journal of chemical physics_ , vol. 21, no. 6, pp. 1087–1092, 1953.
  * [23] W. K. Hastings, “Monte carlo sampling methods using markov chains and their applications,” 1970.
  * [24] W. J. Schakel, V. L. Knoop, and B. van Arem, “Integrated lane change model with relaxation and synchronization,” _Transportation Research Record_ , vol. 2316, no. 1, pp. 47–57, 2012.
  * [25] G. Brockman, V. Cheung, L. Pettersson, J. Schneider, J. Schulman, J. Tang, and W. Zaremba, “Openai gym,” 2016.
  * [26] A. Wegener, M. Piórkowski, M. Raya, H. Hellbrück, S. Fischer, and J.-P. Hubaux, “Traci: an interface for coupling road traffic and network simulators,” in _Proceedings of the 11th communications and networking simulation symposium_ , 2008, pp. 155–163.


# FerroHEMTs: High-Current and High-Speed All-Epitaxial AlScN/GaN
Ferroelectric Transistors

J. Casamento1†, K. Nomoto2†, T. S. Nguyen1, H. Lee2, C. Savant1, L. Li2, A.
Hickman2, T. Maeda3,
J. Encomendero2, V. Gund2, A. Lal2, J. C. M. Hwang1,2, H. G. Xing1,2,3, and D.
Jena1,2,3
1Department of Materials Science and Engineering, Cornell University, Ithaca,
NY 14853, USA,
2School of Electrical and Computer Engineering, Cornell University, Ithaca, NY
14853, USA
3Kavli Institute at Cornell for Nanoscale Science, Cornell University, Ithaca,
NY 14853, USA
†Equal Contribution. Email<EMAIL_ADDRESS><EMAIL_ADDRESS>
Abstract—We report the first observation of ferroelectric gating in AlScN
barrier wide-bandgap nitride transistors. These FerroHEMT devices realized by
direct epitaxial growth represent a new class of ferroelectric transistors in
which the semiconductor is itself polar, and the crystalline ferroelectric
barrier is lattice-matched to the substrate. The FerroHEMTs reported here use
the thinnest nitride high-K and ferroelectric barriers to date to deliver the
highest on-currents at 4 A/mm, and highest speed AlScN transistors with
$f_{MAX}>$ 150 GHz observed in any ferroelectric transistor. The FerroHEMTs
exhibit hysteretic $I_{d}-V_{gs}$ loops with subthreshold slopes below the
Boltzmann limit. A control AlN barrier HEMT exhibits neither hysteretic, nor
sub-Boltzmann behavior. While these results introduce the first epitaxial
high-K and ferroelectric barrier technology to RF and mm-wave electronics,
they are also of interest as a new material platform for combining memory and
logic functionalities in digital electronics.

##  I Introduction

The report of ferroelectric sputtered AlScN in 2019 [1] indicated the
tantalizing possibility of introducing ferroelectric barriers into GaN RF and
mm-wave transistors. One can pursue this either by integrating sputtered
layers on epitaxial channels, or by direct epitaxy of AlScN on GaN transistors
[2]. Direct epitaxial AlScN on GaN revealed high piezoelectric coefficients
[3]. When the leakage was reduced with increased Sc source purity [4],
epitaxial hi-K dielectric constants up to 20 were discovered [5]. These
epitaxial AlScN layers exhibited a low coercive field of $\sim$ 1 MV/cm [6]
compared to those of non-epitaxial sputtered layers reported in [1]. Other
reports by epitaxy showed higher coercive fields [7], and memory functionality
[8]. The RF and mm-wave properties of AlScN barrier HEMTs have been studied
[9, 10, 11], but ferroelectric transistor behavior has not been reported.

Here we report ferroelectric gating behavior of epitaxial AlScN barrier HEMTs
and contrast its ultra high-current, high-speed, and sub-Boltzmann performance
to a control HEMT with a non-ferroelectric AlN barrier. The measured FerroHEMT
behavior is consistent with a low coercive field of epitaxial AlScN. All
measurements presented in this work were performed at room temperature.

##  II Epitaxial Growth

Fig. 1(a) shows a test layer structure comprising of a 200 nm unintentionally
doped (UID) GaN layer grown by MBE followed by a 100 nm AlScN layer with 18%
Sc. A polarization-induced 2D electron gas (2DEG) of density $1.8\times
10^{13}$/cm2 and mobility of $377$ cm2/V$\cdot$s was observed in this
heterostructure by Hall-effect measurement at room temperature. Using a
deposited top electrode and the 2DEG as a bottom electrode, and applying
positive-up-negative-down (PUND) voltage pulses and tracking the currents, the
polarization-electric field (P-E) loops extracted are shown in Fig. 1(b) which
indicate the epitaxial AlScN layer on UID GaN is ferroelectric with a coercive
field $E_{c}\sim 0.9$ MV/cm.

A control AlN/GaN HEMT and a 14% targeted Sc composition AlScN/AlN/GaN
FerroHEMT structure were grown by MBE directly on semi-insulating 6H-SiC
substrates with a 300 nm AlN nucleation layer and a 1 $\mu$m GaN buffer layer
using methods reported in [5] to study ferroelectric gating behavior. Figs.
2(a) and (c) show the corresponding energy band diagrams from the top surface,
calculated using self-consistent Schrödinger-Poisson solutions. 2DEG channels
are expected at the heterojunctions as shown as $\psi_{1}^{2}$ at respective
depths. To form Ohmic contacts to these 2DEGs, lithographically defined
etching and regrowth of heavily doped n+ GaN ($N_{d}\sim$ 1020/cm3 Si) was
performed by MBE. Figs. 2(b) and (d) show the measured 2DEG densities,
mobilities, and sheet resistances over various dies of the wafer measured by
room-temperature Hall effect. The 2DEG densities are consistent with the
expected values from the calculation.

##  III Device design and fabrication

Figs. 3(a) and (b) show the resistances of metal-regrown n+GaN (low) and
n+GaN-2DEG (moderate, not exceptionally low). Figs. 4(a) and (b) show the
control AlN barrier HEMT and the AlScN barrier FerroHEMT cross sections
respectively. Figs. 4(c) shows the representative device process flow of HEMTs
with regrown contacts. A SiO2/Cr hard mask defined the source/drain regions.
Ti/Au source/drain was deposited on the n+GaN, and Ni/Au gate was deposited.
Electron beam lithography (EBL) process was performed to fabricate T-gate
devices for RF and mm-wave performance. Figs. 4(d) and the inset show SEM
images of the final transistor structures. Gate lengths ranged from 90 nm to
18 $\mu$m.

##  IV Results and discussion

Figs. 5(a) and (b) show the measured $I_{d}-V_{ds}$ output characteristics and
$I_{d}-V_{gs}$ transfer characteristics of the control AlN/GaN HEMT sample
respectively. An on current of $\sim 1.5$ A/mm with an on resistance of
$R_{on}=1.34$ $\Omega\cdot$mm, an on/off ratio of $10^{6}$ limited by gate
leakage, with a threshold voltage of $\sim-3.5$ V were observed. Fig. 5(c)
shows a peak transconductance of $\sim 0.5$ S/mm with a barrier thickness of 2
nm GaN + 2 nm AlN. The subthreshold slope shown in Fig. 5(d) indicates very
good normal transistor behavior, grazing the ideal Boltzmann limit of $\sim
60$ mV/decade. The control HEMT thus exhibits excellent performance, as borne
out by its RF performance discussed subsequently.

Fig. 6 shows the transistor characteristics when a 5 nm thick epitaxial AlScN
barrier layer is added between the AlN and the GaN cap layer, as indicated in
Figs. 4(a)-(b). From Fig. 6(a), the maximum $I_{d}$ still reaches $\sim 1.5$
A/mm at the same gate voltage, in spite of a more than double barrier
thickness of 2 nm GaN + 5 nm AlScN + 2 nm AlN compared to the control sample.
This is a result of the high-K dielectric property of AlScN as was reported in
[5]. The $I_{d}-V_{ds}$ curves indicate a higher output conductance, but a far
larger difference from the control sample is observed in the transfer
characteristics in Fig. 6(b). A counterclockwise (CCW) hysteresis loop
develops in the subthreshold characteristics. A sub-Boltzmann steep slope of
23.6 mV/decade is observed for the on$\rightarrow$off voltage sweep. Such
repeatable loops are observed in multiple devices. This translates to a
hysteretic transconductance curve as seen in Fig. 6(c) and drain current as
seen in Fig. 6(d). Note that though the peak transconductance is lower than
the control HEMT, the hi-K AlScN helps maintain a high value in spite of a
more than double barrier thickness.

The hysteresis window of the threshold voltage is between $1.0-2.0$ V as seen
in Figs. 6(c) and (d). These observations are strong signatures of both high-K
dielectric, and ferroelectric gating behavior. Based on the AlScN barrier
thickness, a voltage drop $E_{c}\times t_{AlScN}\sim 0.5$ V across the AlScN
layer is consistent with a low $E_{c}\sim 1.0$ MV/cm. A large portion of the
gate voltage still drops across the GaN and AlN layers between the gate metal
and the 2DEG channel.

Fig. 7(a) shows the hysteresis loop measured on Die 7 of Fig. 2 (d). Several
CCW hysteresis loops based on the voltage step of the measurement are shown in
Fig. 7(a), and the corresponding substhreshold slopes are plotted in Figs.
7(b) and (c). While the majority of the sub-Boltzmann steep slopes are
observed in the leftgoing voltage sweeps, some also appear in the rightgoing
sweeps. It is well known that trapping behavior could lead to false
conclusions of ferroelectricity. The CCW loops for n-channels is a strong
evidence of ferroelectric FETs. Moreover, the absence of such behavior in the
control HEMT sample, and the symmetry, voltage width, and sub-Boltzmann slopes
of the FerroHEMT conclusively indicates ferroelectric gating in the all-
epitaxial AlScN/GaN devices.

Fig. 8(a) shows the high-frequency characteristics of the AlN barrier control
HEMT. It exhibits excellent cutoff frequencies of $f_{T}/f_{MAX}=126/304$ GHz
for a gate length of $L_{g}=90$ nm. The device dimensions and bias conditions
are indicated in the plot. Fig. 8(b) shows the measured values for the
FerroHEMT of similar dimensions but different bias conditions due to the
shifted threshold characteristics. It exhibits lower cutoff frequencies of
$f_{T}/f_{MAX}=78/156$ GHz. Even though the values of the AlScN FerroHEMT are
lower than the control AlN barrier HEMT, they are the highest reported to date
for AlScN barrier transistors, as indicated in Fig. 8(c). Morever, they are
the fastest ferroelectric transistors reported to date: a maximum FerroHEMT
$f_{MAX}=168$ GHz is measured. The higher speed of the control sample is due
to the (expected) higher maximum $g_{m,ext}=0.616$ S/mm of the control HEMT
compared to $g_{m,ext}=0.475$ S/mm for the FerroHEMT, indicating higher speed
FerroHEMTs are possible with further scaling. Fig. 8(d) shows the scaling of
the output current $I_{d}^{max}$ in the AlScN FerroHEMTs when $L_{g}$ is
scaled from 18 $\mu$m to 90 nm. An exceptionally high value of 2.5 A/mm is
observed for an optical gate length of 1 $\mu$m. The deep submicron EBL gates
reach record values of 4 A/mm at 90 nm gate length. These record high on-
currents are enabled by the high 2DEG density generated by the large
difference in polarization between GaN and the epitaxial AlScN layers.

##  V Conclusions and future work

The moderate FerroHEMT channel mobility in Fig. 2(d) can be improved by
3$\times$ for better RF and mm-wave performance. The high-K value of AlScN
will increase the breakdown voltage in FerroHEMTs by reducing the gate leakage
[5]. A negative DIBL effect is predicted [12], but this will be achievable
with careful geometrical design of future epitaxial FerroHEMTs. Thus
FerroHEMTs with thinnest epitaxial ferroelectric barriers show steep
subthreshold slopes, and deliver the highest on-currents (4 A/mm), and highest
speed ($f_{MAX}>$ 150 GHz) in all ferroelectric transistors. They present a
new class of transistors with the potential to blur the boundaries between
memory, logic, and communication devices.

Figure 1: (a) Epitaxial AlScN/GaN heterostructure indicating a polarization-
induced 2DEG at the heterojunction. (b) Measured $P-E$ loops on diodes with
top electrode and the 2DEG of (a) as the bottom electrode. Ferroelectric loops
are observed for the heterostructure. Figure 2: (a) Energy band diagram of
control HEMT. (b) Hall-effect data of control HEMT. (c) Energy band diagram of
FerroHEMT. (d) Hall-effect data of FerroHEMT. Though there are some non-
uniformities, the measured 2DEG densities are consistent with the
calculations. Figure 3: (a) Contact resistances for the control HEMT between
the metal and the regrown n+GaN regions and the n+GaN regions and 2DEGs. (b)
Corresponding contact resistances for the FerroHEMT structure.  Figure 4: (a)
Control HEMT cross section schematic showing the regrown contacts, gate, and
channel regions. (b) Corresponding schematic for the FerroHEMT has an extra 5
nm AlScN layer in the barrier layer. (c) The representative device process
flow of III-Nitride HEMTs with regrown contacts. (d) SEM image of a processed
HEMT, and inset shows Zoomed-in image of a T-gate. Figure 5: Measured
characteristics of the control HEMT with an AlN barrier: (a) Output
characteristics, (b) Transfer characteristics in the log and (inset) linear
scale, (c) Transconductance vs. $V_{gs}$, and (d) subthreshold slope vs.
$I_{d}$.  Figure 6: Measured characteristics of the FerroHEMT with an AlScN
barrier: (a) Output characteristics, (b) Transfer characteristics in the log
scale showing a counterclockwise hysteresis and sub-Boltzmann slope, (c)
Transconductance vs. $V_{gs}$ showing hysteresis, and (d) Linear $I_{d}$ vs
$V_{gs}$ showing hysteresis. Figure 7: (a) Subthreshold characteristics of a
FerroHEMT showing counterclockwise hysteresis loops and steep slopes for
various bias steps. Sub-Boltzmann behavior observed in the subthreshold slopes
for (b) 20 mV gate sweep steps and (c) 10 mV gate sweep steps.  Figure 8:
Measured cutoff frequencies at $L_{g}=90$ nm of (a) the Control HEMT, and (b)
the AlScN barrier FerroHEMT. (c) Speed benchmarks of AlScN HEMTs with previous
reports [9, 10, 11]. (d) Scaling of output current with gate length. In
addition to being the fastest AlScN HEMTs, the FerroHEMT data reported here
also represents the highest speed and highest on-current ferroelectric
transistors realized to date.

## Acknowledgment

This work was supported in part by the SRC and DARPA through the Joint
University Microelectronics Program (JUMP), by the DARPA TUFEN program, and
performed at the Cornell NanoScale Facility, an NNCI member supported by NSF
Grant No. NNCI-2025233.

## References

  * [1] S. Fichtner _et al._ , _Journal of Applied Physics_ , vol. 125, p. 114103, 2019.
  * [2] M. Hardy _et al._ , _Applied Physics Letters_ , vol. 110, p. 162104, 2017\.
  * [3] J. Casamento _et al._ , _Applied Physics Letters_ , vol. 117, p. 112101, 2020.
  * [4] J. Casamento _et al._ , _APL Materials_ , vol. 9, p. 091106, 2021.
  * [5] J. Casamento _et al._ , _Applied Physics Letters_ , vol. 120, p. 152901, 2022.
  * [6] J. Casamento _et al._ , _arXiv:2105.10114_ , 2021.
  * [7] P. Wang _et al._ , _Applied Physics Letters_ , vol. 118, p. 223504, 2021\.
  * [8] D. Wang _et al._ , _Adv. Electronic Materials_ , vol. 40, p. 2200005, 2022\.
  * [9] T. E. Kazior _et al._ , _IEEE MTT-S IMS_ , pp. 1136–1139, 2019.
  * [10] A. J. Green _et al._ , _IEEE Electron Device Lett._ , vol. 40, p. 1056, 2019\.
  * [11] J. Cheng _et al._ , _IEEE Trans. Electron Devices_ , vol. 68, p. 3333, 2021\.
  * [12] Y.-H. Liao _et al._ , _IEEE Electron Device Lett._ , vol. 40, p. 1860, 2019\.