# Automatic AI controller that can drive with confidence: steering vehicle
with uncertainty knowledge

Neha Kumari3,Sumit Kumar4,Sneha Priya1, Ayush Kumar2, Akash Fogla 4
1ICFAI University, 2 Amity University, 3 University of Georgia,4Georgia State
University

##  I ABSTRACT

In safety-critical systems that interface with the real world, the role of
uncertainty in decision-making is pivotal, particularly in the context of
machine learning models. For the secure functioning of Cyber-Physical Systems
(CPS), it is imperative to manage such uncertainty adeptly. In this research,
we focus on the development of a vehicle’s lateral control system using a
machine learning framework. Specifically, we employ a Bayesian Neural Network
(BNN), a probabilistic learning model, to address uncertainty quantification.
This capability allows us to gauge the level of confidence or uncertainty in
the model’s predictions. The BNN based controller is trained using simulated
data gathered from the vehicle traversing a single track and subsequently
tested on various other tracks. We want to share two significant results:
firstly, the trained model demonstrates the ability to adapt and effectively
control the vehicle on multiple similar tracks. Secondly, the quantification
of prediction confidence integrated into the controller serves as an early-
warning system, signaling when the algorithm lacks confidence in its
predictions and is therefore susceptible to failure. By establishing a
confidence threshold, we can trigger manual intervention, ensuring that
control is relinquished from the algorithm when it operates outside of safe
parameters.

### Keywords

Bayesian neural networks, Probabilistic modeling, Torcs, surrogate modeling,
Electronics Control Unit

##  II Introduction

Self-driving vehicle comprises intricate machine learning (ML) and artificial
intelligence (AI) components, functioning as subsystems within a sophisticated
driving system, including a perception system and obstacle detection system.
Although prevalent self-driving systems predominantly employ multiple ML and
AI models for specialized tasks, the paradigm of end-to-end driving systems is
relatively scarce [1]. ML and AI have demonstrated significant efficacy in
control systems[2, 3], predictive modeling for complex phenomena such as
cancer detection and stock market trends[4, 5], and automated design
processes[6]. This study specifically focuses on the utilization of two widely
adopted ML models to govern lateral vehicle motion, with the overarching goal
of scrutinizing their performance under conditions of abundant data
availability. The ensuing comparison aims to serve as a foundational
reference, furnishing insights for AI-based control designers regarding the
judicious selection of models and the discernment of respective strengths and
weaknesses.

For the lateral control of the vehicle, the TORCS open-source car racing
simulator[7] is employed. The data set for this investigation is generated
across diverse speed conditions through the utilization of a conventional
Proportional-Integral-Derivative (PID) controller, with an emphasis on
steering control during the training phase. The dataset encompasses distance
measurements procured from a Light Detection and Ranging (LIDAR) sensor suite,
concomitant with corresponding steering values. A deep learning models, namely
Bayesian Neural network, is selected for training and subsequent testing under
different scenarios. These are the following contributions:

  1. 1.

The Bayesian Neural network-based controller manifests superior generalization
capabilities in instances where access to an extensive dataset is constrained.
It also provide the uncertainty in the prediction.

  2. 2.

The uncertainty is used to decide when the model is not confident, and the
steering control is overtaken by the manual driver.

  3. 3.

When deployed on other similar tracks, the BNN controller successfully drives
on the track without incident. When deployed on the more complex and
convoluted track with hairpin curves and very twisted turns the uncertainty
estimation helped to take manual control when uncertainty was beyond beyond
threshold. It helped in safe navigation even on unseen terrains.

The rest of the paper is organised as follow. Section III formulate the
problem and provide background, approach and training details of controller.
Section IV provides the experimental results of the experimentation. The
related work is discussed in section V and at the end we produce our
conclusion and future direction of researchVI.

##  III Problem Formulation and Approach

###  III-A Problem Formulation

To formalize the problem, we introduce notations. Let $x$ denote the state of
the vehicle estimated by sensors, sampled from an unknown distribution
$P_{x}$. The objective of controller design is to learn a nonlinear function
$f$ mapping $x$ to the control action $c$:

$f:x\mapsto c\;\text{where}\;x\sim P_{x}$

We consider the widely used architecture i.e. deep neural networks based
probabilistic network called Bayesian Neural network as an instances of the
architecture class $\mathcal{A}$. The choice of $f$ is due to two constraints:
first- we want to learn a highly nonlinear control action and second we wnat
some uncertainty in prediction. We treat this problem as a supervised learning
task, utilizing a labeled dataset $D=\\{<x_{i},c_{i}>\\}$ for training. The
dataset is split into training ($D_{train}$) and test ($D_{test}$) sets. The
learning goal is to identify parameters distribution (for neural network) that
effectively represent the training data and perform well on test data and can
also measure the uncertainty in the prediction.

###  III-B Approach

In this section, we prove the background literature on the components used in
the experiment. Torcs Simulator TORCS (The Open Racing Car Simulator) is an
open source 3D racing car simulator that can be use to select and drive a car
on selected track and visualise and log data using a peripheral device or via
a design controller (AI-based or traditional methods). All major aspects of
vehicle dynamics can be accurately simulated, including mass, rotational
inertia, impacts, suspension dynamics, linkages, differentials, friction and
aerodynamics. Simplifying, it perform physical simulations using Euler
integration of dynamical equations using a time discretization level of
$0.002$ seconds. TORCS offers a variety of tracks and cars, sensor and road
types. It is widely used in research community for simulation.
Data generation For data generation purposes, [8] used the TORCS’ asset car
and drove it on the ’Forza’ road track (fig. 1) in order to generate data. For
sensing the environment, Fogla et al [8] deployed multiple LIDAR sensors
attached to the car that can sense the track and the wall. For data generation
purposes, a a tuned PID controller is designed that can drive the car
successfully on a given speed on the track Forza. The data set consists of
input as a set of distances measured by various LIDAR sensors and output as
the steering value. By the finely tuned PID controller, multiple rounds of run
is conducted on this track and the corresponding LIDAR distance data and the
steering values are collected. Our model is trained on this labeled data set.
In our case, we want to train a regression model that estimate and predict the
steering value on the LIDAR distance data during test and run time.

Figure 1: track: Forza

Bayesian neural network, parameters and training  Bayesian Neural Networks
(BNNs) combine the principles of Bayesian statistics and inference with neural
networks. Bayesian inference is a method of statistical inference in which
Bayes’ theorem is used to update the probability for a hypothesis as more
evidence or information becomes available. Bayes’ theorem is stated
mathematically as:

$P(H|E)=\frac{P(E|H)\cdot P(H)}{P(E)}$

where:

  1. 1.

$P(H|E)$ is the probability of hypothesis $H$ given the evidence $E$, known as
the posterior probability.

  2. 2.

$P(H)$ is the probability of hypothesis $H$ being true, known as the prior
probability.

  3. 3.

$P(E|H)$ is the probability of observing evidence $E$ given that hypothesis
$H$ is true, known as the likelihood.

  4. 4.

$P(E)$ is the probability of observing evidence $E$, known as the evidence or
marginal likelihood.

A Bayesian Neural Network (BNN) extends this approach by placing a prior
probability distribution over the weights of the network, rather than
assigning fixed values in case of classical feed forward deep neural network.
This means each weight is considered as a random variable described by a
probability distribution, reflecting our uncertainty about its true value. In
a BNN, the forward pass involves sampling weights from their respective
distributions and using these weights to make predictions. For a given input
$x$, the output $y$ is computed as follows:

$y=f(\mathbf{x},\mathbf{W})$

where $f$ is the neural network function and $\mathbf{W}$ represents the set
of weights and biases sampled from their distributions. Learning in BNNs is
about updating the distributions of the weights. This is done through Bayes’
theorem by calculating the posterior distribution of the weights given the
data:

$P(\mathbf{W}|D)=\frac{P(D|\mathbf{W})\cdot P(\mathbf{W})}{P(D)}$

where:

  1. 1.

$P(\mathbf{W}|D)$ is the posterior distribution of the weights given the data
$D$.

  2. 2.

$P(\mathbf{W})$ is the prior distribution of the weights.

  3. 3.

$P(D|\mathbf{W})$ is the likelihood of the data given the weights.

  4. 4.

$P(D)$ is the evidence or marginal likelihood of the data, often computed as
an integral over all possible weights, which is usually intractable.

Prediction in a BNN involves integrating over all possible weights, weighted
by their posterior probabilities:

$p(y|\mathbf{x},D)=\int p(y|\mathbf{x},\mathbf{W})\cdot
P(\mathbf{W}|D)d\mathbf{W}$

This integral is generally intractable due to the high dimensionality of the
weight space, so approximate methods such as Markov Chain Monte Carlo (MCMC)
or variational inference are used. The true posterior distribution
$P(\mathbf{W}|D)$ is approximated by a more tractable distribution
$q(\mathbf{W}|\theta)$, often chosen to be in a family of distributions that
are easier to sample from, such as Gaussian distributions. The parameters
$\theta$ of this distribution are optimized using variational inference, which
aims to minimize the divergence between $q(\mathbf{W}|\theta)$ and
$P(\mathbf{W}|D)$.

$\text{KL}[q(\mathbf{W}|\theta)||P(\mathbf{W}|D)]$

This is equivalent to maximizing the evidence lower bound (ELBO), which is
often more computationally feasible than computing the posterior directly. The
objective function in variational inference is the Evidence Lower Bound
(ELBO), which is given by:

$\text{ELBO}(\theta)=\mathbb{E}_{q(\mathbf{W}|\theta)}[\log
P(D|\mathbf{W})]-\text{KL}[q(\mathbf{W}|\theta)||P(\mathbf{W})]$

The first term is the expected log likelihood of the data under the
approximate posterior, and the second term is the KL divergence between the
approximate posterior and the prior distribution of the weights. To optimize
the ELBO with respect to the variational parameters $\theta$, we need to
compute its gradient:

$\nabla_{\theta}\text{ELBO}(\theta)=\nabla_{\theta}\mathbb{E}_{q(\mathbf{W}|\theta)}[\log
P(D|\mathbf{W})]-\nabla_{\theta}\text{KL}[q(\mathbf{W}|\theta)||P(\mathbf{W})]$

To apply backpropagation, gradients of random variables (weights) with respect
to the parameters $\theta$ need to be computed. This is facilitated by the
reparameterization trick, which expresses the random variables as
deterministic functions of the parameters and some noise $\epsilon$ that is
independent of the parameters:

$\mathbf{W}=g(\epsilon,\theta)$

where $\epsilon$ is typically sampled from a simple distribution like a
standard Gaussian, and $g$ is a differentiable transformation that maps
$\epsilon$ and $\theta$ to the space of weights. With reparameterization, the
gradient of the expected log likelihood can be approximated by sampling and
can be backpropagated through:

$\nabla_{\theta}\mathbb{E}_{q(\mathbf{W}|\theta)}[\log
P(D|\mathbf{W})]\approx\frac{1}{L}\sum_{l=1}^{L}\nabla_{\theta}\log
P(D|g(\epsilon^{(l)},\theta))$

where $L$ is the number of samples used in the Monte Carlo estimation. Using
gradient-based optimization methods like stochastic gradient descent (SGD) or
Adam, the variational parameters $\theta$ are updated iteratively:

$\theta_{\text{new}}=\theta_{\text{old}}+\alpha\nabla_{\theta}\text{ELBO}(\theta_{\text{old}})$

where $\alpha$ is the learning rate.

Through this process, backpropagation in variational inference enables the
optimization of the variational parameters $\theta$ such that the approximate
posterior $q(\mathbf{W}|\theta)$ gets closer to the true posterior
distribution $P(\mathbf{W}|D)$, all while allowing the use of non-conjugate
priors and complex models such as neural networks. This method is also known
as Stochastic Variational Bayes because of the stochastic nature of the
gradient estimation.

For inference, weights are sampled from an approximation to the posterior, and
then predictions are made based on these samples rather than by performing the
full integration over the weight space. Once we have the approximate
distribution $q(\mathbf{W}|\theta)$, we can sample sets of weights
$\mathbf{W}^{(i)}$ from it:

$\mathbf{W}^{(i)}\sim q(\mathbf{W}|\theta)$

These sampled weights $\mathbf{W}^{(i)}$ are not the exact weights from the
true posterior, but they are drawn from the distribution that we believe
closely approximates the posterior. For each set of sampled weights
$\mathbf{W}^{(i)}$, the BNN can make a prediction for an input $\mathbf{x}$.
This generates a distribution of outputs rather than a single point estimate:

$y^{(i)}=f(\mathbf{x},\mathbf{W}^{(i)})$

where $f$ is the neural network function. The full Bayesian prediction would
integrate over the entire weight space, which is generally intractable.
However, by sampling from the approximate posterior $q(\mathbf{W}|\theta)$, we
can estimate this integral by averaging the predictions from the sampled
weights:

$p(y|\mathbf{x},D)\approx\frac{1}{N}\sum_{i=1}^{N}p(y|\mathbf{x},\mathbf{W}^{(i)})$

where $N$ is the number of weight samples.

This is a Monte Carlo approximation of the integral, which is much more
computationally feasible than the exact integration. The law of large numbers
guarantees that, as $N$ grows, the Monte Carlo approximation converges to the
true integral. The variance in the predictions from the different samples of
weights provides a measure of uncertainty. If the variance is high, it means
the model is less certain about its predictions. This approach allows BNNs to
make probabilistic predictions and quantify uncertainty without explicitly
solving the intractable integral of the full Bayesian treatment. It is
particularly useful in cases where the uncertainty estimate is as critical as
the prediction itself, such as autonomous control systems in self driving
cars.

Figure 2: On the left the classical deep neural network and on the right is
the Bayeisan Neural network ( As shown one of the parameter $w_{11}$ in DNN is
a point value, however for the Bayesian Neural network, it is a Gaussian
probability distribution

Deep Neural Network model, parameters and training DNN has wide ranging
application from image recognition[9], engineering design[10, 11, 6], control
design, anomaly detection[12, 13],etc. The basic architecture of DNN in this
case is the one used in [14]. A DNN is based on a layered network of smaller
computational units called artificial neurons. Artificial neurons, which are
tiny computing units, provide the basis of a multilayer network that powers a
DNN. These neurons are fully linked between the input and output layers and
are arranged in many layers. This experiment uses a fully connected feed-
forward neural network, whose architecture is specified by $a=(L,N,\delta)$,
where $L$ is the number of layers in the network. $L\in N$, where $l\in[L-1]$,
$\delta$ is the activation function $\delta:R\mapsto R$, and $N$ is the set
that reflects the number of neurons in each layer represented by $N_{l}$. A
fully connected feed-forward six-layer neural network with $1$ and
$\\{256,128,64,32,16\\}$ neurons in hidden layers was employed for this
experiment. All hidden neurons have ReLU activation units and the output
neuron is the linear activation unit. The loss function, in this case, is the
mean square error and the weight initialization is Gaussian normal
distribution. The training is done for $500$ epochs by splitting the training
and validation data into $90:10$ and early stopping is used to stop over-
fitting.

Figure 3: Training and validation loss for Bayesian neural network training

##  IV Experimental Results

Once the BNN models are trained, these models are tested on a similar but
another track called ’E-Track 4’ (refer to figure 4). The selection of the
track is based on twists and turns and complexity in the track profile. We set
the target speed of the car to 60 miles/hrs. For the first experiment, we used
both controllers to complete the track without any manual interventions. The
task was to finish one complete lap of this track. One complete lap of this
track is 7.041 Km long with a track width of 15 meters. The BNN controller was
able to complete the task without any crash or off-the-track navigation.

Figure 4: track: E-track 4

In the second experiment, we took a more difficult and track with more hairpin
turns and twists. We monitored the variance in the steering value prediction
and intervened and manually steered when the coefficient of variance becomes
higher than the threshold. The coefficient of variance (CV) measure of the
relative variability of a data set in relation to its mean and is calculated
using following formula :

$CV=\frac{\sigma}{\mu}\times 100$

where $\sigma$ is the standard deviation of the data set and $\mu$ is the mean
of the data set.

The coefficient of variation is useful because it provides a dimensionless
number that allows for comparison of the variability of data sets with
different units of measurement or vastly different means. A lower CV indicates
less variability (more consistency), while a higher CV indicates higher
variability (less consistency).

We observed that the trained random forest controller produced high CoV in
scenarios which was very sharp turns and twist section of road that was never
encountered on the training track. A snapshot of the BNN controller driving
the car on the test track is shown in figure 5.

Figure 5: BNN controller in Torcs and related statistics, the statistics of
our interest are: odd counts -number of samples predictions out of 2 standard
deviations from the mean, total\- total number of odd counts during whole
simulation, steer\- steering value predicted by regressor, std\- standard
deviation of prediction, CoV\- coefficient of variance, that explain the
spread of the distribution.  Figure 6: BNN controller being tested in Torcs
track -Etrack6 Figure 7: BNN controller tested on the track wheel2. It has
some twisted turns as can be seen in the map on right -top corner. At only one
hairpin turns, when controller uncertainty went beyond threshold (set to be
the Coefficient of variance beyond $\pm 3$ continuously for 50 consecutive
simulation steps (each step time is 2 milli seconds) then the manual control
is kicked in. The manual control is relinquished again when the CoV comes
within $\pm 3$ range for 50 consecutive steps.  Figure 8: BNN controller
tested on the track summer. It has highly twisted turns as can be seen in the
map on right -top corner. At some hairpin turns, when controller uncertainty
went beyond threshold (set the Coefficient of variance beyond $\pm 3$
continuously for 50 consecutive steps (each step time is 2 milli seconds) then
the manual control is kicked in. The manual control is relinquished again when
the continuous CoV comes within $\pm 3$ range for 50 consecutive steps.

##  V Related Work

The dominant approach to solving engineering problems are model-driven, like
model-based design[15], model-based control[16], and model-based
optimization[17, 18] etc. With the discovery of data-driven models especially
deep neural networks the domain of model-based engineering is revolutionizing.
In recent times these data-driven models have been used and have worked well
in image classification (AlexNet[9] ), engineering design[19, 20, 21, 22],
autonomous driving car[1, 23, 24, 25], radiology [26], human genome[13], and
many more for developing state of the art control[12, 27, 28] and prediction
systems[14].

According to research[29], self-driving cars could potentially save up to
$35,000$ lives annually in the United States alone by reducing traffic deaths
by up to $99$ percent. $93$ percent of car accidents are the result of driver
mistake, according to information from 2007 US report[30], and rookie
drivers—who seldom have more than $30$ hours of driving experience before
receiving a license—are over-represented in fatal car accidents. Moreover,
crashes among senior drivers rise at the same time because they have a
diminished capacity to assess their surroundings for unforeseen dangers,
according to [29]. Self-driving cars will have been trained using machine
learning data-sets made up of a wide range of driving circumstances and driver
behaviors for the equivalent of hundreds of hours behind the wheel before they
hit the roads[1]. The practical utility of AI-ML model is enormous [31, 32,
33, 34] and a better understanding and insight about the model’s performance
would be useful for practical control designer. Since Deep learning is a data
hungry model, it is not possible to generate an enormous amount of data for
each application. In such cases, this DNN models fail to perform and
alternative learning model can be useful.

##  VI Conclusion and Future Work

###  VI-A Conclusion

In this work, we designed a controller for the lateral control of a vehicle
using AI-ML models. Experiments’ results show that:

  1. 1.

Bayesian neural network based controller is trained and tested in different
track and environment using Torcs simulator. Due to probabilistic nature of
network, BNN can predict uncertainty in prediction and can be used to make
decision about the reliability or confidence of the control action.

  2. 2.

Trained on one track and deployed on other tracks, the BNN-based controller
was able to complete some of the the track without crashing and with out
manual interventions. In some highly twisted tracks , sometime manual
intervention was invoked based on threshold set and prediction confidence
interval.

###  VI-B Future Work

Our experiments are use cases to understand deeper about possibility of using
probabilistic AI model control of safety critical system. For future work, we
want to compare the earlier work [8] with current one based on various
performance metrics.

## References

  * [1] M. Bojarski, D. Del Testa, D. Dworakowski, B. Firner, B. Flepp, P. Goyal, L. D. Jackel, M. Monfort, U. Muller, J. Zhang _et al._ , “End to end learning for self-driving cars,” _arXiv preprint arXiv:1604.07316_ , 2016.
  * [2] P. Abbeel, A. Coates, and A. Y. Ng, “Autonomous helicopter aerobatics through apprenticeship learning,” _The International Journal of Robotics Research_ , vol. 29, no. 13, pp. 1608–1639, 2010.
  * [3] H. Vardhan and J. Sztipanovits, “Rare event failure test case generation in learning-enabled-controllers,” in _2021 6th International Conference on Machine Learning Technologies_ , 2021, pp. 34–40.
  * [4] A. Al Bataineh, “A comparative analysis of nonlinear machine learning algorithms for breast cancer detection,” _International Journal of Machine Learning and Computing_ , vol. 9, no. 3, pp. 248–254, 2019.
  * [5] M. A. Ghazanfar, S. A. Alahmari, Y. F. Aldhafiri, A. Mustaqeem, M. Maqsood, and M. A. Azam, “Using machine learning classifiers to predict stock exchange index,” _International Journal of Machine Learning and Computing_ , vol. 7, no. 2, pp. 24–29, 2017.
  * [6] H. Vardhan and J. Sztipanovits, “Deepal for regression using epsilon-weighted hybrid query strategy,” _arXiv preprint arXiv:2206.13298_ , 2022.
  * [7] B. Wymann, E. Espié, C. Guionneau, C. Dimitrakakis, R. Coulom, and A. Sumner, “Torcs, the open racing car simulator,” _Software available at http://torcs. sourceforge. net_ , vol. 4, no. 6, p. 2, 2000.
  * [8] A. Fogla, K. Kumar, S. Saurav _et al._ , “Vehicle lateral control using machine learning for automated vehicle guidance,” _arXiv preprint arXiv:2303.08187_ , 2023.
  * [9] A. Krizhevsky, I. Sutskever, and G. E. Hinton, “Imagenet classification with deep convolutional neural networks,” _Communications of the ACM_ , vol. 60, no. 6, pp. 84–90, 2017.
  * [10] H. Vardhan, P. Volgyesi, and J. Sztipanovits, “Machine learning assisted propeller design,” in _Proceedings of the ACM/IEEE 12th International Conference on Cyber-Physical Systems_ , 2021, pp. 227–228.
  * [11] S. Kumar, H. Vardhan, S. Priya, and A. Kumar, “Malaria detection using deep convolution neural network,” _arXiv preprint arXiv:2303.03397_ , 2023.
  * [12] H. Vardhan and J. Sztipanovits, “Reduced robust random cut forest for out-of-distribution detection in machine learning models,” _arXiv preprint arXiv:2206.09247_ , 2022.
  * [13] L. Sundaram, H. Gao, S. R. Padigepati, J. F. McRae, Y. Li, J. A. Kosmicki, N. Fritzilas, J. Hakenberg, A. Dutta, J. Shon _et al._ , “Predicting the clinical impact of human mutation with deep neural networks,” _Nature genetics_ , vol. 50, no. 8, pp. 1161–1170, 2018.
  * [14] H. Vardhan and J. Sztipanovits, “Deep learning based fea surrogate for sub-sea pressure vessel,” in _2022 6th International Conference on Computer, Software and Modeling (ICCSM)_. IEEE, 2022, pp. 36–39.
  * [15] H. Neema, H. Vardhan, C. Barreto, and X. Koutsoukos, “Web-Based Platform for Evaluation of Resilient and Transactive Smart-Grids,” in _2019 7th Workshop on Modeling and Simulation of Cyber-Physical Energy Systems (MSCPES)_. IEEE, 2019, pp. 1–6.
  * [16] C. Brosilow and B. Joseph, _Techniques of model-based control_. Prentice Hall Professional, 2002.
  * [17] H. Vardhan, N. M. Sarkar, and H. Neema, “Modeling and optimization of a longitudinally-distributed global solar grid,” in _2019 8th International Conference on Power Systems (ICPS)_. IEEE, 2019, pp. 1–6.
  * [18] H. Neema _et al._ , “Design and Simulation Platform for Evaluation of Grid Distribution System and Transactive Energy,” in _Proceedings of the 6th Annual Symposium on Hot Topics in the Science of Security_. ACM, 2019, p. 15.
  * [19] H. Vardhan, U. Timalsina, P. Volgyesi, and J. Sztipanovits, “Data efficient surrogate modeling for engineering design: Ensemble-free batch mode deep active learning for regression,” _arXiv preprint arXiv:2211.10360_ , 2022\.
  * [20] S. M. Moosavi, K. M. Jablonka, and B. Smit, “The role of machine learning in the understanding and design of materials,” _Journal of the American Chemical Society_ , vol. 142, no. 48, pp. 20 273–20 287, 2020.
  * [21] H. Vardhan and J. Sztipanovits, “Search for universal minimum drag resistance underwater vehicle hull using cfd,” _arXiv preprint arXiv:2302.09441_ , 2023\.
  * [22] M. J. Volk, I. Lourentzou, S. Mishra, L. T. Vo, C. Zhai, and H. Zhao, “Biosystems design by machine learning,” _ACS synthetic biology_ , vol. 9, no. 7, pp. 1514–1533, 2020.
  * [23] C. Badue, R. Guidolini, R. V. Carneiro, P. Azevedo, V. B. Cardoso, A. Forechi, L. Jesus, R. Berriel, T. M. Paixao, F. Mutz _et al._ , “Self-driving cars: A survey,” _Expert Systems with Applications_ , vol. 165, p. 113816, 2021.
  * [24] J. Ni, Y. Chen, Y. Chen, J. Zhu, D. Ali, and W. Cao, “A survey on theories and applications for self-driving cars based on deep learning methods,” _Applied Sciences_ , vol. 10, no. 8, p. 2749, 2020.
  * [25] S. Shalev-Shwartz, S. Shammah, and A. Shashua, “On a formal model of safe and scalable self-driving cars,” _arXiv preprint arXiv:1708.06374_ , 2017.
  * [26] G. W. Gross, J. M. Boone, V. Greco-Hunt, and B. Greenberg, “Neural networks in radiologic diagnosis. ii. interpretation of neonatal chest radiographs.” _Investigative Radiology_ , vol. 25, no. 9, pp. 1017–1023, 1990.
  * [27] Z. Wu, A. Tran, D. Rincon, and P. D. Christofides, “Machine learning-based predictive control of nonlinear processes. part i: theory,” _AIChE Journal_ , vol. 65, no. 11, p. e16729, 2019.
  * [28] S. M. Tahsien, H. Karimipour, and P. Spachos, “Machine learning based solutions for security of internet of things (iot): A survey,” _Journal of Network and Computer Applications_ , vol. 161, p. 102630, 2020.
  * [29] Y. A. Momtaz, R. Kargar, R. Hosseiny, and R. Sahaf, “Rate and pattern of road traffic accidents among older and younger drivers,” _Healthy aging research_ , vol. 7, no. 2, p. e18, 2018.
  * [30] N. T. S. Facts, “Us department of transportation, national highway traffic safety administration, 2008,” 2007.
  * [31] H. Vardhan, P. Volgyesi, and J. Sztipanovits, “Fusion of ml with numerical simulation for optimized propeller design,” _arXiv preprint arXiv:2302.14740_ , 2023.
  * [32] T. Dreossi, D. J. Fremont, S. Ghosh, E. Kim, H. Ravanbakhsh, M. Vazquez-Chanlatte, and S. A. Seshia, “Verifai: A toolkit for the formal design and analysis of artificial intelligence-based systems,” in _Computer Aided Verification: 31st International Conference, CAV 2019, New York City, NY, USA, July 15-18, 2019, Proceedings, Part I 31_. Springer, 2019, pp. 432–442.
  * [33] H. Vardhan, P. Volgyesi, and J. Sztipanovits, “Constrained bayesian optimization for automatic underwater vehicle hull design,” _arXiv preprint arXiv:2302.14732_ , 2023.
  * [34] R. Vaishya, M. Javaid, I. H. Khan, and A. Haleem, “Artificial intelligence (ai) applications for covid-19 pandemic,” _Diabetes & Metabolic Syndrome: Clinical Research & Reviews_, vol. 14, no. 4, pp. 337–339, 2020.