# Dynamics Models in the Aggressive Off-Road Driving Regime

Tyler Han, Sidharth Talia, Rohan Panicker, Preet Shah, Neel Jawale, Byron
Boots
University of Washington This material is based upon work supported by the
National Science Foundation Graduate Research Fellowship under Grant No.
DGE-2140004. Any opinions, findings, and conclusions or recommendations
expressed in this material are those of the author(s) and do not necessarily
reflect the views of the National Science Foundation.BeamNG simulator
interface and model code can be found at: (https://github.com/prl-
mushr/BeamNGRL). Dataset can be found at: https://tinyurl.com/5n9y4akc

###### Abstract

Current developments in autonomous off-road driving are steadily increasing
performance through higher speeds and more challenging, unstructured
environments. However, this operating regime subjects the vehicle to larger
inertial effects, where consideration of higher-order states is necessary to
avoid failures such as rollovers or excessive impact forces. Aggressive
driving through Model Predictive Control (MPC) in these conditions requires
dynamics models that accurately predict safety-critical information. This work
aims to empirically quantify this aggressive operating regime and its effects
on the performance of current models. We evaluate three dynamics models of
varying complexity on two distinct off-road driving datasets: one simulated
and the other real-world. By conditioning trajectory data on higher-order
states, we show that model accuracy degrades with aggressiveness and simpler
models degrade faster. These models are also validated across datasets, where
accuracies over safety-critical states are reported and provide benchmarks for
future work.

##  I INTRODUCTION

In off-road autonomous driving, Model Predictive Control (MPC) is widely used
for fast planning and control to address problems such as obstacle avoidance
or uneven terrain. Historically, MPC has performed well in practice,
accommodating unknown and approximate dynamics through frequent model
prediction and re-optimization [1, 2, 3, 4]. However, the effectiveness of
this approach critically depends on the accuracy of the model prediction step.
Particularly in unstructured environments, MPC methods often suffer due to
models that inadequately capture safety-critical dynamics [2, 5]. Nonetheless,
aggressive driving aims to maximize system performance without inducing
critical failure modes. In unstructured off-road driving, uneven and nonplanar
terrain geometries can cause destabilizing impact forces, wheel airtime, and
loss of control [6, 1]. While driving slower can prevent these failures, an
understanding of vehicle dynamics can mitigate them even at high speeds [1].

Figure 1: This work evaluates dynamics models of varying complexity in
challenging and unstructured environments, both simulated (BeamNG, left) and
real (Washington, right).

Prior works have not extensively studied this regime for off-road driving.

The NATO Reference Mobility Model (NRMM) has been historically used to
characterize ground traversability and difficulty [7, 8, 9]. These models
perform static two-dimensional analyses over distinct ground shapes and
provide estimates of vertical accelerations and their effect on expected
traversal speeds. However, in the aggressive driving regime, speeds change
quickly, ground geometry varies as a continuum, and inertial forces are
exerted in all three dimensions.

In addition, factors such as complex steering mechanisms, suspension, tire
slippage, diverse driving conditions, and deformable terrain can make
analytical dynamics models an extensive engineering effort. Recently,
researchers have employed learned dynamics models to address these issues [3,
10, 11]. In these works, various model architectures are validated against
either positional or velocity errors in both simulated and real environments.
As the field advances towards more aggressive driving and maneuvers, we stress
that validation on only these states is not sufficient.

Off-road driving, or traversability in general, is a complex problem that
fundamentally depends upon both perception and control. For this work, we
limit our scope, assuming reliable perception, to better study aggressiveness
for dynamics modeling. In the real world, we use [12] to achieve this, while
in simulation the perception is the ground truth obtained directly from the
simulator.

We demonstrate that off-road driving is an increasingly difficult dynamics
modeling problem in the aggressive regime. Moreover, we provide results using
analytic and learned dynamics models on off-road driving datasets. Our
contributions are:

  1. 1.

A quantitative study of Aggressiveness and its effect on analytic (no-slip and
slip) and learned dynamics models for off-road driving.

  2. 2.

Safety-critical evaluation of dynamics models on simulated (BeamNG) and real
(Washington) aggressive driving data with steep slopes and banks, high speeds,
vegetation, and diverse terrain.

##  II Literature Review

Aggressive control behavior has been previously explored for on-road
autonomous vehicles, small-scale off-road vehicles, drones, and manipulators
[13, 14, 15, 16]. Interestingly, both quantitative and qualitative definitions
vary from one work to the next. For instance, Mellinger et al. [15] suggest
trajectories that are momentarily unstable (due to some agile maneuver) but
recover to a stable state are aggressive. On the other hand, [14] provides a
rigorous definition for a small-scale wheeled robot, proposing a safety region
defined by the stability limits of an LQR steering controller. All works
generally do agree on some fundamental characteristics of aggressive control
behavior: the autonomy operates at high speeds, maintains safety, and can
saturate actuators reliably. We note that in the off-road setting, stability
can be impractical to model due to highly nonlinear dynamics and diverse
conditions [16, 17, 12, 1].

Rather than studying stability conditions, failure modes can instead be
modeled. For instance, there is substantial work in rollover prevention in the
automotive safety testing industry. Lateral force modeling is used to predict
and prevent rollover failures through metrics such as the static stability
factor, lateral load transfer ratio, and time to rollover [18, 19, 20]. As
proposed in [1, 6], modeling additional components of acceleration and angular
rates is critical to failure prevention in the aggressive off-road regime. By
more rigorously defining this regime in this work, we study its impact on
various dynamics model classes.

Learning dynamics for robot modeling and optimal control has also been
extensively studied [21]. To make model learning more tractable and efficient,
prior work often takes advantage of known holonomic constraints through hybrid
models that combine kinematic and learned functions. This approach has been
shown to demonstrate good experimental results including complex applications
like drifting for a small-scale off-road platform [2]. Gibson et al. employ an
LSTM to learn a multi-step predictive model to be used in optimal real-time
sample-based control for multiple environments [3]. However, the model is
validated using only positional and yaw accuracy. Maheshwari et al. [10] learn
a model that is trained on low-velocity data and demonstrates generalization
to higher-velocity test scenarios. The authors also demonstrate the importance
of predicting slippage, showing a large divergence in aggressive flat turns
between a kinematic and learned model.

##  III Dynamics models

For car-like vehicles with Ackermann steering mechanisms [22], there exist
many models, such as the point mass model, bicycle model, as well as the four-
wheel model [23]. Of these, we consider the bicycle model class. Within this
class, we consider the No-slip model, slip-based models, and Terrain-
conditioned models. These models at their core describe body-frame rates, such
as body frame velocity, rotation rates, or accelerations. These accelerations
are then transformed into the world frame using the vehicle orientation, and
gravity and other inertial effects are added using the predicted or derived
rotation rates. For brevity, we refer the reader to the work by [24] which
describes these transformations and compensations in detail.

Figure 2: Illustration of the coordinate frame used by dynamics models

####  III-1 SE(3) no-slip based bicycle model

This model refers to the no-slip approximate bicycle model with its velocities
and positions projected in 3D using the elevation map. Prior works such as [1,
10] have also used such a model. Here:

$\displaystyle\begin{split}V^{b}_{x}&=V_{w},V^{b}_{y}=0,V^{b}_{z}=0,\quad\omega_{z}=V^{b}_{x}*\tan(\delta)/L_{fr}\end{split}$
(1)

Where $V_{w}$ represents the wheel speed, $V^{b}$ represents the body frame
velocity, $\omega_{z}$ represents the body frame rotation rate around the Z
axis, $\delta$ represents the steering angle in radians, and $L_{fr}$
represents the wheelbase of the car.

####  III-2 SE(3) slip based approximate model

The slip-based approximate model uses a single-track bicycle model as
described by [25], which considers non-planar surfaces, with a simplified
Pacejka tire model [26].

The body frame Forces $F^{b}_{x},F^{b}_{y},F^{b}_{z}$ and rotational
acceleration $\dot{\omega}^{b}_{z}$ are given by:

$\displaystyle\begin{split}F^{b}_{x}&=F_{xr}+F_{xf}\cos(\delta)-F_{yf}\sin(\delta)+mg\sin(\theta)\\\
F^{b}_{y}&=F_{yr}+F_{yf}\cos(\delta)+F_{xf}\sin(\delta)+mg\sin(\phi)\\\
F^{b}_{z}&=m(g\cos{\beta}-V_{x}\omega_{y}+V_{y}\omega_{x})\\\
\dot{\omega}^{b}_{z}&=((F_{xf}\sin(\delta)+F_{yf}\cos(\delta))L_{f}-F_{yr}L_{r})/J_{z}\end{split}$
(2)

Where the terms denoted by $F$ and $L$ represent forces and lengths
respectively, described in Fig2, $m$ represents mass, and $J_{z}$ represents
the mass moment of inertia around its Z axis. These lateral and longitudinal
accelerations are then used to update the body frame velocity $V^{b}$, which
is transformed into the world frame to update the position. We omit details
regarding exact implementation for brevity and refer the reader to prior works
such as [27, 24] that have also used such a model.

####  III-3 Terrain conditioned dynamics modeling

We consider the approach presented in the work by [28] as our fully-learned
baseline. However, for simplicity, we do not re-implement a probabilistic
ensemble (PE) dynamics model [29] and do not consider the history of the past
states. We believe this version represents the core idea and note that it
performs better than the analytical baselines. In this approach, the model
takes the patch of terrain directly under the vehicle $P$, the vehicle’s
intrinsic state and control inputs, and learns to predict the change in
velocity and rotation rates for the next timestep by taking a single step loss
on the velocity and rotation rates.

$\displaystyle\begin{split}\Delta
V^{b},\Delta\omega^{b}=f(V^{b},\omega^{b},\phi,\theta,P,\delta,V_{w})\end{split}$
(3)

During execution, the predicted delta velocity and rotation rates are
integrated to obtain the velocity, orientation, and position using the
“Explicit Kinematic Layer” in [28].

##  IV Aggressiveness

###  IV-A Definition

As prior works have corroborated qualitatively, aggressiveness is generally
considered to be operating at the dynamic limits of the vehicle [16, 13, 15,
17]. Following this intuition, we define aggressive driving to be trajectories
constrained by these dynamic limits, or higher-order states (e.g. velocity,
acceleration, etc.). Let $\dot{\mathbf{x}}_{\text{min}}$ and
$\dot{\mathbf{x}}_{\text{max}}$ be lower and upper bounds on the
differentiated state $\dot{\mathbf{x}}_{t}$ at time $t$.

$\displaystyle\small\tau\coloneqq$
$\displaystyle\begin{bmatrix}\mathbf{x}_{t_{1}}&\mathbf{x}_{t_{1}}&\cdots&\mathbf{x}_{t_{H}}\end{bmatrix}$
(4) s.t.
$\displaystyle\dot{\mathbf{x}}_{\text{min}}\preceq\dot{\mathbf{x}}_{t_{i}}\preceq\dot{\mathbf{x}}_{\text{max}},\quad
i=1,\dots,H$ (5)
$\displaystyle\tau=\operatorname*{arg\,min}_{\tau^{\prime}\in\Phi}C(\tau^{\prime})$
(6)

where the cost function $C(\cdot)$ in (6) represents a combination of
competing objectives, such as moving toward a goal and/or avoiding obstacles.
In practice, this competing objective will make (5) tight [1, 6, 30, 16].
Without assuming knowledge about $C(\cdot)$, our primary interest is in box
constraint (5). These constraints represent fundamental constraints on the
system, be it mechanical or human tolerable limits. For off-road driving,
constraints must be placed on several higher-order states to reliably maintain
control and safety in the aggressive regime. For example, lateral
accelerations are limited by the critical rollover acceleration [19, 18, 20]
and angular velocities and vertical accelerations are limited by airtime and
impact tolerances [1, 6].

###  IV-B Measuring Aggressiveness with Free Energy

Without inductive biases, expressive model classes generally require a diverse
dataset that provides adequate coverage of the relevant state space. However,
there are a few complications with seeking coverage of (5) outright; including
information about the geometry and semantics can result in exploding
dimensionality, parts of the constrained space may not be reachable or
observable at all, or the bounds $\dot{\mathbf{x}}_{\text{min}}$ and
$\dot{\mathbf{x}}_{\text{max}}$ may not be known or can be subject to the data
collection policy’s preferences. To evaluate whether a trajectory $\tau$
improves coverage of our constrained state space, we employ the free-energy
function [31] to score $\tau$ using its distance from $\mathcal{D}$ as
determined by distributions $p_{i}$,

$\displaystyle
E(\tau)=-T\cdot\log{\sum^{K}_{i}\exp{\left\\{(p_{i}\circ\boldsymbol{\phi}[i])(\tau)/T\right\\}}}$
(7)

where $T$ is the temperature hyperparameter and
$\boldsymbol{\phi}:\mathbb{R}^{n\times H}\rightarrow\mathbb{R}^{K}$ is a
feature function that captures the observed bounds,
$\boldsymbol{\phi}(\tau)=\begin{bmatrix}\max_{t}{\dot{\mathbf{x}}_{t}[k_{1}]}&\min_{t}{\dot{\mathbf{x}}_{t}[k_{1}]}&\ldots&\max_{t}{\dot{\mathbf{x}}_{t}[k_{d}]}&\min_{t}{\dot{\mathbf{x}}_{t}[k_{d}]}\end{bmatrix}$.
Note that we choose our state components through $k_{i}$ rather than include
all states to reduce feature space dimensionality and noise.

##  V Experimentation

Figure 3: Freeze-frames of aggressive trajectories in BeamNGRL dataset.

###  V-A Aggressiveness and Dynamics Models

TABLE I: Model Performance on Datasets Across States |  |  $H$-Max Normed Error ($H$-MNE)
---|---|---
Dataset | Model |  Accel. ($\mathrm{m}\mathrm{/}\mathrm{s}^{2}$) |  Ang. Vel. ($\mathrm{r}\mathrm{a}\mathrm{d}\mathrm{/}\mathrm{s}$) |  Velocity ($\mathrm{m}\mathrm{/}\mathrm{s}$) |  Position ($\mathrm{m}$) | Roll ($\mathrm{r}\mathrm{a}\mathrm{d}$) | Pitch ($\mathrm{r}\mathrm{a}\mathrm{d}$) | Yaw ($\mathrm{r}\mathrm{a}\mathrm{d}$)
BeamNGRL | NoSlip3D | 19.24 $\pm$ 13.97 | 1.89 $\pm$ 1.23 | 2.86 $\pm$ 2.05 | 22.20 $\pm$ 15.09 | 0.24 $\pm$ 0.15 | 0.23 $\pm$ 0.15 | 1.92 $\pm$ 1.93
Slip3D | 7.98 $\pm$ 4.71 | 1.19 $\pm$ 0.88 | 3.27 $\pm$ 2.44 | 11.63 $\pm$ 8.80 | 0.20 $\pm$ 0.13 | 0.17 $\pm$ 0.13 | 1.41 $\pm$ 1.63
Learned | 4.79 $\pm$ 1.62 | 0.61 $\pm$ 0.23 | 2.59 $\pm$ 1.10 | 10.39 $\pm$ 6.12 | 0.18 $\pm$ 0.14 | 0.16 $\pm$ 0.12 | 0.96 $\pm$ 1.45
Washington | NoSlip3D | 7.30 $\pm$ 5.48 | 0.59 $\pm$ 0.45 | 8.25 $\pm$ 7.67 | 28.67 $\pm$ 24.82 | 0.18 $\pm$ 0.14 | 0.10 $\pm$ 0.08 | 0.71 $\pm$ 0.96
Slip3D | 5.04 $\pm$ 2.06 | 0.34 $\pm$ 0.21 | 5.89 $\pm$ 3.68 | 16.51 $\pm$ 10.89 | 0.18 $\pm$ 0.14 | 0.08 $\pm$ 0.04 | 0.59 $\pm$ 0.76
Learned | 4.57 $\pm$ 1.79 | 0.24 $\pm$ 0.15 | 2.19 $\pm$ 3.24 | 11.36 $\pm$ 10.89 | 0.18 $\pm$ 0.14 | 0.07 $\pm$ 0.05 | 0.40 $\pm$ 0.97

We perform a study of three classes of dynamics models over trajectories
quantified by our aggressiveness metric (7). The initial dataset $\mathcal{D}$
contains roughly 2000 4-second trajectories in which the vehicle drives over
mildly uneven terrain, at average speeds of $8\pm
1.5~{}$\mathrm{m}\mathrm{/}\mathrm{s}$$ and vertical accelerations of $9.8\pm
2.7~{}$\mathrm{m}\mathrm{/}\mathrm{s}^{2}$$. We choose $k_{i}$ so that the
feature vector $\phi$ is conditioned on the entries of the acceleration
vector. Each $p_{i}$ is then fit to $\mathcal{D}$ using a Gaussian defined on
the feature space determined by $\phi[i]$. This process allows us to quantify
the aggressiveness of additional trajectories relative to $\mathcal{D}$ as
outlined in section IV-B. The learned model is also trained using
$\mathcal{D}$. The remainder of the dataset’s trajectories is collected with
speeds between $7-9~{}$\mathrm{m}\mathrm{/}\mathrm{s}$$, traversing shallower
and steeper slopes.

Each model $f$ is validated on each of these trajectories (on a held-out set
for the learned model) with respect to the ground truth trajectory and the
maximum normed error over the horizon ($H$-MNE). Explicitly,

$\operatorname*{{MaxNormedError}_{H}}(f,\tau)=\max_{\mathbf{x}^{\text{gt}}_{t}\in\tau_{\text{gt}}}\left\lVert\mathbf{x}^{\prime}_{t}-\mathbf{x}_{t}^{\text{gt}}\right\rVert$
(8)

where $\mathbf{x}^{\prime}_{t}$ is obtained from rolling out the model from
the ground truth initial state, i.e.
$\mathbf{x}^{\prime}_{t}=f(\mathbf{x}^{\prime}_{t-1})$ for $t=1,\dots,H$ where
$\mathbf{x}^{\prime}_{0}\coloneqq\mathbf{x}_{0}^{\text{gt}}$. The maximum is
taken over the horizon to reflect the box constraints (5) as any violation
anywhere is safety-critical.

Figure 4: Model performance on validation trajectories. Each marker
corresponds to the validation of a model on a single trajectory. A cubic
polynomial is fit to each model’s performance to show the general trend over
increasing Free Energy Values $E(\tau)$. Higher energy indicates more out-of-
distribution (more aggressive) relative to the initial dataset.

The results of this experiment are shown in fig. 4. With increasing energy
(greater acceleration magnitudes), not only does the performance degrade but
the performance across models of varying complexity diverge. This empirically
confirms our intuition that aggressive driving is a challenging modeling
problem with a nontrivial range of unmodeled dynamics.

###  V-B Model Learning with Aggressive Data

We evaluate the Learned model on two datasets, named the BeamNGRL and
Washington datasets. The BeamNGRL dataset is released with this work and its
collection process is described in section V-A.

The Washington dataset is collected with a full-size Polaris S4 1000 RZR
equipped with IMU, GPS, visual odometry, and a perception network [12] for
terrain mapping. State and sensor messages are synced, or linearly
interpolated as necessary, at a rate of 10 Hz to produce the dataset from
driving data with speeds up to 14 m/s. The time horizon for each trajectory is
5 seconds. The dataset encompasses diverse terrain geometries, ranging from
flat surfaces to steep inclines and to banking and twisting trails. Track
indents, ditches, and sharp elevation changes also occur alongside these
trails. Throughout the dataset, the vehicle additionally interacts with grassy
areas, dirt paths, trees, small rocks, logs, bushes and regions densely
populated with vegetation.

The timestep size for each trajectory in both datasets is 0.1 seconds. The
results of the models are shown in Table I. The numbers displayed are $H$-MNE
errors computed through (8). Vector states (i.e. acceleration, angular
velocity, velocity, position) are demarcated in bold text. It can be observed
in table I that with increasing model complexity, the errors go down. It can
also be observed that there exists a large gap in position error between the
NoSlip3D and Slip3D models for the BeamNGRL dataset, implying that accounting
for wheel slippage addresses most of this error. The velocity errors in the
BeamNGRL dataset are higher for Slip3D than for NoSlip3D due to the first-
order integration of small errors in acceleration. The learned model does not
suffer the same fate as Slip3D in velocity predictions due to lower
acceleration and angular velocity errors. Note that the errors in velocity for
the real dataset may be higher due to errors in system identification, as we
assume vehicle parameters such as friction to be constant throughout the
dataset. It is also observed that the gap between the models is much smaller
for the Washington dataset than it is for the synthetic dataset from BeamNG
[32]. This is a consequence of the driving in the synthetic dataset being far
more aggressive, as there are no safety concerns for the driver in the
simulation. We provide video demonstrations to show examples of simulated low
energy 111https://tinyurl.com/knzy8kzn, medium
energy222https://tinyurl.com/2p993ktu, and high
energy333https://tinyurl.com/ynx8rbhx driving.

##  VI Conclusion

Using BeamNG, we collect a realistic and diverse aggressive off-road driving
dataset. This dataset is used in a quantitative study on the effect of
aggressiveness on models with various modeled dynamics. An energy score
conditioned on higher-order states is proposed for quantifying the
aggressiveness of trajectories. Our results show that model predictions
degrade with increasing aggressiveness and that simpler models degrade faster.

Next, we evaluate the performance of these dynamics models on two different
off-road datasets; the BeamNGRL and the Washington dataset. Importantly, the
models are validated on their accuracy over higher-order states, which are
critical in the aggressive regime.

We acknowledge that the definitions and metrics provided in this work can be
further developed. For instance, semantic or perceptual information (e.g.
grass, dirt, gravel) is readily available through state-of-the-art perception
networks [12, 33]. This information can better differentiate operating regimes
and improve the performance of dynamics models. We aim to incorporate this in
future work.

## References

  * [1] Tyler Han et al.  “Model Predictive Control for Aggressive Driving Over Uneven Terrain”  arXiv, 2023  DOI: 10.48550/arXiv.2311.12284
  * [2] Grady Williams et al.  “Information theoretic MPC for model-based reinforcement learning”  In _2017 IEEE International Conference on Robotics and Automation (ICRA)_ Singapore: IEEE, 2017, pp. 1714–1721  DOI: 10.1109/ICRA.2017.7989202
  * [3] Jason Gibson et al.  “A Multi-step Dynamics Modeling Framework For Autonomous Driving In Multiple Environments”  arXiv, 2023  DOI: 10.48550/arXiv.2305.02241
  * [4] Edoardo Pagot, Mattia Piccinini and Francesco Biral  “Real-time optimal control of an autonomous RC car with minimum-time maneuvers and a novel kineto-dynamical model”  In _2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)_ Las Vegas, NV, USA: IEEE, 2020, pp. 2390–2396  DOI: 10.1109/IROS45743.2020.9340640
  * [5] Juraj Kabzan, Lukas Hewing, Alexander Liniger and Melanie N. Zeilinger  “Learning-Based Model Predictive Control for Autonomous Racing”  In _IEEE Robotics and Automation Letters_ 4.4, 2019, pp. 3363–3370  DOI: 10.1109/LRA.2019.2926677
  * [6] Hojin Lee, Taekyung Kim, Jungwi Mun and Wonsuk Lee  “Learning Terrain-Aware Kinodynamic Model for Autonomous Off-Road Rally Driving With Model Predictive Path Integral Control”  In _IEEE Robotics and Automation Letters_ 8.11, 2023, pp. 7663–7670  DOI: 10.1109/LRA.2023.3318190
  * [7] Daniel W. Carruth, Clayton T. Walden, Christopher Goodin and Sara C. Fuller  “Challenges in Low Infrastructure and Off-Road Automated Driving”  In _2022 Fifth International Conference on Connected and Autonomous Driving (MetroCAD)_ , 2022, pp. 13–20  DOI: 10.1109/MetroCAD56305.2022.00008
  * [8] P. Haley, M. Jurkat, and P. Brady, “Nato reference mobility modelNRMM, Edition I, Users guide, Volume I,” U.S. Army Tank-Automotive Research and Development Command, Warren, MI, Tech. Rep. Operational Modules, Technical Report 12503, 1979.
  * [9] Christopher Goodin et al.  “Fast terrain traversability estimation with terrestrial lidar in off-road autonomous navigation”, 2021, pp. 22  DOI: 10.1117/12.2585797
  * [10] Parv Maheshwari et al.  “PIAug – Physics Informed Augmentation for Learning Vehicle Dynamics for Off-Road Navigation”  arXiv, 2023  arXiv: http://arxiv.org/abs/2311.00815
  * [11] Taekyung Kim, Hojin Lee and Wonsuk Lee  “Physics Embedded Neural Network Vehicle Model and Applications in Risk-Aware Autonomous Driving Using Latent Features”  arXiv, 2022  DOI: 10.48550/arXiv.2207.07920
  * [12] Xiangyun Meng et al.  “TerrainNet: Visual Modeling of Complex Terrain for High-speed, Off-road Navigation”  In _arXiv preprint arXiv:2303.15771_ , 2023
  * [13] R.L. Andersson  “Aggressive trajectory generator for a robot ping-pong player”  In _IEEE Control Systems Magazine_ 9.2, 1989, pp. 15–21  DOI: 10.1109/37.16766
  * [14] Aliasghar Arab, Kaiyan Yu, Jiaxing Yu and Jingang Yi  “Motion Planning and Control of Autonomous Aggressive Vehicle Maneuvers” Conference Name: IEEE Transactions on Automation Science and Engineering  In _IEEE Transactions on Automation Science and Engineering_ , 2023, pp. 1–13  DOI: 10.1109/TASE.2023.3245948
  * [15] Daniel Mellinger, Nathan Michael and Vijay Kumar  “Trajectory generation and control for precise aggressive maneuvers with quadrotors”  In _The International Journal of Robotics Research_ 31.5, 2012, pp. 664–674  DOI: 10.1177/0278364911434236
  * [16] Grady Williams et al.  “Aggressive driving with model predictive path integral control”  In _2016 IEEE International Conference on Robotics and Automation (ICRA)_ Stockholm: IEEE, 2016, pp. 1433–1440  DOI: 10.1109/ICRA.2016.7487277
  * [17] Jacob Knaup, Kazuhide Okamoto and Panagiotis Tsiotras  “Safe High-Performance Autonomous Off-Road Driving Using Covariance Steering Stochastic Model Predictive Control”  In _IEEE Transactions on Control Systems Technology_ 31.5, 2023, pp. 2066–2081  DOI: 10.1109/TCST.2023.3291570
  * [18] Bo-Chiuan Chen and Huei Peng  “Differential-Braking-Based Rollover Prevention for Sport Utility Vehicles with Human-in-the-loop Evaluations”  In _Vehicle System Dynamics_ 36, 2001, pp. 359–389  DOI: 10.1076/vesd.36.4.359.3546
  * [19] M. Doumiati, A. Victorino, A. Charara and D. Lechner  “Lateral load transfer and normal forces estimation for vehicle safety: experimental test” Publisher: Taylor & Francis _eprint: https://doi.org/10.1080/00423110802673091  In _Vehicle System Dynamics_ 47.12, 2009, pp. 1511–1533  DOI: 10.1080/00423110802673091
  * [20] Ronald L. Huston and Fred A. Kelly  “Another look at the static stability factor (SSF) in predicting vehicle rollover”  In _International Journal of Crashworthiness_ 19.6, 2014, pp. 567–575  DOI: 10.1080/13588265.2014.919730
  * [21] Duy Nguyen-Tuong and Jan Peters  “Model learning for robot control: a survey”  In _Cognitive Processing_ 12.4, 2011, pp. 319–340  DOI: 10.1007/s10339-011-0404-1
  * [22] Wm C Mitchell, Allan Staniforth and Ian Scott  “Analysis of ackermann steering geometry”, 2006
  * [23] John K Subosits and J Christian Gerdes  “Impacts of model fidelity on trajectory optimization for autonomous vehicles in extreme maneuvers”  In _IEEE Transactions on Intelligent Vehicles_ 6.3 IEEE, 2021, pp. 546–558
  * [24] Thomas Fork, H Eric Tseng and Francesco Borrelli  “Models for ground vehicle control on nonplanar surfaces”  In _Vehicle System Dynamics_ Taylor & Francis, 2023, pp. 1–25
  * [25] Davide Tavernini et al.  “Minimum time cornering: the effect of road surface and car transmission layout”  In _Vehicle System Dynamics_ 51.10 Taylor & Francis, 2013, pp. 1533–1547
  * [26] Manuel Acosta, Stratis Kanarachos and Michael E Fitzpatrick  “A hybrid hierarchical rally driver model for autonomous vehicle agile maneuvering on loose surfaces.”  In _ICINCO (2)_ , 2017, pp. 216–225
  * [27] Sidharth Talia et al.  “HOUND: An Open-Source, Low-cost Research Platform for High-speed Off-road Underactuated Nonholonomic Driving”  In _arXiv preprint arXiv:2311.11199_ , 2023
  * [28] Hojin Lee, Taekyung Kim, Jungwi Mun and Wonsuk Lee  “Learning Terrain-Aware Kinodynamic Model for Autonomous Off-Road Rally Driving With Model Predictive Path Integral Control”  In _arXiv preprint arXiv:2305.00676_ , 2023
  * [29] Kurtland Chua, Roberto Calandra, Rowan McAllister and Sergey Levine  “Deep reinforcement learning in a handful of trials using probabilistic dynamics models”  In _Advances in neural information processing systems_ 31, 2018
  * [30] David D. Fan et al.  “STEP: Stochastic Traversability Evaluation and Planning for Risk-Aware Off-road Navigation”  arXiv, 2021  arXiv: http://arxiv.org/abs/2103.02828
  * [31] Weitang Liu, Xiaoyun Wang, John Owens and Yixuan Li  “Energy-based Out-of-distribution Detection”  In _Advances in Neural Information Processing Systems_ 33 Curran Associates, Inc., 2020, pp. 21464–21475  URL: https://proceedings.neurips.cc/paper/2020/hash/f5496252609c43eb8a3d147ab9b9c006-Abstract.html
  * [32] BeamNG GmbH  “BeamNG.tech”, 2022  URL: https://www.beamng.tech/
  * [33] Jonas Frey et al.  “RoadRunner - Learning Traversability Estimation for Autonomous Off-road Driving”  arXiv, 2024  arXiv: http://arxiv.org/abs/2402.19341


#  FreeTalky: Don’t Be Afraid! Conversations Made Easier by a
Humanoid Robot using Persona-based Dialogue

Chanjun Park1, Yoonna Jang1, Seolhwa Lee2, Sungjin Park3, and Heuiseok Lim1
1Korea University
2University of Copenhagen
3NAVER Corp.
{bcj1210, morelychee<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>This author is the corresponding author.

###### Abstract

We propose a deep learning-based foreign language learning platform, named
FreeTalky, for people who experience anxiety dealing with foreign languages,
by employing a humanoid robot NAO and various deep learning models. A persona-
based dialogue system that is embedded in NAO provides an interesting and
consistent multi-turn dialogue for users. Also, an grammar error correction
system promotes improvement in grammar skills of the users. Thus, our system
enables personalized learning based on persona dialogue and facilitates
grammar learning of a user using grammar error feedback. Furthermore, we
verified whether FreeTalky provides practical help in alleviating
xenoglossophobia by replacing the real human in the conversation with a NAO
robot, through human evaluation.

## Introduction

Following multiculturalism and hyper-connectivity, many societies value
multilingual skills as a demonstration of the ability to work effectively with
customers, clients, and businesses of various countries and cultures (Damari
et al. 2017). In this context, native-speaking classes and foreign language
phone calls for foreign language learning are available in most countries.

However, the costs are very high and some learners experience a mental block
in learning foreign languages (i.e., foreign language anxiety, also known as
xenoglossophobia). It can be described as a feeling of uneasiness, worry,
nervousness, and apprehension that is experienced while learning or using a
second or foreign language (Horwitz, Horwitz, and Cope 1986; Donmus 2010).

To address these issues, we propose FreeTalky, a software platform that allows
users to communicate comfortably as if they were conversing with a native
speaker by conversing with a robot instead through realistic free talking.
With FreeTalky, users obtain a learning effect beyond the traditional
classroom learning methods, by experiencing direct conversations without the
mental pressure, rather than by only seeing and hearing.

The FreeTalky platform combines speech recognition, a persona-based dialogue
system, English grammar error correction, and a humanoid robot NAO, for
English learning that is user-friendly and practically helpful, by reflecting
user perspectives. Specifically, the persona-based dialogue system provides
customized multi-turn dialogue, while the English grammar error correction
software provides customized grammar correction. These features allow users to
have user-friendly conversations with the robot, learn foreign languages
effectively, and even gain assistance with their xenoglossophobia. The
contributions of this study are as follows.

  * •

To the best of our knowledge, FreeTalky is the first deep learning-based
conversation platform for alleviating xenoglossophobia.

  * •

The persona-based conversation system in FreeTalky allows users to customize
foreign language learning and provides practical feedback on grammatical
errors through a deep neural network-based grammar error correction software.

  * •

Through an evaluation and survey of real users of FreeTalky, the validity and
feasibility of FreeTalky were verified, and various experimental empirical
analyses and error analyses were performed.

Figure 1: Overview of FreeTalky process.

## Related Work

To the best of our knowledge, there is no foreign language learning software
for alleviating xenoglossophobia. In fact, a majority of the commercialized
software products for foreign language learning are developed for the general
public, with the primary objective of improving English language skills, and
do not consider xenoglossophobia.

There are three major foreign language learning platforms: Andy
111https://andychatbot.com/, Cake 222https://mycake.me/, and SpeakNow
333https://www.speaknow.ai. However, despite the commercialized service
system, there is considerable room for improvement.

First, Andy does not provide contextual dialogue, and users cannot receive
evaluation or feedback on their learning. Second, Cake provides feedback on
pronunciation and expressiveness; however, there are no means to have a
natural conversation through an actual dialogue system. Additionally, it only
provides traditional evaluation methods, such as word quizzes and fill in the
blanks. Further, there is no feature that provides practical feedback for
English grammar. Finally, SpeakNow, an educational English conversation
application using 3D avatars, provides evaluation of grammar accuracy,
expressiveness, and pronunciation, exploiting various conversation situations
and users’ voices. However, it is not a type of free conversational model, as
it only provides a service that selects and answers a given sentence, or reads
the given sentences.

In light of these limitations, we propose FreeTalky, which is equipped with a
persona conversation system that enables personalized learning and an English
grammar corrector that provides practical learning feedback.

## FreeTalky

### Workflow of FreeTalky

In this section, we describe the overall workflow of FreeTalky, as shown in
Figure 1. The process is explained step by step as follows.

  1. 1.

The user sets the persona of the robot (e.g., (1) [I like to play basketball.,
…], (2) [I have two daughters., …], (3) [I am a graduate student., …]) by
selecting a number. Each number is assigned a randomly chosen list of
personas, and this information is entered into the system. Automatic persona
setting is provided for the convenience of the user; however, the users can
set the persona of the robot themselves as well.

  2. 2.

The user begins the conversation with the NAO robot to practice speaking in
English. The input of the user is further converted into text using the
speech-to-text (STT) module, and passed on to the dialogue system. The
dialogue model inputs the converted text with persona sentences and previous
history of the dialogue. Next, the model generates a response.

  3. 3.

The generated response is converted into speech using the text-to-speech (TTS)
module of the NAO robot, for it to provide the answers to the user. Through
this process, the user is able to receive the output from the model in the
form of human voice.

  4. 4.

When the conversation is completed by the end command (i.e., Bye.), all
utterances are transmitted to the grammar error correction (GEC) module, where
the grammatical errors in the utterances are detected.

  5. 5.

The robot provides feedback on the sentences that include grammatical errors,
if there are any. By receiving feedback, the users learn English grammar, in
addition to experiencing a seamless conversation in English.

### NAO Robot

According to Shamsuddin et al. (2011), NAO is a humanoid robot that has been
purposely designed to seem approachable. We embedded our FreeTalky in the NAO
robot to allow the robot to be able to communicate with people more easily and
effectively. By utilizing this humanoid robot as an educational assistant,
people who are learning the English language would experience more natural
conversations, rather than studying alone.

### Speech-to-Text

We adopted an off-the-shelf STT module known as CLOVA speech recognition (CSR)
(Ha et al. 2020) API of Naver. It obtains the speech input of the user and
converts it into text. Further, it sends the converted text to the dialogue
system of the model.

### Text-to-Speech

The TTS system is based on ALTextToSpeech, which is a built-in TTS system of
the NAO robot. This module is used for providing feedback to the users on
their grammatical errors. In addition, the robot alters its emotion and voice,
which allows the users to experience a life-like conversation with foreigners.

Figure 2: Overview of persona-based free talking model in FreeTalky.

### Persona-based Free Talking Dialogue

According to Grice’s maxims, there are several maxims that need to be followed
to ensure a good conversation (Dale and Reiter 1995; Looi and See 2012). They
are maxims of quantity, quality, relation, and manners.

To reflect these maxims, FreeTalky adopts a persona-based dialogue model. By
assigning a persona to the system, it can provide consistent and friendly
responses to the users. The users can practice conversing in English with the
NAO robot similarly as they would converse with people. Additionally, they can
set up the persona of the robot such that it includes their various interests
or conversational topics.

Most existing foreign language learning systems provide only a single-turn
dialogue. However, we deployed a multi-turn free-talking model using the pre-
trained language model (Zhang et al. 2019) for a more realistic conversation.
It was trained on a massive dialogue corpus to allow the model to utilize the
learned dialogue patterns and construct colloquial expressions. We adopted
PERSONA-CHAT (Zhang et al. 2018) as the training data for transfer learning of
the model.

In the transfer-learning phase, the concatenations of persona sentences,
dialogue history, and the user’s utterances were used as the input. With this
input, the model learned the patterns inherent in dialogue data. As shown in
Figure 2, the model was trained with two objective functions: next sentence
prediction and masked language modeling. With the masked language modeling,
the model learned next token generation in an auto-regressive manner.
Simultaneously, it learned which answer is suitable for the next utterance
among 20 candidates, using the next sentence prediction function.

The evaluation metrics for the model are hits@1 and perplexity. Hits@1 is used
to measure the accuracy of the correct answer being chosen among 20 candidate
answers. Perplexity is a metric for evaluating a language model for its
ability to predict the next token. Low perplexity indicates low congestion and
high performance of language models. In the results obtained from the
qualitative analysis, hits@1 was 78.68 and perplexity was 14.19, when
evaluated on the validation set of the PERSONA-CHAT dataset (Zhang et al.
2018). In addition, to handle ethical issues (e.g. hate speech, offensive
words) which frequently arises from the language models due to the biases in
toxic language datasets (Sap et al. 2019), we regenerate the sequence, if the
generated sequence has certain words in our pre-built list of offensive words.

Figure 3: Overview of grammar error correction model in FreeTalky.

### Grammar Error Feedback

Several studies have proven the positive effects of feedback on error
correction in foreign language learning (Lyster and Ranta 1997; Loewen 2013;
Nassaji 2016; Park et al. 2021). Inspired by these results, this study adapted
the GEC model for FreeTalky to provide feedback on grammatical errors to
users.

In terms of machine translation, GEC is a task in which the erroneous source
sentence is translated to the correct target sentence. That is, the sequence-
to-sequence (S2S) model can be interpreted as noise sequence to denoise
sequence (NS2DS (Park et al. 2020a, b)). All S2S models, such as CNN (Gehring
et al. 2017), RNN (Bahdanau, Cho, and Bengio 2014), and Transformer (Vaswani
et al. 2017), can be utilized as NS2DS. In addition, existing rule-based and
statistical models can be incorporated with NS2DS models in a hybrid form.

In this study, we adopted the vanilla Transformer (Vaswani et al. 2017) with a
copy mechanism (Gu et al. 2016), which was suitable for the service owing to
its lightweight structure. We applied the copy mechanism based on the
characteristics of the training data in the GEC, where the source and target
sentences had the same character set. Thus, we handled the out-of-vocabulary
(OOV) problem using the copy mechanism in a principled manner.

The overall model structure is shown in Figure 3. It was deployed on a
separate server and connected to FreeTalky by means of REST API using Flask.
To enable scaling-up of the service, we additionally deployed the CPU-based
model, as well as the GPU-based model (Klein et al. 2020). Through this
function, explicit instructions are made available to the user.

Example conversation with FreeTalky |
---|---
FreeTalky: | Hi, I am NAO. Please choose the number between 1 to 3 and tell me, then I will set my personality | (01)
| and tell you about my persona! | (02)
User: | Two | (03)
FreeTalky: | Persona setting is done. Let me introduce myself. I live in colorado. I like to go hiking in the spring. | (04)
| My favorite activity is rock climbing. I am a mechanical engineer. | (05)
FreeTalky: | Now you can have a conversation with me! If you want to finish the conversation, just say bye. | (06)
User: | Hi how are you? | (07)
FreeTalky: | I am doing great! I just got back from rock climbing. What are you up to? | (08)
User: | Wow… That’s awesome. I am going to study English. | (09)
FreeTalky: | I am in colorado, we are going to go hiking sometime soon. | (10)
User: |  I want visit Colorado. I think you like outdoor activities! | (11)
FreeTalky: | I do, but I’m not too big on them. | (12)
User: | I see. You.. engineer? | (13)
FreeTalky: | Yes, I am a mechanical engineer. | (14)
User: | Okay.. It’s nice to meet you. Bye. | (15)
FreeTalky: | Conversation is done. Do you want to get grammatical feedback? | (16)
User: | Yes please. | (17)
FreeTalky: | Okay! You said, “I want visit Colorado”, but “I want to visit Colorado”, is a more grammatically | (18)
| correct expression. You said, “You engineer?”, but “Are you an engineer?” is a more | (19)
| grammatically correct expression. | (20)

Table 1: An example conversation between the user and FreeTalky system. Users
can set persona by choosing number and talk with FreeTalky system. It gives
grammar error feedback to the user after chatting. Full video is attached in
Supp.. The highlighted text indicates the user’s grammar error and FreeTalky’s
feedback sentence.

##  FreeTalky for Xenoglossophobia

Xenoglossophobia, also known as foreign language anxiety, is a consistent and
irrational fear revolving around foreign languages (Abinaya 2016). People who
suffer from xenoglossophobia experience anxiety while having conversations
with people who speak a foreign language. There does not exist a specific
method for treating xenoglossophobia in particular; however, the treatments
suggested for general phobias can be utilized.

Most people who have a phobia are treated with exposure therapy. It is a
method that involves exposing the patients to the situation, thus restricting
the anxiety for a certain period. Though anxiety can temporarily increase
during the treatment, it helps to release it in the long term, when the
patients face real-world situations.

Motivated by this treatment, we experimented to determine whether FreeTalky
could practically help people who have xenoglossophobia. The NAO robot
replaces the second human in the conversation, thus possibly relieving the
anxiety of the user, and simultaneously makes the situation more realistic
using the persona-based free talking system.

## Qualitative Analysis

We analyzed the conversation between the user and FreeTalky in a qualitative
manner. An example conversation is presented in Table 1. As shown in line
(01), the FreeTalky system provides a randomly chosen persona through the
selection of a number between 1 to 3. This setting enables virtual experiences
in various situations where the users can experience the feeling of talking to
different people. In lines (08) and (14), FreeTalky generates persona-based
answers. Its multi-turn conversation not only seems natural, but also
continues until the user ends the conversation themselves. Moreover, it has
the ability to ask an additional question from the user, as in (08). Once the
conversation is over, it provides grammatical error feedback to the user, as
in the last utterance of FreeTalky in Table 1.

### Error Analysis

Although the proposed system can hold a natural multi-turn dialogue and
provide grammatical error feedback, several errors remain in need of
improvement besides the given conversation. We divided the error types into
four cases, as shown in Figure 4. The first case is the error of unnatural
flow of conversation. Although the model attempts to converse about specific
topics, it says things that conflict with what was said before, or with its
persona, as shown in line (12) of Table 1. The second case is the utterance
pattern learned from the pre-trained model. As we utilized the pre-trained
model, the pattern learned from the large dialogue corpus tends to appear, as
a result of which, the model sometimes changes the conversation topic and
talks about itself. In Figure 4, the FreeTalky model suddenly changes the
topic of conversation to travel. The third case is overcorrection.
Overcorrection is a phenomenon wherein a grammatically correct input sentence
is regarded as a subject that requires correction, and the sentence structure
is disturbed. In Figure 4, the model changes a correct sentence of the user.
The fourth case is a generic problem of generative models, that is, a model
tends to generate tokens repeatedly and leads to a [unk] token issue, as shown
in Figure 4.

Figure 4: Four cases of errors. Case 1 is an erroneous sentence that conflicts
with what the model said before. Case 2 is an unnatural utterance that
suddenly changes the conversation topic. Case 3 is an error of overcorrection.
Case 4 is a generic problem of generative models.

##  FreeTalky Satisfaction Survey

We validated the efficiency of the proposed system through five middle school
students, filtered by measurement (Al-Saraj 2014), who experience foreign
language anxiety. Further, we used six different questionnaires for system
validation based on the Likert scale (Likert 1932) that rates situations from
1 (i.e., strongly disagree) to 5 (i.e., strongly agree), as shown in Table 2.

As shown in Figure 5, most questions about the system were scored as strongly
agree or agree. This result implies that the proposed system is suitable for
alleviating people’s xenoglossophobia.

Questionnaire
---
Q1 | I feel comfortable while interacting with a robot in
| a real human-like conversation.
Q2 | I do not feel scared while talking to a robot.
Q3 | Do you think FreeTalky will help alleviate your
| foreign language anxiety?
Q4 | Are you satisfied with the performance of grammatical
| error feedback?
Q5 | Do you think FreeTalky performs naturally in a
| multi-turn conversation?
Q6 | Do you think persona is well applied in FreeTalky?

Table 2:  Questionnaire of FreeTalky satisfaction survey.

User interview before chatting with FreeTalky.
---
Interviewer | Hi, nice to meet you.
| I heard that you experience fear while meeting and conversing with
foreigners.
User | That is right. I have learned English, but when I meet foreigners, I feel nervous, so I am unable to talk to them.
Interviewer | Okay, do you face any problems while reading and writing in English?
User | Yes, reading text in English is not a problem, but I get very nervous when I speak in English.
Interviewer | Have you made any efforts to overcome this?
User | I intended to go to an English conversation academy, but the cost was expensive, and I do not have enough
| time to go there.
Interviewer | Okay, then how about practicing conversing in English with a robot, instead of a human?
User | A robot? I have not talked to a robot before, so I do not know.
Interviewer | Then, why don’t we try and see if it can help relieve your fear now?
User interview after chatting with FreeTalky.
Interviewer | How was it, talking with the robot in English?
User | It was good because I felt more relaxed than when I talk to a real person. Also, it was good that this
| robot gave me feedback on grammatically wrong expressions. The robot had
various voices, so I felt like
| it was a real situation.
Interviewer | You mean that talking with the robot helps relieve your psychological fear.
User | Yes, it does.
Interviewer | Great then, we will attempt to further improve this system to assist with your anxiety. Thank you
User | Thank you.

Table 3: User interview and experience of chatting with FreeTalky. As the user
interview was conducted in Korean, the translated version is attached. The
highlighted text indicates the notable points in the interview (see details in
discussion). Figure 5: Response result of satisfaction survey of FreeTalky.
Figure 6: Correlation result of satisfaction survey of FreeTalky. Each cell
indicates a correlation value, and numbers in parentheses are p-value
(p-value$<$0.05 indicates statistical significance).

Furthermore, we conducted Spearman correlation (de Winter, Gosling, and Potter
2016) on the survey results in Figure 6. The results show a positive
correlation in all question relationships. There is a significant (p$<$0.05)
positive correlation between Q1 and Q4, from Q2 to Q3, Q5, Q6, and between Q5
and Q6. Interestingly, Q3 and Q5, Q6 show a perfect correlation, which implies
that there is a high association between the alleviation of foreign language
anxiety and natural multi-turn dialogue and persona-applied conversation.
These results prove that FreeTalky is appropriate for our objectives, that
applying the persona and multi-turn dialogue to the conversation would result
in the system being more user-friendly. Another highlight of the results
corresponding to our objectives is that the humanoid robot NAO performed well
in alleviating xenoglossophobia, and is user-friendly according to Q2, Q3, Q5,
and Q6. Additionally, it demonstrated that GEC contributes to the comfort of
the users while interacting with humanoid robots, according to the results of
Q1 and Q4. Consequently, these factors determine the system satisfaction.

## Discussion

Following the system satisfaction survey, we interviewed the actual users to
determine an in-depth usability of the system besides the survey. The
interview process aided the system development, as a predefined questionnaire
could not handle unexpected answers, such as a narrative context. The
interview transcript is presented in Table 3. The original interview video is
also attached in the following link444https://youtu.be/P6Xyv3pi6F4.

As illustrated in Table 3, we determined that the user experienced a life-like
situation owing to the various voices of NAO (see yellow box in Table 3). We
provided two modulations of voices for NAO, that is neutral and joyful. The
user in this interview experienced NAO as a joyful voice; thus, this setting
enabled the user to experience a realistic situation.

Although we performed the fundamental quantitative analysis and appropriate
qualitative analysis, limitations that hindered the quantitative analysis of
the system remain because there is no official test set available for further
evaluation. Thus, we focused on the error analysis to address these
limitations. However, these limitations encourage our further research, that
is, to build a test set for evaluating the system and develop an evaluation
metric. We plan to extend our work and release the same, to support
researchers in this field.

## Conclusion

We proposed FreeTalky, where users can receive feedback on English grammar
errors and improve their English conversation skills. In addition, as
FreeTalky is based on a persona dialogue system, the content of multi-turn
conversations can be adaptively altered to suit the users and provide diverse
conversational experience to the users. Furthermore, it is possible to
alleviate the psychological pressure experienced by people suffering from
xenoglossophobia, using the NAO robot.

## Acknowledgment

This research was supported by the MSIT(Ministry of Science and ICT), Korea,
under the ITRC(Information Technology Research Center) support
program(IITP-2018-0-01405) supervised by the IITP(Institute for Information &
Communications Technology Planning & Evaluation) and supported by the
MSIT(Ministry of Science and ICT), Korea, under the ICT Creative Consilience
program(IITP-2021-2020-0-01819) supervised by the IITP(Institute for
Information & communications Technology Planning & Evaluation).

## References

  * Abinaya (2016) Abinaya, R. 2016.  Xenoglossophobia: Language learner’s perspective on learning.  _American College Journal of English Language and Literature_ , 5: 1–4.
  * Al-Saraj (2014) Al-Saraj, T. M. 2014.  Revisiting the foreign language classroom anxiety scale (FLCAS): The anxiety of female English language learners in Saudi Arabia.  _L2 Journal_ , 6(1).
  * Bahdanau, Cho, and Bengio (2014) Bahdanau, D.; Cho, K.; and Bengio, Y. 2014.  Neural machine translation by jointly learning to align and translate.  _arXiv preprint arXiv:1409.0473_.
  * Dale and Reiter (1995) Dale, R.; and Reiter, E. 1995.  Computational interpretations of the Gricean maxims in the generation of referring expressions.  _Cognitive science_ , 19(2): 233–263.
  * Damari et al. (2017) Damari, R. R.; Rivers, W. P.; Brecht, R. D.; Gardner, P.; Pulupa, C.; and Robinson, J. 2017.  The demand for multilingual human capital in the US labor market.  _Foreign Language Annals_ , 50(1): 13–37.
  * de Winter, Gosling, and Potter (2016) de Winter, J. C.; Gosling, S. D.; and Potter, J. 2016.  Comparing the Pearson and Spearman correlation coefficients across distributions and sample sizes: A tutorial using simulations and empirical data.  _Psychological methods_ , 21(3): 273.
  * Donmus (2010) Donmus, V. 2010.  The use of social networks in educational computer-game based foreign language learning.  _Procedia-Social and Behavioral Sciences_ , 9: 1497–1503.
  * Gehring et al. (2017) Gehring, J.; Auli, M.; Grangier, D.; Yarats, D.; and Dauphin, Y. N. 2017.  Convolutional sequence to sequence learning.  In _Proceedings of the 34th International Conference on Machine Learning-Volume 70_ , 1243–1252. JMLR. org.
  * Gu et al. (2016) Gu, J.; Lu, Z.; Li, H.; and Li, V. O. 2016.  Incorporating copying mechanism in sequence-to-sequence learning.  _arXiv preprint arXiv:1603.06393_.
  * Ha et al. (2020) Ha, J.-W.; Nam, K.; Kang, J.; Lee, S.-W.; Yang, S.; Jung, H.; Kim, E.; Kim, H.; Kim, S.; Kim, H. A.; et al. 2020.  ClovaCall: Korean goal-oriented dialog speech corpus for automatic speech recognition of contact centers.  _arXiv preprint arXiv:2004.09367_.
  * Horwitz, Horwitz, and Cope (1986) Horwitz, E. K.; Horwitz, M. B.; and Cope, J. 1986.  Foreign language classroom anxiety.  _The Modern language journal_ , 70(2): 125–132.
  * Klein et al. (2020) Klein, G.; Zhang, D.; Chouteau, C.; Crego, J. M.; and Senellart, J. 2020.  Efficient and High-Quality Neural Machine Translation with OpenNMT.  In _Proceedings of the Fourth Workshop on Neural Generation and Translation_ , 211–217.
  * Likert (1932) Likert, R. 1932.  A technique for the measurement of attitudes.  _Archives of psychology_.
  * Loewen (2013) Loewen, S. 2013.  The role of feedback.  In _The Routledge handbook of second language acquisition_ , 42–58. Routledge.
  * Looi and See (2012) Looi, Q. E.; and See, S. L. 2012.  Applying politeness maxims in social robotics polite dialogue.  In _Proceedings of the seventh annual ACM/IEEE international conference on Human-Robot Interaction_ , 189–190.
  * Lyster and Ranta (1997) Lyster, R.; and Ranta, L. 1997.  Corrective feedback and learner uptake: Negotiation of form in communicative classrooms.  _Studies in second language acquisition_ , 19(1): 37–66.
  * Nassaji (2016) Nassaji, H. 2016.  Anniversary article Interactional feedback in second language teaching and learning: A synthesis and analysis of current research.  _Language Teaching Research_ , 20(4): 535–562.
  * Park et al. (2020a) Park, C.; Kim, K.; Yang, Y.; Kang, M.; and Lim, H. 2020a.  Neural spelling correction: translating incorrect sentences to correct sentences for multimedia.  _Multimedia Tools and Applications_ , 1–18.
  * Park et al. (2021) Park, C.; Seo, J.; Lee, S.; Lee, C.; Moon, H.; Eo, S.; and Lim, H.-S. 2021.  BTS: Back TranScription for speech-to-text post-processor using text-to-speech-to-text.  In _Proceedings of the 8th Workshop on Asian Translation (WAT2021)_ , 106–116.
  * Park et al. (2020b) Park, C.; Yang, Y.; Lee, C.; and Lim, H. 2020b.  Comparison of the evaluation metrics for Neural Grammatical Error Correction with Overcorrection.  _IEEE Access_ , 8: 106264–106272.
  * Sap et al. (2019) Sap, M.; Card, D.; Gabriel, S.; Choi, Y.; and Smith, N. A. 2019.  The risk of racial bias in hate speech detection.  In _Proceedings of the 57th annual meeting of the association for computational linguistics_ , 1668–1678.
  * Shamsuddin et al. (2011) Shamsuddin, S.; Ismail, L. I.; Yussof, H.; Zahari, N. I.; Bahari, S.; Hashim, H.; and Jaffar, A. 2011.  Humanoid robot NAO: Review of control and motion exploration.  In _2011 IEEE international conference on Control System, Computing and Engineering_ , 511–516. IEEE.
  * Vaswani et al. (2017) Vaswani, A.; Shazeer, N.; Parmar, N.; Uszkoreit, J.; Jones, L.; Gomez, A. N.; Kaiser, Ł.; and Polosukhin, I. 2017.  Attention is all you need.  In _Advances in neural information processing systems_ , 5998–6008.
  * Zhang et al. (2018) Zhang, S.; Dinan, E.; Urbanek, J.; Szlam, A.; Kiela, D.; and Weston, J. 2018.  Personalizing dialogue agents: I have a dog, do you have pets too?  _arXiv preprint arXiv:1801.07243_.
  * Zhang et al. (2019) Zhang, Y.; Sun, S.; Galley, M.; Chen, Y.-C.; Brockett, C.; Gao, X.; Gao, J.; Liu, J.; and Dolan, B. 2019.  Dialogpt: Large-scale generative pre-training for conversational response generation.  _arXiv preprint arXiv:1911.00536_.