# Deep Reinforcement Learning for Efficient and Fair Allocation of Health Care
Resources

Yikuan Li*, Chengsheng Mao*, Kaixuan Huang*, Hanyin Wang*, Zheng Yu*
Mengdi Wang†, Yuan Luo†
{yikuan.li, chengsheng.mao, hanyin.wang<EMAIL_ADDRESS>
{kaixuanh, zhengy<EMAIL_ADDRESS>
###### Abstract

Scarcity of health care resources could result in the unavoidable consequence
of rationing. For example, ventilators are often limited in supply, especially
during public health emergencies or in resource-constrained health care
settings, such as amid the pandemic of COVID-19. Currently, there is no
universally accepted standard for health care resource allocation protocols,
resulting in different governments prioritizing patients based on various
criteria and heuristic-based protocols. In this study, we investigate the use
of reinforcement learning for critical care resource allocation policy
optimization to fairly and effectively ration resources. We propose a
transformer-based deep Q-network to integrate the disease progression of
individual patients and the interaction effects among patients during the
critical care resource allocation. We aim to improve both fairness of
allocation and overall patient outcomes. Our experiments demonstrate that our
method significantly reduces excess deaths and achieves a more equitable
distribution under different levels of ventilator shortage, when compared to
existing severity-based and comorbidity-based methods in use by different
governments. Our source code is included in the supplement and will be
released on Github upon publication.

##  1 Introduction

The Institute of Medicine (IOM) defines Crisis Standards of Care as “a
substantial change in usual health care operations and the level of care it is
possible to deliver, which is made necessary by a pervasive (e.g., pandemic
influenza) or catastrophic (e.g., earthquake, hurricane) disaster” [1]. These
guidelines recognize that pandemics such as COVID-19 can strain health systems
into an absolute scarcity of health care resources and could result in the
unavoidable consequence of rationing.

Following the IOM framework, state governments throughout the U.S. have
developed allocation protocols for critical care resources during the pandemic
[2]. Consistent with the broad consensus of ethicists and stakeholders [3],
these protocols aim to triage patients via a pre-specified, transparent, and
objective policy. Despite this common general framework, critical details of
these protocols vary widely across the U.S. [2] Many protocols aim to maximize
total benefits to the population by ranking patients with objective outcome
predictors like the Sequential Organ Failure Assessment (SOFA) score [4], the
conversion of SOFA into “priority scores” differs substantially between
protocols. The protocols also vary on whether they prioritize younger patients
or those without pre-existing medical conditions. For example, the SOFA
protocol (e.g., used in New York [5]) is focused on saving the most lives in
the short term and ignores both age and pre-existing medical conditions. In
contrast, the multiprinciple protocols (e.g., used in Maryland and
Pennsylvania) also prioritize younger patients and those without pre-existing
medical conditions [6]. The fact that these protocols are heuristic based and
substantial variation among the protocols calls for evidence-based protocol
design using machine learning. The allocation protocols are typically assessed
on a daily basis, making it a sequential decision making process and suitable
for reinforcement learning (RL).

In this study, we investigate the use of RL for critical care resource
allocation policy optimization. Our goal is to learn an optimal allocation
policy based on available observations, in order to maximize lives saved while
keeping rationing equitable across different races. It is quite natural that
sequential allocation decisions on critical care resources can be modeled by a
Markov decision process (MDP), and the allocation protocols by RL policies. We
use a large, diverse, multi-hospital population of critically ill patients
amid COVID-19 pandemic to evaluate our proposed method and comparison models.

However, application of reinforcement learning (RL) to this problem is
nontrivial for practical considerations. In particular, empirical assessment
of their performance suggested that most such protocols and their variants
used earlier in US states were significantly less likely to allocate
ventilators to Black patients [7]. In addition, the exact SOFA score is shown
to be only modestly accurate in predicting mortality in COVID-19 patients
requiring mechanical ventilation [8]. There is a vital need to improve both
the utility and equity of these allocation protocols to fairly and effectively
ration resources to critically ill patients. Our main contributions are
summarized as follows:

  1. 1.

We are the first to formulate fair health care resource allocation as a multi-
objective deep reinforcement learning problem. The goal is to integrate the
utilitarian and egalitarian objectives in the RL rewards.

  2. 2.

We propose a Transformer-based deep Q-network capable of making allocation
decisions by incorporating the disease progression of individual patients and
the interaction effects among patients during the critical care resource
allocation.

  3. 3.

We apply our approach to real-world clinical datasets. Experiments show that
our approach leads to fair allocation of critical care resources among
different races, while maintaining the overall utility with respect to patient
survival.

##  2 Related Work

Health care resources, including personnel, facilities, medical equipment,
etc., are never in excess, especially amid pandemics [9] or in the case of
intensive care [10]. Decision-making is vital for health care outcome
optimization [11], bearing the scarcity of resources. A successful strategy
should account for optimal resource utilization, equity and fairness, cost-
effectiveness, and crisis buffering. Health care is a fundamental human right,
and any resource allocation strategy should not disproportionately
disadvantage or discriminate against specific groups. However, implicit or
explicit discrimination has permeated health care and medicine with a long
history, presenting numerous instances of biased outcomes [12, 13, 14].

Due to its inherent nature of comprehending goal-oriented learning and
decision-making challenges, RL possesses the potential to generate optimized
allocation strategies. However, Machine learning approaches, including RL, can
be biased towards favoring the accuracies of majority classes at the expense
of minority classes, and do not automatically align with fairness objectives
[15]. In fact, RL may further exacerbate disparities across patient groups by
ignorance of fairness considerations [16]. In recent years, there have been
growing literature for fairness in machine learning across various fields [15,
17, 18] ), including health care [19, 20, 21, 22, 23]. However, there lack
considerations of fairness of RL algorithms in the application of health care
resource allocation.

Previous studies employed RL to simulate the trajectory of a pandemic to
facilitate the adoption of different levels of lockdown strategies [24], to
infer the future state-level demand for ventilators to inform resource
allocation [25], to efficiently allocate limited PCR tests to screen incoming
travelers for possible COVID-19 infections [26] etc. See survey [27] for more
comprehensive summary of RL application in health care. Previous studies
centered around the application of RL in the simulation of future trajectories
and focus on utilitarian goals such as maximizing efficiency of allocation,
which lacked insight into RL’s impact on the equity among patients. Given
empirical evidences on resulted disparities from in-use critical care resource
allocation protocols [7], the egalitarian aspects of goals deserve focused
attentions. However, using RL for simultaneously reducing disparities across
different races during critical resource allocation and improving overall
patients’ outcomes has been underexplored.

##  3 Problem Formulation

We formulate the ventilator allocation problem as a day-to-day sequential
decision problem. In the event of a ventilator shortage, some patients who
require a ventilator may not receive one due to limited availability. In such
cases, a triage protocol must be applied to prioritize the dispatching of
ventilators to those patients who truly need them, as determined by the
doctors. Following existing clinical literature (e.g., [7]), we assume that
patients who needed ventilators but did not receive one will die. Thus, there
will be no patients "waiting" for a ventilator, as the inability to receive
ventilation will result in their immediate removal from the dataset. Also
following [7], we do not consider in the main results the withdrawal of a
ventilator from one patient to save another, which is a practice drawing
extensive debates without widely agreed consensus on principles (e.g., ranging
from no mechanism in the Maryland protocol to an explicit SOFA based approach
in the New York protocol). But we did include this setting in the supplemental
material, in order to provide a comprehensive picture across the spectrum
connecting both extremes.

The medical condition of a patient at one time point is represented by a
$k$-dimensional vector $x\in[0,1]^{k}\subseteq\mathbb{R}^{k}$, including
demographics, current SOFA score constructs, vital signs, COVID-19 status etc.
Assume the intensive care units have up to $N$ patients arriving in a single
day, with each being assigned a bed. Here we only consider ventilator
scarcity, but bed scarcity can be analogously modeled. We provide a detailed
illustration of our formulation and proposed model in 1. In the following, we
give a rigorous formulation of the RL model for both cases of without and with
the consideration of the fairness in distributing ventilators.

Figure 1: An illustration of the trajectory in our study formulation. Each
color represents a separate patient.

###  3.1 State Space

In the case without consideration of fairness, the state space $\mathcal{S}$
describes the current clinical conditions of all patients in the hospital, as
well as their ventilation status. Each state is represented by

$\displaystyle
s=[x_{1},x_{2},\cdots,x_{N},I_{1},I_{2},\cdots,I_{N}]\in\mathcal{S}\subseteq\mathbb{R}^{kN+N},$
(1)

where $x_{i}\in\mathbb{R}^{k}$ denotes the current medical condition of the
patient on bed $i$, and indicator $I_{i}\in\\{0,1\\}$ denotes whether the bed
$i$ has been ventilated. Apart from normal medical conditions of patients, we
consider three special conditions: the "SURVIVED", "DEAD", and "VACANT"
conditions, corresponding to the cases where the patient in this bed is
recovered or dead after ventilation, as well as currently no patient in this
bed. They act as terminators for patients or separators between patients in
the same bed. Such designs separate different patients on the same bed
explicitly, so that the tasks of learning the progression of medical
conditions given ventilation and the task of recognizing the end of each
patient can be decoupled.

In the case with consideration of fairness, we further record the cumulative
numbers of total and ventilated patients of different ethnoracial groups in
the state vector, denoted by $n_{k},m_{k}\in\mathbb{R}$ respectively, where
$k\in\\{\text{`B'},\text{`W'},\text{`A'},\text{`H'}\\}$ denotes 4 ethnoracial
group in the dataset: non-Hispanic Black, non-Hispanic White, non-Hispanic
Asian, Hispanic. Thus, each state

$\displaystyle
s=[x_{1},x_{2},\cdots,x_{N},I_{1},I_{2},\cdots,I_{N},n_{B},n_{W},n_{A},n_{H},m_{B},m_{W},m_{A},m_{H}]\in\mathcal{S}\subseteq\mathbb{R}^{kN+N+8}$

describes the medical and ventilation status of all current patients, as well
as the number of cumulative total and ventilated patients of different ethnic
groups.

###  3.2 Action Space

For both cases, the action space $\mathcal{A}$ is a discrete space denoting
whether each bed is on ventilation or not. Let us use 1 to denote ventilate
and 0 otherwise, so the action space $\mathcal{A}\subseteq\\{0,1\\}^{N}$. Note
that we have two constraints on actions:

  1. 1.

Maximum number of ventilators:
$\mathcal{A}\subseteq\\{a\in\\{0,1\\}^{N}:\sum_{i=1}^{N}a_{i}\leq B\\}$, where
$B$ is the number of ventilators the hospital has.

  2. 2.

Continuous ventilation: a patient who was ventilated must keep ventilation
until recovery or death:
$\mathcal{A}\subseteq\\{a\in\\{0,1\\}^{N}:a_{i}=1\text{ if
}I_{i}=1,i=1,2,\cdots,N\\}$, where $I_{i}$ in the state information denotes
whether current patient on bed $i$ has been ventilated before.

Therefore, the action space is shrunk to

$\displaystyle\mathcal{A}=\\{a\in\\{0,1\\}^{N}:\sum_{i=1}^{N}a_{i}\leq B\text{
and }a_{i}=1\text{ if }I_{i}=1,\forall i=1,2,\cdots,N\\},$ (2)

which is ensured in our proposed methods as shown in Eq.
(LABEL:eq:best_action).

###  3.3 Transition Model

For ease of notation, we denote the three special conditions of "SURVIVED",
"DEAD" and "VACANT" as $\mathbf{1},\mathbf{-1},\mathbf{0}\in\mathbb{R}^{k}$
respectively. In the case without fairness consideration, given current state
$s=[x_{1},x_{2},\cdots,x_{N},I_{1},I_{2},\cdots,I_{N}]$ and action
$a=[a_{1},a_{2},\cdots,a_{N}]$, it will transit to the next state
$s^{\prime}=[x_{1}^{\prime},x_{2}^{\prime},\cdots,x_{N}^{\prime},I_{1}^{\prime},I_{2}^{\prime},\cdots,I_{N}^{\prime}]$
in the following coordinate-wise way:

  1. 1.

If $x_{i}\neq\mathbf{0},\mathbf{1},\mathbf{-1}$, then the patient will transit
to "DEAD" condition if not ventilated:

$\displaystyle
P^{\text{clinic}}_{i}(x_{i}^{\prime}|s,a)=\begin{cases}1&\text{if
}a_{i}=0,x_{i}^{\prime}=\mathbf{-1},\\\ 0&\text{if
}a_{i}=0,x_{i}^{\prime}\neq\mathbf{-1},\\\
p^{\hbox{on}}(x_{i}^{\prime}|x_{i})&\text{if
}a_{i}=1,x^{\prime}\neq\mathbf{0},\\\ 0&\text{if
}a_{i}=1,x_{i}^{\prime}=\mathbf{0},\end{cases}$ (3)

where $p^{\hbox{on}}(x_{i}^{\prime}|x_{i})$ denotes the probability of a
patient transiting from medical condition $s_{i}$ to $s_{i}^{\prime}$ given
ventilation. The ventilation status is naively transited
$P^{\text{vent}}_{i}(I_{i}^{\prime}|s,a)=1\text{ if }I_{i}^{\prime}=a$ and 0
otherwise.

  2. 2.

If $x_{i}=\mathbf{0},\mathbf{1}\text{ or }\mathbf{-1}$, the action does not
influence the transition dynamic:

$\displaystyle
P^{\text{clinic}}_{i}(s_{i}^{\prime}|s,a)=\begin{cases}0&\text{if
}s_{i}^{\prime}=\mathbf{1}\text{ or }\mathbf{-1},\\\ 1-q_{i}(s)&\text{if
}s_{i}^{\prime}=\mathbf{0},\\\ q_{i}(s)\cdot\xi(s_{i}^{\prime})&\text{if
}s_{i}^{\prime}\neq\mathbf{0},\mathbf{1},\mathbf{-1},\end{cases}$ (4)

where $\xi(\cdot)$ denotes the distribution of the initial medical condition
of a patient, and $q_{i}(s)$ denotes the probability of a new incoming patient
staying in bed $i$. Note $q_{i}(s)$ depends on how new patients are
distributed to empty beds. It satisfies $q_{i}(s)=0$ if the bed $i$ is already
occupied and $\sum_{i=1}^{N}q_{i}(s)=\Lambda$ assuming the number of incoming
patients obeys an Exponential distribution with parameter $\Lambda$. The
ventilation status does not matter at this point, we can set
$P^{\text{vent}}_{i}(I_{i}^{\prime}|s,a)=1\text{ if }I_{i}^{\prime}=0$ and 0
otherwise.

Given above discussion for transition dynamics of each individual
patients/bed, the overall transition probability can be written as

$\displaystyle
P=\text{diag}\left(\text{diag}\\{P^{\text{clinic}}_{i}\\}_{i=1}^{N},\text{diag}\\{P^{\text{vent}}_{i}\\}_{i=1}^{N}\right).$
(5)

In the case with fairness consideration, we further consider the progress of
the number of cumulative patients and ventilated patients of each ethnic
group, whose deterministic transitions are naive by their definitions.

###  3.4 Reward

The reward function consists of the following three parts:

  1. 1.

Ventilation cost $\mathbf{R_{v}}$: if a bed uses a ventilator, it occurs a
small negative reward $c_{1}$:

$\displaystyle\mathbf{R_{v}}(s,a)=c_{1}\cdot\sum_{i=1}^{N}a_{i}.$ (6)

  2. 2.

Terminal condition $\mathbf{R_{t}}$: If a patient is discharged alive after
being on a ventilator, a positive reward of 1 is given. If a patient requires
a ventilator but is not able to receive one, or dies after being on a
ventilator, a penalty of -1 is given:

$\displaystyle\mathbf{R_{t}}(s,a)=\sum_{i=1}^{N}\mathbf{1}[s_{i}=\mathbf{1}]-\sum_{i=1}^{N}\mathbf{1}[s_{i}=\mathbf{-1}].$
(7)

  3. 3.

Fairness penalty $\mathbf{R_{f}}$: in the case with fairness consideration, we
consider the cumulative total and ventilated patients of different ethnic
groups, we hope the distribution of ventilators is fair in terms of the
proportion of ventilated patients of all groups. Therefore, a penalty of KL-
divergence between the frequency distributions of incoming patients of
different ethnoracial groups
$\mathcal{D}_{n}\sim[n_{B},n_{W},n_{A},n_{H}]/(n_{B}+n_{W}+n_{A}+n_{H})$ and
ventilated patients of different ethnoracial groups
$\mathcal{D}_{m}\sim[m_{B},m_{W},m_{A},m_{H}]/(m_{B}+m_{W}+m_{A}+m_{H})$ is
considered:

$\displaystyle\mathbf{R_{f}}(s,a)=\hbox{KL}\left(\mathcal{D}_{n}\large\|\mathcal{D}_{m}\right).$
(8)

Thus, the reward function is given by

$\displaystyle
R(s,a)=\mathbf{R_{v}}(s,a)+\mathbf{R_{t}}(s,a)+\lambda\cdot\mathbf{R_{f}}(s,a),$
(9)

where the parameter $\lambda\geq 0$ balances the trade-off between ventilation
effectiveness and fairness. In the case without fairness consideration, we set
$\lambda=0$.

##  4 Method

###  4.1 Transformer-based Double Deep Q-Network

In our problem formulation, the transition model is decomposable: the
distribution of the next state of the $i$-th patient $s_{i}^{\prime}$ only
depends on the patient’s individual current state-action $(s_{i},a_{i})$ and
is independent of other patients. This structure simplifies the problem to
some extent, as the transitions of the patients’ states can be modeled and
learned separately. Nevertheless, due to the resource constraints imposed
through the restricted action sets and the fairness requirements via reward
penalties, the interaction effects among the patients need to be considered.
However, modeling all the patients jointly creates computational challenges,
as the action space is combinatorial. Also, naively concatenating all
patients’ state vectors introduces an order among patients, which may cause
the model to learn artificial factors on the order of the patients and
potentially biases the model.

To circumvent the computational intractability without losing the
consideration of the interaction effects among patients, we propose
Transformer Q-network parametrization, which is tailored to our problem
structure. Specifically, the Q-network is parametrized as
$Q_{\theta}:\mathbb{R}^{\text{dim}(\mathcal{S})}\to\mathbb{R}^{N\times 2}$,
whose input is the state vectors. The $i$-th row of the output
$Q_{\theta}(s)_{i}\in\mathbb{R}^{2}$ corresponds to the Q-value contributed by
the $i$-th patient given the current state $s$ and the action on the $i$-th
patient. We adopt an additive form of the joint Q-value:

$Q(s,a)=\sum_{i=1}^{N}Q_{\theta}(s)_{i,a_{i}}.$

In this parametrization, the greedy action
$a^{*}=\arg\max_{a\in\mathcal{A}}Q(s,a)$ can be solved efficiently via finding
the extra ($B-|I|$)-largest indices of
$\\{(Q_{\theta}(s)_{i,1}-Q_{\theta}(s)_{i,0})|1\leq i\leq N,I_{i}=0\\}$:

$\displaystyle a^{*}_{i}=1\Leftrightarrow$
$\displaystyle~{}I_{i}=1\qquad\qquad\qquad\qquad\qquad\qquad\qquad\text{($|I|$
patients already in ventilation)}$ (10) $\displaystyle\text{ or
}s:I_{i}=0\text{ and }Q_{\theta}(s)_{i,1}-Q_{\theta}(s)_{i,0}\geq 0\text{ and
}Q_{\theta}(s)_{i,1}-Q_{\theta}(s)_{i,0}\text{ ranks top-($B-|I|$)}$ (11)

where the action set $\mathcal{A}$ are defined in Eq. (2).

In our implementation, we use Transformer [28] for $Q_{\theta}$. Transformers
without positional encodings are permutationally-invariant and can handle
inputs with arbitrary sizes. As our input to the model is essentially an
orderless set of patients with indefinite size, Transformers are a natural fit
for our problem. Each Transformer layer is composed of an MLP layer and an
attention layer. The MLP layer acts independently on each element of the input
as a powerful feature extractor, while the attention layer is able to capture
the interaction effects between all the elements. We combine the Transformer
q-network parametrization with Double-DQN [29], a slight modification to the
original DQN [30] capable of reducing overestimation. We refer to the model
without fairness rewards as TDDQN and the model with fairness rewards as
TDDQN-fair.

##  5 Experiments

###  5.1 Dataset

The dataset is sourced from a comprehensive and integrated repository of all
clinical and research data sources, including electronic health records,
pathology data from multiple real-world hospitals and research laboratories.
We collected all the ICU admissions that have been allocated mechanical
ventilator between March 15, 2020 and January 15, 2023. We filtered the
patients with age between 18 and 95 for consideration. We removed admissions
with a ventilator allocation duration exceeding 30 days to eliminate anomalies
in the data. We extracted 38 features for each admission, including SOFA
components, vital signs, demographics, commorbidities (see supplementary for
more details).

We split our data into 3 splits, admissions between March 15, 2020 and July
14, 2021 were used as training data; admissions between July 15, 2021 and
October 14 2021 were used as validation data on which we selected the best
model for test; admissions between October 15, 2021 and January 15, 2023 were
used as test data. The patient distribution of different races are summarized
in Table 1. The ventilator demands on each day are shown in Figure 2. We
assume that the original dataset obtained from health systems reflects a
scenario where there was an abundant supply of ventilators available. We aim
to investigate the effectiveness of various protocols in mitigating excess
deaths in the event of a reduced number of available ventilators.

Table 1:  Summary statistics of our dataset. For rows ’Train’, ’Validation’,
’Test’ and ’Overall’, N(P%) are the number of patients and the percentage out
of all races. For rows ’Female’ and ’Deaths’, N(P%) are the number of patients
and the percentage of females and deaths within that race. $\pm$ denotes std.
’Vent days’ is the number of days a patient has been allocated a ventilator.

| Asian | Black | Hispanic | White | All races
---|---|---|---|---|---
Train | 206 (3.8%) | 871 (16.0%) | 668 (12.2%) | 3381 (62.0%) | 5455
Validation | 34 (3.2%) | 161 (15.4%) | 109 (10.4%) | 663 (63.3%) | 1047
Test | 239 (4.5%) | 704 (13.4%) | 497 (9.4%) | 3465 (65.7%) | 5271
Overall | 479 (4.1%) | 1736 (14.7%) | 1274 (10.8%) | 7509 (63.8%) | 11773
Age | 62.3$\pm$16.0 | 57.5$\pm$16.2 | 55.9$\pm$15.8 | 64.2$\pm$14.7 | 62.0$\pm$15.5
Female | 184 (38.4%) | 814 (46.9%) | 495 (38.9%) | 2848 (37.9%) | 4630 (39.3%)
Deaths | 122 (25.5%) | 409 (23.6%) | 358 (28.1%) | 1870 (24.9%) | 2980 (25.3%)
Vent days | 4.9$\pm$6.1 | 5.9$\pm$6.7 | 6.5$\pm$7.3 | 4.3$\pm$5.3 | 4.8$\pm$5.9

Figure 2: Daily Ventilator Demands

###  5.2 Technical Details

To train the TDDQN model, we developed a simulator by sampling patients’
trajectories from the training set. This simulator maintained the clinical
trajectory of each patient while randomizing their relative admission order.
At each time step, we sampled the initial condition of $\mathbb{E}$ patients
from the training cohort without replacement. The value of $\mathbb{E}$ is
determined by a Poisson distribution at $\lambda=12$. Patients who are not
allocated with ventilators or those who are discharged (either alive or
deceased) were removed from the ongoing simulator and returned to the sampling
pool. Patients who were allocated ventilators progressed to the conditions at
the next time step of their own trajectory. This simulator allowed us to
simulate and train the model by incorporating realistic patient trajectories
and iteratively updating our models in the offline setting. We derived 85
capacity-specific models, considering the maximum daily demand in the
validation and testing sets to be 85. The ventilator capacity is normalized
from the range of [0, 85] to the range of [0%, 100%].

We compared the performance of the trained protocol with other protocols in
allocating ventilators using real-world trajectories from our evaluation and
testing dataset. We evaluated the performance of our proposed protocol based
on normalized survival rates, with the survival number at no shortage in
ventilators as 100% and no patients alive when no ventilator is available as
0%. Fairness was quantified by comparing the allocation rate across four
ethnoracial groups. The allocation rate was calculated by dividing the total
number of ventilators allocated by the sum of the ventilators requested. The
demographic parity ratio (DPR) [31] served as the group metric of fairness,
and it is defined as the ratio between the smallest and largest group-level
allocation rate. A DPR value close to 1 signifies an equitable allocation,
indicating a non-discriminatory distribution among the different groups.
Additionally, to gain insights into the impact of protocols under varying
shortage levels, we visualized the survival-capacity curve (SCC) and
allocation-capacity curve (ACC). Our experiments were conducted on firewall-
protected servers. The training time for each capacity-specific model was less
than 30 minutes using a GPU backend. For more technical details, such as
hyperparameter selections, abalation studies, and model configurations please
refer to the supplemental material.

###  5.3 Baseline Protocols

Our proposed method was compared with the following existing triage protocols:
Lottery: Ventilators are randomly assigned to patients who are in need.
Youngest First: The highest priority is given to the youngest patients. SOFA:
Patients’ prioritization is discretized into three levels (0-7: high, 8-11:
medium, and >11: low) with the lottery serving as the tiebreaker.
Multiprinciple (MP): Each patient is assigned a priority point based on their
SOFA score (0-8: 1, 9-11: 2, 12-14: 3, 14+: 4). Patients with severe
comorbidities receive an additional 3 points. In case of ties, priority is
given to patients in a younger age group (Age groups: 0–49, 50–69, 70–84, and
>=85). If ties still exist, a lottery is conducted to determine the final
allocation.

###  5.4 Results

Our findings regarding the survival rates for different triage protocols under
various levels of ventilator shortage are illustrated in Figure 3. Across all
triage protocols, higher ventilator capacities are associated with increased
allocation rates, thereby resulting in saving more lives. The Area Under the
Survival-Capacity Curve (AUSCC) serves as an indicator of the overall
performance in terms of life-saving abilities under different levels of
ventilator shortage. Similarly, the Area Under the Allocation-Capacity Curve
(AUACC) reflects the overall performance of ventilator utilization rates under
varying levels of shortage. In Panel A of Figure 3, our methods TDDQN and
TDDQN-fair exhibit higher AUSCC values compared to all other baseline
protocols, demonstrating the superiority of our models in terms of life-saving
efficacy. Likewise, in Panel B, TDDQN and TDDQN-fair demonstrate higher AUACC
values compared to other baselines, indicating their superiority in terms of
ventilator utilization.

Figure 4 showcases the allocation-capacity curves for different ethnoracial
groups and triage protocols in Panels A through E. Panel I displays the
corresponding AUACC values. Only Lottery and our TDDQN-fair model exhibit
minimal disparities, but TDDQN-fair surpasses Lottery in terms of allocation
rates and life-saving efficacy. Conversely, all other triage protocols display
a preference for specific ethnoracial groups. For example, Youngest and MP
favor the Hispanic group, while SOFA and TDQN favor the White group.

We conducted a detailed analysis in Table 2 under the scenario where
approximately 50% of ventilators are unavailable. Our results show that TDDQN-
fair achieves the highest survival rate and allocation rate among the triage
protocols. It ranks second in terms of demographic parity ratio, following
closely behind the Lottery protocol. These findings confirm the effectiveness
of our TDDQN-fair approach in improving both survival rates and fairness
simultaneously. Importantly, the inclusion of fairness rewards does not
compromise its life-saving capabilities compared to the TDDQN configuration.

Table 2: Impact of triage protocols on survival, fairness, and allocation
rates with limited ventilators (B = 40) corresponding to about 50% scarcity.
TDDQN and TDDQN-fair are without and with fairness rewards, respectively.
Fairness metric is demographic parity ratio (DPR, the ratio between the
smallest and the largest allocation rate across patient groups, 100%
indicating non-discriminative). Standard deviations are from 10 experiments
with different seeds. We bold the ethnoracial group results with the highest
survival rate, DPR, and overall allocation rates. Underscore in fairness
metric denotes that Lottery and TDDQN-fair DPRs are not significantly
different from each other.

| Performance | Fairness | Allocation Rates
---|---|---|---
| Survival, % | DPR, % | Overall, % | Asian, % | Black, % | Hispanic, % | White, %
Lottery | 72.23 ± 0.45 | 96.89 ± 1.55 | 75.60 ± 0.28 | 74.95 ± 1.81 | 75.65 ± 1.10 | 76.08 ± 0.90 | 75.68 ± 0.33
Youngest | 74.66 ± 0.07 | 86.39 ± 0.27 | 75.65 ± 0.09 | 75.59 ± 0.50 | 81.73 ± 0.31 | 84.44 ± 0.30 | 72.94 ± 0.12
SOFA | 78.30 ± 0.32 | 92.37 ± 1.01 | 77.92 ± 0.25 | 75.06 ± 1.44 | 73.58 ± 0.77 | 78.66 ± 1.24 | 79.10 ± 0.45
MP | 79.41 ± 0.14 | 90.68 ± 0.95 | 78.34 ± 0.20 | 77.91 ± 1.48 | 74.33 ± 0.66 | 81.96 ± 0.56 | 78.81 ± 0.34
TDDQN | 82.18 ± 0.24 | 86.91 ± 3.45 | 81.80 ± 0.26 | 78.03 ± 0.99 | 72.48 ± 0.74 | 81.71 ± 0.44 | 84.45 ± 0.22
TDDQN-fair | 82.83 ± 0.23 | 95.24 ± 1.65 | 81.90 ± 0.24 | 80.01 ± 1.79 | 79.95 ± 1.05 | 81.26 ± 1.24 | 82.80 ± 0.42

Figure 3: Impact of triage protocols on survival rates and allocation rates under varying levels of ventilator shortages. The maximum daily demand for ventilators in the testing set is considered as full capacity (100%). We scale the number of survivors to a range of [0, 100%] to represent the survival rate. Any capacity below full capacity results in lower survival rates due to ventilator shortages. The allocation rate is calculated by dividing the total number of ventilators allocated by the total number of the ventilators requested. The bar plot associated with each panel indicates the area under the survival-capacity curve and allocation-capactiy curve, respectively, where a larger value indicates that the protocol can save more lives across different levels of shortages. Notably, the MP and SOFA curves exhibit overlap, indicating similar allocation patterns. Similarly, the lottery and youngest curves show close proximity, as do our two TDDQN configurations. Figure 4: Allocation Rates across different protocols and ethnoracial groups. Panels A to F depict the ethnoracial differences in allocation rates when applying different protocols. Each panel focuses on a specific protocol and shows how the allocation rates vary across ethnoracial groups. Panel G presents the area under the allocation-capacity curve across all protocols and all ethnoracial groups. Table 3: Ablation study on rewards | Rewards | Survival, % | DPR, %
---|---|---|---
TDDQN - fair |  $\mathbf{R_{v}}$ \+ $\mathbf{R_{t}}$ \+ $\mathbf{R_{f}}$ | 82.83 | 95.24
TDDQN |  $\mathbf{R_{v}}$ \+ $\mathbf{R_{t}}$ | 82.18 | 90.68
TDDQN-~~$\mathbf{R_{v}}$~~ |  $\mathbf{R_{t}}$ \+ $\mathbf{R_{f}}$ | 80.02 | 94.47
TDDQN-~~$\mathbf{R_{t}}$~~ |  $\mathbf{R_{v}}$ \+ $\mathbf{R_{f}}$ | 73.44 | 95.32

###  5.5 Ablation Studies

We conducted ablation studies to analyze the impact of each reward component.
The results indicate that if the ventilation cost or terminal conditions are
ignored, the model can achieve a fair distribution, but at the expense of
reduced life-saving ability. This finding underscores the superiority of our
proposed model in achieving both optimized and equitable allocation. For
additional ablation studies, such as those related to model architecture,
please refer to the supplemental material.

##  6 Summary

In this study, we formulated fair health care resource allocation as a multi-
objective deep reinforcement learning problem. We developed a transformer-
based deep Q-network (DQN) to integrate disease progression of individual
patients and the interaction effects among patients in order to optimize for
an efficient and fair allocation policy. Our proposed model outperforms
existing protocols used by different state governments in the U.S., by saving
more lives and achieving a more equitable distribution of ventilators, as
demonstrated by experiments on real-world data. We also showed that adding the
fairness component into the reward of our multi-objective TDDQN improved
fairness to an ideal level while still maintaining superior overall patient
survival. We refer the readers to Appendix 5 for a detailed discussion on the
limitations and future directions of this study.

## References

  * Gostin et al. [2012] Lawrence O Gostin, Kristin Viswanathan, Bruce M Altevogt, Dan Hanfling, et al.  _Crisis standards of care: a systems framework for catastrophic disaster response: Volume 1: Introduction and CSC framework_ , volume 3.  National Academies Press, 2012.
  * Piscitello et al. [2020] Gina M Piscitello, Esha M Kapania, William D Miller, Juan C Rojas, Mark Siegler, and William F Parker.  Variation in ventilator allocation guidelines by us state during the coronavirus disease 2019 pandemic: a systematic review.  _JAMA network open_ , 3(6):e2012606–e2012606, 2020.
  * Emanuel et al. [2020] Ezekiel J Emanuel, Govind Persad, Ross Upshur, Beatriz Thome, Michael Parker, Aaron Glickman, Cathy Zhang, Connor Boyle, Maxwell Smith, and James P Phillips.  Fair allocation of scarce medical resources in the time of covid-19, 2020\.
  * Vincent et al. [1996] J L Vincent, Rui Moreno, Jukka Takala, Sheila Willatts, Arnaldo De Mendonça, Hajo Bruining, CK Reinhart, PeterM Suter, and Lambertius G Thijs.  The sofa (sepsis-related organ failure assessment) score to describe organ dysfunction/failure: On behalf of the working group on sepsis-related problems of the european society of intensive care medicine (see contributors to the project in the appendix), 1996.
  * VEN [2015] Ventilator allocation guidelines, Nov 2015.  URL https://www.health.ny.gov/regulations/task_force/reports_publications/docs/ventilator_guidelines.pdf.
  * Biddison et al. [2019] E Lee Daugherty Biddison, Ruth Faden, Howard S Gwon, Darren P Mareiniss, Alan C Regenberg, Monica Schoch-Spana, Jack Schwartz, and Eric S Toner.  Too many patients… a framework to guide statewide allocation of scarce mechanical ventilation during disasters.  _Chest_ , 155(4):848–854, 2019.
  * Bhavani et al. [2021] Sivasubramanium V Bhavani, Yuan Luo, William D Miller, Lazaro N Sanchez-Pinto, Xuan Han, Chengsheng Mao, Burhaneddin Sandıkçı, Monica E Peek, Craig M Coopersmith, Kelly N Michelson, et al.  Simulation of ventilator allocation in critically ill patients with covid-19.  _American journal of respiratory and critical care medicine_ , 204(10):1224–1227, 2021.
  * Raschke et al. [2021] Robert A Raschke, Sumit Agarwal, Pooja Rangan, C William Heise, and Steven C Curry.  Discriminant accuracy of the sofa score for determining the probable mortality of patients with covid-19 pneumonia requiring mechanical ventilation.  _Jama_ , 325(14):1469–1470, 2021.
  * Emanuel and Wertheimer [2006] Ezekiel J Emanuel and Alan Wertheimer.  Who should get influenza vaccine when not all can?  _Science_ , 312(5775):854–855, 2006.
  * Truog et al. [2006] Robert D Truog, Dan W Brock, Deborah J Cook, Marion Danis, John M Luce, Gordon D Rubenfeld, Mitchell M Levy, et al.  Rationing in the intensive care unit.  _Critical care medicine_ , 34(4):958–963, 2006\.
  * Persad et al. [2009] Govind Persad, Alan Wertheimer, and Ezekiel J Emanuel.  Principles for allocation of scarce medical interventions.  _The lancet_ , 373(9661):423–431, 2009.
  * Dresser [1992] Rebecca Dresser.  Wanted single, white male for medical research.  _The Hastings Center Report_ , 22(1):24–29, 1992\.
  * Tamayo-Sarver et al. [2003] Joshua H Tamayo-Sarver, Susan W Hinze, Rita K Cydulka, and David W Baker.  Racial and ethnic disparities in emergency department analgesic prescription.  _American journal of public health_ , 93(12):2067–2073, 2003.
  * Chen et al. [2008] Esther H Chen, Frances S Shofer, Anthony J Dean, Judd E Hollander, William G Baxt, Jennifer L Robey, Keara L Sease, and Angela M Mills.  Gender disparity in analgesic treatment of emergency department patients with acute abdominal pain.  _Academic Emergency Medicine_ , 15(5):414–418, 2008.
  * Mehrabi et al. [2021] Ninareh Mehrabi, Fred Morstatter, Nripsuta Saxena, Kristina Lerman, and Aram Galstyan.  A survey on bias and fairness in machine learning.  _ACM Computing Surveys (CSUR)_ , 54(6):1–35, 2021\.
  * Liu et al. [2018] Lydia T Liu, Sarah Dean, Esther Rolf, Max Simchowitz, and Moritz Hardt.  Delayed impact of fair machine learning.  In _International Conference on Machine Learning_ , pages 3150–3158. PMLR, 2018.
  * Pessach and Shmueli [2022] Dana Pessach and Erez Shmueli.  A review on fairness in machine learning.  _ACM Computing Surveys (CSUR)_ , 55(3):1–44, 2022\.
  * Binns [2018] Reuben Binns.  Fairness in machine learning: Lessons from political philosophy.  In _Conference on fairness, accountability and transparency_ , pages 149–159. PMLR, 2018.
  * Ahmad et al. [2020] Muhammad Aurangzeb Ahmad, Arpit Patel, Carly Eckert, Vikas Kumar, and Ankur Teredesai.  Fairness in machine learning for healthcare.  In _Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining_, pages 3529–3530, 2020.
  * Rajkomar et al. [2018] Alvin Rajkomar, Michaela Hardt, Michael D Howell, Greg Corrado, and Marshall H Chin.  Ensuring fairness in machine learning to advance health equity.  _Annals of internal medicine_ , 169(12):866–872, 2018.
  * Pfohl et al. [2021] Stephen R Pfohl, Agata Foryciarz, and Nigam H Shah.  An empirical characterization of fair machine learning for clinical risk prediction.  _Journal of biomedical informatics_ , 113:103621, 2021.
  * Wang et al. [2022] Hanyin Wang, Yikuan Li, Andrew Naidech, and Yuan Luo.  Comparison between machine learning methods for mortality prediction for sepsis patients with different social determinants.  _BMC Medical Informatics and Decision Making_ , 22(2):1–13, 2022.
  * Li et al. [2022] Yikuan Li, Hanyin Wang, and Yuan Luo.  Improving fairness in the prediction of heart failure length of stay and mortality by integrating social determinants of health.  _Circulation: Heart Failure_ , 15(11):e009473, 2022.
  * Zong and Luo [2022] Kai Zong and Cuicui Luo.  Reinforcement learning based framework for covid-19 resource allocation.  _Computers & Industrial Engineering_, 167:107960, 2022\.
  * Bednarski et al. [2021] Bryan P Bednarski, Akash Deep Singh, and William M Jones.  On collaborative reinforcement learning to optimize the redistribution of critical medical supplies throughout the covid-19 pandemic.  _Journal of the American Medical Informatics Association_ , 28(4):874–878, 2021.
  * Bastani et al. [2021] Hamsa Bastani, Kimon Drakopoulos, Vishal Gupta, Ioannis Vlachogiannis, Christos Hadjichristodoulou, Pagona Lagiou, Gkikas Magiorkinis, Dimitrios Paraskevis, and Sotirios Tsiodras.  Efficient and targeted covid-19 border testing via reinforcement learning.  _Nature_ , 599(7883):108–113, 2021.
  * Yu et al. [2021] Chao Yu, Jiming Liu, Shamim Nemati, and Guosheng Yin.  Reinforcement learning in healthcare: A survey.  _ACM Computing Surveys (CSUR)_ , 55(1):1–36, 2021\.
  * Vaswani et al. [2017] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin.  Attention is all you need.  _Advances in neural information processing systems_ , 30, 2017.
  * Van Hasselt et al. [2016] Hado Van Hasselt, Arthur Guez, and David Silver.  Deep reinforcement learning with double q-learning.  In _Proceedings of the AAAI conference on artificial intelligence_ , volume 30, 2016.
  * Mnih et al. [2013] Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Alex Graves, Ioannis Antonoglou, Daan Wierstra, and Martin Riedmiller.  Playing atari with deep reinforcement learning.  _arXiv preprint arXiv:1312.5602_ , 2013.
  * Bird et al. [2020] Sarah Bird, Miro Dudík, Richard Edgar, Brandon Horn, Roman Lutz, Vanessa Milan, Mehrnoosh Sameki, Hanna Wallach, and Kathleen Walker.  Fairlearn: A toolkit for assessing and improving fairness in ai.  _Microsoft, Tech. Rep. MSR-TR-2020-32_ , 2020.


2021

[1]Steve Macenski

[1]R&D Innovations, Samsung Research America, Clyde Ave, Mountain View, 94043,
CA, United States

2]Department of Computer Science, Manipal Institute of Technology, Udupi -
Karkala Rd, Eshwar Nagar, Manipal, 576104, Karnataka, India

3]Intelligent Robotics Lab, Rey Juan Carlos University, Camino del Molino,
Fuenlabrada, 28943, Madrid, Spain

# Regulated Pure Pursuit for Robot Path Tracking

<EMAIL_ADDRESS>Shrijit Singh<EMAIL_ADDRESS>Francisco Martín<EMAIL_ADDRESS>Jonatan Ginés<EMAIL_ADDRESS>*  [  [

###### Abstract

The accelerated deployment of service robots have spawned a number of
algorithm variations to better handle real-world conditions. Many local
trajectory planning techniques have been deployed on practical robot systems
successfully. While most formulations of Dynamic Window Approach and Model
Predictive Control can progress along paths and optimize for additional
criteria, the use of pure path tracking algorithms is still commonplace.
Decades later, Pure Pursuit and its variants continues to be one of the most
commonly utilized classes of local trajectory planners. However, few Pure
Pursuit variants have been proposed with schema for variable linear velocities
- they either assume a constant velocity or fails to address the point at all.
This paper presents a variant of Pure Pursuit designed with additional
heuristics to regulate linear velocities, built atop the existing Adaptive
variant. The Regulated Pure Pursuit algorithm makes incremental improvements
on state of the art by adjusting linear velocities with particular focus on
safety in constrained and partially observable spaces commonly negotiated by
deployed robots.

We present experiments with the Regulated Pure Pursuit algorithm on
industrial-grade service robots. We also provide a high-quality reference
implementation that is freely included ROS 2 Nav2 framework at
https://github.com/ros-planning/navigation2 for fast evaluation.

###### keywords:

Service Robots, Mobile Robots, Motion Planning, Path Planning

##  1 Introduction

Dynamic Window Approach (DWA) dwa , Pure Pursuit (PP) pure_pursuit , and Model
Predictive Control (MPC) mpc  are by far the most commonly deployed path
trackers. They all have a strong heritage for reliability in a wide range of
environmental conditions. DWA and MPC are often, but not always, formulated as
multi-objective trajectory generation problems to maximize criteria such as
avoiding dynamic obstacle collisions on top of path tracking. This has made
them particularly well suited for many robotics applications where dynamic
robot behaviors are rewarded. A great deal of work has been conducted on these
allowing them to be found on many commercially available robots today.

However, there still exist many applications of deployed robot systems where
this can be considered a detractor. Surveyed among high-end research and
service robot navigation systems, pure path tracking continues to be a common
theme in many robots genora ; se2_nav . Among single-objective path trackers,
a simple and reliable method continues to be exploited decades after its
development: Pure Pursuit.

Pure Pursuit uses simple geometry to find the curvature of a path required to
drive a robot towards a given point on the path. The algorithm itself does not
place any stability restrictions on the translational velocities during
operation, however it also lacks any schema for selecting them. Near-
universally, implementations use a fixed speed. Many variations of Pure
Pursuit exist, however most address the more obvious area of the selection of
lookahead points, which largely aids in stabilizing convergence behaviors
towards the path at a wider range of velocities. The Pure Pursuit algorithm
was not developed with service and industrial robots in mind, which have
additional safety requirements making it further unrealistic to move at a
fixed velocity.

This work proposes an incremental improvement on the Pure Pursuit path
tracking algorithm by describing a reference method of adjusting translational
and rotational velocities to improve safety and operability in a broad range
of common deployed robot applications. We improve the Adaptive Pure Pursuit
(APP) algorithm MITDARPA  by regulating velocities via penalizing sharp
changes in path curvature and proximity to obstacles - two of the most common
events requiring conscientious navigation behaviors. The Regulated Pure
Pursuit (RPP) algorithm slows sharp turns into partially observable dynamic
environments (aisles, hallways, intersections) to reduce the likelihood and
impact of potential collisions. It also reduces linear velocity in close
proximity to obstacles such as people and fixed infrastructure to reduce
likelihood of collision in constrained indoor environments. Finally, it
additionally includes preemptive collision detection missing from other
variations.

Another main contribution of this work is in describing and providing free and
high quality implementations of PP, APP, and RPP for evaluation. It is
integrated into the Nav2 mobile robot navigation system commonly used by
researchers and adds a new capability to the framework nav2 . This work is
well documented, tested, and is in use on robots deployed today.

This paper is organized as follows. First, in section 2, we will describe the
conventional Pure Pursuit Algorithm, providing a mathematical formulation that
will allow us to describe related works that provide variations to the
original algorithm. In section 3, we describe the Regulated Pure Pursuit
Algorithm, our main contribution to this paper. Section 4 will describe its
implementation in a real robotic system, and in section 5, we will
experimentally validate the improvement introduced with our contribution.
Finally, we will provide the conclusions in section 7.

##  2 Related Work

Pure Pursuit MITDARPA  is a widely used algorithm for path tracking. It is
simple but effective. Considering a path $\mathcal{P}$ as an ordered list of
points $\mathcal{P}=\\{p_{0},p_{1},...,p_{n}\\}$ where
$p_{i}=(x_{i},y_{i})\in\mathcal{P}$. A local trajectory planner is a function
$f$ that determines the linear and angular velocity to track a reference path
$\mathcal{P}_{t}$ at time $t$ (Eq. 1).

$(v_{t},\omega_{t})=f(\mathcal{P}_{t})$ (1)

Figure 1 shows visual geometric representations of Pure Pursuit. First, it
determines the closest point $p_{r}$ on $\mathcal{P}_{t}$ to the robot
position. Using a given lookahead distance, $L$, the lookahead point $p_{l}$
is determined as the first $p_{i}$ at least $L$ distance away from $p_{r}$ in
Eq. 2.

$dist(p_{i})=\sqrt{(x_{r}-x_{i})^{2}+(y_{r}-y_{i})^{2}}\\\ $
$p_{l}=p_{i}\in\mathcal{P}_{t},\left\\{\begin{array}[]{l}dist(p_{i-1})<L\\\
dist(p_{i})\geq L\\\ \end{array}\right.$ (2)

With a known $p_{l}$, we can determine curvature of the circle (recall,
$R=1/\kappa$) using simple geometry. If $\mathcal{P}_{t}$ is represented in
vehicle base coordinates, $\mathcal{P^{\prime}}_{t}$, where the robot position
is the origin, then the curvature can be represented as

$\kappa=\frac{2\;y^{\prime}_{l}}{L^{2}}\\\ $ (3)

where $\kappa$ is the path curvature required to drive the robot from its
starting position to the lookahead carrot, $y^{\prime}_{l}$ is the lateral
coordinate of the lookahead point $p^{\prime}_{l}$, and $L$ is the desired
distance between $p_{r}$ and $p_{l}$.

Figure 1 shows this visually, where $L$ is represented geometrically as the
circle’s chord. With curvature to drive towards the lookahead point, the
commands can be send to a robot controller. This process is then updated at
the desired rate.

The primary parameters of the Pure Pursuit path tracker are simply the
velocity of travel $v_{t}$ and the distance $L$ along the path used to select
the lookahead point $p_{l}$. In the standard formulation, this lookahead point
$p_{l}$ is a distance $L$ from the robot tuned to achieve an acceptable trade-
off between oscillations centered around the path (shorter distances) and
slower convergence (longer distances). There exists a broad range of
admissible lookahead distances pure_pursuit .

PP and its variants are applicable to Ackermann and differential-drive robots
due to PP’s formulation supporting dynamics with longitudinal motion and a
turning rate in body-fixed frame. For Ackermann robots to use PP, the path
must be feasible given the kinematic-constraints of the platform - while
differential-drive robots may follow any holonomic path using PP. While omni-
directional robots may utilize PP variants, they will be restricted from
performing lateral movements.

Figure 1: Geometry of finding the path curvature.

There are several known downsides of this approach. In high curvature
situations, Pure Pursuit is known to have overshoot or undershoot behaviors
resulting in path deviations, even in a well tuned system pure_pursuit . This
is typically not a major concern for autonomous driving applications which
naturally has a minimum turning radius limit, but a more substantive issue for
smaller-scale applications like industrial and consumer robots. It also does
not specify any translational velocity specification criteria during
execution. While this can be beneficial as it allows for a great deal of
flexibility with different linear velocity profiles, in practice, without
described methods, nearly all known variants use a constant translational
velocity profile. This is an unsafe defacto-standard.

Variants of this algorithm have been proposed to increase path tracking
stability by varying computations of the lookahead point. MIT’s entry into the
DARPA Urban Challenge implemented the Adaptive Pure Pursuit (APP) algorithm
for lane following while varying the lookahead distances proportionally to the
translational velocity MITDARPA . For velocities in the operating range, a
mapping of lookahead distances to velocities is required such that they have
an acceptable trade-off between oscillation and slower convergence to the
path. A common formulation for this is in Eq. 4, where $L_{t}$ is the
lookahead distance, $v_{t}$ is the translational velocity, and $l_{t}$ is a
lookahead gain representing the amount of time to project $v_{t}$ forward
Interpolation .

$L_{t}=v_{t}l_{t}$ (4)

A recent variant has been created to significantly improve path tracking
accuracy of Pure Pursuit new_pp. This variant also modifies the process of
determining the lookahead point, but instead adjusts this point off the path
to address the long-standing edge case of short-cutting during path tracking
in substantial turns. While it also contains a heuristic change to the
translational velocity during that edge-case, its policy is derived from on-
road properties and ackermann kinematics that does not generalize to other
applications outside of on-road driving of Ackermann vehicles. It does not
solve the problems facing deployment of PP techniques to general mobile
robotics applications for which this work focuses (e.g. practicable
translational velocity control); though its improvements in path tracking
accuracies are notable.

Pure Pursuit nor its variations account for dynamic effects of the vehicle. As
PP is geometrically derived, vehicle dynamics are not modeled in this style of
path tracking algorithm. There are other types of path trackers that do
account for vehicle dynamics survey . This paper uses PP and its variants for
comparison to offer a candid contrast with the same or directly analog
parameters to clearly indicate the differences in performance due to the
contribution 111Not leaving the opportunity for misleading results by
comparing unrelated families of methods with unknown degrees of tuning.

##  3 Regulated Pure Pursuit

The Regulated Pure Pursuit algorithm is designed for service and industrial
mobile robots in real-world constrained and partially observable environments.
It provides methods for adapting the robot’s translational velocity to current
conditions; robot systems cannot simply barrel through aisles and facilities
at full speed without regard. These methods are linear regulation cost
functions that provide high-quality behavior across a variety of practical
mobile robot environments derived from commonplace requirements on reducing
robot velocities in the presence of sharp turns or when operating in confined
regions.

The first phase of the Regulated Pure Pursuit algorithm is to transform the
input path $\mathcal{P}_{t}$ $\\{p_{0},...,p_{j},p_{r},...,p_{n}\\}$ into the
robot’s base coordinate frame $\mathcal{P^{\prime}}_{t}$ and prune it. In
doing so, determining the curvature of the path is reduced to the simple
algebraic expression shown in Eq. 3. Before transformation, $p_{r}$ is
determined and all prior points $\\{p_{0},...,p_{j}\\}$ are permanently pruned
from the stored path to prevent unnecessary future transformation of obsolete
data. The transformed path ($\\{p^{\prime}_{r},...,p^{\prime}_{n}\\}$) is also
pruned for all $p_{i}$ points where
$dist(p^{\prime}_{r},p^{\prime}_{i})>>L_{t}$ as they are sufficiently far away
that they will never need to be considered at $t$. These far path points
continue to be held in the stored path for future iterations, until a new path
is received, as the robot progresses along the path.

Next, RPP will utilize the same lookahead selection mechanics as Adaptive Pure
Pursuit described in Eq. 4. The lookahead distance $L_{t}$ is thusly
proportional to the speed $v_{t}$ and a lookahead gain $l_{t}$ such that
longer distances are used while moving faster. This stabilizes the path
tracking behavior over larger ranges of translational velocities adaptive .
This distance is used to select the lookahead point $p_{l}$. While
interpolating between path points was found to demonstrably improve smoothness
on sparse paths at autonomous vehicle speeds, empirically this did not
contribute much benefit at the service and industrial robot speeds using
typical grid map planning resolutions (0.025 m - 0.1 m) Interpolation .
However, interpolation is beneficial for use in sparser path resolutions (0.1
- 1.0m).

The desired linear velocity, $v_{t}$, is next further processed by the
curvature and proximity heuristics. Both heuristics are applied to linear
velocity and we take the maximum of the two.

The purpose of the curvature heuristic is to slow the robot to
$v^{\prime}_{t}$ during sharp turns into partially observable environments,
such as when entering or exiting hallways and aisles commonly found in retail,
warehouses, factories, schools, and shopping malls. This allows for
significantly safer traversal when making blind turns. This heuristic is
applied to the linear velocity $v_{t}$ when the change in curvature $\theta$
is above a minimum threshold $T_{\theta}$. This minimum radius restricts
velocity scaling in minor turns or path variations that do not require
slowing. The curvature velocity $v^{\prime}_{t}$ returned by this heuristic is
selected as:

$v^{\prime}_{t}=\begin{cases}v_{t}&\kappa>T_{\kappa},\\\
\frac{v_{t}}{r_{min}\;\kappa}&\kappa\leq T_{\kappa}\\\ \end{cases}$ (5)

Where $r_{min}$ is the minimum radius to apply the heuristic. This formulation
is a mathematical reduction of a simple error calculation between the minimum
radius and the radius of a circle represented by $\kappa$ and can be trivially
derived.

The proximity heuristic is applied to the linear velocity $v_{t}$ when the
robot becomes in close proximity to dynamic obstacles or fixed infrastructure.
The purpose of this is to slow the robot when in constrained environments
where the potential for collision is particularly high. Reducing the speed
near fixed infrastructure lowers the likelihood of collision by lessening the
impact of small path variations in tight spaces. Lowering the speed of
industrial and service robots in close proximity to dynamic agents, such as
humans, is a common safety requirement- allowing a robot to reactively stop
faster to prevent potential injury. The linear formulation of this heuristic
reduces the speed by the ratio of $d_{O}/d_{prox}$ with a gain, $\alpha$, to
adjust the response for individual systems. The other formulations tested,
such as exponential and quadratic, far too significantly penalized proximity
to objects and contained only a narrow band of gains which would result in an
acceptable trade-off between proximity to obstacles and velocities to
accomplish the robotic task. This linear heuristic in Eq. 6 has a broad range
of $\alpha$ which may be finely tuned by a system designer and is derivative
of the well-used Adaptive Pure Pursuit’s formulation adaptive .

$v^{\prime}_{t}=\begin{cases}v_{t}\;\frac{\alpha\;d_{O}}{d_{prox}}&d_{O}\leq
d_{prox}\\\ v_{t}&d_{O}>d_{prox}\\\ \end{cases}$ (6)

Where $d_{prox}$ is the proximity distance to obstacles to apply the
heuristic, $d_{O}$ is the current distance to an obstacle, and $\alpha$ is a
gain to scale the heuristic function for aggressive behavior, with the
requirement that $\alpha\leq 1.0$. A higher $\alpha$ lowers the velocity of
the robot in proximity to obstacles more expeditiously. The value of
$d_{prox}$ should be established based on the system requirements of a robot’s
application for how close an obstacle can be before the robot begins slowing
its maximum velocity.

After velocity regulation, the algorithm then determines the path curvature
using Eq. 1. The angular velocity is computed using the regulated velocity,
not desired linear velocity, which prevents consequential undershoot behavior
relative to target curvatures pure_pursuit . Finally, the angular velocity,
$\omega_{t}$, is then trivially found as Eq. 7.

$\omega_{t}=v^{\prime}_{t}\;\kappa$ (7)

The final step of the algorithm is to check our path tracking command for
current or imminent collisions, new to RPP. A given angular velocity
$\omega_{t}$ and regulated linear velocity $v^{\prime}_{t}$ can be projected
forward in time, resulting in a circular arc. Points on the arc are sampled at
the grid map cell resolution forward for a set duration. Collision checking is
done based on a duration to collision rather to the lookahead point such that
the robot is always, at minimum, a set duration from collision. At slow
speeds, it may not be sensible to collision check to a lookahead point tens of
meters, or hundreds of seconds, away. Rather, a temporal schema allows for
fine maneuvers in confined spaces where the current velocity commands may not
be admissible a short distance - but long time - away.

This controller is known to converge from the stability analysis completed in
Ollero95 . Ollera’s work on the general Pure Pursuit algorithm applies to
Regulated Pure Pursuit as well, showing that straight paths and constant
curvature paths are stable. Pure Pursuit itself is independent of velocity in
its formulation. It is only used in the final step to convert the path
curvature into a rotational velocity for a base to track. The regulation
heuristics proposed in this variant do not change the basis of stability of
the root algorithm.

##  4 Implementation

Another contribution of this work is a high-quality reference implementation
of Regulated Pure Pursuit. This reference implementation is available as one
of the default path tracking algorithm plugins in the new and improved ROS 2
Navigation System, known as Nav2. Nav2 is a scalable navigation framework with
multiple algorithm implementations, documentation, and support for building
modern and reliable research and commercial navigation systems nav2 . This
optimized C++ implementation has 92% unit test coverage and is used in the
experiments in Section 5. It also contained a few additional features that are
highlighted in this section.

Rather than creating distinct implementations of Pure Pursuit, Adaptive Pure
Pursuit, and Regulated Pure Pursuit, all three have been built into a single
reference implementation parameterized to enable each specific behavior. This
allows us to test these algorithms easily and analyze their run-time
performance more closely knowing that they share the vast majority of their
computations. This also allows researchers and developers to quickly evaluate
and tune these features by simply changing a handful of parameters from the
available binaries.

Additionally, there is a setting to allow the robot to slow its speed when
approaching the target goal pose. This feature allows the robot to come to a
more gradual stop. When the robot is within close proximity to the goal, the
translational velocity is lowered proportional to the remaining distance, up
to a minimum viable velocity to make progress.

Figure 2: The Pal Robotics Tiago used in the experiments tiago_img .

This work is specifically targeted at industrial and service robots, many of
which are differential drive. While this is of no technical concern to Pure
Pursuit, many differential drive robots will utilize holonomic search-based
path planners (such as Dijkstra’s or A*) rather than kinematically feasible
planners frequently used by Ackermann vehicles brian . This is of concern to
the Regulated Pure Pursuit implementation as the relative heading of the path
may not align with the robot’s starting heading. In this case, it is
beneficial for the robot to be able to rotate to a rough starting heading
before beginning to track the holonomic path. Further, rotating to the final
heading when the robot is within the translational goal tolerances is useful
when being deployed in orientation-sensitive applications. These are non-
issues when using feasible planning algorithms that, no matter the drivetrain.
However, it is useful to consider this situation since holonomic grid search
algorithms are popular.

Further, it is a common approach for local trajectory algorithms to leverage a
rolling environmental model with the robot at its center. The predictive
collision detection algorithm will check for collisions some N seconds in the
future given the robot’s current commanded speed. If a near-future collision
is detected, the robot is stopped. When a robot is traveling at high speeds,
this may actually lapse outside of the bounds of a rolling environmental model
if not carefully tuned. The implementation offers protections from such
situations and issues stern recommendations to prevent the serious safety
violation.

Users are able to tune all of the parameters related to PP, APP, and RPP. In
particular, the heuristics have parameters $r_{min}$, $\alpha$, and
$d_{prox}$. It is possible for these to be poorly tuned such that a robot
would drop to an impractically low speed when in close proximity to an
obstacle or in a turn. To prevent the robot from traveling too slow, a minimum
speed threshold is exposed such that the robot’s speed will never be set below
a parameterized value. Note: the parameters and speeds in the experiment were
set up such that this did not impact our experimental results in Section 5.

An open-source community contributed feature made available is support for
reversing - not only driving forward. When a path contains a cusp or
discontinuity in direction, the controllers will reverse from forward to back
and vis-a-via. When paired with a planner that can generate such paths, such
as the Hybrid-A* planner in Nav2, the robot can track more complex paths.

##  5 Experiments and Analysis

This section describes the experiments to demonstrate the benefits of our
contribution compared to other existing Pure Pursuit variants. One experiment
was conducted in simulation and three others were conducted with a physical
Tiago robot (Figure 2). Tiago is used in industrial and service applications
and the test environment is shown in Figure 7, corresponding to a university
campus building.

In all three hardware experiments below, the same navigation configurations
are utilized. The maximum linear speed $v_{max}$ is set to $0.8m/s$, the
maximum acceleration $a_{max}$ is $0.2m/s^{2}$, and the maximum angular speed
$\omega_{max}$ is $3.2rad/s^{2}$. The lookahead distance $L_{t}$ is set
between $0.25m$ and $1.2m$ and lookahead time is $1.0s$ for Adaptive and
Regulated Pure Pursuit; while Pure Pursuit uses the maximum of $1.2m$. In all
four experiments, no replanning was utilized such that the distance to path
data is regular and can be meaningfully analyzed. SLAM Toolbox was used to
generate the 2D map and AMCL was used to localize within it slam ; nav2 .

###  5.1 Path Tracking Experiment

A simulation experiment was conducted to analyze the improvements Regulated
Pure Pursuit can offer during sharp turns in ideal conditions. This simulation
was conducted using the Turtlebot 3 robot, the Gazebo simulator, and an empty
environment with ground truth information made available to remove the
contribution of odometric and localization error. The robot followed the
reference step function path indicated by the red piece-wise line in Figures 3
and 4 using the three variants of Pure Pursuit.

Figure 3: Comparison of variants of Pure Pursuit.

Figure 3 displays the comparison with the different methods. The same
parameters were utilized in this experiment as the hardware experiments, with
the exception of the desired linear velocity, set to $1.0m/s$ and the
curvature heuristic minimum radius, set to $1.5m$. The overshoot observed
during sharp turns was notably lower in Regulated Pure Pursuit due to the
controlled slow down during sharp changes in curvature allowing the robot to
track the path more finely. The slow down also triggers the use of a closer
adaptive lookahead point further increasing local stability. The mean tracking
error from the reference path of Regulated Pure Pursuit was merely $0.03m$.
The others experienced over an order of magnitude difference at $0.10m$ and
$0.19m$ for APP and PP, respectively. While the main contribution of this work
is not an improvement on path tracking, it is a convenient emergent property
to further ensure safety. This experiment showcases that the algorithm can
follow a computed collision-free path in confined spaces more closely and
thusly more safely.

Figure 4: Effects of varying $r_{min}$ in Regulated Pure Pursuit.

Using the ideal environment, the effects of varying the curvature heuristic’s
minimum radius were also studied. From Figure 4, it can be observed that
larger values of $r_{min}$ allow a robot to follow the reference path more
closely. This is due to the curvature triggering reactions quickly on approach
to the discontinuity. When $r_{min}$ was increased from $1.0m$ to $1.5m$, the
mean tracking error decreases from $0.8m$ to $0.3m$. This reduction in
tracking error does come at a cost: a reduction in the speed resulting in
longer navigation times. For this experiment, when $r_{min}>1.5m$, the
improvement on tracking error became insignificant, but navigation times begin
to spike as the speed during turns trends towards the minimum allowed speed.

###  5.2 Blind Turning Experiment

Figure 5: A blind corner experiment with an obstacle just out of view. When
the robot cross the green line, the obstacle is set.

This experiment evaluates the three algorithms’ performance when the robot
encounters an unexpected obstacle that blocks its path around a blind corner.
The experiment consists of the robot making a sharp 90-degree turn (Figure 5)
behind which there is an obstacle. The obstacle is such that the robot cannot
observe it before beginning to execute the curve and has minimal time to
react, placed after the robot crosses the green line.

For each algorithm, the experiment is repeated ten times. The purpose of this
experiment is to test the reaction of each algorithm in a common but
particularly dangerous situation. While PP and APP do not themselves contain
collision detection capabilities, for a fair comparison, each are evaluated
using the collision checking method described in Section 3.

Table 1: Result of the Blind Turning Experiment. | PP | APP | RPP
---|---|---|---
Avg Stopped Distance ($m$) | $0.15$ | $0.16$ | $0.24$

Table 1 shows the results of this experiment; in all cases, none of the
methods resulted in a collision. This shows that the collision detection
features of RPP (applied to all methods) are sufficient to prevent blind
collisions at service robot speeds. A similar experiment without RPP’s
collision detection yielded collisions in nearly all instances. The RPP
algorithm displayed a 33% increase in average stopped distance relative to
APP. APP and PP, both which do not change their speeds in turns, had
expectedly similar behavior. This is an improvement in reaction time for safe
navigation in environments with many dynamic agents potentially cornering at
the same time. While the $8cm$ increase does not seem like much, with many
robots navigating independently for their own tasks, this additional
precaution is the difference between robots scraping by and colliding with
permanent damage. This highlights that RPP’s slowing of the robot in blind
turns helps to reduce risk in partially observable settings.

###  5.3 Confined Corridor Experiment

The confined corridor experiment was designed to evaluate how well the
proposed approach improves performance in narrow spaces at high speeds when
making continuous changes in direction. Figure 6 shows the test environment
map, creating a ”slalom” pattern, starting from the left. The total width of
the corridor is 1.5 meters and the obstacles each have dimensions of roughly
0.7 m. Each algorithm was tested five times and the data from this experiment
is shown in Table 2.

Table 2: Result of the confined corridor experiment. | PP | APP | RPP
---|---|---|---
Time ($s$) | $13.0$ | $12.7$ | $13.3$
Distance ($m$) | $9.04$ | $9.54$ | $9.63$
Collisions | $0$ | $0$ | $0$
Average Speed ($m/s$) | $0.679$ | $0.736$ | $0.682$
Average Distance Obstacle ($m$) | $0.662$ | $0.681$ | $0.683$
Average Distance to Path ($m$) | $0.100$ | $0.059$ | $0.052$

Figure 6 displays the robot position data overlaying the trajectory each
algorithm was asked to follow. The most telling region of this experiment to
analyze is the final and sharpest turn. RPP’s proximity and curvature
heuristics contributed in this location and resulted in a reduced speed and
14% less path tracking error than APP. The other algorithms had path
undershoot due to the sharp turn and came into much closer proximity to the
obstacle and wouldn’t be considered functionally safe. This undershoot was
corrected in RPP by reacting to the increased curvature partially due to the
deviation from the path, compounded by its approaching proximity to the
obstacle, slowing the robot down. This allowed RPP to better recover from a
minor deviation off the path and navigated safer distance away from the
obstacle. The impact of the regulation heuristics can be seen in blue as the
robot weaves around the final obstacle after quickly changing direction.

Figure 6: Calculated path in gray and the robot’s traveled path, colored by
speed, during the corridor experiment.

The distance traveled by PP is lower than both variants due to the short
cutting it displayed throughout the trajectory in Figure 6. This habitual
undershoot also led to an decreased average distance to an obstacle - passing
much closer to potential collisions.

###  5.4 Full-system experiment

Figure 7: Itinerary for full-system experiment. The total length is 70 meters.

In the final experiment, the robot will follow a route through the three
control points shown in Figure 7 in a campus building. The route contains
collision-free open space, confined halls, and blind sharp turn of
approximately 70 m. The robot repeats the path three times with each of the
algorithms, the data is provided in Table 3. The purpose of this experiment is
to compare the RPP algorithm’s general behaviors at a system-level to the
existing variants.

The data indicates little difference in overall system-level behavior - the
average time to navigate was between 85-95s. Consistent with the previous
experiment, PP does display slightly shorter distance traveled and an
increased tracking error - corresponding to path short-cutting.

Table 3: Result of the full-system experiment. | PP | APP | RPP
---|---|---|---
Time ($s$) | $85.6$ | $94.3$ | $88.6$
Distance ($m$) | $58.24$ | $59.26$ | $58.64$
Collisions | $0$ | $0$ | $0$
Average Speed ($m/s$) | $0.661$ | $0.675$ | $0.646$
Min Distance to Obstacle ($m$) | $1.139$ | $1.133$ | $1.135$
Average Distance to Path ($m$) | $0.062$ | $0.049$ | $0.043$

The most compelling attribute of this experiment is the lack of particularly
unique outcomes between the algorithms. Although RPP slows the robot in narrow
corridors and while completing sharp turns, surprisingly, these maneuvers did
not meaningfully impact the high-level navigation metrics. Time to task
completion and average speeds were consistent across all three algorithms when
considering the standard deviations between the trials. The system-level
performance of RPP is so similar to that of APP, we conclude that the
additional benefits of RPP come at little disadvantage to a system designer.
In fact, it is possible for a solution architect to increase the maximum speed
of the robot modestly when using RPP due to its slowing in turns and in
proximity to obstacles (the common limiting factors of speed in robotics
applications). This meaningfully improves the overall efficiency of a robot
system while securing higher-quality safety features. 222A video with the
experiments with the real robot can be found at https://youtu.be/LQAzzJ8GmS0.

##  6 Limitations

Regulated Pure Pursuit, and other Pure Pursuit variants, suffer from a lack of
modeling of the vehicle. As this class of technique is purely geometric based
on the path, it will not consider dynamic constraints while changing
velocities to track the path. Further, the global path must be feasibly
drivable for a given robot platform since these methods compute velocities in
the absence of kinematic limitations. For differential-drive robots, this may
be any path. However for Ackermann steering robots, this path must already be
drivable considering the kinematic limitations on the minimum possible turning
radius.

Regulated Pure Pursuit also continues to short-cut sections of paths with high
curvature turns, though to a much reduced degree than PP or APP. The work
introduced in new_pp provides an alternative method for selecting the
lookahead point with the goal of more accurately tracking the reference path.
While this work also introduces a velocity heuristic that is not suitable for
our class of applications, a variation on the method for selecting lookahead
points has merit to be applied in mobile robotics.

##  7 Conclusion

Regulated Pure Pursuit builds incrementally on Adaptive Pure Pursuit with a
focus on service robots. This method contains a schema for selecting
velocities that improves functional safety for real-world deployed robot
applications.

Our demonstrations on an industrial robot exhibited improved safety around
blind turns and in confined settings without significant system-level changes.
A high-quality implementation of Regulated Pure Pursuit is freely available at
https://github.com/ros-planning/navigation2 and is in use on robots today.

## Conflict of Interest and Acknowledge

The authors have no conflicts of interest to declare that are relevant to the
content of this article.

This work has been partially funded by Ministerio de Economía and
Competitividad of the Kingdom of Spain under project PID2021-126592OB-C22 and
by the European Commission under grant CoreSense (N1.101070254).

## References

  *   * (1) Fox D, Burgard W, Thrun S.  The dynamic window approach to collision avoidance.  IEEE Robotics Automation Magazine. 1997;4(1):23–33.  10.1109/100.580977.
  * (2) Rösmann C, Hoffmann F, Bertram T.  Timed-Elastic-Bands for time-optimal point-to-point nonlinear model predictive control.  In: 2015 European Control Conference (ECC); 2015. p. 3352–3357.
  * (3) Huskić G, Buck S, Zell A.  GeRoNa: Generic Robot Navigation.  Journal of Intelligent & Robotic Systems. 2019 Aug;95(2):419–442.  10.1007/s10846-018-0951-0.
  * (4) Jelavic E.: Planning and control for car-like vehicles.  [Online; accessed 3-March-2022].  https://github.com/leggedrobotics/se2_navigation.
  * (5) Macenski S, Martin F, White R, Ginés Clavero J.  The Marathon 2: A Navigation System.  In: 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS); 2020. .
  * (6) Coulter RC.  Implementation of the Pure Pursuit Path Tracking Algorithm; 1992. .
  * (7) Campbell SF.  Steering control of an autonomous ground vehicle with application to the DARPA Urban Challenge.  Massachusetts Institute of Technology, Department of Mechanical Engineering.. Massachusetts Ave, Cambridge, MA 02139, USA; 2007.
  * (8) Ohta H, Akai N, Takeuchi E, Kato S, Edahiro M.  Pure Pursuit Revisited: Field Testing of Autonomous Vehicles in Urban Areas.  In: 2016 IEEE 4th International Conference on Cyber-Physical Systems, Networks, and Applications (CPSNA); 2016. p. 7–12.
  * (9) Samuel M, Hussein M, Mohamad M.  A Review of some Pure-Pursuit based Path Tracking Techniques for Control of Autonomous Vehicle.  International Journal of Computer Applications. 2016;135:35–38.
  * (10) Ohta H, Akai N, Takeuchi E, Kato S, Edahiro M.  Adaptive Pure Pursuit Model for Autonomous Vehicle Path Tracking.  International Journal of Science. 2017;4(3):70–73.
  * (11) Ollero A, Heredia G.  Stability analysis of mobile robot path tracking.  In: Proceedings 1995 IEEE/RSJ International Conference on Intelligent Robots and Systems. Human Robot Interaction and Cooperative Robots. vol. 3; 1995\. p. 461–466 vol.3.
  * (12) Robotics P.: Tiago.  [Online; accessed 21-Feb-2021].  http://pal-robotics.com/robots/tiago/.
  * (13) Paden B, Čáp M, Yong SZ, Yershov D, Frazzoli E.  A Survey of Motion Planning and Control Techniques for Self-Driving Urban Vehicles.  IEEE Transactions on Intelligent Vehicles. 2016;1(1):33–55.  10.1109/TIV.2016.2578706.
  * (14) Macenski S, Jambrecic I.  SLAM Toolbox: SLAM for the dynamic world.  Journal of Open Source Software. 2021;6(61):2783.  10.21105/joss.02783.