ADMM

    Alternating Direction Method of Multipliers
AR

    AutoRally
CEM

    Cross-Entropy Method
CNN

    Convolutional Neural Network
ConvLSTM

    Convolutional Long Short Term Memory
CVaR

    Conditional Value-at-Risk
DARPA

    Defense Advanced Research Projects Agency
DDP

    Differential Dynamic Programming
EKF

    Extended Kalman Filter
EM

    Expectation Maximization
ES

    Evolutionary Strategies
GMM

    Gaussian Mixture Model
GP

    Gaussian Process
GRU

    Gated Recurrent Unit
FT

    Fourier Transform
iCEM

    improved Cross-Entropy Method
iFFT

    inverse Fast Fourier Transform
IS

    Importance Sampler
KL

    Kullback Leibler
MOO

    Multi-Objective Optimization
MPC

    Model Predictive Control
MPPI

    Model Predictive Path Integral
NLN

    Normal Log-Normal
NN

    Neural Network
PCA

    Principle Component Analysis
PDF

    Probability Density Function
PSD

    Power Spectral Density
RCNN

    Region-based Convolutional Neural Network
RL

    Reinforcement Learning
RPN

    Region Proposal Network
SQP

    Sequential Quadratic Programming
VaR

    Value-at-Risk
YOLO

    You Only Look Once

# Low Frequency Sampling in Model Predictive Path Integral Control

Bogdan Vlahov1, Jason Gibson1, David D. Fan2, Patrick Spieler2,
Ali-akbar Agha-mohammadi2, Evangelos A. Theodorou1 Manuscript received:
October, 12, 2023; Accepted March, 11, 2024.This work was supported in part by
the Jet Propulsion Laboratory, California Institute of Technology, under
contract with the National Aeronautics and Space Administration
(80NM0018D0004) and the Defense Advanced Research Projects Agency, and in part
by the Office of Naval Research (N00014-21-1-2074). (Corresponding author:
Bogdan Vlahov)1 Bogdan Vlahov, Jason Gibson, and Evangelos Theodorou are with
the Autonomous Control and Decision Systems Lab, Georgia Institute of
Technology, Atlanta GA 30313 USA (e-mail<EMAIL_ADDRESS><EMAIL_ADDRESS>evangelos.theodorou@gatech.edu)2 David D. Fan, Patrick
Spieler, and Ali-akbar Agha-mohammadi are with NASA Jet Propulsion Laboratory,
California Institute of Technology, Pasadena, CA, USA (e-mail:
<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>
###### Abstract

Sampling-based model-predictive controllers have become a powerful
optimization tool for planning and control problems in various challenging
environments. In this paper, we show how the default choice of uncorrelated
Gaussian distributions can be improved upon with the use of a colored noise
distribution. Our choice of distribution allows for the emphasis on low
frequency control signals, which can result in smoother and more exploratory
samples. We use this frequency-based sampling distribution with Model
Predictive Path Integral (MPPI) in both hardware and simulation experiments to
show better or equal performance on systems with various speeds of input
response.

###### Index Terms:

Optimization and Optimal Control; Motion and Path Planning; Integrated
Planning and Control

##  I Introduction

As autonomous systems grow in interest, the choice of methods and algorithms
used to do real-time motion planning and control becomes critical to achieve
complex tasks. In general, there are two approaches to this sort of problem:
gradient-based and sampling-based. Gradient-based approaches, such as
Differential Dynamic Programming (DDP) [1] or Sequential Quadratic Programming
(SQP) [2], generally have requirements on the dynamics or cost functions used
such as they need to be continuously differentiable. In exchange, they provide
controls that converge on the true optimal sequence as the number of
iterations increase. Sampling-based methods such as [3, 4] relax the
requirements on the dynamics and cost functions to allow for completely
arbitrary functions but require many samples to get a good estimation of the
true optimal control trajectory.

Model Predictive Path Integral (MPPI) is one such sampling-based algorithm
that has been used to achieve aggressive behavior in a small-vehicle setting.
While ideally, one would sample every possible control trajectory to determine
the true path integral, this suffers from the curse of dimensionality as both
the control dimensions and the time horizon increase. By instead sampling from
a Gaussian distribution, computationally-feasible solutions has been derived
in path-integral [4], information-theoretic [5], and stochastic search [6]
approaches. The Gaussian distribution gives a clean equation for distribution
calculations and is a natural starting point when constructing a sampling-
based algorithm.

However, sampling a Gaussian at every time step leads to control trajectories
samples containing high-frequency noise. Depending on the dynamics model, the
effect of the high frequency controls can be dampened when calculating the
state trajectory, allowing for control trajectories with high-frequency noise
and low-frequency noise to produce similar costs. This in turn allows the
approximate optimal control trajectories computed from finite samples to
chatter significantly. This can cause damage over time when applied to real
systems. When looking at data collected from human experts on these systems,
it can be seen that they generally take smoother actions over a longer time
horizon to achieve their desired behavior.

Figure 1: (Upper) The off-road vehicle in a desert terrain just before an
autonomy test. (Lower) A screenshot of the Flightmare quadrotor simulator

In order to address these issues, we look to sampling distributions that can
adjust the level of high-frequency noise that appears in control trajectories.
In this paper, we explore sampling focused around the low-frequency domain by
using a colored noise distribution. We show how this choice of sampling
distribution can lead to better state exploration when compared to standard
Gaussian sampling in MPPI, like shown in Fig. 2. Furthermore, we will show the
resulting controls are smoother and thus reduce the wear and tear on systems,
such as those shown in Fig. 1, caused by chattering. Previous works have
attempted to sample smoother controls in various ways but our method provides
an additional parameter, $\gamma$, to adjust the smoothness of sampled control
without adjusting the overall range of sampled controls like adjusting
variance would cause. This smoothness adjustment is done by explicitly
lowering the chance of sampling of higher frequency signals, resulting in less
oscillatory behavior and allows its use with multiple systems with different
control bandwidths. The contributions of this paper can be summarized as
follows:

  * •

We show how a frequency-based sampling distribution can be used in MPPI with
minimal adjustments to the update rules and optimal control calculation in
Section III.

  * •

We perform experiments on a real hardware platform that has large control lag
in Section IV-A as well as highly-reactive systems in Sections IV-B and IV-C.
These tests show that our approach can be considered a generalization of
Gaussian sampling applicable in many scenarios.

The paper is organized as follows. In Section II, we go over other approaches
to choosing sampling distributions. In Section III, we discuss the frequency-
based sampling is performed and the modifications required to MPPI. We
showcase experiments in Section IV and conclude in Section V.

Figure 2: 300 samples of state trajectories generated from running frequency-
based (Left) and Gaussian (Right) control samples through the off-road vehicle
dynamics described in Section IV-A. The frequency-based throttle and steering
samples are generated using $\gamma=4$ and both distributions used the same
$\sigma$ set to $0.4$. Descriptions of $\gamma$ and $\sigma$ can be found in
Section III-A. (Lower) Variance in the x direction of the same 300 samples
from frequency-based and Gaussian distributions over time. There is an
additional Gaussian distribution with $\sigma=2.0$ to show that increasing the
standard deviation does not allow Gaussian sampling to explore as far as
colored samples. Colored samples reach further extremes in both the $x$ and
$y$ directions due to the reduction of high frequency signals in the control
trajectories.

##  II Related Work

Before going into frequency-based techniques, a brief overview of relevant
concepts is required. The Power Spectral Density (PSD) provides a measure of
how much power every frequency, $f$, contributes to a time-domain signal.
Samples from a zero-mean, uncorrelated Gaussian distribution have constant
power at every frequency, $PSD(f)\propto 1$. Colored Noise distributions cover
a large space of distributions that have PSD PSD of the form
$PSD(f)\propto\frac{1}{f^{\gamma}}$.

There have been multiple works on adjusting the sampling distributions used in
MPPI to improve sampling exploration. The original Gaussian sampling
distribution used in [4] provides a good starting point, but can have
difficulty to fully explore the possible state space. [7] uses Gaussian
Processes as their sampling distribution and can show that the resulting
optimal controls are smoother than those found using Gaussian noise. However,
it could require much more computation to generate samples depending on the
time horizons chosen as the prior. [8] uses NLN Normal Log-Normal (NLN)
distribution for sampling and shows that the samples generated can explore a
wider space than standard Gaussian sampling. While NLN distributions allow for
more sampling on the tail ends of a distribution compared to a normal
distribution, there is still only a small chance to sample multiple tail-end
values in a row for slow-acting systems unlike colored noise distributions.
[9] showed that sampling in a derivative ”action space” would also further
improve the exploration and smoothness of the control trajectories MPPI would
produce. Specifically, the authors end up not integrating the Gaussian samples
over time but instead over iterations of MPPI. While this sampling technique
does improve smoothness, it does so by multiplying the samples by $dt$. This
can cause explorations issues with small $dt$. Our frequency-based sampling
can adjust the time correlation of our trajectories independent of $dt$.

Other works have used multi-hypothesis distributions [10, 11] such as Gaussian
Mixture Models and Stein Variational distributions to try to increase the
exploration capabilities of the algorithm. While GMMs can maintain multiple
distributions, there is nothing to prevent the several modes from collapsing
which became the motivation for Stein variational policies. Stein variational
policies maintain multiple particles as Gaussian distributions with fixed
variance but ensure that the means of these Gaussians are pushed apart by
inclusion of a kernel function. Recently, there has also been work in learning
sampling distributions with concepts like normalizing flow [12, 13]. These
methods make use of a starting distribution and then perform a transform of
that base distribution into a new one. This transform can then make use of
machine learning to incorporate outside information such as the location of
obstacles in order to bias the resulting distribution into safe regions.

The use of frequency-based sampling has been explored before as well. [14]
used frequency-based sampling as well as other techniques in a Cross-Entropy
Method (CEM) controller to create improved Cross-Entropy Method (iCEM). They
showed that using frequency-based sampling has one of the largest positive
impacts on the overall performance of the CEM controller. We will use the same
frequency-based sampling technique with MPPI which uses a weighted combination
of every sample to compute the optimal control trajectory rather than using a
percentage cutoff. [15] used a power law noise to improve exploration for
their Reinforcement Learning (RL) agent. They specifically generated a
$\frac{1}{f^{2}}$ distribution by filtering white noise and saw large
improvements over Gaussian sampling in their ablation study. We will be using
a different power law noise generation technique so that we can use any
exponent we desire.

Figure 3: A comparison of Gaussian and Colored noise with various exponents.
As $\gamma$ increases, the samples generated become smoother and can reach
tail-end values more consistently.

##  III Mathematical Background

###  III-A Low Frequency Sampling Distribution

For this paper, we will be using a colored noise distribution found by
sampling Gaussians in the frequency domain and performing the inverse Fast
Fourier Transform (iFFT) to go to the time domain, first described in [16].
These samples are control trajectories with a finite time horizon of $T$
steps. We first will show that samples derived from this method are still
Gaussian in the time domain.

In order to limit our time-domain signal to contain only real components, we
sample a particular frequency-domain sequence known as a Hermitian-symmetric
sequence [17, Fig. 2.5]. A Hermitian-symmetric sequence is one where the
second half of the sequence is the complex conjugate of the first half of the
sequence. Let $Z[n]=Z_{real}[n]+iZ_{imag}[n]$ be a point in the frequency
domain with a real and imaginary component. There will be
$N=\lfloor\frac{T}{2}\rfloor+1$ frequency samples. The sampling of
$Z_{real}[n]$ and $Z_{imag}[n]$ is done from Gaussian distributions of the
form
$\mathcal{N}\left(\mu_{n},\left(\max\left\\{\frac{n}{N},f_{min}\right\\}\right)^{-\gamma}\frac{\sigma^{2}}{\zeta}\right)$
where $f_{min}=\frac{1}{N}$ is a cutoff frequency to ensure non-zero variance,
$\gamma$ is a user-chosen colored exponent, $\sigma$ is a user-chosen standard
deviation, and

$\displaystyle\zeta=T^{-2}N^{\gamma}\left(1+4\sum_{n=1}^{N-1}n^{-\gamma}\right)$
(1)

ensures that the time-domain variance equals $\sigma^{2}$. The effects of
different $\gamma$ on sample control trajectories is shown in Fig. 3. The
Hermitian-symmetric sequence can be compactly represented as
$Z_{N}=\left\\{Z[0],Z[1],...,Z[N-1]\right\\}$ in the frequency domain . As
each point $Z[n]$ in the frequency domain has both a real and imaginary part,
there is no reduction of the total number of points sampled compared to
uncorrelated time-domain Gaussian sampling. The full frequency-domain
sequence, $Z^{\prime}_{T}$ can then be constructed as

$\displaystyle Z^{\prime}[t]=\begin{cases}Z[t]&\text{if}\ 0\leq t\leq N-1\\\
\\\ \overline{Z[T-t]}&\text{if}\ N-1<t<T\end{cases},$ (2)

where $\overline{\cdot}$ is the conjugate operator. The time-domain sequence
is found using the iFFT, referred to as $\Psi$ for notional convenience:

$\displaystyle z(t)$
$\displaystyle=\frac{1}{T}\sum_{n=0}^{T-1}Z^{\prime}[n]e^{i2\pi
nt/T}=\Psi\left(Z_{N}\right).$ (3)

The iFFT is thus a matrix transform of our frequency-domain samples and can be
simplified to the following when sampling Hermitian-symmetric sequences:

$\displaystyle\begin{bmatrix}z(0)\\\ \vdots\\\ z(t)\\\ \vdots\\\
\end{bmatrix}=M\begin{bmatrix}Z_{real}[0]\\\ \vdots\\\ Z_{real}[n]\\\
Z_{imag}[n]\\\ \vdots\\\ \end{bmatrix},\ \forall\begin{matrix}&n=0,...,N-1\\\
&t=0,..,T-1\end{matrix}$ (4) $\displaystyle
M=\frac{1}{T}\begin{bmatrix}1&\cdots&2&0&\cdots\\\ &&\vdots&&\\\
1&\cdots&2\cos\left(\frac{2\pi nt}{T}\right)&-2\sin\left(\frac{2\pi
nt}{T}\right)&\cdots\\\ &&\vdots&&\end{bmatrix}.$ (5)

Note that if $T$ is even, the final sample, $Z[N-1]$, must only have a real
component $\left(Z_{imag}[N-1]=0\right)$ to be considered Hermitian. As the
$M$ matrix does not depend on the samples drawn for $Z[n]$, each $z(t)$ is a
linear combination of Gaussian independent random variables, making it
Gaussian as well. However, these samples will now be time-correlated. In a
slight abuse of notation, we will use the term Gaussian to mean uncorrelated
Gaussian distributions and Colored to refer to our frequency-based sampling
technique throughout the rest of this paper unless otherwise noted.

###  III-B MPPI derivation from Frequency Domain Sampling

Consider a general nonlinear system with discrete-time dynamics and cost
function of the following form:

$\displaystyle{\bf x}_{t+1}$ $\displaystyle={\bf x}_{t}+{\bf F}\left({\bf
x}_{t},{\bf u}_{t}\right)\Delta t+{\bf w}_{t},$ (6) $\displaystyle{\bf
J}(X,U)$ $\displaystyle=\phi({\bf x}_{T})+\sum_{t=0}^{T-1}q({\bf x}_{t}),$ (7)

where ${\bf x}\in\mathbb{R}^{n_{x}}$ is our state, ${\bf
u}\in\mathbb{R}^{n_{u}}$ is our control, and ${\bf w}\in\mathbb{R}^{n_{x}}$
are external disturbances that are unknown but assumed to be bounded. ${\bf
J}(\cdot,\cdot)$ is the cost function of a given state trajectory $X$, and
control trajectory $U$, with a terminal cost $\phi(\cdot)$ and nonlinear state
cost $q(\cdot)$. We exclude control costs from this work for the sake of
brevity but they can be added with no change to the following discussion.

MPPI is trying to minimize the cost function of the system, by sampling and
applying a weighted exponential averaging to produce the optimal control. By
following the stochastic search derivation of MPPI, we can make some small
adjustments to accommodate our new sampling distribution. We make use of the
fact that Gaussian distributions belong to the exponential family of
probability distributions $P$, parameterized by $\theta$, with
$\theta=[\mu,\Sigma]^{T}$ specifically for Gaussians. We define $l(\theta)$ in
order to better align with theory presented in [6],

$\displaystyle l(\theta)$ $\displaystyle=\ln{\mathbb{E}_{\epsilon\sim
P(\theta)}\left[\operatorname{\bf S}(-\operatorname{\bf J}(X,U))\right]}.$ (8)

where $\operatorname{\bf S}(\cdot)$ is a generic shaping function. This
function $l(\theta)$ represents a log-transform on the expectation of the
negated cost function over sampled controls. Maximizing $l(\theta)$ is
equivalent to minimizing $\mathbb{E}\left[\operatorname{\bf J}(X,U)\right]$
which is the goal of our original optimal control problem. [6] shows that,
given $l(\theta)$, the update law for $\theta$ at iteration $k$ can be
gradient ascent,

$\displaystyle\theta^{k+1}_{t}$
$\displaystyle=\theta^{k}_{t}+\alpha_{k}\nabla_{\theta_{t}}l(\theta^{k}_{t})$
(9) $\displaystyle\nabla_{\theta_{t}}l(\theta_{t})$
$\displaystyle=\frac{\mathbb{E}_{\epsilon_{t}\sim
P(\theta)}\left[\operatorname{\bf S}(-\operatorname{\bf
J}(X,U))\nabla_{\theta_{t}}\ln{p\left(\epsilon_{t};\theta_{t}\right)}\right]}{\mathbb{E}_{P(\theta)}\left[\operatorname{\bf
S}(-\operatorname{\bf J}(X,U))\right]},$ (10)

where $\alpha_{k}$ satisfies:

$\displaystyle\alpha_{k}>0\quad\forall k,\
\lim_{k\rightarrow\infty}\alpha_{k}=0,\ \sum_{k=0}^{\infty}\alpha_{k}=\infty,$
(11)

in order to achieve asymptotic convergence [18]. If the shaping function is
chosen to be the exponential function,

$\displaystyle\operatorname{\bf
S}(y;\lambda)=\exp\left(\frac{1}{\lambda}y\right),$ (12)

with $\lambda$ being the inverse temperature, Eq. 10 becomes equivalent to the
update law for information-theoretic MPPI [5],

$\displaystyle{\bf u}^{*}_{t}$ $\displaystyle=\mathbb{E}_{V\sim
P(\theta)}\left[w(V)({\bf v}_{t})\right],$ (13) $\displaystyle w(V)$
$\displaystyle=\frac{1}{\eta}\exp\left(-\frac{1}{\lambda}\operatorname{\bf
J}\left(X,V\right)\right),$ (14)

where $\eta$ is a regularization term, $V=\\{{\bf v}_{0},...,{\bf v}_{T-1}\\}$
is a sampled control trajectory from the Gaussian control distribution
$P(\theta)$, and $w$ is the weight associated with $V$. As our distribution in
the frequency domain is independent and Gaussian, we can change the $t$
variable in Eqs. 9 and 10 to $n$ and subsume the iFFT transform inside of
$\operatorname{\bf J}$ with the following equation,
$\operatorname{\bf\check{J}}(X,U)=\operatorname{\bf J}(X,\Psi(U))$. This
allows us to use the same update law to update the parameters of our
frequency-domain sampling distribution,

$\displaystyle\theta^{k+1}_{n}$
$\displaystyle=\theta^{k}_{n}+\alpha_{k}\nabla_{\theta_{n}}l(\theta^{k}_{n}),$
(15) $\displaystyle\nabla_{\theta_{n}}l(\theta_{n})$
$\displaystyle=\frac{\mathbb{E}_{\epsilon_{n}\sim
P(\theta)}\left[\operatorname{\bf
S}(-\operatorname{\bf\check{J}}(X,U))\nabla_{\theta_{n}}\ln{p\left(\epsilon_{n};\theta_{n}\right)}\right]}{\mathbb{E}_{P(\theta)}\left[\operatorname{\bf
S}(-\operatorname{\bf\check{J}}(X,U))\right]}.$ (16)

For this paper, we choose to only update the mean of our control distribution,
$\theta_{n}=\\{\mu_{n}\\}$, and leave the variance constant; variance updates
would be derived the same way if desired. The gradient is estimated using
Monte-Carlo sampling . Once the gradient is estimated, we choose to do the
mean trajectory update in the time domain for computational convenience,

$\displaystyle\mu_{t}^{k+1}=\Psi\left(\mu_{N}^{k+1}\right)=\Psi\left(\mu_{N}^{k}\right)+\alpha_{k}\Psi\left(\nabla_{\mu_{N}}l\left(\mu_{N}^{k}\right)\right),$
(17)

where $\Psi\left(\mu_{N}^{k}\right)$ is equivalent to $\mu_{t}^{k}$ and

$\displaystyle\Psi\left(\nabla_{\mu_{N}}l\left(\mu_{N}^{k}\right)\right)=\Psi\left(\sum_{m=0}^{M}w_{m}\mathcal{E}_{N}^{m}\right)=\sum_{m=0}^{M}w_{m}z^{m}(t),$
(18)

with $M$ being the total number of sample trajectories, $\mu_{N}$ referring to
the collection of $\\{\mu_{n};\ n=0,..,N\\}$, $\mathcal{E}^{m}_{N}$ refers to
the $m$th sample trajectory’s collection of frequency samples
$\mathcal{E}^{m}_{n}=\\{\epsilon^{m}_{0},\epsilon^{m}_{1},...,\epsilon^{m}_{N}\\}$,
and $w_{m}$ is calculated using Eq. 14. The final algorithm for Frequency
Sampling MPPI can be seen in Algorithm 1 with changes from the standard MPPI
algorithm highlighted in blue.

Given : ${\bf F}\left(\cdot,\cdot\right)$, $q\left(\cdot\right)$,
$\phi\left(\cdot\right)$ $M$, $I$, $T$, $\lambda$, $\sigma$,
$\mathbb{\gamma},\zeta,f_{min},\alpha$: System dynamics, running state cost,
terminal cost, num. samples, num. iterations, time horizon, temperature,
standard deviations, sampling exponents, sampling normalization term, minimum
sampling frequency, update step size;

Input : ${\bf x}_{0}$, ${\bf U}$: initial state, mean control sequence;

Output : $\mathcal{U}$: optimal control sequence

1

// Calculate Frequency range

2 $N\leftarrow\lfloor\frac{T}{2}\rfloor+1$;

// Begin Cost sampling

3 for  _$i\leftarrow 1$ to $I$_ do

4 for  _$m\leftarrow 1$ to $M$_ do

5 $J^{m}\leftarrow 0$;

6 ${\bf x}\leftarrow{\bf x}_{0}$;

// Sample Frequency noise

7
$\mathcal{E}^{m}_{N}=\left(\epsilon_{0}^{m}\dots\epsilon_{N}^{m}\right),~{}\epsilon_{n}^{m}\in\mathcal{N}(0,\zeta^{-1}\left(\max\left\\{\frac{n}{N},f_{min}\right\\}\right)^{-\gamma})$;

8 for  _$t\leftarrow 0$ to $T-1$_ do

9 $z^{m}(t)\leftarrow\Psi\left(\mathcal{E}_{N}^{m}\right)$;

10 ${\bf v}_{t}\leftarrow{\bf u_{t}}+\sigma z^{m}(t)$;

11 ${\bf x}\leftarrow{\bf F}\left({\bf x},{\bf v}_{t}\right)$;

12 $J^{m}\mathrel{+}=q\left({\bf x}\right)$;

13 $J^{m}\mathrel{+}=\phi\left({\bf x}\right)$

// Compute trajectory weights

14 $\rho\leftarrow\min\left\\{J^{1},J^{2},...,J^{M}\right\\}$;

15
$\eta\leftarrow\sum_{m=1}^{M}\exp\left(-\frac{1}{\lambda}\left(J^{m}-\rho\right)\right)$;

16 for  _$m\leftarrow 1$ to $M$_ do

17
$w_{m}\leftarrow\frac{1}{\eta}\exp\left(-\frac{1}{\lambda}\left(J^{m}-\rho\right)\right)$;

18

19

// Control update

20 for  _$t\leftarrow 0$ to $T-1$_ do

21 $\mathcal{U}_{t}\leftarrow{\bf u}_{t}+\alpha\sum_{m=1}^{M}w_{m}z^{m}(t)$;

22

23

Algorithm 1 Frequency-based Sampling MPPI

##  IV Experimental Results

###  IV-A Off-road Vehicle Platform

We conducted experiments on a full-scale autonomous off-road modified Polaris
Razer X, shown in Fig. 1. It is equipped with a sensor array mounted on top
and a onboard computer using 4 RTX 3080s, 256 $\mathrm{GB}$ of RAM, and a
Threadripper 3990x. The autonomy controls the vehicle using the power-assisted
steering wheel actuator, a pump for brake pressure, and direct access to the
throttle. The control inputs for the throttle and brake are combined to create
a range of $[-1,1]$ and the steering wheel control input is normalized to
within $[-1,1]$. Each control input has some form of delay from applying a
signal and seeing an effect on the vehicle. For example, turning the steering
wheel fully from right to left takes about two seconds. We learned the
dynamics and modeling delays, ${\bf F}\left(\cdot,\cdot\right)$, using [19].
The slow response to controls is where the limitations of Gaussian sampling
become clear. In order to achieve tight turning maneuvers like human drivers
can, MPPI needs to sample large steering values in the same direction for
multiple time steps in a row. While increasing the variance of the Gaussian
distribution makes sampling larger values easier, it does nothing to address
the fact that the samples need to have the same sign to actually make a sharp
turn. Running several iterations can move the mean to one extreme but this is
computationally expensive and slow-moving.

In our experiments, both the Gaussian and Colored sampling MPPI algorithms are
implemented in CUDA with $I=3$ iterations, $M=6144$ samples, $\lambda=0.1$,
$dt=0.02$\mathrm{s}$$, and $T=250$ step horizon. The cost function for both
algorithms is

$\begin{split}q({\bf x})&=\textit{CostToGo}\left({\bf
x}\right)+\hat{d}\Bigl{(}\textit{WheelRisk}\left({\bf x}\right)\\\
&+\textbf{BodyRisk}\left({\bf x}\right)+\textbf{Lethal}\left({\bf
x}\right)+\textbf{Rollover}\left({\bf x}\right)\\\ &+\textbf{Roll}\left({\bf
x}\right)+\textbf{Pitch}\left({\bf x}\right)+\textit{Speed}\left({\bf
x}\right)\Bigr{)},\end{split}$ (19)

where $\hat{d}={\bf v}dt$ is the estimated distance traveled at a given
timestep.

(a) Gaussian  (b) Colored

Figure 4: Hardware experiments using Gaussian and Colored Sampling on the
vehicle. These graphs show the vehicle trying to maneuver in a zig-zag
corridor shown in red. The colors along the trajectories indicate the amount
of time spent in autonomy before manual override/goal achieved with purple
dots indicating the starting points. The Colored samples can go from one end
of the corridor to the other while the Gaussian sampling struggles to make
more than a single turn even when started in various locations. Figure 5:
Controls from the vehicle hardware attempts. Picture are the controls from the
first eight seconds of the first attempt of each controller starting from the
same location. We can see that the colored sampling technique is achieves
smoother and larger throttle, brake, and steering commands. Throttle and brake
are combined into a single graph with brake values being below the red line.

In Fig. 4, the vehicle is attempting to stay within the bounds of this zig-zag
corridor which requires hard turns for appreciable periods of time to achieve.
There is a human safety operator in the vehicle that would engage the manual
override whenever the vehicle would exit the corridor. It is important to note
that both controllers look to violate the boundaries of the corridor but this
is due to drift in state estimation between the vehicle position and the
corridor location. Gaussian sampling could not complete more than a single
turn even when started in various locations whereas the Colored sampling only
had one human intervention in its six attempts. The times and success rates
are summarized in Table I. The computation times of the Colored MPPI averaged
$26.846$\mathrm{ms}$$ over these experiments while the vanilla MPPI algorithm
averaged $24.839$\mathrm{ms}$$, showing there is minimal overhead to using the
Colored approach on this hardware.

Furthermore, from the controls plotted in Fig. 5, we can further see that the
Gaussian sampler can hit high steering angles of $0.5$ but only slowly as the
Model Predictive Control (MPC) nature of MPPI adjusts the mean of the steering
trajectory in small steps. For this testing, the control standard deviations
were both set to $0.8$, the combined throttle and brake exponent was
$\gamma_{throttle}=10$, and the steering exponent was $\gamma_{steering}=4$.

TABLE I: Lap Experiment Summaries Sampling | Avg Time[$\mathrm{s}$] | Min Time[$\mathrm{s}$] |  $\mathcal{S}_{R}$[%]
---|---|---|---
Off-road HW Gaussian | N/A | N/A | $0$
Off-road HW Colored | $38.333\pm 3.316$ | $35.99$ | $83.333\,33$
Quad. SIM Gaussian | $8.546\,724\,077\,333\,334\pm 0.206\,235\,800\,013\,380\,1$ | $8.360\,057\,27$ | $24.324\,324\,32$
Quad. SIM Colored | $8.690\,610\,210\,777\,777\pm 0.456\,523\,273\,178\,533\,2$ | $7.695\,057\,759$ | $42.857\,142\,85$

(a) Gaussian  (b) Gaussian  (c) Colored  (d) Colored

Figure 6: Quadrotor flights using both types of sampling. Each controller is
showing the first 9 successful attempts flying through this course. The red
bars indicate gate positions that the quadrotor was expected to fly through.
The left side graphs show the trajectories taken by the two controllers,
colored according to the ground velocity of the quadrotor. On the right side,
the controls that achieved the fastest lap for each controller are shown. Note
that the throttle signal is much smoother for the Colored sampling when
compared to Gaussian.

###  IV-B Simulated Quadrotor Results

We conducted additional experiments on a quadrotor platform in simulation. The
simulation environment used is the Flightmare simulator [20], shown in Fig. 1.
The dynamics function ${\bf F}\left({\bf x}_{t},{\bf u}_{t}\right)$ used
within MPPI had a 13-dimensional state of
$\\{p_{x},p_{y},p_{z},v_{x},v_{y},v_{z},q_{w},q_{x},q_{y},q_{z},\omega_{x},\omega_{y},\omega_{z}\\}$,
where $p$ is position [$\mathrm{m}$], $v$ is linear velocity
[$\mathrm{m}\text{\,}{\mathrm{s}}^{-1}$], $q$ is a quaternion representation
of orientation, and $\omega$ are angular rates
[$\mathrm{rad}\text{\,}{\mathrm{s}}^{-1}$]. It was assumed a low-level
controller existed on the angular rates that could achieve a first order
response of $\tau=0.25$, and the control space was angular rates
[$\mathrm{rad}\text{\,}{\mathrm{s}}^{-1}$] and thrust
[$\mathrm{m}\mathrm{/}\mathrm{s}^{2}$]. Both MPPI algorithms were run with
$I=2$ iterations, $M=4096$ samples, $\lambda=0.3$, $dt=0.01$\mathrm{s}$$, and
$T=150$. The quadrotor was to fly through gates known a-priori. The cost
function was a combination of factors such as distance to next gate,
maintaining height of $2$\mathrm{m}$$ above the ground, tracking a desired
velocity of $9$\mathrm{m}\text{\,}{\mathrm{s}}^{-1}$$, minimizing the
deviation of the thrust axis from vertical to maintain stable flight, and
crash costs for hitting the gate,

$\begin{split}q({\bf x})&=a_{1}\text{Heading}({\bf x})+a_{2}\text{Height}({\bf
x})+a_{3}\text{Speed}({\bf x})\\\ &+a_{4}\text{Stabilize}({\bf
x})+a_{5}\text{GateCrash}({\bf x})\\\ &+a_{6}\text{Waypoint}({\bf
x})+a_{7}\text{Path}({\bf x}).\end{split}$ (20)

When running MPPI with Gaussian sampling, we used standard deviations of
$[0.3,0.3,0.3,3.5]$ for the angular rates and thrust respectively. Note that
the high thrust standard deviation is required for the quadrotor to adjust
quickly when performing turns and reach quick lap times. As seen in Fig. 6a,
the quadrotor successfully flies through the gates. Looking at the controls in
Fig. 6b, we see that the controls end up being quite chattery, especially in
thrust. This sort of chatter may lie outside of the control bandwidth of the
quadrotor (i.e. going from $5$\mathrm{m}\mathrm{/}\mathrm{s}^{2}$$ to
$11$\mathrm{m}\mathrm{/}\mathrm{s}^{2}$$ in $0.02$\mathrm{s}$$ is not possible
given the simulated motors) and can cause wear to physical motors over time.

Figure 7: State trajectories (Left), control trajectories (Middle), and PSD
(Right) from the double integrator system with standard deviation set to
$1.5$. The PSD is shown for the first 2 seconds of the control trajectories to
highlight the frequencies used to reach steady-state. Log-MPPI reaches the
goal fastest but does so with lots of high frequency controls. The Colored
methods using $\gamma=1.0$ and $2.0$ is able to react much quicker compared to
Gaussian sampling by producing a much larger initial control spike and
subsequent control dip to slow down to reach the goal without overshooting.
Smooth-MPPI* does reach the goal sooner than Gaussian sampling but has larger
control deviations around steady state. Finally, the Smooth-MPPI controller
does not perform well with $\sigma=1.5$, keeping the control pretty close to
$0$. The PSD shows that Colored control trajectories have lower contributions
from high frequency components compared to Gaussian, Log-MPPI, and Smooth-
MPPI*.

When running MPPI with Colored sampling, we used standard deviations of
$[0.2,0.2,0.2,0.5]$ and exponents $\gamma$ of $[0.01,0.01,0.01,0.5]$ for the
angular rates and thrust respectively based on best observed performance.
Since the quadrotor is more reactive to control inputs than the hardware
ground vehicle, lower exponents allow us to make sharp maneuvers while still
reducing control chatter. While the lap times do not show much difference, we
can see a difference in Fig. 6d of the controls of the fastest lap. Unlike the
throttle of the Gaussian sampling case, we have lower frequency controls in
throttle even when choosing a relatively small exponent of $0.5$. These are
more likely to be within the control bandwidth of the motors and we only see a
$0.15$\mathrm{s}$$ average time loss from Colored sampling in Table I. The
angular rates of Gaussian and Colored sampling do not see much difference but
this is in line with the angular rates exponent, $\gamma$ being set to $0.01$.
Looking at Fig. 6c, we can also see that the quadrotor achieved speeds of up
to $12$ $\mathrm{m}\text{\,}{\mathrm{s}}^{-1}$ as it went towards the final
gate compared to $10$ $\mathrm{m}\text{\,}{\mathrm{s}}^{-1}$ from the Gaussian
laps.

Looking at Table I, we can see that Colored sampling had the fastest time
through the course but could not reliably achieve that time. The controllers
were both run until they achieved 9 valid attempts. We see that the Gaussian
sampling took many more attempts to get 9 valid laps than the Colored
sampling. Overall, the quadrotor system sees little performance hit for
smoother control inputs.

###  IV-C Double Integrator Results

Finally, we conduct a test on a simple double integrator system. We tested six
different controllers: MPPI with a Gaussian distribution, log-MPPI from [8],
Smooth-MPPI from [9] as well as a modification described below, and MPPI with
Colored sampling and colored exponents $\gamma=1.0$ and $2.0$. The MPPI
controllers had $M=4096$ samples, $I=1$ iteration, $dt=0.015$$\mathrm{s}$,
$T=65$, and $\lambda=1$ for all sampling techniques. The systems started at
$-9$ $\mathrm{m}$ and had a quadratic state cost of

$\displaystyle q({\bf x})$ $\displaystyle=5\left({\bf
x_{0}}+4\right)^{2}+0.5{\bf x_{1}}^{2}.$ (21)

We conducted comparisons of all methods at $\sigma=0.5$, $1.5$, and $3.0$ to
see how adjustments to variance affect the performance of each controller. We
reran all the controllers $1000$ times to also see the variance caused by
Monte-Carlo sampling which is reported in Table II. Note that for Smooth-MPPI,
we saw poor performance using these standard deviations as the control would
barely deviate from $0$. We found that by dividing the provided standard
deviation by $dt$, $\sigma^{*}=\frac{\sigma}{dt}$, performance fell in line
with the other methods; we denote this change to the original method as
Smooth-MPPI* and include both in our comparisons. In Fig. 7, we visualize the
state and control trajectories that achieved the minimum cost for controllers
using $\sigma=1.5$. In addition, we also show the PSD of the first 2 seconds
of each control trajectory to visualize how much each control frequency
contributes to the trajectory as it reaches the goal state. At this
$\sigma=1.5$, log-MPPI reaches the goal first but uses extremely large
controls that would be impossible to achieve on hardware systems. Log-MPPI
also is extremely jerky in attempting to slow down, as evidenced by high power
concentration for large frequencies in the PSD. The Colored sampling
distributions with exponents $\gamma=1$ and $2$ reach the goal next with
smaller peak controls than log-MPPI. We see that the distribution with
$\gamma=2$ reaches the goal slightly faster and has lower power concentraion
at frequencies above $20$\mathrm{Hz}$$. They both have smaller control
deviations from $0$ at steady state compared to Smooth-MPPI* and log-MPPI.
Smooth-MPPI* is the next to reach steady state but has oscillatory control
behavior at steady state that is larger than all the other methods. Gaussian
sampling ends up overshooting the goal and the control trajectory is not as
reactive as log-MPPI or the Colored distributions. Finally, Smooth-MPPI as
described in [9] barely reaches the goal in the time allotted. The control
trajectory does not deviate from $0$ by very much and has low power over all
frequencies, showing that it is unable to respond quickly with this sampling
standard deviation. Overall, Colored sampling allows the controller to be more
reactive compared to Gaussian sampling while also being less oscillatory than
both log-MPPI and Smooth-MPPI* at the same standard deviation.

We show the results across standard deviations of $0.5$, $1.5$, and $3.0$ in
Table II. The metric used in Table II for comparing the algorithms at various
standard deviations is the accumulated cost of the states reached and we
report the standard deviation of these costs over 1000 attempts. Depending on
the standard deviation, log-MPPI, smooth-MPPI, and smooth-MPPI* can have
orders of magnitude more variance in cost when tested multiple times compared
to both Gaussian and Colored sampling. Increasing the control standard
deviation improves all methods in general but the best controller changes as
well. Our Colored sampling distributions maintain good relative performance,
only losing to log-MPPI at higher standard deviations. In addition, we see in
Fig. 7 that our method does not rely on control impulses or exhibit
oscillatory behavior like controllers that do minimize the accumulated cost
better. Interestingly, Smooth-MPPI becomes less consistent as the standard
deviation increases but seems to have the best performance on average around
$\sigma=1.5$. Looking at the Smooth-MPPI* results and Fig. 7, this is most
likely due to the standard deviation choices being too small to properly
excite the double integrator system. Finally, there is some computational
overhead to running Colored sampling compared to other methods, but the
increase in computation time at around $0.12$\mathrm{ms}$$ seems to fall
within the RTX 3080’s performance variance and is only noticeable when the
dynamics and cost function are so simple. While further tuning for each
controller is possible to address any specific issue, we see that our proposed
method performs well across a variety of standard deviations, both in terms of
cost and control trajectory shape.

TABLE II: Double Integrator Results Std. Dev. | Sampling | Accumulated Cost | Calc. Time (ms)
---|---|---|---
0.5 | Gaussian | $27\,919\pm 33.1048$ | $0.259\,161\pm 0.225$
Smooth-MPPI |  $61\,328.7\pm 1766.76$0 | $0.295\,898\pm 0.165\,282$
Smooth-MPPI* |  $21\,853.6\pm 19\,189.7$00 | $0.294\,591\pm 0.231\,628$
Log-MPPI | $24\,545.3\pm 40.2946$ | $0.297\,113\pm 0.124\,21$
Colored 1.0 | $13\,569.7\pm 65.3729$ | $0.378\,722\pm 0.156\,425$
Colored 2.0 | $14\,201.7\pm 61.2216$ | $0.378\,722\pm 0.156\,425$
1.5 | Gaussian | $13\,815.5\pm 64.594$ | $0.258\,975\pm 0.189\,43$
Smooth-MPPI |  $44\,401\pm 3232.29$0 | $0.292\,831\pm 0.154\,399$
Smooth-MPPI* | $11\,446.3\pm 71.1$ | $0.293\,511\pm 0.175\,97$
Log-MPPI | $7316.06\pm 324.471$ | $0.298\,557\pm 0.192\,854$
Colored 1.0 | $10\,636.9\pm 74.4388$ | $0.379\,149\pm 0.177\,628$
Colored 2.0 | $11\,081.2\pm 75.5951$ | $0.3791\pm 0.220\,055$
3.0 | Gaussian | $10\,815.1\pm 75.70$ | $0.258\,632\pm 0.21$
Smooth-MPPI |  $68\,088.2\pm 7521.73$0 | $0.293\,31\pm 0.132\,446$
Smooth-MPPI* | $10\,505.5\pm 75.5529$ | $0.293\,515\pm 0.212\,292$
Log-MPPI | $6963.46\pm 380.457$ | $0.306\,633\pm 0.155\,597$
Colored 1.0 | $9191.69\pm 69.6006$ | $0.380\,484\pm 0.331\,644$
Colored 2.0 | $9699.99\pm 70.19$ | $0.379\,612\pm 0.249\,113$

##  V Conclusions and Future Work

In this work, we proposed use of a new sampling distribution with MPPI to
facilitate smoother controls and a larger coverage of the state space. This
Colored sampling distribution was shown to have nearly identical update laws
when used in MPPI as the Gaussian distribution. This distribution has a
tunable amount of higher frequency components which provides it the ability to
be used across various dynamical systems. In our hardware results, our vehicle
is unable to effectively maneuver when provided high-frequency samples from
Gaussian distributions, reducing the reachable state space significantly.
Meanwhile, our Colored sampling distribution demonstrated tight turns only
achievable by sampling near the limits of the steering wheel for multiple
timesteps in a row. On the simulated quadrotor, we see that by lowering our
sampling exponents, we can achieve nearly similar performance to Gaussian-
based MPPI while maintaining smoother throttle control. Finally, in the double
integrator experiments, Colored sampling is able to achieve more extreme
control trajectories than the Gaussian-based controller and produce lower
frequency control trajectories than other methods such as log-MPPI and Smooth-
MPPI. In the future, we plan to apply this sampling distribution on more
advanced forms of MPPI such as Tube-MPPI and RMPPI as well as combine with
other multi-hypothesis distributions such as GMM and Stein variational
distributions.

##  VI Acknowledgements

This research was developed with funding from the Jet Propulsion Laboratory,
Defense Advanced Research Projects Agency, and the Office of Naval Research.
The views, opinions and/or findings expressed are those of the author and
should not be interpreted as representing the official views or policies of
the Department of Defense or the U.S. Government. Approved for public release
and unlimited distribution.

## References

  * [1] D. H. Jacobson and D. Q. Mayne, _Differential dynamic programming_ , ser. Modern analytic and computational methods in science and mathematics. American Elsevier Pub. Co., 1970. [Online]. Available: https://cir.nii.ac.jp/crid/1130282270933519744
  * [2] P. T. Boggs and J. W. Tolle, “Sequential quadratic programming,” _Acta numerica_ , vol. 4, pp. 1–51, 1995.
  * [3] R. Rubinstein, “The cross-entropy method for combinatorial and continuous optimization,” _Methodology and computing in applied probability_ , vol. 1, pp. 127–190, 1999. [Online]. Available: https://link.springer.com/article/10.1023/A:1010091220143
  * [4] G. Williams, P. Drews, B. Goldfain, J. M. Rehg, and E. A. Theodorou, “Aggressive Driving with Model Predictive Path Integral Control,” in _2016 IEEE International Conference on Robotics and Automation (ICRA)_. IEEE, 2016, pp. 1433–1440. [Online]. Available: https://ieeexplore.ieee.org/document/7487277/
  * [5] ——, “Information-Theoretic Model Predictive Control: Theory and Applications to Autonomous Driving,” _IEEE Transactions on Robotics_ , vol. 34, no. 6, pp. 1603–1622, 2018. [Online]. Available: https://ieeexplore.ieee.org/document/8558663/
  * [6] Z. Wang, O. So, K. Lee, and E. A. Theodorou, “Adaptive Risk Sensitive Model Predictive Control with Stochastic Search,” in _Proceedings of the 3rd Conference on Learning for Dynamics and Control_. PMLR, 2021, pp. 510–522. [Online]. Available: https://proceedings.mlr.press/v144/wang21b.html
  * [7] J. Watson and J. Peters, “Inferring smooth control: Monte carlo posterior policy iteration with gaussian processes,” in _Conference on Robot Learning_. PMLR, 2023, pp. 67–79. [Online]. Available: https://proceedings.mlr.press/v205/watson23a/watson23a.pdf
  * [8] I. S. Mohamed, K. Yin, and L. Liu, “Autonomous Navigation of AGVs in Unknown Cluttered Environments: Log-MPPI Control Strategy,” _IEEE Robotics and Automation Letters_ , vol. 7, no. 4, pp. 10 240–10 247, 2022. [Online]. Available: https://ieeexplore.ieee.org/abstract/document/9834098
  * [9] T. Kim, G. Park, K. Kwak, J. Bae, and W. Lee, “Smooth Model Predictive Path Integral Control Without Smoothing,” _IEEE Robotics and Automation Letters_ , vol. 7, no. 4, pp. 10 406–10 413, 2021. [Online]. Available: https://ieeexplore.ieee.org/document/9835021
  * [10] Z. Wang, O. So, J. Gibson, B. Vlahov, M. S. Gandhi, G.-H. Liu, and E. A. Theodorou, “Variational Inference MPC using Tsallis Divergence,” in _Robotics: Science and Systems XVII_. Robotics: Science and Systems Foundation, Apr. 2021. [Online]. Available: https://www.roboticsproceedings.org/rss17/p073.html
  * [11] A. Lambert, F. Ramos, B. Boots, D. Fox, and A. Fishman, “Stein Variational Model Predictive Control,” in _Proceedings of the 2020 Conference on Robot Learning_. PMLR, 2021, pp. 1278–1297. [Online]. Available: https://proceedings.mlr.press/v155/lambert21a.html
  * [12] J. Sacks and B. Boots, “Learning Sampling Distributions for Model Predictive Control,” in _Proceedings of The 6th Conference on Robot Learning_. PMLR, Mar. 2023, pp. 1733–1742. [Online]. Available: https://proceedings.mlr.press/v205/sacks23a.html
  * [13] T. Power and D. Berenson, “Variational Inference MPC using Normalizing Flows and Out-of-Distribution Projection,” in _Robotics: Science and Systems XVIII_. Robotics: Science and Systems Foundation, Jun. 2022. [Online]. Available: http://www.roboticsproceedings.org/rss18/p027.pdf
  * [14] C. Pinneri, J. Achterhold, M. Rolínek, and G. Martius, “Sample-efficient Cross-Entropy Method for Real-time Planning,” in _Proceedings of the 2020 Conference on Robot Learning_. PMLR, 2021, pp. 1049–1065. [Online]. Available: https://proceedings.mlr.press/v155/pinneri21a.html
  * [15] Z. Zhang, J. Chen, Z. Chen, and W. Li, “Asynchronous Episodic Deep Deterministic Policy Gradient: Toward Continuous Control in Computationally Complex Environments,” _IEEE Transactions on Cybernetics_ , vol. 51, no. 2, pp. 604–613, 2021. [Online]. Available: https://ieeexplore.ieee.org/abstract/document/8946888
  * [16] J. Timmer and M. König, “On generating power law noise,” _Astronomy & Astrophysics_, vol. 300, pp. 707–710, 1995. [Online]. Available: http://jeti.uni-freiburg.de/papers/timmer95.pdf
  * [17] R. Bracewell, “The Fourier Transform and its Applications.” McGraw-Hill, 2000, ch. 2.
  * [18] E. Zhou and J. Hu, “Gradient-Based Adaptive Stochastic Search for Non-Differentiable Optimization,” _IEEE Transactions on Automatic Control_ , vol. 59, no. 7, pp. 1818–1832, 2014. [Online]. Available: https://ieeexplore.ieee.org/document/6756948
  * [19] J. Gibson, B. Vlahov, D. Fan, P. Spieler, D. Pastor, A.-a. Agha-mohammadi, and E. A. Theodorou, “A Multi-step Dynamics Modeling Framework For Autonomous Driving In Multiple Environments,” in _2023 IEEE International Conference on Robotics and Automation (ICRA)_. IEEE, May 2023, pp. 7959–7965. [Online]. Available: https://ieeexplore.ieee.org/abstract/document/10161330
  * [20] Y. Song, S. Naji, E. Kaufmann, A. Loquercio, and D. Scaramuzza, “Flightmare: A Flexible Quadrotor Simulator,” in _Proceedings of the 2020 Conference on Robot Learning_. PMLR, 2021, pp. 1147–1157. [Online]. Available: https://proceedings.mlr.press/v155/song21a.html

  *[DDP]: Differential Dynamic Programming
  *[SQP]: Sequential Quadratic Programming
  *[MPPI]: Model Predictive Path Integral
  *[PSD]: Power Spectral Density
  *[NLN]: Normal Log-Normal
  *[GMMs]: Gaussian Mixture Model
  *[CEM]: Cross-Entropy Method
  *[iCEM]: improved Cross-Entropy Method
  *[RL]: Reinforcement Learning
  *[iFFT]: inverse Fast Fourier Transform
  *[MPC]: Model Predictive Control
  *[GMM]: Gaussian Mixture Model