# DiffPLF: A Conditional Diffusion Model for Probabilistic Forecasting of EV
Charging Load

Siyang Li, Hui Xiong, and Yize Chen  Hong Kong University of Science and
Technology (Guangzhou), Guangzhou
<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>
###### Abstract

Due to the vast electric vehicle (EV) penetration to distribution grid,
charging load forecasting is essential to promote charging station operation
and demand-side management. However, the stochastic charging behaviors and
associated exogenous factors render future charging load patterns quite
volatile and hard to predict. Accordingly, we devise a novel Diffusion model
termed DiffPLF for Probabilistic Load Forecasting of EV charging, which can
explicitly approximate the predictive load distribution conditioned on
historical data and related covariates. Specifically, we leverage a denoising
diffusion model, which can progressively convert the Gaussian prior to real
time-series data by learning a reversal of the diffusion process. Besides, we
couple such diffusion model with a cross-attention-based conditioning
mechanism to execute conditional generation for possible charging demand
profiles. We also propose a task-informed fine-tuning technique to better
adapt DiffPLF to the probabilistic time-series forecasting task and acquire
more accurate and reliable predicted intervals. Finally, we conduct multiple
experiments to validate the superiority of DiffPLF to predict complex temporal
patterns of erratic charging load and carry out controllable generation based
on certain covariate. Results demonstrate that we can attain a notable rise of
39.58% and 49.87% on MAE and CRPS respectively compared to the conventional
method.

###### Index Terms:

EV charging, probabilistic forecasting, diffusion models, deep learning.

##  I Introduction

Rapid advancements of energy storage, fast-charging infrastructure, and
carbon-reduction blueprints [1] facilitate recent proliferation of electric
vehicles (EVs). For instance, there will be 30-42 million EVs supported by
26-35 million charging piles in United States by 2030 [2]. Such substantial EV
penetration exerts additional large-scale, highly stochastic load to power
networks, imposing unprecedented challenges on grid operation [3]. EV charging
load forecasting is crucial to host such vast charging demand, which can
benefit operators of both distribution network and charging stations. For
instance, grid operators are able to design the optimal coordinated dispatch
of EVs and renewables in light of predicted charging power outcomes [4].
Station managers can reduce the electricity procurement cost and optimize the
real-time charging scheduling aided by the demand forecast information [5].

Point forecast is a classical way to procure future charging demand. In [6], a
stacked recurrent neural network is forged by reinforcement learning to
execute demand prediction for different charging scenarios. In [7], charging
demand is estimated by the joint statistical analysis for battery behaviors,
traffic flow and weather data. However, since the real-world charging demand
is extremely stochastic and volatile due to a set of uncertain factors (e.g.
battery dynamics, traveller charging patterns and weather conditions), single
deterministic charging load forecasts can not give station and power grid
operators the most effective tools for load management and operations.
Inaccurate prediction can raise operational costs and jeopardize power quality
[5]. Besides, it is more crucial for stakeholders to gain a group of reliable
forecasts which are beneficial for stochastic optimization and robust decision
making [8]. To this end, probabilistic forecasting is a promising approach to
model the forecast uncertainties, which can be achieved by generating a host
of plausible charging load profiles. Operators can exploit such probabilistic
predictions to lessen charging energy deviation costs [5]. For instance, [9]
proposes to forecast the uncertainty of EV parking duration.

In this work, we are interested in probing an accurate and reliable
probabilistic forecasting model to tackle unknown charging load uncertainties.
Forecasting both the values and uncertainties of EV charging scenarios is an
emergent topic, while several works focus on predicting probable charging
demand in urban areas. Quantile regression like [10] and [11] is a typical way
to explicitly construct the prediction interval (PI) by learning a group of
quantiles of different degrees. They utilize disparate neural networks to
learn the relationships between historical charging data and these quantiles
supervised by certain PI evaluation metrics. Nevertheless, such approach falls
short in modeling complex temporal EV charging processes which are further
complicated by conditional information such as weather and user behaviors.
Another feasible approach is to directly quantify the point forecast
uncertainties, which stem from both predictive model misspecifications and
charging mode variability. For example, hidden state variations in the LSTM-
based point predictor are captured by proximal policy optimization in [12]. A
novel queuing model is proposed in [13] to link mobile EV charging load to
time-varying traffic flow, while a meta-learning method is introduced in [14]
to address the data scarcity issue. However, these methods depend on both
well-trained deterministic prediction model and assumption of the forecast
error distribution.

Essentially, the principal objective of probabilistic charging load
forecasting is to obtain the predictive distribution of future charging demand
profiles based on observed information. The currently heated generative models
[15] are able to approximate the complex distribution of high-dimensional data
and generate various samples of high quality, which are promising to
explicitly model the predictive distribution of charging time-series and yield
a host of plausible future trajectories. Similar work has been done on the
generic time-series analysis, like multivariate time-series forecasting [16],
[17] and imputation [18], all of which leverage generative models to estimate
the target conditional distribution. However, these work do not explicitly
devise a conditioning scheme to entangle the predicted charging load with
historical data and relevant covariates.

In this paper, we aim to develop a data-driven generative model to directly
learn the joint distribution of future charging load, which can be also
conditioned on historical load data and associated covariates including
weather forecasts, calendar variables and EV number. These informative
covariates are non-negligible to achieve high-quality PIs. More importantly,
EV charging station operators and grid operators are also interested in
analyzing the impacts of certain variables on EV charging sessions [19]. We
adopt the denoising diffusion model proposed in [20], which has demonstrated
expressive capacity to produce diverse high-fidelity images and perform
controllable generation guided by text prompts [21]. Diffusion models can
transform the prior Gaussian noise to target charging load time-series by
learning a parameterized reversal of diffusion process. Besides, diffusion
models are quite efficient in training and inference, which also evade the
mode collapse and training instability issues in other generative model such
as generative adversarial networks (GANs) [22].

For the probabilistic forecasting of EV charging load task, we design a
specific conditional denoising diffusion model entitled DiffPLF, to generate
an array of plausible charging load profiles given historical charging demand
and a group of informative covariates. We also incorporate the cross-attention
mechanism [21] into the denoising network, which can condition each perturbed
load time-series in the diffusion process on input conditional terms. In order
to further gain more accurate and reliable probabilistic forecasts, our
training objective is to make the 50%-quantile (i.e. median) of produced
outcomes be close to ground-truth charging load at each prediction step. In
light of it, we propose a fine-tuning block over the pre-trained diffusion
model via a 50%-quantile deviation minimization (QDM) loss. Such QDM loss
imposes a task-informed inductive bias on the diffusion model, and improves
forecasting performance by around $40\%$ in contrast to standard quantile
regression. We verify that DiffPLF can output more accurate predictive
distribution and point forecast. Besides, it can be adapted to various
prediction horizons and execute controllable generation conditioned on
different EV numbers. Our code is publicly available at
https://github.com/LSY-Cython/DiffPLF for better study of EV grid integration.

##  II Problem Formulation

We start from describing the probabilistic charging load prediction task. We
aim at capturing the forecast value along with uncertainties induced by
stochastic and variable charging behaviors. The key is to explicitly model the
conditional distribution of predicted charging load profiles given historical
observations and correlated covariates. Specifically, at time $s$, we collect
a series of past charging demand $p_{i}\in\mathbb{R},i\in$ $[s-\omega,s]$,
where $\omega$ is the length of the context window and $i$ is the time step.
We also prescribe a covariate set $\mathbf{r}$ which will be described
shortly, and utilizing $\mathbf{r}$ is beneficial for accurate forecasts. Note
that $\mathbf{r}$ shall be known in advance for the prediction horizon, which
can be regarded as external information to constrain unwarranted forecasts.
Based on the prior $\mathbf{p}\in\mathbb{R}^{\omega}$ and $\mathbf{r}$, we
want to procure possible outcomes of future charging load
$x^{j}_{0}\in\mathbb{R},\;j\in[s+1,s+\tau]$, where $\tau$ stands for the
length of the prediction horizon and we use subscript $0$ to align with the
original data notation for diffusion models introduced in latter sections.
Thereby, our goal is to approximate the conditional predictive distribution
$q(\mathbf{x}_{0}|\mathbf{p},\mathbf{r})$ of future charging load profiles
$\mathbf{x}_{0}\in\mathbb{R}^{\tau}$. The main challenge of this problem is
how to derive diverse temporal patterns in $\mathbf{x}_{0}$ that are
consistent with conditions $\mathbf{p}$ and $\mathbf{r}$.

Covariates Selection: Choosing informative covariates is critical for
probabilistic time-series forecasting [16]. In this paper, we find three types
of covariates are most helpful: 1) Weather forecasts. Weather conditions are
non-negligible for battery charging dynamics and EV travel behaviors, which
can elicit various temporal modes of charging load [23]. We utilize
temperature forecast $\mathbf{u}\in\mathbb{R}^{\tau}$ and humidity forecast
$\mathbf{v}\in\mathbb{R}^{\tau}$, which have been shown to be the most two
influential weather factors for EV charging load [7]. 2) Calendar variables.
EV mobility can be distinct both temporarily and spatially in terms of the day
type (e.g. weekdays or weekends) [24]. In our setting, we use a one-hot vector
$\mathbf{d}\in\mathbb{R}^{7}$ to signify seven days within a week. 3) EV
number. The total number of charged EV $e$ in the forecast window can affect
both shape and peak value of predicted charging load time-series. We employ EV
number as an unique condition to attest how its variation takes effect on
ultimate forecasted profiles. In summary, the covariate set can be denoted as
$\mathbf{r}=\\{\mathbf{u},\mathbf{v},\mathbf{d},e\\}$.

##  III Denoising Diffusion Models

In this section, we explicate how the denoising diffusion model derive the
complex distribution of charging load time-series, and how to extend it to
perform conditional generation.

Figure 1: The diagram of the conditional diffusion model.

Due to the eminent high-resolution synthesis capacity and efficient training
advantage, denoising diffusion models have been actively applied to various
multi-modal content creation fields [21]. The fundamental principle is
inspired by the non-equilibrium thermodynamics, which implies that it is
feasible to restore the true data distribution by simulating a physically
reversible diffusion process [25]. Firstly, we portray the forward diffusion
process, where real charging load profiles $\mathbf{x}_{0}$ are progressively
transformed into standard Gaussian noise $\mathbf{x}_{T}$ after $T$-step
diffusion procedures in total. This forward process is fixed to a Markov chain
$q(\mathbf{x}_{1:T}|\mathbf{x}_{0})=\prod_{t=1}^{T}q(\mathbf{x}_{t}|\mathbf{x}_{t-1})$,
which suggests step-wise Gaussian noise is gradually imposed on original
$\mathbf{x}_{0}$. The forward transition $q(\mathbf{x}_{t}|\mathbf{x}_{t-1})$
has the form of Gaussian distribution as follows:

$\centering
q(\mathbf{x}_{t}|\mathbf{x}_{t-1})=\mathcal{N}(\mathbf{x}_{t};\sqrt{1-\beta_{t}}\mathbf{x}_{t-1},\beta_{t}\mathbf{I}),\@add@centering$
(1)

where $\beta_{t}$ stands for the magnitude of Gaussian noise added at each
diffusion step $t$, and can be determined by the variance scheduling scheme in
[18]. $\mathbf{x}_{t}$ denotes the perturbed charging load profile by the
Gaussian noise with variance $\beta_{t}$. A critical property [20] of this
forward process is that we can obtain arbitrary $\mathbf{x}_{t}$ based on
$\mathbf{x}_{0}$ in closed form:

$\displaystyle
q(\mathbf{x}_{t}|\mathbf{x}_{0})=\mathcal{N}(\mathbf{x}_{t};\sqrt{\alpha_{t}}\mathbf{x}_{0},(1-\alpha_{t})\mathbf{I});$
(2a)
$\displaystyle\mathbf{x}_{t}=\sqrt{\alpha_{t}}\mathbf{x}_{0}+\sqrt{1-\alpha_{t}}\bm{\epsilon},\bm{\epsilon}\sim\mathcal{N}(\mathbf{0},\mathbf{I}),$
(2b)

where $\alpha_{t}=\prod_{s=1}^{t}(1-\beta_{s})$. This helps significantly
accelerate the forward noising procedure and improve the training efficiency
of diffusion models.

Once the forward diffusion process is fixed, we can restore the genuine
distribution of charging demand time-series $q(\mathbf{x}_{0})$ from the
standard Gaussian $\mathbf{x}_{T}$ by learning a reverse denoising process.
Such reverse process can be defined as a learnable Markov chain
$p_{\theta}(\mathbf{x}_{0:T-1}|\mathbf{x}_{T})=\prod_{t=1}^{T}p_{\theta}(\mathbf{x}_{t-1}|\mathbf{x}_{t})$
parameterized by $\theta$. In light of the physical properties of the
invertible diffusion process [20], if $\beta_{t}$ is small enough, the
diffusion process is continuous and the reverse transition
$p_{\theta}(\mathbf{x}_{t-1}|\mathbf{x}_{t})$ will hold the same function form
as the forward transition $q(\mathbf{x}_{t}|\mathbf{x}_{t-1})$. Hence, we
acquire a Gaussian reverse transition:

$\centering
p_{\theta}(\mathbf{x}_{t-1}|\mathbf{x}_{t})=\mathcal{N}(\mathbf{x}_{t-1};\bm{\mu}_{\theta}(\mathbf{x}_{t},t),\mathbf{\Sigma}_{\theta}(\mathbf{x}_{t},t)).\@add@centering$
(3)

The main target of (3) is to eliminate the noise added at step $t$ from the
perturbed $\mathbf{x}_{t}$. Once the parameters $\theta$ in (3) are
determined, we can transform the prior Gaussian $\mathbf{x}_{T}$ into initial
charging load data $\mathbf{x}_{0}$ through this reverse process.

We utilize the maximum likelihood estimation to learn the parameterized
reverse transition in (3) and approximate the real charging load distribution
$q(\mathbf{x}_{0})$. We opt to minimize the negative log-likelihood of real
demand data $-\log p_{\theta}(\mathbf{x}_{0})$ via its variational upper bound
below:

$\centering-\log
p_{\theta}(\mathbf{x}_{0})\leq\mathbb{E}_{q(\mathbf{x}_{1:T}|\mathbf{x}_{0})}[-\log\frac{p_{\theta}(\mathbf{x}_{0:T})}{q(\mathbf{x}_{1:T}|\mathbf{x}_{0})}].\@add@centering$
(4)

At each diffusion step $t$, we use a denoising network to predict the noise
added on $\mathbf{x}_{0}$. By decomposing (4) into $T+1$ closed-form items, we
get the following training objective for $\bm{\epsilon}_{\theta}$, while we
refer the detailed derivations to [20]:

$\mathcal{L}_{t-1}=\mathbb{E}_{\mathbf{x}_{0},\bm{\epsilon},t}[\left\|\bm{\epsilon}-\bm{\epsilon}_{\theta}(\sqrt{\alpha_{t}}\mathbf{x}_{0}+\sqrt{1-\alpha_{t}}\bm{\epsilon},t)\right\|_{2}^{2}].$
(5)

Until now, we mainly cover generating EV charging load time-series
$\mathbf{x}_{0}$ without any restrictions. In practice, many side features are
collected together with charging load profiles, such as weather and past
charging demand data, which help us better inform the generated curves. We
adopt such features as conditional information, and develop the conditional
diffusion model depicted in Fig. 1. A straightforward way to achieve
conditional generation is to update the reverse transition in (3) into the
conditional normal distribution below [21]:

$\centering
p_{\theta}(\mathbf{x}_{t-1}|\mathbf{x}_{t},\mathbf{c})=\mathcal{N}(\mathbf{x}_{t-1};\bm{\mu}_{\theta}(\mathbf{x}_{t},\mathbf{c},t),\tilde{\beta}_{t}\mathbf{I});\@add@centering$
(6)

where $\mathbf{c}$ represents generic condition terms and in our problem
setup, $\mathbf{c}$ consists of historical charging load and covariate set.
For simplicity, $\mathbf{\Sigma}_{\theta}$ in (3) is fixed to
$\tilde{\beta}_{t}$ and
$\tilde{\beta}_{t}=\frac{1-\alpha_{t-1}}{1-\alpha_{t}}{\beta}_{t}$ [20], which
alleviates the burden to learn $\mathbf{\Sigma}_{\theta}$ separately. Then, we
can naturally condition the training objective in (5) on $\mathbf{c}$:

$\centering\mathcal{L}_{t-1}=\mathbb{E}_{\mathbf{x}_{0},\mathbf{c},\bm{\epsilon},t}[\left\|\bm{\epsilon}-\bm{\epsilon}_{\theta}(\sqrt{\alpha_{t}}\mathbf{x}_{0}+\sqrt{1-\alpha_{t}}\bm{\epsilon},\mathbf{c},t)\right\|_{2}^{2}].\@add@centering$
(7)

Once we train a conditional denoising network $\bm{\epsilon}_{\theta}$, we can
sample $\mathbf{x}_{t-1}$ via the step-wise denoising operation below:

$\mathbf{x}_{t-1}=\frac{1}{\sqrt{1-\beta_{t}}}(\mathbf{x}_{t}-\frac{\beta_{t}}{\sqrt{1-\alpha_{t}}}\bm{\epsilon}_{\theta}(\mathbf{x}_{t},\mathbf{c},t))+\sqrt{\tilde{\beta}_{t}}\mathbf{z},$
(8)

where $\mathbf{z}\sim\mathcal{N}(\mathbf{0},\mathbf{I})$. This is actually a
stochastic sampling procedure, because at each step $t$ in the reverse
process, we also need to sample Gaussian noise $\mathbf{z}$.

##  IV Probabilistic Forecasting Framework

In this section, we detail how to construct DiffPLF by appling the diffusion
model for probabilistic charging load forecasts, which includes a denoising
network with cross-attention conditioning mechanism and task-specific fine-
tuning. Holistic implementation of DiffPLF is depicted in Algorithm 1.

Figure 2: The architecture of the proposed denoising network dedicated for the
conditional diffusion model.

###  IV-A Denoising Network

To model the target conditional distribution
$q(\mathbf{x}_{0}|\mathbf{p},\mathbf{r})$, we should train an effective
denoising network
$\bm{\epsilon}_{\theta}(\mathbf{x}_{t},\mathbf{p},\mathbf{r},t)$ by
$\mathcal{L}_{t-1}$ in (7), whose input is the perturbed charging load profile
$\mathbf{x}_{t}$, diffusion step $t$ and conditional terms
$\mathbf{c}=\\{\mathbf{p},\mathbf{r}\\}$, and output is the noise
$\bm{\epsilon}$ added on $\mathbf{x}_{t}$. How to incorporate the supplemental
$\mathbf{c}$ into $\bm{\epsilon}_{\theta}$, i.e., to condition
$\bm{\epsilon}_{\theta}$ on $\mathbf{c}$ is a vital issue. For instance,
TimeGrad [16] simply concatenates $\mathbf{x}_{t}$ and $\mathbf{c}$ into joint
input vectors for LSTM units and does not explicitly use any conditioning
schemes. Whereas in text-to-image generation, several conditioning ways such
as classifier-based guidance [15], classifier-free guidance [26] and cross-
attention mechanism [21] are proposed to yield proper images which are highly
aligned with text semantics. We employ the cross-attention mechanism to
entangle $\mathbf{x}_{t}$ with $\mathbf{c}$. The cross-attention in our method
aims to discover latent information of conditions $\mathbf{c}$ that are
correlated with predicted profiles $\mathbf{x}_{0}$.

As shown in Fig. 2, the denoising network $\bm{\epsilon}_{\theta}$ contains
four components: 1) Perturbation encoder $f_{\phi}$, which feeds perturbed
time-series $\mathbf{x}_{t}$ along with $t$ to a LSTM and linear layer
respectively, and then utilizes a self-attention module to integrate the
latent features of $\mathbf{x}_{t}$ and $t$. It actually amounts to the
unconditional noise prediction manner defined in (5). 2) Condition encoder
$h_{\varphi}$, which helps represent the condition set
$\mathbf{c}=\\{\mathbf{p},\mathbf{u},\mathbf{v},\mathbf{d},e\\}$. We
concatenate the temporal data $\\{\mathbf{p},\mathbf{u},\mathbf{v}\\}$, and
use LSTM to characterize their time dependencies. The discrete calendar vector
$\mathbf{d}$ and EV number $e$ are handled by a linear layer. Then we also
employ self-attention to fuse their latent features. 3) Cross-attention
mechanism, which is to condition the latent encoding of $\mathbf{x}_{t}$ and
$t$ based on conditions $\mathbf{c}$, and this module is conducive to capture
the conditional predictive distribution by (7). Denote
$\mathbf{Q}=h_{\varphi}(\mathbf{p},\mathbf{r})\cdot\mathbf{W}^{Q}$,
$\mathbf{K}=f_{\phi}(\mathbf{x}_{t},t)\cdot\mathbf{W}^{K}$,
$\mathbf{V}=f_{\phi}(\mathbf{x}_{t},t)\cdot\mathbf{W}^{V}$, then the cross-
attention mechanism is formulated as follows:

$\centering
Attention(\mathbf{Q},\mathbf{K},\mathbf{V})=softmax(\frac{\mathbf{Q}\mathbf{K}^{T}}{\sqrt{d}})\mathbf{V},\@add@centering$
(9)

where $\mathbf{W}^{Q}$, $\mathbf{W}^{K}$, $\mathbf{W}^{V}$ are three linear
transformation matrix weights to be optimized, and $d$ is their hidden
dimension. 4) Forecast network, which finally uses a self-attention module
coupled with a linear projection to transform output features of cross-
attention to the predicted noise $\bm{\epsilon}$ added on $\mathbf{x}_{t}$.

Figure 3: The illustration of the fine-tuning procedure.

###  IV-B Fine-tuning Technique

Indeed, we can train the former denoising network using (7) and employ it to
generate $N$ profiles $\left\\{\hat{\mathbf{x}}^{n}_{0}\right\\}^{N}_{n=1}$
via (8), where $\hat{\mathbf{x}}_{0}$ denotes the synthetic time-series. But
these produced profiles may not form a high-quality PI [16] that is required
by probabilistic forecasting. To this end, we expect the 50%-quantile
$m_{0,i}$ of $N$ generated charging load
$\left\\{\hat{x}^{n}_{0,i}\right\\}^{N}_{n=1}$ at each step $i$ should be
close to the actual value $x_{0,i}$ as much as possible. In this work, we can
treat $\mathbf{m}_{0}\in\mathbb{R}^{\tau}$ as point forecasts. Then our goal
is to minimize the 50%-quantile deviation
$\left\|\mathbf{m}_{0}-\mathbf{x}_{0}\right\|^{2}_{2}$. Such term is an
inductive bias and a task-specific refinement. We leverage it to fine-tune the
former model pre-trained by (7), where we explicitly enforce the conditional
diffusion model to attain a particular objective. Besides, since
$\mathbf{m}_{0}$ is constituted by the predicted outcomes of the pre-trained
diffusion model, this fine-tuning procedure is realized by its own generated
samples. As shown in Fig. 4, we can obtain sharper PI as well as more accurate
deterministic forecasts after the task-informed fine-tuning stage. As
$\hat{\mathbf{x}}^{n}_{0}$ is generated by a stochastic sampling process which
contains multiple times of iterations for $\bm{\epsilon}_{\theta}$, it is
infeasible to directly use the divergence between $\mathbf{m}_{0}$ and
$\mathbf{x}_{0}$ to refine the parameters of $\bm{\epsilon}_{\theta}$. To this
end, we propose to leverage an alternative fine-tuning 50%-quantile deviation
minimization (QDM) objective:

$\centering\mathcal{L}_{QDM}=\left\|\bm{\epsilon}_{\theta}(\mathbf{m}_{t},\mathbf{c},t)-\bm{\epsilon}_{\theta}(\mathbf{x}_{t},\mathbf{c},t)\right\|^{2}_{2},\@add@centering$
(10)

where $\mathbf{m}_{t}$ and $\mathbf{x}_{t}$ indicate we adpot (2b) to corrupt
$\mathbf{m}_{0}$ and $\mathbf{x}_{0}$ by the same noise $\bm{\epsilon}$ at
diffusion step $t$, and $c$ belongs to the ground-truth $\mathbf{x}_{0}$.
Intuitively, minimizing $\left\|\mathbf{m}_{0}-\mathbf{x}_{0}\right\|^{2}_{2}$
can be equivalent to the goal of $\mathcal{L}_{QDM}$, which is also consistent
with the training paradigm of the noise predictor. Moreover, we wish the
conditional predictive distribution learned by the pre-trained diffusion model
can not be impaired by $\mathcal{L}_{QDM}$, thus we also incorporate the
$\bm{\epsilon}$-prediction loss of (7) in this fine-tuning stage, then the
total loss for $\bm{\epsilon}_{\theta}$ refinement is:

$\centering\mathcal{L}_{ref}=\mathcal{L}_{t-1}+\lambda\mathcal{L}_{QDM},\@add@centering$
(11)

where $\lambda$ is the weight of QDM loss. The role $\mathcal{L}_{t-1}$ plays
in $\mathcal{L}_{ref}$ can be comprehended as a prior preservation item, which
indicates that when the pre-trained diffusion model is carrying out the task-
specific fine-tuning, it is able to retain its prior generative capability
simultaneously. Our fine-tuning operation is illustrated in Fig. 3.

Figure 4: Forecasting sample comparisons between the fine-tuned model and
model without fine-tuning.

Input: training data $\mathcal{X}$, testing data $\mathcal{Y}$, diffusion step
$T$

Stage 1: Pretrain the denoising network $\bm{\epsilon}_{\theta}$

1\. Sample $t\sim\text{Uniform}(\\{1,...,T\\})$

2\. Sample $\mathbf{x}_{0},\mathbf{c}\sim\mathcal{X}$,
$\bm{\epsilon}\sim\mathcal{N}(\mathbf{0},\mathbf{I})$

3\. Optimize $\theta$ using $\mathcal{L}_{t-1}$ in (7)

Stage 2: Task-specific fine-tuning

1\. Sample $\mathbf{x}_{0},\mathbf{c}\sim\mathcal{X}$

2\. Generate $\left\\{\hat{\mathbf{x}}^{n}_{0}\right\\}^{N}_{n=1}$ via (8) and
pretrained model

3\. Calculate median $\mathbf{m}_{0}$ for generated
$\left\\{\hat{\mathbf{x}}^{n}_{0}\right\\}^{N}_{n=1}$

4\. Obtain perturbed $\mathbf{x}_{t}$ and $\mathbf{m}_{t}$ using (2a)

5\. Refine $\theta$ using $\mathcal{L}_{ref}$ in (11)

Stage 3: Forecasting via inference

1\. Sample $\mathbf{c}\sim\mathcal{Y}$,
$\mathbf{x}_{T}\sim\mathcal{N}(\mathbf{0},\mathbf{I})$

2\. Predict $\mathbf{x}_{0}$ by iterating (8) for $T$ times

Algorithm 1 Implementation of DiffPLF

##  V Numerical Experiments

###  V-A Experimental Setup

1) Data description: We harness a real-world dataset in the city of Palo Alto,
California termed EV Charging Station Usage Open
Data111https://www.kaggle.com/datasets/venkatsairo4899/ev-charging-station-
usage-of-california-city. It elaborates daily charging session details of
individual stations, including charging durations and delivered energy for
each EV. We resort to the transformation method in [7] to aggregate single
battery charging curves to total charging load profiles with 15-min
resolution. We fetch corresponding weather forecasts for Palo Alto from
Meteostat platform222https://dev.meteostat.net/. For all experiments involved
in this section, we use historical EV charging demand and weather recording
from 2016 to 2018 for both training and fine-tuning, while data in 2019 for
testing.

2) Implementation details: Regarding the historical data utilization, we
exploit the aggregate charging demand of past 5 days to guide future load
forecasts. In terms of the model architecture, we unify the hidden dimensions
of LSTM, cross-attention and self-attention as 32, and the head number of two
attention modules is set to 4. As for the noise scheduling, we follow the
common quadratic scheme adopted in [18], where the start variance
$\beta_{1}=0.0001$, the eventual variance $\beta_{T}=0.5$, and the number of
diffusion steps $T$ is 200.

We compile the whole DiffPLF using Pytorch library and implement it on a Linux
service machine with a 48GB Nvidia A40 GPU. We use Adam optimizer to carry out
the stochastic gradient descent with batch size of 16 for the noise predictor.
During the pre-training and fine-tuning stage, the initial learning rate and
total training epochs are 0.001/0.0002 and 200/100 respectively. Meanwhile, we
find that we can achieve the best results when the weight of QDM loss
$\lambda$ is 0.001 for model refinement. For every testing scenario, we
randomly generate 1,000 possible trajectories of future charging load to
constitute the target PI. After the whole implementation of the proposed
model, we find that the average training time of every epoch is 2.0174s and
2.5514s for the pre-training and fine-tuning stage respectively. Such
discrepancy results from the fact that the denoising network will be executed
only once during each standard diffusion training epoch, whilst being operated
twice within each fine-tuning session. Besides, the mean inference time over
each test case is 5.7511s, and note that when running DiffPLF on every test
sample, we generate 1000 future charging load profiles in parallel.

Figure 5: Randomly selected testing samples. In each subplot, we depict the
real charging load versus the generated day-ahead PI and point forecast.

###  V-B Simulation Results

We focus on the day-ahead charging demand forecasting, where we predict
charging load in the next day based on historical observations of past five
days. We adopt two common metrics to evaluate the holistic performance of the
proposed DiffPLF framework: 1) Mean Absolute Error (MAE) [14]. This indicator
is aimed at assessing the point forecasting ability of the probabilistic
forecasting model. We treat the median of the generated PI as the
deterministic prediction. 2) Continuous Ranked Probability Score (CRPS) [16].
This index is utilized to judge the quality of the predictive distribution
which is supposed to encompass the true realization. We compute the CRPS value
for each prediction step and average that over all time steps as final CRPS
for one testing sample.

In Fig. 5, we randomly draw and show 12 samples out of all testing samples in
2019. Evidently, the ground truth profile can not only be covered by shaded
areas of either 50% or 90% PI, but also keep close to the produced point
prediction to a large margin. It validates that our DiffPLF is able to yield
both accurate and reliable probabilistic forecasts. To further verify the
superiority of the diffusion-driven generative paradigm to model the
predictive distribution, we alter the original training manner (i.e. noise
prediction) of DiffPLF to quantile regression [10]. Such method is trained to
explicitly predict multiple probabilistic intervals and treated as a classical
technique on probabilistic time-series forecasting. Specifically, we employ
the developed noise predictor to directly estimate several quantiles (i.e. 5%,
25%, 75%, 95%) of future charging demand using the pinball loss [27] and
exhibit its results in both the first row of Table I and Fig. 6. We can
observe that DiffPLF can achieve considerably more accurate and sharper PI
than the quantile regression method, because the extreme volatility of EV
charging load renders it quite tough for the neural network to stably optimize
the quantile loss. In Table I we summarize the method comparisons, where w/o
means model without the investigated component.

TABLE I: Quantitative evaluation for different variants of DiffPLF. Method | MAE | CRPS
---|---|---
Quantile Regression | 11.852±3.936 | 10.107±2.124
w/o covariates | 7.842±2.065 | 5.592±1.570
w/o cross-attention | 7.333±1.559 | 5.192±1.099
w/o fine-tuning | 7.227±1.489 | 5.111±1.041
whole fine-tuning | 7.200±1.607 | 5.089±1.122
DiffPLF |  7.161±1.557 |  5.067±1.094
Figure 6: Two forecasts of our generative DiffPLF versus quantile regression.

Cross-attention Mechanism: Previous works also bring up conditioning
mechanisms like feature fusion in the latent space [16], [18]. For comparison,
we utilize the element-wise addition to blend the latent embedding of both
perturbation and condition encoder. From the third line in Table I, a moderate
drop more than 0.12 occurs on two metrics, indicating that cross-attention is
a more effective way compared to latent fusion when modeling the conditional
distribution of temporal charging load data.

Supplementary Covariates: Many previous methods fetch charging demand
forecasts merely based on historical observations, ignoring the benefit of
certain covariates which can be known for the prediction horizon in advance.
Actually, the possible charging load distribution ought to comply with the
covariate set, containing weather forecasts, day type and EV number, which are
actually a kind of conducive constraints for the predictive model to produce
more realistic and accurate forecasts. In order to investigate how DiffPLF
benefits from such additional covariates, we purposely discard the input set
$\mathbf{r}$ for the condition encoder and write its testing results in the
second row of Table I. After the covariate set is eradicated, we observe that
there exists a salient decrease of 9.5% and 10.4% on MAE and CRPS
respectively. It reflects that the appended covariates described in Section II
are crucial for our approach to generate satisfactory probabilistic forecasts.

Task-specific Fine-tuning: In order to render the denoising diffusion method
more amenable to the probabilistic time-series prediction task, we propose to
fine-tune the pretrained diffusion model using the loss function
$\mathcal{L}_{ref}$ defined in (11). $\mathcal{L}_{ref}$ consists of a prior
preservation term $\mathcal{L}_{t-1}$ and a weighted item $\mathcal{L}_{QDM}$
to minimize the discrepancy between the median of generated PI and the ground-
truth signal. To validate the efficacy of this fine-tuning trick, we compare
the performance of entire DiffPLF and its pre-trained version without fine-
tuning in Table I. Apparently, DiffPLF outcomes degrade to a mild extent after
removing its task-specific refinement, which indicates that the proposed fine-
tuning technique is able to improve the effect of diffusion-based generative
modeling on probabilistic forecasting for EV charging load.

Furthermore, we look into how different fine-tuning methods affect the final
outcomes. In the last two rows of Table I, we show that merely fine-tuning the
former part of the noise predictor (while latter weights of cross-attention
and parallel encoders are frozen) is better than refining the whole model. The
performing gap between such two types of fine-tuning may arise from the
implicit adversity of $\mathcal{L}_{QDM}$ for optimizing the latent encoding
of heterogeneous input data and conditioning mechanism, which instead prefer
to be optimized by the $\bm{\epsilon}$-prediction manner.

TABLE II: DiffPLF performance under different diffusion steps. Diffusion step T | MAE | CRPS
---|---|---
100 | 9.127±1.894 | 6.458±1.362
150 | 7.313±1.555 | 5.177±1.106
200 | 7.227±1.489 | 5.111±1.041
250 | 9.569±2.096 | 6.780±1.492
300 | 9.503±1.973 | 6.742±1.415

Varying diffusion step $T$: We investigate the model performance with respect
to different settings of diffusion step $T$, since $T$ is one of the key
factors that can determine the ultimate generation outcomes of discrete-time
diffusion models [28]. We conduct this sensitivity analysis just in the pre-
training context, and forecasting outcomes of default $T=200$ is shown in the
fourth row of Table I, while evaluation results for other $T$ values are
exhibited in Table II. Totally, DiffPLF can achieve the best results on
$T=200$ but be relatively less effective on other four $T$ settings. In light
of [28] and [25], larger $T$ will give rise to heavy tails in the noise
schedule which can deteriorate the learning efficiency of denoising network.
Smaller $T$ can increase the discretization errors of continuous stochastic
diffusion equations and render the Gaussian form presumption on the reverse
transition (3) less valid. How to determine the best $T$ for different
forecasting scenarios more properly is of great significance, we leave it for
future work.

Figure 7: Examples of produced PI with different prediction horizons. Figure
8: Mean and variance of MAE and CRPS for two additional analysis: (a) Varying
prediction horizons. (b) Different deviations of EV number.

Varying prediction horizons: Here, we illustrate DiffPLF can be seamlessly
scaled to different prediction horizons on top of precedent day-ahead
forecasting. For the original 24h horizon $[s+1,s+\tau]$, we assume that
charging demand in $[s+1,s+\eta],0<\eta<\tau$ has been measured and that in
$[s+\eta+1,s+\tau]$ should be forecasted. Then, the prediction length is
changed from $\tau$ to $\tau-\eta$. We can achieve this target by just
modifying the input data for $\mathcal{L}_{QDM}$, without overriding the pre-
trained diffusion model. Specifically, we simply fix
$\mathbf{m}^{s+1:s+\eta}_{t}=\mathbf{x}^{s+1:s+\eta}_{t}$, leaving the output
demand in $[s+\eta+1,s+\tau]$ to be determined. We consider a set of 12h, 6h,
4h and 1h forecasting horizons with examples shown in Fig. 7, where we can
find that DiffPLF can also yield sharp and reliable PI for various prediction
horizons. As is shown in Fig. 8 (a), the fine-tuned model holds consistent
performance under varying forecasting lengths under both MAE and CRPS metrics.

Figure 9: Predicted daily cumulative charging energy profiles under various EV
numbers. The bold line depicts the mean value of generated profiles (plotted
by light lines) for each group with a fixed number of EVs.

Varying EV numbers: In practice, operators may be interested to analyze the
charging load under different number of EV customers. To this end, we wish to
investigate how the variable $e$ affects forecasting outcomes both visually
and quantitatively. We fix the already trained DiffPLF and only adjust the
input $e$ for the condition encoder. We select a testing sample on June 8th
and give five different $e$, and the resulting profiles under each $e$ are
depicted in Fig. 9 respectively. Note that we showcase the cumulative charging
energy, which are integration of instantaneous charging load. This shows
DiffPLF can generate charging energy curves with unique magnitudes and
ascending trends. It also suggests that our diffusion model is able to attain
controllable generation conditioned on various EV numbers. Moreover, we also
want to study the impact of EV quantity deviations on the accuracy of
probabilistic forecasts. In Fig. 8(b), we exhibit numeric assessments when $e$
diverges from its ground-truth value by $\pm 5\%$ and $\pm 10\%$. We find that
DiffPLF is robust to errors of EV number in total, except for the $-10\%$
group, where our model shows a modest drop on prediction results. These
results together verify that DiffPLF can be generalizable to EV numbers,
forecasting horizons and data samples. In future work, it is also intriguing
to benchmark diffusion model’s efficacy across different patterns of EV
charging datasets.

##  VI Conclusion and Outlook

In this paper, we focus on forecasting EV charging load in a probabilistic way
by proposing a novel conditional diffusion model DiffPLF. DiffPLF combines the
denoising diffusion-driven generation and cross-attention mechanism to capture
the predictive distribution conditioned on past demand and complementary
covariates. A task-specific fine-tuning approach is devised to further
ameliorate the quality of produced prediction intervals. Numerical experiments
verify DiffPLF can achieve satisfactory probabilistic demand forecasting and
controllable charging profile prediction under flexible look-ahead horizons.
Since the current method requires an additional task-informed fine-tuning
operation to improve the prediction accuracy, we look forward to developing a
more efficient end-to-end diffusion model which can be specialized in
probabilistic load forecasting in future work. Besides, we hope to extend our
model to predict long-term charging load which is beneficial for charging
infrastructure planning. We also intend to explore multivariate diffusion-
based generative model to handle EV charging and more general energy time-
series.

## References

  * [1] H. Tu, H. Feng, S. Srdic, and S. Lukic, “Extreme fast charging of electric vehicles: A technology overview,” _IEEE Trans. Transp. Electrif._ , vol. 5, no. 4, pp. 861–878, 2019.
  * [2] “The 2030 national charging network,” 2023, https://www.nrel.gov/docs/fy23osti/85970.pdf.
  * [3] Z. Jia, J. Li, X.-P. Zhang, and R. Zhang, “Review on optimization of forecasting and coordination strategies for electric vehicle charging,” _J. Mod. Power Syst. Clean Energy_ , vol. 11, no. 2, pp. 389–400, 2023.
  * [4] Z. Wan, H. Li, H. He, and D. Prokhorov, “Model-free real-time ev charging scheduling based on deep reinforcement learning,” _IEEE Trans. Smart Grid_ , vol. 10, no. 5, pp. 5246–5257, 2019.
  * [5] H. Wang, Y. Jia, M. Shi, C. S. Lai, and K. Li, “A mutually beneficial operation framework for virtual power plants and electric vehicle charging stations,” _IEEE Trans. Smart Grid_ , pp. 1–1, 2023.
  * [6] M. Dabbaghjamanesh, A. Moeini, and A. Kavousi-Fard, “Reinforcement learning-based load forecasting of electric vehicle charging station using q-learning technique,” _IEEE Trans. Ind. Inform._ , vol. 17, no. 6, 2021\.
  * [7] M. B. Arias and S. Bae, “Electric vehicle charging demand forecasting model based on big data technologies,” _Appl. Energy_ , vol. 183, 2016.
  * [8] F. Wu and R. Sioshansi, “A two-stage stochastic optimization model for scheduling electric vehicle charging loads to relieve distribution-system constraints,” _Transportation Research Part B: Methodological_ , vol. 102, pp. 55–82, 2017.
  * [9] K. Phipps, K. Schwenk, B. Briegel, R. Mikut, and V. Hagenmeyer, “Customized uncertainty quantification of parking duration predictions for ev smart charging,” _IEEE Internet of Things Journal_ , 2023.
  * [10] T. Hu, H. Ma, H. Liu, H. Sun, and K. Liu, “Self-attention-based machine theory of mind for electric vehicle charging demand forecast,” _IEEE Trans. Ind. Inform._ , vol. 18, no. 11, pp. 8191–8202, 2022.
  * [11] L. Buzna, P. De Falco, G. Ferruzzi, S. Khormali, D. Proto, N. Refa, M. Straka, and G. van der Poel, “An ensemble methodology for hierarchical probabilistic electric vehicle load forecasting at regular charging stations,” _Appl. Energy_ , vol. 283, p. 116337, 2021.
  * [12] Y. Li, S. He, Y. Li, L. Ge, S. Lou, and Z. Zeng, “Probabilistic charging power forecast of evcs: Reinforcement learning assisted deep learning approach,” _IEEE T. Intell. Veh._ , vol. 8, no. 1, pp. 344–357, 2023.
  * [13] X. Zhang, K. W. Chan, H. Li, H. Wang, J. Qiu, and G. Wang, “Deep-learning-based probabilistic forecasting of electric vehicle charging load with a novel queuing model,” _IEEE T. Cybern._ , vol. 51, no. 6, 2021\.
  * [14] D. W. X. Huang and B. Boulet, “Metaprobformer for charging load probabilistic forecasting of electric vehicle charging stations,” _IEEE Trans. Intell. Transp. Syst._ , pp. 1–11, 2023.
  * [15] P. Dhariwal and A. Nichol, “Diffusion models beat gans on image synthesis,” _Advances in neural information processing systems_ , vol. 34, pp. 8780–8794, 2021.
  * [16] K. Rasul, C. Seward, I. Schuster, and R. Vollgraf, “Autoregressive denoising diffusion models for multivariate probabilistic time series forecasting,” in _Proceedings of the 38th International Conference on Machine Learning_ , vol. 139, 2021, pp. 8857–8868.
  * [17] K. Rasul, A.-S. Sheikh, I. Schuster, U. Bergmann, and R. Vollgraf, “Multivariate Probabilistic Time Series Forecasting via Conditioned Normalizing Flows,” in _International Conference on Learning Representations 2021_ , 2021.
  * [18] Y. Tashiro, J. Song, Y. Song, and S. Ermon, “Csdi: Conditional score-based diffusion models for probabilistic time series imputation,” in _Advances in Neural Information Processing Systems_ , vol. 34, 2021, pp. 24 804–24 816.
  * [19] J. H. Lee, D. Chakraborty, S. J. Hardman, and G. Tal, “Exploring electric vehicle charging patterns: Mixed usage of charging infrastructure,” _Transportation Research Part D: Transport and Environment_ , vol. 79, p. 102249, 2020.
  * [20] J. Ho, A. Jain, and P. Abbeel, “Denoising diffusion probabilistic models,” in _Advances in Neural Information Processing Systems_ , vol. 33, 2020, pp. 6840–6851.
  * [21] R. Rombach, A. Blattmann, D. Lorenz, P. Esser, and B. Ommer, “High-resolution image synthesis with latent diffusion models,” in _2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)_ , 2022, pp. 10 674–10 685.
  * [22] R. Yuan, B. Wang, Y. Sun, X. Song, and J. Watada, “Conditional style-based generative adversarial networks for renewable scenario generation,” _IEEE Trans. Power Syst._ , vol. 38, no. 2, pp. 1281–1296, 2023.
  * [23] S. Shahriar, A. R. Al-Ali, A. H. Osman, S. Dhou, and M. Nijim, “Prediction of ev charging behavior using machine learning,” _IEEE Access_ , vol. 9, pp. 111 576–111 586, 2021.
  * [24] M. Neaimeh, R. Wardle, A. M. Jenkins, J. Yi, G. Hill, P. F. Lyons, Y. Hübner, P. T. Blythe, and P. C. Taylor, “A probabilistic approach to combining smart meter and electric vehicle charging data to investigate distribution network impacts,” _Appl. Energy_ , vol. 157, 2015.
  * [25] Y. Song, J. Sohl-Dickstein, D. P. Kingma, A. Kumar, S. Ermon, and B. Poole, “Score-based generative modeling through stochastic differential equations,” _arXiv preprint arXiv:2011.13456_ , 2020.
  * [26] J. Ho and T. Salimans, “Classifier-free diffusion guidance,” _arXiv preprint arXiv:2207.12598_ , 2022.
  * [27] S. M. J. Jalali, P. Arora, B. Panigrahi, A. Khosravi, S. Nahavandi, G. J. Osório, and J. P. Catalão, “An advanced deep neuroevolution model for probabilistic load forecasting,” _Electric Power Systems Research_ , vol. 211, p. 108351, 2022.
  * [28] A. Q. Nichol and P. Dhariwal, “Improved denoising diffusion probabilistic models,” in _International Conference on Machine Learning_. PMLR, 2021, pp. 8162–8171.


# Learning manipulation of steep granular slopes
for fast Mini Rover turning

Deniz Kerimoglu1,∗, Daniel Soto1,∗, Malone Hemsley2, Joseph Brunner1, Sehoon
Ha3, Tingnan Zhang4
and Daniel I. Goldman1 *Co-first authors1Deniz Kerimoglu, Daniel Soto, Joseph
Brunner and Daniel I. Goldman are with School of Physics, Georgia Institute of
Technology, Atlanta, USA<EMAIL_ADDRESS>Hemsley is with the
Department of Applied Physics and Engineering, Morehouse College, Atlanta,
USA3Sehoon Ha is with the School of Interactive Computing, Georgia Institute
of Technology, Atlanta, USA4Tingnan Zhang is with the Google Robotics,
Mountain View, California, USA

###### Abstract

Future planetary exploration missions will require reaching challenging
regions such as craters and steep slopes. Such regions are ubiquitous and
present science-rich targets potentially containing information regarding the
planet’s internal structure. Steep slopes consisting of low-cohesion regolith
are prone to flow downward under small disturbances, making it very
challenging for autonomous rovers to traverse. Moreover, the navigation
trajectories of rovers are heavily limited by the terrain topology and future
systems will need to maneuver on flowable surfaces without getting trapped,
allowing them to further expand their reach and increase mission efficiency.

In this work, we used a laboratory-scale rover robot and performed maneuvering
experiments on a steep granular slope of poppy seeds to explore the rover’s
turning capabilities. The rover is capable of lifting, sweeping, and spinning
its wheels, allowing it to execute leg-like gait patterns. The high-
dimensional actuation capabilities of the rover facilitate effective
manipulation of the underlying granular surface. We used Bayesian Optimization
(BO) to gain insight into successful turning gaits in high dimensional search
space and found strategies such as differential wheel spinning and pivoting
around a single sweeping wheel. We then used these insights to further fine-
tune the turning gait, enabling the rover to turn 90 degrees at just above 4
seconds with minimal slip. Combining gait optimization and human-tuning
approaches, we found that fast turning is empowered by creating anisotropic
torques with the sweeping wheel.

##  I INTRODUCTION

Almost all portions of the celestial bodies and a significant portion of the
granular surface on Earth remain inaccessible to the majority of wheeled and
tracked vehicles. This is partly due to the limitations of locomoting robots
that struggle to effectively interact with their immediate surroundings.
Particularly, wheeled systems can get stuck in loose granular media as they
sink deeper into the material during the wheel-spinning motion [1]. These
difficulties intensify in the vicinity of craters and other destinations
characterized by steep, flowable surfaces, where a minimal disturbance could
potentially cause yielding and downhill flow. Such regions are scientifically
valuable locations, potentially harboring layers of geological history as well
as containing shaded areas where water ice might be preserved [2, 3]. To
explore these difficult terrains, the next generation of exploratory robots
will need to navigate over steep flowable slopes.

Future missions will also require autonomous systems to negotiate difficult
terrain topologies by performing effective maneuvers to increase access to a
diverse range of terrain types. This will allow currently-denied extreme
terrain to become traversable, hence achieving scientific mission goals would
be less time- and energy-consuming due to the greatly shortened paths.

Recent research has addressed some of the mobility limitations to some extent.
For loose regolith/soil on level ground, refined terramechanical models [4]
and new methods [5] based on granular resistive force theory [6] better
predict the rover-terrain interaction. On the robotics side, various wheeled
and non-wheel robotic platforms have been developed [7, 8, 9]. Few have
achieved robust mobility in loose granular slopes, [10, 11, 12]. However,
robotic turning has not been extensively explored on steep loose granular
slopes yet.

Our previous works explored the climbing gaits of a laboratory-scale rover
robot, namely Mini Rover, a scaled-down version of a robotic lunar rover
prototype called NASA Resource Prospector 15 [13, 14]. The Mini Rover
incorporates wheel and leg-like appendages that actively manipulate flowable
terrain to climb granular slopes [13]. We extended Mini Rover’s capabilities
in [14] using Bayesian optimization, enabling the discovery of fast climb
gaits by exploiting the fluid and solid aspects of granular materials. These
investigations exemplify a wider category of interactions involving locomotor
and flowable terrain. The locomotor becomes integrated with its surroundings
behaving as a unified structure and actively modifies its environment in order
to achieve success. In this work, motivated by leveraging terrain manipulation
strategies, we study the turning of the Mini Rover robot on a steep granular
slope. To this end, simulation environments to investigate the Mini Rover
gaits are impractical since simulating interactions between the robot and the
millions of granular particles is extremely time-consuming even with the use
of high performance computing tools. Moreover, both the robot and the terrain
exhibit complex, nonlinear dynamics, hence deriving closed-form solutions to
represent such interactions is not yet possible, hindering model-based
simulation approaches. Therefore we rely on laboratory experiments and
optimization methods to find and investigate Rover gait patterns.

Inspired by recent progress in robot learning we used Bayesian optimization
[15] to search the extensive gait space and acquire gait cues. We started by
picking a successful gait from our previous work [14] which was optimized to
rapidly climb granular inclines and modified it to prompt turning behavior in
a desired direction. This gait is then used as a seed for the Bayesian
optimizer which learned faster turning strategies. The insights from the
optimization are then used to understand the locomotor-terrain interactions
and to perform targeted systematic experiments for further gait development.

We discovered that the Mini Rover achieves fast turning by combining multiple
terrain manipulation strategies. Mini Rover first creates a yaw motion via
differential wheel spinning, followed by a single rear wheel sweeping motion
that exploits the multiphase feature of the granular media by generating
anisotropic torques. Finally, the rover aligns its roll angle to the changing
slope condition by extending and retracting its wheels accordingly. Hence, our
ML-inspired new gait outperforms gaits found solely by using Bayesian
Optimization and manual tuning by a large margin, turning in 4.5 seconds as
opposed to 2 minutes, respectively.

##  II Materials and Methods

###  II-A Robotic Platform: Mini Rover

The Mini Rover consists of 12 Dynamixel AX-12 actuators and 3D-printed
linkages and grousers (Fig. 1B). Each wheel-leg complex can lift up and down
as well as spin and sweep. The wheel-leg appendages function in a way that
resembles legs for propulsion, but they also spin to disrupt the surrounding
terrain as wheels operate. The rover’s main body is revised to be lighter to
reduce the effect of inertial forces during fast turning.

Figure 1: Experimental apparatus and the rover images. (A) shows the tiltable
and air-fluidized testbed of poppy seeds. (B) shows the Mini Rover on loose
granular slop. (C) illustrates the actuation parameters of the Mini Rover.

The experimental testbed that we run the Mini Rover is a tiltable bed
containing 1 mm diameter poppy seeds (Fig. 1A). The bed size is 2.5 m by 1.2
m, which is large enough to avoid boundary effects during turning. The poppy
seeds represent a flowable and deformable terrain, a laboratory analog for
flowable and deformable surfaces [13]. The granular terrain is reset to a
loosely consolidated state after each experiment via air fluidization,
creating a controlled uniform initial state for systematic experiments (Fig.
1A). For robot tracking purposes, we placed three blue spherical markers on
top of the rover and three webcams were mounted on the testbed to capture the
top and side-view videos. Markers constitute a triangle which is used to track
the position and orientation of the rover using MATLAB Computer Vision
Toolbox. Control of the rover is achieved by creating open loop gaits
trajectories in MATLAB which are tracked internally via Dynamixel actuators’
motor drivers and PID controllers.

To understand the locomotor-terrain interaction we performed experiments to
measure the torque generated by the wheel moving in poppy seeds. The single-
wheel sweep and spin experiments were performed on a platform capable of
moving vertically via a Firgelli linear actuator. A 6-axis Force sensor from
ATI Industries is mounted to the linear actuator. The sweeping and spinning
wheel system is mounted to the force sensor to capture the resistive torques
induced by the poppy seeds. First, the wheel system is immersed 2 cm into the
poppy seeds, and then the substrate is air fluidized, resetting the media in
the vicinity of the wheel. Then the wheel started to spin and sweep at low and
high rates. The resistive torque data is logged to a desktop computer via a
NI-DAQ card and Labview software.

###  II-B BO-RRP Gait to TRRP Gait

To start inducing turning motion with the Mini Rover we modified the Bayesian
Optimized Rear Rotator Pedaling (BO-RRP) gait, a previously studied successful
gait pattern in [14] (see (Fig. 2A)). BO-RRP lifts, spins, and sweeps the rear
wheel-leg appendages in an alternating style, exploiting the multi-phase
feature of granular media by selectively solidifying/fluidizing the mound
around the wheels. In the ”solidifying phase” the wheel spin halts and the
sweeping wheel exerts high pressure that pushes grains backward, creating high
reaction force, as shown with dashed black curves in Fig. 2A. In the
”fluidizing phase”, the wheel spin is activated to create fluidizing shear
when resetting the appendage’s position, generating a low reaction force, as
shown with dashed red curves. Although this strategy significantly increased
the rover’s locomotion speed, it was incapable of creating maneuvering gaits
and hence would only climb straight.

We modified BO-RRP by disabling the selective solidification/fluidization of
the rear wheel, i.e. disabling the spinning motion while sweeping the wheel
toward the rear of the rover (sweep-out), on the turning side of the Mini
Rover (dashed black arrow in Fig. 2B). In addition, we set RRPs of both sides
to run simultaneously, similar to breaststroke motion. In doing so the
modified gait, called TRRP, created a higher push force on one side and a
lower force on the other, yielding a differential thrust and eventually
turning in one direction.

Figure 2: Mini Rover gait diagrams. (A) illustrates the snapshots of the rover
executing BO-RRP. The white rear wheel sweeps-out 100° (toward the rear of the
rover), while the red wheel sweeps-in, resetting the wheel position (toward
the front of the rover). The sweep-out direction is shown in black arrows. The
dashed arrow indicates that the wheel spinning is disabled during sweep-out.
(B) shows the TRRP gait, where both rear wheels perform sweeping-out at the
same time while the right rear wheel spinning is disabled. Once sweep-out is
completed, both rear wheels spin and sweep-in.

###  II-C Bayesian Optimization of TRRP

We designed the new TRRP gait based on our prior experience, however, these
changes may cause the parameters inherited from BO-RRP to be suboptimal.
Therefore, we take another round of Bayesian optimization to optimize control
parameters further, a technique that has demonstrated effectiveness in
handling intricate robot-terrain interactions [16, 17]. Operating efficiently
on systems with low sampling rates, BO treats the objective function as a
stochastic function without making any assumptions by assigning a prior
probability distribution using a Gaussian process. Over multiple iterations,
the outcomes of function evaluations update the priors to create a posterior
distribution, continuing until converging to the maximum of the objective
function.

The Bayesian optimization is initiated with TRRP so that the gait parameter
search starts around a relatively successful gait. The Bayesian optimization
is performed for 30 experiment episodes on a 25∘ granular slope and we omitted
the results ending in failure. The open-loop gaits are set to run for 60
cycles taking about 2 minutes. The turning trials are initiated at a body
orientation of 0∘ and the objective function is set to be 90∘ in the
counterclockwise direction with respect to the world frame. We tracked the
body orientation, and after each experiment, we updated the objective function
to be the achieved turning angle. BO primarily varied the sweep range of the
rear wheels, the spin direction, and the speed of all wheels as well as the
extension/retraction range of wheel-leg appendages. The optimization search
space is illustrated in Fig. 1C.

##  III Results

###  III-A Baseline Gaits

We start maneuvering experiments with the TRRP gait. TRRP achieved gradual
turning, taking over 2.5 minutes to get to 90∘ of body orientation (see Fig. 5
blue curve). Moreover, the Mini Rover drifted and slid down moderately during
turning and failed 2 times out of 7 trials. This is in part due to its
inability to manipulate the underlying terrain to create enough turning thrust
as well as limitations in its capacity to adapt to the changes in the slope.

###  III-B Optimized TRRP Gait (BO-TRRP)

Next, we fed the TRRP gait parameters into Bayesian Optimization and evaluated
the turning performance of the Mini Rover with iterative experiments, with a
random exploration number of 4. Illustrated in Fig. 3A, in the first 10
iterations, the rover turned in the clockwise direction, opposite to the
desired orientation, peaking its performance at iteration 4. Despite being in
the wrong direction, the Mini Rover achieved turning $82^{\circ}$ by
discovering the tank-turning maneuver with the front wheels spinning in the
reverse direction to each other, a conventional turn-in-place strategy. In
doing so, the Mini Rover turns in place via the torque generated by the
reciprocal spinning of the front wheels. The Mini Rover started to turn in the
right direction at iteration 10. Then, at iteration 14 the optimizer learned
the differential spinning (DS) of both the front and rear wheels and turned in
the desired orientation as illustrated with the time-lapse images in Fig. 3B.
The Mini Rover still performed RRP motion with the rear wheels, however,
incorporating DS motion enabled moderate turning. Consequently, at iteration
19 the optimizer learned to perform RRP only with the appendage located at the
opposing side of the desired orientation (single RRP) as shown in Fig. 3C,
while the remaining wheels kept performing DS motion. The cooperation of both
strategies, called BO-TRRP, resulted in almost 90∘ of turning as shown with
the gait diagram in Fig. 3D.

Figure 3: Evaluation of Bayesian Optimization of turning gaits. (A) depicts
the evolution of Mini Rover’s body orientation as a function optimization
iteration. The circles and crosses represent successful and failed
experiments, respectively. (B) and (C) illustrate the tracked images of the
Mini Rover’s CoM (green curves) and orientation ($\theta$) while turning on
25∘ slope at iterations 14 and 19, respectively. The optimization iterations
14 and 19 discovered DS and single RRP gaits, respectively. (D) The snapshot
images of the rover executing the best-performing optimized gait (BO-TRRP).
The straight and curved arrows indicate the wheel’s spinning and sweeping
directions whereas the straight arrows on the wheels depict the direction of
the wheel spinning, respectively. The white and red rear wheels indicate
sweeps-out (toward the rear of the rover) and sweeps-in (toward the front of
the rover), respectively. BO-TRRP starts with the DS motion which is followed
by the single RRP of the right rear wheel. The cooperation of these two
manipulation strategies induces rover turning.

Figure 4: Mini Rover executing ML-inspired gait. (A) and (B) illustrate the
side and top view of the Mini Rover performing fast turning with ML-inspired
gait on a 25∘ slope. Straight and curved arrows depict spin and sweep
directions, respectively. Double-arrows illustrate the difference in the
appendage leg extension and retraction.

Figure 5: Time evolution of Mini Rover’s orientation using various turning
gaits on a slope of 25∘. TRRP gait achieves turning 90∘ gradually in about 2.5
minutes. Optimized gaits start turning fast initially, however, their
performance saturates as they continue turning. ML-inspired gait facilitates
effective terrain manipulation strategies to turn 90∘ in just above 4 seconds.

Figure 6: Single wheel spin and sweep experiments. (A) Illustration of solid
and fluid manipulation of granular terrain with the single wheel-leg system.
In (1) the wheel stops spinning while sweeping at a slow speed, creating more
resistive torque. In (2) the wheel spins and sweeps at a high rate, leading to
less resistive torque. (B) The apparatus used to perform spin and sweep
experiments. (C) The time evaluation of torques obtained from the experiments.
The blue curve depicts the net torque of a non-spinning wheel sweeping 90∘ at
1 rad/s. The torque gradually increases and saturates at 1 second. The orange
curve represents the wheel spinning and sweeping 90∘ at 11 rad/sand and 6
rad/s, respectively. The wheel spin yields linear-like torque where the onset
of sweep generates a peak in the torque.

Continued experiments showed little progress in discovering novel gait
patterns, only yielding faster DS motion at iteration 29. The plateauing of
gait performance is an indication of entrapment into a local minimum, a
commonly observed phenomenon in high-dimensional systems. Additionally,
despite the fact that BO provided us with important strategies for rover
turning, the process was rather slow taking around 2 minutes.

The high-dimensional actuation scheme empowers the Mini Rover to effectively
manipulate the deformable terrain that it stands on. However, the search for
optimized maneuvering gaits in the vast gait space by only using hardware
experiments is challenging and time-consuming. To facilitate faster turning
and avoid local minima, we analyzed instances of optimized gaits. In doing so,
we aim to understand how the locomotor-terrain interaction induces turning and
design more effective gaits. Closer investigation of the BO-TRRP revealed that
Mini Rover significantly benefited from the cooperation of both DS and single
RRP strategies. The DS motion first tilts the rover’s body toward the desired
orientation, creating an angular momentum. Then, the single RRP induces more
body turning by pedaling the right rear appendage while the remaining wheels
keep doing DS motion. The continued DS motion helps the rover to agitate the
material. reducing the material’s resistance. Hence. the pedaling wheel can
turn the rover more easily at each sweep-out, causing the body to pivot around
the right rear appendage.

The Mini Rover experienced unsuccessful outcomes in 15 out of 30 iterations
and 6 of these outcomes were right after the rover turned 90∘. This suggests
that the center of mass (CoM) would be shifting outside the Rover’s support
polygon as its body orientation changed from one slope to another.
Additionally, the Mini Rover’s ability to generate reaction forces by pushing
against the mound decreased a few seconds after the turning maneuver began.
The sweeping leg cannot extend deep enough to reach the mound accumulated
around the wheel. This is due to Mini Rover’s inability to properly align its
roll orientation and CoM position, to the changing slope conditions.
Furthermore, in the single RRP motion, the wheel kept spinning during
sweeping-out which limits the material resistance and creates low turning
torque.

###  III-C ML-inspired Gait Design

We used these observations and gait cues obtained from BO-TRRP to update the
gait scheme for fast turning. We first increased the DS speed of the wheels to
further boost the initial downward tilting of the Mini Rover body. Then, we
addressed the Mini Rover’s roll angle adaptation issue by extending the wheels
that are in the turning direction, facing the downward section of the slope,
and retracting the wheels, facing the upward section of the slope. In doing
so, the rover can adjust its body during and after the completion of its
maneuver. This adjustment also enabled the Mini Rover to push against a larger
granular mound and create high resistive force during sweep-out.

To further enhance the turning behavior, we varied the spin and sweep speed
parameters of the single RRP motion. We first, stopped the wheel spinning
during the sweep-out motion and decreased the speed of the sweep, performing a
solid manipulation on the substrate. In doing so, the wheel benefited more
from the solid-like response of the mound surrounding the wheel by creating
more sweeping torque in the direction of turning (Fig. 6A 1). Once the wheel
started to sweep-in, we initiated the wheel spinning at a high rate and
increased the sweep-in speed, generating a fluid manipulation (Fig. 6A 2)).
This high-rate agitation created less torque during sweep-in, hence yielding
less torque in the opposing direction to the Mini Rover turning.

After these adjustments, we performed systematic maneuvering experiments with
the new ML-inspired turning gait. The ML-inspired gait yielded a body turning
of 90∘ in just above 4 seconds, outperforming previous gaits by a large
margin. To compare the turning performance of the gaits presented in this
study, we plotted the time evolution of body orientation trajectories of the
Mini Rover in Fig. 5. While TRRP achieved gradual and slow turning behavior,
the optimized gaits initiated a fast turning, however, their performance
degraded over time. The success of the ML-inspired gait lies in the
combination of optimization and systematic human investigations. Understanding
the locomotor-terrain interaction helps design more effective gaits,
facilitating the Mini Rover to effectively manipulate its environment to
maneuver its body.

###  III-D Locomotor-Terrain Interaction

To understand and validate the existence of anisotropic torques created during
single RRP, we performed single-wheel spin and sweep experiments to capture
the resistive torques as illustrated with the images in Fig 6B. The blue curve
in Fig. 6C illustrates the experiment where the wheel spins and sweeps at a
higher rate, while the orange curve shows a non-spinning wheel and low-rate
sweeping. The slow sweeping and non-spinning wheel induces a torque profile
that is 2 times larger and 3 times longer than the fast sweeping and spinning
wheel. Therefore, controlling the spin and sweep speed of the wheel enables
creating different torque profiles. The findings in Fig. 6C support our
experimental observations with Mini Rover that the wheel creates anisotropic
torques by controlling the spin/sweep speeds. Hence, the directional torques
facilitate rapid Mini Rover turning in the desired direction.

##  IV Conclusion

In forthcoming extraterrestrial exploration missions, the ability to reach
areas characterized by steep granular slopes and perform various maneuvers
will be advantageous, serving both scientific data acquisition and mission
navigation efficiency objectives. Nevertheless, current exploratory rover
designs favor conventional wheels and passive suspension mechanisms that
primarily allow navigation on flat granular surfaces with limited mobility
over shallow granular slopes. To this end, by experimenting with a laboratory-
scale rover robot capable of applying separate forces to each wheel system, we
exploited the idea of manipulating granular terrain to achieve turning at
steep granular slopes. This paper combines both machine learning and
systematic investigation to obtain effective and robust turning gaits on
granular materials. We first performed Bayesian Optimization to discover
maneuvering gaits starting with a manually tuned turning gait. The
optimization found two terrain manipulation strategies giving us deeper
insight into how the Mini Rover achieved faster turning. Consequently, we
performed systematic investigations and experiments using these strategies and
further developed the optimized gaits. Hence, the combination of multiple
techniques facilitated rover to turn 90∘ in about 4 seconds on a 25∘ granular
slope. The Mini Rover achieved such fast turning by first differentially
spinning its left and right-hand side wheels. Secondly, it creates isotropic
torques with a single sweeping wheel that selectively spins and halts,
creating a pivoting point for the rover. Finally, the rover adapts to the
change of slope that occurs during turning by adjusting its body posture using
extendible/retractable legs.

The interaction between robots and terrain is crucial, particularly in
deformable landscapes. Enhanced comprehension of this interaction through
optimization and human intuition can contribute to the development of improved
gaits and robots. The strategies shown in this study could prove valuable in
designing maneuvering gaits for large-scale rovers and other exploration
robots that are capable of terrain manipulation.

## References

  * [1] B. Schäfer, A. Gibbesch, R. Krenn, and B. Rebele, “Planetary rover mobility simulation on soft and uneven terrain,” _Vehicle System Dynamics_ , vol. 48, no. 1, pp. 149–169, jan 2010.
  * [2] V. L. Sharpton, “Outcrops on lunar crater rims: Implications for rim construction mechanisms, ejecta volumes and excavation depths,” _Journal of Geophysical Research: Planets_ , vol. 119, no. 1, pp. 154–168, 2014.
  * [3] P. Mahanti, M. Robinson, T. Thompson, and M. Henriksen, “Small lunar craters at the apollo 16 and 17 landing sites-morphology and degradation,” _Icarus_ , vol. 299, pp. 475–501, 2018.
  * [4] G. Meirion-Griffith and M. Spenko, “A modified pressure–sinkage model for small, rigid wheels on deformable terrains,” _Journal of Terramechanics_ , vol. 48, no. 2, pp. 149–155, 2011.
  * [5] C. Li, T. Zhang, and D. I. Goldman, “A terradynamics of legged locomotion on granular media,” _Science_ , vol. 339, no. 6126, pp. 1408–1412, 2013.
  * [6] W. Reid, F. J. Pérez-Grau, A. H. Göktoğan, and S. Sukkarieh, “Actively articulated suspension for a wheel-on-leg rover operating on a martian analog surface,” in _2016 IEEE International Conference on Robotics and Automation (ICRA)_. IEEE, 2016, pp. 5596–5602.
  * [7] A. Thoesen and H. Marvi, “Planetary surface mobility and exploration: A review,” _Current Robotics Reports_ , vol. 2, no. 3, pp. 239–249, 2021.
  * [8] T. d. J. M. Sanguino, “50 years of rovers for planetary exploration: A retrospective review for future directions,” _Robotics and Autonomous Systems_ , vol. 94, pp. 172–185, 2017.
  * [9] C. Weibel, G. Valsecchi, H. Kolvenbach, and M. Hutter, “Towards legged locomotion on steep planetary terrain,” in _36th IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2023)_ , 2023.
  * [10] H. Inotsume, C. Creager, D. Wettergreen, and W. Whittaker, “Finding routes for efficient and successful slope ascent for exploration rovers,” in _The international symposium on artificial intelligence, robotics and automation in space (i-SAIRAS)_ , 2016.
  * [11] S. Lyu, W. Zhang, C. Yao, Z. Zhu, and Z. Jia, “Modeling and analysis of a reconfigurable rover for improved traversing over soft sloped terrains,” _Biomimetics_ , vol. 8, no. 1, p. 131, 2023.
  * [12] P. Arm, G. Waibel, J. Preisig, T. Tuna, R. Zhou, V. Bickel, G. Ligeza, T. Miki, F. Kehl, H. Kolvenbach _et al._ , “Scientific exploration of challenging planetary analog environments with a team of legged robots,” _Science robotics_ , vol. 8, no. 80, p. eade9548, 2023.
  * [13] S. Shrivastava, A. Karsai, Y. O. Aydin, R. Pettinger, W. Bluethmann, R. O. Ambrose, and D. I. Goldman, “Material remodeling and unconventional gaits facilitate locomotion of a robophysical rover over granular terrain,” _Science Robotics_ , vol. 5, no. 42, p. eaba3499, 2020.
  * [14] A. Karsai, D. Kerimoglu, D. Soto, S. Ha, T. Zhang, and D. I. Goldman, “Real-time remodeling of granular terrain for robot locomotion,” _Advanced Intelligent Systems_ , vol. 4, no. 12, p. 2200119, 2022.
  * [15] R. Garnett, _Bayesian optimization_. Cambridge University Press, 2023.
  * [16] S. Ha, P. Xu, Z. Tan, S. Levine, and J. Tan, “Learning to walk in the real world with minimal human effort,” _arXiv preprint arXiv:2002.08550_ , 2020.
  * [17] W. Yu, J. Tan, Y. Bai, E. Coumans, and S. Ha, “Learning fast adaptation with meta strategy optimization,” _IEEE Robotics and Automation Letters_ , vol. 5, no. 2, pp. 2950–2957, 2020.