Model Predictive Control with Gaussian-Process-Supported Dynamical Constraints for Autonomous Vehicles

[footnoteinfo]The authors acknowledge support by the research training group DFG-GRK 2297 of the German Research foundation 'Deutsche Forschungsgemeinschaft'.

First]Johanna Bethge
Second]Maik Pfefferkorn
First]Alexander Rose
Third]Jan Peters
First]Rolf Findeisen

[First]Control and Cyber-Physical Systems Laboratory, Technical University of Darmstadt, Darmstadt, Germany
[Second]Systems Theory and Automatic Control Laboratory, Otto-von-Guericke University Magdeburg, Magdeburg, Germany (e-mail<EMAIL_ADDRESS>[Third]Intelligent Autonomous Systems Laboratory, Technical University of Darmstadt, Darmstadt, Germany<EMAIL_ADDRESS>
We propose a model predictive control approach for autonomous vehicles  that exploits learned Gaussian processes for predicting human driving behavior.
The proposed approach employs the uncertainty about the GP’s prediction to achieve safety.
A multi-mode predictive control approach considers the possible intentions of the human drivers.
While the intentions are represented by different Gaussian processes, their probabilities foreseen in the observed behaviors are determined by a suitable online classification.
Intentions below a certain probability threshold are neglected to improve performance.
The proposed multi-mode model predictive control approach with Gaussian process regression support enables repeated feasibility and probabilistic constraint satisfaction with high probability.
The approach is underlined in simulation, considering real-world measurements for training the Gaussian processes.

Gaussian Process based Identification and Control; Nonlinear Model Predictive Control; Gaussian Processes; Robust Control; Intelligent Autonomous Vehicles

§ INTRODUCTION

Safe autonomous driving at intersections in mixed traffic (involving both human-driven and autonomous vehicles) remains challenging – especially as predicting human driving behavior is nontrivial due to the wide range of individual behaviors.

One way to capture the complexity of human driving behavior is to learn models of human-driven vehicles from real data.

We employ Gaussian process (GP) regression [Rasmussen and Williams, 2006] to model human-driven vehicles crossing traffic-light-free intersections.
More specifically, we use for each intention of the human-driven vehicles – turning right, turning left and going straight on – an independent GP-based model.
This enables us to learn a “standard” behavior of human drivers

in form of mean predictions of the GP and capture variations via the uncertainty measure of the GP.
The learned models enable us to predict the future position of a human-driven vehicle.

To enable the autonomous vehicle to safely cross the intersection in mixed traffic, we employ model predictive control (MPC) for decision making [Rawlings et al., 2019, Findeisen and Allgöwer, 2002].
The proposed MPC scheme uses multiple GP-based models for predicting the possible actions of the target vehicle.
Specifically, the safety distance between human-driven and autonomous vehicle is adapted based on the model uncertainty.
This uncertainty enables to capture deviations between the actual behavior of the human-driven vehicle and standard human driving behavior.
Such adaptation ensures that a minimum safety distance is maintained at all times, despite the uncertainty in the predicted behavior of the target vehicle.
However, the challenge that the intention of the human-driven vehicle is a-priori unknown remains – thereby introducing additional uncertainty.
To overcome this challenge, we predict the future positions of the human-driven vehicle for all likely maneuvers and ensure an adapted safety distance for each maneuver using a multi-mode MPC approach.
Supporting the multi-mode MPC scheme with an online classification, we estimate the probability of a specific maneuver being taken and remove unlikely modes - intentions - from the MPC problem to overcome conservatism and to improve performance.

In a similar way, [Bethge et al., 2020] used neural networks instead of Gaussian processes, which does not enable to capture uncertainty due to different driving behaviors.

[Batkovic et al., 2020] proposed an approach to avoid pedestrian obstacles exploiting multiple modes and uncertainty bounds.
Contrary to this work, the pedestrians cross the street orthogonally to the driving direction at a known location and that the safety distance is not adapted online.

While there are many articles on obstacle avoidance [Soloperto et al., 2019], communication design for autonomous vehicles [Di Vaio et al., 2019], and optimization of traffic using traffic light control [De Schutter and De Moor, 1998], these topics are not the focus of this paper.

In summary, we propose a new GP-supported control scheme for safe intersection crossing of autonomous vehicles in mixed traffic.
We show how to derive GP-based models of human-driven vehicles at (traffic-light-free) intersections and how to exploit such models in a multi-mode MPC scheme.
The approach is supported by an online classification approach to estimate the intention of the human-driven vehicle and further exploits these estimate for improved performance.

We guarantee safety, i.e., (robust) constraint satisfaction and recursive feasibility, of the proposed scheme with high probability.

In contrast to other works, we consider all (likely) intentions with corresponding, independent predictions of the behavior of the target vehicle simultaneously in our approach.
This has the potential to reduce conservatism when compared to other approaches, e.g., waiting for all target vehicles to have left the intersection.

The remainder of this paper is structured as follows. We introduce multi-mode MPC in Sec. <ref>.
In Sec. <ref>, we outline GP regression, including the derivation of the prediction models for the human-driven vehicles.
We illustrate our approach using a simulation example in Sec. <ref> and draw conclusions in Sec. <ref>.

§ LEARNING-SUPPORTED MULTI-MODE MPC

We start by outlining the autonomous system (ego) dynamics, followed by a description of the models of human-driven (target) vehicles.
We finish this section by presenting a learning-supported multi-mode MPC scheme and derive probabilistic safety guarantees.

§.§ Autonomous Ego Vehicle

We consider the nonlinear time-invariant system that describes the ego vehicle dynamics:

\begin{align}
& x(k+1) = f(x(k),\,u(k)), \label{eq:SysModel_ego:stateMapping}\\
& y(k) = C x(k). \label{eq:SysModel_ego:outputMapping}
\end{align}

Here, $x \in \mathcal{X} \subseteq \mathbb{R}^{n_x}$ is the system state, $y \in \mathcal{Y} \subseteq \mathbb{R}^{n_y}$ the output and $u \in \mathcal{U} \subseteq \mathbb{R}^{n_u}$ the control input of our autonomous ego vehicle. The state transition map is $f:\, \mathbb{R}^{n_x} \times \mathbb{R}^{n_u} \rightarrow \mathbb{R}^{n_x}$ and $h:\,\mathbb{R}^{n_x} \times \mathbb{R}^{n_u} \rightarrow \mathbb{R}^{n_y}$ denotes the output mapping.
We aim to design a predictive controller with state and output reference $x_\text{ref}$ and $y_\text{ref}$, respectively.

The reference is trackable, i.e., $x_\text{ref}$ and $y_\text{ref}$ can be exactly followed simultaneously at all time steps.

To overcome the challenge of introducing an online trajectory planner, we propose to use a (speed-assigned) path as reference [Faulwasser et al., 2009, Matschek et al., 2019].
The path $\mathcal{P}:\, s \mapsto (x_\text{ref},y_\text{ref})$ is a geometric curve, parameterized by the path parameter $s \in \R{}$, and defined independently of the time.
The path is equipped with a timing law of the form $s(k+1) = \alpha(s,\,u_s), ~ s \in [0,\, \infty), ~ u_s \in [\underline{u}_s,\,\bar{u}_s]$, where $u_s$ is a virtual control input bounded by $\underline{u}_s$ and $\bar{u}_s$.
By a suitable choice of the timing law, forward motion along the path is ensured.
The virtual input $u_s$ is used as an additional degree-of-freedom in the control scheme to actively decide about the reference timing.
The speed-assignment equips the path with a desired reference velocity, which is not strict and can be adapted by the controller for the benefit of low position errors.

§.§ Human-Driven Target Vehicle

The human-driven vehicle follows an a-priori unknown intention $i \in \mathcal{I}$ with

\begin{equation}
\mathcal{I} = \{ \text{turning right},\, \text{turning left},\, \text{straight on} \}. \label{eq:Intention_general}
\end{equation}

To model the target vehicle, we employ for each intention $i$ an independent model of the form

\begin{align}
\xi_i(k+1) = g_i(\xi_i(k)), \, i \in \mathcal{I} ~,  \label{eq:HumanModel_general}
\end{align}

with the measurable output $\xi_i \in \mathbb{R}^{n_\xi}$ of the target vehicle for a specific intention $i \in \mathcal{I}$.
Due to the wide range of individual driving behaviors of humans, it is nearly impossible to obtain an accurate model for an individual driver.
Rather, (<ref>) is designed to model the average behavior of human drivers for each intention and to provide an uncertainty measure to capture deviations of individual behaviors from the standard behavior, see Sec. <ref>.

As the intention of the target vehicle is initially unknown, we employ an online classification approach to determine in each time step $k$ a set of likely intentions $\tilde{\mathcal{I}}_k \subseteq \mathcal{I}$.
Similar to [Bethge et al., 2020], we compare the observed target vehicle behavior to prototypical paths (i.e., the average paths for each intention) to quantify the probability of a specific intention.[Any other online classification scheme can be used as long as it provides a realistic estimate for the probability of the modes being active.]

At the initial time step $k=0$, we set $\tilde{\mathcal{I}}_0 = \mathcal{I}$.

In any subsequent time step, the classification algorithm exploits a sequence $\{ \xi \}(k)$ of previous observations of the target vehicle to provide the probability of intention $i$ being active, which is $\mathrm{P}(i ~|~ \{ \xi \}(k))$.

If $\mathrm{P}(i ~|~ \{ \xi \}(k)) < \upright{\epsilon}{P}$ for a design threshold $\upright{\epsilon}{P} \in (0,1)$, we update $\tilde{\mathcal{I}}_k$ by $\tilde{\mathcal{I}}_{k+1} = \tilde{\mathcal{I}}_k \setminus \{ i \}$.

The probabilities $\mathrm{P}(i ~|~ \{ \xi \}(k))$ provided by the online classification are $\forall i\in {\mathcal{I}}\setminus \tilde{\mathcal{I}}$ upper bounds of the true probabilities and lower bounds $\forall i \in \tilde{\mathcal{I}}$.

In addition to Assumption <ref>, and for simplicity of notation, we assume in the following that $y \in \R{n_y}$ and $\xi \in \R{n_y}$ represent two-dimensional Cartesian positions $(p_x, p_y)$.

§.§ Multi-Mode Model Predictive Controller

We develop a multi-mode MPC scheme to safely navigate the ego vehicle over the intersection.
Such safe crossing requires keeping at least a minimal safety distance $\upright{d}{safe}$ to the target vehicle which depends on the velocity of the ego vehicle.
MPC is an optimal control scheme that is based on the repeated solution of a constrained finite-horizon optimal control problem (OCP) [Rawlings et al., 2019, Findeisen and Allgöwer, 2002].
It can be described by

\begin{align}
\min_{u,u_s}  & \left \{ \sum_{j=0}^{N-1} l(\hat{x}(j), \hat{y}(j), u(j),\hat{s}(j)) \! + \! E(\hat{x}(N), \hat{y}(N), \hat{s}(N)) \right \} \nonumber \\
\text{s.t.} ~ & \forall j \in \{0, \ldots, N-1 \}: \nonumber \\
& ~~ \hat{x}(j+1) = f(\hat{x}(j),\,u(j)),~ \hat{x}(0) = x(k) \\
& ~~ \hat{y}(j) =  C\cdot \hat{x}(j)\\
& ~~ \forall i \in \tilde{\mathcal{I}}_k: \hat{\xi}_i(j+1) = g_i(\hat{\xi}_i(j)),~ \hat{\xi}_i(0) = \xi_i(k) \label{eq:OCP_MMConstrLearn:GPpred}\\
% & ~~ \forall i \in \tilde{\mathcal{I}}_k: d_i(j) = \|\hat{y}(j) - \hat{\xi}_i(j)\|_2 > d_\text{ref}(j) = d_\text{safe}(j) + d_{\sigma,i}(j) \label{eq:OCP_MMConstrLearn:distance} \\
& ~~ \forall i \in \tilde{\mathcal{I}}_k: \hat{d}_i(j) \geq d_\text{ref}(j) \label{eq:OCP_MMConstrLearn:distance} \\
& ~~ \hat{x}(j) \in \mathcal{X}, \hat{y}(j) \in \mathcal{Y},\, u(j) \in \mathcal{U}, \hat{x}(N) \in \mathcal{X}_f\\
& ~~ \hat{s}(j+1) = \alpha(\hat{s}(j), u_s(j)),~ \hat{s}(0) = s(k) \\
& ~~ u_s(j) \in [\underline{u}_s,\, \bar{u}_s],\, \hat{s}(j) \in [0,\,\infty).
\end{align}

Therein, $N$ denotes the horizon length, $\hat{\cdot}$ denotes a prediction, $l(\cdot)$ and $E(\cdot)$ are the stage and terminal cost, respectively, $\mathcal{X}_f$ is the terminal region, $\hat{d}_i(j) = \lVert \hat{y}(j) - \hat{\xi}_i(j) \rVert_2$ is the predicted distance between ego and target vehicle, and $\upright{d}{ref}(j)$ is the required minimum distance between both vehicles.
The latter one is composed of the minimal safety distance $\upright{d}{safe}(j)$, depending on the ego vehicle's speed, and an additional contribution $d_{\sigma,i}(j)$ accounting for the uncertainty in the target vehicle model.
Choosing $d_{\sigma,i}(j)$ according to a high-probability confidence interval of the target vehicle model, the actual vehicle distance is with high probability larger than the minimum safety distance $\upright{d}{safe}$.

From the optimal input sequence $u^*$, which solves (<ref>), the first element is applied to the system throughout the next sampling period.

Note that the controller is supported by the online classification scheme and accounts for all remaining likely intentions of the target vehicle.

Let the above-mentioned assumptions hold.
For the predictions of the target vehicle behavior, assume the confidence interval $\mathrm{P}(\lvert \xi_i(k) - \hat{\xi}_i(k) \rvert \leq \nu \sigma_i(k)) \geq 1 - \omega$, where $\xi_i(k)$ is the actual target vehicle position, $\hat{\xi}_i(k)$ is the prediction, $\omega \in (0,1)$ is a design parameter, $\nu$ is chosen in accordance with the confidence level $1-\omega$ and $\sigma_i(k)$ is the standard deviation describing the model uncertainty for intention $i$ at time step $k$.
Then, constraint satisfaction and recursive feasibility of OCP (<ref>) at time $k$ are given with probability $\mathrm{P}_\text{cs} (k) \geq 1 - \omega - \sum_{j\in \mathcal{I}(k) \setminus \tilde{\mathcal{I}}} \mathrm{P}(j \mid \{ \xi \}(k))$.

The OCP (<ref>) is a standard OCP, satisfying standard assumptions on MPC, except for (<ref>) and (<ref>),

which introduce stochasticity due to the uncertain predictions of the target vehicle positions and the use of confidence-region-based constraint tightening.
However, regarding all other constraints of the OCP (<ref>), standard guarantees as repeated feasibility and constraint satisfaction hold [Rawlings et al., 2019].

Given the $(1-\omega)$-confidence region for the predicted target vehicle positions, $d(k) \geq \upright{d}{safe}(k)$ is ensured with probability $1-\omega$ if $\hat{d}(k) \geq \upright{d}{safe}(k) + d_{\sigma}(k)$, where $d_{\sigma}(k)$ is chosen accordingly to the confidence region, c.f., (<ref>).
Violations of the minimal safety distance constraint between the ego and the vehicle might happen for two reasons: (i) the target vehicle behavior is not captured by the chosen high-probability confidence region obtained from the target vehicle model, or (ii) the true intention of the target vehicle is wrongly removed from the set of likely intentions and thus not considered by the controller anymore.

For (i), each intention considered in the OCP (i.e., $\forall i \in \tilde{\mathcal{I}}_k$), the probability of constraint violation is by design given by $\omega$.
As all $i \in \tilde{\mathcal{I}}_k$ are considered in (<ref>), (<ref>), i.e., they are considered equally likely by the controller, the total probability of constraint violation is not greater than $\omega$ if the true intention $i_*$ is for each time step $k$ contained in $\tilde{\mathcal{I}}_k$.
If $i_* \notin \tilde{\mathcal{I}}_k$ for some time step $k$ (case (ii)), which happens with probability $\mathrm{P}(i_* ~|~ \{ \xi \}(k)) < \upright{\epsilon}{P}$ (Ass. <ref>), the constraints are anyways not automatically violated and $\mathrm{P}(i_* ~|~ \{ \xi \}(k))$ is an upper bound for the probability of constraint violation in this case.
Considering both cases, the probability for violating the constraints is given by $\mathrm{P}_{\text{cv}}(k) \leq \omega + \sum_{j\in \mathcal{I}(k) \setminus \tilde{\mathcal{I}}} \mathrm{P}(j ~|~ \{ \xi \}(k))$.
Hence,  $\mathrm{P}_\text{cs} (k) \geq 1 - \mathrm{P}_{\text{cv}}(k)$ holds.

§ GAUSSIAN PROCESS REGRESSION

We first outline the basics of Gaussian process (GP) regression, followed by an overview of hyperparameter learning.
Thereafter, predictions at uncertain inputs are considered.
We finish by showing how we use GP models to predict human driving behavior in a model predictive controller for safe intersection crossing of an autonomous vehicle.

§.§ Basics of Gaussian Process Regression

A Gaussian process $\rho(\zeta) \sim \mathcal{GP}(m(\zeta), k(\zeta, \zeta')$ extends the concept of Gaussian probability distributions to random functions in order to model an uncertain function $\varphi: \R{d} \rightarrow \R{}$.
Formally, it is defined as a collection of random variables, any finite number of which have a joint Gaussian distribution [Rasmussen and Williams, 2006].

The GP is fully specified through its prior mean function $m: \R{d} \rightarrow \R{}, \zeta \mapsto \mathrm{E}[\rho(\zeta)]$ and prior covariance function $\kappa: \R{d} \times \R{d} \rightarrow \R{}, (\zeta, \zeta') \mapsto \mathrm{Cov}[\rho(\zeta), \rho(\zeta')]$, which are design choices [Rasmussen and Williams, 2006].

The objective is to learn the underlying function $\varphi$ in terms of inferring a predictive (posterior) distribution for so far unobserved function values (test targets) $\varphi_* = \varphi(Z_*)$, where $Z_* \in \R{n_* \times d}$ is a matrix collecting $n_*$ test inputs $\zeta_*^i \in \R{d}, i = 1, \ldots, n_*$.
To this end, we require a set of (noisy) observations (training targets) $\gamma = \varphi (Z) + \varepsilon$ at training inputs $\zeta^i \in \R{d}, i = 1, \ldots, n$, collected in $Z \in \R{n \times d}$, where $\varepsilon \sim \mathcal{N}(\Zero, \upright{\sigma}{n}^2 \I)$ models white Gaussian noise with variance $\upright{\sigma}{n}^2$.
Therein, $\Zero$ denotes the zero vector and $\I$ the identity matrix.

By definition, the GP specifies the joint Gaussian distribution of training and test targets, which is the so-called joint prior distribution.

Conditioning the joint prior distribution on the training targets yields the predictive (posterior) distribution $\varphi_* \mid Z_*, Z, \gamma \sim \mathcal{N}(m^+(Z_*), \kappa^+(Z_*))$ with

\begin{align}
m^+(Z_*) & \! = \! m(Z_*) \! + \! \kappa(Z_*, Z) \kappa_{\gamma}^{-1} (\gamma  \! - \! m(Z)) \label{eq:postmean} \\
\kappa^+(Z_*) & \! = \! \kappa(Z_*, Z_*) \! + \! \upright{\sigma}{n}^2 \I \! - \! \kappa(Z_*, Z) \kappa_{\gamma}^{-1} \kappa(Z, Z_*) ~, \label{eq:postcov}
\end{align}

where $\kappa_{\gamma} = \kappa(Z, Z) + \upright{\sigma}{n}^2 \I$.
The posterior mean (<ref>) is an estimate (prediction) for the unobserved function values $\phi_*$; the posterior variances – the diagonal elements of (<ref>) – quantify the prediction uncertainty [Rasmussen and Williams, 2006].

§.§ Hyperparameter Learning

The Gaussian process is fully defined via the prior mean function $m( \cdot ; \theta)$ and the prior covariance function $\kappa( \cdot, \cdot ; \theta)$, which usually depend on a set of free parameters $\theta$, the so-called hyperparameters.
In consequence, a meaningful predictive distribution (<ref>) requires suitable hyperparameters that adapt the GP model to the underlying problem.

One way to obtain suitable hyperparameters is to learn them from the training data.
To this end, we maximize the GP's capability of explaining the training observations, which is expressed by the logarithmic marginal likelihood

\begin{equation*}
\log ( p(\gamma \mid Z, \theta) ) = -\frac{1}{2} \gamma_0\T \kappa_{\gamma}^{-1} \gamma_0 - \frac{1}{2} \log( \lvert \kappa_{\gamma} \rvert) - \frac{n}{2} \log(2 \pi),
\end{equation*}

where $p(\cdot)$ is a probability density function, $\lvert \cdot \rvert$ denotes the determinant, and $\gamma_0 = \gamma - m(Z)$.

The optimal hyperparameters are determined by maximizing the marginal likelihood w.r.t. the hyperparameters, e.g., by Newton-Raphson method.

§.§ Prediction at Uncertain Inputs

So far, we have considered predictions at deterministic test inputs $\zeta_*$.
However, in many applications as well as in the case of multiple-step predictions, the test inputs are subject to stochastic uncertainty.
To this end, we consider in the following predictions of function values $\varphi_* = \varphi(\zeta_*)$ at uncertain test inputs $\zeta_* \sim \mathcal{N}(\mu_*, \Sigma_*)$, where $\mathcal{N}(\mu_*, \Sigma_*)$ denotes a Gaussian distribution with mean vector $\mu_*$ and covariance matrix $\Sigma_*$.

The exact predictive distribution is then obtained via

\begin{equation*}
p(\varphi(\zeta_*) \mid \mu_*, \Sigma_*, Z, \gamma) = \int p(\varphi(\zeta_*) \mid \zeta_*, Z, \gamma) p(\zeta_*) d \zeta_* ~,
\end{equation*}

where $p(\varphi(\zeta_*) \mid \zeta_*, Z, \gamma)$ denotes the posterior distribution (<ref>).
Applying the law of iterated expectations (Fubini's theorem) and for a suitable choice of the covariance function $\kappa( \cdot , \cdot )$, the mean $\mu_{\varphi_*}$ and the variance $\sigma_{\varphi_*}^2$ of $p(\varphi(\zeta_*) \mid \mu_*, \Sigma_*, Z, \gamma)$ can be computed analytically as [Deisenroth, 2010]

\begin{align}
\mu_{\varphi_*} & = \mathrm{E}_{\zeta_*} [m^+(\zeta_*) \mid \mu_*, \Sigma_*] ~, \label{eq:postmean_uncertain} \\
\sigma_{\varphi_*}^2 & = \mathrm{E}_{\zeta_*}[\kappa^+(\zeta_*) \mid \mu_*, \Sigma_*] + \mathrm{Var}_{\zeta_*}[m^+(\zeta) \mid \mu_*, \Sigma_*], \label{eq:postvar_uncertain}
\end{align}

where conditioning on $\mu_*, \Sigma_*$ indicates the uncertainty of $\zeta_*$.
However, since $m^+(\cdot)$ and $\kappa^+(\cdot)$ are (in general) nonlinear functions, computing (<ref>), (<ref>) is challenging.
Furthermore, the predictive distribution $p(\varphi(\zeta_*) \! \mid \! \mu_*, \Sigma_*, Z, \gamma)$ is non-Gaussian and analytically intractable when propagating an uncertain input through the nonlinear GP model.

We approximate the exact predictive distribution by linearizing the posterior equations (<ref>), (<ref>) around $\mu_*$ and apply (<ref>) and (<ref>), yielding

\begin{align}
\tilde{\mu}_{\varphi_*} & = m^+(\mu_*), \\
\tilde{\sigma}^2_{\varphi_*} & = \kappa^+(\mu_*) + \nabla_{\zeta_*}\T m^+(\mu_*) \Sigma_* \nabla_{\zeta_*} m^+(\mu_*).
\end{align}

Propagating the uncertain input $\zeta_*$ through the linearized posterior GP model results then in an analytically tractable Gaussian distribution $\varphi_* \sim \mathcal{N}(\tilde{\mu}_{\varphi_*},  \tilde{\sigma}^2_{\varphi_*})$ [Deisenroth, 2010, Hewing et al., 2017].

§.§ Gaussian Process Models of Human Drivers for MPC

We aim at using GP regression to build models of human driving behavior for intersection crossing.
To this end, we rely on demonstration data from human-driven vehicles turning right, turning left and going straight on over it.
Those three modes of the human-driven vehicles are considered separately, such that we compute an independent GP model for each intention [Nguyen-Tuong et al., 2008, Nguyen-Tuong et al., 2009].
In the following, we show the general derivation of such a model for a single mode.

The objective is to derive a GP-based model for predicting the vehicle position $(p_x(t_i), p_y(t_i))$ with $ i = 1, \ldots, N$ along the MPC horizon given an initial position $(p_x(t_0), p_y(t_0))$.

We rely on a data set $\{ (\zeta_i = (p_x, p_y)_i, \gamma_{x,i} = v_{x,i}, \gamma_{y,i} = v_{y,i}  \mid i = 1, \ldots, n \}$ of two-dimensional Cartesian vehicle positions $(p_x, p_y)$ and corresponding directed velocities $(v_x, v_y)$.
Based thereon, we train two independent GPs to learn the velocity profiles $v_x = \chi_x(p_x, p_y)$ and $v_y = \chi_y(p_x, p_y)$.
The resulting posterior means represent the average velocities while the posterior variances capture variations in the demonstrations.
Given the two GP models of the directed velocities, we iterate the following procedure along the horizon for $i = 0, \ldots, N-1$:

* Given position $(p_{x,i}, p_{y,i}) \sim \mathcal{N}( \mu_{*,i}, \Sigma_{*,i})$, compute the predictions $v_{x,i} \sim \mathcal{N}(\tilde{\mu}_{x,i}, \tilde{\sigma}_{x,i}^2)$ and $v_{y,i} \sim \mathcal{N}(\tilde{\mu}_{y,i}, \tilde{\sigma}_{y,i}^2))$ according to (<ref>).[Note that for $i=0$ we have $\mu_{*,i} = \begin{bmatrix} p_{x,i} & p_{y,i} \end{bmatrix}\T$ and $\Sigma_{*, i} = \Zero$.]

* Compute the successor position, using the forward Euler integration scheme with sampling time $\upright{T}{s}$, according to $p_{x,i+1} = p_{x,i} + \upright{T}{s} v_{x,i}$, $p_{y,i+1} = p_{y,i} + \upright{T}{s} v_{y,i}$. Then, $(p_{x,i+1}, p_{y,i+1}) \sim \mathcal{N} \big( \mu_{*,i} + \upright{T}{s} \begin{bmatrix} \tilde{\mu}_{x,i} & \tilde{\mu}_{y,i} \end{bmatrix}\T, \Sigma_{*,i} + \upright{T}{s}^2 \mathrm{diag} \big( \begin{bmatrix} \tilde{\sigma}_{x,i}^2 & \tilde{\sigma}_{y,i}^2 \end{bmatrix} \big) \big)$.[Here, we neglect correlations between $(p_{x,i}, p_{y,i})$ and $v_{x,i}$ and $v_{y,i}$, respectively. As the GPs are independent by design, the covariance matrix of the predicted velocity is diagonal, denoted by $\mathrm{diag(\cdot)}$.]

* If $i < N-1$, increase $i$ by one and go to step (i).

To compute confidence bounds on the predicted positions, we employ Chebyshev's inequality, yielding

\begin{align}
\begin{split}
P( \lvert p_{x,i} - \mu_{x,*,i} \rvert \leq \nu \sigma_{x,*,i}) \geq 1 - \omega ~, \\
P( \lvert p_{y,i} - \mu_{y,*,i} \rvert \leq \nu \sigma_{y,*,i}) \geq 1 - \omega ~,
\end{split}
\label{eq:chebyshev}%
\end{align}

with maximum probability of constraint violation $\omega \in (0,1)$ and $\nu = \sqrt{\omega^{-1}}$, see [Olkin and Pratt, 1958].
Furthermore, the mean and variance have to be known exactly.
Chebyshev's inequality holds independently of the particular distribution law and tends to conservative estimates of the confidence intervals.
However, we exploit these properties to compensate for distribution mismatches due to the approximated multiple step predictions as well as to compensate for (accumulated) errors in the predicted means and variances.
Thereby, we achieve a certain robustness although it remains open to establish guarantees.

§ SIMULATION EXAMPLE

We start by deriving the GP-based models of human-driving behavior, including a validation of the derived confidence bounds.
Thereafter, we outline the ego vehicle model used in this article and finish by presenting closed-loop simulation results.

§.§ Modeling of Human Driving Behavior

Trajectories of human-driven vehicles crossing the intersection. Left-turning vehicles approach the intersection on a separate lane. Black lines indicate the average paths.

Given $(p_x, p_y)$-trajectories, with sampling time $\upright{T}{s} = \SI{0.04}{\s}$, of human-driven vehicles crossing an intersection (Fig.  <ref>), we start by estimating the corresponding vehicle velocities using finite differences.
We define for each intention and each velocity profile a zero-mean GP prior with squared exponential covariance function ([Rasmussen and Williams, 2006]), and train them on uniformly selected subsets of the data of the respective intention.
The selection of a smaller number of active training data points (compared to the number of available data points) reduces the computational complexity of the training as well as of the model evaluations.
We finally exploit the learned GPs and derive the models of human-driven vehicles as described in Sec. <ref>.

Having derived these models, we validate them as well as the confidence bounds computed according to Chebyshev's inequality for each intention.
To this end, we choose a horizon length $N$ and assemble from each trajectory, belonging to the respective intention, all horizons that are completely contained in it.
We subsequently evaluate the models along each of the respective horizons and compute for each horizon (exemplarily) the 99% confidence regions according to (<ref>).
The confidence level is in turn estimated empirically as the relative frequency of trajectories, for which at least 99% of the corresponding horizons are contained in the appendant confidence regions.
As increasing horizon lengths usually result in a stronger deterioration of the multiple-step prediction quality, we show the empirical confidence levels dependent on the horizon length and, exemplarily, for the right-turning vehicles in Tab. <ref>.
In accordance with the validation results for all modes, we select – for the MPC – a horizon length of $N=40$, for which the designed confidence bounds are exact.

Empirical confidence levels of the models for human-driven, right-turning vehicles based on 114 trajectories.

$N$     38     39     40     41     42     43
$p_x$-confidence     1     1     1     0.9912     0.9912     0.9912

$p_y$-confidence     1     1     1     0.9912     0.9912     0.9912

Total confidence     1     1     1     0.9825     0.9825     0.9825

§.§ Ego Vehicle Model

The ego vehicle's states

\begin{equation}
x = [p_x,\, p_y, \,v,\, \psi,\, \delta,\, v_s,\, s]^T
\end{equation}

contain the Cartesian x- and y-positions $p_x$ and $p_y$ respectively, the vehicle's speed $v$, its heading $\psi$, the steering rate $\delta$, the path velocity $v_s$ and the path parameter $s$, see [Kong et al., 2015].

The continuous-time dynamics are given by

\begin{equation}
\dot{x} = [v \cos{\psi},\, v \sin{\psi},\, u_a, \, \frac{v}{L} \tan \delta,\,u_\delta,\, s,\, u_s]^T
\end{equation}

with vehicle length $L$ and control inputs

\begin{equation}
u = [u_a,\, u_\delta, \,  u_s]^T,
\end{equation}

where $u_a$ is the acceleration, $u_\delta$ is the steering angle and $u_s$ denotes the virtual input for the path parameter.
The time-discrete state transition map (<ref>) is therefrom derived via discretization with sampling time $\upright{T}{s} = \SI{0.04}{\s}$.
The output in  (<ref>) is $y =  [p_x,\,p_y]^T$.

§.§ Multi-Mode MPC for Safe Intersection Crossing

Scenario of the simulation example. Green points indicate the vehicle positions at the time at which the actual intention of the target vehicle was identified.

Online classification results. Modes with probability below $\upright{\epsilon}{P}$ are considered inactive.

To illustrate our approach, we select a specific trajectory from the data set to represent the target vehicle.
We depict the situation in Fig. <ref>.
The target vehicle starts east and takes a right turn, while the ego vehicle starts south and drives straight on over the intersection.

We employ the control scheme presented in Sec. <ref> to safely cross the intersection with the ego vehicle.

Initially, the intention of the target vehicle is unknown to the ego vehicle.
Consequently, the ego vehicle needs to  consider all possible intentions of the target vehicle.

However, as the target vehicle progresses, we continuously employ the online classification algorithm, c.f., Sec. <ref>, with $\epsilon_{\mathrm{P}} = 0.15$, to estimate the actual intention of the target vehicle.
In Fig. <ref>, we show the result of the classification algorithm at each sampling time point.

Since vehicles that turn left already start on another lane, this intention immediately falls below the threshold $\epsilon_{\mathrm{P}}$ and is thus considered inactive.
However, going straight on or turning right are still possible.
After the target vehicle starts turning right, i.e., after about $\SI{3.4}{\second}$, we can identify the intention of the target vehicle.
The corresponding positions of the vehicles at that time are marked by green points in Fig. <ref>.

Actual velocity profiles of the ego and target vehicle and GP-based prediction of the target vehicle speed. Additionally, the velocity reference for the ego vehicle is shown.

In order to keep the desired minimal safety distance[The desired minimal safety distance is dependent on the ego vehicle speed and chosen as the distance that is covered by the ego vehicle in $\SI{1}{\s}$.], the ego vehicle decelerates such that the target vehicle can pass the intersection first (Fig. <ref>).
Afterwards, the ego vehicle accelerates again while keeping the desired safety distance robustly.
Note that the velocity of the target vehicle is noisy since it is based on an observed trajectory in the data set and thus includes measurement noise.

Finally, we depict in Fig. <ref> the distance between target and ego vehicle during intersection crossing in more detail.

Therein, the actual distance (red line) between ego and target vehicle and the velocity-dependent, minimal desired safety distance $\upright{d}{safe}$ (black line) are shown.
Moreover, for every $20^{\text{th}}$ iteration, we show the predicted minimal safety distances (black dotted lines) according to the planned ego vehicle speeds, the predicted distances (blue dashed lines) and the tightening (blue dotted lines) in (<ref>) by $d_{\sigma,i}$ over the controller's prediction horizon.

We observe that the predicted distances always fulfill the corresponding tightened distance constraints, indicating that the ego vehicle always keeps enough space to account for possible prediction errors of the target vehicle's behavior.

Actual and velocity-dependent, minimal safety distance between ego and target vehicle. At selected times, the minimal safety distances corresponding to the planned ego vehicle speeds ($\hat{d}_{\mathrm{safe}}$), the predicted distances ($\hat{d}$) and the tightened distance constraints ($d_{\mathrm{safe}}+d_{\sigma}$) are shown over the corresponding prediction horizon of length $N$.

§ CONCLUSIONS

We have presented a multi-mode MPC scheme for safe intersection crossing of autonomous vehicles in the presence of human-driven vehicles.
The proposed MPC scheme is supported by learned, GP-based models of human drivers and an online classification algorithm to determine the human's a-priori unknown intention.
We have described the derivation of such GP-based models as well as their embedding in the multi-mode prediction approach used in the MPC scheme.
Furthermore, we have derived probabilistic safety guarantees for the proposed scheme, combining the multi-mode predictions and GP-based confidence sets of human driving behavior.
The proposed approach has been illustrated in a simulation example using real-world traffic data.

Future steps will focus on computationally efficient implementations of the GP evaluations to achieve real-time capability of the proposed approach.
Furthermore, we aim to extend the presented approach to consider multiple human-driven vehicles crossing the intersection and investigate the transferability to other arbitrary intersections.

Finally, we will focus on employing more sophisticated modeling approaches such as structured output GPs or intention-driven dynamics models based on GPs.

[Batkovic et al., 2020]
Batkovic, I., Rosolia, U., Zanon, M., and Falcone, P. (2020).
A robust scenario MPC approach for uncertain multi-modal obstacles.
IEEE Control Systems Letters, 5, 947–952.

[Bethge et al., 2020]
Bethge, J., Morabito, B., Rewald, H., Ahsan, A., Sorgatz, S., and Findeisen, R.
Modelling human driving behavior for constrained model predictive
control in mixed traffic at intersections.
IFAC-PapersOnLine, 53(2), 14356–14362.

[De Schutter and De Moor, 1998]
De Schutter, B. and De Moor, B. (1998).
Optimal traffic light control for a single intersection.
European Journal of Control, 4(3), 260–276.

[Deisenroth, 2010]
Deisenroth, M.P. (2010).
Efficient Reinforcement Learning using Gaussian processes.
Ph.D. thesis, Intelligent Sensor-Actuator-Systems Laboratory,
Karlsruhe Institute of Technology.

[Di Vaio et al., 2019]
Di Vaio, M., Falcone, P., Hult, R., Petrillo, A., Salvi, A., and Santini, S.
Design and experimental validation of a distributed interaction
protocol for connected autonomous vehicles at a road intersection.
IEEE Transactions on Vehicular Technology, 68(10), 9451–9465.

[Faulwasser et al., 2009]
Faulwasser, T., Kern, B., and Findeisen, R. (2009).
Model predictive path-following for constrained nonlinear systems.
In Proc. IEEE Conf. Dec. Cont., 8642–8647.

[Findeisen and Allgöwer, 2002]
Findeisen, R. and Allgöwer, F. (2002).
An introduction to nonlinear model predictive control.
In 21st Benelux Meeting on Systems and Control, volume 11,

[Hewing et al., 2017]
Hewing, L., Kabzan, J., and Zeilinger, M.N. (2017).
Cautious model predictive control using Gaussian process

[Kong et al., 2015]
Kong, J., Pfeiffer, M., Schildbach, G., and Borrelli, F. (2015).
Kinematic and dynamic vehicle models for autonomous driving control
In IEEE Intelligent Vehicles Symposium, 1094–1099.

[Matschek et al., 2019]
Matschek, J., Bäthge, T., Faulwasser, T., and Findeisen, R. (2019).
Nonlinear Predictive Control for Trajectory Tracking and Path
Following: An Introductino and Perspective, 169–198.
Springer International Publishing.

[Nguyen-Tuong et al., 2008]
Nguyen-Tuong, D., Seeger, M., and Peters, J. (2008).
Local Gaussian process regression for real time online model
In Advances in Neural Information Processing Systems,

[Nguyen-Tuong et al., 2009]
Nguyen-Tuong, D., Seeger, M., and Peters, J. (2009).
Model learning with local Gaussian regression.
Advanced Robotics, 23, 2015–2034.

[Olkin and Pratt, 1958]
Olkin, I. and Pratt, J. (1958).
A multivariate Tchebycheff inequality.
The Annals of Mathematical Statistics, 29, 226–234.

[Rasmussen and Williams, 2006]
Rasmussen, C.E. and Williams, C.K.I. (2006).
Gaussian Processes for Machine Learning.
The MIT Press.

[Rawlings et al., 2019]
Rawlings, J.B., Mayne, D.Q., and Diehl, M.M. (2019).
Model Predictive Control: Theory, Computation, and Design.
Nob Hill Publishing, LLC, 2 edition.

[Soloperto et al., 2019]
Soloperto, R., Köhler, J., Allgöwer, F., and Müller, M.A. (2019).
Collision avoidance for uncertain nonlinear systems with moving
obstacles using robust model predictive control.
In 2019 18th European Control Conference (ECC), 811–817.


# A Low-Power Optical Electron Switch

Wayne Cheng-Wei Huang  email<EMAIL_ADDRESS>Roger Bach  email:
<EMAIL_ADDRESS>Peter Beierle  email<EMAIL_ADDRESS>Herman Batelaan
email<EMAIL_ADDRESS>Department of Physics and Astronomy, University of
Nebraska-Lincoln, Lincoln, Nebraska 68588, USA

###### Abstract

An electron beam is deflected when it passes over a silicon nitride surface,
if the surface is illuminated by a low-power continuous-wave diode laser. A
deflection angle of up-to $1.2\,\textrm{mrad}$ is achieved for an electron
beam of $29\,\mu\textrm{rad}$ divergence. A mechanical beam-stop is used to
demonstrate that the effect can act as an optical electron switch with a rise
and fall time of $6\,\mu\textrm{s}$. Such a switch provides an alternative
means to control electron beams, which may be useful in electron lithography
and microscopy.

The motion of electron beams is controlled in technologies such as electron
lithography, microscopy, and diffractometry, in which the use of electric and
magnetic fields to focus and steer beams are proven techniques. The control of
electron motion with laser fields is also possible with the ponderomotive
potential Boot ; Sciaini . In principle, such a technique offers the
interesting possibility that no electrical components or other hardware needs
to be placed in the vicinity of the electron beam. In addition, using the
spatial control at optical wavelength scales, electron-optical elements can be
realized Freimund ; Bucksbaum . However, this optical control requires light
intensities of $10^{14}\,\textrm{W/m${}^{2}$}$. In this paper we report on an
optical electron switch that makes use of a small surface and a low power
laser. Although some material is placed in the vicinity of the electron beam,
no electrical feedthroughs are needed. Moreover, the required laser intensity
is reduced by ten orders of magnitude as compared to techniques based on the
direct interaction between laser light and electrons.

In this letter, it is shown that an electron beam that passes by a surface
deflects when the surface is illuminated by a low-power continuous-wave diode
laser. While searching for a nano-scale related effect at grazing incidence, a
significant and unexpected beam deflection was observed. Deflection angles
reached value of up-to $1.2\,\textrm{mrad}$. At a distance of
$20\,\textrm{cm}$ downstream from the interaction region, this translates to a
beam displacement of $240\,\mu\textrm{m}$. A beam-stop can placed in the
deflected electron beam, so that chopping the laser light results in complete
switching the electron beam on and off. A maximum switching rate of
$10^{5}\,\textrm{Hz}$ is established. Such an optically controlled electron
switch may find applications in electron lithography MAPPER , coherent beam
splitting or provide an alternative route to STM-based techniques that probe
optically induced near-fields Becker ; Elsaesser .

Figure 1: Setup of the low-power optical electron switch. An electron beam
passing close to a surface is deflected by an angle $\theta$ when the surface
is illuminated with a laser beam. The illumination is turned on and off with a
mechanical chopper. (For a detailed descriptions see text.)

Figure 2: Electron counts as a function of time as the laser is switching on
and off. Both chopper data (red dots) at $818\,\textrm{Hz}$ and AOM data
(black dots) at $1000\,\textrm{Hz}$ are shown. Top-left inset: A time-averaged
image shows the initial and deflected electron beam. A semi-transparent
rectangle is added to depict a movable electron beam-stop. Top-right inset:
The deflection magnitude $\theta$ is plotted as a function of the chopping
frequency $f$. The estimated maximum chopping frequency according our
heuristic model, $f_{max}\simeq 2\,\textrm{MHz}$, is also drawn (blue line)
for comparison. The red dots are data collected with a mechanical chopper and
the black dots with a AOM.

A schematic of the experimental setup is shown in figure 1. In our experiment,
the electron beam is emitted from a thermionic source with a beam energy of
$3.98\,\textrm{keV}$. After passing through two collimation slits of width
$5\,\mu\textrm{m}$ and $2\,\mu\textrm{m}$ and separation $24\,\textrm{cm}$,
the beam divergence is reduced to $29\,\mu\textrm{rad}$. At $6\,\textrm{cm}$
after the second collimation slit, a surface is placed parallel to the beam
path. Three different surfaces were tested. The first is a metallic-coated
surface with nano-scale grooves Savas1 ; Savas2 . The other two are a flat
amorphous aluminum (with aluminum oxide on surface) and a silicon nitride
surface. All three surfaces resulted in electron beam deflection.

Continuous-wave diode lasers with maximum powers of $1\,\textrm{mW}$,
$40\,\textrm{mW}$, and $5\,\textrm{mW}$ and wavelengths of $532\,\textrm{nm}$,
$685\,\textrm{nm}$, or $800\,\textrm{nm}$, respectively, were focused by a
cylindrical lens onto the first surface. The other two surfaces were tested
with $800\,\textrm{nm}$ light. The height of the laser beam and electron beam
were matched by using an edge of the surface structure to block part of these
beams. The focal distance is $25\,\textrm{cm}$, and the focused laser beam
waist was about $280\,\mu\textrm{m}\times 1\,\textrm{mm}$ (FWHM). The waist of
the light beam was determined by scanning the intensity profile in situ with a
surface edge. A $10\,\mu\textrm{m}$ wide electron beam passes at a distance of
nominally $20\,\mu\textrm{m}$ from the vertically mounted metallic surface.
Micrometer stages were used to control the horizontal angle (in the
$xz$-plane) as well as the vertical and horizontal travel of the surface.
Downstream from the metallic surface, the electron beam passes through a
parallel plate electrical deflector that aligns the beam with an electrostatic
quadrupole lens. This lens magnifies the electron beam image in the horizontal
direction by a factor of $65$. A chevron multi-channel plate (MCP) detector is
placed $26\,\textrm{cm}$ downstream from the surface. A phosphorescent screen
follows the multichannel plates and a camera is used to record the beam
profile. Amplifiers and discriminators are used in conjunction with a data
acquisition board to record the electron counts as a function of time.
Gaussian fits of the beam profiles are used to find the center positions and
the deflection angles.

The vacuum pressure is about $1.5\times{10}^{-7}\,\textrm{Torr}$. By chopping
the laser, the electron beam image on the MCP detector switches between two
positions. The time-averaged image displays two nearly identical electron beam
images that are horizontally displaced from each other (figure 2, top-left
inset). An electron beam-stop, depicted in the top-left inset of figure 2 as a
semi-transparent rectangle, is added. The electron counts are recorded as a
function of time (figure 2). The dynamical response of the effect and also the
finite electron beam size will limit the rise and fall time. To explore the
limit of the response speed, an $40\,\textrm{MHz}$ acousto-optical modulator
(AOM) was used [IntraAction Corp. AOM-40N]. The amplitude of the acoustic wave
was modulated from $1\,\textrm{Hz}$ to $3\,\textrm{MHz}$. The inset of figure
2 shows the scaling of the deflection magnitude of the electron beam with the
AOM and the chopping frequency. Overall, the deflection magnitude stays
constant for frequencies from $10^{2}\,\textrm{Hz}$ to $3\times
10^{5}\,\textrm{Hz}$. When the chopping frequencies are below
$10^{2}\,\textrm{Hz}$, the deflection magnitude becomes larger. When the AOM
frequency increases above $2\times 10^{5}\,\textrm{Hz}$, the deflection
magnitude decreases to zero.

Figure 3: Distance dependence of the optical electron switch. As the surface
is displaced, the distance $x$ between the surface and the electron beam is
increased (inset). The optical electron switch turns completely on and off up-
to a distance of $200\,\mu\textrm{m}$.

Figure 4: Beam deflection. Left: The measured deflection magnitude is given as
a function of $y$ (black dots). A measurement of the deflection direction is
made at three locations (red circles). The values including sign are indicated
(red crosses). Reversals of deflection sign may be explained by our heuristic
model (blue line) of light-induced surface charge redistribution. Right: A
schematics of electron trajectories (black lines) and surface charge density
(color-coded) is shown (See text for model description). Red represents
positive charge density. Dark blue represents negative charge density. The red
dots indicate the final positions of the electron beams. The interaction
between the electron beams and the surface charges is attractive in the middle
and repulsive at the sides.

In figure 3, deflection larger than the beam divergence is observed to a
distance of up-to $200\,\mu\textrm{m}$. The Rayleigh length of the focused
laser beam is roughly $5\,\textrm{cm}$ for an initial beam width of
$1\,\textrm{mm}$ and a the focal length of $25\,\textrm{cm}$). This is much
larger than $200\,\mu\textrm{m}$, thus the illumination of the surface is
unchanged as the surface is moved with respect to the electron beam. This
measurement indicates that the deflection originates from the electron-surface
interaction rather than the direct electron-laser interaction. As the
interaction range is of the order of $200\,\mu\textrm{m}$, the interacting
part of the surface is expected to have a length scale of that order of
magnitude.

When moving the cylindrical lens in the vertical direction, the laser light
crosses the electron beam at different heights. The deflection angle shown in
figure 4 changes its sign as the light crosses through the electron beam. This
was determined by placing the beam-stop in such a way that the electron beam
is half-blocked when the laser is off. If the laser light deflects the beam
towards the beam-stop, the electron count rate decreases when the light is on.
If the laser light deflect the beam away from the beam-stop, the electron
count rate increases when the light is on. The magnitude of the deflection is
determined by fitting a double Gaussian to the camera image taken with the
beam-stop removed. We observed that as the cylindrical lens is moved
vertically and the light approaches the electron beam from one end, the
electron beam first is deflected away from the surface, then attracted towards
the surface, and back to deflected away again. No significant dependence is
observed for surface tilt angles or laser polarizations.

Measurements have also been performed on different material surfaces such as
uncoated flat silicon-nitride membrane and bulk aluminum. A repulsive
deflection of up-to $1.2\,\textrm{mrad}$ is observed with the silicon-nitride
membrane, while at the aluminum surface some small attractive deflection is
observed. Given that the deflection effect works with different laser
wavelengths at low power, and it can occur at different material surface, we
conclude that an optical electron switch based on such a effect is robust.

In the cases of uncoated silicon-nitride surface, the deflection shows only
one sign unlike that observed with the nano-structured metallic-coated
surface. This suggests that the deflection mechanism could be complex and
involve a host of phenomena including laser heating, plasmon or phonon
excitation, and surface-charge redistribution. Nevertheless, a simplistic
model is constructed to illuminate some features of our experimental data
shown in figure 4. Focused by the cylindrical lens, the laser intensity
profile on the metallic-coated surface can be approximated with an elliptical
Gaussian,

$I(y,z)=I_{0}\times exp{\left[-\left(\frac{y}{\Delta
y}\right)^{2}-\left(\frac{z}{\Delta z}\right)^{2}\right]}.$ (1)

where $\Delta y=170\,\mu\textrm{m}$ and $\Delta z=0.6\,\textrm{mm}$
(corresponding to FWHM of $280\,\mu\textrm{m}\times 1\,\textrm{mm}$). The
maximum intensity is $I_{0}=P_{0}/(\pi\Delta y\Delta z)=1.6\times
10^{4}\,\textrm{W/m${}^{2}$}$ and the laser wavelength is
$\lambda=800\,\textrm{nm}$. The intensity gradient of the laser light can
exert a pondermotive force111 When a light wave propagates in the solid, the
phase relationship between the electric field and the magnetic field is a
complex function of the material properties. For a simplistic model, here we
assume that the electric field and the magnetic field are in phase. on the
electrons in a thin surface layer,

$\mathbf{F}_{p}=-\frac{e^{2}\lambda^{2}}{8\pi^{2}m_{e}c^{3}\epsilon_{0}}\mathbf{\nabla}I.$
(2)

If we assume a linear restoring force for the electron,

$\mathbf{F}_{r}=-\alpha\mathbf{d},$ (3)

where $\alpha$ is a fitting parameter and $\mathbf{d}$ is the displacement,
the induced volume dipole moment can be determined,

$\mathbf{P}=-n_{0}e\mathbf{d}=\frac{1}{\alpha}\frac{n_{0}e^{3}\lambda^{2}}{8\pi^{2}m_{e}c^{3}\epsilon_{0}}\mathbf{\nabla}I,$
(4)

where $n_{0}=5.9\times 10^{28}\,\textrm{m${}^{-3}$}$ is the free electron
density of gold Ashcroft . The volume charge distribution
$\rho_{\textrm{net}}$ is calculated according to
$\rho_{\textrm{net}}=-\nabla\cdot\mathbf{P}$. Assuming that the pondermotive
force is effective through a depth of $\delta_{\textrm{eff}}=1\,\textrm{nm}$
into the metal, the effective surface charge distribution on the metallic-
coated surface can be obtained,

$\sigma_{\textrm{eff}}=\rho_{\textrm{net}}\delta_{\textrm{eff}}=-\frac{1}{\alpha}\frac{n_{0}e^{3}\lambda^{2}\delta_{\textrm{eff}}}{8\pi^{2}m_{e}c^{3}\epsilon_{0}}\mathbf{\nabla}^{2}I$
(5)

The distance between the free electron beam and the surface is
$20\,\mu\textrm{m}$, which is much smaller than the length scale of the
surface charge distribution. Thus, close to the surface the free electron beam
may experience a electric field approximated by
$\mathbf{E}\simeq\sigma_{\textrm{eff}}/2\epsilon_{0}(-\hat{\mathbf{x}})$.
Assuming that the velocity is constant in the $z$-direction because of the
high kinetic energy $K_{0}=3.98\,\textrm{keV}$ in the incoming $z$-direction,
the deflection angle of the electron beam along the $x$-axis is estimated by

$\theta=\frac{\Delta
v_{x}}{v_{0}}=\frac{e}{4\epsilon_{0}K_{0}}\int_{-\infty}^{+\infty}\sigma_{\textrm{eff}}\,dz.$
(6)

After integration, the above equation becomes

$\theta=\theta_{0}\left[1-2\left(\frac{y}{\Delta
y}\right)^{2}\right]e^{-\left(y/\Delta y\right)^{2}},$ (7)

where

$\begin{split}\theta_{0}&\equiv\frac{\sqrt{\pi}eE_{0}\Delta z}{K_{0}},\\\
E_{0}&\equiv\frac{\sigma_{0}}{2\epsilon_{0}},\\\
\sigma_{0}&\equiv\frac{1}{\alpha}\frac{n_{0}e^{3}\lambda^{2}\delta_{\textrm{eff}}I_{0}}{8\pi^{3}\epsilon_{0}m_{e}c^{3}\Delta
y^{2}}.\end{split}$ (8)

The result of this simplistic model is compared with the experimental data in
figure 4. The fitting parameter is determined to be $\alpha\simeq 1.52\times
10^{-16}\,\textrm{N/m}$. The linear restoring force (equation (3)) produces a
harmonic motion with fundamental frequency $\omega_{0}=\sqrt{\alpha/m_{e}}$.
As a damped harmonic oscillator, the frequency response of the electron switch
as shown in the inset of figure 2 is limited to $f_{max}=\omega_{0}/2\pi\simeq
2\,\textrm{MHz}$.

Despite some qualitative agreements, this crude model does not explain many
details, such as the physical origin of linear restoring force (equation (3)),
the increase of the deflection magnitude at very low frequency (figure 2), the
asymmetric side-peak heights (figure 4), and the fact that sign reversal of
deflection direction is only present on the nano-structured metallic-coated
surface but not on the silicon nitride surface. This heuristic model serves to
draw attention to these features of our experimental data.

In summary, when a material surface is placed near an electron beam, a
deflection of the electron beam occurs as the surface is illuminated by a low-
power laser. Thus, the combination of a material surface, a low-power laser,
and a chopping device can make a low-power optical electron switch. Such an
optical electron switch may be used for electron beam control in electron
lithography and in electron microscopy.

The qualitative agreement between our model and the experimental data may be
fortuitous, but it suggests that the deflection mechanism is consistent with a
surface-charge redistribution that is driven by a mechanism that depends on
the intensity gradient of the laser light.

An implication of this work is that instead of using one laser beam for the
optical electron switch, one can use multiple laser beams to form spatial-
temporal controlled structures on a material surface. The near field of the
surface charge may mimic the pattern of the light, and electron matter waves
could be coherently controlled in this manner analogous to the Kapitza-Dirac
effect or temporal lensing Batelaan ; Baum , but without the need for high
laser intensity. Finally, we speculate that the combination of laser pulses
and nano-fabricated structures will make femtosecond manipulation of free
electrons accessible at low intensities Becker ; Zewail ; Muskens .

We gratefully acknowledge the funding support from NSF Grant No. 0969506.

## References

  * (1) H. A. H. Boot and R. B. R. .S. Harvie, _Charged particles in a non-uniform radio-frequency field_ , Nature 180, 1187 (1957).
  * (2) G. Sciaini and R. J. Dwayne Miller, _Femtosecond electron diffraction: heralding the era of atomically resolved dynamics_ , Rep. Prog. Phys. 74, 096101 (2011)
  * (3) D. Freimund, K. Aflatooni, H. Batelaan, _Observation of the Kapitza-Dirac effect_ , Nature 413, 142 (2001).
  * (4) P. H. Bucksbaum, D. W. Schumacher, and M. Bashkansky, _High intensity Kapitza-Dirac effect_ , Phys. Rev. Lett. 61, 1182 (1988).
  * (5) MAPPER Lithography, http://www.mapperlithography.com/
  * (6) C. Ropers, D. R. Solli, C. P. Schulz, C. Lienau, and T. Elsaesser, _Localized Multiphoton Emission of Femtosecond Electron Pulses from Metal Nanotips_ , Phys. Rev. Lett. 98, 043907 (2007).
  * (7) M. Becker, W. Huang, H. Batelaan, E. J. Smythe, and F. Capasso, _Measurement of the ultrafast emporal response of a plasmonic antenna_ , Ann. Phys. (Berlin) 525, A21 (2013).
  * (8) T. A. Savas, S. N. Shah, M. L. Schattenburg, J. M. Carter, and H. I. Smith, _Achromatic interferometric lithography for 100?nm?period gratings and grids_ , J. Vac. Sci. Technol. B 13, 2732 (1995).
  * (9) T. A. Savas, M. L. Schattenburg, J. M. Carter, and H. I. Smith, _Large?area achromatic interferometric lithography for 100 nm period gratings and grids_ , J. Vac. Sci. Technol. B 14, 4167 (1996).
  * (10) N. W. Ashcroft and N. D. Mermin, _Solid State Physics_ , (Holt, Rinehart, and Winston, New York), p.5.
  * (11) H. Batelaan, _Colloquium: Illuminating the Kapitza-Dirac effect with electron matter optics_ , Rev. Mod. Phys. 79, 929 (2007).
  * (12) P. Baum and A. H. Zewail, _Attosecond electron pulses for 4D diffraction and microscopy_ , Proc. Natl. Acad. Sci. 104, 18409 (2007).
  * (13) V. A. Lobastov, R. Srinivasan, and A. H. Zewail, _Four-dimensional ultrafast electron microscopy_ , Proc. Natl. Acad. Sci. 102, 7069 (2005).
  * (14) O. Muskens, _Towards nanoantenna electron switches_ , Ann. Phys. (Berlin) 525, A21 (2013).