# DeepMPCVS: Deep Model Predictive Control for Visual Servoing

Pushkal Katara Robotics Research Center, IIIT Hyderabad, India  Y V S Harish
Robotics Research Center, IIIT Hyderabad, India  Harit Pandya University Of
Lincoln, UK  Abhinav Gupta Robotics Research Center, IIIT Hyderabad, India
AadilMehdi Sanchawala Robotics Research Center, IIIT Hyderabad, India  Gourav
Kumar TCS Innovation Labs, Kolkata, India  Brojeshwar Bhowmick TCS Innovation
Labs, Kolkata, India  K. Madhava Krishna Robotics Research Center, IIIT
Hyderabad, India

###### Abstract

The simplicity of the visual servoing approach makes it an attractive option
for tasks dealing with vision-based control of robots in many real-world
applications. However, attaining precise alignment for unseen environments
pose a challenge to existing visual servoing approaches. While classical
approaches assume a perfect world, the recent data-driven approaches face
issues when generalizing to novel environments. In this paper, we aim to
combine the best of both worlds. We present a deep model predictive visual
servoing framework that can achieve precise alignment with optimal
trajectories and can generalize to novel environments. Our framework consists
of a deep network for optical flow predictions, which are used along with a
predictive model to forecast future optical flow. For generating an optimal
set of velocities we present a control network that can be trained on-the-fly
without any supervision. Through extensive simulations on photo-realistic
indoor settings of the popular Habitat framework, we show significant
performance gain due to the proposed formulation vis-a-vis recent state of the
art methods. Specifically, we show a faster convergence and an improved
performance in trajectory length over recent approaches.

††$*$: denotes equal contribution

> Keywords: Visual Servoing, Deep MPC

##  1 Introduction

The concept of visual servoing has been widely used in the field of robotics
for a variety of tasks from manipulation to navigation [1, 2, 3, 4, 5].
Mapping the observation space to the control/action space is the fundamental
objective behind most of the visual servoing problems. Classically, visual
servoing is highly dependent on handcrafted image features, environment
modeling, and an accurate understanding of system dynamics for the generation
of control commands [6]. Sensitivity to the accuracy of the available
information also prevents such systems from adjusting to the statistical
irregularities of the world. To mitigate these issues, recent approaches aim
to learn supervised models that predict relative camera poses between the
current and the desired images [7, 8, 9]. Although this overcomes the
requirements of having handcrafted image features or any explicit information
about the environment, it fails to generalize in unseen environments. Other
approaches further attempt to exploit information from intermediate
representations like optical flow [10], however, they use classical visual
servoing controller which only looks one-step ahead hence results in sub-
optimal trajectories. On the other hand, many deep reinforcement learning
based approaches such as [11] and [12] do not show their convergence in a
6-DoF domain due to an intractable number of samples in higher dimensional
actions and continuous state space. This narrows down their scope of
performance in real-world visual servoing tasks for precise alignment.
Furthermore, they also need to be extensively retrained in newer environments
as a result, their generalisation capabilities are limited. A few approaches
[13, 14], employ a model based learning framework for visual servoing, they
attempt to learn a forward predictive model to represent the process and a
policy for planning over the predictive model. However, their predictive model
is trained to minimise the appearance loss which is difficult to optimise for
a longer horizon and does not generalize to novel environments. Furthermore,
the policy is obtained either through sampling [13] or by learning from human
demonstrations [14] that do not scale to a higher dimensional action space.

In this work, we present a deep model predictive control strategy that
exploits the visual servoing concept in a principled fashion. This is achieved
by formulating a novel state prediction model for visual servoing based on
dense off-the-shelf/unsupervised optical flow predictions. The prediction
model is employed to generate optimal velocity commands using a control
network, we further propose an efficient unsupervised strategy for online
optimisation of control network. We validate our approach on a photo-realistic
visual servoing dataset proposed in [10], comparing against an exhaustive set
of baselines among classical, deep supervised and model predictive visual
servoing approaches.

Our contributions are summarized as follows:

  * •

We propose a novel unsupervised Deep Model Predictive Control pipeline for
visual servoing using optical flow as intermediate representation in 6-DoF.

  * •

The control actions are learned unsupervised as the LSTM decoder strives to
predict future controls over a time horizon such that the optical flow accrued
as a consequence of these predictions matches the desired flow. The desired
flow between the current and desired images is computed either through a
supervised [15] or unsupervised [16] framework.

  * •

The online optimization over the MPC framework enables the proposed approach
to adapt to any environment even with inaccurate flow predictions and system
dynamics information.

  * •

We compare with existing deep visual servoing methods by evaluating on
parameters like translation error, rotation error, trajectory length, total
no. of iterations and time per iteration. Our approach exhibits significant
convergence and is faster than the state of the art deep visual servoing
methods [10], [17], [18],[19]. We present both qualitative and quantitative
results for the same. We make the code 222Github Link:
https://github.com/pushkalkatara/DeepMPCVS publicly available.

##  2 Related Work

Visual Servoing. Visual Servo (VS) relates to the problem of attaining a goal
pose in the environment using image measurements from a camera sensor attached
to a robot. It can be broadly classified in two approaches: (i) PBVS: Pose
based Visual Servoing where hand-crafted features from images are used to
estimate geometrical information such as pose of the target, camera parameters
and controller is responsible to minimize pose error between current image and
goal image. (ii) IBVS: Image-based Visual Servoing approaches perform feature
extraction and minimize the feature error explicitly in image space. The
control law is to perform gradient descent in feature space, which is then
mapped to the robot’s velocities using an Image Jacobian [6]. Classical visual
servoing approaches [1, 4, 5, 20] employ local appearance based features such
as keypoints, lines and contours to describe the scene. On the other hand,
direct visual servoing approaches [19, 21] skip the feature extraction and
directly minimize the difference in images. Although these direct approaches
mitigate the issue of incorrect matches, they do not converge for larger
camera transformations. One of the limitations of classical visual servoing
methods is the requirement of the depth of the visual features while computing
the Image Jacobian, which is generally not available in the case of a
monocular camera.

Deep Visual Servoing. Recent deep learning driven approaches [7, 8, 18] learn
deep neural networks that directly aim to predict the relative camera pose
between the current and the desired image. The controller then takes a small
step to minimize this error. These neural networks are learned in a supervised
fashion and consequently do not generalize well to unseen environments. A very
recent approach [10] combines deep learning with visual servoing by learning
intermediate flow representations and minimizing their difference using a
classical visual servoing controller. This approach effectively generalizes to
novel environments in 6-DoF. However, since the controller optimizes only for
a single time step, it generates sub-optimal trajectories and could get stuck
in local minima as a result of this greedy behavior.

Visual Navigation using Reinforcement Learning. Visual servoing could also be
posed as a target driven visual navigation problem. Motivated from the success
attained by deep reinforcement learning (DRL) approaches in maze navigation
tasks, there has been an increase in interest in applying imitation learning
[22] or reinforcement learning methods for visual navigation in indoor
environments [11, 12, 23, 24, 25]. In contrast to classical visual servoing
approaches that greedily plan one time step, DRL approaches plan an optimal
policy over a longer horizon even when there is no overlap between the current
and desired image. However, such model free end-to-end learning based
approaches are sample inefficient, thus do not scale to higher dimensional
continuous actions and face difficulty while generalising to novel
environments.

Model Based Visual Control. Model predictive control (MPC) approaches have
been successful in learning complex skills in robotics such as controlling
quadrotors [26], and humanoids [27] using accurate system dynamics. Keypoints
based MPC models proposed by [28, 29] have been previously used in IBVS to
generate optimal policies under the assumption of accurate matching and using
a handful of keypoints. Classical MPC approaches have two limitations:
firstly, they require accurate dynamics, secondly, they do not scale well in
large state spaces. Recent deep learning approaches aim to improve upon this
limitation by simultaneously learning the model dynamics along with the
policies [30, 31]. For learning visio-motor control, recent approaches propose
a deep network to learn forward dynamics by predicting future observations
[13, 14]. To learn the policies along with dynamics, there are a few choices:
(i) projecting the latent space into a Cartesian world and classical
approaches to generate control [32, 33]. However, directly learning camera
pose could be inaccurate for unseen environments and 6-DoF actions. (ii)
Hirose et al. [14] use a neural network to approximate policy which is learned
in a supervised fashion from human demonstrations. However, the issue with
this method is obtaining a large and diverse dataset of human demonstrations
which is expensive. Another approach [13] uses visual MPC where model is
trained using collected data, further perform MPC optimisation using CEM which
combined makes the approach supervised and sampling-dependent making it less
generalizable and less efficient than the proposed method.

This work bridges the gap between classical and learning-based control for
6-DoF image-based visual servoing (IBVS) in novel environments with continuous
action and state space. We select optical flow to represent our states instead
of directly working with images. The dense optical flow is predicted using
deep neural networks. Subsequently, we reformulate a predictive model for
forecasting the evolution of states given a sequence of actions (velocities).
We then learn a recurrent control network on-the-fly in an unsupervised
fashion for computing an optimal set of velocity for a given goal state. We
show superior performance vis-a-vis Deep Visual Servoing methods due to a
receding horizon controller even as the framework generalizes to new
environments without needing to retrain or finetune. The controller regresses
to a continuous space of outputs over 6-DoF.

##  3 Approach

Figure 1:  Given agent observation $I_{t}$ and desired observation $I^{*}$ the
pipeline generates optimal control commands $[v_{t},\omega_{t}]$ for each
time-step t until convergence is achieved. The flow network encodes optical
flow between $I_{t}$ and $I^{*}$ and the control-optimizer architecture learns
to generate optimal control for each time-step $t$ in real-time through
encoded flow.

In this work, we consider the problem of visual servoing as a target driven
navigation in unseen environments using a monocular camera. Given $I_{t}$, the
observation of the robot in the form of monocular RGB image at any time
instant $t$ and the desired observation $I^{*}$, our goal is to generate
optimal control commands $[v_{t},\omega_{t}]$ in 6-DoF that minimizes the
photometric error between $I^{*}$ and $I_{t}$. We assume that the camera is
attached to the robot (also known as eye-in-hand configuration) and is
calibrated. We further assume that the environment is collision free and there
exists a partial overlap between the initial and desired image. In contrast to
existing visual navigation approaches [14], we aim to generate trajectories in
continuous actions (in 6-DoF) and state space. Figure 1 provides an overview
of our approach.

###  3.1 Learning Intermediate Representation

Instead of directly planning in image observation space, we employ optical
flow as an intermediate visual representation for encoding differences in
images. Optical flow encodes displacement for every pixel which is more
relevant information for feature tracking and matching as compared to pixel
intensities. As a result, optical flow has been successfully used in motion
estimation [34] and visual servoing [35, 10]. Furthermore, in the presence of
translation camera motion in a static environment, dense flow features could
also be used for estimating image depth [36]. Although we do not constraint
ourselves to any specific network for flow or depth estimation, in this work
we use a pre-trained neural network, Flownet 2 [15] for flow estimation
without any fine-tuning. To showcase that our approach can achieve similar
performance even when trained without any supervision, we replace Flownet 2
with another network architecture DDFlow [16], which is trained from scratch
using only images in an unsupervised manner. For specifics regarding training
of DDFlow, refer to the supplementary material.

###  3.2 Predictive model for visual servoing

The fundamental objective of IBVS is to minimize the error in visual features
between current and desired images, $e_{t}=\|s(I_{t})-s(I^{*})\|$, which is
solved by performing gradient descent in feature space, resulting in following
control law:

$\mathbf{v}_{t}=-\lambda L^{+}(s(I_{t})-s(I^{*}))$ (1)

where, $\mathbf{v}_{t}=[v_{t},\omega_{t},]$ is the velocity command generated
by the controller in 6-DoF, $s(I_{t})$ and $s(I^{*})$ are the visual features
extracted from images $I_{t}$ and $I^{*}$ respectively. The gradient descent
step size is depicted by $\lambda$ and $(.)^{+}$ denotes psuedo-inverse
operation. The image Jacobian or the interaction matrix mapping the camera
velocity to rate of change of features is given by,

$L(Z_{t})=\begin{bmatrix}-1/Z_{t}&0&x/Z_{t}&xy&-(1+{x}^{2})&y\\\
0&-1/Z_{t}&y/Z_{t}&1+{y}^{2}&-xy&-x\end{bmatrix}.$ (2)

Where, $x$ and $y$ are the normalized image coordinate and $Z_{t}$ is the
scene’s depth at $(x,y,t)$. For a small time interval $\delta t$, IBVS control
law, i.e. equation (1) can be reformulated as $\textstyle s_{t+\delta
t}=s_{t}+L(z)\mathbf{v}_{t}$, which can be rewritten using optical flow as
visual features as,

$\textstyle\mathcal{F}(I_{t},I_{t+\delta t})=L(Z_{t})\mathbf{v}_{t}.$ (3)

This equation gives us the predictive model that could be employed to
reconstruct the future states (flows in our case) from the current velocity.
This also points out that generating an optimal set of actions in terms of
velocities $\mathbf{v}^{*}_{t+1:t+T}$ is a critical step for the convergence
of the system towards the desired image. We achieve this task by online
trajectory optimization using the predictive model and a control network
described in equation (3) as described in section 3.3.

###  3.3 Online Trajectory Optimization using Neural MPC

Model predictive controllers are proven to work well for a wide variety of
tasks in which accurate analytical models are difficult to obtain [30]. A
model predictive controller aims to generate a set of optimal control actions
$\mathbf{v}^{*}_{t+1:t+T}$ which minimize a given cost function
$C(\hat{X}_{t+1:t+T},\mathbf{v}_{t+1:t+T})$ over predicted state $\hat{X}$ and
control inputs $\mathbf{v}$ for some finite time horizon $T$. In order to
formulate visual servoing as MPC with intermediate flow representations, we
define our cost function as mean squared error in flow between any two given
images. Then MPC objective,

$\textstyle\mathbf{v}^{*}_{t+1:t+T}=\underset{\mathbf{v}_{t+1:t+T}}{\arg\min}\lVert\mathcal{{F}}(I_{t},I^{*})-\widehat{\mathcal{F}}(\mathbf{v}_{t+1:t+T})\rVert$
(4)

is then to generate a set of velocities $\textstyle\mathbf{v}^{*}_{t+1:t+T}$
that minimise the error between the desired flow
$\mathcal{{F}}(I_{t},I^{*}_{t})$ predicted by the flow network and the
generated flow $\widehat{\mathcal{F}}(\mathbf{v}_{t+1:t+T})$. Exploiting the
additive nature of optical flows (i.e.
$\mathcal{{F}}(I_{1},I_{3})=\mathcal{{F}}(I_{1},I_{2})+\mathcal{{F}}(I_{2},I_{3})$),
the generated flow could be written as:

$\begin{split}\widehat{\mathcal{F}}(\mathbf{v}_{t+1:t+T})=\sum_{k=1}^{T}[L(Z_{t})\mathbf{v}_{t+k}]\end{split}$
(5)

using the predictive model from equation (3). This formulation provides
optimal trajectory over a horizon $T$ as compared to classical IBVS
controller, equation (1) that greedily optimises a single step. Another
advantage of using the proposed approach is significant decrease in
computation time as compared to classical IBVS controller, as we avoid the
overhead of matrix inversion required for computation of pseudo inverse
$L^{+}=(L^{T}L+\mu diag(L^{T}L))^{-1}$.

The above formulation allows us to control in continuous 6-DoF action space.
However, since the dimensionality of our states (flow representations) are
large, classical MPC solvers could not be used to optimise equation (4). To
tackle this issue, Finn and Levine [13] use a sampling based Cross Entropy
Method (CEM), but the approach is model-based sampling which requires large
number of iterations to train and is less generalizable as supervised on a
dataset. Another recent approach [14] employs a neural network for policy
generation. They train their network using human demonstrations. However,
obtaining human demonstrations are expensive especially for 6 DoF. In
experiments section 4.1, we explicitly compare with these approaches.

In this work, we use a recurrent neural network (RNN) to generate velocity
commands. This choice is more efficient over sampling based approaches such as
CEM since, RNN can be directly trained using Back Propagation Through Time
(BPTT). Furthermore, for sequence prediction tasks RNN is a natural choice
over a feed forward neural network. To train our control network, we employ
the predictions from the flow network as targets. Thus, our target remains
fixed for online trajectory optimization. Note that our linearized predictive
model depends only on the depth of the scene, and assuming depth consistency
for a smaller horizon, we can write the MPC objective only in terms of
control. Hence, our control network could easily be trained in supervised
manner by minimising the flow loss:

$\mathcal{L}_{flow}=\lVert\widehat{\mathcal{F}}(\widehat{\mathbf{v}}_{t+1:t+T})-\mathcal{F}(I_{t},I^{*})\rVert=\lVert\sum_{k=1}^{T}[L(Z_{t})\widehat{\mathbf{v}}_{t+k}]-\mathcal{F}(I_{t},I^{*})\rVert$
(6)

Where, the RNN $g(\mathbf{v}_{t},\theta)$ could be used to generate velocity
at next time instance $\widehat{\mathbf{v}}_{t+1}=g(\mathbf{v}_{t},\theta)$
given the current velocity. Hence, given previous velocity
$\mathbf{{v}_{t-1}}$ and tries to improve the predicted velocity and
compensate for approximations in predictive model by adjusting its parameters
$\mathbf{\theta}$ according such that the flow loss is minimized.

###  3.4 Control Network

In this work, we use a Long-Short-Term Memory architecture (LSTM) [37] with 5
LSTM cell units as RNN for generating velocities. The sequence length (length
of planning horizon as $T$) is a tuning parameter that trades-off between
precision and computation time. In this work we select the sequence length
$T=5$. The network receives previous 6-DoF velocity $\mathbf{v}_{t-1}$ as
input and predicts a sequence of $T$ velocities $\widehat{\mathbf{v}}_{t:t+T}$
in 6-DoF. This is multiplied by the interaction matrix $L(z)$ to obtain
generated flows $\widehat{\mathcal{F}}_{t:t+T}$. The network is trained online
in supervised fashion for $M=100$ iterations by minimizing the flow loss,
equation (6), where the target flow is predicted using our flow network. After
the network is trained for $M$ iterations we execute the $v_{t+1}$ and observe
next image $I_{t+1}$, which is used to predict next target flow
$\mathcal{F}(I_{t+1},I^{*})$. Subsequently, the interaction matrix is
recomputed $L(Z_{t+1})$ based on updated depth $Z_{t+1}$ using
$\mathcal{F}(I_{t},I_{t+1})$ as the proxy. The entire approach can be
summarised by algorithm 1.

Algorithm 1 Model Predictive Visual Servoing using Online Optimisation

1:$I^{*}$, $\epsilon$ $\triangleright$ Goal Image, convergence threshold

2:Initialize $v_{0}$ with random velocity

3:while $\lVert I-I^{*}\rVert\leq\epsilon$ do $\triangleright$ Convergence
criterion

4: $I_{t}:=$ get-current-obs() $\triangleright$ Obtain the current RGB
observation from sensor

5: Predict-Flow ($\mathcal{F}(I_{t},I^{*})$) $\triangleright$ Predict target
flow using flow network

6: $L_{t}:=$ compute-interaction-matrix($\mathcal{F}(I_{t},I_{t-1})$)

7: for $m=0\to M$ do $\triangleright$ Online training the control network

8: $\widehat{\mathbf{v}}_{t:t+T}:=$ g($v_{t-1},\theta_{m}$) $\triangleright$
Velocity predictions from control network

9:
$\widehat{\mathcal{F}}(\widehat{\mathbf{v}}_{t+1:t+T})=\sum_{k=1}^{T}[L_{t}(Z_{t})\widehat{\mathbf{v}}_{t+k}]$
$\triangleright$ Generate flow using prediction model

10:
$\mathcal{L}_{flow}:=\lVert\widehat{\mathcal{F}}(\widehat{\mathbf{v}}_{t+1:t+T})-\mathcal{F}(I_{t},I^{*})\rVert.$
$\triangleright$ Compute flow Loss

11: $\theta_{m+1}:=\theta_{m}-\eta\nabla\mathcal{L}_{flow}$ $\triangleright$
Update control network parameters

12: end for

13: $v_{t+1}=\widehat{\mathbf{v}}_{t+1}$ $\triangleright$ Execute the next
control command

14:end while

##  4 Experiments

In our present work, we have explained an online training process which
enables the robot to learn optimal control commands on the go thus making our
controller performance independent of the environment it is deployed in. To
validate this, we show both qualitative and quantitative results on photo
realistic 3D baseline environments as proposed in [10]. The benchmark
comprises of $10$ indoor photo-realistic environments from the Gibson dataset
[38] in the Habitat simulation engine [39]. We use a free-flying RGB camera as
our agent so that agent can navigate in all 6-DoFs without any constraint. To
compare our approach we consider following baselines: Photometric visual
servoing[19]: PhotoVS is a learning-free classical visual servoing approach
that considers raw image intensities as visual features. Servonet[18], is a
supervised deep learning approach that attempts to predict relative camera
pose from a given image pair. We benchmark this approach without any
retraining/fine-tuning. DFVS[10]: similar to our approach DFVS employs deep
flow representations, however they use classical IBVS controller. To evaluate
our controller’s performance in isolation from flow network and predictive
model we also compare against baselines (a) Fully-connected Neural-Net:  and
(b) Stochastic Optimization [17]: known as the cross-entropy method. We
provide more details on the baseline in supplementary section.

###  4.1 Simulation results on Benchmark

We report both qualitative (refer figure 2) and quantitative ( refer table 1)
results on the benchmark. It can be seen from the table 1 , our controller
outperforms the current state-of-the-art Image Based Visual Servoing technique
[10] in total number of iterations while achieving marginally superior pose
error. PhotoVS [19] and Servonet [18] were unable to converge in 5 and 6 out
of 10 scenes in the benchmark respectively. It can also be seen from
comparison with the baseline (a) Fully-connected Neural-network that it gets
stuck in local minima and results in larger pose error. Furthermore, CEM
results in shorter trajectories but takes significantly large amount of time
to converge as compared to our approach which is intuitive since sampling
approaches are inefficient but statistically complete. We train our control
network for 100 iterations in each MPC optimization step and use Adam
Optimiser with learning rate of 0.0001. The number of training iterations can
be tuned using Early Stopping methods to achieve the best tradeoff between
speed and accuracy for realtime control. We use a "Nvidia Geforce GTX 1080-Ti
Pascal" GPU to benchmark these approaches.

| Initial
---
Image
|  |  |  |
| Desired
---
Image
|  |  |  |
PhotoVS [19] |  |  |  |  |
| Servonet
---
et al.[18]
|  |  |  |
DFVS [10] |  |  |  |  |
| MPC +
---
CEM [13]
|  |  |  |
| MPC +
---
NN [Ours]
|  |  |  |
| MPC +
---
LSTM [Ours]
|  |  |  |
Figure 2:  Qualitative Comparison: We show the photometric error image representation computed between the desired image $I^{*}$ and the final image on termination for various approaches. It is to be noted that the termination criteria for the run is photometric error of 500 or less. It can be seen that Saxena et al. [18] and PhotoVS [19] don’t converge even with large number of iterations hence showing large photometric errors. We compare Flow-depth[ours] with single-view depth of [10]. Comparisons with MPC + CEM [17] and MPC + NN [Ours] is also shown as their result justify our choice of LSTM based network. We show other qualitative results of left over baselines in supplementary. Approaches |  | T. Error
---
(meters)
| R. Error
---
(degrees)
| Tj. Length
---
(meters)
Iterations
Initial Pose Error | 1.6566 | 21.4166 | - | -
DFVS [10] | 0.0322 | 1.7088 | 1.7277 | 1009.2222
NN + MPC [ours] | 0.102 | 2.62 | $1.16^{*}$ | $344^{*}$
CEM [17] \+ MPC | 0.0513 | 0.8644 | 0.89 | 859.6667
LSTM + MPC [ours] | 0.0296 | 0.5977 | 1.081 | 557.4444
Table 1: Quantitative Comparison:We quantitatively compare the average
performance for different approaches across the benchmark scenes proposed in
[10] and report following metrics: Initial Pose Error, translation error(T.
Error), rotation error(R. Error) in degrees and trajectory length(Tj. Length)
in meters. This table does not include [19],[18] as they diverge in majority
of the environments[5 and 6 respectively of the 10 baseline envs].

###  4.2  Controller Performance and Trajectory Comparison

We next compare the trajectories taken by other visual servoing methods like
[10, 19] and controller performance with a standard feed forward neural
network (NN) and CEM [13] to reach the desired goal. The photometric
convergence and trajectory plots are shown in fig. 3. It can be observed from
the figure that photometric error steadily reduces to less than $500$. As seen
from the trajectory plots, CEM with predictive model tends to reach the goal
in an optimal path followed by our approach, while our approach is taking less
MPC steps for convergence.

|  |
---|---|---
Figure 3: Controller Performance: Left: Photometric Error (vs) Number of
Iterations - Our method [MPC + LSTM] converges in least number of iterations
compared to other approaches. Middle: Camera Trajectory. Right: Controller
performance on Unsupervised flow - Dashed lines represents Flownet2 [15] and
bold lines represents DDFlow [16]

###  4.3 Controller performance with Unsupervised flow

To showcase that our deep MPC architecture is also capable of generating
optimal control commands even when trained in fully unsupervised manner i.e.
here even the flow network is trained without supervision using only images.
We use DDFlow [16] instead of Flownet2 [15] to generate the optical flow. For
this experiment, we train our flow network offline on two scenes from the
benchmark. For the training details we refer the readers to the supplementary
material. The photometric error plots in Fig. 3 (Right) shows that our
controller successfully converges on both the environments, thereby making our
pipeline unsupervised. No ground truth or annotations were provided and the
flow was learned in an unsupervised fashion without any unlabeled data.

###  4.4 Generalisation to Real-World Noise

In order to demonstrate robustness of our approach we perform tests with
simulated noise in the Habitat environment. We add Gaussian noise with
mean=0.0 and standard deviation=0.1m in all 6-DoF to control commands. We
perform test on environment Mesic from our simulation benchmark [10] and show
that our pipeline converges to a photometric error of < 500 in 683 iterations
in presence of such a large noise, as compared to 504 iterations without
noise. This showcases ability to handle actuation noise and generalize in a
real world environment.

##  5 Conclusion and Future Work

In this work, we proposed a deep unsupervised model predictive control
architecture for visual servoing in 6-DoF. We demonstrated that our approach
can generate control commands in continuous state space which can adapt and
perform well despite an inaccurate understanding of the underlying system
dynamics. Its ability to be trained in an online fashion makes it easy to
adapt well in a completely unknown environment.
Although the proposed approach addresses some of the most crucial challenges
and performs better than state-of-the-art methods proposed on this problem, it
suffers due to a few drawbacks. Firstly, this approach assumes that the action
space of the robot is unobstructed and there is no chance of collision.
Secondly, like all of the visual servoing problems, it requires a small
minimum overlap between the current and the desired images which constrains
this method to act as a local planner.
The above limitations provide an opportunity to extend this approach by
incorporating methods to handle collision and integrate this method with a
long term global planner.

#### Acknowledgments

This work was done as a collaborative project between International Institute
of Information Technology(IIIT), Hyderabad India and Tata Consultancy
Services(TCS), Innovation Labs, India. This work was supported by research
grant from TCS Innovation Labs India and EPSRC UK (project NCNR, National
Centre for Nuclear Robotics, EP/R02572X/1)

## References

  * Basri et al. [1999] R. Basri, E. Rivlin, and I. Shimshoni.  Visual homing: Surfing on the epipoles.  _International Journal of Computer Vision_ , 1999.
  * Usher et al. [2003] K. Usher, P. Ridley, and P. Corke.  Visual servoing of a car-like vehicle-an application of omnidirectional vision.  In _IEEE ICRA_ , 2003.
  * Bourquardez et al. [2009] O. Bourquardez, R. Mahony, N. Guenard, F. Chaumette, T. Hamel, and L. Eck.  Image-based visual servo control of the translation kinematics of a quadrotor aerial vehicle.  _IEEE Transactions on Robotics_ , 2009.
  * Jagersand et al. [1997] M. Jagersand, O. Fuentes, and R. Nelson.  Experimental evaluation of uncalibrated visual servoing for precision manipulation.  In _ICRA_. IEEE, 1997.
  * Corke [1993] P. I. Corke.  Visual control of robot manipulators–a review.  In _Visual Servoing: Real-Time Control of Robot Manipulators Based on Visual Sensory Feedback_. World Scientific, 1993.
  * Chaumette and Hutchinson [2006] F. Chaumette and S. Hutchinson.  Visual servo control. i. basic approaches.  _IEEE RAM_ , 2006.
  * Yu et al. [2019] C. Yu, Z. Cai, H. Pham, and Q.-C. Pham.  Siamese convolutional neural network for sub-millimeter-accurate camera pose estimation and visual servoing.  _arXiv preprint arXiv:1903.04713_ , 2019.
  * Bateux et al. [2018] Q. Bateux, E. Marchand, J. Leitner, F. Chaumette, and P. Corke.  Training deep neural networks for visual servoing.  In _IEEE ICRA_ , 2018.
  * Sampedro et al. [2018] C. Sampedro, A. Rodriguez-Ramos, I. Gil, L. Mejias, and P. Campoy.  Image-based visual servoing controller for multirotor aerial robots using deep reinforcement learning.  In _IEEE/RSJ IROS_ , 2018.
  * Harish et al. [2020] Y. V. S. Harish, H. Pandya, A. Gaud, S. Terupally, S. Shankar, and K. M. Krishna.  Dfvs: Deep flow guided scene agnostic image based visual servoing.  In _IEEE ICRA_ , 2020.
  * Zhu et al. [2016] Y. Zhu, R. Mottaghi, E. Kolve, J. J. Lim, A. Gupta, L. Fei-Fei, and A. Farhadi.  Target-driven visual navigation in indoor scenes using deep reinforcement learning, 2016.
  * Gordon et al. [ICCV 2019] D. Gordon, A. Kadian, D. Parikh, J. Hoffman, and D. Batra.  Splitnet: Sim2sim and task2task transfer for embodied visual navigation, ICCV 2019.
  * Finn and Levine [ICRA 2016] C. Finn and S. Levine.  Deep visual foresight for planning robot motion, ICRA 2016.
  * Hirose et al. [2019] N. Hirose, F. Xia, R. Martín-Martín, A. Sadeghian, and S. Savarese.  Deep visual mpc-policy learning for navigation.  _IEEE Robotics and Automation Letters_ , 2019.
  * Ilg et al. [2017] E. Ilg, N. Mayer, T. Saikia, M. Keuper, A. Dosovitskiy, and T. Brox.  Flownet 2.0: Evolution of optical flow estimation with deep networks.  In _Proceedings of the IEEE conference on computer vision and pattern recognition_ , 2017.
  * Liu et al. [AAAI, 2019] P. Liu, I. King, M. R. Lyu, and J. Xu.  Ddflow: Learning optical flow with unlabeled data distillation, AAAI, 2019\.
  * Rubinstein and Kroese [2013] R. Y. Rubinstein and D. P. Kroese.  _The cross-entropy method: a unified approach to combinatorial optimization, Monte-Carlo simulation and machine learning_.  Springer Science & Business Media, 2013.
  * Saxena et al. [2017] A. Saxena, H. Pandya, G. Kumar, A. Gaud, and K. M. Krishna.  Exploring convolutional networks for end-to-end visual servoing.  In _IEEE ICRA_ , 2017.
  * Collewet and Marchand [2011] C. Collewet and E. Marchand.  Photometric visual servoing.  _IEEE TRO_ , 2011.
  * Malis et al. [2003] E. Malis, G. Chesi, and R. Cipolla.  212d visual servoing with respect to planar contours having complex and unknown shapes.  _The International Journal of Robotics Research_ , 2003.
  * Bateux and Marchand [2017] Q. Bateux and E. Marchand.  Histograms-based visual servoing.  _IEEE Robotics and Automation Letters_ , 2017.
  * Pathak et al. [IEEE CVPR workshop, 2018] D. Pathak, P. Mahmoudieh, G. Luo, P. Agrawal, D. Chen, Y. Shentu, E. Shelhamer, J. Malik, A. A. Efros, and T. Darrell.  Zero-shot visual imitation, IEEE CVPR workshop, 2018.
  * Gao et al. [2017] W. Gao, D. Hsu, W. S. Lee, S. Shen, and K. Subramanian.  Intention-net: Integrating planning and deep learning for goal-directed autonomous navigation.  _arXiv preprint arXiv:1710.05627_ , 2017.
  * Stein et al. [2018] G. J. Stein, C. Bradley, and N. Roy.  Learning over subgoals for efficient navigation of structured, unknown environments.  In _CoRL_ , 2018.
  * Sax et al. [arXiv preprint arXiv:1912.11121, 2019] A. Sax, J. O. Zhang, B. Emi, A. Zamir, S. Savarese, L. Guibas, and J. Malik.  Learning to navigate using mid-level visual priors, arXiv preprint arXiv:1912.11121, 2019.
  * Zhang et al. [2016] T. Zhang, G. Kahn, S. Levine, and P. Abbeel.  Learning deep control policies for autonomous aerial vehicles with mpc-guided policy search.  In _IEEE ICRA_ , 2016.
  * Williams et al. [2017] G. Williams, N. Wagener, B. Goldfain, P. Drews, J. M. Rehg, B. Boots, and E. A. Theodorou.  Information theoretic mpc for model-based reinforcement learning.  In _IEEE ICRA_ , 2017.
  * Sauvee et al. [2006] M. Sauvee, P. Poignet, E. Dombre, and E. Courtial.  Image based visual servoing through nonlinear model predictive control.  In _Proceedings of the 45th IEEE Conference on Decision and Control_ , pages 1776–1781, 2006.
  * Allibert et al. [2010] G. Allibert, E. Courtial, and F. Chaumette.  Predictive control for constrained image-based visual servoing.  _IEEE Transactions on Robotics_ , 2010.
  * Lenz et al. [2015] I. Lenz, R. A. Knepper, and A. Saxena.  Deepmpc: Learning deep latent features for model predictive control.  In _Robotics: Science and Systems_ , 2015.
  * Bieker et al. [arXiv:1905.10094, 2019] K. Bieker, S. Peitz, S. L. Brunton, J. N. Kutz, and M. Dellnitz.  Deep model predictive control with online learning for complex physical systems, arXiv:1905.10094, 2019.
  * Bansal et al. [2020] S. Bansal, V. Tolani, S. Gupta, J. Malik, and C. Tomlin.  Combining optimal control and learning for visual navigation in novel environments.  In _Conference on Robot Learning_ , 2020.
  * Drews et al. [2017] P. Drews, G. Williams, B. Goldfain, E. A. Theodorou, and J. M. Rehg.  Aggressive deep driving: Combining convolutional neural networks and model predictive control.  In _Conference on Robot Learning_ , 2017.
  * Brox and Malik [2010] T. Brox and J. Malik.  Large displacement optical flow: descriptor matching in variational motion estimation.  _IEEE transactions on pattern analysis and machine intelligence_ , 2010.
  * Argus et al. [arXiv:2007.00291, 2020] M. Argus, L. Hermann, J. Long, and T. Brox.  Flowcontrol: Optical flow based visual servoing, arXiv:2007.00291, 2020\.
  * Shahraray and Brown [1988] B. Shahraray and M. K. Brown.  Robust depth estimation from optical flow.  In _[1988 Proceedings] Second International Conference on Computer Vision_ , 1988.
  * Hochreiter and Schmidhuber [1997] S. Hochreiter and J. Schmidhuber.  Long short-term memory.  _Neural Computation_ , 9(8):1735–1780, 1997.  doi:10.1162/neco.1997.9.8.1735.
  * Xia et al. [2018] F. Xia, A. R. Zamir, Z. He, A. Sax, J. Malik, and S. Savarese.  Gibson env: Real-world perception for embodied agents.  In _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_ , pages 9068–9079, 2018.
  * Manolis Savva* et al. [2019] Manolis Savva*, Abhishek Kadian*, Oleksandr Maksymets*, Y. Zhao, E. Wijmans, B. Jain, J. Straub, J. Liu, V. Koltun, J. Malik, D. Parikh, and D. Batra.  Habitat: A Platform for Embodied AI Research.  In _Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)_ , 2019.


learning fail to protect the training data privacy under gradient leakage
attacks (wei2021gradient, ). Second, these DP solutions also fail to preserve
the same level of model fairness compared to the corresponding non-private
version (truex2019effects, ). We argue that to ensure a DP approach to
federated learning is also gradient leakage resilient and fairness preserving,
it is important to incorporate data and model governance guidelines and
test/evaluation metrics, ensuring that only those $(\epsilon,\delta)$-DP
solutions that are meeting both leakage resilience and fairness preserving
guidelines will be chosen, instead of only optimizing utility factor. One way
to achieve this responsibility-utility co-design is to provide a robust legal
framework that creates policy-driven rights and limits on how data and models
can be used, accompanied by enforcement methods and software tools. Figure 3
summarizes the responsibility-utility co-design to ensure robustness, privacy,
and fairness with governance and compliance.

Figure 3. Co-design of robustness, privacy, and fairness with utility
preserving under trustworthy AI compliance and incentives.

Incentives and compliance. Trustworthy distributed AI compliance can be
promoted and enforced throughout the distributed learning protocols through
feedback control with human-in-the-loop. Although providing discipline for
violation is a form of incentive, tedious and frequent compliance requirements
may put users into consent fatigue (utz2019informed, ) as they are playing
whack-a-mole with consent notifications without taking the time to understand
them. We argue that appropriate and positive incentives are required for
effective compliance. With the increasing levels of misuse and abuse in AI
technology concerning fairness, privacy, and robustness, it becomes critically
important to develop incentive tools to drive the responsible behavior
expected of data and models employed in distributed AI systems and services.
By developing appropriate compliance incentives associated with trustworthy AI
enforcement in distributed learning (e.g., model training and model
inference), the incentives can encourage responsible self-discipline, automate
compliance evaluation and verification, and significantly reduce the risk of
abuse and misuse.

We argue that an incentive-enhanced compliance framework for trustworthy
distributed learning should include the following components: (1) the risk-
specific compliance policy and governance guidelines with human-centered
design; (2) compliance monitoring and audit methods and tools; (3) social and
technical awareness education of trustworthy AI. Game theoretic mechanism
design could be a possible solution for incentive-compatibility while
achieving trustworthiness and high utility simultaneously (zhou2017private, ).
As AI systems play increasingly critical roles, enhancing the interaction
between humans and AI systems is crucial. Incentive programs must adapt to
emerging challenges, technologies, and ethical considerations. Service
providers continue monitor user feedback and introduce human-in-the-loop to
improve the trustworthy AI landscape, together with AI-driven governance
tools.

Toward trustworthy future of foundation models. Foundation models such as GPT
models and Stable Diffusion (openai2023gpt, ; rombach2022high, ) represent a
major advancement of distributed AI, with the promise of transforming domains
through learned knowledge in vast natural language data for downstream tasks
of different data modules. The diverse sources of large-scale training data
and the colossal model sizes of foundation models would inevitably incorporate
distributed learning as its infrastructure. While existing large language
models and foundation models are mostly centralized, many mission-critical and
copy-righted data cannot allow centralized data collection and training.
Therefore, we can anticipate more distributed AI on large language models and
other foundation models. Our discussion on robustness, privacy, and fairness,
together with governance of the trustworthy distributed AI, can take an
initial step to understanding the vulnerabilities faced by the foundation
models. These models face security risks of jailbreaking and manipulation,
privacy risks of revealing the private training data, and additional risks of
bias and hallucination. For example, training data of GPT models and diffusion
models can be exposed from prompt design (carlini2021extracting, ;
carlini2023extracting, ). Backdoor triggers can also be added to induce
targeted outcomes in image generation (chen2023trojdiff, ; chou2023backdoor,
). Shi et al. (shi2023badgpt, ) added a backdoor during the Reinforcement
Learning phase of the ChatGPT training, allowing the model to be susceptible
to backdoor attacks after fine-tuning. The prevalence of foundation models
also leads to new attack surfaces to exploit. For example, prompt injection
attacks aim to elicit an unintended response from the foundation model and
then achieve unauthorized access, manipulate responses, or bypass security
measures (perez2022ignore, ; du2022ppt, ). Besides the malicious uses and
sensitive information disclosure of foundation models, the generative nature
of foundation models also faces challenges from discrimination, exclusion and
toxicity problems, and misinformation harms (weidinger2021ethical, ).

The current safety strategy of GPT models includes pruning the dataset for
toxic or inappropriate content, training the system with Reinforcement
Learning from Human Feedback (RLHF) and rule-based reward models (RBRMs),
providing structured access to the model through an API, and investing in
moderation by humans and by automated moderation and content
classifiers888cdn.openai.com/papers/gpt-4-system-card.pdf. Given the various
safety risks of foundation models, we argue to achieve trustworthy foundation
models from five different perspectives. (1) explaining their decision-making
process; (2) learning to identify the information source and introducing
cross-validation tools to verify any information given by the models. For
misinformation, logic and factual error, the ensemble of the independent
decision maker can find the logical fallacy and discrepancy in the output for
a fact check; (3) integrating toxic language detectors like
RealToxicityPrompts (gehman2020realtoxicityprompts, ) and algorithmic bias
evaluation tools to assess the model output; (4) limiting the capacity of
foundation models on what type of information they can give out; (5) setting
up the minimal information and access permission granted for individuals and
applications, e.g., watermarking for data and model protection
(cao2023invisible, ). Combining these trustworthy AI policy guidelines and
responsibility-utility co-design with governance and compliance specifically
designed for foundation models, we would secure trustworthy distributed AI
systems in the arising foundation model future.

##  8\. Conclusion

We reviewed the representative techniques, algorithms, and theoretical
foundations for trustworthy distributed AI through robustness guarantee,
privacy protection, and fairness awareness in distributed learning. First, we
gave a brief overview of alternative architectures for distributed learning
and discussed the vulnerabilities of AI algorithms in distributed learning
with respect to robustness, privacy, and fairness. Second, we provided a
comprehensive categorization of trustworthy distributed AI, covering (1)
robustness to evasion attacks and irregular queries at inference, and
robustness to poisoning attacks, Byzantine attacks, and irregular data
distribution during training; (2) privacy protection during distributed
learning and model inference at deployment; and (3) AI fairness and governance
regarding both data and models. We concluded the paper with a discussion of
open challenges and future research directions toward trustworthy distributed
AI. To the best of our knowledge, this is the first survey that reviews the
robustness, privacy, fairness, and governance of AI models in the context of
distributed learning through the lens of trustworthy distributed AI. This
survey can benefit AI researchers and AI software developers with the advanced
knowledge of a collection of responsible AI techniques for ensuring
robustness, privacy, and fairness in distributed learning. This survey also
assists cybersecurity researchers, engineers, and stakeholders, who employ AI
technology to solve cybersecurity problems, to gain an in-depth understanding
of the potential issues of AI models concerning robustness, privacy, and
governance. As AI systems play increasingly critical roles in society, the
need for ethical AI and robust governance mechanisms will continue to grow.
Research should focus on developing frameworks and guidelines for ethical AI
design, deployment, and monitoring.

Acknowledgement. This research is partially sponsored by the USA National
Science Foundation under Grants 2038029, 2302720, 2312758, an IBM faculty
award, and a CISCO edge AI grant.

## References

  * [1] Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Aguera y Arcas.  Communication-efficient learning of deep networks from decentralized data.  In AISTATS, 2017.
  * [2] Yue Zhao, Meng Li, Liangzhen Lai, Naveen Suda, Damon Civin, and Vikas Chandra.  Federated learning with non-iid data.  arXiv preprint arXiv:1806.00582, 2018.
  * [3] Vale Tolpegin, Stacey Truex, Mehmet Emre Gursoy, and Ling Liu.  Data poisoning attacks against federated learning systems.  In ESORICS, 2020.
  * [4] Ka-Ho Chow, Ling Liu, Wenqi Wei, Fatih Ilhan, and Yanzhao Wu.  Stdlens: Model hijacking-resilient federated learning for object detection.  In CVPR, pages 16343–16351, 2023.
  * [5] Wenqi Wei, Ling Liu, Yanzhao Wu, Gong Su, and Arun Iyengar.  Gradient-leakage resilient federated learning.  In ICDCS, 2021.
  * [6] Wenqi Wei and Ling Liu.  Gradient leakage attack resilient deep learning.  IEEE TIFS, 17:303–316, 2021.
  * [7] Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy.  Explaining and harnessing adversarial examples.  In ICLR, 2015.
  * [8] Nicholas Carlini and David Wagner.  Towards evaluating the robustness of neural networks.  In IEEE S&P, 2017.
  * [9] Luca Melis, Congzheng Song, Emiliano De Cristofaro, and Vitaly Shmatikov.  Exploiting unintended feature leakage in collaborative learning.  In S&P, 2019.
  * [10] Reza Shokri, Marco Stronati, Congzheng Song, and Vitaly Shmatikov.  Membership inference attacks against machine learning models.  In S&P, 2017.
  * [11] Stacey Truex, Ling Liu, Mehmet Emre Gursoy, Lei Yu, and Wenqi Wei.  Demystifying membership inference attacks in machine learning as a service.  IEEE Trans. on Serv. Comput., 2019.
  * [12] Matt Fredrikson, Somesh Jha, and Thomas Ristenpart.  Model inversion attacks that exploit confidence information and basic countermeasures.  In CCS, 2015.
  * [13] Susan T Fiske.  Stereotyping, prejudice, and discrimination.  1998\.
  * [14] Ninareh Mehrabi, Fred Morstatter, Nripsuta Saxena, Kristina Lerman, and Aram Galstyan.  A survey on bias and fairness in machine learning.  ACM CSUR, 54(6):1–35, 2021.
  * [15] Qiang Yang, Yang Liu, Tianjian Chen, and Yongxin Tong.  Federated machine learning: Concept and applications.  ACM NTIST, 10(2):1–19, 2019.
  * [16] Martín Abadi, Ashish Agarwal, Paul Barham, Eugene Brevdo, Zhifeng Chen, Craig Citro, Greg S Corrado, Andy Davis, Jeff Dean, Matthieu Devin, et al.  Tensorflow: Large-scale machine learning on heterogeneous distributed systems.  arXiv preprint arXiv:1603.04467, 2016.
  * [17] Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al.  Pytorch: An imperative style, high-performance deep learning library.  In NeurIPS, 2019.
  * [18] Chandra Thapa, Pathum Chamikara Mahawaga Arachchige, Seyit Camtepe, and Lichao Sun.  Splitfed: When federated learning meets split learning.  In AAAI, 2022.
  * [19] Enmao Diao, Jie Ding, and Vahid Tarokh.  Heterofl: Computation and communication efficient federated learning for heterogeneous clients.  In ICLR, 2020.
  * [20] Sébastien Bubeck, Nicolo Cesa-Bianchi, et al.  Regret analysis of stochastic and nonstochastic multi-armed bandit problems.  FnT ML, 5(1):1–122, 2012.
  * [21] Keqin Liu and Qing Zhao.  Distributed learning in multi-armed bandit with multiple players.  IEEE TSP, 58(11):5667–5681, 2010.
  * [22] David Martínez-Rubio, Varun Kanade, and Patrick Rebeschini.  Decentralized cooperative stochastic bandits.  In NeurIPS, 2019.
  * [23] Chengshuai Shi and Cong Shen.  Federated multi-armed bandits.  In AAAI, 2021.
  * [24] Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu.  Towards deep learning models resistant to adversarial attacks.  In ICLR, 2018.
  * [25] Weilin Xu, David Evans, and Yanjun Qi.  Feature Squeezing: Detecting Adversarial Examples in Deep Neural Networks.  In NDSS, 2018.
  * [26] Dan Hendrycks and Kevin Gimpel.  A baseline for detecting misclassified and out-of-distribution examples in neural networks.  In ICLR, 2017.
  * [27] Wenqi Wei and Ling Liu.  Robust deep learning ensemble against deception.  IEEE TDSC, 18(4):1513–1527, 2020.
  * [28] Wenqi Wei.  Adversarial resilient and privacy preserving deep learning.  PhD thesis, Georgia Institute of Technology, Atlanta, GA, 2022.
  * [29] Congzheng Song, Thomas Ristenpart, and Vitaly Shmatikov.  Machine learning models that remember too much.  In CCS, 2017.
  * [30] Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus.  Intriguing properties of neural networks.  arXiv:1312.6199, 2013.
  * [31] Ka-Ho Chow, Ling Liu, Mehmet Emre Gursoy, Stacey Truex, Wenqi Wei, and Yanzhao Wu.  Understanding object detection through an adversarial lens.  In ESORICS, 2020.
  * [32] Bin Liang, Hongcheng Li, Miaoqiang Su, Pan Bian, Xirong Li, and Wenchang Shi.  Deep text classification can be fooled.  In IJCAI, 2018.
  * [33] J Li, S Ji, T Du, B Li, and T Wang.  Textbugger: Generating adversarial text against real-world applications.  In NDSS, 2019.
  * [34] Kevin Eykholt, Ivan Evtimov, Earlence Fernandes, Bo Li, Amir Rahmati, Chaowei Xiao, Atul Prakash, Tadayoshi Kohno, and Dawn Song.  Robust physical-world attacks on deep learning visual classification.  In CVPR, 2018.
  * [35] Wenqi Wei, Ling Liu, Margaret Loper, Mehmet Emre Gursoy, Stacey Truex, Lei Yu, and Yanzhao Wu.  Adversarial examples in deep learning: Characterization and divergence.  In IEEE TPS, 2020.
  * [36] Alexey Kurakin, Ian Goodfellow, and Samy Bengio.  Adversarial examples in the physical world.  In ICLR, 2017.
  * [37] Nicolas Papernot, Patrick McDaniel, Somesh Jha, Matt Fredrikson, Z Berkay Celik, and Ananthram Swami.  The limitations of deep learning in adversarial settings.  In IEEE EuroS&P, 2016.
  * [38] Seyed Mohsen Moosavi Dezfooli, Alhussein Fawzi, and Pascal Frossard.  Deepfool: a simple and accurate method to fool deep neural networks.  In CVPR, 2016.
  * [39] Nicolas Papernot, Patrick McDaniel, and Ian Goodfellow.  Transferability in machine learning: from phenomena to black-box attacks using adversarial samples.  arXiv:1605.07277, 2016.
  * [40] Florian Tramèr, Alexey Kurakin, Nicolas Papernot, Ian Goodfellow, Dan Boneh, and Patrick McDaniel.  Ensemble adversarial training: Attacks and defenses.  In ICLR, 2018.
  * [41] Ali Shafahi, Mahyar Najibi, Mohammad Amin Ghiasi, Zheng Xu, John Dickerson, Christoph Studer, Larry S Davis, Gavin Taylor, and Tom Goldstein.  Adversarial training for free!  In NeurIPS, 2019.
  * [42] Florian Tramer and Dan Boneh.  Adversarial training and robustness for multiple perturbations.  In NeurIPS, 2019.
  * [43] Shixiang Gu and Luca Rigazio.  Towards deep neural network architectures robust to adversarial examples.  In ICLR, 2015.
  * [44] Nicolas Papernot, Patrick McDaniel, Xi Wu, Somesh Jha, and Ananthram Swami.  Distillation as a defense to adversarial perturbations against deep neural networks.  In S&P,, 2016.
  * [45] Geoffrey Hinton, Oriol Vinyals, and Jeff Dean.  Distilling the knowledge in a neural network.  arXiv preprint arXiv:1503.02531, 2015.
  * [46] Harini Kannan, Alexey Kurakin, and Ian Goodfellow.  Adversarial logit pairing.  arXiv preprint arXiv:1803.06373, 2018.
  * [47] Chuan Guo, Mayank Rana, Moustapha Cisse, and Laurens van der Maaten.  Countering adversarial images using input transformations.  In ICLR, 2018.
  * [48] Jacob Buckman, Aurko Roy, Colin Raffel, and Ian Goodfellow.  Thermometer encoding: One hot way to resist adversarial examples.  In ICLR, 2018.
  * [49] Pouya Samangouei, Maya Kabkab, and Rama Chellappa.  Defense-gan: Protecting classifiers against adversarial attacks using generative models.  In ICLR, 2018.
  * [50] Yang Song, Taesup Kim, Sebastian Nowozin, Stefano Ermon, and Nate Kushman.  Pixeldefend: Leveraging generative models to understand and defend against adversarial examples.  In ICLR, 2018.
  * [51] Xiaoyu Cao and Neil Zhenqiang Gong.  Mitigating evasion attacks to deep neural networks via region-based classification.  In ACSAC, 2017.
  * [52] Warren He, James Wei, Xinyun Chen, Nicholas Carlini, and Dawn Song.  Adversarial example defense: Ensembles of weak defenses are not strong.  In WOOT, 2017.
  * [53] Kathrin Grosse, Praveen Manoharan, Nicolas Papernot, Michael Backes, and Patrick McDaniel.  On the (statistical) detection of adversarial examples.  arXiv preprint arXiv:1702.06280, 2017.
  * [54] Dongyu Meng and Hao Chen.  Magnet: a two-pronged defense against adversarial examples.  In CCS, 2017.
  * [55] Nicholas Carlini and David Wagner.  Adversarial examples are not easily detected: Bypassing ten detection methods.  In AISec, 2017.
  * [56] Wenqi Wei, Ling Liu, Margaret Loper, Ka-Ho Chow, Emre Gursoy, Stacey Truex, and Yanzhao Wu.  Cross-layer strategic ensemble defense against adversarial examples.  In ICNC, 2020.
  * [57] Ka-Ho Chow, Wenqi Wei, Yanzhao Wu, and Ling Liu.  Denoising and verification cross-layer ensemble against black-box adversarial attacks.  In IEEE Big Data, 2019.
  * [58] Ling Liu, Wenqi Wei, Ka-Ho Chow, Margaret Loper, Emre Gursoy, Stacey Truex, and Yanzhao Wu.  Deep neural network ensembles against deception: Ensemble diversity, accuracy and robustness.  In IEEE MASS, 2019.
  * [59] Tianyu Pang, Kun Xu, Chao Du, Ning Chen, and Jun Zhu.  Improving adversarial robustness via promoting ensemble diversity.  In ICML, 2019.
  * [60] Yanzhao Wu, Ling Liu, Zhongwei Xie, Ka-Ho Chow, and Wenqi Wei.  Boosting ensemble accuracy by revisiting ensemble diversity metrics.  In CVPR, 2021.
  * [61] Yanzhao Wu, Ka-Ho Chow, Wenqi Wei, and Ling Liu.  Exploring model learning heterogeneity for boosting ensemble robustness.  In ICDM, 2023.
  * [62] Guy Katz, Clark Barrett, David L Dill, Kyle Julian, and Mykel J Kochenderfer.  Reluplex: An efficient smt solver for verifying deep neural networks.  In CAV, 2017.
  * [63] Timon Gehr, Matthew Mirman, Dana Drachsler-Cohen, Petar Tsankov, Swarat Chaudhuri, and Martin Vechev.  Ai2: Safety and robustness certification of neural networks with abstract interpretation.  In S&P, 2018.
  * [64] Tsui-Wei Weng, Huan Zhang, Pin-Yu Chen, Jinfeng Yi, Dong Su, Yupeng Gao, Cho-Jui Hsieh, and Luca Daniel.  Evaluating the robustness of neural networks: An extreme value theory approach.  In ICLR, 2018.
  * [65] Eric Wong and Zico Kolter.  Provable defenses against adversarial examples via the convex outer adversarial polytope.  In ICML, 2018.
  * [66] Aditi Raghunathan, Jacob Steinhardt, and Percy Liang.  Certified defenses against adversarial examples.  In ICLR, 2018.
  * [67] Jeremy Cohen, Elan Rosenfeld, and Zico Kolter.  Certified adversarial robustness via randomized smoothing.  In ICML, 2019.
  * [68] Mathias Lecuyer, Vaggelis Atlidakis, Roxana Geambasu, Daniel Hsu, and Suman Jana.  Certified robustness to adversarial examples with differential privacy.  In S&P, 2019.
  * [69] Cynthia Dwork and Aaron Roth.  The algorithmic foundations of differential privacy.  FnT TCS, 9(3–4):211–407, 2014.
  * [70] Akhilan Boopathy, Tsui-Wei Weng, Pin-Yu Chen, Sijia Liu, and Luca Daniel.  Cnn-cert: An efficient framework for certifying robustness of convolutional neural networks.  In AAAI, 2019.
  * [71] Anish Athalye, Nicholas Carlini, and David Wagner.  Obfuscated gradients give a false sense of security: Circumventing defenses to adversarial examples.  In ICML, 2018.
  * [72] Anish Athalye, Logan Engstrom, Andrew Ilyas, and Kevin Kwok.  Synthesizing robust adversarial examples.  In ICML, 2018.
  * [73] Eugene Bagdasaryan, Andreas Veit, Yiqing Hua, Deborah Estrin, and Vitaly Shmatikov.  How to backdoor federated learning.  In AISTATS, 2020.
  * [74] Chulin Xie, Keli Huang, Pin-Yu Chen, and Bo Li.  Dba: Distributed backdoor attacks against federated learning.  In ICLR, 2019.
  * [75] Shihao Zhao, Xingjun Ma, Xiang Zheng, James Bailey, Jingjing Chen, and Yu-Gang Jiang.  Clean-label backdoor attacks on video recognition models.  In CVPR, 2020.
  * [76] Yingqi Liu, Shiqing Ma, Yousra Aafer, Wen-Chuan Lee, Juan Zhai, Weihang Wang, and Xiangyu Zhang.  Trojaning attack on neural networks.  In NDSS, 2017.
  * [77] Luis Muñoz-González, Battista Biggio, Ambra Demontis, Andrea Paudice, Vasin Wongrassamee, Emil C Lupu, and Fabio Roli.  Towards poisoning of deep learning algorithms with back-gradient optimization.  In AISec, 2017.
  * [78] Pang Wei Koh, Jacob Steinhardt, and Percy Liang.  Stronger data poisoning attacks break data sanitization defenses.  ML, 111(1):1–47, 2022.
  * [79] Ji Feng, Qi-Zhi Cai, and Zhi-Hua Zhou.  Learning to confuse: generating training time adversarial data with auto-encoder.  In NeurIPS, 2019.
  * [80] Ali Shafahi, W Ronny Huang, Mahyar Najibi, Octavian Suciu, Christoph Studer, Tudor Dumitras, and Tom Goldstein.  Poison frogs! targeted clean-label poisoning attacks on neural networks.  In NeurIPS, 2018.
  * [81] Chen Zhu, W Ronny Huang, Hengduo Li, Gavin Taylor, Christoph Studer, and Tom Goldstein.  Transferable clean-label poisoning attacks on deep neural nets.  In ICML, 2019.
  * [82] Minghong Fang, Xiaoyu Cao, Jinyuan Jia, and Neil Gong.  Local model poisoning attacks to byzantine-robust federated learning.  In Security, 2020.
  * [83] Arjun Nitin Bhagoji, Supriyo Chakraborty, Prateek Mittal, and Seraphin Calo.  Analyzing federated learning through an adversarial lens.  In ICML, 2019.
  * [84] Matthew Jagielski, Alina Oprea, Battista Biggio, Chang Liu, Cristina Nita-Rotaru, and Bo Li.  Manipulating machine learning: Poisoning attacks and countermeasures for regression learning.  In S&P, 2018.
  * [85] Cong Xie, Oluwasanmi Koyejo, and Indranil Gupta.  Fall of empires: Breaking byzantine-tolerant sgd by inner product manipulation.  In UAI, 2020.
  * [86] Rachid Guerraoui, Sébastien Rouault, et al.  The hidden vulnerability of distributed learning in byzantium.  In ICML, 2018.
  * [87] Xingyu Li, Zhe Qu, Shangqing Zhao, Bo Tang, Zhuo Lu, and Yao Liu.  Lomar: A local defense against poisoning attack on federated learning.  IEEE TDSC, 2021.
  * [88] Shiqi Shen, Shruti Tople, and Prateek Saxena.  Auror: Defending against poisoning attacks in collaborative deep learning systems.  In ACSAC, 2016.
  * [89] Brandon Tran, Jerry Li, and Aleksander Madry.  Spectral signatures in backdoor attacks.  In NeurIPS, 2018.
  * [90] Jonathan Hayase, Weihao Kong, Raghav Somani, and Sewoong Oh.  Spectre: defending against backdoor attacks using robust statistics.  In ICML, 2021.
  * [91] Di Tang, XiaoFeng Wang, Haixu Tang, and Kehuan Zhang.  Demon in the variant: Statistical analysis of $\\{$DNNs$\\}$ for robust backdoor contamination detection.  In Security, 2021.
  * [92] Xiaojun Xu, Qi Wang, Huichen Li, Nikita Borisov, Carl A Gunter, and Bo Li.  Detecting ai trojans using meta neural analysis.  In S&P, 2021.
  * [93] Dan Hendrycks, Mantas Mazeika, Duncan Wilson, and Kevin Gimpel.  Using trusted data to train deep networks on labels corrupted by severe noise.  In NeurIPS, 2018.
  * [94] Lingchen Zhao, Shengshan Hu, Qian Wang, Jianlin Jiang, Chao Shen, Xiangyang Luo, and Pengfei Hu.  Shielding collaborative learning: Mitigating poisoning attacks through client-side detection.  IEEE TDSC, 18(5):2029–2041, 2020.
  * [95] Moses Charikar, Jacob Steinhardt, and Gregory Valiant.  Learning from untrusted data.  In STOC, 2017.
  * [96] Xiaoyu Cao, Minghong Fang, Jia Liu, and Neil Zhenqiang Gong.  Fltrust: Byzantine-robust federated learning via trust bootstrapping.  In NDSS, 2021.
  * [97] Jacob Steinhardt, Pang Wei Koh, and Percy Liang.  Certified defenses for data poisoning attacks.  In NeurIPS, 2017.
  * [98] Elan Rosenfeld, Ezra Winston, Pradeep Ravikumar, and Zico Kolter.  Certified robustness to label-flipping attacks via randomized smoothing.  In ICML, 2020.
  * [99] A Levine and S Feizi.  Deep partition aggregation: Provable defense against general poisoning attacks.  In ICLR, 2021.
  * [100] Akshay Mehra, Bhavya Kailkhura, Pin-Yu Chen, and Jihun Hamm.  How robust are randomized smoothing based defenses to data poisoning?  In CVPR, 2021.
  * [101] Yunjuan Wang, Poorya Mianjy, and Raman Arora.  Robust learning for data poisoning attacks.  In ICML, 2021.
  * [102] Ashwinee Panda, Saeed Mahloujifar, Arjun Nitin Bhagoji, Supriyo Chakraborty, and Prateek Mittal.  Sparsefed: Mitigating model poisoning attacks in federated learning with sparsification.  In AISTATS, 2022.
  * [103] Eitan Borgnia, Valeriia Cherepanova, Liam Fowl, Amin Ghiasi, Jonas Geiping, Micah Goldblum, Tom Goldstein, and Arjun Gupta.  Strong data augmentation sanitizes poisoning and backdoor attacks without an accuracy tradeoff.  In ICASSP. IEEE, 2021.
  * [104] Roei Schuster, Congzheng Song, Eran Tromer, and Vitaly Shmatikov.  You autocomplete me: Poisoning vulnerabilities in neural code completion.  In Security, 2021.
  * [105] Kang Liu, Brendan Dolan-Gavitt, and Siddharth Garg.  Fine-pruning: Defending against backdooring attacks on deep neural networks.  In RAID, 2018.
  * [106] Yige Li, Xixiang Lyu, Nodens Koren, Lingjuan Lyu, Bo Li, and Xingjun Ma.  Anti-backdoor learning: Training clean models on poisoned data.  In NeurIPS, 2021.
  * [107] Wenqi Wei and Ling Liu.  Gradient coupling effect of poisoning attacks in federated learning.  In HICSS, 2024.
  * [108] Leslie Lamport, Robert Shostak, and Marshall Pease.  The byzantine generals problem.  ACM TPLS, pages 382–401, 1982.
  * [109] Peva Blanchard, El Mahdi El Mhamdi, Rachid Guerraoui, and Julien Stainer.  Machine learning with adversaries: Byzantine tolerant gradient descent.  In NeurIPS, 2017.
  * [110] Dong Yin, Yudong Chen, Ramchandran Kannan, and Peter Bartlett.  Byzantine-robust distributed learning: Towards optimal statistical rates.  In ICML, 2018.
  * [111] Jeremy Bernstein, Jiawei Zhao, Kamyar Azizzadenesheli, and Anima Anandkumar.  signsgd with majority vote is communication efficient and fault tolerant.  In ICLR, 2018.
  * [112] Dan Alistarh, Zeyuan Allen-Zhu, and Jerry Li.  Byzantine stochastic gradient descent.  In NeurIPS, 2018.
  * [113] Cong Xie, Sanmi Koyejo, and Indranil Gupta.  Zeno: Distributed stochastic gradient descent with suspicion-based fault-tolerance.  In ICML, 2019.
  * [114] Xudong Pan, Mi Zhang, Duocai Wu, Qifan Xiao, Shouling Ji, and Zhemin Yang.  Justinian’s $\\{$GAAvernor$\\}$: Robust distributed learning with gradient aggregation agent.  In Security. USENIX, 2020.
  * [115] Shiyu Liang, Yixuan Li, and R Srikant.  Enhancing the reliability of out-of-distribution image detection in neural networks.  In ICLR, 2018.
  * [116] Kimin Lee, Kibok Lee, Honglak Lee, and Jinwoo Shin.  A simple unified framework for detecting out-of-distribution samples and adversarial attacks.  In NeurIPS, 2018.
  * [117] Xiang Li, Kaixuan Huang, Wenhao Yang, Shusen Wang, and Zhihua Zhang.  On the convergence of fedavg on non-iid data.  In ICLR, 2019.
  * [118] Jason Van Hulse, Taghi M Khoshgoftaar, and Amri Napolitano.  Experimental perspectives on learning from imbalanced data.  In ICML, 2007.
  * [119] Xian Shuai, Yulin Shen, Siyang Jiang, Zhihe Zhao, Zhenyu Yan, and Guoliang Xing.  Balancefl: Addressing class imbalance in long-tail federated learning.  In IPSN. IEEE/ACM, 2022.
  * [120] Zebang Shen, Juan Cervino, Hamed Hassani, and Alejandro Ribeiro.  An agnostic approach to federated learning with class imbalance.  In ICLR, 2022.
  * [121] Lixu Wang, Shichao Xu, Xiao Wang, and Qi Zhu.  Addressing class imbalance in federated learning.  In AAAI, 2021.
  * [122] Wenqi Wei, Ling Liu, Margaret Loper, Ka-Ho Chow, Mehmet Emre Gursoy, Stacey Truex, and Yanzhao Wu.  A framework for evaluating client privacy leakages in federated learning.  In ESORICS, 2020.
  * [123] Yoshinori Aono, Takuya Hayashi, Lihua Wang, Shiho Moriai, et al.  Privacy-preserving deep learning via additively homomorphic encryption.  IEEE TIFS, 13(5):1333–1345, 2017.
  * [124] Junyi Zhu and Matthew Blaschko.  R-gap: Recursive gradient attack on privacy.  In ICLR, 2021.
  * [125] Ligeng Zhu, Zhijian Liu, and Song Han.  Deep leakage from gradients.  In NeurIPS, 2019.
  * [126] Jonas Geiping, Hartmut Bauermeister, Hannah Dröge, and Michael Moeller.  Inverting gradients - how easy is it to break privacy in federated learning?  In NeurIPS, 2020.
  * [127] Kai Yue, Richeng Jin, Chau-Wai Wong, Dror Baron, and Huaiyu Dai.  Gradient obfuscation gives a false sense of security in federated learning.  arXiv preprint arXiv:2206.04055, 2022.
  * [128] Hongxu Yin, Arun Mallya, Arash Vahdat, Jose M. Alvarez, Jan Kautz, and Pavlo Molchanov.  See through gradients: Image batch recovery via gradinversion.  In CVPR, 2021.
  * [129] Dimitar I Dimitrov, Mislav Balunović, Nikola Jovanović, and Martin Vechev.  Lamp: Extracting text from gradients with language model priors.  arXiv preprint arXiv:2202.08827, 2022.
  * [130] Yangsibo Huang, Samyak Gupta, Zhao Song, Kai Li, and Sanjeev Arora.  Evaluating gradient inversion attacks and defenses in federated learning.  In NeurIPS, 2021.
  * [131] Maximilian Lam, Gu-Yeon Wei, David Brooks, Vijay Janapa Reddi, and Michael Mitzenmacher.  Gradient disaggregation: Breaking privacy in federated learning by reconstructing the user participant matrix.  In ICML, 2021.
  * [132] Xiao Jin, Pin-Yu Chen, Chia-Yi Hsu, Chia-Mu Yu, and Tianyi Chen.  Cafe: Catastrophic data leakage in vertical federated learning.  In NeurIPS, 2021.
  * [133] Chong Fu, Xuhong Zhang, Shouling Ji, Jinyin Chen, Jingzheng Wu, Shanqing Guo, Jun Zhou, Alex X Liu, and Ting Wang.  Label inference attacks against vertical federated learning.  In Security, 2022.
  * [134] Martin Abadi, Andy Chu, Ian Goodfellow, Brendan McMahan, Ilya Mironov, Kunal Talwar, and Li Zhang.  Deep learning with differential privacy.  In CCS, 2016.
  * [135] Brendan McMahan, Daniel Ramage, Kunal Talwar, and Li Zhang.  Learning differentially private recurrent language models.  In ICLR, 2018.
  * [136] Robin C Geyer, Tassilo Klein, and Moin Nabi.  Differentially private federated learning: A client level perspective.  arXiv preprint arXiv:1712.07557, 2017.
  * [137] Wenqi Wei, Ka-Ho Chow, Fatih Ilhan, Yanzhao Wu, and Ling Liu.  Model cloaking against gradient leakage.  In ICDM, 2023.
  * [138] Wenqi Wei, Ling Liu, Jingya Zhou, Ka-Ho Chow, and Yanzhao Wu.  Securing distributed sgd against gradient leakage threats.  IEEE TPDS, 2023.
  * [139] Shiva Prasad Kasiviswanathan, Homin K Lee, Kobbi Nissim, Sofya Raskhodnikova, and Adam Smith.  What can we learn privately?  SICOMP, 40(3):793–826, 2011.
  * [140] Úlfar Erlingsson, Vasyl Pihur, and Aleksandra Korolova.  Rappor: Randomized aggregatable privacy-preserving ordinal response.  In CCS, 2014.
  * [141] Mehmet Emre Gursoy, Acar Tamersoy, Stacey Truex, Wenqi Wei, and Ling Liu.  Secure and utility-aware data collection with condensed local differential privacy.  IEEE TDSC, 18(5):2365–2378, 2019.
  * [142] Stacey Truex, Ling Liu, Ka-Ho Chow, Mehmet Emre Gursoy, and Wenqi Wei.  Ldp-fed: federated learning with local differential privacy.  In EdgeSys, 2020.
  * [143] Lichao Sun, Jianwei Qian, and Xun Chen.  Ldp-fl: Practical private aggregation in federated learning with local differential privacy.  In IJCAI, 2021.
  * [144] Naman Agarwal, Ananda Theertha Suresh, Felix Xinnan X Yu, Sanjiv Kumar, and Brendan McMahan.  cpsgd: Communication-efficient and differentially-private distributed sgd.  In NeurIPS, 2018.
  * [145] Qinqing Zheng, Shuxiao Chen, Qi Long, and Weijie Su.  Federated f-differential privacy.  In AISTATS, 2021.
  * [146] Nicolas Papernot, Shuang Song, Ilya Mironov, Ananth Raghunathan, Kunal Talwar, and Ulfar Erlingsson.  Scalable private learning with pate.  In ICLR, 2018.
  * [147] Ran Canetti.  Security and composition of multiparty cryptographic protocols.  Jour. Crypto., 13(1):143–202, 2000.
  * [148] Keith Bonawitz, Vladimir Ivanov, Ben Kreuter, Antonio Marcedone, Brendan McMahan, Sarvar Patel, Daniel Ramage, Aaron Segal, and Karn Seth.  Practical secure aggregation for privacy-preserving machine learning.  In CCS, 2017.
  * [149] Payman Mohassel and Yupeng Zhang.  Secureml: A system for scalable privacy-preserving machine learning.  In S&P, 2017.
  * [150] Peter Kairouz, Sewoong Oh, and Pramod Viswanath.  Secure multi-party differential privacy.  In NeurIPS, 2015.
  * [151] Craig Gentry.  Fully homomorphic encryption using ideal lattices.  In STOC, 2009.
  * [152] Ronald L Rivest, Adi Shamir, and Leonard Adleman.  A method for obtaining digital signatures and public-key cryptosystems.  CACM, 21(2):120–126, 1978.
  * [153] Pascal Paillier.  Public-key cryptosystems based on composite degree residuosity classes.  In Eurocrypt, 1999.
  * [154] Taher ElGamal.  A public key cryptosystem and a signature scheme based on discrete logarithms.  IEEE TIT, 31(4):469–472, 1985.
  * [155] Chengliang Zhang, Suyi Li, Junzhe Xia, Wei Wang, Feng Yan, and Yang Liu.  $\\{$BatchCrypt$\\}$: Efficient homomorphic encryption for $\\{$Cross-Silo$\\}$ federated learning.  In ATC, 2020.
  * [156] Pramod Subramanyan, Rohit Sinha, Ilia Lebedev, Srinivas Devadas, and Sanjit A Seshia.  A formal foundation for secure remote execution of enclaves.  In CCS, 2017.
  * [157] Fan Mo, Hamed Haddadi, Kleomenis Katevas, Eduard Marin, Diego Perino, and Nicolas Kourtellis.  Ppfl: privacy-preserving federated learning with trusted execution environments.  In MobiSys, 2021.
  * [158] Reza Shokri and Vitaly Shmatikov.  Privacy-preserving deep learning.  In CCS, 2015.
  * [159] Jingwei Sun, Ang Li, Binghui Wang, Huanrui Yang, Hai Li, and Yiran Chen.  Soteria: Provable defense against privacy leakage in federated learning from representation perspective.  In CVPR, 2021.
  * [160] Mehmet Emre Gursoy, Ling Liu, Stacey Truex, Lei Yu, and Wenqi Wei.  Utility-aware synthesis of differentially private and attack-resilient location traces.  In CCS, 2018.
  * [161] Yingzhou Lu, Huazheng Wang, and Wenqi Wei.  Machine learning for synthetic data generation: a review.  arXiv preprint arXiv:2302.04062, 2023.
  * [162] Yuxuan Wan, Han Xu, Xiaorui Liu, Jie Ren, Wenqi Fan, and Jiliang Tang.  Defense against gradient leakage attacks via learning to obscure data.  arXiv preprint arXiv:2206.00769, 2022.
  * [163] Stacey Truex, Ling Liu, Mehmet Emre Gursoy, Wenqi Wei, and Ka Ho Chow.  The tsc-pfed architecture for privacy-preserving fl.  In TPS. IEEE, 2021.
  * [164] Olga Ohrimenko, Felix Schuster, Cédric Fournet, Aastha Mehta, Sebastian Nowozin, Kapil Vaswani, and Manuel Costa.  Oblivious $\\{$Multi-Party$\\}$ machine learning on trusted processors.  In Security, 2016.
  * [165] Milad Nasr, Reza Shokri, and Amir Houmansadr.  Comprehensive privacy analysis of deep learning: Stand-alone and federated learning under passive and active white-box inference attacks.  In S&P, 2019.
  * [166] Jinyuan Jia, Ahmed Salem, Michael Backes, Yang Zhang, and Neil Zhenqiang Gong.  Memguard: Defending against black-box membership inference attacks via adversarial examples.  In CCS, 2019.
  * [167] Klas Leino and Matt Fredrikson.  Stolen memories: Leveraging model memorization for calibrated $\\{$White-Box$\\}$ membership inference.  In Security, 2020.
  * [168] Yijue Wang, Chenghong Wang, Zigeng Wang, Shanglin Zhou, Hang Liu, Jinbo Bi, Caiwen Ding, and Sanguthevar Rajasekaran.  Against membership inference attack: Pruning is all you need.  In IJCAI, 2021.
  * [169] Stacey Truex, Ling Liu, Mehmet Emre Gursoy, Wenqi Wei, and Lei Yu.  Effects of differential privacy and data skewness on membership inference vulnerability.  In TPS, 2019.
  * [170] Xinjian Luo, Yuncheng Wu, Xiaokui Xiao, and Beng Chin Ooi.  Feature inference attack on model predictions in vertical federated learning.  In ICDE, 2021.
  * [171] Dario Pasquini, Giuseppe Ateniese, and Massimo Bernaschi.  Unleashing the tiger: Inference attacks on split learning.  In CCS, 2021.
  * [172] Jinyuan Jia and Neil Zhenqiang Gong.  $\\{$AttriGuard$\\}$: A practical defense against attribute inference attacks via adversarial machine learning.  In Security, 2018.
  * [173] Jingtao Li, Adnan Siraj Rakin, Xing Chen, Zhezhi He, Deliang Fan, and Chaitali Chakrabarti.  Ressfl: A resistance transfer framework for defending model inversion attack in split federated learning.  In CVPR, 2022.
  * [174] Florian Tramèr, Fan Zhang, Ari Juels, Michael K Reiter, and Thomas Ristenpart.  Stealing machine learning models via prediction $\\{$APIs$\\}$.  In Security, 2016.
  * [175] Matthew Jagielski, Nicholas Carlini, David Berthelot, Alex Kurakin, and Nicolas Papernot.  High accuracy and high fidelity extraction of neural networks.  In Security, 2020.
  * [176] Binghui Wang and Neil Zhenqiang Gong.  Stealing hyperparameters in machine learning.  In S&P, 2018.
  * [177] Sanjay Kariyappa and Moinuddin K Qureshi.  Defending against model stealing attacks with adaptive misinformation.  In CVPR, 2020.
  * [178] Tribhuvanesh Orekondy, Bernt Schiele, and Mario Fritz.  Prediction poisoning: Towards defenses against dnn model stealing attacks.  In ICLR, 2020.
  * [179] Mika Juuti, Sebastian Szyller, Samuel Marchal, and N Asokan.  Prada: protecting against dnn model stealing attacks.  In EuroS&P, 2019.
  * [180] Hengrui Jia, Christopher A Choquette-Choo, Varun Chandrasekaran, and Nicolas Papernot.  Entangled watermarks as a defense against model extraction.  In Security, 2021.
  * [181] Wenqi Wei, Ka-Ho Chow, Tiansheng Huang, Sihao Hu, Yanzhao Wu, and Ling Liu.  Demystifying data poisoning attacks in distributed learning as a service.  IEEE Trans. on Serv. Comput., 2024.
  * [182] Yuzhe Ma, Xiaojin Zhu Zhu, and Justin Hsu.  Data poisoning against differentially-private learners: Attacks and defenses.  In IJCAI, 2019.
  * [183] Thien Duc Nguyen, Phillip Rieger, Huili Chen, Hossein Yalame, Helen Möllering, Hossein Fereidooni, Samuel Marchal, Markus Miettinen, Azalia Mirhoseini, Shaza Zeitouni, Farinaz Koushanfar, Ahmad-Reza Sadeghi, and Thomas Schneider.  Flame: Taming backdoors in federated learning.  In Security, 2022.
  * [184] Xiaoyuan Liu, Hongwei Li, Guowen Xu, Zongqi Chen, Xiaoming Huang, and Rongxing Lu.  Privacy-enhanced federated learning against poisoning adversaries.  IEEE TIFS, 16:4574–4588, 2021.
  * [185] Zhuoran Ma, Jianfeng Ma, Yinbin Miao, Yingjiu Li, and Robert H Deng.  Shieldfl: Mitigating model poisoning attacks in privacy-preserving federated learning.  IEEE TIFS, 17:1639–1654, 2022.
  * [186] Saeed Mahloujifar, Mohammad Mahmoody, and Ameer Mohammed.  Data poisoning attacks in multi-party learning.  In ICML, 2019.
  * [187] Aristide CY Tossou and Christos Dimitrakakis.  Algorithms for differentially private multi-armed bandits.  In AAAI, 2016.
  * [188] Abhimanyu Dubey and AlexSandy’ Pentland.  Differentially-private federated linear bandits.  In NeurIPS, 2020.
  * [189] Yuanhao Wang, Jiachen Hu, Xiaoyu Chen, and Liwei Wang.  Distributed bandit learning: Near-optimal regret with efficient communication.  In ICLR, 2019.
  * [190] Cynthia Dwork and Adam Smith.  Differential privacy for statistics: What we know and what we want to learn.  JoPC, 1(2), 2010.
  * [191] Aristide Charles Yedia Tossou and Christos Dimitrakakis.  Achieving privacy in the adversarial multi-armed bandit.  In AAAI, 2017.
  * [192] Sébastien Bubeck, Nicolo Cesa-Bianchi, and Gábor Lugosi.  Bandits with heavy tail.  IEEE TIT, 59(11):7711–7717, 2013.
  * [193] Han Shao, Xiaotian Yu, Irwin King, and Michael R Lyu.  Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs.  In NeurIPS, 2018.
  * [194] Abhimanyu Dubey and Alex ‘Sandy’ Pentland.  Cooperative multi-agent bandits with heavy tails.  In ICML, 2020.
  * [195] Peter Landgren, Vaibhav Srivastava, and Naomi Ehrich Leonard.  Distributed cooperative decision-making in multiarmed bandits: Frequentist and bayesian algorithms.  In CDC, 2016.
  * [196] Ilai Bistritz and Amir Leshem.  Distributed multi-player bandits-a game of thrones approach.  In NeurIPS, 2018.
  * [197] Simina Brânzei and Yuval Peres.  Multiplayer bandit learning, from competition to cooperation.  In COLT, 2021.
  * [198] Kwang-Sung Jun, Lihong Li, Yuzhe Ma, and Jerry Zhu.  Adversarial attacks on stochastic bandits.  In NeurIPS, 2018.
  * [199] Fang Liu and Ness Shroff.  Data poisoning attacks on stochastic bandits.  In ICML, 2019.
  * [200] Abhimanyu Dubey and Alex Pentland.  Private and byzantine-proof cooperative decision-making.  In AAMAS, 2020.
  * [201] Dylan J Foster, Claudio Gentile, Mehryar Mohri, and Julian Zimmert.  Adapting to misspecification in contextual bandits.  In NeurIPS, 2020.
  * [202] Colin R Blyth.  On simpson’s paradox and the sure-thing principle.  J. Amer. Statist. Assoc., 67(338):364–366, 1972.
  * [203] Rogier A Kievit, Willem E Frankenhuis, Lourens J Waldorp, and Denny Borsboom.  Simpson’s paradox in psychological science: a practical guide.  Frontiers in psychology, 4:513, 2013.
  * [204] Charles E Gehlke and Katherine Biehl.  Certain effects of grouping upon the size of the correlation coefficient in census tract material.  J. Amer. Statist. Assoc., 29(185A):169–170, 1934.
  * [205] Samuel Barbosa, Dan Cosley, Amit Sharma, and Roberto M Cesar Jr.  Averaging gone wrong: Using time-aware analyses to better understand behavior.  In WWW, 2016.
  * [206] Pengpeng Xu, Helai Huang, Ni Dong, and Mohamed Abdel-Aty.  Sensitivity analysis in the context of regional safety modeling: Identifying and assessing the modifiable areal unit problem.  Accident Analysis & Prevention, 70:110–120, 2014.
  * [207] Zeynep Tufekci.  Big questions for social media big data: Representativeness, validity and other methodological pitfalls.  In ICWSM, 2014.
  * [208] Ricardo Baeza-Yates.  Bias on the web.  CACM, 61(6):54–61, 2018.
  * [209] Eugene Bagdasaryan, Omid Poursaeed, and Vitaly Shmatikov.  Differential privacy has disparate impact on model accuracy.  In NeurIPS, 2019.
  * [210] Ben Hutchinson and Margaret Mitchell.  50 years of test (un) fairness: Lessons for machine learning.  In FAccT, 2019.
  * [211] Cynthia Dwork, Moritz Hardt, Toniann Pitassi, Omer Reingold, and Richard Zemel.  Fairness through awareness.  In ITCS, 2012.
  * [212] Matt J Kusner, Joshua Loftus, Chris Russell, and Ricardo Silva.  Counterfactual fairness.  In NeurIPS, 2017.
  * [213] Toshihiro Kamishima, Shotaro Akaho, Hideki Asoh, and Jun Sakuma.  Fairness-aware classifier with prejudice remover regularizer.  In ECML-PKDD, 2012.
  * [214] Berk Ustun, Yang Liu, and David Parkes.  Fairness without harm: Decoupled classifiers with preference guarantees.  In ICML, 2019.
  * [215] Cynthia Dwork, Nicole Immorlica, Adam Tauman Kalai, and Max Leiserson.  Decoupled classifiers for group-fair and efficient machine learning.  In FAccT, 2018.
  * [216] Muhammad Bilal Zafar, Isabel Valera, Manuel Gomez Rogriguez, and Krishna P Gummadi.  Fairness constraints: Mechanisms for fair classification.  In AISTATS, 2017.
  * [217] Moritz Hardt, Eric Price, and Nati Srebro.  Equality of opportunity in supervised learning.  In NeurIPS, 2016.
  * [218] Blake Woodworth, Suriya Gunasekar, Mesrob I Ohannessian, and Nathan Srebro.  Learning non-discriminatory predictors.  In COLT, 2017.
  * [219] Alexandra Chouldechova.  Fair prediction with disparate impact: A study of bias in recidivism prediction instruments.  Big data, 5(2):153–163, 2017.
  * [220] Lydia T Liu, Sarah Dean, Esther Rolf, Max Simchowitz, and Moritz Hardt.  Delayed impact of fair machine learning.  In ICML, 2018.
  * [221] Arvind Narayanan and Vitaly Shmatikov.  De-anonymizing social networks.  In S&P, 2009.
  * [222] Loredana Caruccio, Domenico Desiato, Giuseppe Polese, Genoveffa Tortora, and Nicola Zannone.  A decision-support framework for data anonymization with application to machine learning processes.  Inf. Sci., 613:1–32, 2022.
  * [223] Jeroen van de Hoven, Giovanni Comandé, Salvatore Ruggieri, Josep Domingo-Ferrer, Francesca Musiani, Fosca Giannotti, Francesca Pratesi, and Marc Stauch.  Towards a digital ecosystem of trust: Ethical, legal and societal implications.  Opinio Juris In Comparatione, (1/2021):131–156, 2021.
  * [224] Roberta Calegari, Andrea Omicini, and Giovanni Sartor.  Explainable and ethical ai: a perspective on argumentation and logic programming.  In AIxIA, pages 19–36, 2020.
  * [225] Mona Flores, Ittai Dayan, Holger Roth, Aoxiao Zhong, Ahmed Harouni, Amilcare Gentili, Anas Abidin, Andrew Liu, Anthony Costa, Bradford Wood, et al.  Federated learning used for predicting outcomes in sars-cov-2 patients.  Research Square, 2021.
  * [226] Lucas Bourtoule, Varun Chandrasekaran, Christopher A Choquette-Choo, Hengrui Jia, Adelin Travers, Baiwu Zhang, David Lie, and Nicolas Papernot.  Machine unlearning.  In S&P, 2021.
  * [227] Shi-Kuo Chang, Giuseppe Polese, Roshan Thomas, and Souvik Das.  A visual language for authorization modeling.  In VL, pages 110–118. IEEE, 1997.
  * [228] Massimiliano Giordano, Giuseppe Polese, Giuseppe Scanniello, and Genoveffa Tortora.  A system for visual role-based policy modelling.  J Vis Lang Comput, 21(1):41–64, 2010.
  * [229] Christine Utz, Martin Degeling, Sascha Fahl, Florian Schaub, and Thorsten Holz.  (un) informed consent: Studying gdpr consent notices in the field.  In CCS, 2019.
  * [230] Pan Zhou, Wenqi Wei, Kaigui Bian, Dapeng Oliver Wu, Yuchong Hu, and Qian Wang.  Private and truthful aggregative game for large-scale spectrum sharing.  IEEE JSAC, 35(2):463–477, 2017.
  * [231] OpenAI.  Gpt-4 technical report.  arXiv preprint arXiv:2303.08774, 2023.
  * [232] Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer.  High-resolution image synthesis with latent diffusion models.  In CVPR, 2022.
  * [233] Nicholas Carlini, Florian Tramer, Eric Wallace, Matthew Jagielski, Ariel Herbert-Voss, Katherine Lee, Adam Roberts, Tom Brown, Dawn Song, Ulfar Erlingsson, et al.  Extracting training data from large language models.  In Security, 2021.
  * [234] Nicholas Carlini, Jamie Hayes, Milad Nasr, Matthew Jagielski, Vikash Sehwag, Florian Tramer, Borja Balle, Daphne Ippolito, and Eric Wallace.  Extracting training data from diffusion models.  arXiv preprint arXiv:2301.13188, 2023.
  * [235] Weixin Chen, Dawn Song, and Bo Li.  Trojdiff: Trojan attacks on diffusion models with diverse targets.  In CVPR, 2023.
  * [236] Sheng-Yen Chou, Pin-Yu Chen, and Tsung-Yi Ho.  How to backdoor diffusion models?  In CVPR, 2023.
  * [237] Jiawen Shi, Yixin Liu, Pan Zhou, and Lichao Sun.  Badgpt: Exploring security vulnerabilities of chatgpt via backdoor attacks to instructgpt.  arXiv preprint arXiv:2304.12298, 2023.
  * [238] Fábio Perez and Ian Ribeiro.  Ignore previous prompt: Attack techniques for language models.  arXiv preprint arXiv:2211.09527, 2022.
  * [239] Wei Du, Yichun Zhao, Boqun Li, Gongshen Liu, and Shilin Wang.  Ppt: Backdoor attacks on pre-trained models via poisoned prompt tuning.  In IJCAI, 2022.
  * [240] Laura Weidinger, John Mellor, Maribeth Rauh, Conor Griffin, Jonathan Uesato, Po-Sen Huang, Myra Cheng, Mia Glaese, Borja Balle, Atoosa Kasirzadeh, et al.  Ethical and social risks of harm from language models.  arXiv preprint arXiv:2112.04359, 2021.
  * [241] Samuel Gehman, Suchin Gururangan, Maarten Sap, Yejin Choi, and Noah A Smith.  Realtoxicityprompts: Evaluating neural toxic degeneration in language models.  In EMNLP, pages 3356–3369, 2020.
  * [242] Xirong Cao, Xiang Li, Divyesh Jadav, Yanzhao Wu, Zhehui Chen, Chen Zeng, and Wenqi Wei.  Invisible watermarking for audio generation diffusion models.  In TPS, 2019.