# Deep Social Force

Sven Kreiss<EMAIL_ADDRESS>
VITA lab, École Polytechnique Fédérale de Lausanne (EPFL)
Lausanne, CH-1015, Switzerland

###### Abstract

The Social Force model introduced by Helbing and Molnar in 1995 is a
cornerstone of pedestrian simulation. This paper introduces a differentiable
simulation of the Social Force model where the assumptions on the shapes of
interaction potentials are relaxed with the use of universal function
approximators in the form of neural networks. Classical force-based pedestrian
simulations suffer from unnatural locking behavior on head-on collision paths.
In addition, they cannot model the bias of pedestrians to avoid each other on
the right or left depending on the geographic region. My experiments with more
general interaction potentials show that potentials with a sharp tip in the
front avoid locking. In addition, asymmetric interaction potentials lead to a
left or right bias when pedestrians avoid each other.

Keywords: Social Force, Coordinate MLP, Fourier Features

##  1 Introduction

The goal is to model pedestrian behavior with a higher fidelity. So far,
pedestrian simulations do not even model people avoiding each other on the
right in continental, western Europe which limits any application of
pedestrian simulations to counter flows: people passing by each other in
corridors. In fact, most force-based pedestrian simulation models display
unnatural behavior when two people need to avoid each other in head-on paths.

This project brings together classical pedestrian simulations, deep neural
networks and differentiable simulations. However, this project will not do a
sweeping black-box replacement of the entire model. The goal is to generalize
the Social Force model (Helbing and Molnar, 1995) with deep learning while
keeping its interpretable nature of forces derived from gradients of scalar
potentials.

### Social Force in Transportation, Robotics and Crowd Modeling.

The Social Force model is a cornerstone of pedestrian modeling. It is a micro-
simulation of pedestrian behavior based on simple interaction potentials that
exhibit complex behaviors at scale. Social Forces are the gradients of
interpretable potentials.

While the interpretability of the interaction potentials is a feature of the
Social Force model, their shape constraint to a family of falling exponential
functions, or any other parametric form, is unnatural. The model would
preserve its interpretability if the interaction potentials were replaced by
arbitrary, continuous and differentiable functions.

### Universal Functions for Interaction Potentials.

This paper describes interaction potentials by universal function
approximators in the form of deep neural networks. In this form, Social Forces
are derived from gradients of the neural network using automatic
differentiation with efficient BackProp (LeCun et al., 2012). The same
automatic differentiation implementation is used to compute the gradients of
these potentials.

The ultimate goal is to use a more expressive model combined with a stochastic
learning technique to infer model parameters from larger amounts of data. It
will still be possible to plot these potentials and to interpret their shapes
as before.

Figure 1:  Illustration of a pedestrian interaction with the Deep Social Force
model. In this interaction, pedestrian $\alpha$ feels a force $F$ that is
created by the gradient of the repulsive potential $V$ that surrounds
pedestrian $\beta$ (illustrated by blue, dashed equipotential lines). From the
predicted path of pedestrian $\alpha$, an optimizer can modify the generic
shape of $V$ so that the prediction will match the true path.

This paper proposes a method that retains the structure and interpretability
of the Social Force model and removes assumptions on the shapes of the
interaction potentials. The interaction potentials underlying the Social Force
model are of particular parametric forms. While the parameters of these
potentials can be adjusted to better approximate real-world data, the shape is
constrained by the choice of the parametric function. Here, these functions
are learned with neural networks without constraining them to particular
shapes. Figure 1, illustrates how different choices of parameters for the
interaction potential influence the predicted path of a pedestrian. Social
interactions among pedestrians are complex and therefore should not rely on a
particular parametric form.

The software is available at https://github.com/svenkreiss/socialforce. All
results are produced with an executable book (Community, 2020) hosted at
https://www.svenkreiss.com/socialforce/. Figures contain deep links to
animated visualizations with this  Animation symbol or to executable Jupyter
notebooks with the  Notebook symbol.

##  2 Related Work

This paper focuses on predicting human navigation patterns in crowded scenes
where human-space and more importantly human-human interactions are key to
model. As briefly introduced in the previous section, there are two general
streams of methods: interpretable low capacity ones and data-driven high
capacity ones with low interpretability.

### Interpretable models.

Most of interpretable models are based on hand-crafted energy potentials that
require some domain expertise. Helbing and Molnar (Helbing and Molnar, 1995)
have highly impacted the field by introducing a pedestrian motion model based
on attractive and repulsive forces referred to as the _Social Force_ model.
This has been shown to achieve competitive results even on modern pedestrian
datasets (Lerner et al., 2007b; Pellegrini et al., 2009). This method was used
extensively to study panic behavior in dense crowds (Helbing et al., 2005;
Helbing and Johansson, 2009; Helbing et al., 2000), applied in robotics (Luber
et al., 2010) and activity understanding (Mehran et al., 2009; Yamaguchi et
al., 2011; Pellegrini et al., 2010; Leal-Taixe et al., 2011; Leal-Taixé et
al., 2014; Choi and Savarese, 2012, 2014). Variants of the model have been
proposed by extending it to a multi-class case (Robicquet et al., 2016) or by
including rotational inertia (Farina et al., 2017). Using a dataset of dense
crowds, Seer (Seer et al., 2012) have calibrated the Social Force parameters
to produce a model with better predictive power. Recently, Chen (Chen et al.,
2018) presented a nice review on all extensions of the Social Force model.

Related approaches have been used to model human-human interactions. Treuille
et. al. (Treuille et al., 2006) use continuum dynamics, Antonini et. al.
(Antonini et al., 2006) propose a Discrete Choice framework and Wang et. al.
(Wang et al., 2008), Tay et. al. (Tay and Laugier, 2008) use Gaussian
processes. Such functions have also been used to study stationary groups (Yi
et al., 2015; Park and Shi, 2015).

Analyzing the potentials behind all these hand-crafted models is in fact the
backbone for interpretability. However, the manually chosen functions behind
these potentials limit the predictability power of these models.

### Non-interpretable high capacity models.

High capacity models learn human-human and human-space interactions in a more
generic, data-driven fashion. They are data hungry with respect to hand-
crafted methods. Yet, with enough data, they outperform previous works in
terms of prediction accuracy. Since 2016, a collection of methods have been
proposed based on Recurrent Neural Networks (RNN) (Pfeiffer et al., 2017;
Alahi et al., 2016; Gupta et al., 2018). The challenge remains on how to
design the RNN architectures, or how to represent interactions. Alahi et al.
(Alahi et al., 2016) suggest a social tensor to represent human-human
interactions. Gupta et al. (Gupta et al., 2018) present an adversarial
training to learn the distribution of trajectories. While these methods have
the capacity to learn complex interactions, it is difficult to understand the
“why” or simply the impact of surrounding humans on each other. Hence, we
propose to take the benefits of both streams of methods and introduce a non-
parametric, Deep Social Force model.

##  3 Social Force Examples

Several toy examples that demonstrate the properties of Social Force were
introduced in the original paper (Helbing and Molnar, 1995). These examples
are re-implemented here with modern software tools and open sourced.

Figure 2: Corridor example. Pedestrians represented by filled circles walk
towards a goal on the right and pedestrians represented by empty circles walk
towards a goal on the left. The size of the circle represents the walking
speed of a pedestrian as in Helbing and Molnar (1995).  Animation.

### Corridor.

Figure 2 shows a reproduction of the corridor example. Pedestrians represented
by filled circles walk towards a goal on the right and pedestrians represented
by empty circles walk towards a goal on the left. The size of the circle
represents the walking speed of a pedestrian as in the original paper. The
original paper included a study on lane-forming behavior that is not
reproduced here. However, once we have generalized interaction potentials, an
extension of this example will show the effect of left-right asymmetries in
interaction potentials in Section 5.

Figure 3: Gate example. Pedestrians represented by filled circles walk towards
a goal on the right and pedestrians represented by empty circles walk towards
a goal on the left. Five pedestrians have already reached the left goal and
four have reached the right goal. At this moment, a group of two pedestrians
is passing towards the right. The size of the circle represents the walking
speed of a pedestrian as in Helbing and Molnar (1995).

### Gate.

Figure 3 shows the gate example. Pedestrians are trying to pass through a gate
to get to the opposite side of the barrier. In the original paper, the
emergent behavior of grouping was observed: groups of a few people pass the
gate at a time. It was surprising that pedestrians did not pass individually
or that not all of pedestrians of one side pass before the other side. While I
was able to reproduce that behavior, I want to point out that this behavior is
only present when the pedestrians have a preferred walking speed distribution,
i.e. pedestrians move at different preferred speeds.

### Baseline Implementation.

All operations are vectorized. The equations of motion are integrated with the
LeapFrog algorithm (Birdsall and Langdon, 2004) with 25 integration steps per
second.

##  4 Method

The goal is to create a more expressive Social Force model, the _Deep Social
Force_ model. The ultimate aim is to calibrate models on large pedestrian
datasets. The expressive detail of the model is adjustable by the number of
hidden units used for the deep neural networks and can increase as more data
becomes available in the future. We have shown in our own work how a Social
Force model can help a robot navigate a crowded space (Chen et al., 2019). We
have also observed the limits of the Social Force model where two robots that
need to pass each other did not effectively negotiate the situation because
they had no preference to pass each other on the right. The aim is to discover
these intuitive behavioral patterns from data with this Deep Social Force
model.

The technical implementation requires two key ingredients. First, we use
automatic differentiation in PyTorch (Paszke et al., 2017) to calculate the
gradients of the interaction potentials to obtain the forces. Second, we want
to avoid imposing assumptions on the potentials from their parametric form so
we use non-parametric potentials in the form of neural networks.

### Automatic Differentiation of Potentials.

With traditional methods, the dimensionality of the parameter space is crucial
for numerical optimizers and gradient approximation. In addition, second-order
optimizers like L-BFGS (Byrd et al., 1995; Broyden, 1970; Fletcher, 1970;
Goldfarb, 1970; Shanno, 1970) need to store an approximation of the function
curvature. Our proposed method provides exact gradients without costly
numerical gradient estimation and uses first-order optimization methods –
Stochastic Gradient Descent (Bottou, 2010) – that are suitable for the
stochastic setting. This is faster and more stable than numeric
differentiation and more flexible than hand-coded derivatives of particular
potentials.

### Universal Potentials.

Deep neural networks are universal function approximators (Cybenko, 1989;
Hornik et al., 1989) in certain limits.

A common parametric potential (Helbing and Molnar, 1995) for human-human
interactions is a falling exponential:

$V_{\alpha\beta}(b)=V^{0}_{\alpha\beta}\exp(-b/\sigma)$ (1)

with $b$ being a function of the distance and velocities of the pedestrians
and where the parameters to optimize $\theta$ are the peak values of the
potential $V^{0}$ and width of the potential $\sigma$, i.e.
$\theta=\\{V^{0},\sigma\\}$. Here, a multi-layer perceptron (MLP) is used as a
non-parametric111The term “non-parametric” refers to functions that can
increase their capacity as more data becomes available; however, non-
parametric functions do have parameters. replacement:

$V_{\alpha\beta}(b)=\textrm{Softplus}\;\;L_{1\times
5}\;\;\textrm{Softplus}\;\;L_{5\times 1}\;\;b$ (2)

where the parameters to optimize $\theta$ are the entries of the linear
operators $\\{L_{1\times 5},L_{5\times 1}\\}$ and the chosen activation
function and output non-linearity here is a Softplus so that the outputs are
bounded on one side only. This MLP replacement removes all hand-engineered
properties of parametric potentials and discovers those and additional
properties from data instead.

The potentials are not directly observed. The potential parameters $\theta$
are inferred through the observed interactions of pedestrian paths. Every loss
function evaluation is a simulation of all pedestrian paths in the given
scenes. Automatic differentiation is used to obtain the gradients of the
potentials. It is the same technique that is used to obtain the gradients for
the parameter update with Stochastic Gradient Descent (SGD) (Bottou, 2010).

### Fourier Features.

Recent progress in neural rendering has recaptured the research communities
interest in Fourier Features (Rahimi et al., 2007; Tancik et al., 2020) as
inputs to coordinate MLPs. We construct multi-dimensional MLPs with Fourier
Features (FF) that take the usual $b$ and in addition the perpendicular
$d_{\perp}$ and parallel distance $d_{\parallel}$ as inputs. The first
operation on these three inputs is a transformation to Fourier Features. In
particular, we use three one-dimensional feature transformations to 64 Fourier
Features each. The complete MLP is:

$\textrm{MLP}(b,d_{\perp},d_{\parallel})=\textrm{Softplus}\;\;L_{1\times
64}\;\;\big{[}\textrm{Softplus}\;\;L_{64\times
64}\big{]}^{3}\;\;\textrm{Softplus}\;\;L_{64\times
192}\;\;\textrm{FF}_{192\times 3}\;\;\begin{bmatrix}b\\\ d_{\perp}\\\
d_{\parallel}\end{bmatrix}\;\;.$ (3)

##  5 Results

Fusing force-based models with deep neural networks is novel and
unconventional. The resulting model has the benefits of deep learning while
still being interpretable.

### Training / Calibration.

The aim is to train deep neural networks – aka to calibrate the parameters of
interaction potentials – with a method that scales to large datasets. The deep
learning community uses stochastic optimization methods that operate on
batches of data and never on the whole dataset at once. While traditional
Social Force models are calibrated with second order methods (e.g. Yamaguchi
et al. (2011); Seer et al. (2012)), we aim to use the first order Stochastic
Gradient Descent (SGD) (Bottou, 2010). SGD can make use of the gradients that
our differentiable simulation provides and allows to operate on small batches
of data.

(a) circle scenes

(b) trajectory interactions

Figure 4:  In (a), five “circle scenes” are shown and one is highlighted in
blue and orange. The primary pedestrian (blue) always moves from left to right
on the x-axis. The other pedestrian (orange) crosses the scene at a random
angle.  Notebook. In (b), the coordinate frame is fixed to the primary
pedestrian and the trajectories of the secondary pedestrian are plotted for
many synthetic scenes. In this particular instance, the interaction potential
is shaped like a distorted diamond.  Notebook.

### Synthetic Scenes.

A “Scene” is comprised of a few seconds of observed pedestrian trajectories,
see Figure 4. Each trajectory is given as a list of $(x,y)$ coordinates
similar to latitudes and longitudes of GPS tracks. We can use the interaction
potentials given by Helbing and Molnar (1995) or create our own to generate
synthetic scenes. The goal is to have a diverse set of these scenes that probe
all aspects of the interaction potential.

### Automatic Differentiation of Potentials.

With traditional methods, the dimensionality of the parameter space is crucial
for numerical optimizers and the first-order gradient approximation. Automatic
differentiation is faster and more stable than numeric differentiation and
more flexible than hand-coded derivatives of particular potentials. For a
small multi-layer perceptron (MLP) with only 10 parameters (1 input, 1 output,
5 hidden units), it is possible to compare the numeric finite difference
method against back-propagation. Because of the enormous reduction in function
evaluations when back-propagated gradients are available, the finite
difference method is more than an order of magnitude slower even on this small
model. Finite difference methods are infeasible for larger MLPs. Below, we
will see a parameter inference on large MLPs for 2D asymmetric potentials with
over 10,000 parameters using back-propagated gradients through a
differentiable simulation.

Figure 5:  Fit of an MLP-based radial potential for human-human interaction to
a single synthetic path of two pedestrians crossing and avoiding each other
(left). The value $V$ is shown (right) as a function of reduced distance $b$.
Notebook.

### Baseline: Parameter Inference for 1D MLP.

The result of an inference of MLP parameters on synthetic data is shown in
Figure 5. This demonstrates that the differentiable simulation can recover the
exact potential as the generative potential with SGD. Note that the inferred
potential is a universal function in the form of an MLP whereas the generating
potential was a falling exponential function.

(a) Social Force (Helbing and Molnar, 1995)

(b) Coordinate MLP with Fourier Features

Figure 6:  A 2D visualization of the standard interaction potential in the
original Social Force model (Helbing and Molnar, 1995) is shown in (a). In
(b), the result of an inference of a high-capacity MLP with about 10,000
parameters on a synthetic simulation that was generated with an asymmetric
interaction potential is shown.  Notebook.

### Asymmetric 2D Potentials

Above, the interaction potential in the one dimension of reduced distance $b$
(Helbing and Molnar, 1995) was generalized with a neural network. Now, we move
to more generic 2D potentials. Figure 6a shows the interaction potential in 2D
for the classical Social Force model. It is an exact ellipse where the
pedestrian is at one of the focal points and the distance of the focal points
of the ellipse is related to the velocity of the pedestrian. The variable $b$
is the semi-minor axis of this ellipse.

To move to 2D, parallel $d_{\parallel}$ and perpendicular $d_{\perp}$
distances are added to the set of input variables. However, I was not able to
achieve any reasonable inference results with this configuration. Inspired by
recent successes in neural rendering, I added a Fourier Feature layer (FF)
(Tancik et al., 2020; Rahimi et al., 2007). The FF layer takes the three input
variables and converts them to 192 Fourier Features. The FF layer itself does
not contain any trainable parameters. The number of MLP parameters have
nevertheless increased to about 10,000 due to the increased input space and
deeper architecture.

This 2D potential with its many parameters has a high capacity to model many
interaction types. With the differentiable simulation and gradient back-
propagation, we can optimize this large dimensional parameter space. On a
simulation that was generated with an asymmetric potential, we can infer that
asymmetry for this 2D MLP and obtain the potential that is shown in Figure 6b.

Figure 7: Filled circles represent pedestrians that move to the right and
empty circles pedestrians that move to the left. Because of the asymmetry of
the new potential, pedestrians prefer to stay on the right side of oncoming
pedestrians which implies that all pedestrians prefer the right side of the
corridor in their direction of movement.  Animation. The corridor example is
based on a scenario in Helbing and Molnar (1995) which is reproduced for
reference in Figure 2.

We now have all ingredients to investigate implications from asymmetric
potentials in the “corridor” scenario. Figure 7 shows a simulation of an
asymmetric potential. It shows that asymmetric potentials impose a left-right
bias and can model people avoiding each other on the right which ultimately
leads to people preferring the right side of the corridor. In this simulation,
from random initializations, after 40s all pedestrians have moved to the right
in their walking direction as to avoid other pedestrians.

(a)

(b)

Figure 8:  In (a), a scene from the “Crowd by Example” dataset (Lerner et al.,
2007a) as provided in the TrajNet++ challenge (Kothari et al., 2021) is shown.
In (b), a first attempt at infering the shape of a 1D MLP potential on this
real world dataset is shown. The MLP is initialized with synthetic data and
then fine-tuned on the real world scene.  Notebook.

### Inference from real data.

Figure 8 shows a real world scenario on the left. Infering an interaction
potential from that scene is difficult. On the right side of this figure, I
share the result of a first attempt. The MLP is initialized with a fit to
synthetic data and then fine-tuned on the real world example from the “Crowd
by Example” dataset (Lerner et al., 2007a) as provided in the TrajNet++
challenge (Kothari et al., 2021). The optimization does not converge and does
not work without the initialization from synthetic data. If one wanted to
force an interpretation of this result, one would highlight the two distinct
regimes below and above $b=1\,\textrm{m}$. For $b<1\,\textrm{m}$, the
potential is repulsive as in the classical Social Force model. For
$b>1\,\textrm{m}$, the potential is attractive. This type of behavior would
have been impossible to discover with a functional constraint to a falling
exponential as in the classical Social Force model (Helbing and Molnar, 1995).
The reason for the attractive nature of the potential is likely due to the
large number of social groups (groups of friends or classmates) in this
dataset. The classical Social Force model actually does have a separate
interaction term for the attractive forces but here this behavior is
discovered from data.

##  6 Conclusion

This paper and the associated open source software provide an entry point to
reproducible research. The open source implementation allows to critically
investigate previous claims.

Most of this paper has focused on toy problems. It is important to understand
what works and what does not in simplified scenarios. However, when applying
these methods to real data, they generally disappoint. Our modeling might
still not be advanced enough and we might need probabilistic path prediction
and evaluation methods. However, it might also be that the premise of
forecasting pedestrian paths from a few seconds of observations of their
$(x,y)$ coordinates is too optimistic for general situations. This modeling
assumption might be more valid for pedestrians in crowded train station
corridors where the main short-term motion is driven by collision avoidance
and less valid for the motion of students across a sparsly populated open
space on a university campus where motion is driven by other factors not
represented in trajectory coordinates.

Acknowledgments

Sven Kreiss was hosted at the VITA lab at EPFL under Prof. Alexandre Alahi for
the duration of this project. This work was funded by the SNSF Spark grant
190677.

## References

  * Alahi et al. (2016) Alexandre Alahi, Kratarth Goel, Vignesh Ramanathan, Alexandre Robicquet, Li Fei-Fei, and Silvio Savarese.  Social lstm: Human trajectory prediction in crowded spaces.  In _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_ , pages 961–971, 2016.
  * Antonini et al. (2006) Gianluca Antonini, Michel Bierlaire, and Mats Weber.  Discrete choice models of pedestrian walking behavior.  _Transportation Research Part B: Methodological_ , 40(8):667–687, 2006.
  * Birdsall and Langdon (2004) Charles K Birdsall and A Bruce Langdon.  _Plasma physics via computer simulation_.  CRC press, 2004.
  * Bottou (2010) Léon Bottou.  Large-scale machine learning with stochastic gradient descent.  In _Proceedings of COMPSTAT’2010_ , pages 177–186. Springer, 2010\.
  * Broyden (1970) Charles George Broyden.  The convergence of a class of double-rank minimization algorithms 1. general considerations.  _IMA Journal of Applied Mathematics_ , 6(1):76–90, 1970.
  * Byrd et al. (1995) Richard H Byrd, Peihuang Lu, Jorge Nocedal, and Ciyou Zhu.  A limited memory algorithm for bound constrained optimization.  _SIAM Journal on Scientific Computing_ , 16(5):1190–1208, 1995.
  * Chen et al. (2019) Changan Chen, Yuejiang Liu, Sven Kreiss, and Alexandre Alahi.  Crowd-robot interaction: Crowd-aware robot navigation with attention-based deep reinforcement learning.  In _2019 International Conference on Robotics and Automation (ICRA)_ , pages 6015–6022. IEEE, 2019.
  * Chen et al. (2018) Xu Chen, Martin Treiber, Venkatesan Kanagaraj, and Haiying Li.  Social force models for pedestrian traffic–state of the art.  _Transport reviews_ , 38(5):625–653, 2018.
  * Choi and Savarese (2012) Wongun Choi and Silvio Savarese.  A unified framework for multi-target tracking and collective activity recognition.  In _Computer Vision–ECCV 2012_ , pages 215–230. Springer, 2012.
  * Choi and Savarese (2014) Wongun Choi and Silvio Savarese.  Understanding collective activitiesof people from videos.  _Pattern Analysis and Machine Intelligence, IEEE Transactions on_ , 36(6):1242–1257, 2014.
  * Community (2020) Executable Books Community.  Jupyter book.  Feb 2020.  doi: 10.5281/zenodo.4539666.
  * Cybenko (1989) George Cybenko.  Approximation by superpositions of a sigmoidal function.  _Mathematics of Control, Signals and Systems_ , 2(4):303–314, 1989.
  * Farina et al. (2017) Francesco Farina, Daniele Fontanelli, Andrea Garulli, Antonio Giannitrapani, and Domenico Prattichizzo.  Walking ahead: The headed social force model.  _PloS one_ , 12(1):e0169734, 2017.
  * Fletcher (1970) Roger Fletcher.  A new approach to variable metric algorithms.  _The computer journal_ , 13(3):317–322, 1970\.
  * Goldfarb (1970) Donald Goldfarb.  A family of variable-metric methods derived by variational means.  _Mathematics of computation_ , 24(109):23–26, 1970.
  * Gupta et al. (2018) Agrim Gupta, Justin Johnson, Li Fei-Fei, Silvio Savarese, and Alexandre Alahi.  Social gan: Socially acceptable trajectories with generative adversarial networks.  In _IEEE Conference on Computer Vision and Pattern Recognition (CVPR)_ , number CONF, 2018.
  * Helbing and Johansson (2009) Dirk Helbing and Anders Johansson.  Pedestrian, crowd and evacuation dynamics.  In _Encyclopedia of complexity and systems science_ , pages 6476–6495. Springer, 2009.
  * Helbing and Molnar (1995) Dirk Helbing and Peter Molnar.  Social force model for pedestrian dynamics.  _Physical review E_ , 51(5):4282, 1995.
  * Helbing et al. (2000) Dirk Helbing, Illés Farkas, and Tamas Vicsek.  Simulating dynamical features of escape panic.  _Nature_ , 407(6803):487, 2000.
  * Helbing et al. (2005) Dirk Helbing, Lubos Buzna, Anders Johansson, and Torsten Werner.  Self-organized pedestrian crowd dynamics: Experiments, simulations, and design solutions.  _Transportation science_ , 39(1):1–24, 2005.
  * Hornik et al. (1989) Kurt Hornik, Maxwell Stinchcombe, and Halbert White.  Multilayer feedforward networks are universal approximators.  _Neural networks_ , 2(5):359–366, 1989.
  * Kothari et al. (2021) Parth Kothari, Sven Kreiss, and Alexandre Alahi.  Human trajectory forecasting in crowds: A deep learning perspective.  _IEEE Transactions on Intelligent Transportation Systems_ , 2 2021\.
  * Leal-Taixe et al. (2011) L. Leal-Taixe, G. Pons-Moll, and B. Rosenhahn.  Everybody needs somebody: Modeling social and grouping behavior on a linear programming multiple people tracker.  In _ICCV Workshops_ , 2011.
  * Leal-Taixé et al. (2014) Laura Leal-Taixé, Michele Fenzi, Alina Kuznetsova, Bodo Rosenhahn, and Silvio Savarese.  Learning an image-based motion context for multiple people tracking.  In _CVPR_ , pages 3542–3549. IEEE, 2014.
  * LeCun et al. (2012) Yann A LeCun, Léon Bottou, Genevieve B Orr, and Klaus-Robert Müller.  Efficient backprop.  In _Neural networks: Tricks of the trade_ , pages 9–48. Springer, 2012.
  * Lerner et al. (2007a) Alon Lerner, Yiorgos Chrysanthou, and Dani Lischinski.  Crowds by example.  _Comput. Graph. Forum_ , 26:655–664, 2007a.
  * Lerner et al. (2007b) Alon Lerner, Yiorgos Chrysanthou, and Dani Lischinski.  Crowds by example.  In _Computer Graphics Forum_ , volume 26, pages 655–664. Wiley Online Library, 2007b.
  * Luber et al. (2010) Matthias Luber, Johannes Andreas Stork, Gian Diego Tipaldi, and Kai Oliver Arras.  People tracking with human motion predictions from social forces.  In _Robotics and Automation (ICRA), 2010 IEEE International Conference on_ , pages 464–469. IEEE, 2010.
  * Mehran et al. (2009) Ramin Mehran, Akira Oyama, and Mubarak Shah.  Abnormal crowd behavior detection using social force model.  In _Computer Vision and Pattern Recognition, 2009. CVPR 2009. IEEE Conference on_ , pages 935–942. IEEE, 2009.
  * Park and Shi (2015) Hyun Soo Park and Jianbo Shi.  Social saliency prediction.  In _CVPR_ , 2015.
  * Paszke et al. (2017) Adam Paszke, Sam Gross, Soumith Chintala, Gregory Chanan, Edward Yang, Zachary DeVito, Zeming Lin, Alban Desmaison, Luca Antiga, and Adam Lerer.  Automatic differentiation in PyTorch.  In _NIPS-W_ , 2017.
  * Pellegrini et al. (2009) Stefano Pellegrini, Andreas Ess, Konrad Schindler, and Luc Van Gool.  You’ll never walk alone: Modeling social behavior for multi-target tracking.  In _Computer Vision, 2009 IEEE 12th International Conference on_ , pages 261–268. IEEE, 2009.
  * Pellegrini et al. (2010) Stefano Pellegrini, Andreas Ess, and Luc Van Gool.  Improving data association by joint modeling of pedestrian trajectories and groupings.  In _Computer Vision–ECCV 2010_ , pages 452–465. Springer, 2010.
  * Pfeiffer et al. (2017) Mark Pfeiffer, Giuseppe Paolo, Hannes Sommer, Juan Nieto, Roland Siegwart, and Cesar Cadena.  A data-driven model for interaction-aware pedestrian motion prediction in object cluttered environments.  _preprint arXiv:1709.08528_ , 2017.
  * Rahimi et al. (2007) Ali Rahimi, Benjamin Recht, et al.  Random features for large-scale kernel machines.  In _NIPS_ , volume 3, page 5. Citeseer, 2007.
  * Robicquet et al. (2016) Alexandre Robicquet, Amir Sadeghian, Alexandre Alahi, and Silvio Savarese.  Learning social etiquette: Human trajectory understanding in crowded scenes.  In _European conference on computer vision_ , pages 549–565. Springer, 2016.
  * Seer et al. (2012) S Seer, N Brändle, and C Ratti.  Kinects and human kinetics: a new approach for studying crowd behavior, arxiv preprint.  _preprint arXiv:1210.2838_ , 2012.
  * Shanno (1970) David F Shanno.  Conditioning of quasi-newton methods for function minimization.  _Mathematics of computation_ , 24(111):647–656, 1970.
  * Tancik et al. (2020) Matthew Tancik, Pratul P Srinivasan, Ben Mildenhall, Sara Fridovich-Keil, Nithin Raghavan, Utkarsh Singhal, Ravi Ramamoorthi, Jonathan T Barron, and Ren Ng.  Fourier features let networks learn high frequency functions in low dimensional domains.  _arXiv preprint arXiv:2006.10739_ , 2020.
  * Tay and Laugier (2008) Meng Keat Christopher Tay and Christian Laugier.  Modelling smooth paths using gaussian processes.  In _Field and Service Robotics_ , pages 381–390. Springer, 2008.
  * Treuille et al. (2006) Adrien Treuille, Seth Cooper, and Zoran Popović.  Continuum crowds.  In _ACM Transactions on Graphics (TOG)_ , volume 25, pages 1160–1168. ACM, 2006.
  * Wang et al. (2008) Jack M Wang, David J Fleet, and Aaron Hertzmann.  Gaussian process dynamical models for human motion.  _Pattern Analysis and Machine Intelligence, IEEE Transactions on_ , 30(2):283–298, 2008.
  * Yamaguchi et al. (2011) Kota Yamaguchi, Alexander C Berg, Luis E Ortiz, and Tamara L Berg.  Who are you with and where are you going?  In _Computer Vision and Pattern Recognition (CVPR), 2011 IEEE Conference on_ , pages 1345–1352. IEEE, 2011.
  * Yi et al. (2015) Shuai Yi, Hongsheng Li, and Xiaogang Wang.  Understanding pedestrian behaviors from stationary crowd groups.  In _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_ , pages 3488–3496, 2015.