# Observing a group to infer individual characteristics

Arshed Nabeel<EMAIL_ADDRESS>Danny Raj M<EMAIL_ADDRESS>Department of
Chemical Engineering,
Indian Institute of Science, Bangalore

###### Abstract

In the study of collective motion, it is common practice to collect movement
information at the level of the group to infer the characteristics of the
individual agents and their interactions. However, it is not clear whether one
can always correctly infer individual characteristics from movement data of
the collective. We investigate this question in the context of a composite
crowd with two groups of agents, each with its own desired direction of
motion. A simple observer attempts to classify an agent into its group based
on its movement information. However, collective effects such as collisions,
entrainment of agents, formation of lanes and clusters, etc. render the
classification problem non-trivial, and lead to misclassifications. Based on
our understanding of these effects, we propose a new observer algorithm that
infers, based only on observed movement information, how the local
neighborhood aids or hinders agent movement. Unlike a traditional supervised
learning approach, this algorithm is based on physical insights and scaling
arguments, and does not rely on training-data. This new observer improves
classification performance and is able to differentiate agents belonging to
different groups even when their motion is identical. Data-agnostic approaches
like this have relevance to a large class of real-world problems where clean,
labeled data is difficult to obtain, and is a step towards hybrid approaches
that integrate both data and domain knowledge.

††preprint: APS/123-QED

With the advent of sophisticated imaging techniques and machine learning
algorithms, many experimental studies in the field of complex systems and
complex flows turn to computer vision to investigate the underlying dynamics
of the individuals (agents). Examples include the study of the motion of an
intruder and the flow of grains in granular flows [1], hydrodynamics of
droplet interactions in a microchannel [2], dynamics of traffic flows [3] and
crowds [4], and schooling and swarming behavior of organisms such as fish [5].
In most of these examples, a camera can be setup for imaging: high-speed if
the time scales of the processes are milliseconds or smaller, under a
microscope if the length scales involved are of the order of microns or even
use a stereo-pair of images to resolve 3D structures [6]. The agents that make
up the complex system are tracked and their velocities are computed from the
measurements. The motion of these agents is a result of both self-propulsion
(or due to external driving) and interactions between the agents. General
interest is to infer the characteristics of the agents and the underlying
interactions from the observations made of the collective [7, 8, 9, 10, 11].

However, the fundamental question of whether it is even possible to accurately
infer the ‘true’ properties of individuals from measurements, still needs to
be resolved. Depending on the context of the problem, inaccuracy in
identification could prove to be detrimental. For instance, consider the case
of dense pedestrian crowds—e.g. in a pilgrimage site or a crowded
crosswalk—where individuals may be forced in the direction of their neighbors
even when they are unwilling or unable to move in that direction. Can an
observer (i.e. a combination of sensors, computer vision, estimator, etc.)
that is continuously collecting and processing information of the movement of
people in a crowd, predict the onset of a stampede (local or global)? In this
article, we explore this question and investigate how the collective dynamics
of the crowd affects an observer’s ability to infer the underlying behavior of
the agents, from measurements at the level of a collective.

A crowd consisting of different groups of agents is called a _composite
crowd_. Our objective is to classify the agents as belonging to one group or
another based on the information of their movement. In the context of dense-
crowds in pilgrimage sites discussed previously, agents could essentially
belong to two categories: Group 1 where agents move in their desired direction
and Group 2 which consists of agents that either want to change their
direction or speed, or simply stop moving. As the agents are differentiated
solely based on the speed (or velocity) of desired motion, one can use a
simpler and more-widely studied conceptualization of a composite crowd to
study this problem [12, 13, 14, 15, 16, 17, 18, 19, 20]; where the two groups
of agents desire to move in opposite directions with the same speed.

A composite crowd of oppositely moving agents exhibit spontaneous laning
phenomena where agents of the same type occupy a common lane oriented in the
desired direction for movement. Once lanes are formed, agents can move freely,
as they do not encounter those of the oppositely moving type. This feature is
observed in a number of systems that include pedestrian movement in a crossing
or a foot bridge—where people rapidly and spontaneously group with others who
are going the same direction [21, 22], and oppositely charged colloids or
driven complex plasmas—where laned arrangements are formed under externally
imposed force fields [23, 24]. From the perspective of classifying agents, it
may be easier to do so when agents occupy distinct lanes. However, it takes
time for lanes to be formed and when crowd densities are high, lane formation
is not guaranteed, during which it is unclear if agents can be classified
accurately. In some model systems where agents are polar and experience a
torque that makes them turn in order to avoid collisions, they can exhibit a
phase transition to a state where lanes are never formed called, the
critically mingled state [17]. Or oppositely moving agents experiencing high
thermal fluctuations (or Brownian activity to mimic panic in pedestrian
crowds), lead to the formation of a jammed state resulting in dynamic freezing
due to heating [25].

The classification problem becomes particularly challenging when clean,
labeled data is not readily available—as is often the case for problems in
real-world crowd dynamics. In these situations, we cannot rely on the
traditional supervised learning techniques [26] for classification. A physics-
informed approach, based on phenomenological understanding of the crowd
dynamics, is appealing in this context. By incorporating insights about the
dynamics, such an approach can reduce the dependence on labeled data.

In this article, we use the bi-disperse crowd with two groups of agent, each
with its own desired direction of motion. We investigate how well can an
observer—collecting information of the how the agents move—classify agents
according to their desired direction. It is reasonable to expect that
information that aids the classification of a particular agent is encoded in
the motion of that agent itself. Therefore, we first examine the performance
of a simple observer that takes into account only the dynamics of a focal
agent. However, since the dynamics of an agent is also driven by collective
effects arising from interactions with other agents, this approach often
produces misclassifications. To address this, we build an improved observer
that explicitly accounts for collective effects on the movement of an
individual. By using a _neighborhood parameter_ to quantify the influence of
neighboring agents, the new observer can decouple an agent’s intrinsic motion
from the motion caused by interactions with other agents. We conclude by
discussing the limitations of the approach and discussing our outlook about
applying such approaches to real crowds.

## Results

Figure 1: Overview of the crowd dynamics and the simple observer algorithm
where an agent is classified based on its velocity. (a) Examples of dynamics
exhibited by the model at different parameter values. Plots show the
horizontal velocities ($\mathbf{v}_{i,j}\cdot\mathbf{e}_{x}$) for the two
groups of agents from a single run of the simulation. The faint blue and red
traces are the individual agent velocities for Group 1 and Group 2 agents
respectively. The thick blue and red traces are the average velocities
(averaged over all agents in the group) for Group 1 and Group 2 agents. The
dotted grey lines show $\pm s_{0}$, the desired $x$-velocities for the two
groups. Insets show snapshots of the model dynamics at different time-points.
The histograms show the distribution of agent velocities for the two groups,
across all time-points, and across 100 simulations. Notice that the histograms
have significant overlap, and histograms of the minority group (red in the top
and bottom plots) are significantly shifted from their respective $s_{0}$.
(Also see Movie S1, appendix F) _Top:_ Example of clustering
($N_{r}=3/14,\rho=0.46,s_{0}=0.5$. _Middle:_ Example of lane formation
($N_{r}=1/2,\rho=0.46,s_{0}=1$). _Bottom:_ Example of jamming
($N_{r}=3/14,\rho=0.58,s_{0}=0.5$). (b) The model crowd consists of two groups
of agents, Group 1 (desired velocity towards the right, shown as blue) and
Group 2 (desired velocity towards the left, shown as red). The observer
observes the positions and velocities of agents in the crowd, and employs a
classifier to classify agents as _Group 1/Group 2_. (c) Classification
algorithm for the simple observer: For each agent, the classifier finds the
average $x$-velocity of the agent within a short time window, and classifies
the agent based on whether this velocity is positive or negative.

### A simple model of crowd dynamics

We use a simple model of bi-disperse crowds (i.e. consisting of two distinct
groups of agents), similar to the models previously used in literature [25,
19]. Agents are arranged in a 2D periodic domain, with a packing density
$\rho$. Each agent has a desired direction of movement, which is same for all
agents in the group but different between the groups (which we call Group 1
and Group 2). Group 1 agents have a desired velocity along the positive
$x$-direction and Group 2 agents in the negative $x$-direction. An agent is
driven by two forces: a _restitution force_ , denoting the intrinsic effort by
the agent to move in the desired direction, and an _inter-agent force_ ,
denoting the interactions between agents. The forces are designed such that
the agents cannot overlap. For mathematical details of the model, see appendix
A. We study how the model dynamics change with respect to two key aspects of
the crowd: the relative sizes of the two groups, quantified by a _number-
ratio_ $N_{r}$ (the fraction of agents in the minority groups), and the effort
by the agents to move in their desired directions, quantified by the
_intrinsic velocity_ $s_{0}$.

In the absence of any obstacles or collisions, an agent approaches its steady-
state velocity $\mathbf{v}_{0,i}$ with a timescale $\tau$. However, when there
are other agents present, the inter-agent interactions affect the movement in
interesting ways. An agent can be blocked or pushed around by other agents in
its path, which can result in diverse dynamics depending on the model
parameters.

When the packing density $\rho$ is not too high (agents can freely move past
each other), lateral migration due to interactions with other agents causes
agents of the same group to find each other passively and form clusters
(Figure 1(a), top panel). The cluster of agents moves together as a unit, and
collisions with the opposite group appear only at its boundary. For these
reasons, clustered state is an absorbing state: i.e., once a cluster is
formed, it does not break easily, but new agents can join the cluster.
Clustering improves mobility of the agents, as the cluster as a whole is able
to better force its way through opposing agents.

When $N_{r}$ is close to $\frac{1}{2}$ (symmetric or nearly symmetric regime),
clustering can eventually lead to formation of system-spanning lanes (Figure
1(b), middle panel). Since each lane consists only of one group of agents,
mobility is maximum (agent speeds are close to $\pm s_{0}$) in the laned
state.

When $\rho$ is high and the intrinsic velocity $s_{0}$ is relatively small,
the crowd can enter a jammed state, where agents meet head-on and do not have
enough space to move past each other (Figure 1(a), bottom panel). For the
symmetric case ($N_{r}=\frac{1}{2}$), this causes the entire assembly to
freeze. For asymmetric cases ($N_{r}<\frac{1}{2}$), due to an imbalance in the
net force, the jammed assembly of agents slowly drifts in the direction
dictated by the majority group. (See Movie S1, appendix F for an animated
visualization of these cases.)

### Observing agents with a simple observer model

The goal of the observer is to observe the dynamics of the system (velocities
and positions of the agents), and make inferences about the agent
properties—in our case, their respective group memberships (Figure 1(b)). In
other words, the observer has to infer the desired directions of motion for
the agents, based on the observed motion of the collective. We start with a
simple observer algorithm that classifies agents only based on its own
movement information. Specifically, the observer uses the average velocity of
the agent (over a window of length $w$) to classify the agent. An agent $i$ is
classified to be Group 1 if the average velocity $v_{i}^{w}$ is positive, and
Group 2 if $v_{i}^{w}$ is negative. Figure 1(c) for a visual representation of
the algorithm, and see appendix A for details.

For a given window, the number of misclassifications is identified by
comparing the labels of the agents predicted by the observer to the ground-
truth labels (the actual desired directions for the agents, which are unknown
to the observer). For each set of values for the parameters $N_{r}$ and
$s_{0}$ we compute the average number of misclassifications ($n_{m}$),
averaged over multiple time windows and multiple independent realizations of
the simulation.

This simple observer algorithm is the most straightforward way to classify
agents. It assumes that the information necessary to classify an agent is
fully contained in its own motion; and does not explicitly account for its
interactions with the surroundings. The simple observer will serve as a
baseline to study the effect of emergent collective dynamics on
classification, before we develop improved classification models.

Figure 2:  Classification performance depends on the properties of the crowd,
_viz._ composition of the composite crowd ($N_{r}$) and the intrinsic speed of
the agents ($s_{0}$). For larger values of $N_{r}$, the agents start to form
clusters and get classified more accurately. As the collective transitions
from a frozen to a mobile state, agents in the majority group exhibit a
faster-is-slower effect, where misclassifications with increasing $s_{0}$. (a)
Number ratio $N_{r}$ has a non-monotonic effect on the number of
misclassifications for the minority group, due to better clustering at higher
$N_{r}$. Average number of misclassifications versus number-ratio $N_{r}$
($s_{0}=0.75$, $\rho=0.58$). Misclassifications for the smaller group (red,
filled circles) and larger group (blue, empty circles) are shown. (b) Number
of independent clusters $c_{in}$ versus $N_{r}$ for the smaller (red, filled
markers) and larger (blue, empty markers) groups. Initial (circle markers) and
final (square markers) values are shown. Insets show snapshots of model
systems at different values of $N_{r}$. Notice the fore-aft asymmetry around
the minority (red) agents, neighbors are more tightly packed on the left than
on the right. (c) The intrinsic speed $s_{0}$ affects misclassifications non-
monotonically, highlighting a _faster-is-slower_ effect for the majority group
agents (see red highlighted region). Number of misclassifications versus
$s_{0}$ for asymmetric (left, $N_{r}=1/3$) and symmetric (right, $N_{r}=1/2$)
cases, for Group 1 (top, red) and Group 2 (bottom, blue) agents. $\rho=0.46$.
(d) Average drift speed of the frozen assembly versus $N_{r}$ for the smaller
group (red, filled circles) and larger group (blue, empty circles). The grey
dotted line is the mixture velocity, given by $s_{0}\left(1-2N_{r}\right)$.

#### Clustering facilitates movement and helps classification

In the context of stampede prevention, a typical $N_{r}$ of interest would be
$\mathcal{O}\left(\frac{1}{N}\right)$, while in the context of pedestrian
traffic in counter flows, it would be $\mathcal{O}\left(\frac{1}{2}\right)$.
Hence, to understand how the relative population sizes of the two groups in
the composite crowd affect the group-dynamics—and consequently the ability of
an observer to classify—we explore the entire range of $N_{r}$ in our study
and compute the average number of misclassifications $n_{m}$.

We identify three regimes in the range
$N_{r}\in\left[\frac{1}{N},\frac{1}{2}\right]$. When $N_{r}\ll\frac{1}{2}$, we
find that the agents in the minority are well separated in space. We call this
the dilute regime, where the agents in the smaller group do not form any
clusters as they move and remain isolated from other agents of the same group.
In this regime, at high enough crowd densities, the agents in the majority
exert a force high enough to entrain (carry) those in the minority. The
isolated agents in turn serve as obstacles to the flow of those in the
majority resulting in fore-aft asymmetric local packing around the agent. When
$N_{r}$ is increased, agents in the minority start to form small clusters as
time progresses. When $N_{r}$ is close to or equal to $\frac{1}{2}$, we
observe the formation of lanes, which increases the overall mobility (see
Figure 1(a)).

We compute the number of misclassifications $n_{m}$ as a function of $N_{r}$,
as shown in Figure 2(a). We plot $n_{m}$ corresponding to the larger and the
smaller groups separately. The trends observed are qualitatively the same
across different $s_{0}$ and $\rho$ (Figure 5). In the dilute regime
($N_{r}\ll\frac{1}{2}$), the minority agents are isolated in a matrix of
majority agents. Hence, nearly all of them will be pushed by the majority
flow, and will end up being misclassified: as more minority agents are added,
all of them will be misclassified, or in other words, $n_{m}$ increases
linearly with $N_{r}$. In the intermediate regime, $n_{m}$ reaches a maximum
point for the smaller group and then begins to decrease as
$N_{r}\simeq\frac{1}{2}$. For the larger group, $n_{m}$ shows a monotonic
increase with $N_{r}$. It is interesting to observe the reduction of $n_{m}$
with $N_{r}$ close to the symmetric regime. When the number of agents in the
minority group increase beyond a critical fraction, they begin to form
clusters, as described earlier. Clusters begin to move through the crowd
faster than when the agents were isolated—movement velocity of the cluster
increases with cluster size. Once the movement commences, it reduces the
probability of agents within the cluster to get misclassified.

The degree of clustering in the system can be quantified by computing the
number of independent clusters $c_{in}$, for the agents in the minority group,
as a function of $N_{r}$ (Figure 2(b))—as the minority agents become more and
more clustered, $c_{in}$ decreases. Note that i) agents were placed randomly
in the crowd in our simulations, ii) if two agents of the same group are first
Voronoi neighbors of each other, then they belong to the same cluster, iii)
the periodic boundary has to be taken into account while identifying the
number of clusters ($c_{in}$), and iv) an isolated agent is counted as its own
cluster. We observe that $c_{in}$ depends on $N_{r}$ in a manner similar to
$n_{m}$ (compare panels (a) and (b) of Figure 2).

In the dilute regime, agents are isolated and each agent will belong to an
independent cluster. As $N_{r}$ increases, we observe the agents to form small
local clusters which flattens the $c_{in}$ curve. At even higher $N_{r}$,
corresponding to the symmetric regime, system-spanning clusters are observed
which result in the reduction of $c_{in}$. As the simulation proceeds, the
number of clusters in the system would decrease due to passive cluster
formation. However, we observe that the qualitative dependence on $N_{r}$ is
still preserved. Hence, it is clear that the trend in the number of
misclassifications observed as a function of the number-ratio of agents in the
minority group ($n_{m}\text{ vs }N_{r}$) is simply due to the propensity of
agents to form clusters in the crowd.

#### Higher intrinsic speed may worsen classification:
a _faster-is-slower_ effect

Another important parameter to consider is $s_{0}$, the intrinsic speed with
which an agent propels in its desired direction. From an effort-viewpoint, one
would expect the agents that try harder (large values of $s_{0}$) to move in
their desired direction more easily. On the other hand, $s_{0}$ is also a
measure of the difference in velocities (in the principle direction of motion)
between the two groups of agents in the composite crowd. Hence, from a
separation-viewpoint, one could expect higher rates of separation between
agents belonging to different groups, for larger difference in property: for
greater values of $s_{0}$, agents can travel larger distances in a given
amount of time.

We compute the number of misclassifications $n_{m}$ as a function of $s_{0}$
for the asymmetric and symmetric cases, as shown in Figure 2(c). We observe
that classification becomes easier as $s_{0}$ increases for the agents in the
minority, as expected. The trend is similar across a range of $N_{r}$ and
$\rho$ (Figure S2). However, for $N_{r}\neq\frac{1}{2}$, we find a non-
monotonic dependence of $n_{m}$ on $s_{0}$ for the agents in the majority.
This suggests that even though the agents are trying harder to move in their
desired directions, they are unable to. Upon further increase in $s_{0}$, the
decreasing trend begins to appear. Similar trends have been observed in other
problems in the pedestrian dynamics and complex-flows communities and is
commonly known as the faster-is-slower (FIS) effect. One of the first examples
of this phenomenon is the case where a group of pedestrians exit a chamber
through a narrow passage [27]. When agents try to exit faster, they tend to
clog more often which results in intermittent flows and increased exit-times
[28]. While correlations can be drawn between the observed FIS effect with
some flow-features in the crowd, the microscopic mechanisms leading to these
features are not always clear [29].

To understand the microscopic origins of the FIS effect in our system, we
investigate the motion of agents within the crowd. When $s_{0}$ is very small,
the agents in the minority are trapped in a matrix of agents of the majority.
This results in a frozen assembly that drifts in the desired direction of the
majority, with a drift speed $s_{d}$. For a completely frozen assembly with no
relative motion between agents, the expected velocity of the mixture of agents
is simply the average of the individual $\mathbf{v}_{0,i}$ over the whole
crowd, which equals $s_{0}(1-2N_{r})$. The drift velocity $s_{d}$ closely the
matches mixture velocity for small $N_{r}$ (Figure 2(d)). When
$N_{r}\simeq\frac{1}{2}$, the system size spanning clusters that are present
lead to local re-arrangement of agents in the assembly that results in slight
deviation in $s_{d}$ at $N_{r}\simeq\frac{1}{2}$ in Figure 2(d).

Agents get trapped when the force that drives them in the direction they
desire to move is smaller than the sum of all the forces due to the
neighboring agents (in the $x$ direction), i.e.,
$\frac{m}{\tau}(\mathbf{v}_{i}\cdot\mathbf{e}_{-x}-s_{0})<[\sum_{\forall j\neq
i}\mathbf{F}_{ij}(d_{ij};\gamma)]\cdot\mathbf{e}_{x}$. When $s_{0}$ increases,
agents in the minority (which were trapped so far) begin to break free and
move through the crowd (Movie S2, appendix F). In other words, the crowd
transitions from a _jammed state_ to a _mobile state_. This disturbs the
movement of the agents in the majority—which were more ‘fluid’ when the agents
in the minority were trapped—giving rise to local jamming and subsequently,
increased misclassification. This transition from the jammed state to the
mobile state is the reason why $n_{m}$ corresponding to the group in majority
increases with $s_{0}$—the observed FIS effect.

### Correcting a misclassification

Figure 3:  Defining and quantifying the neighborhood parameter which depends
on the states of the nearby neighboring agents. (a) The velocity
$\mathbf{v}_{i}$ of an agent can be decomposed into two components:
$\mathbf{v}_{0,i}$, the intrinsic velocity of the agent, and
$\bm{\upphi}_{i}$, the influence of neighbors. As seen, $\mathbf{v}_{0,i}$ and
$\mathbf{v}_{i}$ may have different directions. (b) The neighborhood parameter
$\widehat{\bm{\upphi}}_{i}$ is an estimate of $\bm{\upphi}_{i}$, using
velocities of neighbors. $\widehat{\bm{\upphi}}_{i}$ for an agent $i$ (deep
blue) is determined by the velocities of its neighbors (pale blue and red).
The contribution of neighbor $j$ is $\tilde{\mathbf{v}}_{j}$, the component of
the velocity vector $\mathbf{v}_{j}$ towards $i$. $\widehat{\bm{\upphi}}_{i}$
is a weighted sum of $\tilde{\mathbf{v}}_{j}$’s, weighted by a function of the
distance between $j$ and $i$.

The simple observer algorithm classifies agents based on their net velocity
measured over a time window $w$. For instance, when the observed velocity
$v_{x}^{w}$ of a Group 2 agent is positive, it will be classified as Group 1
incorrectly by the observer. Thus, correcting a misclassification would
require the observer to classify the agent as belonging to Group 1 even if it
exhibits a net movement in the negative direction. From the perspective of an
observer, this is a counter-intuitive step: the observer has to differentiate
agents in Group 2 from those in Group 1 that happen to move in the same
x-direction. We need to ascertain the conditions under which an observer
should swap the identified group-identity of the agent to correct a possible
misclassification event.

#### An agent’s neighborhood encodes information about its intrinsic direction

In the absence of interactions, an agent moves in its desired direction of
motion with velocity $s_{0}$, and classification is a trivial task. However,
in the previous sections, we observed that inter-agent dynamics affect the
dynamics in interesting ways, making agent classification a harder problem. An
agent’s movement is influenced by both its own intrinsic motion, as well as
interaction-effects from its neighbors which can aid or hinder the motion. For
improved classification performance, an observer needs to decouple the
component of an agent’s motion due to its intrinsic drive and the component of
motion caused by these interaction effects.

In the current formulation of the classifier, the observer uses $v_{i}$ as a
proxy for the agent’s desired velocity, $\mathbf{v}_{0,i}$. However, in
reality, the observed $\mathbf{v}_{i}$ is a combination of the agent’s effort
to move in $\mathbf{v}_{0,i}$ and the support or resistance offered by the
neighborhood $\bm{\upphi}_{i}$.

$v_{i}=\mathbf{v}_{0,i}\cdot\mathbf{e}_{x}+\bm{\upphi}_{i}\cdot\mathbf{e}_{x}$
(1)

See Figure 3(a) for an illustration. With this definition (Equation 1), the
new classification criterion becomes,

$\displaystyle\mathbf{v}_{0,i}\cdot\mathbf{e}_{x}=v_{i}-\phi_{i}>0\text{:
group 1}$ (2)
$\displaystyle\mathbf{v}_{0,i}\cdot\mathbf{e}_{x}=v_{i}-\phi_{i}<0\text{:
group 2}$ (3)

where $\phi_{i}=\bm{\upphi}_{i}\cdot\mathbf{e}_{x}$. In other words,

$\displaystyle v_{i}>\phi_{i}\text{: group 1}$ (4) $\displaystyle
v_{i}<\phi_{i}\text{: group 2}$ (5)

We now discuss an approach to estimate $\bm{\upphi}_{i}$. As mentioned before,
we use information from the local neighborhood to compute a _neighborhood
parameter_ $\widehat{\bm{\upphi}}_{i}$, which will serve as a surrogate for
$\bm{\upphi}_{i}$. To define $\widehat{\bm{\upphi}}_{i}$, we make the
following observations and assumptions: i) an agent’s motion is influenced
predominantly by its immediate neighbors, ii) a neighbor can either aid or
oppose the intrinsic movement of the agent, iii) the influence of a neighbor
on an agent is dependent on the distance between them, and iv) the relative
velocity of the neighbors, i.e. how fast the neighbor is approaching the
position of the agent, can give an indication of the level of influence the
neighbor has.

Based on this, we define $\widehat{\bm{\upphi}}_{i}$ as a weighted sum of
neighbor velocities, weighted by a function of the distance to the neighbor.
The sum is scaled by a coefficient $\mu$, which needs to be chosen
appropriately to maximize classification performance. Appendix A provides
details on the computation of $\widehat{\bm{\upphi}}$; see Figure 3(b) for a
simplified illustration. One could use a data-driven approach to estimate
$\mu$, by fitting a classifier model to labeled data. Since labeled data could
be difficult to obtain in a general real world setting, a phenomenological
approach to the problem becomes relevant. By enforcing that $v_{i}$ and
$\mu\varphi_{i}$ needs to be of the same scale, we estimate $\mu$ from the
parameters of the model using a scaling analysis (see appendix A).

Recall that our classification criterion was $v_{i}\lessgtr\phi_{i}$. In terms
of the surrogate, this becomes $v_{i}\lessgtr\widehat{\phi}_{i}$, or
$v_{i}\lessgtr\mu\varphi_{i}$, where
$\widehat{\bm{\upphi}}_{i}=\mu\bm{\upvarphi}_{i}$ and
$\varphi_{i}=\bm{\upvarphi}_{i}\cdot\mathbf{e}_{x}$. When $\mu=0$, this
reduces to the simple, agent-only observer algorithm. From a classification
viewpoint, we have two features $v_{i}$ and $\varphi_{i}$ and the linear
separator that divides the space into the binary classes (group 1 and 2), is
$v_{i}=\mu\varphi_{i}$.

It is important to note that the neighborhood parameter is independent of the
details of the simulation model. We use relative velocities of the neighbors
as a proxy for inter-agent interactions; in other words, the neighborhood
parameter only depends on the overall phenomenology of the crowd, and not on
specific modeling assumptions.

Figure 4:  The neighborhood observer classifies agents in the minority-group
more accurately at the expense of those in the majority-group, improving the
overall performance for asymmetric crowds. (a) Visualizing the working of the
neighborhood classifier: The top panel shows an example where the neighborhood
observer works well. The blue solid line is the observed velocity $v_{i}^{w}$
of a Group 1 agent, and the blue dashed line is the neighborhood parameter
$\phi_{i}^{w}$. The blue shaded regions are the time-points when the agent-
only observer misclassifies the agent as Group 2, but the neighborhood
observer correctly identifies as Group 1. The bottom panel shows an example
where the neighborhood observer makes an error. In the red shaded region, the
neighborhood observer classifies the agent incorrectly as Group 2, despite the
agent moving in the positive $x$-direction. (b) Comparison of classification
performances of the two observers. In the asymmetric case ($N_{r}=16/21$), the
neighborhood observer makes fewer errors for the minority group (Group 2, red
bars), but makes slightly more errors for the majority group (Group 1, blue
bars). The total number of errors is lower for the neighborhood observer. In
the symmetric case, the neighborhood observer performs worse than the agent-
only observer. ($s_{0}=0.75,\rho=0.58$) (c) A schematic showing the geometry
of classification. the agent-only observer and the neighborhood observer will
have different predictions for agents in the shaded wedge-shaped region
(marked A, B). Group 1 agents (blue) in A and Group 2 agents (red) in B will
be correctly flipped by the neighborhood observer. On the other hand, Group 1
agents in B and Group 2 agents in A will be incorrectly flipped. The relative
classification performances of the observers is determined by the number of
correct versus incorrect flips.

#### Observing agents with the neighborhood-based observer

In what follows, we refer to the simple observer algorithm the _agent-only
observer_ , and the new algorithm as the _neighborhood observer_. In addition
to computing the movement characteristics of just the agent $v_{i}$, the
neighborhood observer calculates $\varphi_{i}$, which is a characteristic of
its neighborhood. As before, we use the time-window-averaged versions of these
quantities, denoted $v_{i}^{w}$ and $\phi_{i}^{w}$ respectively. The parameter
$\mu$ is calculated based on the knowledge of the global properties of the
system—$\rho$ and $N_{r}$.

Figure 4(a) illustrates how the neighborhood observer works. The plot shows
the time-series of the (window-averaged) agent velocity $v_{i}^{w}$ for a
Group 1 agent. When it encounters Group 2 agents in its path, it may get
pushed, causing $v_{i}^{w}$ to dip below 0. The agent-only observer would then
classify the agent—incorrectly—as Group 2: see blue shaded regions in the
plot. However, the classification threshold for the neighborhood observer is
$\phi_{i}^{w}$ (blue dashed line in the plot), incorporates information about
the neighborhood of the agent, and can vary according to how much the agent is
being pushed. Hence, the neighborhood observer is able to correctly classify
the agent as Group 1 even when $v_{i}^{w}<0$, i.e. the agent is moving in the
negative x-direction. On the other hand, if the neighborhood observer over-
estimates $\phi_{i}^{w}$, we may misclassify agents even when the agent-only
observer correctly classifies them. The bottom plot shows an example for a
Group 1 agent moving in a well-laned configuration. Since all neighbors are
moving with high velocity, the neighborhood observer occasionally over-
estimates $\phi_{i}^{w}$ in the red shaded region and misclassifies the agent
as Group 2; even though it is moving in the positive x-direction.

The stacked bar graphs in Figure 4(b) summarize the relative performances of
the two observers. For asymmetric crowds ($N_{r}<1/2$), the neighborhood
observer performs better than the agent-only observer in terms of the total
number of misclassifications: it does so by significantly improving the
performance on the minority group, at the cost of a slight performance hit for
the majority group. For symmetric crowds, the neighborhood observer performs
slightly worse than the agent-only observer. To understand the reason behind
this trend, we explore the mechanism of the neighborhood observer though a
geometric lens.

#### Geometry of the neighborhood classifier

The neighborhood observer can alternatively be viewed as a linear classifier
in the $\varphi_{i}$–$v_{i}$ plane, with zero intercept and slope $1/\mu$ (see
Equation 21). We now take a closer look at the geometry of the classification
algorithm of the two observers. Consider the joint distribution of $v_{i}^{w}$
versus $\varphi_{i}^{w}$. The agent-only observer does not take
$\varphi_{i}^{w}$ into consideration, hence its decision boundary corresponds
to the vertical line through the origin. On the other hand, the decision
boundary of the neighborhood observer is a line through the origin with slope
$1/\mu$. The wedge-shaped regions in the first and third quadrants, marked A
and B in Figure 4(c), are the regions where the predictions are different
between the two observers. The relative performance of the neighborhood
observer over the agent-only observer can be understood by taking a closer
look at these regions.

Recall that in our simulations, the minority group is always Group 2, which
desires to move in the negative x-direction. A Group 2 agent in Region A (and
a Group 1 agent in Region B) will be incorrectly classified by the agent-only
observer, but correctly classified by the neighborhood observer. On the other
hand, a Group 1 agent in Region A (and a Group 2 agent in Region B) will be
correctly classified by the agent-only observer but incorrectly classified by
the neighborhood observer. In other words, the neighborhood observer improves
the classification accuracy of Group 2 agents in Region A, at the cost of
Group 1 agents. A similar statement can be made about Region B. The relative
classification performances of the observers can now be understood in terms of
the relative concentrations of agents in these regions.

We now discuss the effect of different model parameters on the classification
performance of the neighborhood observer.

#### Neighborhood observer improves classification for asymmetric crowds

For asymmetric crowds ($N_{r}<\frac{1}{2}$), the neighborhood observer
outperforms the agent-only observer by making a considerable improvement in
classifying the minority group agents while performing slightly worse on the
majority group (Figure 4(b)). In the asymmetric regime, agents from the
minority group (Group 2) are more prone to get pushed by agents from the
majority group (Group 1), causing the agent-only observer to misclassify them.
However, the neighborhood of any agent will consist mostly of majority agents,
causing $\phi_{i}^{w}$ to be large. Hence, the neighborhood observer is able
to correctly identify many of these agents. That is, even though these agents
are being pushed in the positive x-direction, the neighborhood observer
correctly classifies them as Group 2. However, this comes at a slight cost of
classification performance for the majority group—since $\phi_{i}^{w}$ is
usually large, a slow-moving majority agent can have $v_{i}^{w}<\phi_{i}^{w}$
and can get misclassified.

For symmetric crowds, no one group gets an advantage over the other in getting
classified correctly.

#### Dense and slow crowds are harder to classify for both observers

When density $\rho$ is small, the agents are sparsely distributed and have
relatively more space to move around. This means a clear separation of the
$v_{i}-\phi_{i}$ distributions of Group 1 and Group 2 agents (Figure 8),
resulting in almost no misclassifications for either observer. As density
increases, there is less space for free movement and the agents are blocked
more often: this causes a greater overlap in the distributions, making
classification harder for both observers.

The intrinsic velocity $s_{0}$ has an effect complementary to $\rho$ in the
classification performance. When $s_{0}$ is low, the agents move almost like a
frozen assembly, resulting in a high number of misclassifications for the
agent-only observer. However, since $\phi$ is also correspondingly low, there
is a large overlap in the distributions of the two groups. Hence, the
neighborhood observer cannot improve performance very much.

(See Figure 5, 6 for a quantitative comparison between the performances of the
two observers as a function of the parameters $N_{r}$, $\rho$ and $s_{0}$.)

#### Modes of error for the neighborhood observer

Figure 4(c) shows the key features of the neighborhood classifier in terms of
the distributions of $v_{i}^{w}$ and $\varphi_{i}^{w}$. In the ideal case, the
classification boundary should split the Group 1 and Group 2 agents (blue and
red points in the figure respectively). However, this is not always the case,
and errors can occur due to two main reasons.

Firstly, the classification boundary may ‘clip’ the distributions of Group 1
and Group 2 near high-velocity regions. For example, for symmetric or near-
symmetric crowds, the neighborhood observer is found to clip a small portion
of the regions corresponding to large $|v_{i}^{w}|$ (see Figure 4). This
usually happens when the agent is moving in a well-laned configuration (Movie
S3, appendix F). As an assembly of agents interact to form lanes, they form
heterogeneous regions with high and low densities. When agents are densely
packed locally, they have a larger $\varphi_{i}$ than that predicted by the
scaling which uses the length scale $\frac{1}{\sqrt{\rho}}$, resulting in
inaccurate classifications. Note that the clipping effect is due to mis-
estimation of the parameter $\mu$; an alternate classifier with a slightly
rotated decision boundary can get rid of the problem. An alternate approach
would be a non-linear classifier that takes into account the fact that agents
with high horizontal velocities are highly likely to be moving on their own
volition (and not pushed).

Secondly, in any region in the distribution where Group 1 and Group 2 overlap,
the observer is bound to make some misclassifications. The size of this region
depends on parameters $s_{0}$ and $\rho$, and contains instances of slow-
moving agents: usually caught in a blockade. This is a fundamental limitation
that may be difficult to overcome: in these overlap regions, there is no
discernible difference between the groups in the velocities or neighborhood
characteristics between the two groups. Even an optimal classifier trained
from data (Figure 9) is unable to significantly improve classification
performance in this scenario.

## Discussion

In this article, we set out to understand how the classification process of an
observer is connected to the microscopic details of the crowd dynamics. We
begin with a simple observation/classification technique based on the measured
velocity of agents (averaged over a small time window). We show how the
relative population size of the groups can affect the way clusters are formed
as agents move, which affect the propensity to get classified accurately. We
find that a crowd can exhibit counter-intuitive dynamics, where agents get
misclassified a lot more when they try harder to move past each other—similar
to the so-called faster-is-slower (FIS) effect. As the intrinsic velocity
increases, we find a transition between a frozen state—where minority agents
are embedded in a matrix of the majority agents—to a mobile state—where the
there is relative motion between agents. This transition to the mobile state
causes obstructions to the movement of the majority agents, giving rise to the
FIS effect mentioned above.

Going further, we developed an observer that classifies an agent based on
information not just about itself, but also about its neighbors. Although
simple in principle, this neighborhood observer does something quite non-
trivial: it distinguishes between a Group 1 and a Group 2 agent even when they
are moving in the same direction with identical velocities. The fact that the
neighborhood observer takes into account the influence of the neighboring
agents for classification helps it to ‘read the mind’ of the focal agent by
distinguishing when it is moving on its own volition versus when it is being
pushed.

Note that both the observers treat the time windows as independent: when
making predictions for a given time window, the observers do not use
information from previously observed windows. One could conceive an observer
that uses historical information and updates beliefs with time, perhaps in a
Bayesian fashion. We do not explore this approach; our goal is to explore
techniques that work with minimal observations, and to understand how
instantaneous and short-term dynamics in the system affects our ability to
classify agents.

The 2D classifier that the neighborhood observer employs is not data-driven:
the decision boundary is not obtained by a data fit, but is instead derived
from scale considerations. This makes our approach applicable in scenarios
where labeled data is not easy to obtain. This also makes our classifier
readily interpretable, as it is well-grounded in the physics of the problem.
The assumptions the classification algorithm makes about the details of the
model are minimal. Specifically, the neighborhood parameter is computed based
on neighbor velocities only, and is independent of specific modeling
assumptions. It is not based on the specific nature and functional form of
inter-agent interactions.

We now discuss some ways to improve the classification performance of the
neighborhood observer. Firstly, if labeled data (i.e. data where the group
memberships of agents are known) is readily available, we can fit a linear
classifier to the data to get improved results (Figure 9). When a classifier
is fit in this manner, it will not be susceptible to unfulfilled assumptions
in computing $\mu$, such as mismatch between local and global densities.
Secondly, notice that an agent moving in its desired direction will move
faster than when it is pushed in the opposite direction. If we can estimate
the maximum velocity with which an agent can move when being pushed, this can
be incorporated into a classification logic. The classifier will now be a
hybrid algorithm, with hard velocity thresholds for which it will be
classified into Group 1 or Group 2, and a region in between where a linear
classifier akin to the neighborhood observer operates. This will get rid of
the problem of clipping we observed above. The hard thresholds could be
derived from data, making this a hybrid classifier.

An orthogonal approach is to build a purely data-driven classifier based on
neural networks. One could build a neural-net classifier that takes as input
the positions and velocities of the focal agent and its neighbors, and makes
predictions. Given enough labelled data, such classifiers can learn high-
quality, low-dimensional feature representations that can make effective
predictions. To be effective, such neural network classifiers should be
informed and constrained by the physics and inherent symmetries of the
problem. In our case, it means that the neural network should respect the
rotational and translational invariances of the system, and should be
invariant to any permutation of the agent ordering. Constraints such as these
should be built in to the neural network design. There is an emerging body of
research in graph neural networks [30, 31] and physics-informed deep learning
[32, 33] which can be explored in this regard. Building neural network
classifiers for collective dynamics and studying their feature representations
is an exciting research direction.

Classifying agents in a real crowd is a much harder problem to study in
comparison to the model system we investigate in this article. We assume the
composite crowd to be bi-disperse—consisting of two groups of agents and
uniform—indistinguishable from each other. However, agents in a real crowd are
seldom uniform and one does not really know the total number of distinct
groups they fall under. Note here that in our study, we assumed that the
observer knows the crowd to be bi-disperse. In fact, when the observer
attempts to infer the rule that the agents follow, it never uncovers the same
value for $v_{0,i}$. Almost always, the observer infers a different value for
each agent in a certain group. Simply put, even a uniform group of agents are
identified as non-uniform due to the differences in the conditions they
experience as they move in the crowd. Hence, it will be even harder to
identify agents in poly-disperse settings (real crowds) where the number of
groups are not known apriori.

Therefore, it is worth reiterating that the goal of this study is to
understand the ways in which the microscopic properties of a crowd affect the
classification performance of an observer that is only privy to observables
such as agent positions and velocities. We study this in the context of a
model system, which is an idealised version of collective systems in the real
world. For this reason, hunting for the best data-driven classifier to solve
this specific problem may not be very useful; such a classifier will not be
transferable to a real-world crowd as real-world crowds can be polydisperse
and can involve many complex interactions not included in our model. Hence,
our main goal was to understand how the microscopic properties and inter-agent
interactions can affect classification. A clear understanding of this
relationship is essential to solve similar problems in real-world crowds. Our
phenomenological approach to the classification problem is a first step
towards hybrid techniques, where a data-driven approach is combined with
domain-specific understanding of the crowd dynamics to build better observer
models.

###### Acknowledgements.

The authors would like to thank DST INSPIRE faculty award for the funding. The
authors thank Jason Ryan Picardo, Ganapathy Ayappa, Vishwesha Guttal, Vivek
Jadhav, Basil Thurakkal and Harishankar Muppirala for critical readings of the
manuscript.

## References

  * Candelier and Dauchot [2010] R. Candelier and O. Dauchot, Journey of an intruder through the fluidization and jamming transitions of a dense granular media, Physical Review E - Statistical, Nonlinear, and Soft Matter Physics 81, 1 (2010), arXiv:0909.4628 .
  * Thorsen  _et al._ [2001] T. Thorsen, R. W. Roberts, F. H. Arnold, and S. R. Quake, Dynamic Pattern Formation in a Vesicle-Generating Microfluidic Device, Physical Review Letters 86, 4163 (2001).
  * Kamijo  _et al._ [1999] S. Kamijo, Y. Matsushita, K. Ikeuchi, and M. Sakauchi, Traffic monitoring and accident detection at intersections, IEEE Conference on Intelligent Transportation Systems, Proceedings, ITSC 1, 703 (1999).
  * Murino  _et al._ [2017] V. Murino, M. Cristani, S. Shah, and S. Savarese, The Group and Crowd Analysis Interdisciplinary Challenge, in  _Group and Crowd Behavior for Computer Vision_ (Elsevier, 2017) pp. 1–11.
  * Jhawar  _et al._ [2020] J. Jhawar, R. G. Morris, U. Amith-Kumar, M. D. Raj, T. Rogers, H. Rajendran, and V. Guttal, Noise-induced schooling of fish, Nature Physics 16, 488 (2020).
  * Cavagna and Giardina [2008] A. Cavagna and I. Giardina, The seventh starling, Significance 5, 62 (2008).
  * Herbert-Read  _et al._ [2011] J. E. Herbert-Read, A. Perna, R. P. Mann, T. M. Schaerf, D. J. T. Sumpter, and A. J. W. Ward, Inferring the rules of interaction of shoaling fish, Proceedings of the National Academy of Sciences 108, 18726 (2011), https://www.pnas.org/content/108/46/18726.full.pdf .
  * Klemm  _et al._ [2012] K. Klemm, M. Á. Serrano, V. M. Eguíluz, and M. San Miguel, A measure of individual role in collective dynamics, Scientific reports 2, 1 (2012).
  * Lukeman  _et al._ [2010] R. Lukeman, Y.-X. Li, and L. Edelstein-Keshet, Inferring individual rules from collective behavior, Proceedings of the National Academy of Sciences 107, 12576 (2010), https://www.pnas.org/content/107/28/12576.full.pdf .
  * Katz  _et al._ [2011] Y. Katz, K. Tunstrøm, C. C. Ioannou, C. Huepe, and I. D. Couzin, Inferring the structure and dynamics of interactions in schooling fish, Proceedings of the National Academy of Sciences 108, 18720 (2011), https://www.pnas.org/content/108/46/18720.full.pdf .
  * Heras  _et al._ [2019] F. J. H. Heras, F. Romero-Ferrero, R. C. Hinz, and G. G. de Polavieja, Deep attention networks reveal the rules of collective motion in zebrafish, PLOS Computational Biology 15, 1 (2019).
  * Muramatsu  _et al._ [1999] M. Muramatsu, T. Irie, and T. Nagatani, Jamming transition in pedestrian counter flow, Physica A: Statistical Mechanics and its Applications 267, 487 (1999).
  * Dzubiella  _et al._ [2002] J. Dzubiella, G. P. Hoffmann, and H. Löwen, Lane formation in colloidal mixtures driven by an external field, Physical Review E - Statistical Physics, Plasmas, Fluids, and Related Interdisciplinary Topics 65, 1 (2002).
  * Chakrabarti  _et al._ [2003] J. Chakrabarti, J. Dzubiella, and H. Löwen, Dynamical instability in driven colloids, Europhysics Letters 61, 415 (2003).
  * Delhommelle [2005] J. Delhommelle, Should ”lane formation” occur systematically in driven liquids and colloids?, Physical Review E - Statistical, Nonlinear, and Soft Matter Physics 71, 1 (2005).
  * Tao and Dong [2017] Y. Tao and L. Dong, A Cellular Automaton model for pedestrian counterflow with swapping, Physica A: Statistical Mechanics and its Applications 475, 155 (2017).
  * Bain and Bartolo [2017] N. Bain and D. Bartolo, Critical mingling and universal correlations in model binary active liquids, Nature Communications 8, 1 (2017), arXiv:1612.02565 .
  * Poncet  _et al._ [2017] A. Poncet, O. Bénichou, V. Démery, and G. Oshanin, Universal Long Ranged Correlations in Driven Binary Mixtures, Physical Review Letters 118, 118002 (2017), arXiv:1608.00094 .
  * Reichhardt and Reichhardt [2018] C. Reichhardt and C. J. Reichhardt, Velocity force curves, laning, and jamming for oppositely driven disk systems, Soft Matter 14, 490 (2018), arXiv:arXiv:1707.09438v1 .
  * Liu  _et al._ [2019] L. Liu, K. Li, X. L. Zhou, L. L. He, and L. X. Zhang, Controllable laning phase for oppositely driven disk systems, Chinese Physics B 28, 10.1088/1674-1056/ab4e86 (2019).
  * Helbing  _et al._ [2005] D. Helbing, L. Buzna, A. Johansson, and T. Werner, Self-Organized Pedestrian Crowd Dynamics: Experiments, Simulations, and Design Solutions, Transportation Science 39, 1 (2005).
  * Feliciani and Nishinari [2016] C. Feliciani and K. Nishinari, Empirical analysis of the lane formation process in bidirectional pedestrian flow, Physical Review E 94, 10.1103/PhysRevE.94.032304 (2016).
  * Sütterlin  _et al._ [2009] K. R. Sütterlin, A. Wysocki, A. V. Ivlev, C. Räth, H. M. Thomas, M. Rubin-Zuzic, W. J. Goedheer, V. E. Fortov, A. M. Lipaev, V. I. Molotkov, O. F. Petrov, G. E. Morfill, and H. Löwen, Dynamics of lane formation in driven binary complex plasmas, Physical Review Letters 102, 1 (2009), arXiv:0812.3091 .
  * Vissers  _et al._ [2011] T. Vissers, A. Wysocki, M. Rex, H. Löwen, C. P. Royall, A. Imhof, and A. Van Blaaderen, Lane formation in driven mixtures of oppositely charged colloids, Soft Matter 7, 2352 (2011).
  * Helbing  _et al._ [2000a] D. Helbing, I. J. Farkas, and T. Vicsek, Freezing by Heating in a Driven Mesoscopic System, Physical Review Letters 84, 1240 (2000a).
  * Hastie  _et al._ [2009] T. Hastie, R. Tibshirani, and J. Friedman, Overview of supervised learning, in  _The elements of statistical learning_ (Springer, 2009) pp. 9–41.
  * Helbing  _et al._ [2000b] D. Helbing, I. Farkas, and T. Vicsek, Simulating dynamical features of escape panic, Nature 407, 487 (2000b), arXiv:0009448 [cond-mat] .
  * Garcimartín  _et al._ [2014] A. Garcimartín, I. Zuriguel, J. M. Pastor, C. Martín-Gómez, and D. R. Parisi, Experimental evidence of the ”faster is slower” effect, Transportation Research Procedia 2, 760 (2014).
  * Gershenson and Helbing [2015] C. Gershenson and D. Helbing, When slower is faster, Complexity 21, 9 (2015), arXiv:1506.06796 .
  * Zhou  _et al._ [2020] J. Zhou, G. Cui, S. Hu, Z. Zhang, C. Yang, Z. Liu, L. Wang, C. Li, and M. Sun, Graph neural networks: A review of methods and applications, AI Open 1, 57 (2020).
  * Bronstein  _et al._ [2021] M. M. Bronstein, J. Bruna, T. Cohen, and P. Veličković, Geometric deep learning: Grids, groups, graphs, geodesics, and gauges (2021).
  * Karniadakis  _et al._ [2021] G. E. Karniadakis, I. G. Kevrekidis, L. Lu, P. Perdikaris, S. Wang, and L. Yang, Physics-informed machine learning (2021).
  * Raissi  _et al._ [2019] M. Raissi, P. Perdikaris, and G. E. Karniadakis, Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations, Journal of Computational Physics 378, 686 (2019).
  * Krantz [2007] W. B. Krantz,  _Scaling analysis in modeling transport and reaction processes: a systematic approach to model building and the art of approximation_ (2007) p. 547.

##  Appendix A  Methods

### Crowd model

Agents are modelled as apolar discs in a 2D-periodic domain. Their dynamics is
governed by the following equations:

$\displaystyle
m\frac{d\mathbf{v}_{i}}{dt}=\frac{m}{\tau}(\mathbf{v}_{0,i}-\mathbf{v}_{i})+\sum_{\forall
j\neq i}\mathbf{F}_{ij}$ (6)
$\displaystyle\mathbf{v}_{0,i}=\left\\{\begin{matrix}+s_{0}\mathbf{e}_{x}&i\in\text{{Group
1}}\\\ -s_{0}\mathbf{e}_{x}&i\in\text{{Group 2}}\end{matrix}\right.$ (7)
$\displaystyle\mathbf{F}_{ij}=\left\\{\begin{matrix}-\gamma(d_{ij}-2R)^{-3}\hat{\mathbf{d}}_{ij}&d_{ij}<l_{cr}\\\
0&\text{{otherwise}}\end{matrix}\right.$ (8)

The intrinsic velocity $\mathbf{v}_{0,i}$ has a magnitude $s_{0}$, and is
directed along the positive or negative x-direction depending on the group of
the individual (see Equation 7). The inter-individual force $\mathbf{F}_{ij}$
is a repulsive force, and decays as a power-law with distance. To prevent
collisions, $\mathbf{F}_{ij}$ is chosen such that its magnitude blows up to
infinity when the agent boundaries touch ($d_{ij}=2R$). To avoid spurious
interactions between far-off agents, $\mathbf{F}_{ij}$ is set to 0 beyond a
cut-off radius (see Equation 8). $m$ is the mass of an individual agent,
$\tau$ is the inertial time-scale of the system, and $\gamma$ determines the
strength of the inter-agent interactions. $\mathbf{e}_{x}$ denotes the unit
vector along x-direction. We define the packing density $\rho$ as the number
of agents per unit area.

### Simulations

Agents are placed randomly on a 2-D periodic domain, ensuring that there are
no overlaps. Group identities of the agents are assigned randomly: for a given
value of the number ratio ($N_{r}$), $N_{r}\cdot N$ randomly chosen agents are
assigned to Group 2 and the remaining agents to Group 1. We focus on the
effect of three parameters, $N_{r}$, $\rho$ and $s_{0}$, we vary these over
the following ranges: $\rho\in[0.22,0.58]$,
$N_{r}\in[\frac{1}{N},\frac{1}{2}]$, $s_{0}\in[0.1,3]$. Due to the inherent
symmetry between the groups, we can ignore $N_{r}>\frac{1}{2}$ without loss of
generality. As a result, Group 1 is always the majority group in our
simulations.

The following parameters were fixed: $m=1,\tau=0.2,\gamma=0.2,R=1,l_{cr}=3$.

The parameters $N_{r}$, $\rho$ and $s_{0}$ where varied over the following
sets.

$\displaystyle N_{r}$
$\displaystyle\in\\{1/42,1/21,2/21,1/6,3/14,2/7,1/3,8/21,19/42,1/2\\}$
$\displaystyle s_{0}$ $\displaystyle\in\\{3,2,1.5,1,0.75,0.5,0.25,0.1\\}$
$\displaystyle\rho$
$\displaystyle\in\\{0.57706,0.45792,0.3722,0.30847,0.25981,0.22182\\}$

The total number of agents in the simulations was 42. Since $N_{r}<1/2$, Group
2 was always the minority group.

Phenomena such as laning, clustering, etc. will depend upon the initial
conditions of the simulations. Hence, we perform 100 simulations for each set
of parameter values.

### Simple observer model (agent-only observer)

A simple but effective approach for classification is to classify agents based
on their observed direction of motion. For a better estimate that eliminates
transient fluctuations, one may use the average velocity computed over a time-
window. Therefore, our simple observer algorithm proceeds as follows. First,
the observer computes the average velocity $v_{i}^{w}$ of the agents, averaged
over a time-window of length $w$.

$v^{w}_{i}=\langle\mathbf{v}_{i}.\mathbf{e}_{x}\rangle_{w}$ (9)

The observer classifies an agent based on the net direction of motion during
this window, i.e. the sign of $v_{i}^{w}$:

$\displaystyle v^{w}_{i}\geq 0:\text{{Group 1}}$ (10) $\displaystyle
v^{w}_{i}<0:\text{{Group 2}}$ (11)

In a real-world scenario, an observer has to tackle the problem of sensor
noise that corrupts the data. However, we make the simplifying assumption that
the observer can observe the system perfectly without noise; the only unknown
is the individual group identities of the agents, which the observer desires
to infer.

In our analyses, we used a window-length $w=50$. A shorter window corresponds
to decision-making with fewer data points, while a larger window size may
result in improved classification performance. The exact window length does
not qualitatively affect the results, we verified this by repeating our
analysis with several window sizes between $25$ and $100$.

### Neighborhood observer model

We define a neighborhood parameter $\widehat{\bm{\upphi}}_{i}$ evaluated for
each individual $i$, using information from its first Voronoi neighbors
$\mathcal{N}$ as follows:

$\widehat{\bm{\upphi}}_{i}=\mu\sum_{\forall
j\in\mathcal{N}}f(r_{ij})\big{[}\mathbf{v}_{j}.\mathbf{e}_{ji}\big{]}\mathbf{e}_{ji}$
(12)

The parameter $\widehat{\bm{\upphi}}_{i}$ is a vector that contains the net
effect of all the neighbors (in $\mathcal{N}$) on the agent $i$. Agents
interact as they approach each other, hence the velocities are resolved along
the direction vectors of the line joining the two agents’ center of masses
$\mathbf{e}_{ji}$. In addition, a weighting $f(r_{ij})$, is used to capture
the effect of distance from the focal agent $i$: agents far away will have
lesser impact on the motion of agent $i$. The choice of $f$ is somewhat
arbitrary, as long as it is a monotonically decreasing function that does not
decrease _‘too fast’_. For the purpose of this article, we use a Gaussian
kernel of the form $e^{(-r_{ij}/\epsilon)^{2}}$ as the weighting function,
where $\epsilon=3R$ is the length scale corresponding to the decay of $f$ (see
Appendix C and Figure 7) for an extended discussion on the choice of $f$). The
choice of $f$ and the neighborhood being chosen (Voronoi in our case) decide
the local neighborhood being considered for an agent, the exact choices for
these are not very crucial.

$\mu$ is a scaling parameter which needs to be fixed appropriately. We can
employ a data-based approach to estimate the optimal $\mu$ that maximizes
accuracy of classification using data from our simulations along with the
knowledge of their actual group-identities as labels. As the availability of
labeled data is not a given in real-crowd problems, we turn towards exploring
the phenomenology (Equation 1) to derive a scaling for the parameter $\mu$.

We can begin by employing an order-one scaling analysis[34] on Equation 1. We
express each of the variables $x$ as a product of an $o(1)$ variable denoted
as $\bar{x}$ and an order of magnitude $x_{s}$.

$\displaystyle
v_{s}\bar{v}_{i}^{m}=v_{s}\mathbf{\bar{v}}_{0,i}.\mathbf{e}_{x}+\mu\varphi_{s}\bar{\varphi_{i}}$
(13)
$\displaystyle\bar{v}_{i}^{m}=\mathbf{\bar{v}}_{0,i}.\mathbf{e}_{x}+\frac{\mu\varphi_{s}}{v_{s}}\bar{\varphi_{i}}$
(14)

We choose a scale $\varphi_{s}=\frac{v_{s}}{\mu}$, to make sure that Equation
14 is balanced and all the terms contribute to $\bar{v}_{i}^{m}$, as assumed
initially when proposing Equation 1. Say, for instance, if
$\varphi_{s}\ll\frac{v_{s}}{\mu}$, then Equation 14 becomes similar to
Equation 9, as in our original formulation.

We can now resort to scaling Equation 12 and use the relation
$\varphi_{s}=\frac{v_{s}}{\mu}$ which gives rise to,

$\displaystyle\frac{v_{s}}{\mu}\bar{\varphi}_{i}=f_{s}\sigma_{s}v_{s}\bar{\sum_{\forall
j\in\mathcal{N}}}\bar{f}(r_{ij})\big{[}\mathbf{\bar{v}}_{j}.\mathbf{e}_{ji}\big{]}\mathbf{e}_{ji}.\mathbf{e}_{x}$
(15) $\displaystyle\bar{\varphi}_{i}=\mu f_{s}\sigma_{s}\bar{\sum_{\forall
j\in\mathcal{N}}}\bar{f}(r_{ij})\big{[}\mathbf{\bar{v}}_{j}.\mathbf{e}_{ji}\big{]}\mathbf{e}_{ji}.\mathbf{e}_{x}$
(16) $\displaystyle\mu=\frac{1}{f_{s}\sigma_{s}}$ (17)

A scaling for $\mu$ as shown in Equation 17, arises naturally from Equation
16, when we try to balance the terms. A suitable scale for $f$ would be,

$f_{s}=e^{-\frac{1}{\rho\epsilon^{2}}}$ (18)

where $\sqrt{\frac{1}{\rho}}$ is the typical length scale corresponding to a
crowd with a global packing density $\rho$. $\sigma_{s}$ is the scale of
velocity that arises as a result of the composite nature of the crowd. If all
the agents in the neighborhood $\mathcal{N}$ belong to the same group, then
$\sigma_{s}$ simply takes a value of the mean number of neighbors (which we
can assume as $6$, as the agents in our system are spherical and interact
isotropically). On the other hand, if only half the agents in $\mathcal{N}$
belong to one group, the effect cancels out. Hence, $\sigma_{s}$ is the
expected value of the excess agents in the majority group in $\mathcal{N}$.

If we assume that the probability of finding an agent from a certain group is
independent of other agents in the vicinity, then one can write the expression
for the probability ($p_{k}^{N_{r}}$) of finding $k$ individuals of group 2 in
$\mathcal{N}$ as in Equation 19.

$p_{k}^{N_{r}}=\begin{pmatrix}l\\\ k\end{pmatrix}N_{r}^{k}(1-N_{r})^{6-k}$
(19)

where $|\mathcal{N}|=l$. We take $l=6$ for computations. When there are $k$
agents of group 2, the contribution to $\sigma_{s}$ is $\left|6-2k\right|$.
Using Equation 19, we can evaluate the expected value $\sigma_{s}$ as follows.

$\sigma_{s}=\sum_{k=0}^{6}p_{k}^{N_{r}}\left|6-2k\right|$ (20)

Combining all the terms in Equation 17, using Equation 18 and 20, we get the
final equation of the linear classifier as shown in Equation 21.

$v_{i}^{m}-\Bigg{(}\frac{e^{\frac{1}{\rho\epsilon^{2}}}}{\sum_{k=0}^{6}p_{k}^{N_{r}}\left|6-2k\right|}\Bigg{)}\varphi_{i}=0$
(21)

Where the parameter $\mu$ is,

$\mu=\Bigg{(}\frac{e^{\frac{1}{\rho\epsilon^{2}}}}{\sum_{k=0}^{6}p_{k}^{N_{r}}\left|6-2k\right|}\Bigg{)}$
(22)

### Statistical significance

The average number of misclassifications $n_{m}$ was computed as an average
over 100 simulations and 951 window positions (all positions of a window of
length 50 over a time-series of length 1000). Therefore, the confidence
intervals for $n_{m}$ are too small to be visible in the plots, and are
ignored in Figures 2 and 4. We used a Wilcoxon signed-rank test to test if
$n_{m}$ is significantly different between the agent-only observer and the
neighborhood observer. Due the the very large number of samples, p-values for
all tests were 0 up to machine precision.

##  Appendix B  Model parameters and
classification performance

Figures 5 and 6 show the variation of classification performance over a wide
range of parameters. The number of misclassifications by both observers is
shown.

Figure 2(a) of main text shows the effect of $N_{r}$ on the number of
misclassifications for a particular $\rho$ and $s_{0}$ ($s_{0}=0.75$,
$\rho=0.58$). As $s_{0}$ and $\rho$ are varied, the plots get scaled
accordingly, but the essential qualitative features of the plots are not
affected (Figure 5). The case is similar for misclassifications versus $s_{0}$
plots (Figure 6).

Also, the qualitative features of the misclassification curves for the
neighbourhood observer are similar to that of the agent-only observer.

##  Appendix C  Choice of weighting function $f$

The choice of the weighting function $f$ in Section IV-A merits some
discussion. Clearly, we need the weighting function to assign high weights to
nearby neighbours and low weights to ones that are farther away, i.e. we need
an $f$ such that $f(r)$ decreases monotonically with $r$, where $r$ is the
distance from the focal agent.

Besides, our computation of the scaling factor is dependent on $f$:

$\mu=\frac{1}{\sigma_{s}f_{s}}=\frac{1}{\sigma_{s}f(L_{s})}$ (23)

where $L_{s}$ is a length-scale, which we choose to be
$\sqrt{\frac{1}{\rho}}$. However, this is an estimate based on the global
density parameter: as discussed in Section IV-C, phenomena like lane formation
can cause agents to have very different local densities in their
neighbourhood, causing the effective length scale to be different from
$\sqrt{\frac{1}{\rho}}$. Therefore, to prevent large errors in the estimation
of $\mu$ in these cases, we need to ensure that $f$ is robust to mis-
estimation of $L_{s}$, i.e. does not vary too fast in the vicinity of $L_{s}$.
We chose $f(r)=e^{-r^{2}/\epsilon}$ with $\epsilon=3$ for our analysis.

Figure 7 shows a comparison between two choices of $f$:
$f_{\text{cubic}}(r)=\frac{1}{r^{3}}$ and
$f_{\text{exp}}(r)=e^{-r^{2}/\epsilon}$. Over a range of typical densities,
there is a larger relative variation in $f_{\text{poly}}(L_{s})$, which
translates to a very large variation in $\mu$. Therefore, small mismatches
between the local density around an agent and the global density can result in
a large mis-estimation of $\mu$.

##  Appendix D  Effect of parameters on distributions

Figure 8 shows how the model parameters affect the $v_{i}-\phi_{i}$
distributions. See figure caption for details, and the main text for a
discussion from the viewpoint of dynamics.

##  Appendix E  Learning $\mu$ from data

In the paper, we discussed an approach to compute $\mu$ from the global
parameters of the system, $\rho$ and $N_{r}$, using phenomenological
considerations. When labelled training data (i.e. crowd data where individuals
are already marked as Group 1 or Group 2) is readily available, an alternate
approach would be to fit a linear classification model to the data.

We explore this by fitting a linear SVM (support vector machine) with 0
intercept to the data. Assuming the training-algorithm converges well, this
will give us the _optimal_ linear classifier; optimal in the sense that it
will be the best line that classifies agents on the $v_{i}^{w}-\phi_{i}^{w}$
plane. This classifier overcomes the problem of ‘clipping’ we observed, and
outperforms the agent-only observer for all parameter values. Figure 9 shows
how the decision boundary of the SVM avoids the clipping problem, and compares
the performance of the three algorithms.

##  Appendix F Supplementary videos

For supplementary videos, see: https://www.dannyraj.com/obsinf-supp-info

Figure 5:  Number of misclassifications versus $N_{r}$ for the agent-only
observer (light colour, circle markers) and the neighbourhood observer (dark
color, square markers) for the smaller (red lines, filled markers) and larger
(blue lines, empty markers) groups, for various choices of $s_{0}$ and $\rho$.
Figure 6:  Number of misclassifications versus $s_{0}$ for the two observers.
Figure conventions are the same as Figure 5.  Figure 7:  Left panel:
$f(L_{s})$ against $\rho$ for $f:=f_{\text{poly}}$ (blue) and
$f:=f_{\text{exp}}$ (red); where $L_{s}=1/\sqrt{\rho}$. Left panel: $\mu$
against $\rho$, with $\mu$ computed using $f_{\text{poly}}$ (blue) and
$f_{\text{exp}}$ (red). Notice the large variation when $\mu$ is computed
using $f_{\text{poly}}$.  Figure 8:  Joint distribution of $v_{i}^{w}$ and
$\varphi_{i}^{w}$, and its dependence on parameters. (a) A typical
distribution of agent velocity $v_{i}^{w}$ and neighbourhood parameter
$\varphi_{i}^{w}$. $N_{r}=1/2,\rho=0.31,s_{0}=3$. The blue points correspond
to agents actually belonging to Group 1, and red points correspond to agents
belonging to Group 2. The black line is the classification boundary (of the
neighbourhood observer) with slope $1/\mu$. (b) When $N_{r}$ decreases
($N_{r}=1/6$), the distribution of the minority agents (red) shifts upwards,
indicating an increase in $\varphi_{i}^{w}$ in the positive direction. The
distribution of the majority agents (blue) shifts leftwards and upwards,
indicating that fewer agents are being pushed. (c) When $\rho$ decreases
($\rho=0.22)$ the distributions of the two groups move farther apart. (d) When
$s_{0}$ decreases ($s_{0}=0.1$), all agents are moving very slowly, which also
results in low $\varphi_{i}^{w}$ values. There is a large overlap between the
distributions of the two groups, making classification difficult.  Figure 9:
(a) Comparison between the decision boundaries of the neighbourhood observer
and the trained SVM classifier. The black line is the decision boundary of the
neighbourhood observer, which is a line through the origin with slope $1/\mu$,
The grey line is the decision boundary found by training an SVM. Note how the
black line clips the edges of the distributions while the grey line avoids
this issue. (b) Comparison of the performances of the 3 classification
algorithms: the agent-only observer, the neighbourhood observer (with
analytically computed $\mu$) and the trained SVM classifier. The SVM
classifier is as good as or better than the neighbourhood observer for
asymmetric cases (data shown from $N=14$. In the symmetric case, the SVM
classifier obtains equal or better performance than the agent-only observer,
while the neighbourhood observer performs slightly worse.