# A Spatio-Temporal Attentive Network for Video-Based Crowd Counting
††thanks: This work was partially funded by: Extension (ESA, n.
4000132621/20/NL/AF), AI4Media - A European Excellence Centre for Media,
Society and Democracy (EC, H2020 n. 951911), “Intelligenza Artificiale per il
Monitoraggio Visuale dei Siti Culturali” (AI4CHSites) CNR4C program, CUP
B15J19001040004, and by the Italian Ministry of University and Research MUR-
FOE 2020 TIRS project.

1st Marco Avvenuti  Dept. of Information Engineering
University of Pisa
Pisa, Italy
<EMAIL_ADDRESS>2nd Marco Bongiovanni  Dept. of Information
Engineering
University of Pisa
Pisa, Italy
<EMAIL_ADDRESS>3rd Luca Ciampi  Inst. of Information
Science and Tech.
National Research Council (ISTI-CNR)
Pisa, Italy
<EMAIL_ADDRESS>4th Fabrizio Falchi  Inst. of Information Science and
Tech.
National Research Council (ISTI-CNR)
Pisa, Italy
<EMAIL_ADDRESS>5th Claudio Gennaro  Inst. of Information Science
and Tech.
National Research Council (ISTI-CNR)
Pisa, Italy
<EMAIL_ADDRESS>6th Nicola Messina  Inst. of Information Science
and Tech.
National Research Council (ISTI-CNR)
Pisa, Italy
<EMAIL_ADDRESS>
###### Abstract

Automatic people counting from images has recently drawn attention for urban
monitoring in modern Smart Cities due to the ubiquity of surveillance camera
networks. Current computer vision techniques rely on deep learning-based
algorithms that estimate pedestrian densities in still, individual images.
Only a bunch of works take advantage of temporal consistency in video
sequences. In this work, we propose a spatio-temporal attentive neural network
to estimate the number of pedestrians from surveillance videos. By taking
advantage of the temporal correlation between consecutive frames, we lowered
state-of-the-art count error by $5\%$ and localization error by $7.5\%$ on the
widely-used FDST benchmark.

###### Index Terms:

Crowd Counting, Deep Learning, Visual Counting, Smart Cities

##  I Introduction

Computer Vision obtained a tremendous boost in the last few years thanks to
the astonishing advances in Machine Learning. In particular, Deep Learning
allowed the research community to define new state-of-the-arts in many
Computer Vision tasks, such as object detection [1], or image retrieval [2],
to name a few. With the increasing interest in Smart Cities and the grown
availability of surveillance cameras that have become pervasive, there is a
unanimous effort to employ these novel technologies for urban monitoring and
surveillance. Like no other sensing mechanism, networks of city cameras can
observe and simultaneously provide visual data to AI systems to extract
relevant information from this deluge of data. In this context, many smart
applications, ranging from lot occupancy detection [3] to pedestrian detection
[4, 5, 6] and re-identification [7], have been proposed and are nowadays
widely employed worldwide. In this work, we treat the counting task, which
consists of providing the number of instances of a specific class present in
the scene — for example, the number of vehicles that are transiting a
particular road. Specifically, we focus on crowd counting to automatically
estimate the number of people present in images gathered from city
surveillance cameras. This application is crucial in many scenarios, like
monitoring and eventually limiting people aggregations during the recent
COVID-19 pandemic.

Recent literature extensively faced the crowd counting task by estimating and
integrating density maps of still, individual images. Nevertheless, relatively
few works take advantage of the temporal consistency of video streams.
However, using temporal constraints across consecutive frames could be an
essential key point for enhancing the counting performance.

Driven by these concerns, we propose an enhanced extension to the video-
counting framework introduced by Liu et al. [8, 9]. These works presented an
interesting semi-supervised learning method that infers people density maps
starting from the estimation of people flows among adjacent keyframes. At
inference time, the flow is integrated to obtain the actual people count.
Although the learning framework is solid, their proposed network comprises a
simple fully-convolutional encoder-decoder pipeline, which estimates the flow
from a pair of consecutive images. In this work, inspired by the recent
attentive mechanisms proposed to process visual data, such as Vision
Transformer [10], we enhance the architecture in [8, 9] with self-attentive
connections. Specifically, we introduce an attentive-based temporal fusion
layer to improve the flow predictor, taking advantage of the temporal
correlation between consecutive frames. Through an experimental evaluation, we
show that our proposed method can boost the performance compared with the
original framework on the widely-used FDST dataset [11], one of the largest
and most diverse collections of temporally correlated frames, suitable for
video-based counting. We assess not only the counting performance considering
the counting errors occurring at inference time (i.e., the difference between
the predicted and the actual person numbers) but also the ability to correctly
localize the counted persons. Indeed, count errors do not take into account
where the pedestrians have been detected in the images and, consequently,
counting models might achieve low values of errors while providing wrong
predictions (e.g., a high number of false positives and false negatives).

To sum up, we propose the following contributions:

  * •

We propose an extension to a recent semi-supervised video-based counting
framework [8, 9], employing an attentive-based temporal fusion layer that
takes advantage of the temporal correlation between consecutive images and
improves the people flow estimation.

  * •

We demonstrate through detailed experiments that the proposed variation can
reach state-of-the-art results on the FDST dataset, lowering the count error
by $5\%$.

  * •

We conduct a performance evaluation also considering the ability to correctly
localize the counted persons, lowering the localization error by $7.5\%$
compared to the original counting framework.

The code and the trained models will be publicly available at
https://tinyurl.com/yb42ce38.

##  II Related Work

###  II-A Image-based Counting

Image-based counting aims at estimating the number of object instances, like
people [12, 13], cells [14, 15], or vehicles [16, 17], in still images or
video frames [18]. Current solutions are formulated as supervised deep
learning-based problems belonging to one of two main categories: counting by
detection and counting by regression. Detection-based approaches, such as in
[19] and [20], require prior detection of the single instances of objects. On
the other hand, regression-based techniques like [21] and [22] try to
establish a direct mapping between the image features and the number of
objects in the scene, either directly or via the estimation of a density map
(i.e., a continuous-valued function). Regression techniques show superior
performance in crowded and highly-occluded scenarios [18].

###  II-B Video-based Counting

Crowd counting approaches are mainly based on single-image inputs, even when a
video sequence is available, leading to the impossibility of exploiting the
temporal interdependence between consecutive frames in the sequence.
Nevertheless, in the literature, there are a handful of works that rely on the
estimation of density maps, but, on the other hand, try to exploit also
temporal information to improve counting accuracy. For instance, [23]
introduced one of the first video-based counting approaches, based on an LSTM-
based method called ConvLSTM, i.e., a fully connected LSTM extended with
convolutional layers in both the input-to-state and state-to-state
connections. In [24], the authors exploited a Locality-constrained Spatial
Transformer (LST) module to model the spatial-temporal correlation between
estimated neighboring density maps. Another remarkable work is [25], where the
authors introduced a Temporal Aware Network (TAN), which can compute the
number of people present at frame $X_{t}$ by exploiting frames from $X_{t-k}$
to $X_{t+k}$. More recently, Liu et al. [8, 9] proposed an alternative
approach where density maps are not directly regressed from images but are
inferred from people flows across image locations between consecutive frames.
However, the proposed network in charge of estimating flows from pair of
images lies in a simple fully-convolutional encoder-decoder pipeline. In this
work, we extend and enhance this framework by exploiting self-attentive
connections, introducing an attentive-based temporal fusion to enhance the
flow predictor.

###  II-C Attentive Models

Attention mechanisms have been largely used in the last two years and had a
significant impact on tasks that involve both vision and language, such as VQA
[26], image captioning [27, 28] or image-text matching [29, 2]. Recently, the
Transformer-like attention mechanism [30] obtained the best results in
processing images and videos. In particular, the authors in [10] introduced
the Vision Transformer, demonstrating the power of the self-attentive
mechanism in the image classification task. Similarly, the DETR architecture
[31] used the full Transformer architecture for tackling the object detection
task, obtaining remarkable results with respect to state-of-the-art fully-
convolutional approaches. This work is inspired by the recent advances in
attentive video processing, where most methods use spatio-temporal attention
to understand frame patches from multiple timesteps [32, 33].

##  III Method

The proposed architecture tries to reconstruct the people flows, enhancing the
weak-supervised learning framework proposed in [8, 9]. In Section III-A we
briefly review their original framework; in Section III-B we explain our
attentive-based temporal fusion used to improve the flow predictor.

Figure 1: Visualization of a neighborhood $N(j)$ of an image patch j (shown in
red). This is used to impose people conservation constraints, as formulated by
Equations 2 and 3.

###  III-A The People-Flow Approach

The approach presented in [8, 9] is a video-based counting scheme that does
not directly estimate people densities from images but infers them from the
so-called people flow between two consecutive frames. People flows are vector
fields that associate pedestrian movement vectors to every point in the frame
space. People flows are zeros at a certain point in space if there are no
moving pedestrians.

Once the people flow $\boldsymbol{f}^{t-1,t}$ between two consecutive frames
$\boldsymbol{I}^{t-1}$ and $\boldsymbol{I}^{t}$ has been predicted, the j-th
spatial location of the density map $d^{t}_{j}$ for the frame
$\boldsymbol{I}^{t}$ can be reconstructed by summing all the flow
contributions entering $j$ from neighboring locations of the previous frame:

$\displaystyle d^{t}_{j}=\sum_{i\in N(j)}f^{t-1,t}_{i,j}$ (1)

where the neightbouring locations $N(j)$ of the $j$-th path are shown in
Figure 1. The final people count at time $t$ can then be found by summing up
all the pixel values of the obtained density map: $\sum_{j}d^{t}_{j}$.
However, regressing the flows is not a straightforward operation, as many
video-counting datasets do not embed any explicit people flow ground-truth
that can be used to supervise the network.

For this reason, the work in [8, 9] proposed a weak-supervised learning
approach that estimates the flows using only the ground-truth density maps
$\boldsymbol{\bar{d}}^{t-1}$ and $\boldsymbol{\bar{d}}^{t}$ at consecutive
timesteps $(t-1,t)$. In particular, the flows are constructed by only imposing
strong people conservation constraints, i.e., people cannot appear or
disappear between consecutive frames if not in the frame edges. In particular,
the constraints can be expressed as follows:

$\displaystyle\sum_{i\in N(j)}f_{i,j}^{t-1,t}$ $\displaystyle=\sum_{k\in
N(j)}f_{j,k}^{t,t+1}$ (2) $\displaystyle f_{i,j}^{t-1,t}$
$\displaystyle=f_{j,i}^{t,t-1}$ (3)

where Eq. 2 imposes people conservation in the neighborhood of a location $j$
(Figure 1) across consecutive frame intervals, and Eq. 3 enforces the spatio-
temporal symmetry of the flows, i.e. the people should move in the opposite
direction when the time flows backwards. More details can be found in [8].

With this weakly-supervised learning framework, the only missing piece is the
function that regresses the flows. In particular, this function is a deep
neural network $\mathcal{R}(\boldsymbol{I}^{t-1},\boldsymbol{I}^{t},\theta)$
that outputs the flow $\boldsymbol{f}^{t-1,t}$ given two consecutive images in
input. Its parameters $\theta$ are optimized during the training process by
enforcing the constraints in Eq. 2 and 3. In the next paragraph, we propose to
use a spatio-temporal attentive network as $\mathcal{R}$.

Figure 2: The overall spatio-temporal attentive regressor architecture. Figure
3: Inner architecture of the spatio-temporal fusion module.

###  III-B The Attentive Flow Regressor

The proposed network resembles a convolutional encoder-decoder architecture,
which takes two consecutive RGB images in input and produces the predicted
flows. Specifically, the encoder $\mathcal{E}$ processes the input images
$\boldsymbol{I}^{t-1}$ and $\boldsymbol{I}^{t}$ to obtain the corresponding
internal feature maps
$\boldsymbol{w}^{t-1},\boldsymbol{w}^{t}\in\mathbb{R}^{W\times H\times C}$,
where $(W,H,C)$ are the width, the height and the number of channels,
respectively. Formally:

$\displaystyle\boldsymbol{w}^{t-1}$
$\displaystyle=\mathcal{E}(\boldsymbol{I}^{t-1})$ (4)
$\displaystyle\boldsymbol{w}^{t}$
$\displaystyle=\mathcal{E}(\boldsymbol{I}^{t})$ (5)

These feature maps are then composed together using an aggregator function
$\tilde{\boldsymbol{w}}=\mathcal{A}(\boldsymbol{w}^{t-1},\boldsymbol{w}^{t}$),
which outputs a feature map $\tilde{\boldsymbol{w}}\in\mathbb{R}^{W\times
H\times C^{\prime}}$, with the same spatial resolution but possibly a
different number $C^{\prime}$ of channels. In the end, the final flows are
obtained by applying the decoder to the aggregated feature maps:
$\boldsymbol{f}^{t}=\mathcal{D}(\tilde{\boldsymbol{w}})\in\mathbb{R}^{W\times
H\times 10}$. Notice that the output flow is 10-dimensional, as there are ten
possible directions in which a person can move inside the frame (nine
locations in the neighborhood including the starting cell, as depicted in
Figure 1, plus one cell representing the rest of the world at the edges of the
image).

In the original formulation in [8], the aggregator function is a
straightforward concatenation along the channels dimension:
$\mathcal{A}=[\cdot,\cdot]^{C}$, which therefore outputs
$\tilde{\boldsymbol{w}}\in\mathbb{R}^{W\times H\times 2C}$. In this work, we
propose an attentive spatio-temporal aggregation module that can produce more
space-time aware feature maps $\tilde{\boldsymbol{w}}$, which, in turn,
provide better flows.

The idea is to employ the feature maps coming from one of the two input frames
to condition the features of the other frame. We denote the feature maps from
the two frames as $\boldsymbol{w}_{\text{input}}$ and
$\boldsymbol{w}_{\text{target}}$, where input refers to the visual information
that conditions the target one in the attention mechanism. In particular, we
used an approach similar to the convolutional self-attention by [34].
Specifically, we initially derive other three feature maps from the two input
maps produced by the encoder, namely
$\boldsymbol{v}=g_{v}(\boldsymbol{w}_{\text{target}})\in\mathcal{R}^{W\times
H\times d_{v}}$ and
$\boldsymbol{q}=g_{q}(\boldsymbol{w}_{\text{target}})\in\mathcal{R}^{W\times
H\times d_{k}}$ from $\boldsymbol{w}_{\text{target}}$, and
$\boldsymbol{k}=g_{k}(\boldsymbol{w}_{\text{input}})\in\mathcal{R}^{W\times
H\times d_{k}}$ from $\boldsymbol{w}_{\text{input}}$. These
$\boldsymbol{v},\boldsymbol{k},\boldsymbol{q}$ are the values, the keys, and
the queries respectively, used to drive the Transformer-like attention
mechanism. They are produced from the input feature maps using three different
convolutional layers $g_{v},g_{k},g_{k}$ having a $3\times 3$ kernel and
padding=1 to leave the spatial resolution untouched.

The output, as in the Transformer [30] attention mechanism, is computed
through the scaled dot-product attention:

$\displaystyle\boldsymbol{o}$
$\displaystyle=\text{softmax}(\frac{\boldsymbol{q}\boldsymbol{k}^{\top}}{\sqrt{d_{k}}})\boldsymbol{v}$
(6)
$\displaystyle=\text{softmax}(\frac{g_{q}(\boldsymbol{w}_{\text{target}})g_{k}(\boldsymbol{w}_{\text{input}})^{\top}}{\sqrt{d_{k}}})g_{v}(\boldsymbol{w}_{\text{target}})$
(7)

Note that, differently from the original Transformer-like attention, we
empirically found that computing values from the target sequence — instead of
the input sequence — led to better results. Still, the core idea aims at
incorporating the context from one frame into the feature map of the other
frame.

To account for the original feature maps after the attentive processing, we
concatenate a transformation of the original feature maps
$\boldsymbol{w_{\text{input}}}$ to the output $\boldsymbol{o}$ along the
channel dimension:

$\displaystyle\boldsymbol{\bar{o}}$
$\displaystyle=[g_{\text{skip}}(\boldsymbol{w_{\text{input}}}),\boldsymbol{o}]^{C}$
(8)

where $g_{\text{skip}}(\cdot)$ is a convolution operation that does not change
the spatial resolution, and outputs $C-d_{v}$ channels so that
$\boldsymbol{\bar{o}}$ has again C channels. We denote the above-described
spatio-temporal attentive fusion module as
$\boldsymbol{\bar{o}}=\mathcal{ST}(\boldsymbol{w}_{\text{input}},\boldsymbol{w}_{\text{target}})$
(see Figure 3). Now, the final goal is to condition the past frame
$\boldsymbol{I}^{t-1}$ given the present frame $\boldsymbol{I}^{t}$, and vice-
versa. For this reason, we compute the two symmetric attentional feature maps,
by simply swapping the two frames in input:

$\displaystyle\boldsymbol{\bar{o}}^{t}$
$\displaystyle=\mathcal{ST}(\boldsymbol{w}^{t-1},\boldsymbol{w}^{t})$ (9)
$\displaystyle\boldsymbol{\bar{o}}^{t-1}$
$\displaystyle=\mathcal{ST}(\boldsymbol{w}^{t},\boldsymbol{w}^{t-1})$ (10)

At this point, using plain concatenation along the channel direction, we merge
the obtained attentive feature maps obtaining the final spatio-temporal aware
feature map:
$\boldsymbol{\bar{w}}=[\boldsymbol{\bar{o}}^{t-1},\boldsymbol{\bar{o}}^{t}]^{C}$.
Finally, as in the original encoder-decoder convolutional architecture, the
10-dimensional output flow $\boldsymbol{f}^{t}$ is obtained by applying the
decoder to this last feature map:
$\boldsymbol{f}^{t}=\mathcal{D}(\tilde{\boldsymbol{w}})$. The overall
architecture is shown in Figure 2.

##  IV Experimental Evaluation

This section describes the experiments performed to validate our approach and
discusses the obtained results. As benchmark, we exploit the widely-used FDST
dataset [11]. It consists of 100 videos gathered from 13 different scenarios.
A total of 150,000 frames have been extracted, thus representing one of the
largest and most diverse collections of real-world images suitable for this
task. Annotations are expressed using dots localizing peoples’ heads, as usual
for the counting task, for a total of 394,081 labeled pedestrians. We follow
the same setting as in [11], considering 60 videos (9,000 frames) as the
training set and the remaining 40 videos (6,000 frames) as the test set.

In the first stage of the evaluation, we compare the counting errors obtained
using our proposed solution against the framework introduced in [8, 9], and
other recent state-of-the-art counting solutions present in the literature. On
the other hand, in the second part of our experiments, we also assess the
ability to correctly localize the counted pedestrians.

###  IV-A Comparison with the State-of-the-art

Here, we compare our solution in terms of counting with other state-of-the-art
methodologies. Following standard counting benchmarks, we used the Mean
Absolute Error (MAE) and the Root Mean Square Error (RMSE) to measure the
counting performance. Specifically, they are defined as:

$\text{MAE}=\frac{1}{N}\sum_{n=1}^{N}\left|c_{\text{gt}}^{n}-c_{\text{pred}}^{n}\right|\,,$
(11)

$RMSE=\sqrt{\frac{1}{N}\sum_{n=1}^{N}(c_{gt}^{n}-c_{pred}^{n})^{2}}.$ (12)

where $N$ is the number of test images, $c_{\text{gt}}^{n}$ is the actual
count (i.e., the ground truth), and $c_{\text{pred}}^{n}$ is the predicted
count of the $n$-th image. It is worth noting that, as a result of the
squaring of each difference, the RMSE effectively penalizes large errors more
heavily than small ones, and so it is more useful when outliers are
particularly undesirable.

We report our quantitative results obtained on the FDST dataset in Table I. We
called our solution People Flow Temporal Fusion (TF), to distinguish it from
the original framework People Flow Plain Concatenation (PC). We divided the
training set into train and validation splits, considering the $80\%$ and the
$20\%$ of the available data, respectively. To mitigate the overfitting
problem, we considered training and validation images belonging to different
video sequences (12 sequences for validation and the remaining 48 for the
training). For better capturing movement between consecutive images, we
sampled the previous and the consecutive frames by setting an offset of 5
frames. More, we performed data augmentation randomly, applying common
transformations such as horizontal flipping, cropping, and normalization. We
repeated the experiments using our solution three times, reporting the mean.
Finally, images are resized to a dimension of $640\times 360$ pixels (width
and height, respectively). As can be seen, our approach outperforms the
competing methods. In particular, we lowered the MAE by about $5\%$ compared
to the baseline framework, with a comparable RMSE value.

TABLE I: Comparison with SOTA on FDST dataset. We obtained SOTA results in terms of MAE and we lowered the error by $5\%$ compared to the original framework.  Model |  Temporal |  MAE $\downarrow$ |  RMSE $\downarrow$
---|---|---|---
ConvLSTM [23] |  ✓ |  4.48 |  5.82
WithoutLST [24] |  |  3.87 |  5.16
MCNN [13] |  |  3.77 |  4.88
LST [24] |  ✓ |  3.35 |  4.45
CAN [22] |  |  2.44 |  2.96
People Flow PC [8, 9] |  ✓ |  2.17 |  2.62
People Flow TF (Our) |  ✓ |  2.07 |  2.69

Figure 4: Some qualitative examples showing the predicted density maps.

###  IV-B Localization Analysis

Although the MAE is a fair metric for establishing a comparative in terms of
count error, it can often lead to masking erroneous estimations. A potential
pitfall of the counting approaches is that the models may miss hard-to-detect
instances. To compensate for these missed detections and estimate the correct
count, they may falsely mark background sub-regions having similar regional
image properties as possible object instances instead. The reason is that the
MAE does not take into account where the estimations have been done in the
images. In this section, we conduct experiments to assess the ability of our
solution to localize the counted pedestrians correctly, comparing the obtained
results against the framework introduced by [8, 9]. Specifically, we consider
the Grid Average Mean absolute Error (GAME) [35], a hybrid metric that
simultaneously considers the object count and the estimated locations of the
persons. It is computed by sub-dividing the image in $4^{L}$ non-overlapping
regions and summing the MAE computed in each of these sub-regions:

$GAME(L)=\frac{1}{N}\sum_{n=1}^{N}(\sum_{l=1}^{4^{L}}|c_{gt}^{l}-c_{pred}^{l}|),$
(13)

where $N$ is the total number of test images, $c_{pred}^{l}$ is the estimated
count in a region $l$ of the n-th image, and $c_{gt}^{l}$ is the ground truth
for the same region in the same image. The higher $L$, the more restrictive
the GAME metric will be.

In this work, we considered a grid with a fixed dimension of $80\times 45$
(width and height, respectively), corresponding to the spatial resolution of
the feature maps produced by the encoder $\mathcal{E}$ fed with input images
of $640\times 360$ pixels. Table II shows our quantitative results obtained on
the FDST dataset exploiting our solution People Flow Temporal Fusion (TF)
compared with the original framework People Flow Plain Concatenation (PC). We
repeated the experiments three times, reporting the mean. Specifically,
concerning the original framework, we performed three different experiments
considering the model publicly provided
111https://github.com/weizheliu/People-Flows by the authors of [8, 9], and two
models obtained re-training from scratch the original network using different
seeds (as illustrated in the Table, the mean concerning the MAE is fully
comparable with the one reported in [8, 9]). As can be seen, our approach
outperforms the original methodology, lowering the GAME by about $7.5\%$, thus
suggesting that our solution provides better performance not only in terms of
count errors but that it is also capable of better localizing the found
pedestrians. In Figure 4 we report some qualitative outputs from our approach.

TABLE II: Localization Analysis. We lowered the GAME by about $7.5\%$ compared
to the original framework on the FDST dataset.

Model |  MAE $\downarrow$ |  GAME $\downarrow$
---|---|---
People Flow PC∗ |  2.17 |  16.00
People Flow TF (Our) |  2.07 |  14.81

* Retrained in this work.

##  V Conclusions and Future Work

This paper proposed an attentive neural network to estimate the number of
pedestrians in videos from surveillance cameras. In particular, we extended a
promising method that estimates people flows to obtain people densities, which
are then integrated to get the final count. To create a more suitable spatial
and temporal context for predicting the flows, we proposed a spatio-temporal
attentive network, which can contextualize the features maps from the present
with those from the past frame vice-versa. In our experiments on the largely-
used FDST dataset, we demonstrated the effectiveness of our architecture. We
obtained a considerable improvement in counting performance compared to other
state-of-the-art approaches in video counting. Furthermore, through the GAME
metric, we demonstrated that our method achieves a more precise localization
of the pedestrians inside the frame than the network without the spatio-
temporal attentive module.

In the future, we plan to extend the experimentation to other datasets, also
employing virtual data to train the network, to avoid manual annotation of
large dot-annotated pedestrian videos. Furthermore, we plan to use domain
adaptation techniques to fill the well-known domain-gap existing between the
different monitored scenarios.

## References

  * [1] M. Traquair, E. Kara, B. Kantarci, and S. Khan, “Deep learning for the detection of tabular information from electronic component datasheets,” in _2019 IEEE Symposium on Computers and Communications (ISCC)_. IEEE, jun 2019.
  * [2] N. Messina, G. Amato, A. Esuli, F. Falchi, C. Gennaro, and S. Marchand-Maillet, “Fine-grained visual textual alignment for cross-modal retrieval using transformer encoders,” _ACM Transactions on Multimedia Computing, Communications, and Applications_ , vol. 17, no. 4, pp. 1–23, nov 2021.
  * [3] G. Amato, F. Carrara, F. Falchi, C. Gennaro, and C. Vairo, “Car parking occupancy detection using smart camera networks and deep learning,” in _2016 IEEE Symposium on Computers and Communication (ISCC)_. IEEE, jun 2016.
  * [4] G. Amato, L. Ciampi, F. Falchi, C. Gennaro, and N. Messina, “Learning pedestrian detection from virtual worlds,” in _Image Analysis and Processing - ICIAP 2019 - 20th International Conference, Trento, Italy, September 9-13, 2019, Proceedings, Part I_ , ser. Lecture Notes in Computer Science, vol. 11751. Springer, 2019, pp. 302–312.
  * [5] L. Ciampi, N. Messina, F. Falchi, C. Gennaro, and G. Amato, “Virtual to real adaptation of pedestrian detectors,” _Sensors_ , vol. 20, no. 18, p. 5250, sep 2020.
  * [6] M. D. Benedetto, F. Carrara, L. Ciampi, F. Falchi, C. Gennaro, and G. Amato, “An embedded toolset for human activity monitoring in critical environments,” _Expert Systems with Applications_ , vol. 199, p. 117125, aug 2022.
  * [7] Y. Ma, T. Bai, W. Zhang, S. Li, J. Hu, and M. Lu, “Multi-scale relation network for person re-identification,” in _2021 IEEE Symposium on Computers and Communications (ISCC)_. IEEE, sep 2021.
  * [8] W. Liu, M. Salzmann, and P. Fua, “Estimating people flows to better count them in crowded scenes,” in _Computer Vision – ECCV 2020_. Springer International Publishing, 2020, pp. 723–740.
  * [9] ——, “Counting people by estimating people flows,” _IEEE Transactions on Pattern Analysis and Machine Intelligence_ , pp. 1–1, 2021.
  * [10] A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly, J. Uszkoreit, and N. Houlsby, “An image is worth 16x16 words: Transformers for image recognition at scale,” in _9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021_. OpenReview.net, 2021.
  * [11] Y. Fang, B. Zhan, W. Cai, S. Gao, and B. Hu, “Locality-constrained spatial transformer network for video crowd counting,” in _2019 IEEE International Conference on Multimedia and Expo (ICME)_. IEEE, jul 2019.
  * [12] L. Liu, Z. Qiu, G. Li, S. Liu, W. Ouyang, and L. Lin, “Crowd counting with deep structured scale integration network,” in _2019 IEEE/CVF International Conference on Computer Vision (ICCV)_. IEEE, oct 2019.
  * [13] Y. Zhang, D. Zhou, S. Chen, S. Gao, and Y. Ma, “Single-image crowd counting via multi-column convolutional neural network,” in _2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)_. IEEE, jun 2016.
  * [14] L. Ciampi, F. Carrara, G. Amato, and C. Gennaro, “Counting or localizing? evaluating cell counting and detection in microscopy images,” in _Proceedings of the 17th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications_. SCITEPRESS - Science and Technology Publications, 2022.
  * [15] J. P. Cohen, G. Boucher, C. A. Glastonbury, H. Z. Lo, and Y. Bengio, “Count-ception: Counting by fully convolutional redundant counting,” in _2017 IEEE International Conference on Computer Vision Workshops (ICCVW)_. IEEE, oct 2017.
  * [16] G. Amato, L. Ciampi, F. Falchi, and C. Gennaro, “Counting vehicles with deep learning in onboard UAV imagery,” in _2019 IEEE Symposium on Computers and Communications (ISCC)_. IEEE, jun 2019.
  * [17] L. Ciampi, C. Gennaro, F. Carrara, F. Falchi, C. Vairo, and G. Amato, “Multi-camera vehicle counting using edge-ai,” _CoRR_ , vol. abs/2106.02842, 2021.
  * [18] V. S. Lempitsky and A. Zisserman, “Learning to count objects in images,” in _Advances in Neural Information Processing Systems 23: 24th Annual Conference on Neural Information Processing Systems 2010_. Curran Associates, Inc., 2010, pp. 1324–1332.
  * [19] G. Amato, P. Bolettieri, D. Moroni, F. Carrara, L. Ciampi, G. Pieri, C. Gennaro, G. R. Leone, and C. Vairo, “A wireless smart camera network for parking monitoring,” in _2018 IEEE Globecom Workshops (GC Wkshps)_. IEEE, dec 2018.
  * [20] I. H. Laradji, N. Rostamzadeh, P. O. Pinheiro, D. Vazquez, and M. Schmidt, “Where are the blobs: Counting by localization with point supervision,” in _Computer Vision – ECCV 2018_. Springer International Publishing, 2018, pp. 560–576.
  * [21] L. Ciampi, C. Santiago, J. Costeira, C. Gennaro, and G. Amato, “Domain adaptation for traffic density estimation,” in _Proceedings of the 16th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications_. SCITEPRESS - Science and Technology Publications, 2021.
  * [22] W. Liu, M. Salzmann, and P. Fua, “Context-aware crowd counting,” in _2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)_. IEEE, jun 2019.
  * [23] F. Xiong, X. Shi, and D.-Y. Yeung, “Spatiotemporal modeling for crowd counting in videos,” in _2017 IEEE International Conference on Computer Vision (ICCV)_. IEEE, oct 2017.
  * [24] Y. Fang, B. Zhan, W. Cai, S. Gao, and B. Hu, “Locality-constrained spatial transformer network for video crowd counting,” in _2019 IEEE International Conference on Multimedia and Expo (ICME)_. IEEE, jul 2019.
  * [25] X. Wu, B. Xu, Y. Zheng, H. Ye, J. Yang, and L. He, “Fast video crowd counting with a temporal aware network,” _Neurocomputing_ , vol. 403, pp. 13–20, aug 2020.
  * [26] M. Malinowski, C. Doersch, A. Santoro, and P. Battaglia, “Learning visual question answering by bootstrapping hard attention,” in _Proceedings of the European Conference on Computer Vision (ECCV)_ , 2018, pp. 3–20.
  * [27] S. He, W. Liao, H. R. Tavakoli, M. Yang, B. Rosenhahn, and N. Pugeault, “Image captioning through image transformer,” in _Proceedings of the Asian Conference on Computer Vision_ , 2020.
  * [28] M. Cornia, M. Stefanini, L. Baraldi, and R. Cucchiara, “Meshed-memory transformer for image captioning,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 2020, pp. 10 578–10 587.
  * [29] N. Messina, F. Falchi, A. Esuli, and G. Amato, “Transformer reasoning network for image-text matching and retrieval,” in _2020 25th International Conference on Pattern Recognition (ICPR)_. IEEE, 2021, pp. 5222–5229.
  * [30] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,” _Advances in neural information processing systems_ , vol. 30, 2017.
  * [31] N. Carion, F. Massa, G. Synnaeve, N. Usunier, A. Kirillov, and S. Zagoruyko, “End-to-end object detection with transformers,” in _European conference on computer vision_. Springer, 2020, pp. 213–229.
  * [32] G. Bertasius, H. Wang, and L. Torresani, “Is space-time attention all you need for video understanding,” _arXiv preprint arXiv:2102.05095_ , vol. 2, no. 3, p. 4, 2021.
  * [33] A. Arnab, M. Dehghani, G. Heigold, C. Sun, M. Lučić, and C. Schmid, “Vivit: A video vision transformer,” in _Proceedings of the IEEE/CVF International Conference on Computer Vision_ , 2021, pp. 6836–6846.
  * [34] I. Bello, B. Zoph, A. Vaswani, J. Shlens, and Q. V. Le, “Attention augmented convolutional networks,” in _Proceedings of the IEEE/CVF international conference on computer vision_ , 2019, pp. 3286–3295.
  * [35] R. Guerrero-Gómez-Olmedo, B. Torre-Jiménez, R. López-Sastre, S. Maldonado-Bascón, and D. Oñoro-Rubio, “Extremely overlapping vehicle counting,” in _Pattern Recognition and Image Analysis_. Springer International Publishing, 2015, pp. 423–431.


* [115] A. L. Gutjahr and G. L. Nemhauser, “An algorithm for the line balancing problem,” _Management science_ , vol. 11, no. 2, pp. 308–315, 1964.
  * [116] B. Chachuat, “Mixed-integer linear programming (MILP): Model formulation,” http://macc.mcmaster.ca/maccfiles/chachuatnotes/07-MILP-I_handout.pdf, accessed: 2021-02-09.
  * [117] “Integer programming,” Jan 2021. [Online]. Available: https://en.wikipedia.org/wiki/Integer_programming
  * [118] M. Bénichou, J.-M. Gauthier, P. Girodet, G. Hentges, G. Ribière, and O. Vincent, “Experiments in mixed-integer linear programming,” _Mathematical Programming_ , vol. 1, no. 1, pp. 76–94, 1971.
  * [119] T. H. Cormen, C. E. Leiserson, R. L. Rivest, and C. Stein, _Introduction to algorithms_. MIT press, 2009.
  * [120] C. U. Manual, “Ibm ilog cplex optimization studio,” _Version_ , vol. 12, pp. 1987–2018, 1987.
  * [121] H.-J. Bandelt, P. Forster, and A. Röhl, “Median-joining networks for inferring intraspecific phylogenies.” _Molecular biology and evolution_ , vol. 16, no. 1, pp. 37–48, 1999.
  * [122] M. Hanafy and H. ElMaraghy, “Developing assembly line layout for delayed product differentiation using phylogenetic networks,” _international journal of production research_ , vol. 53, no. 9, pp. 2633–2651, 2015.
  * [123] G. Boothroyd and P. Dewhurst, _Product design for manufacture and assembly_. CRC Press, 2011.
  * [124] A. Pugh, _An introduction to tensegrity_. Univ of California Press, 1976.
  * [125] “Tensegrity.” [Online]. Available: https://www.creativemachineslab.com/tensegrity.html
  * [126] I. Goodfellow, Y. Bengio, A. Courville, and Y. Bengio, _Deep learning_. MIT press Cambridge, 2016, vol. 1, no. 2.
  * [127] “Convolutional neural network,” Apr 2021. [Online]. Available: https://en.wikipedia.org/wiki/Convolutional_neural_network
  * [128] “Deep reinforcement learning,” Feb 2021. [Online]. Available: https://en.wikipedia.org/wiki/Deep_reinforcement_learning
  * [129] Y. Lin and Y. Sun, “Grasp planning to maximize task coverage,” _The International Journal of Robotics Research_ , vol. 34, no. 9, pp. 1195–1210, 2015\.
  * [130] L. C. Hale, “Principles and techniques for designing precision machines,” Lawrence Livermore National Lab.(LLNL), Livermore, CA (United States), Tech. Rep., 1999.
  * [131] “Degrees of freedom (mechanics),” Mar 2021. [Online]. Available: https://en.wikipedia.org/wiki/Degrees_of_freedom_(mechanics)
  * [132] J. Wolfartsberger, J. D. Hallewell Haslwanter, and R. Lindorfer, “Perspectives on assistive systems for manual assembly tasks in industry,” _Technologies_ , vol. 7, no. 1, p. 12, 2019.
  * [133] Q. Liu, Z. Liu, W. Xu, Q. Tang, Z. Zhou, and D. T. Pham, “Human-robot collaboration in disassembly for sustainable manufacturing,” _International Journal of Production Research_ , vol. 57, no. 12, pp. 4027–4044, 2019.
  * [134] I. Spectrum, “Unimate,” May 2018. [Online]. Available: https://robots.ieee.org/robots/unimate/
  * [135] “Industrial robot,” Feb 2021. [Online]. Available: https://en.wikipedia.org/wiki/Industrial_robot
  * [136] “Cartesian coordinate robot,” Jan 2021. [Online]. Available: https://en.wikipedia.org/wiki/Cartesian_coordinate_robot
  * [137] “Sciclops microplate handler,” Dec 2020. [Online]. Available: https://hudsonrobotics.com/microplate-handling-2/platecrane-sciclops-3/
  * [138] Dejan and N. Kerley, “Scara robot: How to build your own arduino based robot,” Feb 2021. [Online]. Available: https://howtomechatronics.com/projects/scara-robot-how-to-build-your-own-arduino-based-robot/
  * [139] [Online]. Available: https://www.youtube.com/
  * [140] M. Wilson, _Implementation of robot systems: an introduction to robotics, automation, and successful systems integration in manufacturing_. Butterworth-Heinemann, 2014.
  * [141] “The top six types of industrial robots in 2020: Diy-robotics blog,” Feb 2021\. [Online]. Available: https://diy-robotics.com/blog/top-six-types-industrial-robots-2020/
  * [142] [Online]. Available: https://www.iso.org/obp/ui/#iso:std:iso:8373:ed-2:v1:en
  * [143] “United states department of labor.” [Online]. Available: https://www.osha.gov/dts/osta/otm/otm_iv/otm_iv_4.html#2
  * [144] X. Yang, H. Liu, J. Xiao, W. Zhu, Q. Liu, G. Gong, and T. Huang, “Continuous friction feedforward sliding mode controller for a trimule hybrid robot,” _IEEE/ASME Transactions on Mechatronics_ , vol. 23, no. 4, pp. 1673–1683, 2018.
  * [145] D. L. Peiper, “The kinematics of manipulators under computer control,” Stanford univ ca dept of computer science, Tech. Rep., 1968.
  * [146] “Inverse kinematics,” Mar 2021. [Online]. Available: https://en.wikipedia.org/wiki/Inverse_kinematics
  * [147] S. Iwamoto, “Inverse theorem in dynamic programming i,” _Journal of Mathematical Analysis and Applications_ , vol. 58, no. 1, pp. 113–134, 1977.
  * [148] G. J. Monkman, S. Hesse, R. Steinmann, and H. Schunk, _Robot grippers_. John Wiley & Sons, 2007.
  * [149] “Robot end effector,” Jan 2021. [Online]. Available: https://en.wikipedia.org/wiki/Robot_end_effector
  * [150] W. Wang, R. N. Loh, and E. Y. Gu, “Passive compliance versus active compliance in robot-based automated assembly systems,” _Industrial Robot: An International Journal_ , 1998.
  * [151] D. Logan, “A first course in the finite element method. cengage learning, stamford, ct.” 2012.
  * [152] “Finite element method,” Mar 2021. [Online]. Available: https://en.wikipedia.org/wiki/Finite_element_method
  * [153] “Natural language processing,” Apr 2021. [Online]. Available: https://en.wikipedia.org/wiki/Natural_language_processing
  * [154] “Cyber-physical system,” Apr 2021. [Online]. Available: https://en.wikipedia.org/wiki/Cyber-physical_system
  * [155] “Pick-and-place machine,” Oct 2020. [Online]. Available: https://en.wikipedia.org/wiki/Pick-and-place_machine
  * [156] “Petri net,” Apr 2021. [Online]. Available: https://en.wikipedia.org/wiki/Petri_net
  * [157] M. Ochalek, B. Jenett, O. Formoso, C. Gregg, G. Trinh, and K. Cheung, “Geometry systems for lattice-based reconfigurable space structures,” in _2019 IEEE Aerospace Conference_. IEEE, 2019, pp. 1–10.
  * [158] N. A. Fleck, “An overview of the mechanical properties of foams and periodic lattice materials,” _Cell Met Polym_ , pp. 3–7, 2004.
  * [159] V. Deshpande, M. Ashby, and N. Fleck, “Foam topology: bending versus stretching dominated architectures,” _Acta materialia_ , vol. 49, no. 6, pp. 1035–1040, 2001.
  * [160] E. Kroll and B. S. Carver, “Disassembly analysis through time estimation and other metrics,” _Robotics and Computer-Integrated Manufacturing_ , vol. 15, no. 3, pp. 191–200, 1999.
  * [161] K. Kimble, K. Van Wyk, J. Falco, E. Messina, Y. Sun, M. Shibata, W. Uemura, and Y. Yokokohji, “Benchmarking protocols for evaluating small parts robotic assembly systems,” _IEEE Robotics and Automation Letters_ , vol. 5, no. 2, pp. 883–889, 2020.
  * [162] J. Reap and B. Bras, “Design for disassembly and the value of robotic semi-destructive disassembly,” in _International Design Engineering Technical Conferences and Computers and Information in Engineering Conference_ , vol. 36231, 2002, pp. 275–281.
  * [163] L. Murray, W. Liu, A. Winfield, J. Timmis, and A. Tyrrell, “Analysing the reliability of a self-reconfigurable modular robotic system,” in _International Conference on Bio-Inspired Models of Network, Information, and Computing Systems_. Springer, 2011, pp. 44–58.
  * [164] E. Melnik, I. Korovin, and A. Klimenko, “Improving dependability of reconfigurable robotic control system,” in _International Conference on Interactive Collaborative Robotics_. Springer, 2017, pp. 144–152.
  * [165] S. K. Das, P. Yedlarajiah, and R. Narendra, “An approach for estimating the end-of-life product disassembly effort and cost,” _International Journal of Production Research_ , vol. 38, no. 3, pp. 657–673, 2000.
  * [166] A. Renteria and E. Alvarez-de-los Mozos, “Human-robot collaboration as a new paradigm in circular economy for weee management,” _Procedia Manufacturing_ , vol. 38, pp. 375–382, 2019.
  * [167] X. Guo, M. Zhou, S. Liu, and L. Qi, “Multiresource-constrained selective disassembly with maximal profit and minimal energy consumption,” _IEEE Transactions on Automation Science and Engineering_ , 2020.
  * [168] L. K. MOYER and S. M. GUPTA, “Environmental concerns and recycling/disassembly efforts in the electronics industry,” _Journal of Electronics Manufacturing_ , vol. 7, no. 01, pp. 1–22, 1997.
  * [169] T. C. Kuo, “Enhancing disassembly and recycling planning using life-cycle analysis,” _Robotics and Computer-Integrated Manufacturing_ , vol. 22, no. 5-6, pp. 420–428, 2006.
  * [170] A. Padilla-Rivera, S. Russo-Garrido, and N. Merveille, “Addressing the social aspects of a circular economy: A systematic literature review,” _Sustainability_ , vol. 12, no. 19, p. 7912, 2020.
  * [171] K. Arredondo-Soto, A. Realyvasquez-Vargas, A. Maldonado-Macías, and J. García-Alcaraz, “Impact of human resources on remanufacturing process, internal complexity, perceived quality of core, numerosity, and key process indicators,” _Robotics and computer-integrated manufacturing_ , vol. 59, pp. 168–176, 2019.
  * [172] R. Gerbers, K. Wegener, F. Dietrich, and K. Dröder, “Safe, flexible and productive human-robot-collaboration for disassembly of lithium-ion batteries,” in _Recycling of Lithium-Ion Batteries_. Springer, 2018, pp. 99–126.
  * [173] M. Dalle Mura, F. Pistolesi, G. Dini, and B. Lazzerini, “End-of-life product disassembly with priority-based extraction of dangerous parts,” _Journal of Intelligent Manufacturing_ , pp. 1–18, 2020.
  * [174] S. Robla-Gómez, V. M. Becerra, J. R. Llata, E. Gonzalez-Sarabia, C. Torre-Ferrero, and J. Perez-Oria, “Working together: A review on safe human-robot collaboration in industrial environments,” _IEEE Access_ , vol. 5, pp. 26 754–26 773, 2017.
  * [175] H. Fakhurldeen, F. Dailami, and A. G. Pipe, “Cara system architecture-a click and assemble robotic assembly system,” in _2019 International Conference on Robotics and Automation (ICRA)_. IEEE, 2019, pp. 5830–5836.
  * [176] K. Ashton _et al._ , “That ‘internet of things’ thing,” _RFID journal_ , vol. 22, no. 7, pp. 97–114, 2009.
  * [177] “What the internet of everything really is - a deep dive,” Jun 2020. [Online]. Available: https://www.i-scoop.eu/internet-of-things-guide/internet-of-everything/
  * [178] “Industry 4.0,” May 2020. [Online]. Available: https://en.wikipedia.org/wiki/Industry_4.0
  * [179] “Smart manufacturing,” May 2020. [Online]. Available: https://en.wikipedia.org/wiki/Smart_manufacturing
  * [180] K. Chailoet and E. Pengwang, “Assembly of modular robot for cleaning various length of solar panels,” in _IOP Conference Series: Materials Science and Engineering_ , vol. 639, no. 1. IOP Publishing, 2019, p. 012014.
  * [181] H. Poschmann, H. Brüggemann, and D. Goldmann, “Disassembly 4.0: a review on using robotics in disassembly tasks as a way of automation,” _Chemie Ingenieur Technik_ , vol. 92, no. 4, pp. 341–359, 2020.

## Acronyms

1D

    One-Dimensional
2D

    Two-Dimensional
3D

    Three-Dimensional
6D

    Six-Dimensional
AAOG

    Asassembly [4]AOG
AC

    Ant Colony
ACO

     [4]AC Optimization
BC

    Bee Colony
BCO

     [4]BC Optimization
A/DML

    Assembly/Disassembly [4]ML
AI

    Artificial Intelligence
AIS

    Advanced Immune System
AL

    Assembly Line
ALBP

     [4]AL [4]BP
AOG

    And/Or Graph
AR

    Augmented Reality
AS

    Assembly Sequence
ASAGA

    Adaptive [4]SA [4]GA
ASG

     [4]AS Generation
A/D-SP

    Assembly/Disassembly Sequence Planning
ASP

     [4]AS Planning
ASTP

    Assembly [4]STP
AtO

    Assemble to Order
BACO

    Bidirectional [4]ACO
BCC

    Body Centered Cubic
BCI

    Brain-Computer Interface
BFS

    Best-First Search
BIM

    Building Information Modelling
BP

    Balancing Problem
CA

    Computer-Aided
CAAD

     [4]CA Architectural Design
CAD

     [4]CA Design
CAM

     [4]CA Manufacturing
CAPP

     [4]CA [4]PP
CARA

    Click and Assemble [4]RA
CAS

    Complex Autonomous System
CASTP

    Cooperative [4]ASTP
CATIA

     [4]CA Three-dimensional Interactive Application
CCU

    Central Control Unit
CDSTP

    Cooperative [4]DSTP
CE

    Circular Economy
CIM

    Computer-Integrated Manufacturing
CNC

    Computer Numerical Control
CNN

    Convolutional Neural Network
CPH

    Components Per Hour
CPPS

    Cyber-Physical Production System
CPR

    Cable-driven Parallel Robot
CPS

    Cyber-Physical System
CPU

    Central Processing Unit
CSP

    Constraint Satisfaction Problem
DAOG

    Disassembly [4]AOG
DAWM

    Discretely-Assembled Walking Motor
DCB

    Disconnecting Circuit Breaker
DCI

    Disassembly Cost Index
DDI

    Disassembly Demand Index
DfA

    Design for Assembly
DfD

    Design for Disassembly
DfEoL

    Design for [4]EoL
DfR

    Design for Remanufacturing
DHGM

    Disassembly Hybrid Graph Model
DHI

    Disassembly Handling Index
DIG

    Disassembly Interference Graph
DL

    Disassembly Line
DLBP

     [4]DL [4]BP
DoF

    Degrees of Freedom
DOG

    Disassembly Order Graph
DOI

    Disassembly Operation Index
DP

    Dynamic Programming
DPD

    Delay Product Differentiation
DRA

    Discrete [4]RA
DRL

    Deep [4]RL
DS

    Disassembly Sequence
DSTP

    Disassembly [4]STP
DSP

     [4]DS Planning
DSS

    Decision Support System
DTW

    Dynamic Time Warping
EA/DML

    Educational [4]A/DML
EE

    End-Effector
EEG

    ElectroEncephaloGraphy
EF-M

    Enhanced Function-Means
EoL

    End-of-Life
EV

    Exploded View
EVD

     [4]EV Diagram
EVG

     [4]EV Generation
FEA

    Finite Element Analysis
FPCA

    Functional Principal Component Analysis
GA

    Genetic Algorithm
GEA

    Games and Entertainment Artifacts
GWO

    Grey Wolf Optimization
HAWA

    Hybrid Ant–Wolf Algorithm
HCBA

    Hybrid Cuckoo–Bat Algorithm
HitL

    Human-in-the-Loop
HPN

    Hybrid [4]PN
HRC

    Human-Robot Collaboration
HRCA

     [4]HRC for Assembly
HRCD

     [4]HRC for Disassembly
IC

    Integrated Circuit
IDP

    Inverse [4]DP
IIoT

    Industrial [4]IoT
IKP

    Inverse Kinematic Problem
IoE

    Internet of Everything
IoT

    Internet of Things
IPS

    Industrial Path Solutions
KT

    Knowledge Transfer
LbD

    Learning by Demonstration
LC

    Life Cycle
LCA

     [4]LC Assessment
LLL

    Left–Left–Left
LLR

    Left–Left–Right
LoD

    Level Of Development
LRTDP

    Labeled Real-Time [4]DP
L-system

    Lindenmayer system
M2M

    Machine to Machine
MBGA

    Many-objective Best-order-sort [4]GA
MDBA-Pareto

    Modified Discrete Bees Algorithm based on Pareto
MDoF

    Multi [4]DoF
MDPs

    Markov Decision Processes
MILP

    Mixed Integer Linear Programming
MJPN

    Median-Joining Phylogenetic Network
ML

    Mechatronics Line
MO

    Multi-Objective
MOO

     [4]MO Optimization
MOOP

     [4]MOO Problem
MR

    Multi-Robot
MRAL

     [4]MR [4]AL
MRDL

     [4]MR [4]DL
MRR

    Modular Reconfigurable Robot
MTM

    Methods [4]TM
NPV

    Net Present Value
NLP

    Natural Language Processing
NP

    Non-deterministic Polynomial-time
NS

    Non-dominated Sorting
OASP

    Optimal [4]ASP
ODSP

    Optimal [4]DSP
OEM

    Original Equipment Manufacturer
OSA

    One-Side Assembly
PaP

    Pick-and-Place
PCB

    Printed Circuit Board
PCI

    Platforms Customisation Index
PD

    Parametric Design
PE

    Profit-oriented and Energy-efficient
PEASP

     [4]PE [4]ASP
PEDSP

     [4]PE [4]DSP
PN

    Petri Net
pose

    position and orientation
PP

    Process Planning
PRI

    Platforms Reconfiguration Index
QC

    Quality Control
RA

    Robotic Assembly
RAD

    Robotic Assembly and Disassembly
RAM

    Random-Access Memory
RC

    Reinforced Concrete
RD

    Robotic Disassembly
RFID

    Radio-Frequency IDentification
RL

    Reinforcement Learning
RM

    Robotic Manipulator
RP

    Rapid Prototyping
RRT

    Rapidly-exploring Random Trees
SA

    Simulated Annealing
SAAS

    Set of All [4]AS
SADS

    Set of All [4]DS
SAML

    Smart Assembly [4]ML
SC

    Simple Cubic
SCARA

    Selective Compliance Assembly Robot Arm
SDK

    Software Development Kit
SDR

    Set of Directions of Removal
SEMS

    Smart ElectroMechanical System
SHPN

    Synchronized Hybrid [4]PN
SM

    Smart Manufacturing
SMD

    Surface Mount Device
SMT

    Surface Mount Technology
SO

    Single-Objective
SP

    Sequence Planning
SOO

     [4]SO Optimization
SOOP

     [4]SOO Problem
SR

    Single-Robot
SS

    Scatter Search
STP

    Sequence and Task Planning
SWOT

    Strengths, Weaknesses, Opportunities, and Threats
TaMP

    Task and Motion Planning
TAOG

    Transformed [4]AOG
Tensegrity

    Tensional integrity
TM

    Time Measurement
TMU

     [4]TM Units
TNM

    TheNewMakers
TPD

    Task Precedence Diagram
TP-GMM

    Task Parametrized Gaussian Mixture Model
TPN

    Timed [4]PN
VA

    Virtual Assembly
VD

    Virtual Disassembly
VR

    Virtual Reality
WD

    Wavelet Decomposition
WEEE

    Waste Electrical and Electronic Equipment
WMR

    Wheeled Mobile Robot
WRS

    World Robot Summit
WS

    WorkStation
  *[RAD]: Robotic Assembly and Disassembly
  *[[]:
  *[RA]: Robotic Assembly
  *[RD]: Robotic Disassembly
  *[SP]: Sequence Planning
  *[3D]: Three-Dimensional
  *[CE]: Circular Economy
  *[EE]: End-Effector
  *[HRC]: Human-Robot Collaboration