# MVM3Det: A Novel Method for Multi-view Monocular 3D Detection

Haoran Li1, Zicheng Duan2, Mingjun Ma1, Yaran Chen1, Jiaqi Li3 and Dongbin
Zhao1 1Haoran Li, Mingjun Ma, Yaran Chen and Dongbin Zhao are with the State
Key Laboratory of Management and Control for Complex Systems, Institute of
Automation, Chinese Academy of Sciences, Beijing, 100190, China, and also with
the University of Chinese Academy of Sciences, Beijing, China (email :
<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>dongbin.zhao@ia.ac.cn)2Zicheng Duan is with the College of Engineering and
Computer Science, Australian National University, ACT, 2601, Australia (email
: zicheng.duan@anu.edu.au)3Jiaqi Li is with the School of Mechanical
Engineering, Beijing Institute of Technology, Beijing, 100081, China (email:
xuer0324@gmail.com)This work is supported by the Strategic Priority Research
Program of Chinese Academy of Sciences (CAS) under Grant XDA27030400, and the
National Natural Science Foundation of China (NSFC) under Grants No. 62006226.

###### Abstract

Monocular 3D object detection encounters occlusion problems in many
application scenarios, such as traffic monitoring, pedestrian monitoring,
etc., which leads to serious false negatives. Multi-view object detection
effectively solves this problem by combining data from different perspectives.
However, due to label confusion and feature confusion, the orientation
estimation of multi-view 3D object detection is intractable, which is
important for object tracking and intention prediction. In this paper, we
propose a novel multi-view 3D object detection method named MVM3Det which
simultaneously estimates the 3D position and orientation of the object
according to the multi-view monocular information. The method consists of two
parts: 1) Position proposal network, which integrates the features from
different perspectives into consistent global features through feature
orthogonal transformation to estimate the position. 2) Multi-branch
orientation estimation network, which introduces feature perspective pooling
to overcome the two confusion problems during the orientation estimation. In
addition, we present the first dataset for multi-view 3D object detection
named MVM3D111We will release the code and dataset at https://github.com/DRL-
CASIA/MVM3D.. Comparing with State-Of-The-Art (SOTA) methods on our dataset
and public dataset WildTrack, our method achieves very competitive results.

##  I INTRODUCTION

Object detection is a fundamental task for autonomous robot system, which has
developed vigorously and has been widely deployed in perception systems.
However, due to the lack of 3D information, 2D object detection is hard to
provide sufficient information for decision-making and planning of autonomous
robots. Therefore, a large amount of 3D object detection methods have been
developed for 3D pose estimation, such as monocular-based methods [1, 2],
LiDAR-based methods [3, 4, 5], and multi-sensor fusion methods [6, 7, 8].
However, these methods only use the sensor data obtained from a single
perspective, and the mutual occlusion between objects causes serious
information loss. Even for multi-sensor fusion methods whose relative
positions between the sensors are close, the ability of these methods to
alleviate blind spots in the field of view is limited. Since almost current
methods are based on a single view, the false negative caused by the occlusion
has become one of the main problems in the application.

Multi-view 3D object detection is one of the possible solutions for this
problem, which fuses the information from different perspectives to overcome
detection failures caused by occlusion in single perspective scenarios, hence
providing robust results and accurate pose estimation. However, estimating a
unified 3D pose from sensor data captured from different perspectives is the
main challenge of multi-view object detection, especially in the case of lack
of depth information. Firstly, since the information from each view is
different, how to effectively fuse multi-view information into unified global
information is inevitable problem for 3D location estimation. Secondly,
orientation is vital for trajectory tracking and intention prediction, but
orientation estimation with multiple viewing information is intractable with
two confusion problems: 1) label confusion and 2) feature confusion. The first
problem is common in monocular 3D detection. Relative orientation labels are
usually used to eliminate label ambiguity caused by different line of sight
angles. However, since the relative orientation labels in each view are
different, it is difficult to find a consistent relative orientation label for
all different views in multi-view detection. The second problem is that due to
the perspective transformation of multiple cameras, the global features of
objects which have same orientation and different positions are different. At
present, there is little work focusing on overcoming this feature confusion
for accurate orientation estimation.

Another important factor restricting the development of multi-view 3D object
detection methods is the lack of datasets. At present, although there are many
datasets [9, 10] for monocular 3D object detection, these datasets are only
limited to verifying the detection methods from a single perspective. Arnold
_et al._ [11] uses LiDAR to collect roundabout data in the simulation
environment to develop LiDAR-based multi-view 3D object detection method, but
does not involve multi-view monocular data. The WildTrack dataset [12] is
closest to the requirements of multi-view monocular 3D detection method.
However, this dataset not only contains a small amount of data, but also has
no orientation labels of objects. Therefore, the development of multi-view
monocular 3D object detection method is largely limited to the lack of
appropriate datasets.

In this paper, we propose a novel multi-view monocular 3D object detection
method MVM3Det that simultaneously estimates the position and orientation.
This method contains position proposal network which achieves the consistent
global features through feature orthogonal projection, and multi-branch
orientation estimation network which alleviates features confusion problem
with feature perspective pooling. In addition, we present the dataset MVM3D
for multi-view 3D object detection, hoping to promote the development of
multi-view 3D object detection methods. The main contributions of this paper
are as follows.

  * 1)

In order to estimate unique position of the object with multi-view
information, we propose feature orthogonal projection and construct a position
proposal network. This network generates the consistent global features from
multi-view cues.

  * 2)

We design multi-branch orientation estimation network with feature perspective
pooling to estimate the corresponding relative orientation for each
perspective. This network alleviates features confusion problem during
orientation estimation. Based on above modules, a multi-view 3D detection
method is proposed. It is worth mentioning that our method is the first method
that estimates the orientation and position simultaneously with multi-view
monocular information.

  * 3)

We present the first multi-view 3D object detection dataset named MVM3D.
Compared with the SOTA methods on our dataset and public dataset WildTrack,
the proposed method achieves very competitive results. On MV3D dataset, our
method achieves 95.9% MODA, an 1.1% increases over SOTA method. Our method
achieves 49.0% $AP_{3D}$, an 9.7% increases over monocular based 3D detection
method.

##  II REALTED WORK

###  II-A Monocular based detection

There are serveral monocular based 3D detection methods for autonomous robots.
Due to the lack of depth information in the image, some assumptions are
usually required to restore the 3D pose of the object from the image. For
example, objects are assumed to have the same altitude [13]. This assumption
cannot be satisfied in many automatic driving scenes. Therefore, OFTNet [1]
realizes orthogonal feature transformation within a certain height range to
weaken the previous assumption. In addition, some methods [14, 15, 16] abandon
the altitude hypothesis. They use the 2D bounding boxes obtained by the 2D
object detection method, and predict the size and orientation of the object.
Finally, the spatial position of the object is estimated by solving
perspective N – points problem. These methods are sensitive to the prediction
accuracy of object size and orientation. Another kind of methods is based on
monocular depth estimation [17, 18, 19]. This kind of methods needs to pre-
train the depth estimation network, and then estimate the 3D pose of the
object combined with the 2D detection frame.

###  II-B Multi-view based detection

Monocular based methods confront the false negative problem caused by the
occlusion, especially when the objects are crowded. In the past two years,
some multiple perspectives based methods have been developed to overcome the
occlusion problem by capturing information from multiple perspectives. Fleuret
_et al._ [20] propose probabilistic occupancy map method which estimates the
probabilities of occupancy on the ground plane with multi-view information.
Peng _et al._ [21] builds a multi-view network which combines multiple
Bayesian networks from per view and predicts the localization of the objects.
Baque _et al._ [22] fuses the results of 2D target detection and used
convolutional neural network and conditional random field to estimate
pedestrian occupancy jointly. Hou _et al._ [23] chooses an efficient anchor
free feature aggregation method to regress pedestrian occupancy as a Gaussian
distribution. However, these methods only realize position estimation for
pedestrian detection, and do not estimate the orientation of objects. In this
paper, a novel multi-view 3D object detection method is proposed, which
estimates the position and orientation simultaneously.

Figure 1: The architecture of MVM3Det. The network consists of PPN and multi-
branch orientation estimation network. PPN estimates the position of the
object by fusing the information from multiple perspectives through the
Feature Orthogonal Transformation (FOT). Multi-branch orientation estimation
network predicts the orientation in each perspective through the Feature
Perspective Pooling (FPP). Non-Maximum Suppression (NMS) is used to select
final prediction.

##  III Method

In this paper, we propose a novel multi-view 3D object detection method. This
method simultaneously estimates 3D position and orientation from multi-view
monocular information, and effectively alleviate the problem of false negative
caused by the occlusion. This method mainly consists of two parts: position
proposal network and multi-branch orientation estimation network. Position
proposal network obtains the consistent global features from data of different
perspectives, and uses anchor-based method to estimate the spatial position of
objects. Multi-branch orientation estimation introduces feature perspective
pooling to realize orientation estimation from different perspectives
according to the previous position, so as to alleviate the confusion problem.
The network structure is shown in Fig. 1.

###  III-A Position proposal network

The purpose of Position Proposal Network (PPN) is to estimate the possible 3D
spatial position of objects according to the data obtained from different
perspectives. Different from the classical Region Proposal Network (RPN), PPN
includes three processes: feature extraction, feature fusion and position
proposal. In this paper, the monocular camera is used for each view to monitor
the objects. Considering the great breakthrough of deep convolutional networks
in the field of image processing, we employ ResNet-18 [24] as the feature
extractor of the image in each perspective to capture deep features.

Since the information observed from each perspective is different, it is
meaningless to directly fuse the deep features obtained by ResNet in image
space. Therefore, the premise of feature fusion is to align the features
captured from different perspectives in a unified feature space. In this
paper, we assume that the altitude of the objects is approximately distributed
on a horizontal plane, then we introduces the feature orthogonal
transformation to project the deep features from different perspectives into a
Birds-Eye-View (BEV) space. Similar to [1, 23, 25], when the altitude of the
distribution plane of the objects is known, we calculate the position of each
pixel from deep features in the BEV space by orthogonal transformation.

$\Gamma=\tau(R^{-1}K^{-1}U+R^{-1}T).$

Here $\Gamma$ is 3D position, and $U$ is the homogeneous coordinates of the
pixel in the camera image. $K$ is the camera matrix. $R$ and $T$ are the
rotation matrix and translation vector of the camera, respectively. $\tau$ is
scale factor which is estimated by

$\tau=\frac{(R^{-1}K^{-1}U+R^{-1}T)|_{z}}{z^{P}}.$

$V|_{z}$ means the $z$ axis value of the vector $V$. $z^{P}$ is the altitude
of the objects. For each deep feature of each view, the above orthogonal
transformation is used to convert image features to the BEV space. We stack
the projected features with coordinate maps and fuse them in the BEV space by
the convolutional network, so as to obtain consistent global features.

Based on the global features, we introduce anchor-based method for location
estimation. Similar to RPN, the global feature is input into the fully
convolutional network to predict the probability of an object at the
corresponding position of each anchor and the offset of the object center
relative to the anchor center. During the training process, the prediction
bounding box whose Intersection over Union (IoU) with the ground truth is
greater than 0.7 is taken as the positive sample, and the prediction bounding
box whose IoU is less than 0.3 is taken as the negative sample. During the
inference process, the confidence threshold and Non-Maximum Suppression (NMS)
[26] are applied to predict the possible position.

###  III-B Multi-branch orientation estimation network

Orientation estimation is an essential part of 3D object detection. However,
there are several challenges to estimate orientation with multi-view monocular
information. Similar to the monocular 3D object detection methods, multi-view
orientation estimation also faces the same label confusion problem. This
problem means that the different line of sight relative to the camera makes
the observed states various when the object orientation is fixed, resulting in
the same label corresponding to different states, as shown with ②, ④ and ⑤ in
Fig. 2. The usual solution is to redefine the label according to the line of
sight angle to eliminate this ambiguity. When objects with the same
orientation are in the same line of sight, there is no label confusion
problem, but due to the influence of the feature orthogonal projection, the
previous consistent global features face another confusion problem. This is
due to the altitude assumption, which causes the streak near the projection
center. This streak leads to the global features of object at different
positions are quite different, even though they have the same orientation on
the same line of sight angle. This phenomenon is called feature confusion, as
shown with ①, ② and ③ in Fig. 2.

Since feature confusion is caused by the orthogonal transformation of
features, the global features are not suitable for orientation estimation.
Here we propose feature perspective pooling which combines features from each
perspective and the position proposals. Firstly, according to the position
obtained by PPN, the pool region corresponding to the 3D position in the image
of each view is calculated through perspective transformation. Specifically, 8
vertices of the bounding box are obtained according to the estimated center
position of the object and the predefined orientation. According to the
parameters of the camera, the position of each vertex in the image is
calculated by perspective transformation. We obtain the Region Of Interest
(ROI) by calculating the minimum outer rectangle of the 8 projection vertices.
Then, ROI pooling is used to obtain the features for orientation prediction
under each perspective. During the perspective pooling process, we assume the
altitude and height of the object are known.

Similar to [14], we do not directly regress the orientation, but divide the
value into $N$ intervals. Each branch network predicts the confidence of the
object included in the ROI, the probability that the object angle falls in
region $[\frac{2\pi}{N}(i-1),\frac{2\pi}{N}i],i\in\\{1,2,\cdots,N\\}$, and the
offset of the object angle relative to the center value of the region. During
training, the label of offset $o_{i}$ is calculated as follows

$o_{i}=\beta-\frac{\pi}{N}(2*i-1),\text{where }i\in\\{1,2,\cdots,N\\},$

where $\beta$ is the ground truth of the orientation. In order to eliminate
label confusion, the value is the angle relative to the line of sight. Since
the multi-branch method predicts the orientation of the object in each
perspective, each object may have multiple prediction bounding boxes with
different orientation. During the inference, we first rank the boxes according
to the confidence, and then apply NMS to determine the final prediction
result.

Figure 2: Confusion problems for orientation estimation. The upper row
describes that 5 robots with the same orientation and different positions. The
second line is the content of the robots in the image space. The last line is
the content of the robots in BEV space.

###  III-C Multi-view based 3D object detection

Based on the PPN and multi-branch orientation estimation network, we propose a
multi-view 3D detection method MVM3Det. This method takes the monocular image
from multiple perspectives as the input and extracts the features through the
sharing ResNet-18, then obtains the consistent global feature through the
feature orthogonal transformation, and applies the anchor based method to
estimate the possible spatial position of the object. After confidence
threshold and NMS, the estimated position is combined with feature perspective
pooling to obtain the features of ROI in each perspective and estimates the
orientation of the object in each perspective. Finally, the bounding boxes are
obtained by NMS. In the training process, considering that the orientation
estimation network is based on accurate position estimation, we adopt a phased
training method: first train the PPN, and then train the multi-branch
orientation estimation network.

####  III-C1 Position proposal network loss

PPN loss $\mathcal{L}^{PPN}$ consists of two parts

$\displaystyle\mathcal{L}^{PPN}=$
$\displaystyle\frac{1}{N_{conf}}\sum_{i}\mathcal{L}_{conf}(\hat{p}_{i},p_{i})$
(1)
$\displaystyle+\lambda^{PPN}\frac{1}{N_{val}}\sum_{i}p_{i}\mathcal{L}_{offset}(\hat{t}^{BEV}_{i},t^{BEV}_{i})$
$\displaystyle+\lambda^{PPN}_{2D}\frac{1}{N_{val}}\sum_{i}\sum_{v}p_{i}\mathcal{L}_{offset}(\hat{t}^{v}_{i},t^{v}_{i}),$

where the first part represents the loss of predicted confidence of each
anchor. $\hat{p}_{i}\in\\{0,1\\}$ and $p_{i}$ are the ground truth label and
predicted confidence, respectively. $N_{conf}$ represents the total number of
anchors. The second part represents the offset loss of anchor in BEV space,
$\hat{t}^{BEV}_{i}$ represents the offset between the true position and
anchor, and $t^{BEV}_{i}$ represents the predicted offset. Here, only offset
loss of anchor involving objects are considered. $N_{val}$ represents the
number of the valide anchors which involve objects. The last part is multi-
brach regression loss which represents the offset loss of 2D bounding box in
the image space under the different view $v$. The box is projected from the
BEV position prediction. Experiments show that this loss function improves the
accuracy of position estimation. Softmax loss and smooth $L1$ loss are adopted
for $\mathcal{L}_{conf}$ and $\mathcal{L}_{offset}$ respectively. We follow
[27] to encode $t^{BEV}_{i}$ and $t^{v}_{i}$. $\lambda^{PPN}$ and
$\lambda^{PPN}_{2D}$ are balance parameters.

####  III-C2 Multi-branch orientation estimation network loss

For multi-branch orientation estimation network, we predict three parts: 1)
the confidence of object existence , 2) the orientation interval
classification probability and 3) the orientation offsets. Therefore, the loss
function also consists of three parts.

$\displaystyle\mathcal{L}^{MBON}=$
$\displaystyle\sum_{v}\frac{1}{N^{v}_{conf}}\sum_{i}\mathcal{L}_{conf}(\hat{p}^{v}_{i},p^{v}_{i})$
(2)
$\displaystyle+\frac{1}{N^{v}_{val}}\sum_{i}p^{v}_{i}\mathcal{L}_{cls}(\hat{b}^{v}_{i},b^{v}_{i})$
$\displaystyle+\lambda^{MBON}\frac{1}{N^{v}_{val}}\sum_{i}p^{v}_{i}\mathcal{L}_{ori}(\hat{o}^{v}_{i},o^{v}_{i})$

where $p^{v}_{i}$ represents the true label of the $i$-th predicted bounding
box in the $v$-th perspective. Second part $\mathcal{L}_{cls}$ is multi-bin
classification loss which calculates distribution distance between the one-hot
label and predicted probability distribution. Here softmax loss is used for
this loss. $b_{i}$ represents the probability distribution of the $i$-th box
over orientation intervals, and $o_{i}$ represents the offset between the
orientation and the interval center. $N^{v}_{conf}$ is the total number of the
predicted bounding boxes in view $v$. $N^{v}_{val}$ is the number of the
validate boxes. Considering the periodicity of the orientation, we employ the
cosine function to encode the offset prediction error as the offset prediction
loss $\mathcal{L}_{ori}$, which is same as [14].

##  IV EXPERIMENT

In order to verify the performance of the proposed multi-view 3D detection
method, we present a multi-view monocular 3D object dataset, and conduct
ablation experiments on the dataset. In addition, we compare the performance
of position detection and orientation estimation with the current SOTA methods
on the proposed dataset and public dataset.

###  IV-A Multi-view monocular 3D object dataset

Although multi-view 3D object detection is very important for autonomous robot
system, there are few relevant algorithms so far, which is largely due to the
lack of relevant datasets. There are abundant 3D detection datasets for
autonomous driving, such as KITTI, Waymo open dataset, etc., which contain a
large amount of images, point cloud and 3D bounding boxes. However, the data
are collected from a single perspective, and there is still a problem of the
occlusion. The WildTrack dataset is the most similar to our proposed dataset,
but it only contains 400 samples and does not provide the orientation label of
the object. Therefore, it is unable to verify the orientation estimation
performance of the methods. Our proposed dataset contains 4330 pairs monocular
images collected from multiple perspectives and a large number of bounding
boxes including position and orientation.

The collection of datasets is completed in an 8m$\times$4.5m site. A pair of
monocular cameras are erected diagonally on the site to capture images of the
site. There are some obstacles with different heights and moving objects in
the site. In MVM3D dataset, the object to be detected is mobile robot. The
orientation distribution of labels in the dataset is very important for
orientation estimation. We try our best to ensure that the number of samples
is distributed uniformly in each angle interval. More information about
dataset can be found https://github.com/DRL-CASIA/MVM3D.

TABLE I: Comparison on WildTrack Methods | MODA | MODP | Prec. | Recall
---|---|---|---|---
(%) | (%) | (%) | (%)
RCNN & clustering [28] | 11.3 | 18.3 | 68 | 43
POM-CNN [20] | 23.2 | 30.5 | 75 | 55
DeepMCD [29] | 67.8 | 64.2 | 85 | 82
Deep-Occlusion [22] | 74.1 | 53.8 | 95 | 80
MVDet [23] | 88.2 | 75.7 | 94.7 | 93.6
MVM3Det(Ours) | 84.0 | 75.8 | 93.6 | 90.2
|  |  |  |

###  IV-B Evaluation metrics

Following the localization methods such as [23, 22], we choose Multi-Object
Detection Accuracy (MODA) and Multi-Object Detection Precision (MODP), recall,
precision as the evaluation metrics for the object localization [30]. Average
Orientation Similarity (AOS), Average Precision of 3D detection ($AP_{3D}$)
[31] and Orientation Score (OS) [14] are selected for the orientation
estimation.

###  IV-C Implementation details

During experiments, random brightness, random contrast and random saturation
are used for image augmentation. Similar to [23], ResNet-18 is applied as the
backbone network. The features in BEV space are interpolated into a fixed size
[120, 160]. During training PPN, 0.7 and 0.3 are the IoU thresholds and
employed to select positive samples and negative sample, respectively.
$\lambda^{PPN}$ is 3, and $\lambda^{PPN}_{2D}$ is 1. During training multi-
branch orientation estimation network, we select 128 samples from PPN with IoU
greater than 0.5 to train the network. $\lambda^{MBON}$ is 0.4. We use Adam
optimizer with learning rate $0.15\times 10^{-5}$ to train the networks for 15
epochs, and batchsize is set to 1.

TABLE II: Comparison on MVM3D Methods | MODA | MODP | Prec. | Recall
---|---|---|---|---
(%) | (%) | (%) | (%)
MVDet [23] | $94.8$ | $\bf 87.7$ | $97$ | $\bf 98.2$
MVM3Det(Ours) | 95.9 | 83.2 | 99.2 | 95.8
|  |  |  |

###  IV-D Localization performance

####  IV-D1 Results on WildTrack dataset

On WildTrack dataset, we compare the proposed method with the current SOTA
methods, and the results are shown in Table I. The results show that MVDet is
the best positioning method at present. Compared with that, the proposed
method is slightly better than MVDet in MODP, and the results of other metrics
are close. Compared with anchor free method MVDet, location estimation of
MVM3Det is an anchor based method. Due to the limitation of anchor, the recall
of this method is lower than that of anchor free method when the objects are
dense, so the MODA also is lower. Compared with the remaining methods, our
method has obvious advantages in positioning performance.

####  IV-D2 Results on MVM3D dataset

Compared with WildTrack dataset, MVM3D dataset is larger and the illumination
changes are richer. Moreover, due to the existence of obstacles, the occlusion
situation is more complex. It should be noted that although the MVM3D dataset
is larger than WildTrack dataset, the density of objects in each frame is less
than that of WildTrack. This is also the reason why MVM3Det has high recall on
this dataset. With close recall, anchor based method MVM3Det has better
precision, and higher MODA than MVDet, as shown in Table II. The visualization
of results on MVM3D dataset are shown in Fig 3. It can be seen from the
results in the figure that our proposed method can accurately estimate the
position and orientation of the object even when it is seriously blocked by
obstacles.

TABLE III: Ablation studies on MVM3D

| FOT | Multi-view | MBR | MODA | MODP | Prec. | Recall
---|---|---|---|---|---|---|---
(%) | (%) | (%) | (%)
1 |  |  |  | 12.7 | 43.2 | 50.6 | 22.0
2 | ✓ |  |  | 69.0 | 76.2 | 90.4 | 56.3
3 |  | ✓ |  | 27.9 | 69.3 | 91.8 | 30.6
4 | ✓ | ✓ |  | 94.7 | 80.6 | 96.6 | 99
5 | ✓ | ✓ | ✓ | 95.9 | 83.2 | 99.2 | 95.8
|  |  |  |  |  |  |

Ground truth

MVDet[23]

MVM3Det

Figure 3: Visualization of the results. Since MVDet does cannot provide
orientation information, we generate cylinders according to the predicted
location and robots ground truth radius.

###  IV-E Ablation studies for localization

We set up several ablation experiments for the proposed method on MVM3D to
analyze the localization performance. The results are shown in Table III.

####  IV-E1 Basic monocular model

The simplest baseline model is to project the original image into BEV space by
feature orthogonal transformation, and then uses ResNet-18 as feature
extractor to predict the position of objects in BEV space. The results are
shown in the 1-th row of Table III.

####  IV-E2 Feature orthographic transformation (FOT)

The baseline model of the 2-th row is modified based on the model of the 1-th
row. Specifically, the model applies ResNet-18 to extract features in image
space, and then uses feature orthogonal transformation to obtain the features
in BEV space and predict the spatial position of the object. Compared with the
results in the 1-th row, it is noticeable that the pre-trained ResNet-18 model
has better performance in the original image space.

####  IV-E3 Multi-view image fusion

Based on first baseline model, the model in the 3-th row adds the input from
multiple perspectives. Images projected from multiple perspectives into the
BEV space are stacked as the input to ResNet-18. Compared with the results in
the 1-th row, it can be seen that multi-view information increases the
location performance to a certain extent.

####  IV-E4 Multi-view feature fusion

The model of the 4-th row is based on the 3-th row, which employs ResNet-18 to
extract the deep features from the original perspective, and then uses the
feature orthogonal transformation to obtain the features in the BEV space.
This model is trained with first two part of PPN loss. The results in the 2-th
and 4-th rows show that feature fusion under multi-view greatly reduces the
false negatives and improves the performance of the method.

####  IV-E5 Multi-branch regression (MBR)

The last line is the multi-view 3D object detection method proposed in this
paper, which adds a multi-branch 2D bounding box regression loss. Compared
with the results in the 4-th and 5-th rows, it can be seen that the joint BEV
space loss and multi-branch regress loss increases the position estimation
accuracy to a certain extent.

###  IV-F Orientation performance

TABLE IV: Orientation performance on MVM3D Methods | IoU = 0.25 | IoU = 0.5
---|---|---
$AP_{3D}$ | AOS | OS | $AP_{3D}$ | AOS | OS
Monocular | 77.2% | 67.1% | 0.87 | 30.3% | 26.6% | 0.88
MVM3Det | 90.2% | 82.6% | 0.91 | 49.0% | 45.5% | 0.92
|  |  |  |  |  |

As far as we know, there is no method to realize orientation estimation with
multi-view monocular information. However, orientation is necessary to
monocular 3D detection. The baseline model compared here is a degraded model
based on our proposed method, which only uses single view information. It can
be seen from the results in Table IV that multi-view fusion greatly improves
the accuracy of orientation estimation.

##  V CONCLUSIONS

In this paper, a novel multi-view monocular 3D object detection method is
proposed, which overcomes false negatives caused by the occlusion and the
confusion problem of orientation estimation under multiple perspectives. This
method is mainly composed of PPN and multi-branch orientation estimation
network. PPN fuses the data from different perspectives through the feature
orthogonal transformation to estimate the spatial position of the objects.
Multi-branch orientation estimation introduces feature perspective pooling to
realize orientation estimation from various perspectives, so as to alleviate
the problems of label confusion and feature confusion in orientation
estimation. In addition, in order to promote the development of multi-view 3D
object detection, we present the first multi-view monocular 3D dataset MVM3D,
covering different illumination and complex occlusion scenes. Through the
experiments on our dataset and public dataset, the proposed method achieves
competitive results to the current SOTA in position estimation, and realizes
orientation estimation for the first time.

## References

  * [1] T. Roddick, A. Kendall, and R. Cipolla, “Orthographic feature transform for monocular 3D object detection,” _arXiv preprint arXiv:1811.08188_ , 2018\.
  * [2] B. Tekin, S. N. Sinha, and P. Fua, “Real-time seamless single shot 6D object pose prediction,” in _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)_ , 2018, pp. 292–301.
  * [3] Y. Zhou and O. Tuzel, “VoxelNet: End-to-end learning for point cloud based 3D object detection,” in _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)_ , 2018, pp. 4490–4499.
  * [4] W. Shi and R. Rajkumar, “Point-GNN: Graph neural network for 3D object detection in a point cloud,” in _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)_ , 2020, pp. 1708–1716.
  * [5] H. Li, X. Zhou, Y. Chen, Q. Zhang, D. Zhao, and D. Qian, “Comparison of 3D object detection based on lidar point cloud,” in _Proceedings of the IEEE 8th Data Driven Control and Learning Systems Conference (DDCLS)_ , 2019, pp. 678–685.
  * [6] X. Chen, H. Ma, J. Wan, B. Li, and T. Xia, “Multi-view 3D object detection network for autonomous driving,” in _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)_ , 2017, pp. 1907–1915.
  * [7] J. Ku, M. Mozifian, J. Lee, A. Harakeh, and S. L. Waslander, “Joint 3D proposal generation and object detection from view aggregation,” in _Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)_. IEEE, 2018, pp. 1–8.
  * [8] Y. Chen, H. Li, R. Gao, and D. Zhao, “Boost 3D object detection via point clouds segmentation and fused 3D giou-l1 loss,” _IEEE Transactions on Neural Networks and Learning Systems_ , 2020.
  * [9] A. Geiger, P. Lenz, C. Stiller, and R. Urtasun, “Vision meets robotics: The KITTI dataset,” _International Journal of Robotics Research (IJRR)_ , 2013\.
  * [10] “Waymo open dataset: An autonomous driving dataset,” 2019.
  * [11] E. Arnold, M. Dianati, R. de Temple, and S. Fallah, “Cooperative perception for 3D object detection in driving scenarios using infrastructure sensors,” _IEEE Transactions on Intelligent Transportation Systems_ , pp. 1–13, 2020.
  * [12] T. Chavdarova, P. Baqué, S. Bouquet, A. Maksai, C. Jose, T. Bagautdinov, L. Lettry, P. Fua, L. Van Gool, and F. Fleuret, “WILDTRACK: A multi-camera HD dataset for dense unscripted pedestrian detection,” in _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)_ , 2018, pp. 5030–5039.
  * [13] C. Yan and E. Salman, “Mono3D: Open source cell library for monolithic 3D integrated circuits,” _IEEE Transactions on Circuits and Systems I: Regular Papers_ , vol. 65, no. 3, pp. 1075–1085, 2017.
  * [14] A. Mousavian, D. Anguelov, J. Flynn, and J. Kosecka, “3D bounding box estimation using deep learning and geometry,” in _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)_ , 2017, pp. 7074–7082.
  * [15] L. Liu, J. Lu, C. Xu, Q. Tian, and J. Zhou, “Deep fitting degree scoring network for monocular 3D object detection,” in _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)_ , 2019, pp. 1057–1066.
  * [16] B. Li, W. Ouyang, L. Sheng, X. Zeng, and X. Wang, “GS3D: An efficient 3D object detection framework for autonomous driving,” in _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)_ , 2019, pp. 1019–1028.
  * [17] F. Manhardt, W. Kehl, and A. Gaidon, “ROI-10D: Monocular lifting of 2D detection to 6D pose and metric shape,” in _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)_ , 2019, pp. 2064–2073.
  * [18] J. Ku, A. D. Pon, and S. L. Waslander, “Monocular 3D object detection leveraging accurate proposals and shape reconstruction,” in _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)_ , 2019, pp. 11 859–11 868.
  * [19] Z. Qin, J. Wang, and Y. Lu, “MonoGRNet: A general framework for monocular 3D object detection,” _IEEE Transactions on Pattern Analysis and Machine Intelligence_ , pp. 1–1, 2021.
  * [20] F. Fleuret, J. Berclaz, R. Lengagne, and P. Fua, “Multicamera people tracking with a probabilistic occupancy map,” _IEEE Transactions on Pattern Analysis and Machine Intelligence_ , vol. 30, no. 2, pp. 267–282, 2008.
  * [21] P. Peng, Y. Tian, Y. Wang, J. Li, and T. Huang, “Robust multiple cameras pedestrian detection with multi-view bayesian network,” _Pattern Recognition_ , vol. 48, no. 5, pp. 1760–1772, 2015.
  * [22] P. Baqué, F. Fleuret, and P. Fua, “Deep occlusion reasoning for multi-camera multi-target detection,” in _Proceedings of the IEEE International Conference on Computer Vision (ICCV)_ , 2017, pp. 271–279.
  * [23] Y. Hou, L. Zheng, and S. Gould, “Multiview detection with feature perspective transformation,” in _Proceedings of the European Conference on Computer Vision (ECCV)_. Springer, 2020, pp. 1–18.
  * [24] K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)_ , 2016, pp. 770–778.
  * [25] H. Li, Y. Chen, Q. Zhang, and D. Zhao, “BiFNet: Bidirectional fusion network for road segmentation,” _IEEE Transactions on Cybernetics_ , pp. 1–13, 2021\.
  * [26] R. Girshick, J. Donahue, T. Darrell, and J. Malik, “Rich feature hierarchies for accurate object detection and semantic segmentation,” in _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)_ , 2014, pp. 580–587.
  * [27] S. Ren, K. He, R. Girshick, and J. Sun, “Faster R-CNN: Towards real-time object detection with region proposal networks,” _IEEE Transactions on Pattern Analysis and Machine Intelligence_ , vol. 39, no. 6, pp. 1137–1149, 2017\.
  * [28] Y. Xu, X. Liu, Y. Liu, and S.-C. Zhu, “Multi-view people tracking via hierarchical trajectory composition,” in _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)_ , 2016, pp. 4256–4265.
  * [29] T. Chavdarova and F. Fleuret, “Deep multi-camera people detection,” in _Proceedings of the IEEE International Conference on Machine Learning and Applications (ICMLA)_ , 2017, pp. 848–853.
  * [30] R. Kasturi, D. Goldgof, P. Soundararajan, V. Manohar, M. Boonstra, and V. Korzhova, “Performance evaluation protocol for face, person and vehicle detection & tracking in video analysis and content extraction (vace-ii),” _Computer Science & Engineering University of South Florida, Tampa_, 2006\.
  * [31] A. Geiger, P. Lenz, and R. Urtasun, “Are we ready for autonomous driving? the KITTI vision benchmark suite,” in _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)_. IEEE, 2012, pp. 3354–3361.


#  $\pi$-ROAD: a Learn-as-You-Go Framework for
On-Demand Emergency Slices in V2X Scenarios

Armin Okic1, Lanfranco Zanzi23, Vincenzo Sciancalepore2, Alessandro Redondi1,
and Xavier Costa-Pérez24 2NEC Laboratories Europe, Heidelberg, Germany,
<EMAIL_ADDRESS>
3 Technische Universität Kaiserslautern, Kaiserslautern, Germany
4 i2CAT Foundation and ICREA, Barcelona, Spain.
1Politecnico di Milano, Milano, Italy<EMAIL_ADDRESS>
###### Abstract

Vehicle-to-everything (V2X) is expected to become one of the main drivers of
5G business in the near future. Dedicated _network slices_ are envisioned to
satisfy the stringent requirements of advanced V2X services, such as
autonomous driving, aimed at drastically reducing road casualties. However, as
V2X services become more mission-critical, new solutions need to be devised to
guarantee their successful service delivery even in exceptional situations,
e.g. road accidents, congestion, etc. In this context, we propose $\pi$-ROAD,
a _deep learning_ framework to automatically learn regular mobile traffic
patterns along roads, detect non-recurring events and classify them by
severity level. $\pi$-ROAD enables operators to _proactively_ instantiate
dedicated _Emergency Network Slices (ENS)_ as needed while re-dimensioning the
existing slices according to their service criticality level. Our framework is
validated by means of real mobile network traces collected within
$400\leavevmode\nobreak\ km$ of a highway in Europe and augmented with
publicly available information on related road events. Our results show that
$\pi$-ROAD successfully detects and classifies non-recurring road events and
reduces up to $30\%$ the impact of ENS on already running services.

##  I Introduction

Despite the high investment volume in public transportation, many people daily
commute to work with their private vehicles over major roads around cities.
Accordingly, both drivers and passengers consume and generate a large amount
of data along the road mobile infrastructure for a wide variety of purposes:
navigation systems, car sensors, infotainment but also phone calls, social
media, streaming, etc. Along highly crowded roads, this may lead to network
congestion and, in the worst case, service disruptions.

In order to avoid these situations, the next generation of mobile networks
(5G) has defined the _Network Slicing_ concept, which allows infrastructure
providers to dynamically instantiate _on-demand customized virtual network
instances_ with dedicated Service Level Agreements (SLAs). Standardization
bodies have defined the overarching architecture [1] to support such isolated
slices, thus fostering research on dynamic resource orchestration mechanisms
based on well-known technologies such as Network Function Virtualization (NFV)
and Software Defined Networking (SDN) [2, 3].

As 5G gets rolled-out and advanced V2X services deployed, solutions to protect
mission-critical services will become increasingly important. While
significant road safety improvements have been introduced in the last decades,
road fatalities are still a major cause of injuries and death worldwide [4].
Thus, guaranteeing public safety along roads is still a major challenge that
requires novel solutions. Due to the combined effect of high-mobility
patterns, traffic volumes and advanced automotive-related use cases (V2X),
mobile network infrastructure along roads will face daunting challenges to
guarantee mission critical services in unexpected congestion scenarios. In
this context, _Emergency Network Slices (ENS)_ are expected not only to
improve situation awareness during emergencies but also to support the
provisioning of enhanced communication schemes, e.g., Ultra-Reliable and Low-
latency Communication (URLLC) for virtual and augmented reality (VR/AR) to
first-responder teams, e.g., ambulances, police, firefighters, that have to
reach the event location and manage emergencies in a faster and safer manner
[5]. As ENS will get top priority, solutions need to be devised to re-
dimension already running services according to their criticality level.
Unexpected road events progressively cause traffic congestion to nearby areas
and, in turn, saturate the networking resources of adjacent base stations. If
such a _propagation effect_ could be _predicted_ , their congestion effects
could be alleviated by means of _proactive slices resource management_.

We take on this challenge and propose _Passive Information-based Resource
Orchestration in Automotive Driving scenarios ( $\pi$-ROAD)_ that relies on a
deep learning framework to _analyze_ passive information from real-time mobile
network traffic statistics, _learn_ regular traffic patterns, and accurately
_detect_ anomalous deviations due to unexpected road events.

Our contributions can be summarized as follows: $i$) we provide an in-depth
analysis of _real mobile network traces_ and statistics from an operational
network along 400 Km of a major road in Europe, $ii$) we design a deep
learning model, namely $\pi$-ROAD, that applies the well-known concept of
learn-as-you-go, i.e., it accurately detects and localizes road events and
classifies them according to their severity, $iii$) we focus on the network
orchestration of emergency scenarios where an Emergency Network Slice (ENS)
must be instantiated according to the type of road event, $iv$) we formulate
an optimization problem to minimize the impact of ENSs on already existing
slices and $v$) we validate our model and assess its capabilities by means of
a full dataset of network monitoring data in realistic scenarios.

The remainder of the paper is structured as follows. Section II provides an
overview of road dynamics from a mobile network perspective. Section III
details the main building blocks of our solution. Leveraging on the output of
$\pi$-ROAD, Section IV formulates the ENS problem, whereas Section V presents
an exhaustive simulation campaign to validate our design principles through
real operational data. Section VI summarizes the related literature and VII
concludes the paper.

#### Privacy issues

The research activity presented in this paper does not violate user’s privacy
rights. The dataset contains only aggregated and anonymous information
collected by the network operator for monitoring purposes.

##  II Data Analysis

We carry out an analysis of the network traffic dynamics occurring on the
vehicular roads.

The considered dataset consists of $6$ months of real network data (February-
July $2020$) collected from an operational LTE network deployed alongside the
Italian A4 highway shown in Fig. 1. The highway has a length of approximately
$400$ km and is located in the north of Italy, connecting the city of Turin
with Venice, passing through the metropolitan area of Milan. Along this
highway segment, more than $1000$ LTE cells, corresponding to about $200$
eNodeBs (eNBs), are deployed to provide connectivity to the users commuting or
traveling on this path, and to citizens leaving in the proximity of the
highway. Available data exploit local monitoring information of LTE eNBs
including both averaged and aggregated Key Performance Indicators (KPIs)
within a time granularity of $15$ minutes.

Figure 1: Overview of the Italian highway (A4).

###  II-A Spatial consideration

Fig. 2 (top) depicts a snapshot of normalized DL and UL traffic volumes (per
single base station, where the approximate base station location is
highlighted in the upper part of the plot) aggregated over a month. We
highlight areas of the main cities located nearby the A4 roadway. As expected,
DL volumes increase in the proximity of major cities, due to urban traffic
leakage of base stations covering the highway. Additional traffic peaks can be
identified in specific geographical locations, e.g., intersections between
different highways, main inter-urban roads, or train lines. Generally, these
locations are characterized by a higher density of base stations, which means
that occurrence of any road events in these points will probably lead to major
impact on the mobile network. Moreover, a large number of mobile terminals
characterizes such locations over time, due to a significantly higher user
mobility. Similarly, UL traffic shows analogous behaviors but with a limited
volume (about $10\%$ of the overall DL traffic).

###  II-B Temporal patterns consideration

Several works in the literature suggest a strong relationship that correlates
end-users mobility patterns with cellular network statistics [6, 7, 8] in
urban environments. In a similar way, the mobility patterns identified on
highways present repetitive trends following regular working routines. In this
respect, we plot at the bottom of Fig. 2 normalized number of active users
during working days and weekends for different areas of the highway, in
particular two areas around metropolitan cities as well as two sections of the
highway between two main cities. The signatures are extracted by calculating
the median values of same time periods over several weeks, separately for
working days of the week and weekends, as proposed by [9].

The first and second subplots show that areas around big industrial cities,
such as Milan and Turin, are characterized by commuting patterns with a
presence of mobility peaks in the morning, noon and evening during the working
days, which are absent on weekends. Conversely, mobility patterns of users
during working and weekend days are comparable when considering highway
segments interconnecting major cities (third and fourth subplots).

Figure 2: Aggregated spatial distribution of DL/UL traffic volumes along the
highway (top), with the focus on working day and weekend signatures for
specific highway areas (bottom).

###  II-C Service-based QoS consideration

Different services with heterogeneous requirements might be seamlessly managed
by the mobile network infrastructure. In real deployments, this is usually
achieved by labelling each traffic flow with a specific QoS Class Identifier
(QCI) to ensure that each traffic bearer [10] is allocated with the
appropriate set of resources to guarantee an affordable Quality of Service
(QoS). In Fig. 3 (left-hand side), we depict the temporal distribution of the
DL volumes differentiated by QCI traffic types. It can be noticed a dominance
of non-guaranteed bit-rate (NON-GBR) traffic types (QCIs $6$ to $9$), mostly
related to video-streaming and social media activities, and almost negligible
volumes for guaranteed bit-rate (GBR) traffic types (QCIs $1$ to $4$). Note
that the satisfaction of service requirements also depends on the network
congestion level and on the instantaneous channel quality experienced by end-
users, together with the corresponding modulation and coding scheme (MCS)
selected at the eNodeB scheduler. Interestingly, DL traffic perceives, on
average, a lower MCS index with respect to the UL one (right-hand side of Fig.
3). This is due to the high end-user mobility and longer data exchange
sessions, which suffer from a wider communication distance along the path
[11].

Figure 3: Temporal distribution of different traffic types volumes.

###  II-D Road events consideration

The considered dataset takes into account monitoring information from the
mobile network infrastructure, but it lacks of information related to
vehicular traffic and unexpected congestions/events that affect its dynamic.
To fill this gap and help us to accurately relate network traffic dynamics
with road events, we rely on publicly available information coming from
heterogeneous social media sources like Twitter notification service of
Autostrade per l’Italia111Online available at: https://twitter.com/trafficoa,
Google Maps222Online available at: https://www.google.com/maps and
Waze333Online available at: https://www.waze.com/livemap, which facilitate an
exhaustive catalog of real-time road event information and useful metadata,
e.g., timestamp, exact location and a short description.

After $6$ months of data collection, the final dataset includes about $800$
road events,444Note that our experimental campaign has been carried out during
the Covid-19 pandemic, when the imposed lockdown limited the overall end-user
mobility in the north of Italy, therefore, reported numbers may be biased and
differ from yearly regular data. as shown in Fig. 4. As expected, road events
mainly occur during morning and evening commuting periods, and are
geographically placed close to main cities, e.g., Milan and Verona. We will
match the information contained in this supplementary dataset with the network
geographical deployment information to assign each road event with the closest
base station and obtain the ground truth of occurred events, as later detailed
in Section III.

###  II-E Event propagation effect

Road events might have a consistent impact on the vehicular traffic conditions
and, in turn, on overall network resource availability. However, it might be
easily confused with common network traffic outliers and therefore be ignored.
Current network deployments may come to help: the RAN deployment along the
highway has quite a regular pattern and might reveal implicit information as
monitoring data are simultaneously retrieved from different observation
points. In particular, a road event might affect the end-users activity or
drop the mobility rate gradually for adjacent base stations. We call it _event
propagation effect_. It usually depends on the severity of the road event, the
day time, the specific location and the network deployment (e.g., base station
density, etc.).

From the auxiliary information contained in our event dataset, we make a
straightforward association between the location of each event and the closest
base station, thereby identifying the source of the road event in the network.
Then, analyzing the propagation of anomalous traffic patterns onto nearby
cells, we can infer which direction of the highway, or vehicular traffic flow,
has been affected.

Figure 4: Spatio-Temporal distribution of road events. Figure 5: Event
propagation on nearby base stations in both directions for selected metrics.

An example of this is shown in Fig. 5. The scenario accounts for a major
accident occurred nearby Verona at $9$:$00$ AM. The red line in each plot
represents the regular behaviour of the corresponding metric, while the blue
line depicts the daily data. After the accident, we can notice a clear
deviation from regular patterns for multiple metrics lasting few hours. The
higher number of users in RRC connected state indicates that multiple users
are actively using mobile network resources, suggesting the presence of
traffic congestions. Similarly, this affects DL volumes and average Physical
Resource Block (PRB) utilization. Moreover, a higher number of affected base
stations on the east side suggests that the accident occurred in east-west
direction (Venice-Turin). Given the high variability and intrinsic
characteristic of each monitoring metric, an accurate anomalous pattern
characterization would involve time, space and metric-specific considerations.
Therefore, road events can hardly be detected by simple anomaly detection
algorithms, e.g., those based on outlier detection that may run on a single
base station. This motivates us to investigate advanced deep learning
solutions capable of dealing with such multi-dimensional information matrix
while keeping a global view of the system.

Figure 6: $\pi$-ROAD model architecture

##  III  $\pi$-ROAD Design and Model Formulation

Hereafter, we introduce the design of $\pi$-ROAD, a deep learning-based
framework to detect road events. An overview of the model architecture is
depicted in Fig. 6. The overall solution requires two different stages to deal
with road event predictions: $i$) an autoencoder-based approach consisting of
Long Short-Term Memory Recurrent Neural Networks (LSTM) layers to detect
anomalies within highly dynamic and heterogeneous time patterns and $ii$) a
Deep-Learning approach to approximate complex functions, e.g., those that
relate road events occurrences with temporal and spatial distributions of
anomalies detected in adjacent base stations. The overall $6$ months dataset
has been splitted according to a $60/20/20$ ratio for the purposes of
training, validation and testing procedures, respectively.

###  III-A Input of $\pi$-ROAD

Let us consider a time slotted system whereby time is divided into time
intervals $t=\\{1,\dots,\mathcal{T}\\}$, and define $\mathcal{N}$ as the set
of base stations deployed along the highway. As detailed in Section II, each
base station $n\in\mathcal{N}$ gathers information about a multitude of
heterogeneous network statistics, defined as $\mathcal{M}$, which include,
among the others, PRB utilization, physical channel quality information,
traffic volumes, X2/S1 hand-over statistics, etc. In our experiments we
consider $25$ different monitoring metrics.

Let $x^{n}_{m}(t)$ be the time series describing the $m$-th monitoring metric
observed under the base station $n$ at the time $t$. As suggested in [12], an
accurate organization of the input traces helps enhancing deep learning models
performances. In light of this, for all metrics $m\in\mathcal{M}$ and base
stations $n\in\mathcal{N}$, we collect the $x^{n}_{m}(t)$ traces and order
them by preserving their spatial location (from Turin to Venice) in a matrix
$X_{m}(t)$ with dimensions $N\times L$, representing a metric-specific
_snapshot_ of the network, where $N$ is the total number of base stations, and
$L$ is the number of monitoring samples within the observation period (or
lookback time window), fixed to $4$ hours throughout our experiments. The
input matrices $X_{m}(t)$ are normalized with respect to maximum values of
each $m$ in the training dataset.

###  III-B LSTM-based anomaly detection

The possibility to detect changes in traffic conditions is subject to the
capabilities of our model to correctly identify irregular statistics from the
monitoring samples collected along the highway. Given the multi-dimensionality
of our dataset, we build an autoencoder $A_{m}$ for every collected metric
thereby allowing for better scalability. Autoencoders imply the setup of an
encoding-decoding architecture. The encoding part, commonly implemented as
feed-forward neural networks, provides a compressed representation of the
input data to subsequent layers [13]. The decoding phase follows the same
steps over a symmetric architecture. Analytically, the two phases applied to
the input metric $X_{m}$ can be described as follows:

$\displaystyle h_{m}(X_{m}(t))=\eta(W^{\eta}_{m}X_{m}(t)+b_{m}^{\eta}),$ (1)
$\displaystyle\hat{X}_{m}(t)=\delta(W^{\delta}_{m}h_{m}(X_{m}(t))+b_{m}^{\delta}),$
(2)

where $\eta$ and $\delta$ are the encoding and decoding functions with their
corresponding weights $W^{\eta}_{m}$ and $W^{\delta}_{m}$, $b_{m}^{\eta}$ and
$b_{m}^{\delta}$ are bias vectors, and $h_{m}(X_{m}(t))$ and $\hat{X}_{m}(t)$
are the compressed input and the reconstructed output sequences, respectively.

The performance of autoencoders, defined as their capability to reconstruct
the input sequence, depends on variability and complexity of data provided at
the input. Differently, in our work we exploit autoencoders for anomaly
detection [14] instead of simple dimensionality reduction.

The data analysis performed in Section II suggests the adoption of deep
learning techniques able to deal with the spatio-temporal characteristics of
mobile traces. For this reason, we implement our encoder-decoder architecture
by means of Long Short-Term Memory Recurrent Neural Networks (LSTM-RNNs). Each
autoencoder $A_{m}$ accounts for $4$ LSTM layers, two for each phase. LSTM is
a type of RNN architecture that has proven its value when dealing with
repetitive patterns and unstructured time series, while solving a problem of
vanishing gradient for long-term dependencies present in other RNNs [15].
LSTMs can be represented as a chain of $G$ modules, or cells, each one
applying a set of operations to the input data. In our case, the two LSTM
layers are composed by $128$ and $64$ cells, respectively [16].

The output of each cell $g\in G$, also known as cell state, is transferred to
the subsequent cell in a recursive manner. The possibility of handling long-
term trends in LSTM is provided by structures, dubbed as gates, which
carefully remove or add information to the cell state. Each cell has three
gates, namely, input $I_{g}$, output $O_{g}$ and forget gate $F_{g}$, which
controls the amount of information that should be added (or dropped) to the
cell state before transferring it to the next unit [17]. This effect is
achieved combining the influence of different non-linear activation functions,
i.e., $\sigma$ (sigmoid) and $\tanh$ (hyperbolic tangent function), at each
gate. The impact of these functions on the input data needs to be learned
during the training phase aiming to minimize a loss function [17], which in
our case is the mean squared error (MSE).

To accomplish the anomaly detection task, we make use of labeled data and
train the autoencoders offline exclusively on _eventless_ snapshots taken from
historical data as to capture the behavior of the system without anomalies.
Once trained, we feed the model with online monitoring traces. Given that
anomalous patterns have not been part of the training phase, it is expected
that the model will exhibit performance degradation during the reconstruction
phase. Simple classifiers would mark the input snapshot as anomalous if the
reconstruction error exceeds a given threshold. Instead, we derive the squared
error matrix $e_{m}(t)=\|{X_{m}(t)-\hat{X}_{m}(t)}\|^{2}$ from each
autoencoder $A_{m}$ and combine the individual error matrices into a 3D tensor
$e(t)=\\{e_{1}(t),e_{m}(t),\dots,e_{M}(t)\\}$ which is passed to the second
stage of our model.

###  III-C Road Event Localization

The function linking anomalies detected by the first stage of our model with
the occurrence of road events along the highway is unknown and hard to be
characterized due to the multitude of system-related and external variables
affecting the entire detection process. The simple detection of anomalies in
monitoring statistics does not imply the occurrence of an emergency, as
uncorrelated events—like hardware failures and/or unexpected traffic peaks—may
trigger alerts leading to erroneous estimations. For this reason in the
following, we exploit the information described in Section II-D as ground-
truth to train a Convolutional Neural Network (CNN)-based model that captures
the spatio-temporal correlation of different anomalies and maps them into
geographical information [18]. Our design choice is further motivated by the
fact that in case of road events, as shown in Fig. 5, affected base stations
present significant levels of correlation between each other, which further
improves the learning task.

More in details, the second stage of $\pi$-ROAD consists of two stacked 3D-CNN
layers and a final Multi-Layer Perceptron (MLP) fully connected layer. In
order to exploit local correlation from nearby base stations, each neuron of
the 3D-CNN layers has a limited receptive field, or kernel, whose size
determines its observation area. For a given input tensor, convolutions with
3D kernels are iteratively applied to provide the subsequent layers with a
compressed representation of the input information.

Through extensive hyper-parameters optimization, we used two 3D-CNN layers
with filter sizes of $32$ and $16$ neurons, respectively, and corresponding
convolutional kernel sizes of $(3,3,3)$ and $(3,3,3)$ [19]. Each neuron of the
CNN runs a filter $H(\sum_{t}e(t)*k(t)+b)$, where $e(t)$ is the input tensor
at time $t$, $k(t)$ is the kernel filter, $*$ is the convolution operator, $b$
is a bias, and $H(\cdot)$ is a non-linear activation function, in our case
Rectified Linear Unit (ReLU) [20].

In order to map the encoded representation of the anomalies into geographical
information about the emergency, we make use of a final Multi-Layer Perceptron
layer. MLP is a class of neural networks with fully-connected neurons among
layers which has the capabilities to approximate, through supervised learning,
the function that relates the input with the output. In our case, we are
interested in the function that links the encoded spatio-temporal anomalies of
multiple metrics and different base stations with the exact event location.
The MLP classifier consists of three layers with $512$, $256$ and $220$
neurons, respectively, where the output layer matches the number of base
stations [21]. To regularize the output and reduce over-fitting, we introduce
a dropout rate of $0.2$. Overall, the second stage of the model is trained
using Adam optimizer with learning rate $10^{-4}$, and adopting cross-entropy
as loss function over $150$ training epochs. Due to high system variability
caused by user social behaviors and external causes affecting it (e.g.,
weather conditions), it clearly rises the need to design a model capable of
adapting to new (anomalous) patterns. Therefore, we ensure that the model is
retrained as soon as new observations are made available.

###  III-D Road Event Classification

Upon detection, it is important to classify the magnitude of each event to
promptly alert first aid responders and, if necessary, identify the set of
networking requirements to be allocated for an emergency slice setup. We rely
on the impact of the _propagation effect_ over the set of base stations near
to the road event to provide an empirical classification. Let us introduce
$\tilde{\mathcal{N}}\subset\mathcal{N}$ as the set of base stations affected
by the road event. Due to the irregular density that characterizes the Radio
Access Network (RAN) deployment along the highway, we argue that a simple road
event classification based on the cardinality of $\tilde{\mathcal{N}}$ would
not be accurate, as road events occurring in the proximity of main cities
would involve a wider number of base stations than those occurring in rural
areas. Therefore, we define our classification metric $\mu$ as
$\frac{\tilde{N}}{\psi}$, where $\tilde{N}$ is the cardinality of
$\tilde{\mathcal{N}}$, and $\psi$ is the number of base stations deployed in
the area within a radius of $10$ km surrounding the road event. We empirically
select $10$ km as this value represents the longest vehicular queue registered
in our event dataset. Clearly, $\psi$ should be re-dimensioned to generalize
our findings within other network topologies.

Finally, we differentiate among three different categories of events: Light,
Moderate, and Severe. The event duration of each category reflects the most
common scenarios contained in our real dataset, while networking throughput
requirements are meant to support the provisioning of Augmented and Virtual
Reality (AR/VR) streaming in mission critical scenarios [22]. The details
about each road event class and corresponding ENS requirements are summarized
in Table I.

Table I: Event characterization and ENS Requirements.

Event category |  $\Theta$ \- Time Duration | NS Requirements |  $\mu$ Value
---|---|---|---
Light |  $20$min |  $10$Mbps | $(0,1]$
Moderate |  $40$min |  $15$Mbps | $(1,2]$
Severe |  $60$min |  $25$Mbps | $\geq 2$

###  III-E Performance Comparison and Practical Considerations

Hereafter, we compare the performance of $\pi$-ROAD against state-of-the-art
models trained to perform similar anomaly detection and classification tasks,
highlighting each model drawback (compared to our solution) through practical
considerations. Benchmarks include a simple threshold-based autoencoder
classifier (AE) [16], and a more advanced 3D-CNN-based classifier [19, 21].

Figure 7: ROC and PR diagrams with indicated AuC and AuPRC scores for
different anomaly detection and classification approaches.

Fig. 7 resumes the overall performances using Receiver Operating
Characteristics (ROC) and Precision-Recall (PR) metrics. On the left-hand
side, the ROC metric provides a compact representation of the capabilities of
the model to deal with the trade-off between True Positive Rate (TPR) and
False Positive Rate (FPR). On the right-hand side, the PR curve states the
performance of a classifier in terms of Precision and Recall, where Precision
is a measure of result relevancy, and Recall is a measure of how many truly
relevant results are returned by the classifier. To ease the comparison, we
also quantify the Area under the ROC Curve (AuC) and Area under the PR Curve
(AuPRC). The unsupervised approach adopted in AE requires the definition of
a-priori thresholds to mark anomalous patterns from the reconstruction errors.
Clearly, the definition of these settings depends on the intrinsic variability
of the traces in input, and requires considerations over the statistics of
each feature. This approach hardly scales when considering multiple
heterogeneous metrics, as in our case. Despite accurate tuning of the
parameters, it clearly appears how this simple approach fails to reveal the
majority of events, achieving the lowest precision score. This further
emphasizes how detection schemes based on simple threshold/outlier detection
are not enough to address the scenario considered in our work. Conversely, the
3D-CNN classifier adopts a supervised approach, which prevents from specifying
user-defined thresholds for decision-making. The resulting ROC and PR curves
show better performances when compared against the baseline AE approach. This
result can be explained through the ability of 3D-CNN network to capture
spatio-temporal correlations between different network measurements [23].
However, when Recall value increases over a certain level (i.e., moving from
left to right on the PR diagram), the Precision score drops drastically,
suggesting poor performances when differentiating among different types of
anomalies. In other words, the model detects only some types of anomalies,
e.g., those deriving from major events with very strong impact on monitoring
traces, and fails to generalize over probably smaller ones. Conversely,
$\pi$-ROAD outperforms the two stand-alone approaches in both ROC and PR
metrics. The advantage deriving from the two-stage approach adopted by
$\pi$-ROAD is two-fold $i$) the initial anomaly detection task performed by
first stage of the model, together with an accurate input organization,
facilitates the learning task of the 3D-CNN layer and $ii$) the capabilities
of the 3D-CNNs to deal with bare reconstruction errors removes the need to
provision user-defined thresholds which may bias the final results.

##  IV ENS orchestration

The outcome of $\pi$-ROAD can be used to tackle a variety of public safety
issues along the highway. For example, in case of road accidents, it is
important to enable dynamic RAN resource allocation in order to provide first
responder teams with enough communication capabilities, regardless of active
user sessions or services. To this aim, in what follows we introduce our
formulation to address the Emergency Network Slice (ENS) orchestration
problem.

Radio Access Network. Let us consider a RAN deployment with network slicing
support covering the area surrounding the highway and comprising a set of base
stations $\mathcal{N}$. Each base station $n\in\mathcal{N}$ is characterized
by a capacity $C_{n}$, defined as the sum of its available physical resource
blocks (PRBs). As result of the road event localization process executed by
$\pi$-ROAD and described in Section III-C, we identify the subset of base
stations $\widetilde{\mathcal{N}}\subset\mathcal{N}$ that will host the
emergency slice.

Active slice services. Let us assume a set of network slices $\mathcal{S}$
supporting V2X services, being already installed and active on the considered
RAN deployment, whereas let us mark $s=0$ the Emergency Network Slice (ENS) to
be temporarily installed. We assume each V2X slice $s\in\mathcal{S}$ described
by means of a predefined network slice template that suggests slice
requirements in terms of throughput $\Delta_{s}$, and latency $\Lambda_{s}$.
Typical values for V2X services are shown in Table II.

Table II: V2X Slice requirements (c.f. [24, 25, 26])

V2X category | Latency | Data rate | Reliability
---|---|---|---
Autonomous driving |  $10$ms |  $10$Mbps | 99.999%
Tele-operated driving |  $20$ms |  $25$Mbps Sensor Data Streaming | 99.999%
Vehicular Internet/Infotainment |  $>100$ms |  $15$Mbps Video Streaming | Not Specified
Road Safety |  $100$ms |  $1$Mbps | 99%

###  IV-A Problem design

In real scenarios, when setting up an emergency slice, advanced orchestration
operations are required to still guarantee service level agreements (SLAs) of
active slices. Assuming that an admission control process has been executed to
accept and install V2X slices on the network with the aim of maximizing the
resource efficiency (while still honouring expected service requirements), it
might appear challenging to add on top of active slices an additional high-
priority service, such as ENS. However, an efficient admission control will
accommodate slices with heterogeneous requirements to compensate unexpected
slice behaviors or traffic peaks [27]. Analytically, we assume that predefined
SLAs include for each slice $s\in\mathcal{S}$ and base station
$n\in\mathcal{N}$ a minimum number of reserved PRBs, namely $Q^{(t)}_{n,s}$
555When SLAs disclose information on expected throughput or user rate, the
amount of required PRBs could be dynamically obtained by monitoring the
traffic demand of the different slices. For further details, we refer the
reader to [28, 29].. Without loss of generality, we assume that the overall
resource availability is assigned to the set of running slices. In addition,
we translate the slice latency requirements $\Lambda_{s}$ into a tolerance
value $\lambda_{s}$ that defines the maximum number of time intervals packets
shall wait into the queue before being served, or dropped due to lack of
resource availability within the latency requirements. Note that if all
running V2X slices have high priority, i.e., very low delay tolerance, it
might be infeasible to install the ENS without impairing other active
services. ENS parameters depend on the severity of the emergency road event
which can be obtained as output of a $\pi$-ROAD execution. We resume the
corresponding settings in Table I, in light of the discussions of Section
III-D. We define $\Theta$ as the envisioned emergency time duration, and
assume, for each ENS to be deployed, a fixed amount of PRBs allocation
$Q_{n,0}^{(t)},\forall t\in\\{0,\cdots,\Theta\\}$ and the lowest delay
tolerance $\lambda_{0}=1$. If not properly scheduled, the additional ENS may
lead in the worst case to resource deficit and service disruption in one or
multiple base stations. Therefore, we aim at minimizing the instantaneous
resource deficit $\pi_{n}^{(t)}$, while still guaranteeing defined SLAs for
other active running slices. Our problem can be formulated as follows:

######  Problem 1 ($\pi$-Orchestrator).

$\displaystyle\mathrm{minimize}$
$\displaystyle\displaystyle\sum_{n\in\tilde{\mathcal{N}}}\sum_{t\in\mathcal{T}}\pi_{n}^{(t)}$
$\displaystyle\mathrm{subject\leavevmode\nobreak\ to}$
$\displaystyle\sum_{s}z_{n,s}^{(t)}\leq
C_{n}+\pi_{n}^{(t)},\quad\quad\quad\forall
n\in\tilde{\mathcal{N}},t\in\mathcal{T};$
$\displaystyle\sum_{t=0}^{\Theta}Q_{n,s}^{(t)}-z_{n,s}^{(t)}\leq\sum_{t=\Theta+1}^{\Theta+\lambda_{s}}Q_{n,s}^{(t)},\,\forall
s\in\mathcal{S},n\in\tilde{\mathcal{N}};$ $\displaystyle
z_{n,0}^{(t)}=Q_{n,0}^{(t)};\quad\forall
n\in\tilde{\mathcal{N}},t\in\mathcal{T};$ $\displaystyle
z_{n,s}^{(t)}\in\mathbb{N}^{+},\qquad\forall
n\in\tilde{\mathcal{N}},s\in\mathcal{S},t\in\mathcal{T};$
$\displaystyle\pi_{n}^{(t)}\in\mathbb{N}^{+},\qquad\forall
n\in\tilde{\mathcal{N}},t\in\mathcal{T}.$

where (integer) decision variables are $\pi_{n}^{(t)}$ and $z_{n,s}^{(t)}$
indicating the number of PRBs to be assigned to slice $s$ on base station $n$
at time interval $t$. Note that $z_{n,0}^{(t)}=Q_{n,0}^{(t)}$ is due to the
highest priority assigned to the ENS ($s=0$). The first set of constraints
introduces flexibility into the problem by adding a non-zero fictitious value
(i.e., the resource deficit $\pi_{n}$) to avoid infeasible solutions. The
second set of constraints specify that the slice resource reservation must be
performed fulfilling the slice delay tolerance, i.e., waiting traffic still in
the queue must be scheduled within $\lambda_{s}$ time slots. We run Problem 1
for a set of time slots $\mathcal{T}$ that includes the time window $\Theta$
required by the ENS. Problem 1 is an Integer Linear Programming (ILP), which
can be efficiently approximated by means of relaxation techniques (e.g.,
[30]), and commercial solvers to provide near-optimal results 666The
implementation of the problem has been carried out using the framework of IBM
ILOG CPLEX and its Python API..

######  Proposition 1.

Problem 1 is NP-Hard and difficult to approximate within $N^{1-\epsilon}$, for
$\epsilon>0.$

###### Proof:

The NP-Hardness proof goes by reduction. Problem 1 clearly belongs to the NP
problems. Let us consider an instance of a bin packing problem (BPP) with
arbitrary constants $a$ and $B$, and binary decision variables $x_{ij}$ and
$y_{j}$, where $\mathcal{I}$ and $\mathcal{J}$ are the sets of items and bins,
respectively [31].

######  Problem 2 (BPP).

$\displaystyle\mathrm{minimize}$ $\displaystyle\displaystyle\sum_{j}y_{j}$
$\displaystyle\mathrm{subject\leavevmode\nobreak\ to}$
$\displaystyle\sum_{i}ax_{ij}\leq By_{j},\quad\quad\quad\forall
j\in\mathcal{J};$ $\displaystyle\sum_{j}x_{ij}\leq 1,\quad\quad\quad\forall
i\in\mathcal{I}.$

If we consider $N=1$ base station, and $z_{n,s},\pi_{n}$ as binary decision
variables, Problem 1 reduces to a BPP that shows that for all $\epsilon>0$,
packing items within the minimum number of bins within $N^{1-\epsilon}$ is NP-
hard [31]. Since such a problem is trivial compared to our original Problem
1—that includes a number of base stations $N>1$ and integer decisions
variables—this result is rather strong. ∎

###  IV-B V2X traffic scheduling

The instantaneous resource deficit
$\sum_{n\in\tilde{\mathcal{N}}}\pi_{n}^{(t)}$ depends on the severity of the
detected road event and, in turn, on the specific ENS resource reservation and
time duration settings. Therefore, Problem 1 considers the worst case
scenario, i.e., when each active slice fully utilizes reserved resources.
However, as shown in Section II, network congestions rarely occur outside the
commuting time periods and some of reserved network resources may be
underutilized. So, the impact of the ENS on the system can be further
mitigated relaxing the fixed PRB allocation scheme envisioned to assure slice
isolation at RAN scheduling level [32, 33]. Specifically, if a network slice
is underutilized, other slices with pending traffic can use some of its
network resources (PRBs) 777While this concept may fail to comply with the
network slice isolation principle, we assume that the isolation is still
guaranteed at higher (abstracted) scheduling layers [32].. This allows us to
devise a practical algorithm that reduces the overall slice resource deficit.
In particular, the algorithm sorts slice traffic requests based on the slice
priority (or traffic delay tolerance). Slices with higher priority will use
assigned PRBs as per the solution of Problem 1 ($z_{n,s}^{(t)}$) within time
interval $t$ to serve incoming traffic requests. Once all requests have been
served, remaining slice PRBs ($z_{n,s}^{(t)}>0$) are used to serve traffic
requests of the next slice in the priority-ranked list. When all PRBs are used
($\sum_{s}z_{n,s}^{(t)}=0$), the algorithm proceeds to the next time interval
$t$ keeping unserved traffic requests in the slice queues. If slice traffic
requests are not served within the slice delay tolerance $\lambda_{s}$, these
are dropped. The resulting resource deficit is further minimized as the RAN
slice scheduler efficiently assign resources to slices with pending traffic.
Results are shown and discussed in Section V-B.

##  V  $\pi$-ROAD Performance Evaluation

In the following, we investigate the performance of $\pi$-ROAD while detecting
road events, and use this information to assess our solution’s capabilities in
orchestrating radio resources in case of road accidents and realistic traffic
conditions involving different types of V2X network slices.

###  V-A Localization of the road event

In order to evaluate event localization performances, we provide as input the
test set of monitoring samples taken from our dataset. In Fig. 8, we summarize
the results, focusing on the set of base stations and a representative time
period. Each point on the map represents a pair of base station and time
interval, and its color indicates the detected probability of event occurrence
given the monitoring information. Circular markers on the same map highlight
the ground-truth location of road accidents. From the picture it can be
noticed how $\pi$-ROAD detects most of the events, not only revealing their
temporal duration (x-axis), but also suggesting how much the propagation
effect influences adjacent base stations (y-axis). Interestingly, the heatmap
also reveals some cases in which $\pi$-ROAD detects the occurrence of events
before online notification services and social media platforms. For
visualization purposes, at the top of the figure we highlight two geographical
areas of the highway covering impacted base stations in two different event
occurrences happening at time $t_{77}$ and $t_{84}$, respectively. In the
first case, $\pi$-ROAD classifies the road event as Light since only two base
stations reported anomalous statistics. In the second case, given the wider
number of affected eNodeBs, the event is classified as Moderate. Finally, few
events are not detected. Either this can be due to very limited impact of such
events on the monitoring traces or, they may be approximated with patterns
that are unknown to the model. In the latter case, we expect that similar
events would be detected in the future as the model follows the learn-as-you-
go approach to continuously train with the latest data.

###  V-B Resource Orchestration

Hereafter, we test our ENS orchestration solution exploiting $\pi$-ROAD’s
outputs and the realistic monitoring information contained in our dataset. We
consider a $4$-slices network scenario, including two enhanced Mobile
Broadband (eMBBs) slices dedicated to streaming and infotainment services
(with high delay tolerance), and two Ultra-reliable low-latency communication
(URLLC) slices dedicated to Autonomous and Tele-operated driving, with
corresponding networking requirements as detailed in Table II. We assume that
DL slice traffic volumes are generated under the reference set of base
stations in the considered event period, according to the specific service QCI
and corresponding traces as depicted in Fig. 3. In particular, eMBB slices are
mapped to QCIs traces $7$ and $8$, while URLLCs to QCIs $1$ and $6$,
respectively.

Figure 8: Example of road event detection and localization using $\pi$-ROAD
model along the highway within one day, compared against the event ground-
truth information.

(a) Resource orchestration at system level.

(b) Resource orchestration at base station level.

Figure 9: Performance evaluation of a $4$-slices scenario when an ENS is
installed.

Intuitively, the impact of an ENS installation over the radio access network
depends, among the others, by the networking capacity deployed in the area of
the road event, instantaneous traffic demands, slice requirements, and user
density. In Fig. 9(a) (top) we provide a quantitative measure in case of
different road event types and corresponding duration. We consider channel
conditions as perceived by the base station during the event, thereby limiting
the base station capacity (due to lower assigned MCS) when compared to the
ideal scenario. The results of each category are averaged over $10$ road event
occurrences taken from our real dataset. It can be noticed that eMBB slices
unveil worst performance (w.r.t. URLLC) due to their higher throughput
requirements and lower slice priority (i.e., higher delay tolerance). However,
an advanced slice orchestration solution may alleviate this issue.

Fig. 9(a) (bottom) focuses on the Moderate road events evaluating the impact
of different slice types (delay tolerance parameters $\lambda_{s}$) with a
fixed decision interval $t$ duration of $100$ ms [34]. Specifically,
$\pi$-Orchestrator may suggest different solutions with corresponding slice
configurations that translate into variable slice scheduling opportunities per
decision interval. Therefore, we evaluate the Cumulative Distribution
Functions (CDFs) of traffic latency considering different scheduling
opportunities. Despite the resource deficit introduced by the ENS, the average
slice latency improves when the number of scheduling opportunities increases.
Fig. 9(b) depicts the temporal evolution of the overall effects caused by the
ENS slice setup at transmission time interval (TTI) level. All traffic traces,
MCS and channel quality values are generated with a millisecond granularity
according to their temporal evolution and statistical distributions resulting
from our monitoring dataset. The initial scenario accounts for a fixed
resource allocation scheme (without any ENS running) which allows to satisfy
all the different slice requirements, as expected in normal traffic conditions
thanks to the adoption of calibrated admission and control algorithms, e.g.
[28]. In particular, among the $4$ slices populating the system, eMBB slices
have a fixed quota of $30$% of available resources while URLLCs have $20$%,
respectively. After $10$ seconds, an ENS is instantiated. Its higher priority
implies the execution of the $\pi$-Orchestrator to minimize service
disruption. From the upper plot, it can be noticed how our solution reduces
the quota of resources assigned to non-delay sensitive services which,
unavoidably, suffer from a resource deficit (highlighted in red). However, in
emergency scenarios the RAN slice scheduler may assign unused reserved
resources to active slices with traffic pending to be served, further reducing
the resource deficit and, in turn, the SLA violation of certain slices. In the
lower plot, we depict the overall traffic potentially violating the SLAs with
a red dashed line, whereas the traffic actually exceeding it with a black
continuous line. Results show how scheduling relaxation would help in reducing
the SLA violations up to $30\%$. Clearly, this result assumes that none of the
resources allocated for ENSs can be consumed by other slices, even in absence
of emergency traffic.

##  VI Related Work

The ever-increasing vehicular traffic fosters the need to deeply understand
the complex relationship that regulates mobility patterns which, in turn,
affects mobile network operational conditions [35, 36]. From the one side,
this effort requires constant monitoring of the communication infrastructure.
From the other side, the highly heterogeneous set of monitoring KPIs demands
for advanced solutions to automate the early detection of anomalous
conditions. The authors of [37] initially addressed this scenario proposing a
Bayesian network working on a set of discrete metrics collected from an
operational UMTS infrastructure. Similarly, the work of [38] focused on a
large-scale cellular network scenario, where traffic traces are modeled into
regular and random components. Their decomposition approach well suits
predictable patterns, but fails in highly variable scenarios. More recently,
state-of-the-art solutions start combining monitoring traces and heterogeneous
contextual information to improve the effectiveness of model predictions and
decisions. In [27, 29] the authors leverage spatio-temporal characteristics of
mobile traces to predict resource utilization in the context of network
slicing.

Most of the works in the literature address the urban environment and aims to
mitigate the cause of traffic congestions by predicting traffic flows and
offloading strategies to alternative paths [39], while less studies targeted
the highway scenario. In [40] the authors present a spatio-temporal analysis
of highways travel patterns exploiting per-vehicle data records collected by a
centralized authority in China. Given the fine granularity of per-vehicle
information, there is lack of a discussion about anomaly detection from a
mobile network perspective. Additionally, note that the majority of the
related works presented above tackle the vehicular traffic prediction by means
of active systems, e.g., GPS location exchange, or networks of sensors
deployed along the road, which clearly ease the prediction task but also
increases the volume of information generated along the process. To the best
of our knowledge, this is the first work considering the prediction of road
events and their impact on the mobile network infrastructure exclusively
accounting for _aggregated_ and _passive_ RAN monitoring samples.

##  VII Conclusions

In this paper, we presented $\pi$-ROAD, a deep learning-based solution for the
analysis and detection of road events over the highway. The model accounts for
two complementary stages $i$) an autoencoder-based stage to identify anomalous
patterns in the temporal evolution of operational mobile network data, and
$ii$) a 3D-CNN-based stage to overcome simple threshold detection schemes and
automatically learn the relationship that links multiple metric-specific
anomalies to road event occurrences. The output of $\pi$-ROAD may be used in a
multitude of emergency scenarios. We focused on the design and validation of a
slice orchestration solution dealing with the setup of Emergency Network Slice
(ENS) in V2X scenarios. Considering real mobile traffic distributions from a
major highway in Italy, our results show that the information provided by
$\pi$-ROAD can significantly reduce, up to $30\%$, the impact of Emergency
Network Slice setup, also decreasing the probability of service disruption on
other running network slices.

## References

  * [1] Third Generation Partnership Project (3GPP), “Technical Specification Group Services and System Aspects; System architecture for the 5G System (Rel. 16),” 3GPP TS 23.501 V16.5.1, Aug. 2020.
  * [2] P. Rost _et al._ , “Network Slicing to Enable Scalability and Flexibility in 5G Mobile Networks,” _IEEE Communications Magazine_ , vol. 55, no. 5, pp. 72–79, May 2017.
  * [3] J. Ordonez-Lucena, P. Ameigeiras, D. Lopez, J. J. Ramos-Munoz, J. Lorca, and J. Folgueira, “Network Slicing for 5G with SDN/NFV: Concepts, Architectures, and Challenges,” _IEEE Communications Magazine_ , vol. 55, no. 5, pp. 80–87, May 2017.
  * [4] W. H. Organization, Jul. 2020. [Online]. Available: {https://www.who.int/gho/road_safety/mortality/en/}
  * [5] K. Donoughe, J. Whitestone, and H. Gabler, “Analysis of Firetruck Crashes and Associated Firefighter Injuries in the United States,” vol. 56, no. 1. Association for the Advancement of Automotive Medicine, Oct. 2012, pp. 69–76, Available: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3503424/.
  * [6] C. Marquez, M. Gramaglia, M. Fiore, A. Banchs, and X. Costa-Pérez, “How Should I Slice My Network?: A Multi-Service Empirical Evaluation of Resource Sharing Efficiency,” in _ACM International Conference on Mobile Computing and Networking_ , Oct. 2018, pp. 191–206.
  * [7] G. Khodabandelou _et al._ , “Estimation of Static and Dynamic Urban Populations with Mobile Network Metadata,” _IEEE Transactions on Mobile Computing_ , vol. 18, no. 9, pp. 2034–2047, Sept. 2019.
  * [8] A. Okic, A. E. Redondi, I. Galimberti, F. Foglia, and L. Venturini, “Analyzing different mobile applications in time and space: a city-wide scenario,” in _IEEE Wireless Communications and Networking Conference_ , 2019, pp. 1–6.
  * [9] A. Furno, M. Fiore, R. Stanica, C. Ziemlicki, and Z. Smoreda, “A Tale of Ten Cities: Characterizing Signatures of Mobile Traffic in Urban Areas,” _IEEE Transactions on Mobile Computing_ , vol. 16, no. 10, pp. 2682–2696, Oct. 2017.
  * [10] Third Generation Partnership Project (3GPP), “Technical Specification Group Services and System Aspects; Policy and charging control architecture (Rel. 16),” 3GPP TS 22.203 V16.2.0, Dec. 2019.
  * [11] C. E. Andrade, S. D. Byers, V. Gopalakrishnan, E. Halepovic, D. J. Poole, L. K. Tran, and C. T. Volinsky, “Connected Cars in Cellular Network: A Measurement Study,” in _ACM Internet Measurement Conference - IMC’17_ , Nov. 2017, p. 235–241.
  * [12] X. Wang, Z. Zhou, F. Xiao, K. Xing, Z. Yang, Y. Liu, and C. Peng, “Spatio-Temporal Analysis and Prediction of Cellular Traffic in Metropolis,” _IEEE Transactions on Mobile Computing_ , vol. 18, no. 9, pp. 2190–2202, Sept. 2019.
  * [13] B. Liao, J. Zhang, C. Wu, D. McIlwraith, T. Chen, S. Yang, Y. Guo, and F. Wu, “Deep Sequence Learning with Auxiliary Information for Traffic Prediction,” in _ACM International Conference on Knowledge Discovery and Data Mining_ , Jun. 2018, pp. 537––546.
  * [14] S. Tariq, S. Lee, Y. Shin, M. S. Lee, O. Jung, D. Chung, and S. S. Woo, “Detecting Anomalies in Space Using Multivariate Convolutional LSTM with Mixtures of Probabilistic PCA,” in _ACM International Conference on Knowledge Discovery and Data Mining_ , Jul. 2019, pp. 2123––2133.
  * [15] S. Hochreiter and J. Schmidhuber, “Long Short-Term Memory,” _Neural Computation_ , vol. 9, no. 8, pp. 1735––1780, Nov. 1997.
  * [16] H. D. Trinh, E. Zeydan, L. Giupponi, and P. Dini, “Detecting Mobile Traffic Anomalies Through Physical Control Channel Fingerprinting: A Deep Semi-Supervised Approach,” _IEEE Access_ , vol. 7, pp. 152 187–152 201, Oct. 2019.
  * [17] Y. Zhu, H. Li, Y. Liao, B. Wang, Z. Guan, H. Liu, and D. Cai, “What to Do Next: Modeling User Behaviors by Time-LSTM,” in _International Joint Conference on Artificial Intelligence_ , vol. 17, 2017, pp. 3602–3608.
  * [18] S. Ji, W. Xu, M. Yang, and K. Yu, “3D Convolutional Neural Networks for Human Action Recognition,” _IEEE transactions on pattern analysis and machine intelligence_ , vol. 35, no. 1, pp. 221–231, Jan. 2013.
  * [19] Y. Yang, Z. Hu, K. Bian, and L. Song, “ImgSensingNet: UAV vision guided aerial-ground air quality sensing system,” in _IEEE Conference on Computer Communications - INFOCOM’19_ , Jul. 2019, pp. 1207–1215.
  * [20] A. Zappone _et al._ , “Wireless Networks Design in the Era of Deep Learning: Model-based, AI-based, or both?” _IEEE Transactions on Communications_ , vol. 67, no. 10, pp. 7331–7376, Jun. 2019.
  * [21] K. Sankhe, M. Belgiovine, F. Zhou, S. Riyaz, S. Ioannidis, and K. Chowdhury, “ORACLE: Optimized radio classification through convolutional neural networks,” in _IEEE Conference on Computer Communications - INFOCOM’19_ , Jul. 2019, pp. 370–378.
  * [22] J. Pérez-Romero _et al._ , “Supporting Mission Critical Services through Radio Access Network Slicing,” in _International Conference on Information and Communication Technologies for Disaster Management_ , Dec. 2019, pp. 1–8.
  * [23] L. Zanzi, V. Sciancalepore, A. Garcia-Saavedra, X. Costa-Pérez, G. Agapiou, and H. D. Schotten, “ARENA: A Data-driven Radio Access Networks Analysis of Football Events,” _IEEE Transactions on Network and Service Management_ , vol. 17, no. 4, pp. 2634–2647, Dec. 2020.
  * [24] Third Generation Partnership Project (3GPP), “Study on enhancement of 3GPP Support for 5G V2X Services (Release 16) ,” 3GPP TR 22.886 V14.0.0, Dec. 2018\.
  * [25] ——, “Feasibility Study on New Services and Markets Technology Enablers, Stage 1 (Release 14) ,” 3GPP TR 22.891 V14.2.0, Sep. 2016.
  * [26] J. Mei, X. Wang, and K. Zheng, “Intelligent Network Slicing for V2X Services Toward 5G,” _IEEE Network_ , vol. 33, no. 6, pp. 196–204, 2019\.
  * [27] D. Bega, M. Gramaglia, M. Fiore, A. Banchs, and X. Costa-Perez, “DeepCog: Cognitive Network Management in Sliced 5G Networks with Deep Learning,” in _IEEE Conference on Computer Communications - INFOCOM’19_ , Apr. 2019, pp. 280–288.
  * [28] J. X. Salvat, L. Zanzi, A. Garcia-Saavedra, V. Sciancalepore, and X. Costa-Pérez, “Overbooking Network Slices Through Yield-driven End-to-end Orchestration,” in _ACM CoNEXT_ , Nov. 2018, pp. 353–365.
  * [29] D. Bega, M. Gramaglia, M. Fiore, A. Banchs, and X. Costa-Perez, “AZTEC: Anticipatory Capacity Allocation for Zero-Touch Network Slicing,” in _IEEE Conference on Computer Communications - INFOCOM’20_ , Jul. 2020, pp. 280–288.
  * [30] Weihua Wang, P. B. Luh, J. H. Yan, and G. A. Stern, “An Improved Lagrangian Relaxation Method for Discrete Optimization Applications,” in _IEEE International Conference on Automation Science and Engineering_ , Aug. 2008, pp. 359–364.
  * [31] S. Martello and P. Toth, _Knapsack Problems: Algorithms and Computer Implementations_. USA: John Wiley & Sons, Inc., 1990.
  * [32] X. Foukas, M. K. Marina, and K. Kontovasilis, “Orion: RAN Slicing for a Flexible and Cost-Effective Multi-Service Mobile Network Architecture,” in _ACM International Conference on Mobile Computing and Networking_ , Oct. 2017\.
  * [33] L. Zanzi, V. Sciancalepore, A. Garcia-Saavedra, H. D. Schotten, and X. Costa-Pérez, “LACO: A Latency-Driven Network Slicing Orchestration in Beyond-5G Networks,” _IEEE Transactions on Wireless Communications_ , pp. 1–15, Oct. 2020.
  * [34] X. Ge, “Ultra-Reliable Low-Latency Communications in Autonomous Vehicular Networks,” _IEEE Transactions on Vehicular Technology_ , vol. 68, no. 5, pp. 5005–5016, Mar. 2019.
  * [35] R. Li, Z. Zhao, J. Zheng, C. Mei, Y. Cai, and H. Zhang, “The Learning and Prediction of Application-Level Traffic Data in Cellular Networks,” _IEEE Transactions on Wireless Communications_ , vol. 16, no. 6, pp. 3899–3912, Jun. 2017.
  * [36] J. Zhang, Y. Zheng, J. Sun, and D. Qi, “Flow Prediction in Spatio-Temporal Networks Based on Multitask Deep Learning,” _IEEE Transactions on Knowledge and Data Engineering_ , vol. 32, no. 3, pp. 468–478, Mar. 2020.
  * [37] R. M. Khanafer, B. Solana, J. Triola, R. Barco, L. Moltsen, Z. Altman, and P. Lázaro, “Automated Diagnosis for UMTS Networks Using Bayesian Network Approach,” _IEEE/ACM Transactions on Vehicular Technology_ , vol. 57, no. 4, pp. 2451–3058, Oct. 2008.
  * [38] F. Xu, Y. Lin, J. Huang, D. Wu, H. Shi, J. Song, and Y. Li, “Big Data Driven Mobile Traffic Understanding and Forecasting: A Time Series Approach,” _IEEE Transactions on Services Computing_ , vol. 9, no. 5, pp. 796–805, Aug. 2016.
  * [39] C. Ide _et al._ , “LTE Connectivity and Vehicular Traffic Prediction Based on Machine Learning Approaches,” in _IEEE Vehicular Technology Conference (VTC-Fall)_ , Sept. 2015, pp. 1–5.
  * [40] W. Ji, Z. Lu, and T. Zhu, “Mining Spatial-Temporal Travel Patterns from Highway Transaction Data,” in _ACM Symposium on Computer Science and Intelligent Control_ , Sept. 2018.