# Operationalizing AI/ML in Future Networks:
A Bird’s Eye View from the System Perspective

Qiong Liu, Tianzhu Zhang, Masoud Hemmatpour, Han Qiu, Dong Zhang, Chung Shue
Chen, Marco Mellia, and Armen Aghasaryan Q. Liu is with the Department of
Computer Sciences and Networks, Telecom Paris, Palaiseau, France. (E-mail:
qiong.liu@telecom-paris.fr)T. Zhang, C. S. Chen, and A. Aghasaryan are with
Nokia Bell Labs, Massy, France. (E-mail: {tianzhu.zhang, chung_shue.chen,
armen.aghasaryan}@nokia-bell-labs.com)M. Hemmatpour is with Simula Research
Laboratory, Oslo, Norway. (E-mail: mashemat@simula.no)H. Qiu is with the
Institute for Network Sciences and Cyberspace, BNRist, Tsinghua University,
Beijing, China. (E-mail: qiuhan@tsinghua.edu.cn)D. Zhang is with the College
of Computer Science and Big Data, Fuzhou University, Fuzhou, China. (E-mail:
zhangdong@fzu.edu.cn)M. Mellia is with the Department of Control and Computer
Engineering, Politecnico di Torino, Turin, Italy. (E-mail:
<EMAIL_ADDRESS>
###### Abstract

Modern Artificial Intelligence (AI) technologies, led by Machine Learning
(ML), have gained unprecedented momentum over the past decade. Following this
wave of “AI summer”, the network research community has also embraced AI/ML
algorithms to address many problems related to network operations and
management. However, compared to their counterparts in other domains, most ML-
based solutions have yet to receive large-scale deployment due to insufficient
maturity for production settings. This article concentrates on the practical
issues of developing and operating ML-based solutions in real networks.
Specifically, we enumerate the key factors hindering the integration of AI/ML
in real networks and review existing solutions to uncover the missing
considerations. Further, we highlight a promising direction, i.e., Machine
Learning Operations (MLOps), that can close the gap. We believe this paper
spotlights the system-related considerations on implementing & maintaining ML-
based solutions and invigorate their full adoption in future networks.

###### Index Terms:

AI/ML for networking, Network Systems, Network engineering

##  I Introduction

To drive digital transformation, modern telecommunication networks are
undergoing a disruptive evolution. The ongoing 5G rollout promises to deliver
customized network services to billions of subscribers with ultra-high speed,
ultra-high reliability, ultra-low latency, and ubiquitous connectivity. The
massification of Internet-of-Things is expected to connect trillions of
devices and create brand-new business models. Next-generation digital realms,
e.g., Metaverse, also call for high-quality, customizable communication
mediums to drive human-machine interaction and digital-physical fusion. These
technical headways inevitably make modern networks increasingly diverse,
decentralized, and complex. Traditionally, networks were mainly managed by
predefined or ad-hoc rules. However, these methods either bear oversimplified
assumptions about the underlying systems or incur huge computation overhead,
which disaccords with the continuing network complexification. As of today,
human participation is still essential for in-depth problem diagnoses and
decision-making.

With the accruing breakthroughs over the last decade, modern AI/ML has
achieved human-level performance in many challenging tasks, e.g., computer
vision, gaming, and, more recently, natural language processing. Nowadays,
AI/ML-empowered products have permeated various industrial and business
sectors, including healthcare, manufacturing, entertainment, and education.
According to the findings of Gartner and MIT Sloan Management Review, AI has
led to $3.9T of business value in 2022 and is deemed a strategic priority by
83% of CEOs [1]. Inspired by these successes, network researchers are
extensively exploring AI/ML algorithms for network automation and management.
This new breed of ML-based networking solutions111Note that we focus on
techniques of AI’s data-driven branch, frequently denoted as ML. By abuse of
language, we will use the two acronyms interchangeably in this paper., i.e.,
applications, functions, and services, has demonstrated more optimistic
outcomes than the traditional fixed-policy approaches [2].

Despite the enormous interest, AI/ML is still immature for deployment in real
networks. According to a recent report [3], $88\%$ of the telco industry’s
proof-of-concept AI/ML projects fail to reach live deployment. The major
deterrent stems from inadequate “system thinking”. Based on our observation,
existing AI/ML-based solutions have two fundamental disparities with real-
network deployments: (i) One-dimensional design: ML solutions mainly aim to
outperform prior solutions on specific metrics, especially accuracy, without
vetting other network-/system-critical imperatives. For example, as networks
usually become increasingly complex and intertwined, optimization becomes
multi-metric and multi-dimensional. (ii) System discrepancy: These solutions
were mostly demonstrated in controlled environments and became costly to fit
into real network systems with much higher scale, complexity, and dynamism.
For instance, given the data-driven nature of ML-based solutions, ensuring
their performance under sporadic data drifts and system evolutions is non-
trivial [4]. Consequently, such a “reality gap” hampers the integration and
deployment of AI/ML in real networks.

In information technology (IT) industry, the development and delivery of
software products is usually streamlined via Development and Operations
(DevOps) practices, which are further customized by the telco industry to
enhance service quality and reduce time-to-market [5]. Although network-
oriented DevOps practices offset part of the exertions via continuous
integration and delivery, they can hardly accommodate the unique
characteristics of AI/ML [4]. To smoothly operationalize (i.e., develop,
deploy, and manage) AI/ML in production, network operators must master skills
in ML, data engineering, and systems, which is extremely burdensome.

This paper aspires to elucidate the practical challenges of integrating AI/ML
into the future network landscape. Specifically, we present network-oriented
AI/ML research and its gap with real networks in Sec. II. In Sec. III, we
enumerate the practical considerations to actualize AI/ML in production-ready
networks. Afterward, we prospect a promising direction - Machine Learning
Operations (MLOps), which closes the gap of operationalizing AI/ML in telco
networks in Sec. IV.

##  II Background

In this section, we briefly review the current status of AI/ML and elaborate
on the practical barriers obstructing their pervasive adoption in operational
networks.

###  II-A Landing AI in Networks

In recent years, AI/ML has sparked tremendous hype in the telco industry
thanks to (i) the innovative breakthroughs in theoretical research, (ii) the
success in other fields such as computer vision and NLP, and (iii) the
presence of optimized development toolkits with hardware acceleration.
Compared to fixed-policy approaches, AI/ML algorithms exhibit exceptional
pattern matching, incremental learning, and automation capabilities on large-
scale, multi-dimensional data [6].

Standardization bodies (e.g., ETSI, 3GPP) anticipate AI/ML techniques to play
a pivotal role in automating future networks and have formed multiple working
groups to investigate different use cases [2]. In industry, carrier-grade
platforms are under active development to bolster AI/ML-augmented network
services: Nokia’s AVA Ecosystem offers telco operators cloud-native AI/ML and
analytic services to automate network operations, enhance service assurance
and subscriber experience and reduce cost [3]; Huawei’s ADN ecosystem features
network automation with dedicated support for AI operations [2], which
consists of three tiers, i.e., on-device AI, online fog/cloud AI, and offline
cloud AI, to support network and AI operations with assorted temporal-spatial
properties. In academia, ML algorithms are widely developed to tackle a large
spectrum of “networking” problems, such as traffic classification [7],
resource scheduling [6], anomaly detection [8], load balancing [9], QoE
management [10]. Given the rapid expansion of the AI/ML frontier (e.g.,
generative AI), their growth in telco networks will continue to enrich.

###  II-B The reality gap

Despite the plethora of solutions, a closer inspection reveals a less rosy
picture - These solutions are generally inadequate for live deployment in real
network systems [3]. Based on our study, such a reality gap originates from
three factors:

#### Data complexity

Compared to other prevalent AI/ML application domains, e.g., computer vision
and language processing, network data has much more diverse formats, e.g., raw
packets, flow, configuration files, system logs, and event alarms. They may
contain categorical, temporal, spatial, or even graph semantics. Such multi-
modal data with high variety, velocity, and volume can be exceedingly onerous
to model and process [10], not to mention their natural distribution drifts
caused by data and system evolvements.

Figure 1: ML lifecycle in production settings.

#### One-dimensional nature

Existing solutions generally focus on optimizing specific performance metrics
rather than comprehensively assessing the overall readiness, which is
incompatible with the stipulations of real-world ML systems. In particular,
some solutions strive for high prediction accuracy using supersized Deep
Neural Networks (DNNs) models that can hardly fit into resource-limited
network devices. The potentially high inference latency can make them
unsuitable for real-time constraints. They may also interfere with the
critical data path. But most of all, in production networks, all Key
Performance Indicators (KPIs) are naturally intertwined and must be jointly
analyzed and attuned to network/system constraints to avoid one-dimensional
solutions.

#### Hidden technical debts

This term was coined by Sculley et al. [11], which refers to the massive
operational costs of operationalizing ML-based systems by non-experts. Similar
debts also apply in network systems. As existing solutions were mostly
developed in simulated or controlled environments, the practical deployment
and maintenance issues were usually sidelined. In real systems, instead, ML
models should be deployed as part of a data-processing pipeline. Owing to
disparate development toolkits and deployment targets, integrating them into
real networks can be laborious and error-prone. As network devices can come
from sundry vendors with bespoke configuration, optimization, and execution
routines, deploying AI/ML on them can result in complicated manual tuning,
customization, and feasibility tests. In addition, rather than a one-off
process, ML-based solutions must be continuously upgraded to meet business
requirements and sustain long-term value over the rapid evolution of the telco
industry.

##  III Operationalizing AI/ML in production networks: the status quo

To close the gap and seamlessly operationalize AI/ML in production, many
critical system-related considerations exist throughout the ML lifecycle,
i.e., data preparation, development, and operations phases, as illustrated in
Fig. 1. This section encapsulates these considerations and explores associated
studies within the networking domain. The included works were chosen based on
two criteria: (i) they address one or more practical aspects, and (ii) the
methodologies proposed have undergone implementation and verification within
actual network systems.

###  III-A Data preparation

Data quality directly determines the ceilings of any AI/ML-based product,
spurring the recent trend towards data-centric AI [1]. Due to the complexities
in real networks, good datasets are not always available. Ensuring data
quality can averagely cost $60\%$ of time in AI/ML projects [3]. Special
considerations should be enforced upon data preparation to supply the ML
algorithms with high-quality data: the constituent data acquisition and
feature extraction processes.

Data acquisition. In existing solutions, data can generally originate from
three sources: (i) live networks, (ii) controlled environments, or (iii)
(curated) public datasets. In case (i), despite the multitudinous data
measurement and collection methods, the process can incur huge operational
costs, which obligates considerate tradeoffs [9]. For example, sampling is
usually prioritized over the per-packet collection in high-speed networks to
attenuate the impact on the traffic datapath. Also, network data collection
can incur uncontrollable situations, such as packet drops, sampling biases, or
schema changes, hence aberrations and outliers [7]. Labeling the collected
data still remains a daunting problem as it consumes substantial human effort
and does not scale with the data volume and scenarios. In cases (ii) and
(iii), as data are from outside the target networks, their statistical
properties can be unaligned with deployment assumptions, which leads to
unexpected consequences. The lack of labeled data (and datasets) often limits
the reproducibility and the solution generalization. Testing in real
environments becomes thus necessary to disclose the potential biases/anomalies
before model deployment. At last, existing solutions often assume
statistically stationary input properties, making them susceptible to
uncertainties and novelties commonly found in real networks.

Feature extraction. Raw network data must be converted to features conformant
with the ensuing AI/ML algorithms. Feature extraction is challenging -
different feature sets imply varied system costs (and model performance), thus
merit closer scrutiny: many existing ML-based solutions empirically define
custom features, which may become hard to obtain and scale in deployment.
Furthermore, feature selection schemes, when applied, might face revamping
upon network evolution.

Existing solutions: In contemporary network research, several seminal works
approached the practical challenges of data acquisition and feature
extraction: Bronzino et al. [10] introduced Traffic Refinery, an efficient
automation pipeline for flow-level data collection and feature extraction. It
aligns network operator goals by consolidating multiple design choices to
alleviate packet losses. Additionally, a dedicated profiler quantifies system-
level costs, offering operators a trade-off between feature selection and
model accuracy. In a distinct exploration, Yao et al. [9] proposed the
Aquarius framework to enable flexible data collection and feature extraction
for data center networks. This system embeds a transport-layer collector for
effective TCP traffic feature extraction, storing them in shared memory to
facilitate seamless ML algorithm interactions on the control plane, devoid of
data plane disruption. Lastly, Holland et al. [12] proposed the nPrint
framework, which transforms packets into a consistent binary format without
sacrificing contextual meaning. This mechanism empowers ML algorithms to
automatically identify key features, avoiding the efforts of manual feature
extraction.

The data preparation phase involves direct interaction with real networks,
which can be extremely challenging given the high-speed data flows and system-
level complexities. Despite the limited test scenarios, these solutions
provide valuable first-hand guidelines for efficiently collecting &
representing data and initiate a broader trend in network research toward
developing more sophisticated, ML-compatible methods for data preparation.

TABLE I: Synoptic of the related works. | Data | Feature | Algorithm | Hyperparam. | Model |  |  |  | Target |
---|---|---|---|---|---|---|---|---|---|---
Reference | acquisition | extraction | design | tuning | training | Validation | Deployment | Management | network | Use cases
Bronzino et al. [10] | ✓ | ✓ |  |  |  |  |  |  | - | QoE inference
Yao et al. [9] | ✓ | ✓ |  |  |  |  |  |  | Datacenter | Load balancing
|  |  |  |  |  | network | Traffic classification
|  |  |  |  |  | Resource scheduling
Holland et al. [12] |  | ✓ | ✓ | ✓ | ✓ | ✓ |  |  | - | Traffic analysis
Swamy et al. [8] |  |  | ✓ | ✓ | ✓ | ✓ | ✓ |  | Datacenter | Anomaly detection
|  |  | network | Traffic classification
|  |  | Botnet detection
Lacoboaiea et al. [6] |  |  |  |  | ✓ | ✓ |  |  | WLAN | Resource scheduling
Zheng et al. [13] |  |  |  |  | ✓ | ✓ | ✓ | ✓ | Datacenter | Anomaly detection
|  |  |  | network | QoE inference
Yang et al. [4] |  |  |  |  |  |  |  | ✓ | - | Traffic classification

###  III-B Development

Model development consists of two fundamental steps, i.e., algorithm design,
model training & validation, each crucial to determine a solution’s overall
readiness for the target network.

Algorithm design. The purpose of ML can be threefold: (i) making effective use
of existing knowledge, (ii) gathering a structured understanding of unknown
phenomena, and (iii) learning to achieve a goal, which can be mapped to three
branches, i.e., Supervised, Unsupervised, and Reinforcement Learning (RL) –
with potential intersections among them (e.g., semi-supervised or self-
supervised learning).

Supervised ML techniques, such as regression and classification, excel at
tracking well-specified problems in open-loop settings to increase visibility
about network traffic or distill insight from raw data. In particular,
regression techniques are fit for forecasting (e.g., traffic demand or user
behavior) or learning complex relationships, such as relating network Quality
of Service (QoS) indicators to user Quality of Experience (QoE).
Classification techniques are another related example where AI techniques are
useful: traffic prioritization requires coarse-grained traffic class labels
for policing and may additionally require fine-grained application labels.

Unsupervised ML operates by identifying patterns and structures within data
without labeling, relying instead on the algorithm ability to discern
intrinsic features and relationships within the dataset. For example,
unsupervised AI employs algorithms in anomaly detection to discern data
deviations by autonomously learning underlying distributions. These algorithms
identify outliers representing significant departures from established
patterns without reliance on pre-labeled normal data instances.

RL is suitable for sustained and efficient closed-loop AI automation
environments. An example is the automation of resource management by using RL,
implemented through centralized cloud agents or distributed device agents. In
this context, AI agents are dedicated to improving QoS, e.g., enhancing
transmission efficiency and reducing latency. To attain such a goal, agents
are rewarded for their actions, effectively balancing exploration and
exploitation within a vast state space, thus providing network operators with
automated and optimized solutions [12].

Model training & validation. In the system context of model training &
validation, factors such as inference efficiency, generalizability, and safety
hold similar significance as the traditional focus on accuracy. For instance,
generalizability ensures timely adaptation in dynamic environments like
disaster-resilient networks, safety is crucial for ML algorithms that require
frequent interaction with real systems, and inference efficiency is crucial
for quick decision-making.

Existing solutions: Two prior works explore AutoML to automatically carry out
model selection and hyper-parameter tuning to hide the AI/ML-specific
complexities from network operators. Holland et al. [12] leverage the
AutoGluon-Tabular framework to locate and ensemble models with high predictive
accuracy and low inference latency, given the features and labels. Similarly,
Swamy et al. [8] employ an optimization framework that automatically performs
algorithm selection and model generation as a Bayesian optimization problem
based on user intents and network constraints. Lacoboaiea et al. [6] address
the challenges of building a Deep RL-based channel manager, specifically
focusing on training safety, efficiency, environment realism, and
generalization. They leverage digital twins for secure training, adjust
learning rates for efficiency, enhance simulator fidelity with real-world
data, and bolster generalization via synthetic noise and actual data
integration.

These initial research endeavors provide viable directions for model
development in the context of real network systems. Moving forward, the focus
of network-oriented AI/ML-based solutions will likely shift towards further
simplifying the complexity of these technologies for more diverse network
environments and application scenarios.

###  III-C Operations

In this part, we elaborate on AI/ML-based solutions that require attention in
real-world networks concerning deployment and management.

Deployment. Operational deployment encompasses packaging, customization, and
feasibility tests. As ML-based solutions were mainly intended for the control
plane, these tasks can be handled by general-purpose model serving tools.
Recently, intrigued by the advantages of in-network ML, researchers began to
push the ML frontier into the network data plane to capitalize on the
voluminous data there [13]. Model deployment becomes a Sisyphean task due to
the distinctions between the local implementation environment and network
infrastructure, and the divergent tooling can sorely impede customization.
Moreover, as networks are replete with a plethora of specialized hardware
devices with disparate architectures, configuration routines, and resource
footprints, the deployment process entails refactoring a solution into a
generic data-processing pipeline with minimal interference on the network
service [8].

Management. Furthermore, managing the deployed ML-based solutions involves
model serving, resource & operation management, and drifting monitoring tasks.
In particular, as network systems can evolve expeditiously, the intrinsic
concept/data drifts can result in model decay and service degradation. The
inference quality should thus be constantly inspected to detect performance
diminishments and trigger the model-rebuilding process whenever applicable. In
real networks, the correct quality metrics and triggers should be carefully
scoped, and the monitoring overhead should also be balanced with the quality
assessment accuracy [4]. Depending on the problem context, the rebuilding
process can start from the data preparation and labeling or model development
stage, which must be specified beforehand.

Existing solutions. To cope with these challenges, Zheng et al. [13] introduce
the Planter framework as a countermeasure to these limitations. This modular
architecture facilitates the seamless deployment of diverse in-network ML
algorithms across three prominent hardware platforms: Intel Tofino, BMv2, and
P4Pi. Notably, Planter is adept at accommodating a slew of mainstream ML
algorithms. The post-training phase sees these models automatically transform
into P4 code tailored to the specific target, subsequently undergoing
compilation and data plane integration for functional validation and
deployment. On the other hand, Swamy et al. [8] craft compiler tools designed
to autonomously render target-oriented code for three widely adopted
deployment platforms: FPGA, Tofino, and Taurus. Additionally, they harness a
cycle-accurate simulator to preemptively gauge the model’s key performance
indicators (KPIs), encompassing throughput, latency, and resource allocation
metrics. Yang et al. [4] focus on the pivotal concern of monitoring. Their
proposed methodology amalgamates gradient-based techniques with Open Set
Recognition and explainable AI paradigms to scrutinize and gauge the quality
of inferences. Comparative evaluations have been conducted to validate the
proficiency of their approach in delivering meticulous model evaluations,
consistently monitoring inference quality, and detecting nuanced data drifts.

These works constitute the vital advancements in deploying and managing the
pre-trained AI/ML-based solutions in real network systems. As the network
vendors keep introducing more sophisticated features and devices, the
supported scope of ML models, deployment targets, and drift detection methods
still require further expansion, which the tools and methods of these works
can assist.

Based on the discussions in Sec. III-A, III-B, III-C. We summarize all the
pioneering works in Table I in terms of the tackled lifecycle stages,
supported types of ML algorithms, targeted network environment, and already-
validated networking use cases. Essentially, each work covers part of the ML
lifecycle stages.

Figure 2: Conventional vs. Automated ML lifecycle.

###  III-D Missing pieces to the puzzle

Based on the proceeding review, we identify three missing pieces to the fully
operationalized AI/ML puzzle. First, despite the optimistic individual
advancements, they have not been cumulatively translated into global benefits.
In real systems, all the individual stages must be seamlessly articulated as
an end-to-end data processing pipeline to reduce the management & maintenance
overhead. With the current reliance on manual interventions, ML-based
solutions will become heavy to manage in future networks. Second,
reproducibility is not enforced due to the absence of systematic logging and
tracking. Traditional version control tools cannot sufficiently capture the
nuances of ML workflows’ datasets, parameters, and configuration dependencies,
which must be consistently reproducible for scientific rigor and regulatory
compliance. Third, communication barriers or silos can also arise due to the
disparate expertise & priorities of data scientists and network engineers,
hampering productivity and slowing time-to-value.

We specifically illustrate two approaches for managing the ML lifecycle in
Fig. 2. The traditional workflow involves a one-off process involving data
collection, model development, and deployment. This approach prioritizes rapid
delivery for the initial time. Nonetheless, as the temporal dimension extends,
this method becomes less efficient. In particular, the data/system shifts
necessitate continuous model retraining. Without proper management of the
model development data, reproducing and enhancing existing models become
laborious as the whole process can involve multiple teams, from data science
to network engineering. Manual asset transferring is inefficient and
burdensome.

Conversely, the second approach adopts a more systematic approach. Initially,
the involved teams dedicate significant time to constructing an automated
pipeline with established tracking mechanisms. Compared to the conventional
manual approach, it confers substantial long-term benefits. Being fully
automated, the process ensures consistent reproducibility, enabling continuous
model enhancement, and paves the way for concerted efforts across different
teams.

##  IV MLOps: towards end-to-end pipelines

This section discusses MLOps as a potential solution for relieving the
operational overhead.

###  IV-A Why MLOps?

MLOps is an emerging set of engineering practices in the ML field aiming at
applying DevOps principles to unify the development and operation of ML-based
systems. Traditionally, the operational costs of delivering software products
can be countered with DevOps, which encompasses an assemblage of principles to
break the silo between software developers and IT operations engineers,
promoting Automation and Continuous Integration (CI)/Continuous Deployment
(CD) throughout the product lifecycle. These principles help drive IT and
business outcomes for many businesses and organizations. The network community
has adopted DevOps to fuel technological innovation and revenue growth [4].

However, though DevOps can curb the operational overhead of productionalizing
traditional software projects, they lack supplemental support for the unique
characteristics of ML. There are five fundamental discrepancies between
conventional software and ML: First, code quality predominantly decides the
performance in traditional software; In AI/ML, the model and data all impact
the outcome [1]. Second, traditional software is usually built on full-fledged
libraries with clear abstraction boundaries [11]. ML-based solutions, au
contraire, often involve a broader range of tools, libraries, and platforms,
subject to extra integration and maintenance costs. Third, unlike traditional
software that conveys deterministic outputs, ML models are intrinsically
stochastic and entail disparate processes to validate their behaviors. Fourth,
ML models are susceptible to data/concept drifts, which are common in real
networks and thus necessitate drift monitoring and model rebuilding [4].
Finally, building and operating ML-based solutions call for data science
skillsets, which are missing in traditional software/network routines.
According to a recent survey, $55\%$ of telcos lack the pertinent data science
talent [3]. Although network practitioners can gradually get acquainted with
AI/ML and data science, mastering the theories and technical details requires
time.

Figure 3: MLOps for networking: A tenable architecture.

###  IV-B What is MLOps?

Layered on the DevOps tenets, MLOps accommodates the unique traits of AI/ML
with the following practices:

  * •

Continuous Monitoring (CM) / Continuous Training (CT): MLOps addresses the
model decay problem by constantly monitoring the data and inference quality
and rebuilding the model whenever applicable.

  * •

Automation: To alleviate the operational cost, MLOps aims to streamline all
the AI/ML lifecycle stages as a fully automated pipeline without human
intervention.

  * •

Versioning: Based on the DevOps version control for code, MLOps advocates the
version control of artifacts involved in the process, including data, model,
and code. The accompanying data and feature stores also simplify data
governance.

  * •

Experiment tracking: All experiments should be systematically tracked to
ensure reproducibility and auditability.

  * •

Collaboration: MLOps advocates a common platform and language to build synergy
across the involved persons with different priorities and expertise.

With these practices, MLOps proposes a promising way to consolidate
innovations across the ML lifecycle and dramatically curtail operational
costs, even though this burgeoning discipline is still nascent in the AI/ML
network community. We envision a plausible architecture in Fig. 3, where most
MLOps practices can be adopted for real networks, though considerable tooling
and engineering efforts are still inevitable.

###  IV-C MLOps for networking: A case study

This section demonstrates the benefits of MLOps with a case study on a real
network benchmark. Specifically, we implement an ML-based solution for KPI
forecasting, a crucial task for predictive maintenance in operational
networks. To retain production setting, we deploy software network services
across a distributed cluster inside a small-scale enterprise datacenter, where
each node is a commodity off-the-shelf server equipped with Intel® Xeon CPUs
and Intel® 10/40-Gigabit network cards. The network services attain line-rate
traffic processing. We predict the overall network service performance, e.g.,
throughput, using the features of the underlying hardware subsystems,
including Processor, Memory, and PCI buses, to circumvent the intricacy of in-
band data collection. These features can be readily obtained using standard
profiling tools. As detailed in [14], though these features seem less relevant
than the packet-/flow-level statistics, they contain valuable insights on
service performance and can be utilized for KPI predictions. To elude the
aforementioned manual exertions throughout the data preparation, development,
and operations phases, we adopt common MLOps practices to build a prototypical
data processing pipeline, whose workflow is detailed as follows:

  * •

Data preparation: We define standard procedures for the entailed steps (i.e.,
data collection, validation, and labeling) to automatically obtain high-
quality data without interfering with the normal operations in the data plane.

  * •

Feature selection: To extract the most relevant features from the $\sim$600
features collected by standard profiling tools (e.g., Linux perf, Intel PCM),
we employ Pearson’s correlation coefficient to assay their statistical
dependencies with the target service KPIs.

  * •

Automated model selection: According to the studies of Manousis et al. [14],
we consider three ML models: Linear Regression, Gradient Boosting Regression,
and Artificial Neural Network, for automated model selection.

  * •

Automated hyper-parameter tuning: We employ Bayesian optimization to evaluate
the performance of different hyperparameter combinations and build a
probabilistic model to automatically select the optimal one.

  * •

Performance monitoring and feedback loop: We employ the Jensen-Shannon
divergence to inspect the input data distribution to establish a continuous
monitoring mechanism. When the new data diverges from the training data, the
feedback loop should be activated to retrain and roll out an up-to-date model.

  * •

Versioning and experiment tracking: we employ MLflow [1] to maintain detailed
records of all the involved model artifacts, datasets, (hyper-)parameters, and
other assets in the model development process, ensuring both reproducibility
and auditability.

Figure 4: The MLOps benefits on accuracy and efficiency.

An empirical evaluation of our data processing pipeline is presented in Fig.
4. We focus on two test scenarios: a singleton virtual network function (L2
bridge) and a Linear Service Chain (SFC) with five network functions. For each
scenario, we collect the training data under varied input loads. We first
investigate the benefits of incrementally applying the customized data
preparation, feature selection, model selection, and hyper-parameter tuning
procedures. As shown in Fig. 4, our systematic refinement strategies can
incrementally enhance the overall accuracy, i.e., $76\%\rightarrow 96\%$ for
bridge and $62\%\rightarrow 84\%$ for SFC. Furthermore, the implemented
automation can significantly reduce the incurred model delivery time compared
to the conventional, manual approaches. In particular, the time required to
detect data drift has been reduced to merely $0.4$ seconds, thanks to the
seamless identification of distribution shifts. Given that our prototype has
not even integrated the most advanced tools and techniques, we believe MLOps
can offer huge performance boosts and smooth the adoption of AI/ML in future
networks.

##  V Conclusion

Due to the lack of system-related considerations, AI/ML is still not an
integral part of modern networks. This paper analyzed the inconsistencies
between existing AI/ML-based solutions and real network systems and discussed
all the practical considerations throughout their product lifecycle. We also
reviewed the related works and identified the missing pieces. Then, we
conducted a case study to validate the advantages of MLOps in a real network
system. Based on our experience, we recommend MLOps as a promising way to
erase operational concerns. We believe this paper can raise awareness about
the practical hurdles of developing, deploying, and managing AI/ML-based
solutions in production settings and expedite the integration of AI/ML in
future network systems.

## References

  * [1] J. Bradley _et al._ , “The Big Book of MLOps,” https://www.databricks.com/p/ebook/the-big-book-of-mlops, 2022.
  * [2] D. Rossi _et al._ , “Landing AI on networks: An equipment vendor viewpoint on autonomous driving networks,” _IEEE TNSM_ , 2022.
  * [3] Nokia Networks, “AVA AI and analytics,” https://nokia.com/networks/ai-and-analytics, 2022.
  * [4] L. Yang _et al._ , “Quality monitoring and assessment of deployed deep learning models for network AIOps,” _IEEE Network_ , 2021.
  * [5] W. John _et al._ , “Service provider devops,” _IEEE Communications Magazine_ , 2017.
  * [6] O. Iacoboaiea _et al._ , “From design to deployment of zero-touch deep reinforcement learning WLANs,” _IEEE Commun. Mag._ , 2023.
  * [7] F. Pacheco _et al._ , “Towards the deployment of machine learning solutions in network traffic classification: A systematic survey,” _IEEE Commun. Surveys Tuts._ , 2018.
  * [8] T. Swamy _et al._ , “Homunculus: Auto-generating efficient data-plane ML pipelines for datacenter networks,” _arXiv:2206.05592_ , 2022.
  * [9] Z. Yao _et al._ , “Aquarius-enable fast, scalable, data-driven service management in the cloud,” _IEEE TNSM_ , 2022.
  * [10] F. Bronzino _et al._ , “Traffic Refinery: Cost-aware data representation for machine learning on network traffic,” _ACM POMACS_ , 2021.
  * [11] D. Sculley _et al._ , “Hidden technical debt in machine learning systems,” _NeurIPS_ , 2015.
  * [12] J. Holland _et al._ , “New directions in automated traffic analysis,” in _ACM CCS_ , 2021.
  * [13] C. Zheng _et al._ , “Automating in-network machine learning,” _arXiv:2205.08824_ , 2022.
  * [14] A. Manousis _et al._ , “Contention-aware performance prediction for virtualized network functions,” in _ACM SIGCOMM_ , 2020.


# Risk-Aware Navigation for Mobile Robots in Unknown 3D Environments

Elie Randriamiarintsoa 1, Johann Laconte 2, Benoit Thuilot1, Romuald Aufrère1
1Université Clermont Auvergne, Clermont Auvergne INP, CNRS, Institut Pascal,
F-63000 Clermont-Ferrand, France<EMAIL_ADDRESS>of
Toronto Institute for Aerospace Studies (UTIAS), 4925 Dufferin St, Ontario,
Canada<EMAIL_ADDRESS>
###### Abstract

Autonomous navigation in unknown 3D environments is a key issue for
intelligent transportation, while still being an open problem. Conventionally,
navigation risk has been focused on mitigating collisions with obstacles,
neglecting the varying degrees of harm that collisions can cause. In this
context, we propose a new risk-aware navigation framework, whose purpose is to
directly handle interactions with the environment, including those involving
minor collisions. We introduce a physically interpretable risk function that
quantifies the maximum potential energy that the robot wheels absorb as a
result of a collision. By considering this physical risk in navigation, our
approach significantly broadens the spectrum of situations that the robot can
undertake, such as speed bumps or small road curbs. Using this framework, we
are able to plan safe trajectories that not only ensure safety but also
actively address the risks arising from interactions with the environment.

##  I INTRODUCTION

Navigating in unknown 3D environments is a crucial task for mobile robots.
Before being able to move, a robot must represent the environment in which it
evolves. A well-known and efficient way to map the environment is the
occupancy grids introduced by [1]. An occupancy grid provides the robot with
information about the potential presence of an obstacle at a given position.
However, this information alone does not encompass all the capabilities of the
robot to maneuver within its environment. For example, a wheeled robot can
safely cross grasses, a curb, or a speed bump at low speed, as shown in Figure
1. Therefore, a way to assess their hazardous nature is necessary to evolve in
these environments. Motivated by this fact, numerous recent works [2] have
contributed to risk-aware navigation. However, [3] demonstrated that the
Bayesian occupancy grid, which stores the probability of collision of a given
position, is ill-suited to compute the risk over paths. Indeed, the
probability of collision, computed as the joint probability that every cell is
free of obstacles, is highly dependent on the grid tessellation size. To
overcome this difficulty, they developed a novel framework called Lambda-Field
to assess physics-based risks on occupancy grids.

Figure 1: Example of a situation that a vehicle might encounter while
navigating. Speed bumps are frequent obstacles an intelligent vehicle has to
safely overcome. With our framework, the vehicle is able to make more
effective decisions based on the level of risk it is allowed to take.

Moreover, in occupancy grids framework, most navigation methods [4] use
geometric and semantic reasoning to handle the obstacles present in the
environment, such as curbs, traffic cones, speed bump, sidewalk, buildings,
traffic circle and traffic lights shown in Figure 1. [3] showed that path
planning becomes more intuitive and meaningful when a physical risk metric is
used. However, their work has been developed solely for 2D environments. Here,
we adapt their work to 3D environments and use physical, risk-based reasoning
to deal with the obstacles represented in Figure 1.

The main contributions of this paper are i) an extension of the Lambda-Field
[3] to 3D environments; ii) a generalization of the risk that take into
account traversable obstacles; iii) a mathematical formulation of a local
risk-aware path planning algorithm in the Lambda-Field; iv) a demonstration of
the applicability of the method, using data acquired in real urban
environment.

##  II RELATED WORK

An important challenge in local motion planning is to find a feasible path
that matches the robot’s capabilities while being safe to traverse. Prior to
identifying such a path, it is essential to accurately map the immediate
surroundings of the robot, thereby representing its local environment. A
popular representation of the robot surroundings is the occupancy grids,
introduced by [1]. The main idea of the occupancy grids is to tessellate the
environment into cells and to store in each cell the information of occupancy.
[5] enhanced the previous idea by adding a Bayesian layer to the approach.
Their work led to the Bayesian Occupancy Filter (BOF) techniques.

Using these Bayesian occupancy grids, standard motion planning frameworks plan
collision-free paths by assessing the probability of collision of each
potential robot path. However, assessing only the risk of colliding with an
obstacle will always lead the robot to go around them. Therefore, the
collision risk does not fully exploit the robot ability. Indeed, a mobile
robot can cross tall grass or a speed bump even if it results in collisions.
To allow robots to make more insightful choices, several approaches propose to
integrate the notion of risk in motion planning by using risk maps. Unlike
conventional Bayesian occupancy grids, where each cell stores the occupancy
probability of a given position, a risk map stores the risk at this position.
[6] designed a risk map for cognitive vehicles. The defined risk map is used
to prevent the robot from approaching the hazardous obstacles, such as
pedestrians or cars. [7] used historical shipping traffic and bathymetry data
of coastal regions to create a risk map for underwater vehicles. Assigning a
probabilistic risk value to each position that is likely to be occupied allows
to avoid potential hazardous collisions. [8] used a risk map to quantify the
risk of an unmanned aerial vehicle flying over a given position to cause
lethal incidents in an inhabited area. Even though these techniques produce
good results, they all make the assumption that the risk is only dependent on
the robot position, while intuitively the risk depends also on the robot state
and capabilities. As such, our work proposes a risk model that exploits both
the state of the environment and the robot ability to assess the risk of a
given path.

In risk-aware navigation, [9] provide insight about which metrics should be
used. Among them, the expected value and the Conditional value-at-risk (CVaR)
are identified as valid candidates. The CVaR is used to capture the worst-case
expected hazardous event that could happen over a given horizon of time, and
has been used by several recent works [10, 11]. [12] measure the risk of
colliding with randomly moving obstacles. They define the problem of path
planning as a constrained optimization where the distance to the obstacles
must be superior or equal to a fixed threshold. They demonstrate that this
formulation allows to plan an effective path for a quadrotor in a 3D
environment, while adjusting the safety and prudence of the motions. For
mobile robots, [13] enhance the previous work by adding multiple sources of
risk, such as the slippage risk. A new risk map is then created by aggregating
the risks that come from these different sources. [14] quantify the risk by
converting a learned speed distribution map to a risk cost value. In constrast
to our work, traversability is captured via experienced trajectories. [15]
propose another risk-aware path planning which is also based on a priori data,
precisely on a known map. In [3], the authors introduce the Lambda-Field, a
generic method to assess a physical risk over a continuous path. The risk is
defined as the expected force of collision along a path. They introduced a new
mapping framework where each cell stores the density of collisions that could
be hazardous to the robot. In contrast to the Bayesian occupancy grids
framework, the Lambda-Field framework computes the integral of the risk over a
given path. This approach has the ability to retain the physical units of the
risk. They have demonstrated the relevance of the approach in 2D static
environments. They extended their work to unstructured [16] and dynamic
environments [17], yet still only considering 2D obstacles. In this article,
we adapt their framework to 3D static environments and propose a new
physically coherent risk measure.

##  III PRELIMINARIES

In this section, we briefly present the framework developed by [3]. In a
similar way as occupancy grid frameworks, the environment is tessellated into
cells, where all cells have a fixed size and area $\Delta
a\in\mathbb{R}_{>0}$. The core difference of their approach, compared to a
standard Bayesian occupancy grid, is the information that is stored in each
cell. Instead of estimating the probability of collision, they estimate the
density of collision $\lambda\in\mathbb{R}_{\geq 0}$ for each cell, also
called the intensity. The probability of collision within the cell is then
$\lambda\Delta a$. The higher the intensity of a cell, the more likely it is
that a hazardous event will happen in this cell. On the contrary, an intensity
of zero means that the cell will never lead to a harmful collision and can be
crossed safely. As shown in [3], the probability of facing at least one
hazardous event (i.e., collision) over a path $\mathcal{P}$ in the discretized
field is

$\mathbb{P}(\text{coll}|\mathcal{P})=1-\exp\left(-\Delta
a\sum_{c_{i}\in\mathcal{C}}\lambda_{i}\right),$ (1)

where $\mathcal{C}$ is the set of cells crossed by the path $\mathcal{P}$, and
$\lambda_{i}$ is the intensity of a cell $c_{i}\in\mathcal{C}$.

In order to build this map, [16] use a 2D lidar. If $e$ is the error region
area attached to every lidar measurement, then a cell $c_{i}$ is measured as
hazardous if it is located within the error region area of a lidar beam
impact. Otherwise, if the lidar beam traversed the cell without returning a
collision, the cell is measured as safe. Under these considerations, the
intensity $\lambda_{i}$ of each cell is computed using an expectation
maximization approach. Let $s_{i}$ be the number of times a cell $c_{i}$ is
measured as safe and $h_{i}$ the number of times a cell $c_{i}$ is measured as
hazardous, it can be shown, see [16], that the intensity of the cell $c_{i}$
can be expressed as

$\lambda_{i}=\frac{1}{e}\ln\left(1+\frac{h_{i}}{s_{i}}\right).$ (2)

Furthermore, the main advantage of the Lambda-Field is that it enables the
computation of the probability of collision, but also of a generic risk over a
path. Following the demonstration in [16], the expected value of a generic
deterministic risk over a path $\mathcal{P}$ crossing the cells
$\\{c_{i}\\}_{0:N-1}$, is given by

$\mathbb{E}[r(X)]=\sum_{i=0}^{N-1}K_{i}r(x_{i}),$ (3)

where $x_{i}$ corresponds to the position on the path associated to the cell
$i$ and $r(x_{i})$ is a generic risk function that gives the value of the
deterministic risk if a collision would happen at $x_{i}$. $K_{i}$ gives the
probability of encountering an harmful event at $x_{i}$ and is given by

$K_{i}=\exp\left(-\Delta a\sum_{j=0}^{i-1}\lambda_{j}\right)(1-\exp(-\Delta
a\lambda_{i})).$ (4)

$\mathbb{E}[r(X)]$ encompasses both the state of the environment through the
intensity field, and the state of the robot when it comes at a given position
along the path, though the risk function $r(x_{i})$. As such, this framework
provides a way to compute a meaningful risk over a path in occupancy grids.

##  IV MAPPING

We present here our extension of the Lambda-Field to take into account
traversable 3D obstacles in the environment. For that, we use a 3D lidar to
compute a Digital Elevation Map (DEM) [18]. By aggregating multiple point
clouds over time, we compute the difference in elevation of the environment.
This is achieved by taking into account the elevations of the eight
neighboring cells for each grid cell. Then, this DEM is used to compute the
Lambda-Field.

To model traversable obstacles in the context of urban environments containing
road curbs and speed bumps. We define a cell $c_{i}$ as safe if the value of
the difference in elevation is below a threshold value $H_{\text{safe}}$;
otherwise, this cell is defined as hazardous. The intensity equation Equation
2 is however modified to take into account the severity of the event and will
stop the robot in its course. As such, the intensity of a cell $c_{i}$ is
computed with

$\lambda_{i}=\frac{1}{e}\ln{\left(1+\frac{h_{i}}{s_{i}}\right)}p_{i}$ (5)

where quantity $p_{i}$ describing the severity of this collision is defined by

$p_{i}=\min\left(\frac{|H_{i}|}{R},1\right),$ (6)

with $H_{i}$ the difference in elevation of the cell $c_{i}$ with reference to
its neighbors and $R$ the radius of the wheel. As such, the likelihood it stop
$p_{i}$ is one when the obstacle is higher than the wheel radius and the robot
has no chance to go over the obstacle, and tends to zero for small obstacles
which are harmful. In the conservative case where we assume the robot is not
able to go over any obstacle, then $p_{i}=1$ and the measurement equation
returns to Equation 2. Equation 5 is then a generalization of the approach
where we consider that the robot is also able to traverse over small
obstacles.

As an example, Figure 2 depicts a Lambda-Field computed using Equation 5, for
the environment shown in Figure 1. The global Lambda-Field map is shown at the
top in Figure 2 and the construction of the map around the speed bump (in
yellow in Figure 1) at different times is illustrated below. At time $t_{1}$,
the speed bump is in the lidar range and the intensities $\lambda_{i}$
representing the speed bump began to converge. The speed bump is better
represented at time $t_{2}$ through new and additional measurements. As the
speed bump has been completely scanned at time $t_{3}$, the intensities
associated with it have completely converged.

Figure 2: Example of Lambda-Field of the environment depicted in Figure 1. The
area containing the speed bump and the cones is enlarged and shown at
different times. The lidar range (vertical line) and the robot (box) pose for
each of these times are illustrated in teal ($t_{1}$), dull green ($t_{2}$)
and blue ($t_{3}$). The left curb (light blue) and the traffic lights (green)
seen in Figure 1 are outlined in dashed lines.

One can note that the curbs, traffic cones, speed bump, buildings and traffic
lights poles are well detected. The borders of the speed bump have high
intensity values, when its ascending and descending portions have more nuanced
intensities, meaning that harmful collisions are less prone to happen. As a
result, the robot will be able to cross the speed bump by taking a reasonable
risk. Additionally, buildings, traffic light poles, and traffic cones have
high intensities. These obstacles cannot be crossed by the robot without
taking an unreasonable risk.

##  V RISK ASSESSMENT

In order to navigate in the Lambda-Field, we define a risk function that
reflects the potential obstacles that the robot can face. We choose to
describe the risk by the maximum potential energy absorbed by the wheels.

We illustrate in Figure 3 the modeling of the wheel. The wheel is modeled as a
deformable disk, see Figure 3. When the wheel crosses an elevated obstacle, it
deforms at the point where the two objects collide. This deformation is
approximated with the deformation of a spring of stiffness $k_{r}$.

Figure 3: Modeling of the wheel of radius $R$ in collision with the curb (in
blue) of height $H$, with a speed $v$ and angle $\Psi$. The deformation of the
wheel due to the collision is approximated with the deformation of a spring of
stiffness $k_{r}$.

In order to find the maximum amplitude $l_{m}$ of the tire compression due to
the collision, we solve the following differential equation:

$\ddot{l}+\omega^{2}l=0\quad\text{with }\omega=\sqrt{\frac{k_{r}}{m}}$ (7)

where $l$ is the compression at the contact point and $m$ is the mass of the
vehicle. As such, the maximum amplitude $l_{m}$ of the tire compression is

$l_{m}=\frac{v\cos(\Psi)}{\omega}$ (8)

where $v$ is the linear velocity of the robot and
${\Psi=\arcsin(\frac{R-\min(H,R)}{R})}$ is the angle of attack of the
collision. The angle $\Psi$ depends on the elevation of the obstacle and the
radius $R$ of the wheel. To be conservative, we take the maximum difference in
elevation $H$ of the obstacles that lie in the transverse axis of the robot
path $\mathcal{P}$. Finally, the risk function is defined as the maximum
potential energy that will be absorbed in the wheels at the time of collision
and is computed by

$r(X)=\dfrac{1}{2}k_{r}l_{m}^{2}$ (9)

As such, the risk function $r(X)$ has a clear physical meaning and is
expressed in Joule. In the next section, we use this risk assessment to
construct safe paths in 3D environments.

##  VI PATH PLANNING

The goal of the robot is to follow a local reference path
$\mathcal{P}_{ref}\subset\mathbb{R}^{3}$ that is assumed to be known. However,
unforeseen events, detected by the robot’s sensors, can make this path
impassable. The aim of our path planning framework is to find the optimal path
$\mathcal{P}^{*}\subset\mathbb{R}^{3}$ that is the closest to the reference
path $\mathcal{P}_{ref}$, while maintaining a reasonable level of risk along
the path. Namely, a path $\mathcal{P}$ is accepted only if the risk undertaken
by the robot along the path is below a user-defined risk threshold
$r_{threshold}$, consistent with the preferences of the user and the robot
abilities. The optimal path $\mathcal{P}^{*}$ also minimizes the traversal
time of the robot.

To address the problem, we rely on a Nonlinear Model Predictive Control (NMPC)
approach, where we use the Ackermann steering model as the prediction model.
Using a small sample time $\Delta t$, the kinematic equations can be
discretized into

$\displaystyle\begin{bmatrix}x_{k+1}\\\ y_{k+1}\\\
\theta_{k+1}\end{bmatrix}=\begin{bmatrix}x_{k}\\\ y_{k}\\\
\theta_{k}\end{bmatrix}+\Delta t\begin{bmatrix}v_{k}\cos{\theta_{k}}\\\
v_{k}\sin{\theta_{k}}\\\ v_{k}\tan{\delta_{k}}/L\end{bmatrix},$ (10)

where $k$ represent the discrete time, $L$ is the length of the wheelbase of
the robot, $(x_{k},y_{k})$ and $\theta_{k}$ are respectively the centre
position of the rear axle and the yaw angle of the robot in the absolute
frame. Finally, $v_{k}$ is the linear velocity of the robot, and $\delta_{k}$
the steering angle of the robot. The compact form of Equation 10 is written as
$\textbf{x}_{k+1}=f_{d}(\textbf{x}_{k},\textbf{u}_{k})$, where
$\textbf{x}_{k+1}=[x_{k+1},y_{k+1},\theta_{k+1}]^{\intercal}$ is the predicted
state vector, $\textbf{x}_{k}=[x_{k},y_{k},\theta_{k}]^{\intercal}$ is the
state vector, and $\textbf{u}_{k}=[v_{k},\delta_{k}]^{\intercal}$ is the
control vector. The reference trajectory associated with the path
$\mathcal{P}_{ref}$ is defined as a set of desired states
$[\textbf{x}^{r}_{0},\cdots,\textbf{x}^{r}_{N_{p}}]^{\intercal}$, where
$\textbf{x}^{r}_{k}$ is the desired state at time $k$.

According to our objective and constraints, we want to minimize the errors
between the predicted and the reference paths over a finite prediction horizon
of length $N_{p}$ while considering kinematic, control, and risk constraints.
We also want to minimize the traversal time and penalize the error in the
final state. The penalization of this latter error is used to force the robot
to approach the farthest reference goal $\textbf{x}^{r}_{N_{p}}$ located in
the perceived and therefore secure navigable area. As a result, the risk-aware
navigation based on NMPC consists in computing a trajectory
$[\textbf{x}_{0},\cdots,\textbf{x}_{N_{p}}]^{\intercal}$ by the following
quadratic constraint optimization:

$\displaystyle\min$ $\displaystyle Z=Z_{1}+Z_{2}+Z_{3}$ (11) s. t.
$\displaystyle\textbf{x}_{k+1}=f_{d}(\textbf{x}_{k},\textbf{u}_{k})$
$\displaystyle\textbf{u}_{\rm min}\leq\textbf{u}_{k}\leq\textbf{u}_{\rm max}$
$\displaystyle\mathbb{E}[\textit{r}(X)]\leq r_{threshold}$

where $Z$ is the cost function of the optimization problem, defined as a
combination of three penalty terms described below, and
$(\textbf{u}_{min},\textbf{u}_{max})$ are respectively the lower and upper
bounds of the control vector.

The first penalty term is used to prevent the robot from deviating too far
from the reference path $\mathcal{P}_{ref}$:

$Z_{1}=\sum_{k=0}^{N_{p}-1}(\textbf{x}_{k}-\textbf{x}_{k}^{r})^{\intercal}\textbf{Q}(\textbf{x}_{k}-\textbf{x}_{k}^{r}),$
(12)

where $\textbf{Q}\in\mathbb{R}^{3\times 3}$ is the weight matrix used to
penalize the state component parts. The second penalty term ensures that the
robot approaches to the farthest desired position:

$Z_{2}=(\textbf{x}_{N_{p}}-\textbf{x}_{N_{p}}^{r})^{\intercal}\textbf{Q}_{N}(\textbf{x}_{N_{p}}-\textbf{x}_{N_{p}}^{r}),$
(13)

where $\textbf{x}_{N_{p}}$ is the last predicted state, and
$\textbf{Q}_{N}\in\mathbb{R}^{3\times 3}$ is the weight matrix used to
penalize the final state component parts. The last penalty term is used to
prioritize a minimum traversal time:

$Z_{3}=\sum_{k=0}^{N_{p}-1}\text{w}_{v}(v_{k}-v_{max})^{2},$ (14)

where $v_{max}$ is the maximum velocity the robot can reach and
$\text{w}_{v}\in\mathbb{R}$ is the weighting coefficient that penalizes low
velocities. The final constraint $\mathbb{E}[\textit{r}(X)]\leq r_{threshold}$
forces the robot to find a trajectory that is safe to traverse. Therefore, any
trajectory that exceeds the risk threshold $r_{threshold}$ will be rejected.
Note that we do not add the risk to the cost function, as we would lose its
physical meaning. The risk is here defined as a hard constraint to ensure that
it is never exceeded. Finally, the optimal path $\mathcal{P}^{*}$ is the one
that minimizes the cost function in respects to the risk and the control
constraints.

##  VII VALIDATION OF THE FRAMEWORK

Figure 4: Example of an environment where an unexpected obstacle (speed bump)
occurred on the reference path (dashed green). We investigated three risk
thresholds, $0\text{\,}\mathrm{J}$ (left), $3\text{\,}\mathrm{J}$ (middle) and
$40\text{\,}\mathrm{J}$ (right). For each threshold, the path taken by the
robot is in blue. Its velocity and risk are depicted in purple and red.
Events: (1): Speed bump came into the robot’s perception field. (2): Robot
started to climb the speed bump. (3): Robot reached the speed bump top. (4):
Robot started to get down. (5): Robot reached the asphalt. The gray shaded
areas show the duration when the robot traversed the speed bump.

To show the applicability of this framework, we ran three scenarios in the
environment depicted in Figure 1, in which three different risk thresholds
were considered. We used real data for the perception part and simulated the
path planning with a simulation model of the robot seen in Figure 1. The
simulations were performed on a computer equipped with an 11th Gen Intel Core
i7-11850H processor and an NVIDIA GeForce RTX 3080 graphics card. In all
scenarios, the robot starts in front of the speed bump with zero steering
angle and zero speed. The Lambda-Field construction involves setting the
difference in elevation threshold $H_{\text{safe}}$ to $5\text{\,}\mathrm{cm}$
and the $e$ value to $1\text{\,}{\mathrm{cm}}^{2}$. Each Lambda-Field grid map
consist of $200\times 200$ cells, where each cell has a size of
$10\times$10\text{\,}\mathrm{cm}$$, resulting in a map of
$20\times$20\text{\,}\mathrm{m}$$. The simulated robot has a wheel radius $R$
of $25\text{\,}\mathrm{cm}$ and a mass $m$ of $50\text{\,}\mathrm{kg}$. The
stiffness $k_{r}$ of each robot wheel is set to
$150\,000\text{\,}\mathrm{N}\mathrm{/}\mathrm{m}$. The maximum velocity
$v_{\text{max}}$ of the robot is set to
$1.5\text{\,}\mathrm{m}\mathrm{/}\mathrm{s}$, while the steering angle range
$(\delta_{\text{min}},\delta_{\text{max}})$ is set to
$\pm$11\text{\,}\mathrm{\SIUnitSymbolDegree}$$. The sampling time $\Delta t$
is set to $100\text{\,}\mathrm{ms}$. The weight matrix Q is a diagonal matrix
of elements $[0.05,0.05,0.05]$. The weight matrix $\textbf{Q}_{N}$ is a
diagonal matrix of elements $[1,1,1]$. Therefore, if an unexpected obstacle
prevents the robot from tracking the reference path, but the reference goal
can be reached by a path deviating from the reference path, then the path
planner will be able to choose this path. Finally, the weight coefficient
$\text{w}_{v}$ is equal to $0.1$.

We show the results in Figure 4, where each column corresponds to the result
of a scenario. The first row in Figure 4 shows the path of the robot as well
as the reference path, which is a straight line passing through the speed
bump. The second row in Figure 4 shows the velocity profile and the risk
during the traversal. The graphs are labeled with numbers corresponding to
different times of interests: $t_{1}$: the speed bump is detected; $t_{2}$:
the robot ascend the speed bump; $t_{3}$: the robot reaches the top speed
bump; $t_{4}$: the robot descends the speed bump; and $t_{5}$: the robot comes
back to the road.

In the first scenario, the risk threshold is set to zero, meaning that the
robot is not allowed to take any risk. As long as the robot does not see the
speed bump, the reference path is tracked. At time $t_{1}$, the robot can no
longer track the reference path, as crossing the speed bump is risky. The path
planning formulation leads the robot to go around the speed bump. One can note
that throughout the traversal, the robot maximizes its velocity without ever
crossing a potential risky cell, as no risk is allowed. This scenario shows
that setting the risk to zero is the same as assessing the risk of collision
and imposing that the robot goes around the obstacles.

However, in some cases, avoiding may not be possible (for example, the left
lane is forbidden due to traffic rules or occupied by another vehicle) and if
no risk is allowed the robot would then stop in front of the speed bump. With
our approach, it is possible to reach the goal by managing the risk threshold
in accordance with the capability of the robot. In the scenario shown in the
second column in Figure 4, we set the risk threshold $r_{threshold}$ to
$3\text{\,}\mathrm{J}$, meaning in our case that $6\text{\,}\mathrm{mm}$
compression of the wheel is the maximum allowed. One can see that the robot
passes through the speed bump, but at a low speed. The robot slows down at
time $t_{1}$, as the speed bump is detected by the robot. Then, from time
$t_{2}$ to time $t_{5}$, our path planner regulates the speed of the robot to
stay below the risk threshold. The path planner maintains an adequate speed to
cross the speed bump safely, finding a balance between minimizing the
traversal time and the risks taken by the robot. From time $t_{3}$ to $t_{4}$,
the robot is on the speed bump. Before leaving it, the robot again detects
hazardous events on the reference path, leading the robot to slow down. At
time $t_{4}$, the robot starts to descend the speed bump at a reasonable speed
to manage the risk that has appeared on the path. Finally, at time $t_{5}$,
the robot leaves the obstacle, accelerating again. This scenario shows that
the robot can handle a consistent amount of risk in complicated situations.

In the last scenario, we increase the risk threshold to
$40\text{\,}\mathrm{J}$, meaning that we accept a $23\text{\,}\mathrm{mm}$
maximum tire’s compression. Intuitively, the robot goes faster than in the
previous scenario. At time $t_{1}$, the speed bump is detected. The planner
stabilizes the velocity of the robot to traverse the speed bump, while not
expecting a risk exceeding $40\text{\,}\mathrm{J}$. Until it reaches its
target, the robot tracks the reference path at its greatest allowable
velocity. As such, our framework is able to produce meaningful paths by
considering the potential hazards associated with navigating through 3D
obstacles.

##  VIII CONCLUSION

In this paper, we presented a method of risk-aware navigation in unknown 3D
environments. We propose a generalization of the work of [3], taking into
account the traversal of small obstacles such as road curbs or speed bumps. We
showed how to compute the associated Lambda-Field, using 3D lidar
measurements. The resulting map is used to evaluate the expected maximum
potential energy that the robot’s wheels will absorb in the event of a
collision along a given path. Finally, we presented a path planning
formulation that take the risk into account as a hard constraint. Using our
formulation, we showed that the risk function is well-fitted for risk-aware
navigation in urban environment. While being able to mimic a standard path
planning approach by setting the risk threshold to zero, our framework can
also generate a path going over obstacles if the risk is tolerable. This work
focused on explaining the theoretical framework and its applicability.

Future works will focus on improving the path planning part and demonstrating
the pertinence of the framework in larger-scale experiments. The exploration
of alternative risk metrics, such as CVaR, will be pursued to account for tail
events. We intend to extend our framework to enable the robot to perform long-
term missions. Indeed, providing mobile robots with such capabilities will
augment the autonomy of intelligent vehicles in rural environments. A global
path planner based on OpenStreetMap [19] will be included to the framework.
Furthermore, we will conduct extensive experiments on the framework,
incorporating quantitative evaluations. Finally, we will investigate the
possibility of adding several risks to further constrain the path-planning
algorithm, such as the risk of crossing a continuous lane marking.

## ACKNOWLEDGMENT

This work has been supported by the AURA Region and the European Union (FEDER)
through the LOG SSMI project of CPER 2020 MMaSyF challenge.

## References

  * [1] A. Elfes  “Using occupancy grids for mobile robot perception and navigation”  In _Computer_ 22.6, 1989, pp. 46–57  DOI: 10.1109/2.30720
  * [2] Yining Shi et al.  “Grid-Centric Traffic Scenario Perception for Autonomous Driving: A Comprehensive Review”  In _arXiv preprint arXiv:2303.01212_ , 2023  arXiv:2303.01212 [cs.CV]
  * [3] Johann Laconte et al.  “Lambda-Field: A Continuous Counterpart of the Bayesian Occupancy Grid for Risk Assessment”  In _2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)_ , 2019, pp. 167–172  DOI: 10.1109/IROS40897.2019.8968100
  * [4] B.K. Patle et al.  “A review: On path planning strategies for navigation of mobile robot”  In _Defence Technology_ 15.4, 2019, pp. 582–606  DOI: https://doi.org/10.1016/j.dt.2019.04.011
  * [5] Christophe Coué et al.  “Bayesian occupancy filtering for multitarget tracking: an automotive application”  In _The International Journal of Robotics Research_ 25.1 Sage Publications Sage CA: Thousand Oaks, CA, 2006, pp. 19–30
  * [6] Joachim Schroder, Tobias Gindele, Daniel Jagszent and Rudiger Dillmann  “Path planning for cognitive vehicles using risk maps”  In _2008 IEEE Intelligent Vehicles Symposium_ , 2008, pp. 1119–1124  DOI: 10.1109/IVS.2008.4621290
  * [7] Arvind A. Pereira et al.  “Toward risk aware mission planning for Autonomous Underwater Vehicles”  In _2011 IEEE/RSJ International Conference on Intelligent Robots and Systems_ , 2011, pp. 3147–3153  DOI: 10.1109/IROS.2011.6095157
  * [8] Stefano Primatesta, Giorgio Guglieri and Alessandro Rizzo  “A risk-aware path planning strategy for UAVs in urban environments”  In _Journal of Intelligent & Robotic Systems_ 95 Springer, 2019, pp. 629–643
  * [9] Anirudha Majumdar and Marco Pavone  “How should a robot assess risk? towards an axiomatic theory of risk in robotics”  In _Robotics Research: The 18th International Symposium ISRR_ , 2020, pp. 75–84  Springer
  * [10] R Tyrrell Rockafellar and Stanislav Uryasev  “Optimization of conditional value-at-risk”  In _Journal of risk_ 2 Citeseer, 2000, pp. 21–42
  * [11] R.Tyrrell Rockafellar and Stanislav Uryasev  “Conditional value-at-risk for general loss distributions”  In _Journal of Banking & Finance_ 26.7, 2002, pp. 1443–1471  DOI: https://doi.org/10.1016/S0378-4266(02)00271-6
  * [12] Astghik Hakobyan, Gyeong Chan Kim and Insoon Yang  “Risk-Aware Motion Planning and Control Using CVaR-Constrained Optimization”  In _IEEE Robotics and Automation Letters_ 4.4, 2019, pp. 3924–3931  DOI: 10.1109/LRA.2019.2929980
  * [13] David D Fan et al.  “Step: Stochastic traversability evaluation and planning for risk-aware off-road navigation”  In _arXiv preprint arXiv:2103.02828_ , 2021
  * [14] Xiaoyi Cai, Michael Everett, Jonathan Fink and Jonathan P. How  “Risk-Aware Off-Road Navigation via a Learned Speed Distribution Map”  In _2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)_ , 2022, pp. 2931–2937  DOI: 10.1109/IROS47612.2022.9982200
  * [15] Anton Koval, Samuel Karlsson and George Nikolakopoulos  “Experimental evaluation of autonomous map-based Spot navigation in confined environments”  In _Biomimetic Intelligence and Robotics_ 2.1, 2022, pp. 100035  DOI: https://doi.org/10.1016/j.birob.2022.100035
  * [16] Johann Laconte et al.  “A Novel Occupancy Mapping Framework for Risk-Aware Path Planning in Unstructured Environments”  In _Sensors_ 21.22, 2021  DOI: 10.3390/s21227562
  * [17] Johann Laconte et al.  “Dynamic Lambda-Field: A Counterpart of the Bayesian Occupancy Grid for Risk Assessment in Dynamic Environments”  In _2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)_ , 2021, pp. 4846–4853  DOI: 10.1109/IROS51168.2021.9636804
  * [18] Yiyuan Pan et al.  “GPU accelerated real-time traversability mapping”  In _2019 IEEE International Conference on Robotics and Biomimetics (ROBIO)_ , 2019, pp. 734–740  DOI: 10.1109/ROBIO49542.2019.8961816
  * [19] Mordechai Haklay and Patrick Weber  “OpenStreetMap: User-Generated Street Maps”  In _IEEE Pervasive Computing_ 7.4, 2008, pp. 12–18  DOI: 10.1109/MPRV.2008.80

  *[BOF]: Bayesian Occupancy Filter
  *[CVaR]: Conditional value-at-risk
  *[DEM]: Digital Elevation Map
  *[NMPC]: Nonlinear Model Predictive Control