# Autonomous Navigation for Robot-assisted Intraluminal and Endovascular
Procedures: A Systematic Review

Ameya Pore1,3,∗, Zhen Li2,5,∗, Diego Dall’Alba1, Albert Hernansanz3, Elena De
Momi2, Arianna Menciassi6, Alicia Casals3, Jenny Dankelman5, Paolo Fiorini1
and Emmanuel Vander Poorten4
1Department of Computer Science, University of Verona, Verona, Italy
2Department of Electronics, Information and Bioengineering, Politecnico di
Milano, Milan, Italy
3Center of Research in Biomedical Engineering, Universitat Politècnica de
Catalunya, Barcelona, Spain
4Department of Mechanical Engineering, KU Leuven, Leuven, Belgium
5Department of Biomechanical Engineering, Delft University of Technology,
Delft, Netherlands
6 The BioRobotics Institute, Scuola Superiore Sant’Anna, Pisa, Italy
∗Ameya Pore and Zhen Li contributed equally to this manuscript.
Corresponding author: Zhen Li (email<EMAIL_ADDRESS>
###### Abstract

Increased demand for less invasive procedures has accelerated the adoption of
Intraluminal Procedures (IP) and Endovascular Interventions (EI) performed
through body lumens and vessels. As navigation through lumens and vessels is
quite complex, interest grows to establish autonomous navigation techniques
for IP and EI for reaching the target area. Current research efforts are
directed toward increasing the Level of Autonomy (LoA) during the navigation
phase. One key ingredient for autonomous navigation is Motion Planning (MP)
techniques. This paper provides an overview of MP techniques categorizing them
based on LoA. Our analysis investigates advances for the different clinical
scenarios. Through a systematic literature analysis using the PRISMA method,
the study summarizes relevant works and investigates the clinical aim, LoA,
adopted MP techniques, and validation types. We identify the limitations of
the corresponding MP methods and provide directions to improve the robustness
of the algorithms in dynamic intraluminal environments. MP for IP and EI can
be classified into four subgroups: node, sampling, optimization, and learning-
based techniques, with a notable rise in learning-based approaches in recent
years. One of the review’s contributions is the identification of the limiting
factors in IP and EI robotic systems hindering higher levels of autonomous
navigation. In the future, navigation is bound to become more autonomous,
placing the clinician in a supervisory position to improve control precision
and reduce workload.

_Keywords_ Medical robotics $\cdot$ Intraluminal procedures $\cdot$
Endovascular interventions $\cdot$ Autonomy $\cdot$ Navigation $\cdot$
Continuum robots

##  1 Introduction

Figure 1: IPEI considered in this paper with respective standard
interventional tools used and clinical target sites. (a) Endovascular
catheterisation (b) Transanal colorectal procedures with a standard endoscope
(c) Transurethral and transvaginal access for prostate or bladder procedures
(d) Transoral procedures for airways or oesophagus (e) Transnasal procedure to
access bronchi. A primary difference between IP and EI is the sensing
modalities used, i.e., IP commonly use images from a camera as sensory input,
whereas EI mostly use X-Ray fluoroscopy.

Intraluminal Procedures (IP) and Endovascular Interventions (EI) are emerging
medical therapies that make use of body lumens and vessels to reach otherwise
difficult-to-reach regions deep into the body (Fig. 1). To enable these
procedures, snake-like flexible instruments are needed that can adapt to the
complex intraluminal and endovascular anatomy [1]. Intraluminal Procedures and
Endovascular Interventions (IPEI) have shown significant improvements in
patient outcomes, such as reduced blood loss, post-operative trauma, wound
site infection and recovery/hospitalisation time [2]. However, the flexible
tools used in IPEI have non-ergonomic designs. It is also difficult to control
these instruments precisely as a complex mapping between input and output
motion is present. This design limitation drastically increases the cognitive
and physical workload of the clinician. Overall, it is well-known that
clinicians undergo a long learning curve before becoming proficient in using
such highly dexterous instruments [3].

IPEI are composed of several complex tasks that must be performed in the right
order and following strict procedures. The first task (which may take a large
proportion of time) consists of carefully navigating to reach the targeted
area [4, 5]. A major challenge during this first navigation phase consists of
the complexity of operating in a deformable but constrained workspace with a
device that itself is quite compliant. The interventional instruments have to
traverse the anatomical passageways. While doing so, they constantly keep
contact with the lumen or vessels along at least a certain portion of their
body length [2]. Such contacts generally happen outside the field of view due
to restrictive perception of the endoluminal or endovascular tool architecture
[6]. Contacts may be dangerous, and their response is generally hard to
predict, especially when there is no direct sight of the local anatomy.
Moreover, the movement of the tools is hard to predict. Movement at the
proximal end may lead to no, limited or unexpectedly large movement of the
distal tip [7]. Here, friction, slack, and deformation of the instrument and
vascular or luminal wall prevent a desirable 1-to-1 relation between the
proximal and distal tip motion.

All these aspects make navigation in IPEI very challenging, and robotic
systems have been introduced to improve the current situation. However, the
introduction of robotic assistance has only partially reduced the procedure
complexity [8], due to non-intuitive mapping between user and robot motions,
limits on tool dexterity and poor shape sensing capabilities affecting
situational awareness [9]. It is believed that automation could provide
benefits to reduce clinicians’ workload while improving the overall outcome of
the procedure [8, 9, 10]. For instance, navigation assistance could minimise
path-related complications such as perforation, embolisation, and dissection
caused by excessive interaction forces between interventional tools and the
lumen or vessels. Furthermore, with the increasing demand for IPEI and the
limited number of experts [11], autonomous navigation will place clinicians in
a supervisory role requiring minimal and discontinuous intervention. It will
allow them to focus on high-level decisions rather than low-level execution.

An autonomy framework for robot-assisted Minimally Invasive Surgical (MIS) was
recently proposed with different Levels of Autonomy (LoA) based on robot
assistance, task automation, conditional autonomy, and high level autonomy
[12]. A detailed analysis of the framework mentioned above was carried out by
Haidegger et al. and Attanasio et al. where they map out technologies that
provide distinct features at different LoA for robot-assisted MIS [10, 9].
These studies use a top-down approach to define LoA based on general features
of robot-assisted MIS. Hence applying these levels for specific subtasks such
as navigation in IPEI is not trivial. A bottom-up granular approach is
required to define LoA, considering specific clinical phases. Therefore, this
article introduces a set of characteristic features essential for defining the
LoA for the IPEI navigation phase. Characteristic features refer to subtasks
associated with a specific clinical phase (i.e. IPEI navigation), such as
target localisation, motion planning and motion execution. These
characteristic features are used to define the LoA for the IPEI navigation.
The inclusion of autonomous features raises several ethical and regulatory
concerns due to incorrect robot behaviour. This article discusses recent
regulatory developments for high-risk applications, such as autonomous robotic
systems in IPEI.

One of the initial steps towards enabling autonomous navigation for IPEI is
through implementing Motion Planning (MP) techniques [13]. MP refers to
obtaining a path from a start to a goal configuration, respecting a collision-
free workspace.

It is a well-studied problem for rigid robotic manipulators [14]. Recent
studies have explored MP for flexible continuum robots with a large number of
degrees of freedom [15, 16]. However, there is a lack of an organised survey
of MP for IPEI and other biomedical applications using continuum robotic
systems. We consider the problem of a continuum robotic system operating in a
clustered and highly variable environment relevant to IPEI scenarios. Thus, we
conduct a survey of existing MP methods for IPEI, the associated challenges
and potential promising directions. Capsule robots are excluded from this
survey since they are generally used for imaging or drug delivery with limited
diagnostic capabilities. We consider IPEI robots with diagnostic capabilities,
a large proportion of which are continuum robots.

The contributions of this review article are, first, to identify the Levels of
Autonomy (LoA) for the IPEI navigation phase; second, to provide an overview
of existing MP methods that could enable autonomous navigation; and third, to
provide future directions towards autonomous navigation for IPEI. This paper
is structured as follows: Sec. 2 provides an overview of different IPEI
considered in this work, the challenges associated and the robotic systems
available. Sec. 3 describes the LoA for IPEI navigation and the recent
regulatory measures developed. Sec. 4 introduces the survey analysis for MP
methods. It presents the taxonomy and classification of MP algorithms for IPEI
procedures. Finally, the future development directions of IPEI navigation are
proposed in Sec. 5.

##  2 Robotic automation in IPEI

IP can be categorised into endoluminal and transluminal procedures [17, 6].
Endoluminal procedures involve interventions whereby the instruments move
through and stay in natural body orifices and lumens. In transluminal
procedures, instruments operate in body lumens. However, they also can create
incisions in lumen walls to access target sites beyond the lumen, such as
natural orifice transluminal endoscopic surgery. Examples of endoluminal
procedures include transoral interventions of the airways or oesophagus,
transanal access to the lower digestive tract, transnasal access to bronchi
and transurethral bladder and upper urinary tract procedures. Examples of
transluminal procedures include transgastric and transvaginal abdominal
procedures, transoesophagal thoracic and transanal mesorectal procedures (Fig.
1). In the context of this paper, we use IP as an inclusive term for referring
to both endoluminal and transluminal procedures. EI use a percutaneous
approach to reach target areas in the vasculature. Typical EI include aneurysm
repair, stent-graft, transcatheter aortic valve implantation, radio-frequency
ablation, mitral valve repair, etc [18]. While the technical innovation for
IPEI remains similar, EI are carried out typically using external image
guidance such as through X-Ray fluoroscopy or echography [17].

Some hospital units use consolidated robot-assisted MIS systems [19] for IPEI,
however a large proportion of robotic systems consists of continuum robots [6,
16]. Continuum robots are actuated structures that form curves with continuous
tangent vectors and are considered to have an infinite number of joints and
Degrees-of-Freedom (DoFs) [16, 2]. They have produced a step-change in medical
robotics as they offer better access and safer interactions making new
interventions possible. However, they are highly complex to model, sense and
control [2]. Current robotic solutions for IPEI in the research phase are
advancing the state-of-the-art through integrating new technologies that
enhance the ability to recognise and interact with tissues through increased
dexterity and sensory feedback [9]. These technological advances can help in
navigation guidance and building higher levels of autonomy. Some systems are
used in multiple procedures due to the lack of specific robotic technologies,
multi-functionality and the ability of robotic systems to adapt to different
IPEI procedures that share similar technical or clinical characteristics [20].
This section outlines the available robotic platforms for IPEI. Our study
considers endovascular interventions and transanal, transurethral,
transvaginal, transoral, and transnasal procedures target clinical
applications (See Fig. 2). We do not take into account procedures in which the
development of continuum robotic systems is in its infancy or where the
navigation phase does not constitute the predominant phase, such as auditory
canal access, transvascular interventions and exploratory procedures of the
lymphatic system.

###  2.1 Endovascular interventions

In a general endovascular intervention, cardiologists introduce a guidewire
through a small incision on the groin, the arm or the neck. The guidewire is
advanced to the desired location and acts as the stable track for the catheter
to follow. Two major challenges in controlling catheters and guidewires exist
in this procedure. One difficulty is steering guided through a 2D fluoroscopy
image [21, 22]. Hence, it requires a precise understanding of the 3D anatomy
projected in a 2D image plane. The other difficulty is steering the instrument
tip by combining insertion, retraction and torque actions at the proximal end
of the catheter and guidewire. These actions give rise to haptic feedback due
to friction and collision between the catheter and the vascular walls [23].
Robotic advancements in computer assistance, such as enhanced instrumentation,
imaging and navigation, have improved the current state of endovascular
procedures. In addition, robotic platforms provide controlled steering of the
catheter tip with improved stability. As a result, there is a growing interest
in teleoperated robotic catheterisation systems, which offer reduced radiation
exposure, increased precision, elimination of tremors and added operator
comfort.

Figure 2: Selection of some commercial robotic systems for IPEI. For
endovascular interventions: Corpath™ system (Corindus, Waltham, USA) and
Niobe™ system (Stereotaxis, St. Louis, USA), Sensei–Magellan (Hansen Medical,
Mountain View, USA) and Monarch system (Auris Health, Redwood, USA); For
transurethral and transvaginal procedures: Roboflex™ (ELMED, Ankara, Turkey)
and Sensei–Magellan (Hansen Medical, Mountain View, USA); For gastrointestinal
transanal procedures: Invendoscope™ (Invendo Medical, Weinheim, Germany) and
Aer-O-Scope (GI View Ltd, Ramat Gan, Israel); For transnasal procedures: da
Vinci (Intuitive Surgical, Sunnyvale, USA) and Flex® (Medrobotics, Raynham,
USA) For bronchoscopic transoral intervention: Monarch system (Auris Health,
Redwood, USA), ION™ (Intuitive Surgical, Sunnyvale, USA), da Vinci (Intuitive
Surgical, Sunnyvale, USA) and Flex® (Medrobotics, Raynham, USA) are used.

Recent developments in CorPath™ GRX (Corindus, Waltham, USA) provide guided
robotic control that allows clinicians to navigate endovascular tools through
a joystick. Other robotic catheter systems formerly introduced by Hansen
Medical (Mountain View, USA) and later acquired by J&J robotics (New
Brunswick, USA) are the Sensei™ X and Magellan platforms. Although used for
different cardiovascular applications, they are not commercially available
anymore. These platforms are considered in the article since they were
milestones in robotic systems for endovascular interventions [20]. Part of
this technology entered into the Monarch platform (Auris Health, Redwood,
USA), which targets bronchoscopy. The mechanically driven Amigo™ (Catheter
Robotics Inc. Budd Lake, USA) and the R-One™ (Robocath, Rouen, France) robotic
assistance platform allows steering standard catheters in 3 DoFs using an
intuitive remote controller that replicates the standard handle of a catheter.
The Niobe™ (Stereotaxis, St. Louis, USA) is a remote magnetic navigation
system in which a magnetic field guides the catheter tip. The tip deflection
is controlled by changing the orientation of outer magnets by utilising a
mouse or a joystick at the workstation. These robotic systems have reported
excellent intravascular navigation. However, the absence of haptic feedback
affects the procedural outcome when manoeuvring in smaller vessels like
coronary, cerebral and visceral vessels [24, 25].

###  2.2 Transanal IP

Transanal colonoscopy is a widely used method for the diagnosis and treatment
(screening and surveying) of colonic diseases such as Colorectal Cancer (CRC)
[26, 7]. In a standard colonoscopy procedure, an insertion tube is introduced
through the anus and pushed forward to inspect the colonic wall [27]. Early
detection and diagnosis of CRC lesions is essential for improving the overall
outcome of the patient [7, 27]. The rise in the number of colonoscopies has
increased the workload of endoscopists. However, not enough attention is given
to the ergonomic aspects of conventional colonoscopy. Several studies have
reported work-related musculoskeletal injuries of the hand, wrist, forearm and
shoulder among colonoscopists [28, 29]. Although colonoscopy-related adverse
events rarely occur, the proportion of subjects with risk factors is
increasing. Severe colonoscopic complications such as perforation and bleeding
can be fatal [30, 31]. Furthermore, even well-experienced endoscopists are
often limited by the lack of manoeuvrability, which can result in about 20% of
missed polyp localisation [32]. The rate of missed polyp detection varies by
the polyp type, often early-stage malignancies being difficult to detect [33].

Robotic Colonoscopy has been investigated to simplify the use of flexible
endoscopes, reducing the procedure time and improving the overall outcome of
the procedure [26]. Some cost-efficient solutions have shown advantages in
reducing pain, the need for sedation, and the possibility of being disposable
[7]. These platforms have a self-propelling semi-autonomous or teleoperated
navigation system. Several robotic colonoscopy platforms have received
clearance to enter the market. These include the NeoGuide Endoscopy System
(NeoGuide Endoscopy System Inc., Los Gatos, USA) [34], the Invendoscope™ E210
(Invendo Medical GmbH, Weinheim, Germany), the Aer-O-Scope System (GI View
Ltd., Ramat Gan, Israel)[35], the ColonoSight (Stryker GI Ltd., Haifa, Israel)
[36] and the Endotics System (ERA Endoscopy Srl, Pisa, Italy) [37]. The
NeoGuide Endoscopy system and the ColonoSight are no longer commercially
available. The Neoguide system is a cable-driven system that consists of 16
independent segments with 2DoFs each, position sensors at the tip to obtain
the insertion depth and real-time 3D mapping of the colon. Whereas the
Invendoscope™ E210 is a single-use, pressure-driven colonoscope that grows
from the tip using a double layer of an inverted sleeve, reducing the forces
applied to the colonic wall. The device has a working channel with
electrohydraulic actuation at the tip. The ColonoSight is composed of a
reusable endoscope wrapped with a disposable sheath to prevent infection. The
locomotion is provided by the air inflated inside the sleeve that covers an
inner tube. The tip consists of a bendable section with two working channels.
The Aer-O-Scope is a disposable self-steering and propelling endoscope that
uses electro-pneumatic actuation through two sealed balloons. Recent proof-of-
concept of the device showed successful caecum intubation with no need for
sedation [35]. The Endotic System uses a remotely controlled disposable
colonoscope that mimics inchworm locomotion.

###  2.3 Transurethral and transvaginal IP

Transurethral interventions have been used generally for bladder cancer
resection, radical prostatectomy, and partial cystectomy [38]. Transvaginal
access has been utilised for nephrectomy [39]. Both these interventions use an
endoscopic device to intentionally puncture a viscera (e.g. vagina, ureter and
urinary bladder) to access the abdominal cavity and perform intra-abdominal
operations [40]. There are considerable challenges that limit the widespread
adoption of transurethral and transvaginal access for urological applications,
such as the unmet need for dedicated specially designed instruments resulting
in lack of distal dexterity, limited tool accuracy, and limited depth
perception [41, 39]. These factors lead to the under-resection of tumours and
difficulty in enucleating tissue with minimal tilting of the rigid tools and
the urethral anatomy, motivating research in robot-assisted techniques [38].

In 2008, Robotic Flexible Ureteroscopy (fURS) was accomplished using the
Sensei–Magellan system (Hansen Medical, Mountain View, USA), which was
designed for cardiology and angiography [42]. Since 2010, ELMED (Ankara,
Turkey) developed the Roboflex™ Avicenna for fURS that directly drives the
endoscope and an arm enabling rotation by a joystick. Compared to traditional
flexible ureteroscopy, this system’s advantage lies in improved movement
precision and better ergonomics [43].

###  2.4 Transoral IP

Conventional Transoral Endoscopy (TOE) is the standard diagnostic method used
to examine the oesophagus, stomach, and proximal duodenum. In TOE, varying
lengths of flexible endoscopes are used, e.g. gastroscopes (925mm–1.1 m),
Duodenoscopes (approximately 1.25 m) and Enteroscopes (1.52- 2.2 m) [44]. The
diagnostic and therapeutic capabilities of TOE strongly correlate with the
technical and decision-making skills of the operator with a steep learning
curve [45]. Standard endoscopic surgical approach for laryngeal lesions uses
laryngoscope, microscope and laser [46]. This approach requires the surgeon to
work within the limits of the laryngoscope and gain line-of-sight observation
to complete the operation [46]. Transoral access is also used for bronchoscopy
to reach the lungs farther down the airways. Conventionally, a bronchoscope is
used for such procedures [47]. However, the average diagnostic yield remains
low because of limited local view in the peripheral airways [48].
Electromagnetic navigation was introduced to guide the bronchoscope through
the peripheral pulmonary lesions, but it lacked direct visualisation of the
airways, hence motivating the need for robotic assistance [49].

Available robotic systems for TOE includes the EASE system (EndoMaster Pte,
Singapore) and EndoSamurai™ (Olympus Medical Systems Corp., Tokyo, Japan). The
EASE system is based on a teleoperated device that remotely controls the
endoscopic medical arms. The EndoSamurai™ system consists of instruments
mounted at the end of the endoscope for submucosal dissection procedures. Some
other robotic systems in an early development phase are reviewed in [45].

Commercially available systems for laryngeal procedures are the da Vinci
Robotic System (Intuitive Surgical, Sunnyvale, USA) and the Flex Robotic
System (Medrobotics, Raynham, USA) [19]. The Flex robotic system includes a
rigid endoscope controlled through a computer interface, with two external
channels for flexible instruments.

In robotic bronchoscopy, Monarch™ (Auris Health Inc, Redwood, USA) is
pioneering robotic endoscopy. The platform consists of an outer sheath, an
inner bronchoscope with 4DoF steering control, electromagnetic navigation
guidance and continuous peripheral visualisation [49]. Another robotic
platform called ION™ Endoluminal System by Intuitive Surgical includes an
articulated, flexible catheter with shape sensing capabilities, which provides
positional and shape feedback along with a video probe for live visualisation
while driving the catheter. [49].

###  2.5 Transnasal IP

Systems for the transnasal procedure have been investigated with several
exploratories in mind. These procedures, ranked according to the distance to
the target from the entry point include transnasal navigation for sinuses,
transnasal skull base procedure, and transnasal micro-procedure of the upper
airways.

One of the challenges with diseases of the sinuses lies in the difficulty of
monitoring their progression, obtaining a biopsy, and facilitating
intervention in the frontal and maxillary sinuses while avoiding visible
scarring or obliteration of bone scaffolds of the nose. Conventionally, a
flexible endoscope is used in clinical practice [50]. Skull base surgeries are
carried out through transnasal access. A typical target for these surgeries is
the removal of pituitary gland tumours through a transsphenoidal approach [51,
52]. The standard endoscopic approach for these surgeries is limited by
restricted access, cumbersome manual manipulation of interventional tools near
susceptible anatomy and lack of distal dexterity [53].

Another interventional target using transnasal access is the upper airways and
throat [54]. Transnasal Endoscopy (TNE) is performed using an ultrathin
endoscope with a shaft diameter of $6\text{\,}\mathrm{mm}$ which is inserted
through the nasal passage. Once the instrument is beyond the upper oesophageal
sphincter, endoscopy is conducted in the standard fashion. However, there are
some technical limitations of TNE, namely, a smaller working channel can
result in limited suction and the availability of fewer endoscopic
accessories.

In general practice, the robotic systems mentioned in transoral approaches
such as da Vinci Robotic System (Intuitive Surgical, Sunnyvale, USA) and Flex®
Robotic System (Medrobotics Corp., Raynham, USA) are also used in transnasal
interventions [19]. The Flex® Robotic System is an operator-controlled
flexible endoscope system primarily designed for an Ear-Nose-Throat procedure
that includes a steerable endoscope and computer-assisted controllers, with
two external channels for the use of compatible $3.5\text{\,}\mathrm{mm}$
flexible instruments. However, specific robotic systems with appropriate
ergonomics and dimensions suited for transnasal passage are still under
development [19].

Table 1: Descriptive classification of LoA for IPEI navigation. H: Performed by a human operator, M: Performed by a machine. H/M: Performed by a human, assisted by a machine, M/H: Performed by a machine, assisted by a human. M1: Performed under human supervision. LoA |  Description |  Target localisation |  Motion planning |  Execution & re-planning
---|---|---|---|---
0 |  Direct robot control: The clinician exclusively controls all cognitive functions without any support or assistance [12]. Most IPEI systems used in clinical practice operate at Level-0 autonomy. |  H |  H |  H
1 |  Navigation assistant: The human operator maintains continuous control of the robotic navigation intraoperatively; however, it is assisted robotically during the execution of the motion. Other cognitive functions are carried out manually. |  H |  H or M |  H/M
2 |  Navigation using waypoints: The operator provides discrete high-level navigation tasks such as waypoints or predefined trajectories. These trajectories are derived during preoperative planning. The robot carries out the required motion between the waypoints during the execution time, with the clinician in a supervisory role to approve or override the strategy. |  H |  M/H |  M1 or M/H
3 |  Semi-autonomous navigation: The final goal of navigation is provided by a human operator, and the system generates the strategies required to carry out the complete navigation task. During the execution time, it relies on the operator’s supervision to approve or override the choice. In IPEI navigation, the robot would extract waypoints and then plan the trajectory to reach the point. |  H |  M |  M1
4 |  High-level autonomous navigation: This level is characterised by the ability of the system to make clinical decisions and execute the control solution under the clinician’s supervision. The system should interpret preoperative imaging modalities such as CT, MRI and ultrasound to detect target regions and extract all the information required for proper navigation. |  M |  M |  M1

##  3 Levels of Autonomy (LoA)

One of the promising features of upcoming IPEI robotic systems is autonomy
since it provides the ability to perceive, analyse, plan and take actions
automatically [55]. An autonomous robotic system can deal with non-programmed
situations and has the capability of self-management and self-guidance [56].
The most notable aspect of autonomy is the transfer of decision-making from a
human operator to a robotic system. To allow this transfer, two conditions
must be met [57]. First, the operator must transfer the control to the robotic
system, including the related responsibilities (i.e., the human operator must
“trust" the autonomous system). Second, the system must be certified, i.e., it
must fulfil all ethical, legal and certification requirements. However, these
certification standards are not fully developed for medical robotic systems
due to a lack of consideration, and clear understanding of autonomy [58].
Therefore, we first introduce the ethical and regulatory aspects related to
autonomy in Sec. 3.1, then we define generic LoA in Sec. 3.2, while in Sec.
3.3, we present the specific LoA for IPEI navigation systems.

###  3.1 Ethical and regulatory aspects of autonomy

The ethical concerns can be addressed from multiple perspectives, including
human rights, law, economics, policy and ethics [59]. We highlight the
viewpoints of medical robot practitioners. When transferring the decisions
from a human operator to an autonomous system, one of the main ethical
concerns is the consequences of errors resulting from the decisions taken.
These errors can be due to incorrect robot behaviours, leading to hazardous
situations [60]. Hence, robot-assisted intervention is considered a high-risk
category [61]. To address the ethical concerns, the European Commission
proposed a regulatory framework for AI applications in the high-risk category,
known as the Artificial Intelligence Act (AI Act) [61]. Similar efforts are
being formalised in the United States under the National AI Initiative [62].
The AI Act describes the role of a human operator: the obligation to provide
human supervision, the right for a human to override an automated decision,
and the right to obtain human intervention which forbids full autonomy.
Therefore, human intervention needs to be carefully designed into the system
at different levels of integration [63]. Furthermore, the reliability of
medical robotics is associated with the notion of certification, which
requires legal approval that the system has reached a particular standard.
Several regulatory standards exist in the robotics domain; for instance, the
standards for medical electrical systems are defined by the International
Electrotechnical Commission (IEC) in the Technical Report (TR) 60601-4-1 [64],
which guides risk management, basic safety and essential performance towards
systems with some degrees of autonomy. These regulatory standards are not
fully developed for robot-assisted intervention, and the introduction of LoA
could support this development by facilitating the system verification and
validation with improved risk management [58]. As a consequence of upcoming
regulations like the AI Act, it is expected that earlier phases of the design
process will progressively consider safety and system integration concerns.

###  3.2 Levels of Autonomy: Definition

Quantifying system autonomy based on its capabilities presents a significant
challenge due to different levels of advances in underlying technologies. In
medical robotics, as the autonomous capabilities of the robot are increasing,
the role of medical specialist is shifting from manual dexterity and
interventional skills towards diagnosing and high-level decision-making.

Prior work identified five LoA for medical robot systems considering a
complete clinical procedure and the capability of a human operator/clinician
[12, 10]. At level 0, the robot has no decision autonomy, and the clinician
controls all aspects of the system. i.e., the clinician exclusively controls
it. At level 1, the robot can assist the clinician, while at level 2, it can
autonomously perform an interventional subtask. At level 3, the robot can
autonomously perform longer segments of the clinical procedure while making
low-level cognitive decisions. Finally, at level 4, the robotic system
executes the complete procedure based on human-approved clinical plans or
surgical workflow. Few studies have defined level 5, which refers to full
autonomy in which the robotic clinician can perform the entire procedure
better than the human operator; hence human approval is not required [12, 10].
However, level 5 is still in the realm of science fiction, so we consider it
outside the scope of this article. In higher LoA, the robot responding to
various sensory data will be highly sophisticated while it could replicate the
sensorimotor skills of an expert clinician more closely.

Attanasio et al. [9] outlined the enabling technologies and the practical
applications for different levels. Haidegger et al. provided a top-down
classification of LoA for general robot-assisted MIS [10]. Their
classification considers four robot cognitive functions (i.e., generate,
execute, select and monitor options), where the overall LoA is the normed sum
of the four system functions assessed on a linear scale, “0" meaning fully
manual and “1" fully autonomous.

In clinical practice, an interventional procedure workflow is decomposed into
several granular levels, such as phase, steps, and gestures [65]. Many of the
interventional phases and skills that are used in robot-assisted MIS are not
considered in IPEI, e.g., luminal navigation. Hence, LoA defined for robot-
assisted MIS can not be directly applied for IPEI. Moreover, using the
proposition provided by Haidegger et al., it is challenging to identify a
clear boundary between human and automated control required for specific
phases/steps of robot-assisted MIS. It introduces an additional problem of
defining the system’s overall level that implements different LoA for
different phases of the procedure. Hence, we propose a bottom-up solution
where an intermediate LoA is defined for specific interventional phases.
Having knowledge of a subtask will enable a better understanding of the amount
of human intervention required at a granular scale. A bottom-up classification
would better estimate the overall system autonomy since underlying phases can
be at a different intermediate LoA. Moreover, it can be applied to all medical
procedures, from robot-assisted MIS to IPEI. The target of this article is
IPEI navigation; hence we define the intermediate LoA for this interventional
phase.

###  3.3 LoA for IPEI

LoA for robot-assisted MIS has been derived from the degree of autonomy
introduced by ISO, who, jointly with IEC, created a technical report (IEC/TR
60601-4-1) [64] to propose an initial standardisation of autonomy levels in
medical robotics. The report parameterises DoFs along a system’s four
cognition-related functions: generate, execute, monitor and select options
strategy. A similar classification approach has been followed by Haidegger et
al. for robot-assisted MIS. We identify three specific cognitive functions for
an IPEI navigation task: 1) Target localisation, 2) Motion planning, and 3)
Execution and replanning. Target localisation is usually based on preoperative
images, such as Computed Tomography (CT), Magnetic Resonance Imaging (MRI) or
X-Ray imaging. It is a critical feature, as inaccurate target identification
can lead to inaccuracies in the subsequent steps. MP can be considered in two
phases: preoperative and intraoperative. Preoperative MP refers to the
planning performed before the procedure based on multi-modal medical images
[66]. This may be done in static virtual models of the lumen or vessels.
Execution and replanning is an intraoperative phase to carry out the required
motion to reach the target while continuously replanning intraoperatively. It
can include target relocalisation when adjustment is needed due to unexpected
situations.

Table. 1 illustrates the LoAs defined for IPEI navigation. In LoA 0, all the
features from target localisation, MP and motion execution are carried out by
a human operator. Commercially available robotic system (as described in Sec.
II) can be considered in this category since the human operator has complete
control of the robotic motion. LoA 1 is characterised by target localisation
and preoperative planning manually carried out by the clinician. The clinician
executes the actual motion with the assistance of the robotic system. Systems
that use external tracking devices and registration methods to align the
preoperative data with the intraoperative condition and support the clinician
in executing a clinical procedure can be considered LoA 1 [67, 68, 69].
Taddese et al. developed a teleoperated magnetically controlled endoscope,
where the system provides navigation assistance by controlling the magnetic
field [70]. These systems represent the first implementations of LoA 1, where
the manipulator executes the command imparted by the operator. In LoA 2, the
robotic system fully controls the specific navigation steps. Target
localisation is carried out by the clinician, who provides input in the form
of waypoints or demonstration trajectories. The path planner uses these cues
to generate a global trajectory. Further, the robotic system carries out the
required motion indicated by the path planner. During execution, the human
operator supervises the autonomous navigation and approves the robot’s actions
or overrides it (to comply with AI Act indications). In LoA 3, after target
localisation by the clinician, the path planner generates the global path in
the preoperative phase without any manual intervention. This level includes
automatically splitting the entire navigation task into specific subtasks that
could be performed autonomously. The robotic system executes the motion
indicated by the path planner and adapts to environmental changes through
real-time replanning. The local real-time knowledge will provide information
regarding the anatomical environment, and the motion will be adjusted as the
autonomously steering is performed. All the features from target localisation,
MP and execution are autonomously carried out without any human intervention
by a system reaching LoA 4. The main difference between LoA 3 and LoA 4 is the
addition of automatic target identification. This additional feature requires
enabling technologies such as autonomous segmentation of organs to detect
abnormal tissues such as polyps, automatic localisation and shape sensing
mechanisms [8] (Sec. 5.2). Fig. 3 shows a case study of LoA for the transanal
IP. In the next section, the proposed LoA will be used to classify all the
work considered in the field of IPEI navigation.

Figure 3: Case study of LoA for endoscopic navigation for transanal IP. The
complete navigation task is divided into three cognitive functions: target
localisation through preoperative imaging, planning the motion preoperatively
and executing the motion. (Row 1) Target localisation using preoperative
images: The identified target is depicted with a red circle. (Row 2)
Preoperative MP: Path representation inside the colon shown with a yellow
line. (Row 3) Intraoperative motion execution and MP: Intraoperative
endoscopic visualisation. (left to right). LoA0-LoA4 respectively. For each
level, we indicate the agent that operates each cognitive function. Agent
refers to either a human operator, path-planning system or robotic
manipulator. In the case of two agents, the supervisor agent is depicted on
the right side, while the main agent executing the actions is on the left and
its icon is larger.

##  4 Systematic review of MP for IPEI

###  4.1 Literature review

A systematic analysis was conducted, following the PRISMA methodology [71], to
survey the developments of automation and MP in IPEI.

####  4.1.1 Search method

A systematic analysis was conducted using the following digital libraries:
Google Scholar, Scopus and IEEE Xplore. Search queries were programmatically
generated from the search term matrix. Query results were automatically
retrieved and checked for duplicates via the Scopus API. The list of
references was saved as a .csv file and manually evaluated according to the
inclusion criteria. All items that did not meet the inclusion criteria were
excluded. The search terms used in this survey were chosen by generalising the
term “motion planning for intervention".

Search terms are combined with the logical operators AND and OR such that a
large search space can be covered in sufficient detail. Fig. 4 provides an
overview of all the search terms and the flow of the conducted review. This
matrix, once all possible combinations have been exhausted, yields 520
entries. To automatically manage all the generated entries and remove the
duplicates, a python library, pybliometrics, was used [72]. The cutoff date
for the earliest work included is 2005, and the latest work is from July 2022.

####  4.1.2 Selection criteria

The paper was selected by:

  1. (i)

considering only continuum robots (excluding capsule mobile robots [73, 74])
for IPEI;

  2. (ii)

excluding low-level controller studies based on force control, position
control, impedance control and similar;

  3. (iii)

considering only full papers drafted in English. Extended abstracts reporting
preliminary findings were omitted;

  4. (iv)

excluding transluminal procedures that require incisions such as hydrocephalus
ventricles.

Figure 4: (a) Search matrix used for the survey (b) Prisma flow diagram
summarizing how the systematic review was conducted.

####  4.1.3 Post processing and analysis

The search script returned 11404 references. Prisma flow diagram in Fig. 4b
summarises how the systematic review was conducted. 10833 references and 515
references were excluded after title check and abstract check, respectively.
Additional 9 references were included manually because search results did not
cover 100% of the current studies for different reasons. Finally, this process
yielded a list of 65 references.

The outcomes of various studies were classified based on several criteria,
shown in Fig. 5, including the targeted procedure, the LoA, the MP method, the
validation, and the environment’s dynamics. The MP methods are categorised
into subgroups presented in Fig. 6 for an in-depth analysis. The summary of
the state-of-the-art on IPEI MP publications are presented in Table
LABEL:tab:reflist, and its development is shown in Fig. 7a. Besides the MP
approach, we have highlighted the distribution of the targeted IPEI procedures
in Fig. 7b. Moreover, Table LABEL:tab:reflist shows that some studies involved
intraoperative path replanning with a dynamic environment (last column).

Figure 5: Schematics of the analysis carried out for each paper. These
criteria include the targeted procedure, the LoA, the MP method, the
validation and the dynamics of the environment.

###  4.2 Taxonomy on MP for IPEI navigation

MP has been a well-documented field for navigation tasks since the 1980s,
supporting robotic manipulators and mobile platform operations in indoor and
outdoor industrial applications. During MP, robot characteristics are usually
considered to find a feasible path solution, such as its geometrical
dimensions to avoid collisions and kinematic constraints to respect its
movement capability. The robot kinematics describes the relationship between
the configuration and task spaces [75]. The configuration space $\mathcal{C}$
is defined as all robot configurations. The task space $\mathcal{T}$ is
referred to as the workspace that the robot can reach for each specific
configuration $\mathbf{q}$. The robot kinematics can be expressed in a general
form as

$\mathcal{T}=f(\mathbf{q})\quad\mathbf{q}\in\mathcal{C}$ (1)

MP is an essential component of autonomous IPEI robotic systems, even under
complex operating conditions and stringent safety constraints. As shown in
Fig. 6, MP methods can be decomposed into four sub-groups by adjusting the
taxonomy of path planning in general robots from [76]: node, sampling,
optimisation and learning-based techniques. The node-based (or graph-based)
algorithms use a graph-searching strategy along with a tree structure. The
sampling-based algorithms construct a tree structure based on random samples
in a configuration space. Therefore these methods find a collision-free path
and ensure compatibility with the robot’s motion capabilities. Optimisation-
based algorithms formulate the MP problem as a mathematical problem by
minimising or maximising an objective function with respect to some
constraints and obtaining the optimal case through a solver. Learning-based
methods use a Markov decision process to learn a goal-directed policy based on
a reward function. A brief general definition of different MP methods is
provided in Table. LABEL:tab:env, while Table III summarizes different MP
works for IPEI applications.

Table 2: Background of path-planning methods. No. |  Path Planning |  Description
---|---|---
1. |  Node-based |
a. |  Centerline-based Structure (CBS) |  This method is long-established to keep the tip of the instruments away from the walls [77]. A tree structure is built from the anatomical information of the lumen, where each node contains the information of the lumen centerline position and the corresponding lumen radius.
b. |  Depth First Search (DFS) |  DFS algorithm traverses a graph by exploring as far as possible along each branch before backtracking [78]
c. |  Breadth First Search (BFS) |  BFS algorithm [79] starts at the tree root and explores the k-nearest neighbor nodes at the present depth before moving on to the nodes at the next depth level.
d. |  Dijkstra |  The Dijkstra algorithm [80] is an algorithm for finding the shortest paths between nodes in a graph. It is also called Shortest Path First (SPF) algorithm. The Dijkstra algorithm explores a graph by expanding the node with minimal cost.
e. |  Potential field |  Artificial potential field algorithms [81] define a potential field in free space and treat the robot as a particle that reacts to forces due to these fields. The potential function is composed of an attractive and repulsive force, representing the different influences from the target and obstacles, respectively.
f. |  A* & Lifelong Planning A* (LPA*) |  A-star [82] is an extension of the Dijkstra algorithm, which reduces the total number of states by introducing heuristic information that estimates the cost from the current state to the goal state.
g. |  Wall-following |  Wall-following algorithms move parallel and keep a certain distance from the wall according to the feedback received from sensors.
2. |  Sampling-based |
a. |  Rapidly-exploring Random Tree (RRT) |  RRT [83] and its derivatives are widely used sampling-based methods. These methods randomly sample in the configuration space or workspace to generate new tree vertices and connect the collision-free vertices as tree edges. In addition, these methods can consider the kinematic constraints (i.e., curvature limitations) during MP.
b. |  Probabilistic RoadMap* (PRM*) |  A probabilistic roadmap is a network graph of possible paths in a given map based on free and occupied spaces [84, 85]. PRM* takes random samples from the robot’s configuration space, tests them for whether they are in the free space, and uses a local planner to attempt to connect these configurations to other nearby configurations. Then, the starting and goal configurations are added in, and a graph search algorithm is applied to the resulting graph to determine a path between these two configurations.
3. |  Optimization-based |
a. |  Mathematical Model |  MP can be formulated as a path optimization problem with constraints on the robot model, such as its kinematic model [86].
b. |  Evolutionary algorithms |  Evolutionary algorithms use bio-inspiration to find approximate solutions to difficult optimization problems. [86]. Ant Colony Optimization (ACO) is one of the population-based metaheuristic algorithms [87]. Artificial ants incrementally build solutions biased by a pheromone model, i.e. a set of parameters associated with graph components (either nodes or edges) whose values are modified at runtime by the ants.
4. |  Learning-based |
a. |  Learning from Demonstrations (LfD) |  LfD is the paradigm where an agent acquires new skills by learning to imitate an expert. LfD approach is compelling when ideal behavior cannot be easily scripted, nor defined easily as an optimization problem, but can be demonstrated [88].
b. |  Reinforcement Learning (RL) |  In RL, an agent learns to maximise a specific reward signal through trial and error interaction with the environment by taking actions and observing the reward [89].

####  4.2.1 Node-based algorithms

Node-based algorithms use an information structure to represent the
environment map and are commonly used for navigation assistance [76]. Table
LABEL:tab:reflist shows different MP works for IPEI that exploit node-based
methods. As schematised in Fig. 6, algorithms that have been adopted here are
Centerline-based Structure (CBS), Depth First Search (DFS), Breadth First
Search (BFS), Dijkstra, potential field, A*, Lifelong Planning A* (LPA*), and
wall-following.

Figure 6: Classification of IPEI MP methods for continuum robots found in
literature

##### Centerline-based Structure

Geiger et al. extracts the 3D skeleton for bronchoscopy planning by computing
the skeleton of the segmented structure and then converting this skeleton into
a hierarchical tree model of connected branches [90]. Sánchez et al. [91]
obtains the skeleton of the bronchial anatomy via the fast marching method
firstly and then defines the skeleton branching points as a binary tree
(B-tree). Sánchez’ study gives a path corresponding to a sequence of nodes
traversing the B-tree. Intraoperatively, a geometry likelihood map is used to
match the current exploration to the path planned preoperatively. The airway
centerlines serve as the natural pathways for navigating through the airway
tree. They are represented by a discrete set of airway branches in [67].
Starting with each target Region of Interest (ROI) associated airway route,
the method from Khare et al. [67] automatically derives a navigation plan that
consists of natural bronchoscope manoeuvres abiding by the rotate-bend-advance
paradigm learned by physicians during their training. This work is evaluated
both in phantoms and in a human study.

Wang et al. developed a method to build a navigation information tree based on
the vasculature’s centerline for catheterisation [92]. The authors made a tree
structure assuming the vascular system was rigid and interrogated the tree to
find the nearest node during intraoperative navigation. The navigation
experiments were carried out on a resin vessel phantom. Another study proposed
a 3D vasculature’s centerline extraction approach via a Voronoi diagram [93].
It treated the centerlines as the minimal action paths on the Voronoi diagrams
inside the vascular model surface. The experimental results show that the
approach can extract the centerlines of the vessel model. Further Zheng et al.
[68] firstly proposed to extract the preoperative 3D skeleton via a parallel
thinning algorithm for medical axis extraction [94]. Secondly, they proposed
to use a graph matching method to establish the correspondence between the 3D
preoperative and 2D intraoperative skeletons, extracted from 2D intraoperative
fluoroscopic images. However, the proposed graph matching is sensitive to
topology variance and transformation in the sagittal and transverse planes. A
recent study on transnasal exploration proposed central path extraction
algorithm based on pre-planning for the roaming area [95].

Nevertheless, a common disadvantage of work available in the literature
describing this approach is that they focus on constructing an information
structure, but path exploration inside the information structure is not
mentioned [90, 91, 67, 92, 93, 68]. Specifically, the tree structure is built,
but the path solution is not generated autonomously through a graph search
strategy, especially when there are multiple path solutions simultaneously.

##### Depth First Search

As an extended method to travel the tree formed in [67], the studies by Zang
et al. implement a route search strategy of DFS for an integrated
endobronchial ultrasound bronchoscope, exploring a graph by expanding the most
promising node along the depth [69], [96]. In another study by Gibbs et al., a
DFS to view sites is regarded as the first phase search, followed by a second
search focusing on a ROI localisation phase and a final refinement to adjust
the viewing directions of the bronchoscope [97]. A DFS approach is also
developed in Huang et al. for endovascular interventions [98]. Instead of
considering path length as node weights in the typical DFS approach, this work
defines the node weights as an experience value set by doctors.

The search time and the planned path are significantly dependent on the order
of nodes in that same graph layer. Even though a DFS approach can search for a
feasible path by first exploring the graph along with the depth, it does not
ensure that the first path found is the optimal path.

##### Breadth First Search

The BFS algorithm was employed in [99] for a magnetically-actuated catheter to
find a path reaching the target along vascular centerline points. However, the
BFS algorithm would take much more time to find a solution in a complex
vascular environment with multi-branches.

##### Dijkstra

A graph structure based on vasculature’s centerlines that are determined using
a volume growing and a wavefront technique is designed by Schafer et al. in
[100]. The optimal path is then determined using the shortest path algorithms
from Dijkstra. However, Schafer et al. assume that the centerline points are
input as an ordered set, which would be a strict assumption. Moreover, they
only report the scenario of a single lumen without branches, which does not
reflect the advantages of the Dijkstra algorithm. A similar method but in a
backward direction is presented by Egger et al. [101]. This work determines an
initial path by Dijkstra. Users define initial and destination points. After
that, the initial path is aligned with the blood vessel, resulting in the
vasculature’s centerline. However, this methodology is not fully autonomous,
and it involves manually tuned parameters. Another work extracts the
centerline and places a series of guiding circular workspaces along the
navigation path that are perpendicular to the path [102]. The circular planes
jointly form a safe cylindrical path from the start to the target. The
Dijkstra algorithm is implemented to find the minimal cumulative cost set of
voxels within the airway tree for bronchoscope navigation [103, 104] and find
the shortest path along vasculature’s centerlines [105], [106, 107].

Compared to DFS, Dijkstra keeps tracking and checking the cost until it
reaches the target. So there is a higher possibility of getting a better
solution. Nevertheless, these researches still focus on tracking anatomical
centerlines that are difficult to follow precisely and often not desirable.
Because aligning the instrument tip with the centerline may call for excessive
forces at more proximal points along the instrument’s body where contact with
the anatomy occurs.

Figure 7: Chronological development of endoluminal navigation. (a) MP
approaches (b) The targeted IPEI procedures. Until 2010, the majority of
studies have implemented node-based and sampling algorithms for MP. While
lately, with the exponential increase in computational resources, the field is
transitioning towards learning-based methods.

##### Potential field

The work by Rosell et al. [108] computes the potential field over grids based
on the L1 distance to obstacles. It is used to search a path by wavefront
propagation for bronchoscopy. Rosell’s approach considers the geometry and
kinematic constraints while selecting the best motion according to a cost
function. Yang et al. [109] extract centerlines via a distance field method,
establish and navigate the tree after that. However, the authors only
considered the curvature constraint at 180∘ turns along vasculature’s
centerlines and assumed that all the path points have the same Y coordinate.
Martin et al. [110] employ a potential field approach by defining an
attractive force from the endoluminal image centre mass to the colon centre
mass. A linear translation between the colon centre mass and the image centre
is reconstructed and regarded as the linear motion of the colonoscope tip.
This work is validated both in the synthetic colon and pig colon (in-vivo). A
similar approach is followed by Zhang et al. where a robotic endoscope
platform is employed to bring surgical instruments at the target site [111].
Girerd et al. [112] use a 3D point cloud representation of a tubular structure
and compute a repulsive force to ensure that the concentric tube needle tip
remains inside the contour.

The Potential field has an advantage in local planning by maintaining the
centre of the image close to the centre of the cross-section of the lumen or
the vessels. Nevertheless, it only considers a short-term benefit rather than
global optimality during this local planning and might get stuck in a local
minimum during global path planning.

##### A* and Lifelong Planning A*

He et al. [113] compute and optimise endoscopic paths using the A* algorithm.
The effectiveness of the preoperatively planned path is verified by an
automatic virtual nasal endoscopy browsing experiment. Ciobirca et al. search
shortest airway paths through voxels of a bronchus model using the A*
algorithm [114]. They claimed that this method could potentially improve the
diagnostic success rate with a system for tracking the bronchoscope during a
real procedure. However, this statement has not been validated yet. Some
studies proposed a path planning method for Concentric Tube Robots (CTRs) in
brain surgery. The authors of these studies build a nearest-neighbour graph
and use LPA* algorithm for efficient replanning to optimise the insertion pose
[115, 116]. Compared to A*, LPA* [117] can reuse information from previous
searches to accelerate future ones. Ravigopal et al. proposed a modified
hybrid A* search algorithm to navigate a tendon-actuated coaxially aligned
steerable guidewire robot along a pre-computed path in 2D vasculature phantoms
under C-arm fluoroscopic guidance [118]. Recently, Huang et al. showed colon
navigation using a real-time heuristic searching method, called Learning real-
time A* (LRTA*) [119]. LRTA* with designed directional heuristic evaluation
shows efficient performance in colon exploration compared to BFS and DFS.
Directional biasing avoids the need for unnecessary searches by constraining
the next state based on local trends.

A* and LPA* use heuristic information to reach the goal. The first is commonly
used for static environments, while the second can adapt to changes in the
environment. They can converge very fast while ensuring optimality because
both the cost from the start and the cost to the goal are taken into account.
But their execution performance depends on the accuracy of the heuristic
information. If inaccurate heuristic information is employed, searching in
non-optimal directions severely affects its performance.

##### Wall-following

The study in [120] uses a wall-following algorithm to assist catheter
navigation. Fagogenis et al. [120] employ haptic vision to accomplish wall-
following inside the blood-filled heart for a catheter. The wall-following
algorithm could be considered an efficient navigation approach if there are
few feasible routes to reach the target state. Otherwise, the solution of a
wall-following algorithm cannot ensure optimality.

####  4.2.2 Sampling-based algorithms

As observable in Table LABEL:tab:reflist, different works, in the context of
MP for IPEI, exploit sampling-based methods. As schematised in Fig. 6,
algorithms based on Rapidly-exploring Random Tree (RRT) and its variants and
Probabilistic RoadMap* (PRM*) have been proposed.

##### Rapidly-exploring Random Tree and its variants

Some studies compare several RRT-based algorithms looking for the optimal
option for the virtual bronchoscopy simulator, such as RRT, RRT-connect,
dynamic-domain RRT, and RRT-Connect with dynamic-domain [121, 122]. Results
reveal that the RRT-Connect with Dynamic Domain is the optimal method
requiring the minimum number of samples and computational time for finding the
solution path. Fellmann et al. use a collision-free path via RRT as a baseline
[123]. Then different trajectory generation strategies are applied and
evaluated. Inside the narrow and straight nasal passage, Fellmann et al.
report that the best strategy is synchronous point-to-point. However, that
strategy could become infeasible as the distance between intermediate
configurations increases. Kuntz et al. [124] introduce a RRT-based algorithm
in a three-step planning approach for a novel transoral lung system consisting
of a bronchoscope, a CTR, and a bevel-tip needle. Their approach considers the
ability of needle steering during path planning. Kuntz et al. demonstrate the
motion planner’s ability to respect a maximum needle steering curvature. The
time to find a motion plan significantly depends on the steering capability
and the target location.

The study in [125] implements an improved RRT algorithm for cerebrovascular
intervention. The expansion direction of the random tree is a trade-off
between the new randomly sampled node and the target. This strategy can
improve the convergence speed of the algorithm, even if catheter constraints
are not considered.

Alterovitz et al. [126] proposed a Rapidly-exploring RoadMap (RRM) method that
initially explores the configuration space like RRT. Once a path is found, RRM
uses a user-specified parameter to weigh whether to explore further or to
refine the explored space by adding edges to the current roadmap to find
higher-quality paths in the explored space. Their method is presented for CTRs
in a tubular environment with protrusions as bronchus. Some studies develop
the RRM method and improve it with more accurate mechanics-based models in a
skull base surgery scenario and static lung bronchial tubes for CTRs
respectively [127], [128]. In Torres et al. [127], the planner required
$1077\text{\,}\mathrm{s}$ to get a motion plan that avoids bone, critical
blood vessels and healthy brain tissue on the way to the skull base tumour.
The same authors extend the previous studies in [129] by proposing a modified
Rapidly-exploring Random Graph (RRG) method that computes motion plans at
interactive rates. This work improves the computation cost and allows
replanning when the robot tip position changes. However, generating such a
roadmap requires an extensive amount of computation. Therefore, the method
could behave well in a static environment but not in deformable lumens.

Fauser et al. use the formulation of RRT-connect (or bi-directional RRT, Bi-
RRT) introduced earlier by them [130] to solve a common MP problem for
instruments that follow curvature constrained trajectories [131]. In [132],
Fauser et al. implement the RRT-connect algorithm for a catheter in a 3D
static aorta model, under the allowed maximal curvature
$0.1\text{\,}{\mathrm{mm}}^{-1}$. Further extension of this work proposes path
replanning from different robot position states along the initial path
starting from the descending aorta to the goal in the left ventricle [133].

##### Probabilistic RoadMap*

Kuntz et al. propose a method based on a combination of a PRM* method and
local optimisation to plan motions in a point cloud representation of a nasal
cavity anatomy [134]. The limitation is that the anatomy model is only updated
within the visible region of the endoscope, while deformations of the rest of
the anatomy are not considered. If tissue deformation is negligible, this
planning method could be used for intraoperative planning. Otherwise, the
deformations of the overall model must be considered beforehand.

####  4.2.3 Optimisation-based algorithm

MP can be formulated as an optimisation problem and solved by numerical
solvers [86]. Moreover, these methods can be programmed to consider also the
robotic kinematics.

##### Mathematical model

An optimisation-based planning algorithm that optimises the insertion length
and orientation angle of each tube for a CTR with five tubes is proposed by
Lyons et al. [135]. Firstly, the authors formulate the MP problem as a non-
linear constrained optimisation problem. Secondly, the constraint is moved to
the objective function, and the problem is converted to a series of
unconstrained optimisation problems. Lastly, the optimal solution is found
using the Limited-memory Broyden-Fletcher-Goldfarb-Shanno (L-BFGS) algorithm
[136] and Armijo’s Rule [137]. The robot kinematics is modelled using a
physically-based simulation that incorporates beam mechanics. This work is
evaluated in simulation on a patient’s lung anatomy. However, the
computational time of the proposed method is high,

Table 3: Summary of motion planning methods for IPEI navigation  Ref. |  Procedure |  Method |  Robot |  Kinematics |  Validation |  Environment
---|---|---|---|---|---|---
Level 1: Navigation assistant
Node-based:
[90] Geiger 2005 |  Transoral |  CBS |  Bronchoscope |  N |  in-silico (3D pulmonary vessels) |  Static
[100] Schafer 2007 |  Endovascular |  Dijkstra |  Guidewire |  N |  in-vitro (3D cardiovascular) |  Static
[101] Egger 2007 |  Endovascular |  Dijkstra |  Catheter |  N |  in-silico (3D aorta) |  Static
[103] Gibbs 2007 |  Transoral |  Dijkstra |  Bronchoscope |  N |  in-silico (3D bronchus) |  Static
[104] Gibbs 2008 |  Transoral |  Dijkstra |  Bronchoscope |  N |  in-silico (3D bronchus) |  Static
[102] Liu 2010 |  Endovascular |  Dijkstra |  Catheter |  N |  in-vitro (3D aorta) |  Static
[92] Wang 2011 |  Endovascular |  CBS |  Catheter |  N |  in-vitro (3D resin vessel) |  Static
[98] Huang 2011 |  Endovascular |  DFS |  Guidewire |  N |  in-silico (3D aorta) |  Static
[108] Rosell 2012 |  Transoral |  Potential field |  Bronchoscope |  Y |  in-silico (3D tracheobronchial) |  Static
[97] Gibbs 2013 |  Transoral |  DFS |  Bronchoscope |  Y |  in-vivo (3D bronchus) |  Dynamic
[93] Yang 2014 |  Endovascular |  CBS |  Guidewire |  N |  in-silico (3D aorta) |  Static
[67] Khare 2015 |  Transoral |  CBS |  Bronchoscope |  Y |  in-vivo (3D bronchus) |  Dynamic
[91] Sánchez 2016 |  Transoral |  CBS |  Bronchoscope |  N |  in-silico (3D bronchus) |  Static
[68] Zheng 2018 |  Endovascular |  CBS |  Catheter |  N |  in-vitro (3D aorta) |  Dynamic
[114] Ciobirca 2018 |  Transoral |  A* |  Bronchoscope |  N |  in-silico (3D bronchus) |  Static
[115] Niyaz 2018 |  Transnasal |  LPA* |  Concentric tube robot |  Y |  in-silico (3D nasal cavity) |  Static
[116] Niyaz 2019 |  Transnasal |  LPA* |  Concentric tube robot |  Y |  in-silico (3D nasal cavity) |  Static
[69] Zang 2019 |  Transoral |  DFS |  Bronchoscope |  Y |  in-vivo (3D bronchus) |  Dynamic
[109] Yang 2019 |  Transurethral |  Potential field |  Ureteroscope |  Y |  in-silico (3D ureter) |  Static
[120] Fagogenis 2019 |  Endovascular |  wall-following |  Concentric tube robot |  Y |  in-vivo (3D cardiovascular) |  Dynamic
[113] He 2020 |  Transnasal |  A* |  Endoscope |  N |  in-silico (3D nasal cavity) |  Static
[96] Zang 2021 |  Transoral |  DFS |  Bronchoscope |  Y |  in-vivo (3D bronchus) |  Static
[119] Huang 2021 |  Transanal |  LRPA* |  Colonoscope |  Y |  in-vitro (2D colon) |  Dynamic
[95] Wang 2021 |  Transnasal |  CBS |  Endoscope |  N |  in-silico (3D nasal cavity) |  Static
[118] Ravigopal 2021 |  Endovascular |  Hybrid A* |  Robotic guidewire |  Y |  in-vitro (2D vessel) |  Static
sampling-based:
[126] Alterovitz 2011 |  Transoral |  RRM |  Concentric tube robot |  Y |  in-silico (3D bronchus) |  Static
[127] Torres 2011 |  Transnasal |  RRM |  Concentric tube robot |  Y |  in-silico (3D nasal cavity) |  Static
[128] Torres 2012 |  Transoral |  RRM |  Concentric tube robot |  Y |  in-silico (3D bronchus) |  Static
[129] Torres 2014 |  Transnasal |  RRG |  Concentric tube robot |  Y |  in-silico (3D nasal cavity) |  Static
[123] Fellmann 2015 |  Transnasal |  RRT |  Concentric tube robot |  Y |  in-silico (3D nasal cavity) |  Static
[124] Kuntz 2015 |  Transoral |  RRT |  Steerable needle |  Y |  in-silico (3D bronchus) |  Static
[121] Aguilar 2017 |  Transoral |  bi-RRT |  Bronchoscope |  Y |  in-silico (3D bronchus) |  Static
[122] Aguilar 2017 |  Transoral |  bi-RRT |  Bronchoscope |  Y |  in-silico (3D bronchus) |  Static
[131] Fauser 2018 |  Endovascular |  bi-RRT |  Catheter |  Y |  in-silico (3D vena cava) |  Static
[132] Fauser 2019a |  Endovascular |  bi-RRT |  Steerable guidewire |  Y |  in-silico (3D aorta) |  Static
[133] Fauser 2019b |  Endovascular |  bi-RRT |  Steerable guidewire |  Y |  in-silico (3D aorta) |  Static
[134] Kuntz 2019 |  Transnasal |  PRM* |  Concentric tube robot |  Y |  in-silico (3D nasal cavity) |  Dynamic
[125] Guo 2021 |  Endovascular |  RRT |  Catheter |  N |  in-silico (cerebrovascular) |  Static
Optimisation-based:
[135] Lyons 2010 |  Transoral endotracheal |  Mathematical model |  Concentric tube robot |  Y |  in-silico (3D bronchus) |  Static
[138] Gao 2015 |  Endovascular |  ACO |  Catheter |  Y |  in-silico (3D lower limb arteries) |  Static
[139] Qi 2019 |  Endovascular |  Mathematical model |  Continuum robot |  Y |  in-vitro (blood vessels) |  Static
[140] Li 2021 |  Endovascular |  GA |  Catheter |  Y |  in-silico (3D aorta and coronaries) |  Static
[141] Guo 2021 |  Endovascular |  Mathematical model |  Catheter |  Y |  in-silico, in-vitro (3D vessel model) |  Static
[142] Abah 2021 |  Endovascular |  Mathematical model |  Catheter |  Y |  in-vitro (3D cerebrovascular) |  Static
Learning-based:
[143] Zhao 2022 |  Endovascular |  LfD using GAN |  Guidewire |  N |  in-vitro (3D vessel model) |  Static
[144] Meng 2021 |  Endovascular |  RL |  Catheter |  N |  in-silico (3D aorta) |  Static
Level 2: Navigation using waypoints
Learning-based:
[145] Trovato 2010 |  Transanal |  RL |  Fibre optic endoscope |  N |  ex-vivo (3D swine colon) |  Dynamic
[146] Rafii-Tari 2013 |  Endovascular |  LfD using GMM |  Catheter |  N |  in-vitro (3D aorta) |  Static
[147] Rafii-Tari 2014 |  Endovascular |  LfD using H+HMM |  Catheter |  Y |  in-vitro (3D aorta) |  Static
[148] Chi 2018a |  Endovascular |  LfD using DMPs |  Catheter |  Y |  in-vitro (3D aorta) |  Dynamic
[149] Chi 2018b |  Endovascular |  LfD using GMMs |  Catheter |  N |  in-vitro (3D aorta) |  Static
[150] Chi 2020 |  Endovascular |  LfD using GAIL |  Catheter |  N |  in-vitro (3D aorta) |  Static
Level 3: Semi-autonomous navigation
Node-based:
[105] Qian 2019 |  Endovascular |  Dijkstra |  Guidewire |  N |  in-vitro (3D femoral arteries, aorta) |  Static
[112] Girerd 2020 |  Transnasal |  Potential field |  Concentric tube robot |  Y |  in-silico (3D nasal cavity), in-vitro (origami tunnel) |  Static
[110] Martin 2020 |  Transanal |  Potential field |  Endoscope |  N |  in-vivo (3D colon) |  Dynamic
[111] Zhang 2020 |  Transanal |  Potential field |  Endoscope |  Y |  in-vitro (2D colon model) |  Dynamic
[107] Cho 2021 |  Endovascular |  Dijkstra |  Guidewire |  N |  in-vitro (2D vessel) |  Static
[99] Fischer 2022 |  Endovascular |  BFS |  Catheter |  N |  in-vitro (2D vessel) |  Static
[106] Schegg 2022 |  Endovascular |  Dijkstra |  Guidewire |  N |  in-silico (3D coronary arteries) |  Static
Learning-based:
[28] You 2019 |  Endovascular |  RL |  Catheter |  N |  in-vitro (3D heart) |  Static
[151] Behr 2019 |  Endovascular |  RL |  Catheter |  N |  in-vitro (2D vessel) |  Static
[152] Karstensen 2020 |  Endovascular |  RL |  Catheter |  N |  in-vitro (2D vessel) |  Static
[153] Kweon 2021 |  Endovascular |  RL |  Guidewire |  N |  in-vitro (2D coronary artery) |  Static
[154] Pore 2022 |  Transanal |  RL |  Endoscope |  N |  in-silico (3D colon) |  Dynamic
[155] Karstensen 2022 |  Endovascular |  RL |  Guidewire |  N |  ex-vivo (2D venous system) |  Dynamic

which restricts the possibility of applying this method to real-time
scenarios. Moreover, the authors manually define the skeleton and treat the
structure as a rigid body, confining its applicability.

An inverse kinematics MP method for continuum robots is expressed as an
optimisation problem based on the backbone curve method by Qi et al. [139].
The technique minimises the distance to the vasculature’s centerline under
kinematic constraints independently during each step without considering a
long-term cumulative cost. Therefore, optimal inverse kinematics that does not
consider the past and future phases might not be globally optimal.

Guo et al. [141] employed directional modeling of a teleoperated catheter and
proposed a hybrid evaluation function to find the optimal trajectory. This
work conducted wall-hit experiments and compared the response time of obstacle
avoidance with and without path planning. However, the optimal solution is
obtained with an exhaustive enumeration, which is a computationally expensive
solution. Abah et al. [142] consider the path planning as a nonlinear least-
squares problem to minimize the passive deflection of the steerable catheter.
It is achieved by matching the shape of the steerable segment as closely as
possible to the centerline of the cerebrovascular. Nevertheless, the
centerline might not be the optimal reference route for steerable catheters.

##### Evolutionary algorithms

An improved ACO method is proposed to plan an optimal vascular path with
overall consideration of factors such as catheter diameter, vascular length,
diameter, as well as curvature and torsion [138]. The associated computational
time varied from $2\text{\,}\mathrm{s}$ to $30\text{\,}\mathrm{s}$, with an
average value $12.32\text{\,}\mathrm{s}$. The high computational time cost
limits its application in real-time scenarios. Li et al. [140] proposed a fast
path planning approach under the steerable catheter curvature constraint via a
local Genetic Algorithm (GA) optimisation. The reported results showed the
planner’s ability to satisfy the robot curvature constraint while keeping a
low computational time cost of $0.191\pm 0.102$s.

####  4.2.4 Learning-based algorithms

Learning-based methods are a viable candidate for real-time MP. These methods
use statistical tools such as Artificial Neural Networks, Hidden Markov models
(HMMs), and dynamical models to map perceptual and behaviour spaces. In the
context of this article, we identified Learning from Demonstrations (LfD) and
Reinforcement Learning (RL) approaches as sub-fields of learning methods.

##### Learning from Demonstrations

Rafii-Tari et al. provides a system for human-robot collaboration for
catheterisation [147]. The catheterisation procedure is decomposed manually
into a series of catheter movement primitives. These primitive motions are
modelled as Hidden Markov models (HMMs) and are learnt using a Learning from
Demonstrations (LfD) approach. Additionally, a high-level HMM is learnt to
sequence the motion primitives. Another system, proposed by the same authors,
provides a semi-automated approach for navigation, in which guidewire
manipulation is controlled manually, and catheter motion is automated by the
robot [146]. Catheter motion is modelled here using a Gaussian Mixture Model
(GMM) to create a representation of temporally aligned phase data generated
from demonstrations. Chi et al. extend this work by showing subject-specific
variability among type I aortic arches through incorporating the anatomical
information obtained from preoperative image data [149]. In all the above
methods, expectation maximisation was used to perform maximum-likelihood
estimation to learn the model parameters. Another study presents a LfD method
based on Dynamical Movement Primitives (DMPs) [148]. DMPs are compact
representations for motion primitives formed by a set of dynamic system
equations [156]. The study uses DMPs to avoid unwanted contact between the
catheter tip and the vessel wall. DMPs were trained from human demonstrations
and used to generate motion trajectories for the proposed robotic
catheterisation platform. The proposed methods can adapt to different flow
simulations, vascular models, and catheterisation tasks. In a recent
continuation of their prior study, Chi et al. improves the RL part by
including model-free Generative Adversarial Imitation Learning (GAIL) loss
that learns from multiple demonstrations of an expert [150]. In this work, the
catheterisation policies adapt to the real-world setup and successfully
imitate the task despite unknown simulated parameters such as blood flow and
tissue-tool interaction. Zhao et al. proposed a Generative Adversarial Network
(GAN) framework by combining Convolutional Neural Network (CNN) and Long Short
Term Memory (LSTM) [143] to estimate suitable manipulation actions for
catheterization. The Deep Neural Network (DNN) is trained using experts’
demonstration data and evaluated in a phantom with a grey-scale camera
simulating X-ray imaging.

##### Reinforcement Learning

Trovato et al. developed a hardware system for a robot colonic endoscope. It
showed that the voltage for propulsion could be controlled through classic RL
algorithms such as State-Action-Reward-State-Action (SARSA) and Q-learning
that could determine the forward and backward motion [145]. Existing state-of-
the-art RL algorithms use DNN to learn from high-dimensional and unstructured
state inputs with minimal feature engineering to accomplish tasks, called Deep
Reinforcement Learning (DRL) [157]. Recently, Behr et al. [151], Karstensen et
al. [152] and Meng et al. [144] proposed a closed-loop control system based on
DRL, which uses the kinematic coordinates of the guidewire tip and manipulator
as input and outputs continuous actions for each degree of freedom for
rotation and translation. [155] showed the translation in ex-vivo veins of a
porcine liver. To improve the previously closed-loop control, You et al. [28]
and Kweon et al. [153] automate control of the catheter using DRL based on
image inputs in addition to the kinematic information of the catheter. The
authors train a policy in a simulator and show its translation to a real
robotic system. The real robotic experiments are carried out using the tip
position from an electromagnetic sensor sent to the simulator to realise the
virtual image input.

For transanal IP, Pore et al. proposed a deep visuomotor control to map the
endoscopic images to the control signal [154]. The study reported efficient
colon navigation in various in-silico colon models and better navigation
performance compared to experts in terms of overall trajectory properties.
Other efforts where some applications of DRL are emerging is tracheotomy. For
example, Athiniotis et al. uses a snake-like clinical robot to navigate down
the airway [158] autonomously. In this work, they employ a Deep Q-Network
(DQN) based navigation policy that utilises images from a monocular camera
mounted on its tip. The system serves as an assistive device for medical
personnel to perform endoscopic intubation with minimal human intervention.

###  4.3 Limitations of present MP methods

MP is a key ingredient in enabling autonomous navigation. However, it suffers
some limitations that hinder their universal application in IPEI procedures.
In this section, we identify the limitations of the aforementioned MP methods.

Node-based: The searching strategy of node-based algorithms is based on
specific cost functions. The optimality and completeness of the solution
obtained using this strategy could be guaranteed. However, (i) node-based
algorithms usually lack the consideration to satisfy robot capability during
MP, such as robots’ kinematic constraints; (ii) the uncertainty of sensing is
rarely considered; (iii) the proposed methods are only applied in rigid
environments, tissue deformations during procedures are not incorporated; (iv)
node-based algorithms usually rely on the thorough anatomical graph
structures. Accurate reconstructions of the anatomical environment in the
preoperative phase are needed to build the data structure and search inside
it. The mentioned limitations reduce the usability of these methods. In
theory, they may work, but in practice, they are difficult to be applied for
autonomous real-time navigation in real-life conditions.

Sampling and Optimisation based: Sampling and optimisation-based approaches
can account for the robot-specific characteristics. Nevertheless, the
performance of these methods is affected significantly by the robot model.
Moreover, especially for continuum soft robots, [2], the modelling methods and
soft constraints of obstacle collision are challenging and still under
investigation. Sampling-based approaches reduce computational time compared to
optimisation approaches but do not ensure the solution’s optimality. The
“probabilistic" completeness of sampling-based methods is their intrinsic
property due to their random sampling. In other words, finding a feasible path
solution is not always guaranteed. Existing optimisation-based methods are
time-consuming and mainly applied in static environments for preoperative MP.
Hybrid methods that fuse multiple approaches could maximise their respective
advantages.

Learning-based: Learning-based methods implemented in robotics have been
rising. However, current challenges associated with learning-based methods
limit their universal application in the clinical scenario [159]: One of the
major concerns is safety [160]. Recently developed learning methods make use
of DNN that can show unpredictable behaviour for unseen data outside the
training regime. Hence ensuring that the DNN never makes decisions that can
cause a safety violation is crucial [161, 162]. In addition, DNN-based
learning methods require a huge amount of training data due to their inherent
complexity, the large number of parameters involved and the learning
optimisation [163]. Therefore, a massive amount of data need to be acquired,
moved, stored, annotated and queried in an efficient way [164]. In the
surgical domain, high-quality diverse information is rarely available [48].
Various groups have proposed shared standards for device integration, data
acquisition systems and scalable infrastructure for data transmission, such as
the CONDOR (Connected Optimized Network and Data in Operating Rooms) project
(https://condor-h2020.eu/) and OR black box [165]. A general trend to overcome
data limitations is through the use of simulators. However, it is challenging
to generalise the knowledge gained through training in a simulator to a real
situation, called the “sim-to-real" reality gap. Discrepancies between reality
and virtual environment occur due to modelling errors [166]. Notably, model-
free DRL is a widely popular way of learning goal-directed behaviours and has
shown promising success in controlled robotic environments [159]. Some
commonly used algorithms include PPO (on-policy) [167], SAC (off-policy)
[168]. However, model-free DRL suffers from several limitations. First, there
is a need to design a reward function implicitly. This need requires the
developer to have domain knowledge of the dynamics of the environment [159],
which is highly complex for deformable objects and tissues [169, 170]. Second,
sensitivity to hyperparameters and under-optimised parameters can cause a
significant difference in performance. Hence, a considerable amount of time
has to be invested in tuning hyperparameters. Third, learning from high-
dimensional inputs such as images is challenging compared to low-dimensional
state features such as robot kinematic data and has shown generalisation
problems due to the high capacity of DNN [159]. Fourth, continuum robots such
as endoscopes add to the dimensionality of the action space since they have a
high number of DoFs with complex architectures, compared to industrial robots
[171]. Some algorithm difficulty involves restricted policy search.

LfD is a preferred way to learn human gestures in the context of imitation
learning [172]. However, a significant drawback of LfD methods is that they
require many demonstrations to be adequately trained, which is unfeasible in
clinical settings considering the time, resources and ethical constraints.
Furthermore, LfD typically only enables the robot to become as good as the
human’s demonstrations since a large deviation of the policy from the
demonstrated data could lead to unstable policy learning [173].

##  5 FUTURE DIRECTIONS

Navigation is one of the crucial interventional phases of an IPEI procedure.
The need for automation in IPEI navigation will increasingly support the
adoption of novel MP techniques capable of working in unstructured and dynamic
luminal environments. In this section, we describe the improvements in MP
algorithms that have been applied in other robotics domains and can be
extended to IPEI. Moreover, robot navigation relies on robot design and its
sensing capabilities. Therefore, we discuss the essential robotics
capabilities still missing to enable navigation systems with a higher level of
autonomy (e.g., LoA 4).

###  5.1 Improvements in motion planning algorithms

MP for continuum robots is a complex problem because many configurations exist
with multiple internal DoFs that have to be coordinated to achieve purposeful
motion [171, 16]. 32 of 65 publications consider MP for the robot without
considering its kinematics, as shown in Table LABEL:tab:reflist. Future
studies need to focus on the robotic constraints for active MP. Moreover,
replanning is required to adapt the current plan to deformable environments
using sensorial information. The objective of replanning is to reduce the
navigation error measured according to defined metrics. Therefore, the
computational efficiency of MP becomes essential for real-time scenarios. This
section highlights insights that can improve existing MP techniques, as
discussed in Sec. 4.

Some novel studies on the path planning of a steerable needle for neurosurgery
could give some inspiration for IPEI, as these studies considered curvature
constraints of a robotic needle. Parallel path exploration is used in the
Adaptive Fractal Trees (AFT) proposed for a programmable bevel-tip steerable
needle [174]. This method uses fractal theory and Graphics Processing Units
(GPUs) architecture to parallelize the planning process, and enhance the
computation performance and online replanning, as demonstrated with simulated
3D liver needle insertions. An Adaptive Hermite Fractal Tree (AHFT) is later
proposed, where the AFT is combined with optimised geometric Hermite curves
that allow performing a path planning strategy satisfying the heading and
targeting curvature constraints [175]. Although developed and tested only for
a preoperative neurosurgical scenario, AHFT is well-suited for GPU
parallelisation for rapid replanning.

Hybrid approaches can take advantage of individual methods to show enhanced
performance and overcome the limitation of each method. The emerging learning-
based approaches can be combined with other methods to overcome their
limitations. For example, Wang et al. propose a hybrid approach combining RL
and RRT algorithms for MP in narrow passages [176]. Their method can enhance
the local space exploration ability and guarantee the efficiency of global
path planning. Some other authors also present hybrid MP methods for IPEI
navigation. For example, Meng et al. propose a hybrid method using Breadth-
First Search (BFS) and GA for micro-robot navigation in blood vessels of rat
liver, aiming to minimise the energy consumption [74].

Optimisation-based methods are also an active area of research for obtaining
an optimal preoperative plan under complex constraints. Particle Swarm
Optimisation (PSO) is implemented by Granna et al. for a concentric tube
robotic system in neurosurgery [177]. Dynamic programming is employed for
micro-robot path planning in rigid arteries under a minimum effort criterion
[178]. However, the search space reduction technique for the constrained
optimisation problem is essential for intraoperative MP. Howell et al. propose
an augmented Lagrangian trajectory optimiser solver for constrained trajectory
optimisation problems in [179]. It handles general nonlinear state and input
constraints and offers fast convergence and numerical robustness. For an IPEI
motion planner, an efficient optimisation solver with reduced search space
would be potentially applied for intraoperative planning.

As demonstrated in Fig. 7, the recent shift towards learning-based approaches
has shown promising success. The guarantee of a provable behaviour using DNN
is still an open problem, and it is crucial to incorporate safety constraints
for the automation of IPEI navigation tasks to avoid hazardous actions. Some
studies have proposed safe RL frameworks for safety-critical paradigms using
barrier functions to restrict the robot actuation in a safe workspace [180,
160] and its behaviour is formally verified to guarantee safety [161, 162].
Robot unsafe behaviour can also be generated due to large policy updates of
gradient-based optimisation. Such large deviations can be limited by
restricting the policy update in a trust region, leading to monotonic
improvement in policy performance. Some works use f-divergences methods such
as KL-divergence to constrain the policy search from being greedy [167]. To
tackle the problem of high cost and danger of interacting with the environment
and data inefficiency of existing DRL methods, recent studies have explored
offline RL that learns exclusively from static datasets of previously
collected experiences [181].

Commonly used model-free RL techniques do not consider the dynamics of the
environment [182]. However, various complexities, such as pulsatile flow
within the vasculature or nonlinear behaviour of the instrument, hinder the
implementation of model-free algorithms and compel to simplify the problem
sets. Thus, the future trend could involve implementing model-based approaches
in endoluminal or endovascular environments [183]. Model-based approaches are
sample-efficient and require less data for training [184]. Hierarchical RL is
another untapped field for long navigation tasks, which is oriented to
subdivide the interventional phase into steps and applying specific policies
to each. This approach better adapts to the specifications of each phase. For
example, in the case of IPEI navigation, the complete navigation task could be
subdivided and learnt incrementally [185]. Recently, curriculum learning has
been proposed to learn in increasingly complex environments [186].

###  5.2 Robotic capabilities

Reaching higher LoA in navigation requires accurate control and enhanced
shape-sensing capabilities. In this subsection, we discuss various missing
capabilities in current IPEI robotic systems that hinder the development of a
LoA 4 navigation system.

####  5.2.1 Robotics actuation

Continuum robots employed in IPEI procedures are developed based on different
designs and technologies. For instance, several continuum instruments use
concentric tube mechanisms or multi-link systems [16, 15]. Soft-robotics
systems are an emerging paradigm that can enable multi-steering capabilities
and complex stress-less interventions through narrow passageways. IPEI
scenarios reflect an environment where the snake-like robot can use the wall
as a support to propel forward. Bio-inspired robots imitate biological systems
such as snake locomotion [187, 188], octopus tentacles [189], elephant trunks
[190], and mammalian spine [191]. They have been an emerging research
direction in soft-robotic actuation [192]. Pressure-driven eversion of
flexible, thin-walled tubes, called vine robots, has shown increased
applications to navigate confined spaces [193].

####  5.2.2 Proprioception and Shape-sensing

To achieve precise and reliable motion control of continuum robots, accurate
and real-time shape sensing is needed. However, accurately modelling the robot
shape is challenging due to friction, backlash, the inherent deformable nature
of the lumen or vessels and inevitable collisions with the anatomy [194]. Some
emerging sensor-based shape reconstruction techniques for interventional
devices rely on Fiber Bragg Gratings (FBG) and Electromagnetic (EM) sensors
[194, 195, 196, 197, 198]. Both FBG and EM enabled techniques provide real-
time shape estimation due to their short response time, miniature size,
biocompatibility, non-toxicity, and high sensitivity. Multiple sensors can be
attached along the length of the continuum robot to track the robot and
measure the axial strain. However, FBG sensors provide a poor response in
high-strain conditions and EM sensors suffer from the problem of EM
interference [195]. Hence, a sensor-fusion method between FBG, EM sensors and
sparse fluoroscopic images could improve 3D catheter shape reconstruction
accuracy [198].

####  5.2.3 Lumen/vessel modelling

Intraoperative imaging modalities such as ultrasound and optical coherence
tomography can support direct observation and visualisation [199, 200, 201].
Sensor fusion between IntraVascular UltraSound (IVUS) and EM can provide an
intravascular reconstruction of vessels [202, 199]. For computer-assisted
navigation, Simultaneous Localisation And Mapping (SLAM) has been successfully
demonstrated in inferring dense and detailed depth maps and lumen
reconstruction [203]. Depth prediction models have been developed recently to
estimate lumen features [204].

##  6 CONCLUSIONS

Navigation is one of the crucial steps of IPEI that requires extensive
interventional dexterity and skills. This work provides a detailed overview of
several critical aspects required to improve IPEI navigation. We propose a
classification of dedicated autonomy levels and provide a systematic review of
the governing motion planning methods. Autonomous navigation could improve the
overall execution of IPEI procedures, enabling the interventionist to focus on
the medical aspects rather than on control issues with the instruments.
Therefore, in this article, we define the levels of autonomy required for IPEI
navigation and the foreseeable human intervention associated with each level.
This classification will improve risk and safety management while we advance
towards higher levels of autonomy. One of the essential steps towards
achieving automation is through employing MP methods. A comprehensive overview
of MP techniques used in IPEI navigation is provided in this work. At the same
time, the limitations associated with existing methods are provided. These
voids in capabilities need to be overcome if one wants to raise the level of
autonomy of today’s existing robotic systems. These include improvements in MP
techniques and in enhanced robotic capabilities such as actuation and
proprioception modelling. Autonomous navigation can positively impact IPEI
procedures, making them widely accessible to a greater population.

##  7 ACKNOWLEDGEMENTS

This work was supported by the ATLAS project. This project has received
funding from the European Union’s Horizon 2020 research and innovation
programme under the Marie Sklodowska-Curie grant agreement No 813782.

IVUS

    IntraVascular UltraSound
ACO

    Ant Colony Optimization
BFS

    Breadth First Search
BPH

    Benign Prostatic Hyperplasia
CBS

    Centerline-based Structure
CNN

    Convolutional Neural Network
CRC

    Colorectal Cancer
CRISP

    Continuum Reconfigurable Incisionless Surgical Parallel
CT

    Computed Tomography
CTRs

    Concentric Tube Robots
C-TBNA

    Conventional Transbronchial Needle Aspiration
DDPG

    Deep Deterministic policy gradient
DFS

    Depth First Search
DMPs

    Dynamical Movement Primitives
DNN

    Deep Neural Network
DoFs

    Degrees-of-Freedom
DQN

    Deep Q-Network
DRL

    Deep Reinforcement Learning
EBUS

    EndoBronchial UltraSound
EI

    Endovascular Interventions
ELSE

    Ethical, Legal, Social and Economic
EM

    Electromagnetic
ESD

    Endoscopic Submucosal Disection
FBG

    Fiber Bragg Gratings
FEM

    Finite-Element Method
fURS

    Robotic Flexible Ureteroscopy
GA

    Genetic Algorithm
GAN

    Generative Adversarial Network
GAIL

    Generative Adversarial Imitation Learning
GMM

    Gaussian Mixture Model
HMMs

    Hidden Markov models
IP

    Intraluminal Procedures
IPEI

    Intraluminal Procedures and Endovascular Interventions
LoA

    Levels of Autonomy
LfD

    Learning from Demonstrations
LPA*

    Lifelong Planning A*
L-BFGS

    Limited-memory Broyden-Fletcher-Goldfarb-Shanno
LSTM

    Long Short Term Memory
MIS

    Minimally Invasive Surgical
MP

    Motion Planning
MRI

    Magnetic Resonance Imaging
MSM

    Mass-Spring Model
OCT

    Optical Coherence Tomography
PBD

    Position-Based Dynamics
PRM*

    Probabilistic RoadMap*
RL

    Reinforcement Learning
ROI

    Region of Interest
RRG

    Rapidly-exploring Random Graph
RRM

    Rapidly-exploring RoadMap
RRT

    Rapidly-exploring Random Tree
RRT-connect

    bi-directional RRT, or bi-RRT
SLAM

    Simultaneous Localisation And Mapping
TNE

    Transnasal Endoscopy
TOE

    Transoral Endoscopy
TORS

    Transoral Robotic Surgery
TURBT

    Trans-Urethral Resection of Bladder Tumours
TURP

    Trans-Urethral Resection of the Prostate

## References

  * [1] Jenna Seetohul et al.  Snake robots for surgical applications: A review.  Robot., 11(3):57, 2022.
  * [2] Tomas da Veiga et al.  Challenges of continuum robots in clinical context: a review.  Prog. Biomed. Eng., 2020.
  * [3] Nabil Simaan et al.  Medical technologies and challenges of robot-assisted minimally invasive intervention and diagnostics.  Annu. Rev. Control Robot. Auton. Syst., 1:465–490, 2018.
  * [4] J Micah Prendergast et al.  Autonomous localization, navigation and haustral fold detection for robotic endoscopy.  In Proc. IEEE Int. Conf. Intell. Robot. Syst., pages 783–790, 2018\.
  * [5] Junsun Hwang et al.  A review of magnetic actuation systems and magnetically actuated guidewire-and catheter-based microrobots for vascular interventions.  Intell. Service Robot., 13(1):1–14, 2020.
  * [6] A Orekhov et al.  Snake-like robots for minimally invasive, single-port, and intraluminal surgeries.  The Enciclopedia of Med. Robot. World Sci., pages 203–243, 2018\.
  * [7] Luigi Manfredi.  Endorobots for colonoscopy: Design challenges and available technologies.  Front. in Robot. AI, page 209, 2021.
  * [8] Paolo Fiorini et al.  Concepts and trends in autonomy for robot-assisted surgery.  Proc. IEEE, 110(7):993–1011, 2022.
  * [9] Aleks Attanasio et al.  Autonomy in surgical robotics.  Annu. Rev. Control Robot. Auton. Syst., 4, 2020.
  * [10] Tamás Haidegger.  Autonomy for surgical robots: Concepts and paradigms.  IEEE Trans. Med. Robot. Bionics, 1(2):65–76, 2019.
  * [11] Rachel Hargest.  Five thousand years of minimal access surgery: 1990–present: organisational issues and the rise of the robots.  J. Royal Soc. Medicine, 114(2):69–76, 2021.
  * [12] Guang-Zhong Yang et al.  Medical robotics—regulatory, ethical, and legal considerations for increasing levels of autonomy.  Sci. Robot., 2(4):8638, 2017.
  * [13] BK Patle et al.  A review: On path planning strategies for navigation of mobile robot.  Defence Technol., 15(4):582–606, 2019.
  * [14] Jean-Claude Latombe.  Robot motion planning, volume 124.  Springer Science & Business Media, 2012.
  * [15] Olatunji Mumini Omisore et al.  A review on flexible robotic systems for minimally invasive surgery.  IEEE Trans. Syst., Man, and Cybernetics: Syst., 2020.
  * [16] Jessica Burgner-Kahrs et al.  Continuum robots for medical applications: A survey.  IEEE Trans. Robot., 31(6):1261–1280, 2015.
  * [17] Valentina Vitiello et al.  Emerging robotic platforms for minimally invasive surgery.  IEEE Rev. Biomed. Eng., 6:111–126, 2012.
  * [18] Matthew Blecha et al.  Modern endovascular therapy.  World J. Surg., pages 1–10, 2020.
  * [19] Andrea Bolzoni Villaret et al.  Robotic transnasal endoscopic skull base surgery: systematic review of the literature and report of a novel prototype for a hybrid system (brescia endoscope assistant robotic holder).  World neurosurgery, 105:875–883, 2017.
  * [20] Brian S Peters et al.  Review of emerging surgical robotic technology.  Surg. Endoscopy, 32(4):1636–1655, 2018.
  * [21] Hedyeh Rafii-Tari et al.  Current and emerging robot-assisted endovascular catheterization technologies: a review.  Ann. Biomed. Eng., 42(4):697–715, 2014.
  * [22] Johannes Bonatti et al.  Robotic technology in cardiovascular medicine.  Nature Rev. Cardiol., 11(5):266, 2014.
  * [23] Yili Fu et al.  Steerable catheters in minimally invasive vascular surgery.  Int. J. Med. Robot. Comput. Assist. Surg., 5(4):381–391, 2009.
  * [24] Ali Pourdjabbar et al.  The development of robotic technology in cardiac and vascular interventions.  Rambam Maimonides Med. J., 8(3), 2017.
  * [25] Marton Berczeli et al.  Catheter robots in the cardiovascular system.  Latest Develop. Med. Robot. Syst., page 95, 2021.
  * [26] Gastone Ciuti et al.  Frontiers of robotic colonoscopy: A comprehensive review of robotic colonoscopes and technologies.  J. Clinical Medicine, 9(6):1648, 2020.
  * [27] Chung-Kwong Yeung et al.  Emerging next-generation robotic colonoscopy systems towards painless colonoscopy.  J. Digestive Diseases, 20(4):196–205, 2019.
  * [28] Hyeonseok You et al.  Automatic control of cardiac ablation catheter with deep reinforcement learning method.  J. Mech. Sci. Tech., 33(11):5415–5423, 2019.
  * [29] Madeline Lemke et al.  Colonoscopy trainers experience greater stress during insertion than withdrawal: implications for endoscopic curricula.  J. Can. Assoc. Gastroenterology, 4(1):15–20, 2021.
  * [30] Rajib Ahmed et al.  Colonoscopy technologies for diagnostics and drug delivery.  Med. Devices & Sensors, 2(3-4):e10041, 2019.
  * [31] Karen J Wernli et al.  Risks associated with anesthesia services during colonoscopy.  Gastroenterology, 150(4):888–894, 2016.
  * [32] Cesare Hassan et al.  Diagnostic yield and miss rate of endorings in an organized colorectal cancer screening program: the smart (study methodology for adr-related technology) trial.  Gastrointestinal Endoscopy, 89(3):583–590, 2019.
  * [33] Mary Than et al.  Diagnostic miss rate for colorectal cancer: an audit.  Ann. gastroenterology, 28(1):94, 2015.
  * [34] A Eickhoff et al.  In vitro evaluation of forces exerted by a new computer-assisted colonoscope (the neoguide endoscopy system).  Endoscopy, 38(12):1224–1229, 2006.
  * [35] Nathan Gluck et al.  A novel self-propelled disposable colonoscope is effective for colonoscopy in humans (with video).  Gastrointestinal endoscopy, 83(5):998–1004, 2016.
  * [36] Moshe Shike et al.  Sightline colonosight system for a disposable, power-assisted, non-fiber-optic colonoscopy (with video).  Gastrointestinal endoscopy, 68(4):701–710, 2008.
  * [37] E Tumino et al.  Use of robotic colonoscopy in patients with previous incomplete colonoscopy.  Eur. Rev. Med. Pharmacol Sci., 21(4):819–826, 2017.
  * [38] S Duke Herrell et al.  Future robotic platforms in urologic surgery: recent developments.  Current opinion in urology, 24(1):118, 2014.
  * [39] Mark D Tyson et al.  Urological applications of natural orifice transluminal endoscopic surgery.  Nature Rev. Urology, 11(6):324–332, 2014.
  * [40] Wassim M Bazzi et al.  Natural orifice transluminal endoscopic surgery in urology: Review of the world literature.  Urology Ann., 4(1):1, 2012.
  * [41] Yuyang Chen et al.  Review of surgical robotic systems for keyhole and endoscopic procedures: state of the art and perspectives.  Front. of med., 14(4):382–403, 2020.
  * [42] Jens Rassweiler et al.  Robot-assisted flexible ureteroscopy: an update.  Urolithiasis, 46(1):69–77, 2018.
  * [43] Giorgio Gandaglia et al.  Novel technologies in urologic surgery: a rapidly changing scenario.  Current urology Rep., 17(3):1–8, 2016.
  * [44] Pietro Valdastri et al.  Advanced technologies for gastrointestinal endoscopy.  Ann. Rev. Biomed. Eng., 14, 2012.
  * [45] Wojciech Marlicz et al.  Frontiers of robotic gastroscopy: A comprehensive review of robotic gastroscopes and technologies.  Cancers, 12(10):2775, 2020.
  * [46] Armando De Virgilio et al.  Trans-oral robotic surgery in the management of parapharyngeal space tumors: a systematic review.  Oral Oncology, 103:104581, 2020.
  * [47] Udaya BS Prakash et al.  Bronchoscopy.  J. Bronchology & Interv. Pulmonology, 1(4):340, 1994.
  * [48] Lauren R Kennedy-Metz et al.  Computer vision in the operating room: Opportunities and caveats.  IEEE Trans. Med. Robot. Bionics, 3(1):2–10, 2020.
  * [49] Abhinav Agrawal et al.  Robotic bronchoscopy for pulmonary lesions: a review of existing technologies and clinical data.  J. thoracic disease, 12(6):3279, 2020.
  * [50] Heinz Stammberger et al.  Functional endoscopic sinus surgery.  Eur. archives of oto-rhino-laryngology, 247(2):63–76, 1990.
  * [51] Jessica Burgner et al.  A telerobotic system for transnasal surgery.  IEEE Trans. Mechatronics, 19(3):996–1006, 2013.
  * [52] Alba Madoglio et al.  Robotics in endoscopic transnasal skull base surgery: literature review and personal experience.  Control Syst. Design of Bio-Robot. and Bio-mechatronics with Adv. Appl., pages 221–244, 2020.
  * [53] Tokuma Tanuma et al.  Current status of transnasal endoscopy worldwide using ultrathin videoscope for upper gastrointestinal tract.  Digestive Endoscopy, 28:25–31, 2016.
  * [54] Nabil Simaan et al.  Design and integration of a telerobotic system for minimally invasive surgery of the throat.  Int. J. Robot. Res., 28(9):1134–1153, 2009.
  * [55] Shimon Y Nof.  Automation: What it means to us around the world.  In Springer Handbook Automat., pages 13–52. 2009.
  * [56] Joanna Isabelle Olszewska et al.  Ontology for autonomous robotics.  In Proc. IEEE Int. Symp. Robot. Human Interact. Commun., pages 189–194, 2017.
  * [57] Hualong Chen et al.  From automation system to autonomous system: An architecture perspective.  J. Mar. Sci. Eng., 9(6):645, 2021.
  * [58] Michael Fisher et al.  Towards a framework for certification of reliable autonomous systems.  Auton. Agents and Multi-Agent Syst., 35(1):1–65, 2021.
  * [59] Federica Merenda et al.  Ethics, safety and human centricity: Intelligent machines under the scope of the european ai regulation act.  In Workshop Conf. Italian Inst. Robot. Intell. Mach., 2021.
  * [60] Shane O’sullivan et al.  Legal, regulatory, and ethical frameworks for development of standards in artificial intelligence (ai) and autonomous robotic surgery.  Int. J. Med. Robot. Comput. Assist. Surg., 15(1):e1968, 2019.
  * [61] European Parliament and Council of the European Union.  Artificial intelligence act: Regulation laying down harmonised rules on artificial intelligence and amending certain union legislative acts.  Proposal for Regulation COM/2021/206, 2021.
  * [62] National artificial intelligence initiative.  https://www.congress.gov/bill/116th-congress/house-bill/6216/text.
  * [63] Maria-Camilla Fiazza.  The eu proposal for regulating ai: Foreseeable impact on medical robotics.  In Proc. IEEE Int. Conf. Adv. Robot., 2021.
  * [64] International Electrotechnical Commission (2017c).  Iec tr 60601-4-1 – medical electrical equipment – part 4-1: Guidance and interpretation - medical electrical equipment and medical electrical systems employing a degree of autonomy.  In URL https://webstore.iec.ch/publication/29312.), 2017.
  * [65] Darko Katić et al.  Lapontospm: an ontology for laparoscopic surgeries and its application to surgical phase recognition.  Int. J. Comput. Assist. Radiol. Surg., 10(9):1427–1434, 2015.
  * [66] Sharan R Ravigopal et al.  Fluoroscopic image-based 3-d environment reconstruction and automated path planning for a robotically steerable guidewire.  IEEE Robot. Automat. Lett., 7(4):11918–11925, 2022.
  * [67] Rahul Khare et al.  Hands-free system for bronchoscopy planning and guidance.  IEEE Trans. Biomed. Eng., 62(12):2794–2811, 2015.
  * [68] Jian-Qing Zheng et al.  Towards 3d path planning from a single 2d fluoroscopic image for robot assisted fenestrated endovascular aortic repair.  In Proc. IEEE Int. Conf. Robot. Automat., pages 8747–8753, 2019\.
  * [69] Xiaonan Zang et al.  Optimal route planning for image-guided ebus bronchoscopy.  Comput. in biology and medicine, 112:103361, 2019.
  * [70] Addisu Z Taddese et al.  Enhanced real-time pose estimation for closed-loop robotic manipulation of magnetically actuated capsule endoscopes.  Int. J. Robot. Res., 37(8):890–911, 2018.
  * [71] Matthew J Page et al.  The prisma 2020 statement: an updated guideline for reporting systematic reviews.  Brit. Med. J., 372, 2021.
  * [72] Michael E Rose et al.  pybliometrics: Scriptable bibliometrics using a python interface to scopus.  SoftwareX, 10:100263, 2019.
  * [73] Fanny ER Vuik et al.  Colon capsule endoscopy in colorectal cancer screening: a systematic review.  Endoscopy, 53(08):815–824, 2021.
  * [74] Ke Meng et al.  Motion planning and robust control for the endovascular navigation of a microrobot.  IEEE Trans. Ind. Inform., 2019.
  * [75] Bruno Siciliano et al.  Springer handbook of robotics, volume 200.  Springer, 2008.
  * [76] Liang Yang et al.  Survey of robot 3d path planning algorithms.  J. Control Sci. Eng., 2016.
  * [77] Irene Cheng et al.  Enhanced segmentation and skeletonization for endovascular surgical planning.  In Proc. Med. Imag.: Image-Guided Procedures, Robot. Interv., and Model., volume 8316, pages 868–874, 2012.
  * [78] Robert Tarjan.  Depth-first search and linear graph algorithms.  SIAM J. Comput., 1(2):146–160, 1972.
  * [79] Rina Dechter et al.  Generalized best-first search strategies and the optimality of a.  J. ACM, 32(3):505–536, 1985.
  * [80] Edsger W Dijkstra et al.  A note on two problems in connexion with graphs.  Numerische mathematik, 1(1):269–271, 1959.
  * [81] Yong Koo Hwang et al.  A potential field approach to path planning.  IEEE Trans. Robot. Automat., 8(1):23–32, 1992.
  * [82] Peter E Hart et al.  A formal basis for the heuristic determination of minimum cost paths.  IEEE Trans. Syst. Sci. Cybernetics, 4(2):100–107, 1968.
  * [83] Steven M LaValle et al.  Rapidly-exploring random trees: A new tool for path planning.  Comput. Sci. Dept. Oct., 98(11), 1998.
  * [84] Roland Geraerts et al.  A comparative study of probabilistic roadmap planners.  In Algorithmic Found. Robot., pages 43–57. 2004.
  * [85] Sertac Karaman et al.  Sampling-based algorithms for optimal motion planning.  Int. J. Robot. Res., 30(7):846–894, 2011.
  * [86] Purushothaman Raja et al.  Optimal path planning of mobile robots: A review.  Int. J. Phys. Sci., 7(9):1314–1320, 2012.
  * [87] Marco Dorigo et al.  Ant colony optimization.  IEEE Comput. Intell. Mag., 1(4):28–39, 2006.
  * [88] Harish Ravichandar et al.  Recent advances in robot learning from demonstration.  Annu. Rev. Control, Robot., and Auton. Syst., 3:297–330, 2020.
  * [89] Richard S Sutton et al.  Reinforcement learning: An introduction.  MIT press, 2018.
  * [90] Bernhard Geiger et al.  Virtual bronchoscopy of peripheral nodules using arteries as surrogate pathways.  In Amir A. Amini and Armando Manduca, editors, Proc. Med. Imag.: Physiol. Func. Struct. from Med. Imag. SPIE, 2005.
  * [91] Carles Sánchez et al.  Navigation path retrieval from videobronchoscopy using bronchial branches.  In Workshop on Clin. Image. Based Procedures, pages 62–70. 2016\.
  * [92] Junchen Wang et al.  Intravascular catheter navigation using path planning and virtual visual feedback for oral cancer treatment.  Int. J. Med. Robot. Comput. Assist. Surg., 7(2):214–224, 2011.
  * [93] Fan Yang et al.  Centerlines extraction for lumen model of human vasculature for computer-aided simulation of intravascular procedures.  In Proc. IEEE World Congr. Intell. Control Automat., pages 970–975, 2014.
  * [94] Michael Kerschnitzki et al.  Architecture of the osteocyte network correlates with bone material quality.  J. bone and mineral Res., 28(8):1837–1845, 2013.
  * [95] WANG Yudong et al.  Rapid path extraction and three-dimensional roaming of the virtual endonasal endoscope.  Chin. J. Electronics, 30(3):397–405, 2021.
  * [96] Xiaonan Zang et al.  Image-guided ebus bronchoscopy system for lung-cancer staging.  Inform. in medicine unlocked, 25:100665, 2021.
  * [97] Jason D Gibbs et al.  Optimal procedure planning and guidance system for peripheral bronchoscopy.  IEEE Trans. Biomed. Eng., 61(3):638–657, 2013.
  * [98] Dongjin Huang et al.  An interactive 3d preoperative planning and training system for minimally invasive vascular surgery.  In Pro. IEEE Int. Conf. Comput. Aided Des. Comput. Graph., pages 443–449, 2011.
  * [99] Cedric Fischer et al.  Using magnetic fields to navigate and simultaneously localize catheters in endoluminal environments.  IEEE Robot. Automat. Lett., 2022.
  * [100] Sebastian Schafer et al.  Planning image-guided endovascular interventions: guidewire simulation using shortest path algorithms.  In Proc. Med. Imag.: Visualization and Imag. Guided Procedures, volume 6509, pages 813–822, 2007.
  * [101] J. Egger et al.  A fast vessel centerline extraction algorithm for catheter simulation.  In Proc. IEEE Int. Symp. Comput. Based Med. Syst., pages 177–182, 2007.
  * [102] H Liu et al.  An in vitro investigation of image-guided steerable catheter navigation.  Proc. Institution of Mech. Eng.: J. Eng. Medicine, 224(8):945–954, 2010.
  * [103] Jason D Gibbs et al.  3d path planning and extension for endoscopic guidance.  In Med. Imag.: Visualization and Imag. Guided Procedures, 2007.
  * [104] Jason D Gibbs et al.  Integrated system for planning peripheral bronchoscopic procedures.  In Med. Imag.: Physiol. Func. Struct. Med. Imag., 2008.
  * [105] Hanxin Qian et al.  Towards rebuild the interventionist’s intra-operative natural behavior: A fully sensorized endovascular robotic system design.  In Proc. IEEE Int. Conf. Med. Imag. Phys. Eng., 2019.
  * [106] Pierre Schegg et al.  Automated planning for robotic guidewire navigation in the coronary arteries.  In Proc. IEEE Int. Conf. Soft Robot., pages 239–246, 2022.
  * [107] Yongjun Cho et al.  Image processing based autonomous guidewire navigation in percutaneous coronary intervention.  In Proc. IEEE Int. Conf. Consum. Electron. Asia, 2021.
  * [108] Jan Rosell et al.  Motion planning for the virtual bronchoscopy.  In Proc. IEEE Int. Conf. Robot. Automat., pages 2932–2937, 2012\.
  * [109] Fan Yang et al.  Path planning of flexible ureteroscope based on ct image.  In Proc. IEEE Chin. Control Conf., pages 4667–4672, 2019.
  * [110] James W Martin et al.  Enabling the future of colonoscopy with intelligent and autonomous magnetic manipulation.  Nature Mach. Intell., 2(10):595–606, 2020.
  * [111] Qi Zhang et al.  Enabling autonomous colonoscopy intervention using a robotic endoscope platform.  IEEE Trans. Biomed. Eng., 68(6):1957–1968, 2020.
  * [112] Cedric Girerd et al.  Slam-based follow-the-leader deployment of concentric tube robots.  IEEE Robot. Automat. Lett., 5(2):548–555, 2020.
  * [113] Yucheng He et al.  Endoscopic path planning in robot-assisted endoscopic nasal surgery.  IEEE Access, 2020.
  * [114] C Ciobirca et al.  A new procedure for automatic path planning in bronchoscopy.  Mater. Today: Proc., 5(13):26513–26518, 2018.
  * [115] Sherdil Niyaz et al.  Following surgical trajectories with concentric tube robots via nearest-neighbor graphs.  In Proc. Int. Symp. Exp. Robot., 2018.
  * [116] Sherdil Niyaz et al.  Optimizing motion-planning problem setup via bounded evaluation with application to following surgical trajectories.  In Proc. IEEE Int. Conf. Intell. Robot. Syst., pages 1355–1362, 2019.
  * [117] Nathan Koenig et al.  Design and use paradigms for gazebo, an open-source multi-robot simulator.  In Proc. IEEE Int. Conf. Intell. Robot. Syst., volume 3, pages 2149–2154, 2004.
  * [118] Sharan R Ravigopal et al.  Automated motion control of the coast robotic guidewire under fluoroscopic guidance.  In Proc. Int. Symp. Med. Robot., 2021.
  * [119] Hao-En Huang et al.  Autonomous navigation of a magnetic colonoscope using force sensing and a heuristic search algorithm.  Sci. Rep., 11(1):1–15, 2021.
  * [120] G Fagogenis et al.  Autonomous robotic intracardiac catheter navigation using haptic vision.  Sci. Robot., 4(29), 2019.
  * [121] Wilbert G Aguilar et al.  Rrt-based path planning for virtual bronchoscopy simulator.  In Proc. IEEE Int. Conf. Augmented Reality, Virtual Reality and Comput. Graph., pages 155–165, 2017.
  * [122] Wilbert G Aguilar et al.  Virtual bronchoscopy motion planner.  In Proc. IEEE Int. Conf. Electronics, Electrical Eng. and Comput., 2017.
  * [123] Carolin Fellmann et al.  Implications of trajectory generation strategies for tubular continuum robots.  In Proc. IEEE Int. Conf. Intell. Robot. Syst., pages 202–208, 2015\.
  * [124] Alan Kuntz et al.  Motion planning for a three-stage multilumen transoral lung access system.  In Proc. IEEE Int. Conf. Intell. Robot. Syst., pages 3255–3261, 2015.
  * [125] Jian Guo et al.  A training system for vascular interventional surgeons based on local path planning.  In Proc. IEEE Int. Conf. Mechatronics and Automat., pages 1328–1333, 2021.
  * [126] Ron Alterovitz et al.  Rapidly-exploring roadmaps: Weighing exploration vs. refinement in optimal motion planning.  In Proc. IEEE Int. Conf. Robot. Automat., pages 3706–3712, 2011\.
  * [127] Luis G. Torres et al.  Motion planning for concentric tube robots using mechanics-based models.  In Proc. IEEE Int. Conf. Intell. Robot. Syst., 2011.
  * [128] Luis G Torres et al.  Task-oriented design of concentric tube robots using mechanics-based models.  In Proc. IEEE Int. Conf. Intell. Robot. Syst., pages 4449–4455, 2012.
  * [129] Luis G Torres et al.  Interactive-rate motion planning for concentric tube robots.  In Proc. IEEE Int. Conf. Robot. Automat., pages 1915–1921, 2014\.
  * [130] Johannes Fauser et al.  Planning nonlinear access paths for temporal bone surgery.  Int. J. Comput. Assist. Radiol. Surg., 13(5):637–646, 2018.
  * [131] Johannes Fauser et al.  Generalized trajectory planning for nonlinear interventions.  In Comput. Assist. Robot. Endoscopy. pp. 46–53, 2018.
  * [132] Johannes Fauser et al.  Optimizing clearance of bézier spline trajectories for minimally-invasive surgery.  In Proc. Int. Conf. Med. Imag. Comput. and Imag. Assist. Interv., pages 20–28, 2019.
  * [133] Johannes Fauser et al.  Planning for flexible surgical robots via bézier spline translation.  IEEE Robot. Automat. Lett., 4(4):3270–3277, 2019.
  * [134] Alan Kuntz et al.  Planning high-quality motions for concentric tube robots in point clouds via parallel sampling and optimization.  In Proc. IEEE Int. Conf. Intell. Robot. Syst., pages 2205–2212, 2019.
  * [135] Lisa A Lyons et al.  Planning active cannula configurations through tubular anatomy.  In Proc. IEEE Int. Conf. Robot. Automat., pages 2082–2087, 2010\.
  * [136] Dong C Liu et al.  On the limited memory bfgs method for large scale optimization.  Math. Program., 45(1):503–528, 1989.
  * [137] Mokhtar S Bazaraa et al.  Nonlinear programming: theory and algorithms.  John Wiley & Sons, 2013.
  * [138] Mingke Gao et al.  Three-dimensional path planning and guidance of leg vascular based on improved ant colony algorithm in augmented reality.  J. Med. Syst., 39(11):133, 2015.
  * [139] Fei Qi et al.  Kinematic analysis and navigation method of a cable-driven continuum robot used for minimally invasive surgery.  Int. J. Med. Robot. Comput. Assist. Surg., 15(4):e2007, 2019.
  * [140] Zhen Li et al.  Path planning for endovascular catheterization under curvature constraints via two-phase searching approach.  Int. J. Comput. Assist. Radiol. Surg., 16(4):619–627, 2021.
  * [141] Jian Guo et al.  Design a novel of path planning method for the vascular interventional surgery robot based on dwa model.  In Proc. IEEE Int. Conf. Mechatronics and Automat., pages 1322–1327, 2021.
  * [142] Colette Abah et al.  Image-guided optimization of robotic catheters for patient-specific endovascular intervention.  In Proc. IEEE Int. Symp. Med. Robot., pages 1–8, 2021.
  * [143] Yan Zhao et al.  Surgical gan: Towards real-time path planning for passive flexible tools in endovascular surgeries.  Neurocomputing, 2022.
  * [144] Fanxu Meng et al.  Evaluation of a reinforcement learning algorithm for vascular intervention surgery.  In Proc. IEEE Int. Conf. Mechatronics and Automat., pages 1033–1037, 2021.
  * [145] Gabriele Trovato et al.  Development of a colon endoscope robot that adjusts its locomotion through the use of reinforcement learning.  Int. J. Comput. Assist. Radiol. Surg., 5(4):317–325, 2010.
  * [146] Hedyeh Rafii-Tari et al.  Learning-based modeling of endovascular navigation for collaborative robotic catheterization.  In Adv. Inf. Syst. Eng., pages 369–377. 2013.
  * [147] Hedyeh Rafii-Tari et al.  Hierarchical hmm based learning of navigation primitives for cooperative robotic endovascular catheterization.  In Proc. Int. Conf. Med. Imag. Comput. and Comput. Assist. Interv., pages 496–503, 2014.
  * [148] Wenqiang Chi et al.  Trajectory optimization of robot-assisted endovascular catheterization with reinforcement learning.  In Proc. IEEE Int. Conf. Intell. Robot. Syst., 2018.
  * [149] Wenqiang Chi et al.  Learning-based endovascular navigation through the use of non-rigid registration for collaborative robotic catheterization.  Int. J. Comput. Assist. Radiol. Surg., 13(6):855–864, 2018.
  * [150] Wenqiang Chi et al.  Collaborative robot-assisted endovascular catheterization with generative adversarial imitation learning.  In Proc. IEEE Int. Conf. Robot. Automat., 2020.
  * [151] Tobias Behr et al.  Deep reinforcement learning for the navigation of neurovascular catheters.  Current Directions Biomed. Eng., 5(1):5–8, 2019.
  * [152] Lennart Karstensen et al.  Autonomous guidewire navigation in a two dimensional vascular phantom.  Current Directions in Biomed. Eng., 6(1), 2020.
  * [153] Jihoon Kweon et al.  Deep reinforcement learning for guidewire navigation in coronary artery phantom.  IEEE Access, 9:166409–166422, 2021.
  * [154] Ameya Pore et al.  Colonoscopy navigation using end-to-end deep visuomotor control: A user study.  arXiv preprint arXiv:2206.15086, 2022.
  * [155] Lennart Karstensen et al.  Learning-based autonomous vascular guidewire navigation without human demonstration in the venous system of a porcine liver.  Int. J. Comput. Assist. Radiol. Surg., pages 1–8, 2022.
  * [156] Matteo Saveriano et al.  Dynamic movement primitives in robotics: A tutorial survey.  arXiv preprint arXiv:2102.03861, 2021.
  * [157] Volodymyr Mnih et al.  Human-level control through deep reinforcement learning.  Nature, 518(7540):529–533, 2015.
  * [158] S Athiniotis et al.  Deep q reinforcement learning for autonomous navigation of surgical snake robot in confined spaces.  In Proc. Hamlyn Symp. Med. Robot., 2019.
  * [159] Julian Ibarz et al.  How to train your robot with deep reinforcement learning: lessons we have learned.  Int. J. Robot. Res., 40(4-5):698–721, 2021.
  * [160] Javier Garcıa et al.  A comprehensive survey on safe reinforcement learning.  J. Mach. Learn. Res., 16(1):1437–1480, 2015.
  * [161] Ameya Pore et al.  Safe reinforcement learning using formal verification for tissue retraction in autonomous robotic-assisted surgery.  In Proc. IEEE Int. Conf. Intell. Robot. Syst., pages 4025–4031. IEEE, 2021.
  * [162] Davide Corsi et al.  Constrained reinforcement learning and formal verification for safe colonoscopy navigation.  arXiv preprint arXiv:2303.03207, 2023.
  * [163] Yann LeCun et al.  Deep learning.  Nature, 521(7553):436–444, 2015.
  * [164] David C Birkhoff et al.  A review on the current applications of artificial intelligence in the operating room.  Surg. Innov., 28(5):611–619, 2021.
  * [165] Mitchell G Goldenberg et al.  Using data to enhance performance and improve quality and safety in surgery.  JAMA Surg., 152(10):972–973, 2017.
  * [166] Andrei A Rusu et al.  Sim-to-real robot learning from pixels with progressive nets.  arXiv preprint arXiv:1610.04286, 2016.
  * [167] John Schulman et al.  Proximal policy optimization algorithms.  arXiv preprint arXiv:1707.06347, 2017.
  * [168] Tuomas Haarnoja et al.  Soft actor-critic algorithms and applications.  arXiv preprint arXiv:1812.05905, 2018.
  * [169] Xingyu Lin et al.  Softgym: Benchmarking deep reinforcement learning for deformable object manipulation.  arXiv preprint arXiv:2011.07215, 2020.
  * [170] Zhen Li et al.  Position-based dynamics simulator of vessel deformations for path planning in robotic endovascular catheterization.  Med. Eng. & Physics, 110:103920, 2022.
  * [171] Pierre Dupont et al.  Continuum robots for medical interventions.  Proc. IEEE, 2022.
  * [172] Ameya Pore et al.  Learning from demonstrations for autonomous soft-tissue retraction.  In Proc. IEEE Int. Symp. Med. Robot., 2021.
  * [173] Seyed Kamyar Seyed Ghasemipour et al.  A divergence minimization perspective on imitation learning methods.  In Proc. PMLR Conf. Robot. Learn., pages 1259–1277, 2020.
  * [174] Fangde Liu et al.  Fast and adaptive fractal tree-based path planning for programmable bevel tip steerable needles.  IEEE Robot. Automat. Lett., 1(2):601–608, 2016.
  * [175] Marlene Pinzi et al.  The adaptive hermite fractal tree (ahft): a novel surgical 3d path planning approach with curvature and heading constraints.  Int. J. Comput. Assist. Radiol. Surg., 14(4):659–670, 2019.
  * [176] Wei Wang et al.  A learning-based multi-rrt approach for robot path planning in narrow passages.  J. Intell. Robot. Syst., 90(1-2):81–100, 2018.
  * [177] Josephine Granna et al.  Computer-assisted planning for a concentric tube robotic system in neurosurgery.  Int. J. Comput. Assist. Radiol. Surg., 14(2):335–344, 2019.
  * [178] Mohammad Javad Pourmanda et al.  Navigation and control of endovascular helical swimming microrobot using dynamic programing and adaptive sliding mode strategy.  Control Syst. Des. Bio-Robot. Bio-Mechatronics Adv. Appl., page 201, 2019.
  * [179] Taylor A Howell et al.  Altro: A fast solver for constrained trajectory optimization.  In Proc. IEEE Int. Conf. Intell. Robot. Syst., pages 7674–7679, 2019.
  * [180] Richard Cheng et al.  End-to-end safe reinforcement learning through barrier functions for safety-critical continuous control tasks.  In Proc. AAAI Conf. Artif. Intell., volume 33, pages 3387–3395, 2019.
  * [181] Rafael Figueiredo Prudencio et al.  A survey on offline reinforcement learning: Taxonomy, review, and open problems.  arXiv preprint arXiv:2203.01387, 2022.