# Towards Safer Robot-Assisted Surgery:
A Markerless Augmented Reality Framework

Ziyang Chen  Laura Cruciani  Ke Fan  Matteo Fontana  Elena Lievore  Ottavio De
Cobelli
Gennaro Musi  Giancarlo Ferrigno  and Elena De Momi Ziyang Chen, Laura
Cruciani, Ke Fan, and Giancarlo Ferrigno are with the Department of
Electronics, Information and Bioengineering, Politecnico di Milano, 20133,
Italy (e-mail<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>giancarlo.ferrigno@polimi.it).Matteo Fontana, and Elena
Lievore are with the Department of Urology, European Institute of Oncology,
IRCCS, Milan, 20141, Italy (e-mail<EMAIL_ADDRESS>elena.lievore@ieo.it).Ottavio De Cobelli, and Gennaro Musi are with the
Department of Urology, European Institute of Oncology, IRCCS, Milan, 20141,
Italy, and also with the Department of Oncology and Onco-haematology, Faculty
of Medicine and Surgery, University of Milan, Milan, Italy (e-mail:
<EMAIL_ADDRESS>gennaro.musi@ieo.it).Elena De Momi is with the
Department of Electronics, Information and Bioengineering, Politecnico di
Milano, 20133, Italy, and also with European Institute of Oncology, IRCCS,
Milan, 20141, Italy (e-mail: elena.demomi@polimi.it).

###### Abstract

Robot-assisted surgery is rapidly developing in the medical field, and the
integration of augmented reality shows the potential of improving the
surgeons’ operation performance by providing more visual information. In this
paper, we proposed a markerless augmented reality framework to enhance safety
by avoiding intra-operative bleeding which is a high risk caused by the
collision between the surgical instruments and the blood vessel. Advanced
stereo reconstruction and segmentation networks are compared to find out the
best combination to reconstruct the intra-operative blood vessel in the 3D
space for the registration of the pre-operative model, and the minimum
distance detection between the instruments and the blood vessel is
implemented. A robot-assisted lymphadenectomy is simulated on the da Vinci
Research Kit in a dry lab, and ten human subjects performed this operation to
explore the usability of the proposed framework. The result shows that the
augmented reality framework can help the users to avoid the dangerous
collision between the instruments and the blood vessel while not introducing
an extra load. It provides a flexible framework that integrates augmented
reality into the medical robot platform to enhance safety during the
operation.

Robot-assisted surgery, markerless augmented reality, stereo reconstruction,
segmentation, da Vinci Research Kit.

##  1 INTRODUCTION

Robot-assisted surgery (RAS) got improved patient outcomes in both intra-
operative operation and post-operative recovery compared to traditional open
surgery, and it also provides the possibility to integrate artificial
intelligence in this platform for autonomous and safe operation. An emerging
technology, i.e., Augmented Reality (AR) fusing virtual targets on real
scenes, provides more visual information for the users, and it has been
introduced in the field of robotic surgery to enhance the safety [1, 2, 3].
Generally, the surgeon needs to capture the pre-operative images of the
patient such as Computed Tomography (CT) and Magnetic Resonance Imaging (MRI)
for the preliminary surgical planning. These CT/MRI slices can be segmented
using software like 3D Slicer to generate the pre-operative 3D model, and then
projected on the intra-operative images to implement the AR effect. These
augmented intra-operative images can guide the operation of the surgeons by
providing more visual information and improve the operation performance. One
major challenge is how to localize the intra-operative soft tissues or organs
so that the pre-operative model can register with the intra-operative target
correctly, to implement the overlapping of the pre-operative model on the
corresponding position of intra-operative images. In [4, 5], the authors
implemented the AR in robot-assisted radical prostatectomy by overlapping the
pre-operative 3D model on the endoscopic images, they used the software vMIX
(StudioCoast Pty Ltd, Australia) to manually align the position of the pre-
operative model, which hinders the practice in real-time AR visualization
during the operation. Similarly, a manual alignment between the pre-operative
model and the intra-operative anatomy was used in some other operations, such
as the robotic thyroidectomy [6], the transoral surgery [7] and the partial
nephrectomy [8]. In [9], the authors introduced three possible solutions to
implement AR registration including landmarks, laparoscopic video and intra-
operative ultrasound to recognize the 3D position of surgical scenes, and they
summarized that the laparoscopic video-based approach will be the mainstream
since it does not require external hardware. Also, the authors in [10]
introduced the registration process using the robotic instruments by pointing
at pre-installed markers or the extra projector-camera system. To implement
the automatic AR alignment using the laparoscopic video based on the da Vinci
Surgical System (dVSS), the authors in [11] proposed to track the interested
soft tissue and then recover the corresponding 3D position information of the
soft tissue using stereo reconstruction. The experimental result showed that
the AR effect can be implemented on the intra-operative images. Nevertheless,
this approach relied on an external device to manually draw the boundary of
the interested target at the beginning of the operation, and the pre-operative
model was substituted using a simple ellipsoid that is different from the
intra-operative tissue.

Many stereo reconstruction methods have been proposed to reconstruct the 3D
scene by estimating the disparity map, and the depth information can be
recovered by a transformation with the focal length and the baseline of the
stereo endoscope. [12] is a classic stereo reconstruction network, the authors
adopted the Convolutional Neural Network (CNN) with embedding a Spatial
Pyramid Pooling (SPP) module to extract the high-level features of the stereo
image, then they were concatenated as a 4D cost volume and fed into a stacked
hourglass architecture for the disparity estimation. To simplify this
hourglass architecture, the authors in[13] adopted fewer skipping connections
in the decoder and designed a novel 4D cost volume by calculating group-wise
correlation. Next, [14] was proposed to predict image pairs with a high
resolution, different-level feature maps were employed to foster the multiple
cost volumes and then gradually connected to estimate the disparity map based
on a coarse-to-fine manner. More methods [15, 16, 17] were proposed to adopt
different strategies to foster the cost volume since it is a key factor for
the final prediction. Different from the traditional CNN architecture,
attention-based transformer [18] provides a new network architecture and
vision transformer [19] opens the path to utilize this module in the vision
field by encoding the image into many tokens. The works in [20, 21] started to
fuse the transformer and CNN for the disparity estimation and their results
showed a satisfactory performance by evaluating a public stereo endoscopic
dataset [22].

3D reconstruction recovers the depth information of the whole intra-operative
scene, which raises another issue, i.e., keeping the interested region while
removing other background points for an accurate registration between the pre-
operative model and the intra-operative target. Hence, another segmentation
network needs to be integrated to distinguish the background and the target by
predicting a binary mask. UNet [23] is the most representative model in
medical image segmentation, and it adopted a U-shape architecture with
downsampling and upsampling operation and implemented the fusion of different-
level features using skipping connection. Following the similar UNet
architecture, the authors in [24] proposed a multi-path feature fusion
strategy by combining two UNet networks, the authors in [25] integrated an
attention gate module to enhance the learning of target structures, and the
authors in [26] designed a nested UNet architecture by densely aggregating
different-scale features. The emerging Transformer module has also been
utilized in the segmentation field. In [27, 28, 29], the authors employed the
transformers as the encoder to extract the features, and then adopted the CNN
architecture as the decoder to predict the segmentation mask. More recently, a
fancy model named Segment Anything (SAM) [30] was released which had been
trained based on 11 million images and showed a strong generalization ability
in various segmentation tasks. With the explosion of big data, it can be
foreseen that such kind of models will lead a new era because of the promising
zero-shot performance.

Intra-operative bleeding is a risky situation that affects surgical quality
and post-operative recovery of patients, and it generally occurs due to
unconscious collisions between the surgical instruments and the delicate blood
vessel (i.e., the artery and the vein) during the operation. Hence, we propose
a markerless augmented reality framework to relieve the occurrence of this
situation. Different from the existing approaches such as the manual location
or the pre-installed markers, we combined the stereo reconstruction and
segmentation networks to locate the intra-operative soft tissue. The proposed
framework integrating the advanced neural networks can implement the
visualization of the pre-operative model, and it also performs the minimum
distance detection between the instruments and the blood vessel to avoid a
dangerous collision. Furthermore, it does not rely on the extra external
device, which means high generalization in other robotic systems and surgical
tasks. The main contributions can be summarized as follows:

1) A markerless augmented reality framework was proposed to visualize the pre-
operative model on intra-operative scenes, and it provides the minimum
distance detection between the surgical instruments and the delicate blood
vessel for safety.

2) A comprehensive evaluation of advanced neural networks in stereo
reconstruction and segmentation fields was performed to find out the best
combination to recover 3D information of the intra-operative blood vessel
accurately and fastly.

3) A user study involving ten human subjects who performed a robot-assisted
lymphadenectomy based on the da Vinci Research Kit in a dry lab, was achieved
to explore the usability of the proposed AR framework compared to the standard
setup.

The remainder of this paper is structured as follows. Section 2 describes the
details of our proposed augmented reality framework. In Section 3, it presents
the framework evaluation metrics and the specific experimental protocol of the
usability study, and the results are given in Section 4. Section 5 discusses
the findings and limitations of our work, and the conclusion of this paper and
future work are drawn in Section 6.

##  2 Methodology

Figure 1: The architecture of the markerless augmented reality framework. The
image pair is fed into two different networks to estimate the disparity map
and the binary mask. Then, the disparity pixels that belong to the blood
vessel can be reprojected to generate the 3D intra-operative blood vessel. The
pre-operative model can be overlapped on the corresponding region of the
endoscopic images by registration, and the minimum distance between the
instruments and the blood vessel can be detected based on the dVRK kinematics
and the 3D reconstructed blood vessel.

The proposed augmented reality framework is shown in Fig. 1, and it is
integrated into the popular da Vinci Research Kit (dVRK, Intuitive Surgical
Inc., US, and Johns Hopkins University). The stereo image pair is input into a
stereo reconstruction network to estimate the left disparity map, and the left
image is also input into a segmentation network to segment the blood vessel.
Then, the intra-operative blood vessel can be generated in the 3D space by
combining the disparity map and the binary mask. The pre-operative model is
utilized to perform the registration with the intra-operative blood vessel so
that it can be projected on the corresponding position of the endoscopic image
pairs to implement an AR effect. Furthermore, the minimum distances between
the surgical instruments and the soft tissue are calculated based on the dVRK
kinematics and the 3D position of the reconstructed blood vessel. The specific
description of this framework is given below.

###  2.1 dVRK system and calibrations

The dVRK system is an open-source robotic platform composed by the hardware of
the first generation of da Vince surgical system as well as the customized
software and electronics. It can be mainly divided into the leader side and
the follower side. There are two Patient Side Manipulators (PSMs) that mount
various surgical instruments such as the large needle driver at the follower
side, and a stereo endoscope (1920$\times$1080 resolution) mounted on an
Endoscopic Camera Manipulator (ECM) is used to capture in vivo surgical
scenes. By adjusting the position of the Setup Joint (SUJ) connecting these
robotic arms, the respective Remote Center of Motion (RCM) can be located at
the skin entry point of the abdomen [11]. On the other hand, the surgeon at
the leader side can observe the endoscopic scenes using a High Resolution
Stereo Viewer (HRSV) and remotely control the movements of PSMs and ECM by
operating Master Tool Manipulators (MTMs) as well as a foot pedal tray [31,
32].

Fig. 2 shows the details of our dVRK system, and it also presents the
reference frame definition adopted in our framework. The Cartesian position of
each instrument can be obtained from the dVRK kinematics, and a 9$\times$6
chessboard with a square length of 1cm is adopted for our calibrations. First,
the camera calibration is done based on Zhang’s calibration approach [33] to
generate the intrinsic and extrinsic parameters for the image rectification,
undistortion and projection. Then, a hand-to-hand calibration is conducted to
search for the rigid transformation between left and right end effectors based
on Horn’s method [34], since the position subscribed from the direct
kinematics is not accurate enough. By collecting 40 non-collinear points, we
can generate the rigid transformation matrix $T_{\emph{EE1}}^{\emph{EE2}}$ and
transfer the 3D points in _{ EE1}_ to _{ EE2}_. Furthermore, a hand-eye
calibration is performed to obtain the transformation between the left end
effector _{ EE2}_ and the left camera _{ L_CAM}_. We operate the left end
effector to point at 54 corner points of the chessboard to obtain the 3D
coordinates, and the corresponding 2D coordinates on the left image are
obtained using cv.findChessboardCorners and cv.cornerSubPix (OpenCV) functions
so that the transformation can be calculated based on the Random Sample
Consensus (RANSAC) scheme (cv.solvePnPRansac function) [35, 36]. Since our end
effector is referencing the frame _{ ECM}_, the transformation matrix
$T_{\emph{ECM}}^{\emph{L\\_CAM}}$ is generated based on the hand-eye
calibration.

Figure 2: The presentation of the dVRK system in a dry lab. In (a), the user
is operating the MTMs and observing the surgical scenes using HRSV at the
leader side, and the surgical instruments mounted on the PSMs are performing
the operation following the remote control of the user at the follower side.
In (b), it shows the three types of system calibrations, including camera
calibration, hand-hand calibration and hand-eye calibration.

###  2.2 Intra-operative blood vessel reconstruction

The stereo image is first rectified to align the polar lines in the horizon
axis, and the resolution is resized from 1920$\times$1080 to 640$\times$360 to
accelerate the framework. Then, the images are fed into a stereo
reconstruction network to estimate the disparity map [14]. In this way, we can
reproject the disparity map to generate the 3D intra-operative point cloud.
The conversion between the estimated disparity value $\hat{d}$ and the depth
value $\tilde{d}$ is formulated as,

$\tilde{d}(i,j)=\frac{bf}{\hat{d}(i,j)}$ (1)

where $i$,$j$ are the pixel position on the 2D image, $b$ is the baseline of
the stereo camera, and $f$ is the focal length.

However, the reconstructed point cloud involves the whole intra-operative
scene, and we need to extract the interested region (i.e., the blood vessel in
our case) for the registration. Considering that the estimated disparity map
is referencing the left rectified image, we propose to adopt another
segmentation network to generate the binary mask of the interested region. The
left rectified image is fed into the segmentation network to estimate the
indices of the blood vessel and the background. Next, we use the disparity
pixels that belong to the blood vessel by referencing these indices and
reproject them to the 3D space to reconstruct the intra-operative blood
vessel. The 3D position of the reconstructed blood vessel directly influences
the quality of the following registration and the distance detection, so a
comprehensive comparison study including the stereo reconstruction networks
and segmentation networks is provided in Section 3 to determine the best
combination in our framework. Finally, two post-processing approaches are
adopted to improve the segmentation estimation, including mask boundary
eroding and small object removal. Mask boundary eroding can remove the
possible misclassified pixels near the boundary of the blood vessel, and the
small object removal is used to remove the possible outliers in other regions,
which can refine the segmentation quality in some challenging cases such as
the blurred scenes caused by the fast movement of the instruments.

The recontructed 3D blood vessel is referencing the rectified left camera
coordinate system _{ Rec_L_CAM}_ , and we transform it to the reference _{
ECM}_ for the following distance calculation based on the equation,

$P_{\emph{intra\\_obj}}^{\emph{ECM}}=\left(T_{\emph{ECM}}^{\emph{L\\_{CAM}}}\right)^{-1}*\left(T_{\emph{L\\_CAM}}^{\emph{Rec\\_L\\_CAM}}\right)^{-1}*P_{\emph{intra\\_obj}}^{\emph{Rec\\_L\\_CAM}}$
(2)

where $P_{\emph{intra\\_obj}}^{\emph{Rec\\_L\\_CAM}}$ is the reconstructed 3D
points of the intra-operative blood vessel referencing the rectified left
camera coordinate system, $T_{\emph{L\\_CAM}}^{\emph{Rec\\_L\\_CAM}}$ is the
transformation between the unrectified left camera system and the rectified
left camera system obtained from the stereo image rectification, and
$T_{\emph{ECM}}^{\emph{L\\_{CAM}}}$ is obtained based on the hand-eye
calibration.

###  2.3 Registration between pre-operative and intra-operative targets

The pre-operative 3D model is generally captured using CT or MRI in the
hospital, and then using some software such as the 3D Slicer to segment the
interested region to generate the 3D structure. In our case, we utilize a 3D
modeling software, i.e., Blender, to create a pre-operative 3D model for
simplification since we don’t have an external device to capture these CT/MRI
slices. Next, we can perform the registration between the pre-operative model
and the intra-operative reconstructed point cloud. Here, the pre-operative
blood vessel is a mesh model while the intra-operative one is a point cloud,
so we sample plenty of points from the mesh model for the registration
process. Considering that there is an apparent position difference between the
pre-operative model and the intra-operative blood vessel in the initial state,
the global registration-based RANSAC algorithm [36] is adopted to conduct the
initial transformation, and it is only implemented once at the beginning.
Then, the local registration-based Iterative Closest Point (ICP) [37]
algorithm is performed to finetune the position of the pre-operative model. It
is found that this registration strategy can register the models accurately
and fastly in our experiment.

###  2.4 Distance detection and AR visualization

Our framework not only provides the augmented pre-operative model
visualization on intra-operative images, but also detects the minimum
distances between the surgical instruments and the blood vessel. After
aligning the position of the pre-operative model with the intra-operative
blood vessel, we could overlap the pre-operative model on the left intra-
operative scenes $P_{\emph{pre\\_obj}}^{\emph{L\\_img}}$ by the
transformation,

$P_{\emph{pre\\_obj}}^{\emph{L\\_img}}=K_{\emph{L}}*T_{\emph{ECM}}^{\emph{L\\_CAM}}*T_{\emph{BL}}^{\emph{ECM}}*P_{\emph{pre\\_obj}}^{\emph{BL}}$
(3)

where $P_{\emph{pre\\_obj}}^{\emph{BL}}$ is the 3D pre-operative points
referencing the Blender coordinate system {BL}, $T_{\emph{BL}}^{\emph{ECM}}$
is the transformation matrix generated by the registration,
$T_{\emph{ECM}}^{\emph{L\\_CAM}}$ is the matrix obtained from the hand-eye
calibration and $K_{\emph{L}}$ contains the intrinsic and distortion matrices
of the left camera obtained from the camera calibration. Similarly, we can use
the following equation to project the pre-operative model on the right intra-
operative scenes $P_{\emph{pre\\_obj}}^{\emph{R\\_img}}$,

$P_{\emph{pre\\_obj}}^{\emph{R\\_img}}=K_{\emph{R}}*T_{\emph{L\\_CAM}}^{\emph{R\\_CAM}}*T_{\emph{ECM}}^{\emph{L\\_CAM}}*T_{\emph{BL}}^{\emph{ECM}}*P_{\emph{pre\\_obj}}^{\emph{BL}}$
(4)

where $T_{\emph{L\\_CAM}}^{\emph{R\\_CAM}}$ is the transformation between the
left and right camera coordination systems, and $K_{\emph{R}}$ contains the
intrinsic and distortion matrices of the right camera. Then, we can observe
that the pre-operative model is overlapped on the corresponding regions of the
left and right images, respectively.

Finally, we calculate the minimum distance between the surface of the
instruments and the reconstructed blood vessel based on the fast k-nearest-
neighbor search strategy [38, 39]. The Cartesian position of the end effectors
and the RCM points of instruments are subscribed from the robot kinematics so
that we can model the instruments as cylinders with a radius of 4mm and sample
them as the point clouds (here, the position of PSM1 is aligned to PSM2 by the
hand-hand rigid transformation $T_{\emph{EE1}}^{\emph{EE2}}$). Two gauges are
provided in the left upper and right upper corners of intra-operative images,
and they can visualize the respective minimum distance of left and right
instruments. Also, the color of the pre-operative model changes automatically
according to the smaller distance by comparing the left and right minimum
distances to remind the surgeons during the operation.

##  3 Experimental protocol and performance metrics

###  3.1 Framework characterisation evaluation

####  3.1.1 Reconstruction and segmentation networks

The prerequisite for performing AR visualization and distance detection is
that the intra-operative position of soft tissue needs to be accurately
restored, so we explored 14 state-of-the-art methods in the stereo
reconstruction field to find out the best model that can be utilized in the
medical scenes. Among them, ELAS [40] is an optimation based method while
others [21, 41, 20, 15, 16, 12, 13, 17, 42, 43, 44, 45, 14] utilize the neural
networks. The stereo endoscopic dataset, SERV-CT [46], was adopted to conduct
the quantitative evaluation of these methods, and it contains sixteen image
pairs captured from porcine samples based on the dVSS and provides the dense
ground truth. To reconstruct the endoscopic scenes, we run these models based
on their official weights without any task-specific finetuning for the
generalization [47]. A set of accuracy-related metrics was chosen to evaluate
the reconstruction error, comparing the estimated depth with the provided
ground truth, both expressed in millimeters. The metrics include Median
Absolute Error (MeAE), Mean Absolute Error (MAE), Root Mean Square Error
(RMSE), Absolute Relative Error (Abs Rel), Squared Relative Error (Sq Rel), as
well as $\delta_{ratio}$ [48, 49].

$\text{MeAE}=\text{Median}\left\\{\tilde{d}(i,j)\in
S\mid|\tilde{d}(i,j)-d^{\prime}(i,j)|\right\\}$ (5)
$\text{MAE}=\frac{1}{|S|}\sum_{(i,j)}\left|\tilde{d}(i,j)-d^{\prime}(i,j)\right|$
(6)
$\text{RMSE}=\sqrt{\frac{1}{|S|}\sum_{(i,j)}\left|\tilde{d}(i,j)-d^{\prime}(i,j)\right|^{2}}$
(7)
$\text{Abs~{}Rel}=\frac{1}{|S|}\sum_{(i,j)}\frac{\left|\tilde{d}(i,j)-d^{\prime}(i,j)\right|}{d^{\prime}(i,j)}$
(8)
$\text{Sq~{}Rel}=\frac{1}{|S|}\sum_{(i,j)}\frac{\left|\tilde{d}(i,j)-d^{\prime}(i,j)\right|^{2}}{d^{\prime}(i,j)}$
(9) $\delta_{ratio}:\%\text{ of }\tilde{d}(i,j)\text{ s.t.
}\max\left(\frac{\tilde{d}(i,j)}{d^{\prime}(i,j)},\frac{d^{\prime}(i,j)}{\tilde{d}(i,j)}\right)<\tau$
(10)

where $S$ is the set of predicted depth values for each frame,
$\tilde{d}(i,j)$ is the predicted depth value related to pixel in position
$(i,j)$ and $d^{\prime}(i,j)$ is the ground truth of depth value. The last
metric evaluates the depth fluction error between the reconstructed points and
the ground truth, and three different thresholds
$\tau\in[1.25^{1},1.25^{2},1.25^{3}]$ were adopted. Unlike the other metrics,
the higher $\delta_{ratio}$ means the better reconstruction result. Also, we
provided the inference time of one frame when evaluating these models.

Another segmentation network is required to extract the interested soft
tissue, so 8 state-of-the-art segmentation methods [23, 24, 25, 26, 27, 28,
29, 30] were evaluated for the blood vessel segmentation. The recent
segmentation network SAM [30] has a strong generalization in different fields,
while other neural networks need to be trained. Hence, we captured six
endoscopic videos containing the 3D-printed blood vessel based on the dVRK
platform in our lab and extracted around 100 images from each video for the
manual annotation (551 frames in total). We performed the annotation using
Computer Vision Annotation Tool (CVAT) [50]. Six-fold cross validation was
adopted to train and evaluate the models. During the training process, we
cropped the images into 128$\times$128 patches and trained the models for 100
epochs based on a batch size of 64 (100000 patches in every epoch and 10$\%$
of images from the training images were used for the model validation). Two
methods [27, 28] loaded the pre-trained weight following the authors’ original
configuration while other models were trained from the scratch. The images
were also split into patches with the resolution of 128$\times$128 without
overlapping during the test phase. Common evaluation metrics in the
segmentation field were used for a comprehensive comparison, including Dice
coefficient ($\frac{2TP}{2TP+FP+FN}$), accuracy ($\frac{TP+TN}{TP+TN+FP+FN}$),
specificity ($\frac{TN}{TN+FP}$), sensitivity ($\frac{TP}{TP+FN}$),
precision($\frac{TP}{TP+FP}$), the area of the precision-recall (PR) curve
[51] as well as the inference time.

Figure 3: A simulated lymphadenectomy based on the dVRK platform in a dry lab.
The first row presents the standard endoscopic scenes, while the second row
shows the augmented scenes with AR assistance. The two gauges show the
respective minimum distance between the two instruments and the blood vessel.
The pre-operative model is overlapping on the intra-operative blood vessel,
and its color will change automatically following the smaller distance by
comparing the left and right distances. The defined task is to remove the ten
lymph nodes while not touching the delicate blood vessel for safety.

####  3.1.2 other characterisation

The quantitative performance of this framework needs to be measured for
practicality, and the experimental platform is based on an NVIDIA RTX 3080 GPU
in a local laptop. On the one hand, we provided the running time of the
proposed framework by calculating the time spent in 100 consecutive frames,
and we also gave the specific time distribution in each component of this
framework. On the other hand, we calculated the error distribution in
different components of this framework. The system calibrations based on the
dVRK introduce errors, and the respective performance metric can be formulated
as:

$\bullet$ Camera calibration error $E_{\text{cam}}$ considers the difference
between the coordinates of reprojected 3D points on the 2D plane and the
actual coordinates of the chessboard corners by calculating the root mean
square value,

$E_{\text{cam }}=\sqrt{\frac{1}{N_{\text{cam }}}\sum_{i=1}^{N_{\text{cam
}}}\left\|\varepsilon_{i}^{\text{pro}}-\varepsilon_{i}^{\text{act}}\right\|_{2}^{2}}$
(11)

where $\varepsilon_{i}^{\text{pro}}$ is the 2D points reprojected using the
camera calibration parameters, while $\varepsilon_{i}^{\text{act}}$ denotes
the actual 2D coordinates on the chessboard corners. $N_{\text{cam}}$ is the
number of the points used for the camera calibration, and $\|,\|_{2}$
represents the Euclidean norm.

$\bullet$ Hand-hand calibration error $E_{\text{hand\\_hand}}$ was calculated
based on the Cartesian position difference between the 3D points subscribed
from the left end effector and the same points transformed from the right end
effector based on the hand-to-hand matrix $T_{\emph{EE1}}^{\emph{EE2}}$,

$E_{\text{hand\\_hand}}=\sqrt{\frac{1}{N_{\text{hh}}}\sum_{i=1}^{N_{\text{hh}}}\left\|\rho_{i}^{L}-T_{\emph{EE1}}^{\emph{EE2}}\rho_{i}^{R}\right\|_{2}^{2}}$
(12)

where $\rho_{i}^{L}$ and $\rho_{i}^{R}$ are the 3D Cartesian positions of the
left and right end effectors, respectively. $N_{\text{hh}}$ is the number of
the 3D points used for the hand-hand calibration.

$\bullet$ Hand-eye calibration error $E_{\text{hand\\_eye}}$ was evaluated
based on the pixel position difference between the reprojected 2D coordinates
$\gamma_{i}^{\text{pro}}$ from the 3D points of the left end effector using
the hand-eye transformation matrix and the actual 2D coordinates
$\gamma_{i}^{\text{act}}$,

$E_{\text{hand\\_eye}}=\sqrt{\frac{1}{N_{\text{he}}}\sum_{i=1}^{N_{\text{he}}}\left\|\gamma_{i}^{\text{pro}}-\gamma_{i}^{\text{act}}\right\|_{2}^{2}}$
(13)

where $N_{\text{he}}$ is the number of the points adopted for the hand-eye
transformation matrix.

Furthermore, the errors of the reconstruction and segmentation networks have
been reported in the last subsection, and the registration process also
introduces errors. We calculated the Cartesian position error
$E_{\text{regis}}$ between the pre-operative blood vessel after registration
using the transformation matrix $T_{\emph{BL}}^{\emph{ECM}}$ and the
reconstructed blood vessel,

$E_{\text{regis}}=\sqrt{\frac{1}{N_{\text{re}}}\sum_{i=1}^{N_{\text{re}}}\left\|T_{\emph{BL}}^{\emph{ECM}}\varphi_{i}^{\text{pre-
op }}-\varphi_{i}^{\text{recon }}\right\|_{2}^{2}}$ (14)

where $\varphi_{i}^{\text{pre-op }}$ is the pre-operative 3D points, while
$\varphi_{i}^{\text{recon }}$ is the intra-operative 3D reconstructed points
referencing the frame _{ ECM}_. $N_{\text{re}}$ is the point number for the
registration.

Table 1: Quantitative depth estimation result based on the SERV-CT stereo endoscopic dataset (The image resolution is 720$\times$576). | MeAE (mm) | MAE (mm) | RMSE (mm) | Abs Rel | Sq Rel | $\delta<1.25^{1}$ | $\delta<1.25^{2}$ | $\delta<1.25^{3}$ | Time (s)
---|---|---|---|---|---|---|---|---|---
HybridStereoNet [21] | 46.98±19.32 | 54.30±16.86 | 73.66±13.79 | 0.67±0.08 | 70.27±18.33 | 0.14±0.07 | 0.24±0.12 | 0.39±0.21 | 0.52±0.01
ACVNet [40] | 8.43±13.46 | 17.88±12.24 | 28.26±14.52 | 0.20±0.12 | 10.12±8.65 | 0.69±0.19 | 0.72±0.19 | 0.77±0.20 | 0.42±0.01
STTR [20] | 3.82±6.81 | 15.65±10.24 | 42.26±22.28 | 0.22±0.20 | 31.88±33.58 | 0.83±0.18 | 0.88±0.14 | 0.94±0.06 | 0.43±0.00
ELAS [39] | 3.81±3.14 | 9.05±8.51 | 18.86±19.72 | 0.13±0.16 | 11.73±29.87 | 0.87±0.18 | 0.95±0.07 | 0.98±0.04 | 0.05±0.01
GraftNet [15] | 1.27±0.55 | 8.23±4.59 | 35.42±19.45 | 0.11±0.08 | 23.43±21.61 | 0.96±0.03 | 0.96±0.03 | 0.97±0.02 | 0.33±0.00
Cascade-Stereo [16] | 1.80±0.67 | 5.53±3.00 | 13.15±6.72 | 0.06±0.02 | 2.15±1.72 | 0.93±0.04 | 0.96±0.03 | 0.98±0.02 | 0.34±0.00
PSMNet [12] | 1.43±0.61 | 3.43±1.94 | 7.13±4.81 | 0.04±0.02 | 0.72±1.05 | 0.97±0.03 | 0.98±0.02 | 0.99±0.02 | 0.40±0.01
GwcNet [13] | 1.77±0.56 | 3.06±1.25 | 5.22±2.77 | 0.04±0.01 | 0.35±0.39 | 0.98±0.03 | 1.00±0.01 | 1.00±0.00 | 0.31±0.01
CFNet [17] | 1.29±0.49 | 3.04±2.13 | 6.99±7.21 | 0.04±0.05 | 2.01±6.16 | 0.98±0.02 | 0.99±0.02 | 1.00±0.01 | 0.29±0.00
W-Stereo-Disp [41] | 1.35±0.41 | 3.02±1.38 | 6.59±4.35 | 0.04±0.02 | 0.71±1.04 | 0.98±0.03 | 0.99±0.02 | 1.00±0.01 | 0.42±0.00
LEAStereo [42] | 1.44±0.82 | 2.96±1.28 | 6.29±2.60 | 0.04±0.01 | 0.58±0.44 | 0.98±0.02 | 1.00±0.01 | 1.00±0.00 | 0.52±0.01
IGEV-Stereo [43] | 1.12±0.47 | 2.59±1.18 | 5.64±3.10 | 0.03±0.01 | 0.40±0.36 | 0.98±0.02 | 1.00±0.01 | 1.00±0.00 | 0.32±0.00
CREStereo [44] | 1.32±0.71 | 2.38±1.58 | 4.18±2.90 | 0.03±0.01 | 0.23±0.27 | 0.99±0.01 | 1.00±0.00 | 1.00±0.00 | 1.14±0.01
HSM [14] | 1.10±0.60 | 2.05±1.17 | 3.72±2.08 | 0.02±0.01 | 0.17±0.17 | 0.99±0.02 | 1.00±0.00 | 1.00±0.00 | 0.03±0.00
Figure 4: Qualitative surgical scene reconstruction result. The reconstructed
3D surgical surfaces based on six representative models are provided.

###  3.2 Framework usability study

To explore the usability of our proposed framework, a common surgical
operation named lymphadenectomy was designed in a dry lab environment based on
the proposal of an oncology surgeon. As shown in Fig. 3, the 3D-printed soft
blood vessel and kidney were adopted to simulate the surgical scene, and the
defined task is to remove the ten lymph nodes one by one (the white soft
objects) near the blood vessel while not touching it. Frame 1 shows the
initial position of the instruments, then the users operate the left
manipulator to catch the left six lymph nodes and put them at the bottom-left
corner as shown in Frames 2 and 3, next the users operate the right
manipulator to remove the right four lymph nodes and put them at the same
corner as shown in Frame 4 and 5, finally the users move the instrument to the
initial position in Frame 6. Frame 3 and Frame 5 are challenging cases since
the instruments are more likely to collide with the blood vessel. Ten human
subjects (6 males and 4 females, aged between 21 and 28) who have a biomedical
background were invited to join our experiment. Before the experiment, the
users had spent 10 minutes for them to be familiar with the dVRK system. Then,
the experiment was repeated in three rounds and we analyzed the user data of
the last round to avoid the possible influence of the learning curve. In each
round, two different modalities were performed in a random sequence for each
user: Control (the standard endoscopic scene without the AR assistance), and
Experiment (the endoscopic scene with the AR assistance).

Our AR framework not only visualizes the corresponding pre-operative model on
the intra-operative blood vessel, but it also provides the visualization of
the minimum distances between the instruments and the blood vessel. We added
two gauges on the scenes to visualize the minimum distances (SA: 6cm means the
safe area and RA: 3cm means the risk area), and the color of the pre-operative
model changes following the smaller distance between the left and right
distances. Five performance metrics were utilized to observe if the AR
framework can help improve the surgical performance,

$\bullet$ Minimum distance $D_{\text{min}}$ between the instruments and the
blood vessel:

$D_{\text{min}}=\min\left\\{d_{1L},d_{2L},\cdots,d_{ML},d_{1R},d_{2R},\cdots,d_{MR}\right\\}$
(15)

where $d_{ML}$ is the minimum distance of the left instrument in the M-th
frame, while $d_{MR}$ denotes the minimum distance of the right instrument in
the M-th frame during the operation.

$\bullet$ Mean distance $D_{\text{mean}}$ when the instruments are in the risk
area of 3cm:

$D_{\text{mean}}=\frac{1}{\widetilde{M}+\widetilde{N}}\left(\sum_{m=1}^{\widetilde{M}}d_{mL}+\sum_{n=1}^{\widetilde{N}}d_{nR}\right)$
(16)

where $\widetilde{M}$ is the number of the minimum distance points of the left
instrument when they are smaller than 3cm, while $\widetilde{N}$ represents
the number when the points of the right instrument are less than 3cm.

$\bullet$ Collision number $N_{\text{c}}$ when the distance points are smaller
than the threshold $r$:

$N_{\text{c}}=\sum_{m=1}^{M}\left\\{1\mid
d_{mL}<r\right\\}+\sum_{m=1}^{M}\left\\{1\mid d_{mR}<r\right\\}$ (17)

where $r$ is defined as 0.5cm in our case. Here, we regard it as one time of
collision if the points keep smaller than 0.5cm for one consecutive second
during the task.

$\bullet$ Overall movement path $S_{\text{p}}$ of the instruments during the
operation:

$S_{p}=\sum_{m=2}^{M}\left(\left\|C_{L}^{m}-C_{L}^{m-1}\right\|_{2}+\left\|C_{R}^{m}-C_{R}^{m-1}\right\|_{2}\right)$
(18)

where $C_{L}^{m}$ is the 3D Cartesian coordinate of the left end effector in
the m-th frame, while $C_{R}^{m}$ is the 3D coordinate of the right one in the
m-th frame.

$\bullet$ Execution time $T_{\text{exe }}$ to perform the complete operation:

$T_{\text{exe }}=T_{\text{end }}-T_{\text{start }}$ (19)

where $T_{\text{start }}$ and $T_{\text{end }}$ are the start time and the end
time of the task, respectively.

Table 2: Quantitative segmentation result using the self-made dataset (resolution: 1920$\times 1080$) based on 6-fold cross validation. | DSC | Accuracy | Specificity | Sensitivity | Precision | Area of PR Curve | Time (s)
---|---|---|---|---|---|---|---
Attention UNet [25] | 0.6708±0.2296 | 0.9089±0.0778 | 0.9059±0.0856 | 0.9486±0.0404 | 0.5860±0.3075 | 0.7974±0.1324 | 0.3509±0.0018
Segment Anything [30] | 0.9661±0.0372 | 0.9957±0.0041 | 0.9997±0.0003 | 0.9404±0.0633 | 0.9958±0.0049 | N/A | 0.6219±0.0071
UNETR [29] | 0.9780±0.0082 | 0.9971±0.0010 | 0.9986±0.0006 | 0.9757±0.0116 | 0.9805±0.0093 | 0.9931±0.0063 | 0.2056±0.0014
SETR [28] | 0.9794±0.0076 | 0.9973±0.0009 | 0.9989±0.0004 | 0.9747±0.0126 | 0.9843±0.0064 | 0.9933±0.0068 | 0.1970±0.0010
TransUNet [27] | 0.9823±0.0059 | 0.9976±0.0006 | 0.9988±0.0006 | 0.9818±0.0101 | 0.9829±0.0085 | 0.9985±0.0015 | 0.3066±0.0029
UNet++ [26] | 0.9825±0.0062 | 0.9976±0.0007 | 0.9986±0.0007 | 0.9842±0.0080 | 0.9809±0.0099 | 0.9984±0.0017 | 0.3498±0.0016
LadderNet [24] | 0.9835±0.0061 | 0.9978±0.0007 | 0.9989±0.0005 | 0.9824±0.0100 | 0.9846±0.0071 | 0.9987±0.0017 | 0.0898±0.0012
UNet [23] | 0.9855±0.0038 | 0.9980±0.0005 | 0.9990±0.0004 | 0.9842±0.0072 | 0.9869±0.0059 | 0.9990±0.0010 | 0.3178±0.0017
Figure 5: Qualitative segmentation result using a self-made dataset containing
the 3D-printed blood vessel captured from the dVRK platform.

To investigate the feasibility and friendliness of the AR framework, the users
were invited to fill in a System Usability Scale (SUS) questionnaire
containing ten typical questions [52] after the experiment. They evaluated the
two systems (the standard system and the extended system with the AR
assistance) by giving a score for each question (from score 1: strongly
disagreement to score 5: strongly agreement), and the final SUS score of each
user can be calculated as,

$\text{SUS}_{\text{score
}}=\left(\sum_{k=1,3,5,7,9}\left(S_{k}-1\right)+\sum_{k=2,4,6,8,10}\left(5-S_{k}\right)\right)*2.5$
(20)

where $S_{k}$ denotes the score of the k-th question provided by the user.

Table 3: The running time distribution of the framework (the image resolution is resized into 640$\times$360) Phase | Time (s)
---|---
Stereo image preprocessing | 0.0095±0.0017
Disparity map estimation | 0.0365±0.0018
Binary mask estimation with postprocessing | 0.0470±0.0015
Point cloud generation and alignment | 0.0032±0.0004
Distance calculation | 0.0158±0.0020
Registration between pre-op and intra-op targets | 0.0015±0.0002
Augmented reality visualization | 0.0312±0.0043
Whole pipeline | 0.1448±0.0079
Table 4: The error distribution in the framework. Component | Metric | Value / Unit
---|---|---
Camera calibration | $E_{\text{cam}}$ | 0.60±0.17 (pixels)
Hand-hand calibration | $E_{\text{hand\\_hand}}$ | 0.10±0.05 (cm)
Hand-eye calibration | $E_{\text{hand\\_eye}}$ | 1.64±0.80 (pixels)
Reconstruction | MAE | 0.205±0.117 (cm)
Segmentation | Dice | 0.9855±0.0038
Registration | $E_{\text{regis}}$ | 0.4328±0.0385 (cm)

##  4 Results

###  4.1 Results on framework characterisation evaluation

####  4.1.1 Reconstruction and segmentation networks

Table 1 gives the quantitative evaluation result using these advanced depth
estimation models. The model HSM[14] gets the best performance in both
accuracy and inference time. The qualitative comparison based on three frames
was also provided in Fig. 4. It can be observed that HSM can reconstruct
smoother soft tissue surfaces with fewer outliers, so we chose this method for
the depth estimation in our framework.

The quantitative segmentation result was provided in Table 2, and the
qualitative comparison result was shown in Fig. 5. Here, we provided the
specific boxes as the prompt for the mask estimation of the SAM model, and the
area of PR curve is not applicable for this model since it predicts the pixel
classification directly instead of the probability. We noticed that UNet could
provide a reliable segmentation quality in our phantom environment, so we
adopted this model to estimate the mask in our experiment.

Figure 6: The data of the ten human subjects based on five different metrics.
“Control” means the users complete the operation based on the standard
endoscopic scenes, while “Experiment” is based on the scenes with the AR
assistance. The result of the Wilcoxon signed-rank test is shown as
$ns:0.05<p\leq 1$, $*:0.01<p\leq 0.05$, $**:0.001<p\leq 0.01$,
$***:0.0001<p\leq 0.001$, and $***~{}*:p\leq 0.0001$.

####  4.1.2 Other characterisation

Table 3 presents the running time distribution of this framework. Among them,
stereo image preprocessing includes image subscribing, rectification and
resizing from 1920$\times$1080 to 640$\times$360 based on a fixed scale
factor. It can be noticed that the three phases, i.e., the disparity map
estimation, the binary mask estimation and the AR visualization, spent the
most time in this framework, and the whole time to process one frame takes
0.1448±0.0079 (6.91FPS), which can provide a smooth visual feedback during the
operation. Also, Table 4 shows the error distribution in this framework. It
can be noted that these errors are small enough in our experiment.

###  4.2 Results on framework usability study

Fig. 6 presents the box plots of the user data, and the Wilcoxon signed-rank
test (p$<$0.05) is conducted to explore if there is a significant difference
between the control modality and the experiment modality. It shows that there
are significant differences when referencing the minimum distance
$D_{\text{min}}$, the mean distance $D_{\text{mean}}$ and the collision number
$N_{\text{c}}$. Table 5 also provides the average values using the five
metrics. With the AR assistance, the minimum distance increases from 0.0472cm
to 0.1864cm, and the mean distance in the risk area also increases from
1.3387cm to 1.4641cm. When considering the collision number, the value reduces
from 23.8 to 13.7. The statistical differences show that the AR assistance can
help reduce the collision risk between the instruments and the blood vessel
when operating. Furthermore, there is no statistical difference when
referencing the overall path $S_{\text{p}}$ and execution time $T_{\text{exe
}}$, which indicates that the AR assistance does not introduce an extra load
in operating the robot.

The final result of the SUS questionnaire is given in Fig. 7. The average SUS
score of the control modality is 66, while the experiment modality has a
higher SUS score of 73. The statistical test presents there is a significant
difference between the two modalities (p = 0.0104).

Table 5: The average values and p value based on the users’ data. | Control | Experiment | P value
---|---|---|---
Minimum Distance $D_{\text{min}}$ (cm) | 0.0472 | 0.1864 | 0.0098
Mean Distance $D_{\text{mean}}$ (cm) | 1.3387 | 1.4641 | 0.0371
Collision Number $N_{\text{c}}$ | 23.8 | 13.7 | 0.0077
Overall Path $S_{\text{p}}$ (cm) | 267.28 | 264.32 | 0.5566
Execution Time $T_{\text{exe }}$ (s) | 175.11 | 167.44 | 0.2324

##  5 Discussion

Augmented reality is a popular direction in various fields, including robotic
surgery since it provides the possibility to enhance safety during the
operation, such as the distance visualization in our case or the visualization
of some invisible tissues during the operation. However, an unsolved challenge
is to locate the region of interest so that the pre-operative model can be
registered on the corresponding intra-operative soft tissues or organs. In
this work, we proposed a vision-based markerless location approach to perform
the AR effect with the distance visualization on the intra-operative scenes,
which releases the burden of the manual alignment or landmarks. It can be
integrated into other tasks and platforms because of the high independence on
the specific device.

Advanced neural networks have been proposed for different vision tasks with
promising performance by evaluating some public or self-made datasets, and it
is worth integrating them to implement applications in practice. Hence, we
compared the state-of-the-art networks in the stereo reconstruction and
segmentation fields to track the soft tissue in the 3D intra-operative space.
We adopted the model HSM [14] in our case since it shows a reliable depth
estimation in both depth and inference time. Furthermore, we added a
segmentation mask to extract the region of interest from the whole scene. The
segmentation performance relies on the specific scenes, so we captured and
annotated an endoscopic dataset based on the dVRK platform in our lab. By
referencing the segmentation results of 6-fold cross validation, we noticed
that UNet [23] can provide a more reliable segmentation quality compared to
other models, so we adopted it in our framework. But it should be noted that
the segmentation quality is influenced by different scenes and training
strategies. For instance, the transformer-based networks [27, 28, 29] may
produce a better estimation with a large of annotated training images, which
needs to be compared in the specific tasks. In particular, we evaluated the
recently emerging big model SAM [30]. Although it does not perform a better
result compared to some other models, it should be pointed out that this model
has a strong generalization ability in different scenes without finetuning.
With the rise of big models and data, it can be foreseen that such kind of
model may dominate the vision tasks since it releases the burden of annotation
and training.

Figure 7: The specific SUS score distribution provided by the users as well as
the average SUS scores in two different modalities.

By simulating a robot-assisted lymphadenectomy based on the dVRK platform, we
obtained some surgical data from ten human subjects. It can be observed that
the proposed AR framework can increase the distance and reduce the collisions
between the instruments and the delicate blood vessel when operating.
Moreover, it does not introduce the extra physical and cognitive load since
the overall path and the complete time are similar in two different
modalities. Based on the feedback from the users, sometimes they could not
catch the lymph nodes well caused by the inaccurate depth perception when they
operated. Under this circumstance, the color information of the pre-operative
model can assist them in judging the proper occasion to catch the lymph nodes,
especially the pink and red colors (the distances are within 1cm and 0.5cm,
respectively), which can enhance their confidence and accuracy when catching
the objects. Finally, the average SUS score given by the users increases by 7%
when adopting the AR assistance, which means the proposed AR framework is
friendly to the users. Nevertheless, one limitation comes from the modeling of
surgical instruments. There are many types of instruments in real surgery,
such as monopolar scissors, bipolar forceps, Cadiere forceps, clip appliers
and needle drivers, which means the grippers of the instruments are different,
and modeling them into a cylinder introduces slight errors. A possible
solution is to add an extra neural network to recognize the tips of the
instruments [53] so that the modeling can be more accurate even though it may
slow down the framework. Another limitation comes from the phantom environment
for the simulated surgical operation. In clinical practice, the in vivo
environment of patients will be more complex and dynamic.

##  6 CONCLUSION

This paper proposed a markerless augmented reality framework to implement a
pre-operative structure visualization on the intra-operative scenes, and it
also provides the minimum distance detection between the instruments and the
delicate blood vessel for safety. It can be integrated into other existing
robotic platforms and tasks because it does not rely on specific devices.
Comprehensive comparison studies are performed to explore the best combination
for intra-operative blood vessel reconstruction, and the framework usability
evaluation from ten human subjects presents that the proposed framework can
enhance safety during the operation and does not introduce extra burden, which
shows its potential in clinical applications.

Augmented reality provides the possibility to enhance surgical safety based on
visual feedback, and another popular direction, virtual fixtures, can also
enhance safety by providing force feedback. In the next step, we will conduct
the virtual fixtures based on the dVRK platform and compare the difference
between the visual feedback and the force feedback in surgical assistance.

## References

  * [1] X. Hu, F. R. y Baena, and F. Cutolo, “Head-mounted augmented reality platform for markerless orthopaedic navigation,” _IEEE Journal of Biomedical and Health Informatics_ , vol. 26, no. 2, pp. 910–921, 2021.
  * [2] Z. Chen, A. Marzullo, D. Alberti, E. Lievore, M. Fontana, O. De Cobelli, G. Musi, G. Ferrigno, and E. De Momi, “Frsr: Framework for real-time scene reconstruction in robot-assisted minimally invasive surgery,” _Computers in Biology and Medicine_ , p. 107121, 2023.
  * [3] R. Hussain, A. Lalande, C. Guigou, and A. Bozorg-Grayeli, “Contribution of augmented reality to minimally invasive computer-assisted cranial base surgery,” _IEEE journal of biomedical and health informatics_ , vol. 24, no. 7, pp. 2093–2106, 2019.
  * [4] L. Bianchi, F. Chessa, A. Angiolini, L. Cercenelli, S. Lodi, B. Bortolani, E. Molinaroli, C. Casablanca, M. Droghetti, C. Gaudiano _et al._ , “The use of augmented reality to guide the intraoperative frozen section during robot-assisted radical prostatectomy,” _European Urology_ , vol. 80, no. 4, pp. 480–488, 2021.
  * [5] R. Schiavina, L. Bianchi, S. Lodi, L. Cercenelli, F. Chessa, B. Bortolani, C. Gaudiano, C. Casablanca, M. Droghetti, A. Porreca _et al._ , “Real-time augmented reality three-dimensional guided robotic radical prostatectomy: preliminary experience and evaluation of the impact on surgical planning,” _European urology focus_ , vol. 7, no. 6, pp. 1260–1267, 2021.
  * [6] D. Lee, H.-J. Kong, D. Kim, J. W. Yi, Y. J. Chai, K. E. Lee, and H. C. Kim, “Preliminary study on application of augmented reality visualization in robotic thyroid surgery,” _Annals of surgical treatment and research_ , vol. 95, no. 6, pp. 297–302, 2018.
  * [7] J. Y. Chan, F. C. Holsinger, S. Liu, J. M. Sorger, M. Azizian, and R. K. Tsang, “Augmented reality for image guidance in transoral robotic surgery,” _Journal of Robotic Surgery_ , vol. 14, pp. 579–583, 2020.
  * [8] P. De Backer, C. Van Praet, J. Simoens, M. P. Lores, H. Creemers, K. Mestdagh, C. Allaeys, S. Vermijs, P. Piazza, A. Mottaran _et al._ , “Improving augmented reality through deep learning: Real-time instrument delineation in robotic renal surgery,” _European Urology_ , 2023.
  * [9] T. Wendler, F. W. van Leeuwen, N. Navab, and M. N. van Oosterom, “How molecular imaging will enable robotic precision surgery: The role of artificial intelligence, augmented reality, and navigation,” _European Journal of Nuclear Medicine and Molecular Imaging_ , vol. 48, no. 13, pp. 4201–4224, 2021.
  * [10] L. Qian, J. Y. Wu, S. P. DiMaio, N. Navab, and P. Kazanzides, “A review of augmented reality in robotic-assisted surgery,” _IEEE Transactions on Medical Robotics and Bionics_ , vol. 2, no. 1, pp. 1–16, 2019.
  * [11] V. Penza, E. De Momi, N. Enayati, T. Chupin, J. Ortiz, and L. S. Mattos, “Envisors: Enhanced vision system for robotic surgery. a user-defined safety volume tracking to minimize the risk of intraoperative bleeding,” _Frontiers in Robotics and AI_ , vol. 4, p. 15, 2017.
  * [12] J.-R. Chang and Y.-S. Chen, “Pyramid stereo matching network,” in _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_ , 2018, pp. 5410–5418.
  * [13] X. Guo, K. Yang, W. Yang, X. Wang, and H. Li, “Group-wise correlation stereo network,” in _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_ , 2019, pp. 3273–3282.
  * [14] G. Yang, J. Manela, M. Happold, and D. Ramanan, “Hierarchical deep stereo matching on high-resolution images,” in _The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)_ , June 2019.
  * [15] B. Liu, H. Yu, and G. Qi, “Graftnet: Towards domain generalized stereo matching with a broad-spectrum and task-oriented feature,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 2022, pp. 13 012–13 021.
  * [16] X. Gu, Z. Fan, S. Zhu, Z. Dai, F. Tan, and P. Tan, “Cascade cost volume for high-resolution multi-view stereo and stereo matching,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 2020, pp. 2495–2504.
  * [17] Z. Shen, Y. Dai, and Z. Rao, “Cfnet: Cascade and fused cost volume for robust stereo matching,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 2021, pp. 13 906–13 915.
  * [18] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,” _Advances in neural information processing systems_ , vol. 30, 2017.
  * [19] A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly _et al._ , “An image is worth 16x16 words: Transformers for image recognition at scale,” _arXiv preprint arXiv:2010.11929_ , 2020.
  * [20] Z. Li, X. Liu, N. Drenkow, A. Ding, F. X. Creighton, R. H. Taylor, and M. Unberath, “Revisiting stereo depth estimation from a sequence-to-sequence perspective with transformers,” in _Proceedings of the IEEE/CVF International Conference on Computer Vision_ , 2021, pp. 6197–6206.
  * [21] X. Cheng, Y. Zhong, M. Harandi, T. Drummond, Z. Wang, and Z. Ge, “Deep laparoscopic stereo matching with transformers,” in _Medical Image Computing and Computer Assisted Intervention–MICCAI 2022: 25th International Conference, Singapore, September 18–22, 2022, Proceedings, Part VII_. Springer, 2022, pp. 464–474.
  * [22] M. Allan, J. Mcleod, C. Wang, J. C. Rosenthal, Z. Hu, N. Gard, P. Eisert, K. X. Fu, T. Zeffiro, W. Xia _et al._ , “Stereo correspondence and reconstruction of endoscopic data challenge,” _arXiv preprint arXiv:2101.01133_ , 2021.
  * [23] O. Ronneberger, P. Fischer, and T. Brox, “U-net: Convolutional networks for biomedical image segmentation,” in _Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18_. Springer, 2015, pp. 234–241.
  * [24] J. Zhuang, “Laddernet: Multi-path networks based on u-net for medical image segmentation,” _arXiv preprint arXiv:1810.07810_ , 2018.
  * [25] O. Oktay, J. Schlemper, L. L. Folgoc, M. Lee, M. Heinrich, K. Misawa, K. Mori, S. McDonagh, N. Y. Hammerla, B. Kainz _et al._ , “Attention u-net: Learning where to look for the pancreas,” _arXiv preprint arXiv:1804.03999_ , 2018.
  * [26] Z. Zhou, M. M. R. Siddiquee, N. Tajbakhsh, and J. Liang, “Unet++: Redesigning skip connections to exploit multiscale features in image segmentation,” _IEEE transactions on medical imaging_ , vol. 39, no. 6, pp. 1856–1867, 2019.
  * [27] J. Chen, Y. Lu, Q. Yu, X. Luo, E. Adeli, Y. Wang, L. Lu, A. L. Yuille, and Y. Zhou, “Transunet: Transformers make strong encoders for medical image segmentation,” _arXiv preprint arXiv:2102.04306_ , 2021.
  * [28] S. Zheng, J. Lu, H. Zhao, X. Zhu, Z. Luo, Y. Wang, Y. Fu, J. Feng, T. Xiang, P. H. Torr _et al._ , “Rethinking semantic segmentation from a sequence-to-sequence perspective with transformers,” in _Proceedings of the IEEE/CVF conference on computer vision and pattern recognition_ , 2021, pp. 6881–6890.
  * [29] A. Hatamizadeh, Y. Tang, V. Nath, D. Yang, A. Myronenko, B. Landman, H. R. Roth, and D. Xu, “Unetr: Transformers for 3d medical image segmentation,” in _Proceedings of the IEEE/CVF winter conference on applications of computer vision_ , 2022, pp. 574–584.
  * [30] A. Kirillov, E. Mintun, N. Ravi, H. Mao, C. Rolland, L. Gustafson, T. Xiao, S. Whitehead, A. C. Berg, W.-Y. Lo _et al._ , “Segment anything,” _arXiv preprint arXiv:2304.02643_ , 2023.
  * [31] T. Da Col, A. Mariani, A. Deguet, A. Menciassi, P. Kazanzides, and E. De Momi, “Scan: System for camera autonomous navigation in robotic-assisted surgery,” in _2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)_. IEEE, 2020, pp. 2996–3002.
  * [32] Z. Chen, S. Terlizzi, T. Da Col, A. Marzullo, M. Catellani, G. Ferrigno, and E. De Momi, “Robot-assisted ex vivo neobladder reconstruction: preliminary results of surgical skill evaluation,” _International journal of computer assisted radiology and surgery_ , vol. 17, no. 12, pp. 2315–2323, 2022.
  * [33] Z. Zhang, “A flexible new technique for camera calibration,” _IEEE Transactions on pattern analysis and machine intelligence_ , vol. 22, no. 11, pp. 1330–1334, 2000.
  * [34] B. K. Horn, “Closed-form solution of absolute orientation using unit quaternions,” _Josa a_ , vol. 4, no. 4, pp. 629–642, 1987.
  * [35] G. Bradski, “The OpenCV Library,” _Dr. Dobb’s Journal of Software Tools_ , 2000.
  * [36] M. A. Fischler and R. C. Bolles, “Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography,” _Communications of the ACM_ , vol. 24, no. 6, pp. 381–395, 1981.
  * [37] P. J. Besl and N. D. McKay, “Method for registration of 3-d shapes,” in _Sensor fusion IV: control paradigms and data structures_ , vol. 1611. Spie, 1992, pp. 586–606.
  * [38] F. Williams, “Point cloud utils,” 2022, https://www.github.com/fwilliams/point-cloud-utils.
  * [39] V. Garcia, E. Debreuve, and M. Barlaud, “Fast k nearest neighbor search using gpu,” in _2008 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops_. IEEE, 2008, pp. 1–6.
  * [40] A. Geiger, M. Roser, and R. Urtasun, “Efficient large-scale stereo matching,” in _Computer Vision–ACCV 2010: 10th Asian Conference on Computer Vision, Queenstown, New Zealand, November 8-12, 2010, Revised Selected Papers, Part I 10_. Springer, 2011, pp. 25–38.
  * [41] G. Xu, J. Cheng, P. Guo, and X. Yang, “Attention concatenation volume for accurate and efficient stereo matching,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 2022, pp. 12 981–12 990.
  * [42] D. Garg, Y. Wang, B. Hariharan, M. Campbell, K. Q. Weinberger, and W.-L. Chao, “Wasserstein distances for stereo disparity estimation,” _Advances in Neural Information Processing Systems_ , vol. 33, pp. 22 517–22 529, 2020.
  * [43] X. Cheng, Y. Zhong, M. Harandi, Y. Dai, X. Chang, H. Li, T. Drummond, and Z. Ge, “Hierarchical neural architecture search for deep stereo matching,” _Advances in Neural Information Processing Systems_ , vol. 33, 2020.
  * [44] G. Xu, X. Wang, X. Ding, and X. Yang, “Iterative geometry encoding volume for stereo matching,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 2023.
  * [45] J. Li, P. Wang, P. Xiong, T. Cai, Z. Yan, L. Yang, J. Liu, H. Fan, and S. Liu, “Practical stereo matching via cascaded recurrent network with adaptive correlation,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 2022, pp. 16 263–16 272.
  * [46] P. E. Edwards, D. Psychogyios, S. Speidel, L. Maier-Hein, and D. Stoyanov, “Serv-ct: A disparity dataset from cone-beam ct for validation of endoscopic 3d reconstruction,” _Medical image analysis_ , vol. 76, p. 102302, 2022.
  * [47] J. Lu, A. Jayakumari, F. Richter, Y. Li, and M. C. Yip, “Super deep: A surgical perception framework for robotic tissue manipulation using deep learning for feature extraction,” in _2021 IEEE International Conference on Robotics and Automation (ICRA)_. IEEE, 2021, pp. 4783–4789.
  * [48] D. Eigen, C. Puhrsch, and R. Fergus, “Depth map prediction from a single image using a multi-scale deep network,” _Advances in neural information processing systems_ , vol. 27, 2014.
  * [49] C. Zhao, Q. Sun, C. Zhang, Y. Tang, and F. Qian, “Monocular depth estimation based on deep learning: An overview,” _Science China Technological Sciences_ , vol. 63, no. 9, pp. 1612–1627, 2020.
  * [50] B. Sekachev, N. Manovich, M. Zhiltsov, A. Zhavoronkov, D. Kalinin, B. Hoff, D. Kruchinin, A. Zankevich, and D. Sidnev, “opencv/cvat: v1. 1.0,” _Zenodo_ , 2020.
  * [51] A. Venugopal, S. Moccia, S. Foti, A. Routray, R. A. MacLachlan, A. Perin, L. S. Mattos, A. K. Yu, J. Leonardo, E. De Momi _et al._ , “Real-time vessel segmentation and reconstruction for virtual fixtures for an active handheld microneurosurgical instrument,” _International Journal of Computer Assisted Radiology and Surgery_ , vol. 17, no. 6, pp. 1069–1077, 2022.
  * [52] J. Brooke _et al._ , “Sus-a quick and dirty usability scale,” _Usability evaluation in industry_ , vol. 189, no. 194, pp. 4–7, 1996.
  * [53] R. Moccia, C. Iacono, B. Siciliano, and F. Ficuciello, “Vision-based dynamic virtual fixtures for tools collision avoidance in robotic surgery,” _IEEE Robotics and Automation Letters_ , vol. 5, no. 2, pp. 1650–1655, 2020.


# Existence of solutions for a biological model using topological degree
theory
Existencia de soluciones para un modelo biológico empeando la teoría de grado
topológico

Carlos Héctor Daniel Alliera Departamento de Matemática, FCEyN - Universidad
de Buenos Aires, Pabellón I, Ciudad Universitaria, Buenos Aires, Argentina,
<EMAIL_ADDRESS>http://cms.dm.uba.ar

###### Abstract

Topological degree theory is a useful tool for studying systems of
differential equations. In this work, a biological model is considered.
Specifically, we prove the existence of positive $T$-periodic solutions of a
system of delay differential equations for a model with feedback arising on
Circadian oscillations in the Drosophila period gene protein.
Keywords: Differential equations with delay; Periodic solutions; Models with
feedback; Topological degree.

Resumen

La teoría de grado topológico es una herramienta útil para estudiar sistemas
de ecuaciones diferenciales. En este trabajo analizamos un modelo biológico;
específicamente, probamos la existencia de soluciones positivas $T$-periódicas
de un sistema de ecuaciones diferenciales con retardo basado en el modelo
auto-regulado de los ciclos Circadianos de proteínas a nivel genético de la
mosca de la fruta (Drosophila).
Palabras Clave: Ecuaciones diferenciales con retardo; Soluciones periódicas,
Modelos auto-regulados; Grado topológico.

MSC 2010: $34K13$, $92B05$.

##  1 Introduction

Let us consider a model proposed by Goldbeter [1], who showed the variation on
PER: Period of messenger of Ribo-Nucleic Acid (mRNA) in Drosophila (often
called “fruit flies”) related to circadian rhythms. Here, a nonautonomous
version of the model is considered with the aim of proving the existence of
periodic solutions by means of a powerful topological tool: the Leray-Schauder
degree. In the original model, the existence of a positive steady state can be
shown, under appropriate conditions, by the use of the Brouwer degree. As we
shall see, when the parameters are replaced by periodic functions, essentially
the same conditions yield the existence of positive periodic solutions.

Figure 1: The model for the circadian variation in PER.

##  2 The model

###  2.1 General features

  1. I)

This negative feedback will be described by an equation of Hill type in which
$n$ denotes the degree of cooperativity, and $K(t)$, the threshold repression
function.

  2. II)

To simplify the model, we consider that $P_{N}$ behaves directly as a
repressor.

  3. III)

The constants $K,K_{i}$ and $V_{j}$ denote the maximum rate and Michaelis
constant of the kinase(s) and phosphatase(s) involved in the reversible
phosphorylation of $P_{0}$, into $P_{1}$, and of $P_{1}$, into $P_{2}$ are not
negative.

  4. IV)

Maximum accumulation rate of cytosol is denoted by $V_{s}$.

  5. V)

Cytosol is degraded enzymically, in a Michaelian manner, at a maximum rate
$V_{m}$.

  6. VI)

Functions of this system are:

    1. (a)

Cytosolic concentration is denoted by $M$.

    2. (b)

We consider only three states of the protein: unphosphorylated ($P_{0}$),
monophosphorylated ($P_{1}$) and bisphosphorylated ($P_{2}$).

    3. (c)

Fully phosphorylated form of PER ($P_{2}$) is degraded in a Michaelian manner,
at a maximum rate $V_{d}$ and also transported into the nucleus, at a rate
characterized by the apparent first-order rate constant $k_{1}$.

  7. VII)

The rate of synthesis of PER, proportional to $M$, is characterized by an
apparent first-order rate constant $K_{s}$.

  8. VIII)

Transport of the nuclear, bisphosphorylated form of PER ($P_{N}$) into the
cytosol is characterized by the apparent first-order rate constant $k_{2}$.

  9. IX)

The model could be readily extended to include a larger number of
phosphorylated residues.

With this in mind, our non-autonomous version of Goldbeter’s system reads:

$\begin{array}[]{l}\dfrac{dM}{dt}=\dfrac{V_{S}(t)K_{1}(t)^{n}}{K_{1}^{n}(t)+P_{N}(t)^{n}}-\dfrac{V_{m}(t)M(t)}{K_{m_{1}}(t)+M(t)},\\\
\\\
\dfrac{dP_{0}}{dt}=K_{s}(t)M(t)+\dfrac{V_{2}(t)P_{1}(t)}{K_{2}(t)+P_{1}(t)}-\dfrac{V_{1}(t)P_{0}(t)}{K_{1}(t)+P_{0}(t)},\\\
\\\
\dfrac{dP_{1}}{dt}=\dfrac{V_{1}(t)P_{0}(t)}{K_{1}(t)+P_{0}(t)}+\dfrac{V_{4}(t)P_{2}(t)}{K_{4}(t)+P_{2}(t)}-P_{1}(t)\left(\dfrac{V_{2}(t)}{K_{2}(t)+P_{1}(t)}+\dfrac{V_{3}(t)}{K_{3}(t)+P_{1}(t)}\right),\\\
\\\
\dfrac{dP_{2}}{dt}=\dfrac{V_{3}(t)P_{1}(t)}{K_{3}(t)+P_{1}(t)}+k_{2}(t)P_{N}(t)-P_{2}(t)\left(k_{1}(t)+\dfrac{V_{4}(t)}{K_{4}(t)+P_{2}(t)}+\dfrac{V_{d}(t)}{K_{d}(t)+P_{2}(t)}\right),\\\
\\\ \dfrac{dP_{N}}{dt}=k_{1}(t)P_{2}(t)-k_{2}(t)P_{N}(t)\end{array}$ (1)

where $K_{i},\ i=1,2,3,4,d,m_{1}$, $k_{1},\ k_{2}$ and $V_{j},\
j=1,2,3,4,S,m,d$ are strictly positive, continuous $T$-periodic functions. We
shall prove that, under accurate assumptions to be specified below, the system
admits at least one positive $T$-periodic solution.

##  3 Existence of positive periodic solutions

In order to apply the topological degree method to problem (2), let us
consider the space of continuous $T$-periodic vector functions

$C_{T}:=\\{u\in C(\mathbb{R},\mathbb{R}^{5}):u(t)=u(t+T)\,\hbox{ for all
$t$}\\},$

equipped with the standard uniform norm, and the positive cone

$\mathcal{K}:=\\{u\in C_{T}:u_{j}\geq 0,j=1\ldots,5\\}.$

Thus, the original problem can be written as $Lu=Nu$, where $L:C^{1}\cap
C_{T}\to C$ is given by $Lu:=u^{\prime}$ and the nonlinear operator
$N:\mathcal{K}\to C_{T}$ is defined as the right-hand side of system (1). For
convenience, the average of a function $u$ shall be denoted by $\overline{u}$,
namely $\overline{u}:=\frac{1}{T}\int_{0}^{T}u(t)\,dt$. Also, identifying
$\mathbb{R}^{5}$ with the subset of constant functions of $C_{T}$, we may
define the function $\phi:[0,+\infty)^{5}\to\mathbb{R}^{5}$ given by
$\phi(x):=\overline{Nx}$.

For the reader’s convenience, let us summarize the basic properties of the
Leray-Schauder degree which, roughly speaking, can be regarded as an algebraic
count of the zeros of a mapping $F:\overline{\Omega}\to E$, where $E$ is a
Banach space and $\Omega\subset E$ is open and bounded. In more precise terms,
assume that $F=I-K$, where $K$ is compact and $F\neq 0$ on $\partial\Omega$.
The degree $deg_{LS}(F,\Omega,0)$ is defined as the Brouwer degree $deg_{B}$
of its restriction $F|_{V}:\Omega\cap V\to V$, where $V$ is an accurate
finite-dimensional subspace of $E$. In particular, if the range of $K$ is
finite dimensional, then one may take $V$ as the subspace spanned by ${\rm
Im}(K)$. If $deg_{LS}(F,\Omega,0)$ is different from $0$, then $F$ vanishes in
$\Omega$; moreover, the degree is invariant over a continuous homotopy
$F_{\lambda}:=I-K_{\lambda}$ with $K_{\lambda}$ compact and $F_{\lambda}\neq
0$ over $\partial\Omega$. Finally, we recall that if
$T:\mathbb{R}^{n}\to\mathbb{R}^{n}$ is a diffeomorphism and $0\in T(A)$ for
some open bounded $A\subset\mathbb{R}^{n}$ then $deg_{B}(T,A,0)$ is just the
sign of the jacobian determinant of $T$ at the (unique) pre-image of $0$. The
following continuation theorem is a direct consequence of the standard
topological degree methods (see e.g. [2]).

###### Theorem 1

Assume there exists $\Omega\subset\mathcal{K}^{\circ}$ open and bounded such
that:

  1. a)

The problem $Lu=\lambda Nu$ has no solutions on $\partial\Omega$ for
$0<\lambda<1$.

  2. b)

$\phi(u)\neq 0$ for all $u\in\partial\Omega\cap\mathbb{R}^{5}$.

  3. c)

$deg_{B}(\phi,\Omega\cap\mathbb{R}^{5},0)\neq 0$.

Then $(\ref{t1})$ has at least one solution in $\overline{\Omega}$.

###  3.1 A priori bounds

In this section, we shall find appropriate bounds for the solution of the
problem $Lu=\lambda Nu$ with $\lambda\in(0,1)$. For convenience, let us fix
the following notation for the minima and maxima of all the functions involved
in the model,namely

$0<\mathit{v}_{i}\leq V_{i}(t)\leq\mathcal{V}_{i},\ 0<\kappa_{j}\leq
K_{j}(t)\leq\mathcal{K}_{j},\ 0<\hat{k}_{l}\leq k_{l}(t)\leq\Bbbk_{l},\
\forall\ i,\ j,\ l.$

Now assume that $u\in\mathcal{K}^{\circ}$ satisfies $Lu=\lambda Nu$ for some
$0<\lambda<1$.
Let us firstly consider a value $t^{*}$ where $M$ achieves an absolute
maximum, then $M^{\prime}(t^{*})=0$ and hence

$\dfrac{V_{S}(t^{*})K_{1}(t^{*})^{n}}{K_{1}^{n}(t^{*})+P_{N}(t^{*})^{n}}=\dfrac{V_{m}(t^{*})M(t^{*})}{K_{m_{1}}(t^{*})+M(t^{*})}\geq\dfrac{\mathit{v}_{m}M(t^{*})}{\mathcal{K}_{m_{1}}+M(t^{*})}:=b_{M}(M(t^{*})).$

If

$\mathit{v}_{m}>\mathcal{V}_{S}$ (2)

then

$M(t^{*})=b_{M}^{-1}\left(\dfrac{V_{S}(t^{*})K_{1}(t^{*})^{n}}{K_{1}^{n}(t^{*})+P_{N}(t^{*})^{n}}\right)<b_{M}^{-1}\left(V_{S}(t^{*})\right)\leq\frac{\mathcal{V}_{S}\mathcal{K}_{m_{1}}}{\mathit{v}_{M}-\mathcal{V}_{S}}:=\mathcal{M}$

Next, suppose that $P_{0}$ achieves its absolute maximum at some point,
denoted again $t^{*}$, then

$K_{s}(t^{*})M(t^{*})+\dfrac{V_{2}(t^{*})P_{1}(t^{*})}{K_{2}(t^{*})+P_{1}(t^{*})}=\dfrac{V_{1}(t^{*})P_{0}(t^{*})}{K_{1}(t^{*})+P_{0}(t^{*})}\geq\dfrac{\mathit{v}_{1}P_{0}(t^{*})}{\mathcal{K}_{1}+P_{0}(t^{*})}:=b_{0}(P_{0}(t^{*})).$

Thus, under the condition

$\mathcal{K}_{S}\mathcal{M}+\mathcal{V}_{2}<v_{1},$ (3)

we deduce that

$P_{0}(t^{*})=b_{0}^{-1}\left(K_{s}(t^{*})M(t^{*})+\dfrac{V_{2}(t^{*})P_{1}(t^{*})}{K_{2}(t^{*})+P_{1}(t^{*})}\right)<\frac{\mathcal{K}_{S}\mathcal{M}+\mathcal{V}_{2}}{v_{1}-(\mathcal{K}_{S}\mathcal{M}+\mathcal{V}_{2})}\mathcal{K}_{1}:=\mathcal{P}_{0}.$

Next, an upper bound $\mathcal{P}_{1}$ for $P_{1}$ is readily obtained in the
following way. Let us denote again by $t^{*}$ a value where $P_{1}$ achieves
its absolute maximum, then

$\dfrac{V_{1}(t^{*})P_{0}(t^{*})}{K_{1}(t^{*})+P_{0}(t^{*})}+\dfrac{V_{4}(t^{*})P_{2}(t^{*})}{K_{4}(t^{*})+P_{2}(t^{*})}=P_{1}(t^{*})\left(\dfrac{V_{2}(t^{*})}{K_{2}(t^{*})+P_{1}(t^{*})}+\dfrac{V_{3}(t^{*})}{K_{3}(t^{*})+P_{1}(t^{*})}\right).$

When $P_{1}(t^{*})\gg 0$, the right-hand side gets close to
$V_{2}(t^{*})+V_{3}(t^{*})$, while the left-hand side is always less or equal
than
$\frac{\mathcal{V}_{1}\mathcal{P}_{0}}{\kappa_{1}+\mathcal{P}_{0}}+\mathcal{V}_{4}$.
Thus, the existence of $\mathcal{P}_{1}$ is guaranteed by the condition

$\frac{\mathcal{V}_{1}\mathcal{P}_{0}}{\kappa_{1}+\mathcal{P}_{0}}+\mathcal{V}_{4}<\min_{t\in\mathbb{R}}\\{V_{2}(t)+V_{3}(t)\\}.$
(4)

The remaining upper bounds are obtained as follows. In the first place, define
a new variable $Q:=P_{N}+P_{2}$ which satisfies the equation:

$\dfrac{dQ}{dt}=\dfrac{V_{3}(t)P_{1}(t)}{K_{3}(t)+P_{1}(t)}-P_{2}(t)\left(\dfrac{V_{4}(t)}{K_{4}(t)+P_{2}(t)}+\dfrac{V_{d}(t)}{K_{d}(t)+P_{2}(t)}\right).$

If $Q$ achieves its absolute maximum at $t^{*}$, then

$\dfrac{\mathcal{V}_{3}\mathcal{P}_{1}}{\kappa_{3}+\mathcal{P}_{1}}\geq\dfrac{V_{3}(t^{*})P_{1}(t^{*})}{K_{3}(t^{*})+P_{1}(t^{*})}-P_{2}(t^{*})\left(\dfrac{V_{4}(t^{*})}{K_{4}(t^{*})+P_{2}(t^{*})}+\dfrac{V_{d}(t^{*})}{K_{d}(t^{*})+P_{2}(t^{*})}\right).$

As before, if the condition

$\dfrac{\mathcal{V}_{3}\mathcal{P}_{1}}{\kappa_{3}+\mathcal{P}_{1}}<\min_{t\in\mathbb{R}}(V_{4}(t)+V_{d}(t))$
(5)

is assumed, then $P_{2}(t^{*})\leq\tilde{P}$ for some $\tilde{P}$. Moreover,
from the fourth equation of the system we deduce the existence of a constant
$C$ such that $\dfrac{dP_{2}}{dt}\geq-CP_{2}(t)$. Hence we obtain, for all
$t$, that $P_{2}(t)\leq e^{CT}\tilde{P}:=\mathcal{P}_{2}$. This provides also
an upper bound for $Q(t)$ and, consequently, an upper bound $\mathcal{P}_{N}$
for $P_{N}(t)$.

After upper bounds are established, we proceed with the lower bounds as
follows. Assume that $M$ achieves its absolute minimum at some $t_{*}$, then
we use again the fact that $M^{\prime}(t_{*})=0$ to obtain:

$\dfrac{V_{m}(t_{*})M(t_{*})}{K_{m_{1}}(t_{*})+M(t_{*})}=\dfrac{V_{S}(t_{*})K_{1}(t_{*})^{n}}{K_{1}^{n}(t_{*})+P_{N}(t_{*})^{n}}\geq\dfrac{\mathit{v}_{S}\kappa_{1}^{n}}{\kappa_{1}^{n}+\mathcal{P}_{N}^{n}}.$

This shows that $M_{1}(t)\geq\mathfrak{m}$ for some positive constant
$\mathfrak{m}$. In the same way, we find a lower bound $\mathfrak{p}_{0}$ for
$P_{0}$ using the fact that

$\dfrac{V_{1}(t_{*})P_{0}(t_{*})}{K_{1}(t_{*})+P_{0}(t_{*})}=K_{s}(t_{*})M(t_{*})+\dfrac{V_{2}(t_{*})P_{1}(t_{*})}{K_{2}(t_{*})+P_{1}(t_{*})}\geq\kappa_{s}\mathfrak{m}.$

Next, suppose that $P_{1}$ achieves its absolute minimum at $t_{*}$, then

$P_{1}(t_{*})\left(\dfrac{V_{2}(t_{*})}{K_{2}(t_{*})+P_{1}(t_{*})}+\dfrac{V_{3}(t_{*})}{K_{3}(t_{*})+P_{1}(t_{*})}\right)>\dfrac{V_{1}(t_{*})P_{0}(t^{*})}{K_{1}(t_{*})+P_{0}(t_{*})}\geq\dfrac{v_{1}\mathfrak{p}_{0}}{\mathcal{K}_{1}+\mathfrak{p}_{0}}>0$

which yields the existence of a positive lower bound $\mathfrak{p}_{1}$.
Finally, positive lower bounds for $P_{2}$ and $P_{N}$ are obtained by means
of the function $Q=P_{2}+P_{N}$. Indeed, if $Q$ achieves its absolute minimum
at some $t_{*}$, then

$P_{2}(t_{*})\left(\dfrac{V_{4}(t_{*})}{K_{4}(t_{*})+P_{2}(t_{*})}+\dfrac{V_{d}(t_{*})}{K_{d}(t_{*})+P_{2}(t_{*})}\right)\geq\dfrac{v_{3}\mathfrak{p}_{1}}{\mathcal{K}_{3}+\mathfrak{p}_{1}}$

and we deduce that $P_{2}(t_{*})$ cannot be arbitrarily small. As before,
using the fact that $P_{2}^{\prime}\geq-CP_{2}$ it is seen that $P_{2}(t)\geq
e^{-CT}P_{2}(t_{*})$ and the conclusion follows. This, in turn, yields a lower
bound $\mathfrak{p}_{N}>0$ for $P_{N}$.

We are already in conditions of defining the open set
$\Omega\subset\mathcal{K}^{\circ}$ as

$\Omega:=\\{(M,P_{0},P_{1},P_{2},P_{N})\in
C_{T}:\mathfrak{m}<M(t)<\mathcal{M},\mathfrak{p}_{0}<P_{0}(t)<\mathcal{P}_{0},$
$\mathfrak{p}_{1}<P_{1}(t)<\mathcal{P}_{1},\mathfrak{p}_{2}<P_{2}(t)<\mathcal{P}_{2},\mathfrak{p}_{N}<P_{N}(t)<\mathcal{P}_{N}\\}$

and

###### Theorem 2

Assume that the previous conditions (2), (3), (4) and (5) hold. Then problem
(1) has at least one positive $T-$periodic solution.

###  3.2 Degree computation

In the previous section, the first condition of the continuation theorem was
verified. It remains to prove that $b)$ and $c)$ are fulfilled as well. With
this aim, set $\mathcal{Q}:=\Omega\cap\mathbb{R}^{5}$ and recall that the
function $\phi:\overline{\mathcal{Q}}\to\mathbb{R}^{5}$ is defined by
$\phi(x)=\overline{Nx}$. We claim that each coordinate $\phi_{j}$ has
different signs at the corresponding opposite faces of $\mathcal{Q}$.

Indeed, compute for example $\phi_{1}(\mathcal{M},P_{0},P_{1},P_{2},P_{N})$
and $\phi_{1}(\mathfrak{m},P_{0},P_{1},P_{2},P_{N})$ for $\mathfrak{p}_{j}\leq
P_{j}\leq\mathcal{P}_{j}$:

$\phi_{1}(\mathcal{M},P_{0},P_{1},P_{2},P_{N})=\frac{1}{T}\int_{0}^{T}\left(\dfrac{V_{S}({t})K_{1}({t})^{n}}{K_{1}^{n}({t})+P_{N}}-\dfrac{V_{m}({t})\mathcal{M}}{K_{m_{1}}({t})+\mathcal{M}}\right)\,dt$
$<\mathcal{V}_{S}-\dfrac{v_{m}\mathcal{M}}{\mathcal{K}_{m_{1}}+\mathcal{M}}=0,$
$\phi_{1}(\mathfrak{m},P_{0},P_{1},P_{2},P_{N})=\frac{1}{T}\int_{0}^{T}\left(\dfrac{V_{S}({t})K_{1}({t})^{n}}{K_{1}^{n}({t})+P_{N}}-\dfrac{V_{m}({t})\mathfrak{m}}{K_{m_{1}}({t})+\mathfrak{m}}\right)\,dt$
$>\dfrac{v_{S}\kappa_{1}^{n}}{\mathcal{K}_{1}^{n}+\mathcal{P}_{N}^{n}}-\dfrac{\mathcal{V}_{m}\mathfrak{m}}{\kappa_{m_{1}}+\mathfrak{m}}\geq
0$

provided that $\mathfrak{m}$ is small enough. In the same way, making the
lower bounds smaller if necessary, we deduce that

$\phi_{2}(M,\mathcal{P}_{0},P_{1},P_{2},P_{N})<0<\phi_{2}(M,\mathfrak{p}_{0},P_{1},P_{2},P_{N})$
$\phi_{3}(M,P_{0},\mathcal{P}_{1},P_{2},P_{N})<0<\phi_{2}(M,{p}_{0},\mathfrak{p}_{1},P_{2},P_{N})$
$\phi_{4}(M,P_{0},P_{1},\mathcal{P}_{2},P_{N})<0<\phi_{2}(M,{p}_{0},p_{1},\mathfrak{p}_{2},P_{N})$
$\phi_{5}(M,P_{0},P_{1},P_{2},\mathcal{P}_{N})<0<\phi_{2}(M,{p}_{0},p_{1},p_{2},\mathfrak{p}_{N}).$

Thus, condition $b)$ of Continuation Theorem is verified. Moreover, we may
define a homotopy as follows. Consider the center of $\mathcal{Q}$ given by

$\wp:=\left(\dfrac{\mathcal{M}+\mathfrak{m}}{2},\dfrac{\mathcal{P}_{0}+\mathfrak{p}_{0}}{2},\dfrac{\mathcal{P}_{1}+\mathfrak{p}_{1}}{2},\dfrac{\mathcal{P}_{2}+\mathfrak{p}_{2}}{2},\dfrac{\mathcal{P}_{N}+\mathfrak{p}_{N}}{2}\right)$

and the function
$\mathcal{H}:\overline{\mathcal{Q}}\times[0;1]\to\mathbb{R}^{5}$ given by

$\mathcal{H}(x,\lambda)=(1-\lambda)(\wp-x)+\lambda\phi.$

We need to verify that $\mathcal{H}$ does not vanish at $\partial\mathcal{Q}$.
To this end, suppose for example that
$\mathcal{H}(\mathcal{M},P_{0},P_{1},P_{2},P_{N})=0$ for some
$\hat{\lambda}\in[0;1]$, then

$0=\mathcal{H}_{1}(\mathcal{M},\hat{\lambda})=(1-\hat{\lambda})\underbrace{\left(\dfrac{\mathcal{M}+\mathfrak{m}}{2}-\mathcal{M}\right)}_{<0}+\hat{\lambda}\underbrace{\phi_{1}(\mathcal{M},P_{0},P_{1},P_{2},P_{N})}_{<0}<0,$

a contradiction. All the remaining cases follow in an analogous way. By the
homotopy invariance of the Brouwer degree, it follows that

$deg_{B}(\phi,\mathcal{Q},0)=deg_{B}(\wp-I,\mathcal{Q},0)=(-1)^{5}\neq 0.$

This proves the third condition of the continuation theorem and, therefore,
the existence of a $T$-periodic solution is deduced. $\square$

## References

  * [1] A. Goldbeter, _Biochemical Oscillations and Cellular Rhythms. The molecular bases of periodic and chaotic behaviour_ , Cambridge University Press, 1996.
  * [2] P. Amster, _Topological Methods in the Study of Boundary Value Problems_ , Springer, New York, 2014.
  * [3] P. Amster, L. Idels, _Existence theorems for some abstract nonlinear non-autonomous systems with delays_ , Commun Nonlinear Sci Numer Simulat 19 (2014) 2974–2982.