# The Bleeps, the Sweeps, and the Creeps:
Convergence Rates for Dynamic Observer Patterns via Data Assimilation for the
2D Navier-Stokes Equations

Trenton Franz  School of Natural Resources, University of Nebraska–Lincoln,
Lincoln, NE 68588-0130, USA<EMAIL_ADDRESS>,  Adam Larios  Department
of Mathematics, University of Nebraska–Lincoln, Lincoln, NE 68588-0130, USA
<EMAIL_ADDRESS>and  Collin Victor  Department of Mathematics, University of
Nebraska–Lincoln, Lincoln, NE 68588-0130, USA<EMAIL_ADDRESS>
###### Abstract.

We adapt a continuous data assimilation scheme, known as the Azouani-Olson-
Titi (AOT) algorithm, to the case of moving observers for the 2D
incompressible Navier-Stokes equations. We propose and test computationally
several movement patterns (which we refer to as “the bleeps, the sweeps and
the creeps”), as well as Lagrangian motion and combinations of these patterns,
in comparison with static (i.e. non-moving) observers. In several cases,
order-of-magnitude improvements in terms of the time-to-convergence are
observed. We end with a discussion of possible applications to real-world data
collection strategies that may lead to substantial improvements in predictive
capabilities.

###### Key words and phrases:

Continuous data assimilation; Azouani-Olson-Titi algorithm; Navier-Stokes
equations; moving observers; Lagrangian particles; observational interpolation
MSC 2010: 35Q30; 93C20; 37C50; 76D55; 76F70.

“I’m having trouble with the radar, sir. […] I’ve lost the bleeps, I’ve lost
the sweeps, and I’ve lost the creeps.” –Michael Winslow as Radar Technician in
Spaceballs [51].

##  1\. Introduction

One difficulty in the simulation of many physically interesting dynamical
systems is that initial data is often incomplete or inaccurate. However, one
can circumvent this difficulty by using the tools of data assimilation, which
is a class of techniques used to increase accuracy by combining time-dependent
observational data together with a physical model, driving simulations to
converge to the “true” solution. In this work, we concentrate on a simple yet
powerful data assimilation tool known as the Azouani-Olson-Titi (AOT)
algorithm, since it readily lends itself to the case of dynamic observation
points, which is the major focus of this manuscript. In a previous work [42],
two of the authors first proposed the idea of dynamic observers for the AOT
algorithm, and showed that in a simplified 1D model (the Allen-Cahn reaction-
diffusion equations), moving the observation points in time resulted in order-
of-magnitude increases in the time to convergence to machine precision error.
In the present work, we extend the idea of continuous data assimilation via
moving observers to the 2D Navier-Stokes equations, and show similar gains in
convergence rates. Moreover, we propose and investigate several movement
schemes for observers, and compare these to the case of static (i.e., non-
moving) observers. During the completion of this manuscript, we learned of
another work [6], which shows convergence of the AOT algorithm for the 2D
Navier-Stokes equations with observers in a moving square patch. However, the
purpose of the present work is to examine the effectiveness of several
different, arguably more realistic, movement patterns with the aim of real-
world data collection strategies.

Since its inception in [3, 4], the AOT algorithm has been the subject of much
recent study in both analytical studies [1, 5, 6, 8, 9, 10, 11, 13, 15, 16,
21, 22, 23, 24, 25, 26, 27, 28, 29, 31, 30, 32, 34, 35, 36, 37, 39, 41, 44,
45, 48, 49, 50, 53] and computational studies [2, 12, 14, 18, 20, 33, 38, 40,
42, 43].

In this work we investigate the AOT algorithm as applied to the 2D Navier-
Stokes equations (NSE). Writing the NSE in an abstract form111After solving
for the pressure, one can write $F(u)=\nu\triangle u-u\cdot\nabla
u-\nabla\triangle^{-1}\nabla\cdot(u\otimes u)$ (see, e.g., [19, 52] for
details), but this specific form is not especially relevant for present
discussion. we have the following equation

(1.1) $\frac{d}{dt}u=F(u).\\\ $

Note that in (1.1), we do not know the initial condition for this system.
However, we assume that we can observe the solution at certain spatial
locations, which may be sparsely distributed, say with some characteristic
spatial distance between observations $h>0$. Using these measurements, the AOT
algorithm [3, 4] constructs the system

(1.2) $\displaystyle\begin{cases}\frac{d}{dt}v=F(v)+\mu(I_{h}(u)-I_{h}(v))\\\
v(0)=v_{0}\end{cases}$

where $I_{h}(u)$ is a suitable interpolation of the observed values of a
strong solution $u$ of (1.2), and $\mu>0$ is a constant, sometimes called the
“nudging parameter.” It was proven in [3] that for sufficiently large $\mu$
and sufficiently small $h$ that $\left\|u(t)-v(t)\right\|_{L^{2}}\to 0$ and
$\left\|\nabla u(t)-\nabla v(t)\right\|_{L^{2}}\to 0$ exponentially fast in
time as $t\to\infty$ for any choice of $v_{0}\in L^{2}$ such that $v_{0}$ is
divergence-free. Note that in many works including the present one, for
simplicity data is assumed to be observed continuously in time and without
noise. However, similar exponential convergence results for discrete-in-time
observations were proven in [28], and for noisy data and stochastic forcing in
[5, 12].

An important aspect to consider when capturing physical phenomena via data
assimilation is where to place observers. Observers can be thought of as
physical objects such as weather stations in the case of predicting the
weather. These observers may require financial investment and manpower to
place, thus it is crucial to minimize the number of observers required for
convergence of the method. Computational tests in the case of 2D NSE indicated
exponential convergence in time using a static uniform grid of observers [33].
The rate of convergence can typically be sped up by adding more observers,
however computational studies [7, 42] seem to suggest that the rate of
convergence can be improved by moving existing observers instead. In a
previous work we found in the case of the 1D Allen-Cahn equations that the
number of observers can be drastically reduced by replacing the static uniform
grid with a mobile cluster of fine length scale [42]. Recently in [7] it was
shown computationally that mobile local data assimilation in the case of 2D
NSE achieved fast convergence rates when compared to a non-mobile local data
assimilation algorithm. There mobile local data assimilation was investigated
in the single regime of continuously moving a square subdomain around the
domain. In this work we investigate computationally the effect of movement
strategies for observers on convergence rates for 2D NSE for both local and
global data assimilation.

The effect of sweeps has been recently observed in the reduction of skill in
weather forecasting models due to the elimination of cross Atlantic air
travel. For example, it was shown in [17] the 50-75% reduction in air flights
in 2020 has lead to a decrease of $2^{\circ}$C in air temperature forecasts
over Greenland and Siberia and a sustained reduction in flights will lead to
further degradation of the air pressure and wind speed forecast skill.

In the present work, we focus on the 2D incompressible Navier-Stokes
equations, given by:

(1.3a) $\displaystyle u_{t}+u\cdot\nabla u-\nu\triangle u$
$\displaystyle=\nabla p+f,$ (1.3b) $\displaystyle\nabla\cdot u$
$\displaystyle=0.$

Here $u(x,t)=(u_{1}(x,t),u_{2}(x,t))$ is the velocity, $\nu>0$ is the
viscosity, $p$ is the pressure and $f$ is a forcing term. Applying the
continuous data assimilation algorithm (1.2) to this system yields the
following equations:

(1.4a) $\displaystyle v_{t}+v\cdot\nabla v-\nu\triangle v$
$\displaystyle=\nabla p+f+\mu(I_{h,t}(u)-I_{h,t}(v)),$ (1.4b)
$\displaystyle\nabla\cdot v$ $\displaystyle=0.$

Here, $I_{h,t}$ is a linear operator representing nodal piecewise-linear
interpolation, and $\mu>0$ is a algorithmic parameter often referred to as the
“nudging” parameter. In the static case, more general interpolation operators
were considered in [3], but for the sake of simplicity, we only consider the
case of piecewise-linear interpolation. Note that, in contrast to the usual
AOT algorithm for static observers, here, we allow the observer locations to
depend on time, as denoted by the subscript $t$ in $I_{h,t}$.

This paper is laid out as follows. In Section 2, we describe the various
observer movement patterns that we consider. In Section 3, we describe our
computational methods and results, but to keep the discussion concise, we only
refer to the graphics, which are in the appendix. In Section 4, we conclude
with a comparative discussion on the merits of the various observer movement
patterns, and contrast them with the results of the static observer vase.

##  2\. Movement Patterns

For this study we computationally examined different methods of moving
observers dynamically. We group these into three general movement patterns
that we call “The Bleeps”, “The Sweeps” and “The Creeps”. In addition to these
movement patterns we also investigate a fourth pattern given by observers as
Lagrangian particles suspended in the fluid.

###  2.1. The Bleeps

“The Bleeps” refers to the strategy of observing data at random locations that
change over time. Initially observers are placed at random locations and at
each time-step. The old observers are then removed, and new observers are
placed at new random locations. These random locations are determined by
picking uniformly random integers $x_{i}\in\left\\{0,1,...,N\right\\}$ with
locations given by $x=(x_{1}\cdot\Delta x-\pi,x_{2}\cdot\Delta y-\pi)$. The
locations are chosen in this manner so that their coordinates align with the
underlying spatial grid.

While we do not claim that there is a physically-realistic scenario
corresponding to this movement strategy, studying “The Bleeps” can give some
idea of how non-static observations can influence convergence rates in the
absence of any coherent spatial structure. Thus, we examine this case as
guideline or comparative test to shed light on the importance of domain
coverage vs. certain geometric patterns in observer motion.

###  2.2. The Sweeps

“The Sweeps” refers to the general strategy of moving a group observers at a
constant velocity periodically over the domain. For this method we
investigated three separate movement strategies referred to as thin, thick,
and random sweeps.

“Thin-sweeps” refers to the strategy of utilizing a large cluster of observers
placed in a thin rectangular strip of the domain that moves continuously to
the right at a constant velocity. The rectangular strip utilized by this
strategy has length $x=a\Delta x$ and width $y=L_{y}=2\pi$ and moves at
constant velocity $b\frac{\Delta x}{\Delta t}$ for some $a,b\in\mathbb{N}$.
Note that the form of the velocity and $x$-dimension were chosen specifically
so that the rectangular region would align with the underlying spatial grid at
every timestep. In all of the simulations conducted we choose values
$a\in\left\\{1,...,20\right\\}$ and additionally with $b=a$. In simulations
this rectangular box contains observers placed at each gridpoint along the
spatial grid, meaning that the full solution of $u$ is observed locally in
this rectangular region. Outside of this rectangle, $I_{h,t}(u-v)=0$ for any
given time $t$. Thus $I_{h,t}$ is given as follows:

$\displaystyle I_{h,t}(u-v)=(u-v)\chi_{R(t)}$

Here $R(t)$ refers to the sliding rectangular box of observers at the $t$.

“Thick-sweeps” refers to the strategy of utilizing a uniform grid of observers
placed on a rectangular region that does not vary in size with the number of
observers contained within. We fix the rectangular region to be a box
containing the fourth of the domain given by
$R=\left\\{(x,y)\in\mathbb{T}^{2}:-\pi\leq x\leq-\frac{\pi}{2}\right\\}$. This
rectangular region moves to the right at the constant velocity $b\frac{\Delta
x}{\Delta t}$. In all of the simulations whose results are presented here we
used $b=1$. For certain velocities one can view this method as a uniform
static grid with only a fourth of the observers active at any given time.
$I_{h,t}$ is given as follows:

$\displaystyle I_{h,t}(u-v)=(I_{h}(u-v))\chi_{R(t)}$

Figure 2.1. Graph displaying movement of observers using the thick sweeps
movement strategy displayed over the vorticity of the reference solution at
time $t=2$. Observers were initially placed at locations indicated by * and
travel to the right along the black paths with ending locations marked by
$\times$.

Random-sweeps refers to the strategy of placing observers at random locations
with an additional random velocity. A major difference between this and all of
the previous methods is that the location of the observers are not restricted
to the underlying spatial grid. This means that the value of $u-v$ must be
interpolated from the spatial grid at each timestep in order obtain
approximate values for the observer to observe. The velocities of each
observer are given by vectors with each component having a random uniformly
distributed value from the interval $(-1,1)$. $I_{h,t}$ is given as follows:

$\displaystyle I_{h,t}(u-v)=I_{h,t}\circ J_{\Delta x,t}(u-v)$

Here $J$ is a linear interpolation operator applied to $u-v$ in order to get
the approximate values for the observers with $I$ interpolating these observed
values onto $\mathbb{T}^{2}$.

Figure 2.2. Graph displaying movement of observers using the random sweeps
movement strategy displayed over the vorticity of the reference solution at
time $t=2$. Observers were initially placed at 10 locations indicated by * and
travel along random trajectories indicated by black paths with ending
locations marked by $\times$.

We study each of these sweep-movement strategies as each can be potentially
realized in physical scenarios. The thin-sweep observers can be used to model
observations given by, e.g., a laser sweeping through a domain in particle
image velocimetry (PIV) applications, where a velocity profile is taken as an
observation. Thick sweeps can be used to model observers on mobile platforms
that move at a fixed velocity. Random sweeps can be thought of as modeling
observers with a fixed velocity, e.g., airplanes or satellites.

###  2.3. The Creeps

“The creeps” refers to the strategy of having observers move according to
random walks (“creeping” along). After a given amount of time, each observer
will (uniformly) randomly pick an integer $j\in\left\\{1,2,3,4,5\right\\}$.
These integers determine the direction of motion $d(j)$ as follows:

$\displaystyle d(j)=\left(\begin{matrix}x_{1}\Delta x\\\ x_{2}\Delta
y\end{matrix}\right)\ \text{ where }\begin{cases}x_{1}=1,x_{2}=0,&\text{ if
}j=1,\\\ x_{1}=-1,x_{2}=0,&\text{ if }j=2,\\\ x_{1}=0,x_{2}=1,&\text{ if
}j=3,\\\ x_{1}=0,x_{2}=-1,&\text{ if }j=4,\\\ x_{1}=0,x_{2}=0&\text{ if
}j=5.\end{cases}$

That is, they randomly move in a cardinal direction, or remain still, with
uniform probability.

Figure 2.3. Graph displaying movement of observers using the creeps movement
style displayed over the vorticity of the reference solution at time $t=2$.
Observers were initially placed at 100 locations indicated by * and travel
along a random walk indicated by black paths with ending locations marked by
$\times$.

###  2.4. Lagrangian Particles

The final movement strategy we considered is given by observers following
Lagrangian particles trajectories. Observers are initialized into a uniform
grid of spatial resolution $h>0$ and then follow the Lagrangian trajectory
$\ell(t)$ as determined by

(2.1) $\displaystyle\frac{d\ell}{dt}=u(\ell(t),t).$

This is a very natural type of dynamic observer, as the movement of the
observer comes exactly from the behavior of the system that we are attempting
the capture in simulations of $v$. Once can consider sensors attached to buoys
(e.g., Argo floats) set adrift in the ocean.

Lagrangian trajectories for observers were evolved forward using the third-
order Adams-Bashforth method with values of $u$ at locations $(\ell(t),t)$. As
in the case for random-sweeps, observers in the case of Lagrangian particles
the observations do not in general occur at location on the underlying spatial
grid. Therefore we must approximate the value $u(\ell(t),t)$ via a linear
interpolation of $u$. In this case we can consider $I_{h,t}(u-v)$ as follows:

$\displaystyle I_{h,t}(u-v)=I_{\ell(t),t}\circ J_{\ell(t)}(u-v)$

Here $J_{\ell(t)}(u-v)$ is a piecewise-linear interpolant used to approximate
the values of $u-v$ in order to find the observational data and
$I_{\ell(t),t}$ is another piecewise-linear interpolant used to extend the
observational data to the numerical grid.

Figure 2.4. Graph displaying movement of observers given by Lagrangian
particles displayed over the vorticity of the reference solution at time
$t=50$. Observers were initially placed at 10 locations indicated by * and
travel along random trajectories indicated by black paths with ending
locations marked by $\times$.

##  3\. Computational Results

All of our computations were done on the 2D NSE in the vorticity-stream
function formulation using a fully dealiased pseudospectral code with physical
domain $\mathbb{T}^{2}=[-\pi,\pi]^{2}$. As in [7] we used viscosity
$\nu=10^{-4}$ with forcing as given in [33] that was multiplied by a scalar
constant to obtain a forcing $f$ with Grashof number $G=10^{6}$. Additionally,
consistent with [7, 47, 46], we used a third-order Adams-Bashforth method with
an integrating factor to solve for the linear term exactly. We used step size
$\Delta t=0.005$, and our initial data was generated by evolving zero initial
data out to time $t=25,000$. The spectrum of the initial data for the
reference solution $u$ is plotted in Figure Figure A.1 and one can see that
spectrum is well-resolved (it also remained well-resolved for all times). Note
that the spectrum of the assimilation solution $v$ was temporarily slightly
under-resolved in . Additionally, we used spatial resolution $N^{2}=1,024^{2}$
in our simulations in order to fully resolve the energy spectrum to machine
precision (roughly $2.22\times 10^{-16}$) before the 2/3’s dealiasing cutoff
(see, e.g., Figure A.1).

Note that while the equation was simulated using the vorticity-stream function
formulation, the interpolation term was calculated at the velocity level using
nodal interpolation in physical space. To preserve the periodicity in the
interpolation across the periodic “boundary” we interpolated using observed
values over $[-3\pi,3\pi]\times[-3\pi,3\pi]$ instead of just over
$\mathbb{T}^{2}=[-\pi,\pi]\times[-\pi,\pi]$, and then truncated the result
back to the physical domain. Since we are simulating the equations at the
vorticity-stream function level, we calculate $\|\psi-\tilde{\psi}\|_{L^{2}}$
as the error between the simulated and reference solutions. We additionally
calculate both $\left\|\omega-\tilde{\omega}\right\|_{L^{\infty}}$ and
$\left\|\omega-\tilde{\omega}\right\|_{L^{2}}$ (see figs. A.2, A.8, A.4, A.10,
A.6, A.12, A.3, A.9, A.5, A.11, A.7 and A.13). Here $\psi$ denotes the stream
function of the $u$ equation with $\omega$ being the vorticity of $u$, and
$\tilde{\cdot}$ denoting the corresponding quantity in system (1.4).

Using the numerical methods discussed above, we simulated dynamic observers
that move in accordance with the movement styles. We found that the choice of
$\mu$ varied for different movement methods. For the uniform static grid it
was found that $\mu=10$ was the optimal choice of $\mu$. It appears that the
choice of larger $\mu$ values is restricted by a CFL condition on the time-
step; namely $\Delta t<2/\mu$. We observed, similar to our previous work, [42]
that highly mobile movement schemes, such as thin-sweeps, do not appear to be
subject to the same restriction on $\mu$, perhaps due to the numerically
destabilizing effect of large $\mu$-values being only briefly concentrated in
any given spatial location. For simulations of sweeps, we used the value
$\mu=30$, which was unstable in the case of the static uniform grid of
observers. In our simulations we observed that larger values of $\mu$ tended
to lead to faster convergence in the case of thin-sweeps. In simulations, we
tested thick-sweeps and found that the method did allow for larger choices of
$\mu$ when the speed of the observer movement was increased to that of the
thin-sweeps (from $1\frac{\Delta x}{\Delta t}$ to $3\frac{\Delta x}{\Delta
t}$), further corroborating the “brief local destabiliztion” explanation
discussed above. While thick-sweeps did allow for a larger choice of $\mu$ we
did not observe faster convergence rates in these trials. We conjecture that,
at least typically, the larger values of $\mu$ are optimal for very fast
moving observer regimes in which any given point will only be observed for a
short amount of time.

In preliminary simulations, we observed that certain $\mu$ values for which
the static uniform grid remained stable could introduce instabilities in
mobile observer schemes (except in the case of thin-sweeps and thick-sweeps).
This is perhaps due to a CFL condition restricting the length-scale $h$, the
minimum length between observers. Since $h$ is not restricted for most of the
mobile observer paradigms, $\mu$ values should be smaller to avoid this
instability. We found that the value $\mu=10$ was optimal for the uniform
static grid and did not cause instability in the mobile observers for our
choice of spatial resolution $N^{2}=1024^{2}$. In preliminary simulations with
spatial resolution $N^{2}=512^{2}$ we saw that the mobile observer methods
required $\mu$ values smaller than the the optimal choice of $\mu$ for the
uniform static grid to avoid instability. However this instability was not
seen in any of the higher resolution tests. While $\mu$ can potentially be
further optimized, as generally higher $\mu$ values coincide with faster decay
in the error, it is worth noting that our results showed faster rates of
convergence using the same $\mu$ values, but with smaller numbers of observers
required, (see figs. A.2 and A.8).

To test these methods computationally, we first ran simulations for the
uniform static grid case to establish a baseline for comparison. We found
that, for our choices of parameters, a uniform grid of $5,625$ observers would
converge to machine precision $10^{-14}$ by approximately time $t=70$ (see
figs. A.2, A.8, A.4, A.10, A.6, A.12, A.3, A.9, A.5, A.11, A.7 and A.13).
Using this, we varied the number of observers in the case of the bleeps, the
sweeps, and the creeps in order to obtain convergence at approximately the
same time ($t=70$) for the sake of comparison. The results from these studies
can be seen in figs. A.2, A.8, A.4, A.10, A.6, A.12, A.3, A.9, A.5, A.11, A.7
and A.13. As we see from these results, all three of these methods enjoy
exponential convergence to the reference solution in the $L^{2}$ norm.
Moreover, all three methods converge to machine precision faster than the
uniform grid using significantly fewer observers. The bleeps in particular
demonstrated almost an order of magnitude improvement in the number of
observers compared to the uniform grid.

Next, we ran simulations utilizing mobile observers with each initialized to
approximately 5,625 observers to compare with the static uniform grid of 5,625
observers. As expected, we observed that all of the mobile observer methods
converged exponentially fast to the reference solution $u$ at a faster rate
than the uniform static grid. Our results can be seen in figs. A.2, A.8, A.4,
A.10, A.6, A.12, A.3, A.9, A.5, A.11, A.7 and A.13.

One additional metric we used for comparison with the uniform static grid was
the CPU time required to obtain convergence to machine precision. It is
important to check that the extra complexity built into simulating moving
observers does not make these methods prohibitively expensive to simulate when
compared to the static uniform grid. In all of the simulations described above
we also captured the CPU time. Graphs of the error vs the CPU time can be seen
in figs. A.2, A.8, A.4, A.10, A.6, A.12, A.3, A.9, A.5, A.11, A.7 and A.13.
Note that the CPU time required to update the locations is included in these
calculations of total CPU time. This includes the time to randomly determine
locations, move the rectangular subdomain, and determine random walks for
bleeps, sweeps, and creeps, respectively (this would likely not be needed when
working with real-world data, but on the other hand, working real-world data
may bring other CPU-intensive operations, such as I/O costs or off-grid
interpolation on non-uniform grids). Additionally, note that since the thin-
sweeps strategy use the exact solution over a small rectangular subdomain
there is no interpolation performed. This leads to a very low CPU time in
those figures. In the plots, we found that when the number of observers is
chosen to ensure convergence by (roughly) time $t=70$, all of the dynamic
observer movement strategies except Lagrangian particles obtain convergence
significantly faster in CPU time than the uniform grid case. This is perhaps
due to the fewer number of observers used, simplifying the calculations of the
interpolation term. Additionally, when all of the methods are simulated using
approximately $5,625$ observers, the mobile observer methods still obtain
faster convergence than the static uniform grid. In this case, it is likely
due to the rate of convergence, which can be seen in figs. A.2, A.8, A.4,
A.10, A.6, A.12, A.3, A.9, A.5, A.11, A.7 and A.13.

We note that all of the data assimilation methods, except thin and thick
sweeps, show a steep decay in the error initially until time $t\approx 1$.
This steep decay is possibly due to the global distribution of observers
across the domain. This steeper decay can be seen in the case of thick-sweeps
when initialized with 1,400 observers moving with constant velocity
$3\frac{\Delta x}{\Delta t}$ in the $x$ direction until approximately time
$t=8$ (see fig. A.2). Moreover we note that this steep decay was not seen for
thick-sweeps with the constant velocity $1\frac{\Delta x}{\Delta t}$ in the
$x$ direction (see fig. A.14).

In the above simulations it appears that performing thin-sweeps is a good
strategy for mobile data assimilation however it requires the exact solution
in a small rectangle. All of the other movement schemes are adaptable to
higher resolution with no difficulties, but with thin-sweeps for a fixed
rectangular region the number of observers scales with the resolution.
Physically this may be feasible if the data collection comes from a device
that gathers a velocity profile along one dimension, however it is not
feasible if the velocity is measured using observers placed at discrete
points. This lead to the generalization of thin-sweeps to thick-sweeps, which
allows for mobile observers that sample locally in space and move continuously
in time without requiring a number of observers proportional to the spatial
resolution of the underlying numerical grid. Here, instead of having access to
the full solution in a rectangle, we extend the rectangular area to cover a
quarter of the domain and interpolate locally over a uniform grid of observers
distributed locally on this subdomain. In our simulations we found that the
velocity of the observers was an important factor in the convergence rate (see
Figures A.14 and A.15). One may be able to optimize the velocity of the
observers based on the size of the domain being sampled to improve convergence
rates further, but we do not explore this in the present work.

##  4\. Conclusions

Our simulations of mobile observers provide evidence that highly mobile
observers such as the bleeps give the fastest convergence rates using the
fewest number of observers when compared to a static uniform grid of
observers. All mobile methods demonstrated convergence faster in simulation
time than the static uniform grid when initialized with the same number of
observers; however, the methods that demonstrated the highest convergence
rates were the bleeps and random-sweeps, both of which require some level of
randomness. While sensors with random movement may be difficult to implement
physically, this does demonstrate the need for observing data throughout the
domain.

While all of the methods converged to machine precision faster in simulation
time than the static uniform grid, the thick-sweeps and Lagrangian mobile
observer movement schemes performed poorly in comparison to the rest of the
mobile methods. In particular, neither thick-sweeps nor Lagrangian particle
observers demonstrate the improvements in CPU time seen in the other mobile
methods. In the case of thick-sweeps, the poor performance relative to the
other mobile methods is likely due to lack of mobility. While thick-sweeps do
move observers continuously, all of the simulations done in this study used
$b=1$ which results in slow movement across the domain. Larger $b$ values
would likely make thick-sweeps behave similarly to thin-sweep in convergence
rates. Similarly, we believe that the Lagrangian particle observers perform
poorly due to the slow movement of fluid particles and the tendency for
particles to become trapped in vorticity structures (e.g., eddies).

An additional point of note is the performance of thin-sweeps. Recall that
thin-sweeps measures the full solution within a small rectangle and inserts
this directly into the $v$ equation without interpolation of any sort. This
leads to a drastic decrease in CPU time when compared to the other methods
which can be seen in figs. A.8, A.10, A.12, A.9, A.11, A.13, A.2, A.4, A.6,
A.3, A.5 and A.7. Additionally, due to the requirement of measuring the full
solution inside a small rectangle, the number of observers is misleading. As
the resolution increases, the thin-sweep strategy will need an increasing
amount of observers if the rectangular area remains the same size. The need to
measure the full solution essentially means that the thin-sweep strategy
requires an infinite number of observers. As currently implemented, the number
of observers determines the width of the rectangular area, which for $3072$ is
$3072/N\Delta x=3\Delta x$. Another restriction of thin-sweeps is that this
requirement that the number of observers must be a multiple of the resolution
$N$.

Due to the shortcomings of both thin-sweeps and thick-sweeps, we propose that
a hybrid of the two methods would be practical to implement and would have
some of the beneficial properties of both while being more readily
implementable physically. This hybrid method would replace the rectangular
area of thick-sweeps with the much smaller rectangle from thin-sweeps.
Additionally, as in thin-sweeps, this hybrid method would keep the movement of
the rectangular area proportional to its width. However, unlike thin-sweeps,
we would actually interpolate over the area within the rectangle instead of
requiring the full solution. We will explore these ideas in a future work.

From our simulations it appears that the bleeps method requires the smallest
number of observers to achieve convergence comparable to the uniform static
grid. In the description of the bleeps movement strategy, we noted that the
locations of observers are changed at each timestep. In our previous
simulations, it was found that the bleep strategy also demonstrates
exponential convergence to the reference solution when observers are placed in
random locations that changed after a specified amount of time. We noted that
the error decreased sharply for a short amount of time after the observers
were moved. In order to maintain this steep rate of decay, we chose to
randomize the observer locations at every timestep. This suggests that one
could optimize the randomization time, which will be the subject of a future
work.

When considering the viability of mobile methods, it is important to consider
the feasibility of physical implementations. In particular, one should ask if
one can realize these methods using actual sensors physically moving in time.
Moreover, in physical implementations, mobile observers may be more expensive
than non-mobile observers and more prone to break down and potentially suffer
measurement error. Therefore, factoring in cost of number of observers should
be taken into account in real-world applications, but this is beyond the scope
of the present work. In this study, we assumed that all observers had the same
level of accuracy (namely, they are assumed to have perfect accuracy).
Although studies of the AOT algorithm have been carried out in the context of
noisy data [5, 12], showing that the algorithm with static observations is
robust with respect noisy measurements, factoring in different levels of
accuracy for different observers may be crucial in real-world implementations.
(For example, sensors on airplanes tend to be much more accurate, but more
expensive that ground-based sensors.)

We note that, although we made an effort to follow good coding practices, no
specific effort was made to optimize the computation of the various moving
observer schemes, nor their interpolation, nor the nudging constants $\mu$,
except in the case of static observers. However, the moving-observer schemes
still outperform the static observer case in simulation time. When measured in
CPU time the moving-observer schemes outperform the static observer case
except in the case of Langrangian observers. It appears that Lagrangian tracer
particles tend to get stuck in eddies, decreasing their global coverage.
However, random the random sampling locations of the bleeps demonstrated
significantly better error. Therefore, for example, in the case of
observational devices attached to buoys, it may be worth considering
mechanisms to “kick” the buoys into a different location. For example, a small
motor or wind sail attached to Argo buoys could lead to significant decreases
the error.

In summary, this work demonstrates the enormous potential for improving data
assimilation speed, accuracy, and the number of observers required by allowing
for observer to move through the domain in certain patterns, sampling wider
regions of the flow. These results open the door to many new possibilities,
including optimizing movement patterns (using, e.g., machine learning
strategies), combining data from moving and non-moving sources (using, e.g.,
data from ground-based stationary sensors and fast-moving drones),
heterogeneous data sources (e.g., temperature and salinity of the ocean), and
many others. We will explore these ideas in future works.

##  Appendix A Additional Figures

Figure A.1. Energy spectrum of the initial data with $\nu=10^{-4}$,
$G=10^{6}$, and $\Delta t=0.005$. The vertical red line is the 2/3’s
dealiasing cutoff at $\frac{2}{3}\frac{N}{2}=341.\overline{3}$. Figure A.2.
Comparison of error over time for static uniform grid and dynamic observer
strategies with number of observers optimized to ensure convergence around the
time $t=70$ (log-linear plot). See legend entries for the number of observers
used in each method. $\mu=10$ for all methods except thin-sweeps, which uses
$\mu=30$. Thick-sweeps has constant $x$ velocity $3\frac{\Delta x}{\Delta t}$.
Figure A.3. Comparison of error over time for static uniform grid and dynamic
observer strategies all initialized using approximately 5,625 observers (log-
linear plot). See legend entries for the number of observers used in each
method. $\mu=10$ for all methods except thin-sweeps, which uses $\mu=30$.
Thick-sweeps has constant $x$ velocity $3\frac{\Delta x}{\Delta t}$.  Figure
A.4. Comparison of error over time for static uniform grid and dynamic
observer strategies with number of observers optimized to ensure convergence
around the time $t=70$ (log-linear plot). See legend entries for the number of
observers used in each method. $\mu=10$ for all methods except thin-sweeps,
which uses $\mu=30$. Thick-sweeps has constant $x$ velocity $3\frac{\Delta
x}{\Delta t}$.  Figure A.5. Comparison of error over time for static uniform
grid and dynamic observer strategies all initialized using approximately 5,625
observers (log-linear plot). See legend entries for the number of observers
used in each method. $\mu=10$ for all methods except thin-sweeps, which uses
$\mu=30$. Thick-sweeps has constant $x$ velocity $3\frac{\Delta x}{\Delta t}$.
Figure A.6. Comparison of error over time for static uniform grid and dynamic
observer strategies with number of observers optimized to ensure convergence
around the time $t=70$ (log-linear plot). See legend entries for the number of
observers used in each method. $\mu=10$ for all methods except thin-sweeps,
which uses $\mu=30$. Thick-sweeps has constant $x$ velocity $3\frac{\Delta
x}{\Delta t}$.  Figure A.7. Comparison of error over time for static uniform
grid and dynamic observer strategies all initialized using approximately 5,625
observers (log-linear plot). See legend entries for the number of observers
used in each method. $\mu=10$ for all methods except thin-sweeps, which uses
$\mu=30$. Thick-sweeps has constant $x$ velocity $3\frac{\Delta x}{\Delta t}$.
Figure A.8. Comparison of error over CPU time for static uniform grid and
dynamic observer strategies with number of observers optimized to ensure
convergence around the time $t=70$ (log-linear plot). See legend entries for
the number of observers used in each method. $\mu=10$ for all methods except
thin-sweeps, which uses $\mu=30$. Thick-sweeps has constant $x$ velocity
$3\frac{\Delta x}{\Delta t}$.  Figure A.9. Comparison of error over CPU time
for static uniform grid and dynamic observer strategies all initialized using
approximately 5,625 observers (log-linear plot). See legend entries for the
number of observers used in each method. $\mu=10$ for all methods except thin-
sweeps, which uses $\mu=30$. Thick-sweeps has constant $x$ velocity
$3\frac{\Delta x}{\Delta t}$.  Figure A.10. Comparison of error over CPU time
for static uniform grid and dynamic observer strategies with number of
observers optimized to ensure convergence around the time $t=70$ (log-linear
plot). See legend entries for the number of observers used in each method.
$\mu=10$ for all methods except thin-sweeps, which uses $\mu=30$. Thick-sweeps
has constant $x$ velocity $3\frac{\Delta x}{\Delta t}$.  Figure A.11.
Comparison of error over CPU time for static uniform grid and dynamic observer
strategies all initialized using approximately 5,625 observers (log-linear
plot). See legend entries for the number of observers used in each method.
$\mu=10$ for all methods except thin-sweeps, which uses $\mu=30$. Thick-sweeps
has constant $x$ velocity $3\frac{\Delta x}{\Delta t}$.  Figure A.12.
Comparison of error over CPU time for static uniform grid and dynamic observer
strategies with number of observers optimized to ensure convergence around the
time $t=70$ (log-linear plot). See legend entries for the number of observers
used in each method. $\mu=10$ for all methods except thin-sweeps, which uses
$\mu=30$. Thick-sweeps has constant $x$ velocity $3\frac{\Delta x}{\Delta t}$.
Figure A.13. Comparison of error over CPU time for static uniform grid and
dynamic observer strategies all initialized using approximately 5,625
observers (log-linear plot). See legend entries for the number of observers
used in each method. $\mu=10$ for all methods except thin-sweeps, which uses
$\mu=30$. Thick-sweeps has constant $x$ velocity $3\frac{\Delta x}{\Delta t}$.
Figure A.14. Comparison of Thick-Sweeps methods initialized with different
speeds (log-linear plot). Both trials were initialized with the 5,700
observers, $\mu=10$, moving with constant velocity in the $x$ direction as
given in legend parentheses.  Figure A.15. Comparison of Thick-Sweep methods
initialized with different speeds (log-linear plot).. Both trials were
initialized with the 5,700 observers, $\mu=10$, moving with constant velocity
in the $x$ direction as given in legend parentheses.

## Acknowledgments

Author T.E.F. acknowledges the financial support of the USDA National
Institute of Food and Agriculture Hatch project #1020768 and project
#2019-67021-29312. Author A.L. acknowledges the financial support of National
Science Foundation (NSF) grants DMS-1716801 and CMMI-1953346. Author C.V.
acknowledges the financial support of NSF GRFP grant DMS-1610400.

## References

  * [1] D. A. Albanez, H. J. Nussenzveig Lopes, and E. S. Titi.  Continuous data assimilation for the three-dimensional Navier–Stokes-$\alpha$ model.  Asymptotic Anal., 97(1-2):139–164, 2016.
  * [2] M. U. Altaf, E. S. Titi, O. M. Knio, L. Zhao, M. F. McCabe, and I. Hoteit.  Downscaling the 2D Benard convection equations using continuous data assimilation.  Comput. Geosci, 21(3):393–410, 2017.
  * [3] A. Azouani, E. Olson, and E. S. Titi.  Continuous data assimilation using general interpolant observables.  J. Nonlinear Sci., 24(2):277–304, 2014.
  * [4] A. Azouani and E. S. Titi.  Feedback control of nonlinear dissipative systems by finite determining parameters—a reaction-diffusion paradigm.  Evol. Equ. Control Theory, 3(4):579–594, 2014.
  * [5] H. Bessaih, E. Olson, and E. S. Titi.  Continuous data assimilation with stochastically noisy data.  Nonlinearity, 28(3):729–753, 2015.
  * [6] A. Biswas, Z. Bradshaw, and M. S. Jolly.  Data assimilation for the Navier-Stokes equations using local observables.  (submitted) arXiv 2008.06949.
  * [7] A. Biswas, Z. Bradshaw, and M. S. Jolly.  Data assimilation for the Navier-Stokes equations using local observables.  2020\.
  * [8] A. Biswas, C. Foias, C. F. Mondaini, and E. S. Titi.  Downscaling data assimilation algorithm with applications to statistical solutions of the Navier–Stokes equations.  In Annales de l’Institut Henri Poincaré C, Analyse non linéaire, pages 295–326. Elsevier, 2019.
  * [9] A. Biswas and V. R. Martinez.  Higher-order synchronization for a data assimilation algorithm for the 2D Navier–Stokes equations.  Nonlinear Anal. Real World Appl., 35:132–157, 2017.
  * [10] A. Biswas and R. Price.  Continuous data assimilation for the three dimensional Navier-Stokes equations.  (submitted) arXiv 2003.01329.
  * [11] E. Carlson, J. Hudson, and A. Larios.  Parameter recovery for the 2 dimensional Navier-Stokes equations via continuous data assimilation.  SIAM J. Sci. Comput., 42(1):A250–A270, 2020.
  * [12] E. Carlson, J. Hudson, A. Larios, V. R. Martinez, E. Ng, and J. P. Whitehead.  Dynamically learning the parameters of a chaotic system using partial observations.  (submitted) arXiv 2108.08354.
  * [13] E. Carlson and A. Larios.  Sensitivity analysis for the 2D Navier-Stokes equations with applications to continuous data assimilation.  J. Nonlinear Sci., 2021.  (to appear).
  * [14] E. Carlson, . Van Roekel, Luke, M. Petersen, H. C. Godinez, and A. Larios.  CDA algorithm implemented in MPAS-O to improve eddy effects in a mesoscale simulation.  2021\.  (submitted).
  * [15] E. Celik, E. Olson, and E. S. Titi.  Spectral filtering of interpolant observables for a discrete-in-time downscaling data assimilation algorithm.  SIAM J. Appl. Dyn. Syst., 18(2):1118–1142, 2019.
  * [16] N. Chen, Y. Li, and E. Lunasin.  An efficient continuous data assimilation algorithm for the sabra shell model of turbulence.  (submitted) arXiv 2105.10020.
  * [17] Y. Chen.  Covid-19 pandemic imperils weather forecast.  Geophysical research letters, 47(15):e2020GL088613, 2020.
  * [18] P. Clark Di Leoni, A. Mazzino, and L. Biferale.  Inferring flow parameters and turbulent configuration with physics-informed data assimilation and spectral nudging.  Phys. Rev. Fluids, 3(10):104604, 2018.
  * [19] P. Constantin and C. Foias.  Navier–Stokes Equations.  Chicago Lectures in Mathematics. University of Chicago Press, Chicago, IL, 1988.
  * [20] S. Desamsetti, H. Dasari, S. Langodan, O. Knio, I. Hoteit, and E. S. Titi.  Efficient dynamical downscaling of general circulation models using continuous data assimilation.  Quarterly Journal of the Royal Meteorological Society, 2019.
  * [21] A. E. Diegel and L. G. Rebholz.  Continuous data assimilation and long-time accuracy in a $c^{0}$ interior penalty method for the Cahn-Hilliard equation.  (submitted) arXiv 2106.14744.
  * [22] Y. J. Du and M.-C. Shiue.  Analysis and computation of continuous data assimilation algorithms for Lorenz 63 system based on nonlinear nudging techniques.  J. Comput. Appl. Math., 386:113246, 2021.
  * [23] A. Farhat, M. S. Jolly, and E. S. Titi.  Continuous data assimilation for the 2D Bénard convection through velocity measurements alone.  Phys. D, 303:59–66, 2015.
  * [24] A. Farhat, E. Lunasin, and E. S. Titi.  Abridged continuous data assimilation for the 2D Navier–Stokes equations utilizing measurements of only one component of the velocity field.  J. Math. Fluid Mech., 18(1):1–23, 2016.
  * [25] A. Farhat, E. Lunasin, and E. S. Titi.  Data assimilation algorithm for 3D Bénard convection in porous media employing only temperature measurements.  J. Math. Anal. Appl., 438(1):492–506, 2016.
  * [26] A. Farhat, E. Lunasin, and E. S. Titi.  On the Charney conjecture of data assimilation employing temperature measurements alone: the paradigm of 3D planetary geostrophic model.  Mathematics of Climate and Weather Forecasting, 2(1), 2016.
  * [27] A. Farhat, E. Lunasin, and E. S. Titi.  Continuous data assimilation for a 2D Bénard convection system through horizontal velocity measurements alone.  J. Nonlinear Sci., pages 1–23, 2017.
  * [28] C. Foias, C. F. Mondaini, and E. S. Titi.  A discrete data assimilation scheme for the solutions of the two-dimensional Navier–Stokes equations and their statistics.  SIAM J. Appl. Dyn. Syst., 15(4):2109–2142, 2016.
  * [29] K. Foyash, M. S. Dzholli, R. Kravchenko, and È. S. Titi.  A unified approach to the construction of defining forms for a two-dimensional system of Navier–Stokes equations: the case of general interpolating operators.  Uspekhi Mat. Nauk, 69(2(416)):177–200, 2014.
  * [30] B. García-Archilla and J. Novo.  Error analysis of fully discrete mixed finite element data assimilation schemes for the Navier-Stokes equations.  Adv. Comput. Math., 46(4):Paper No. 61, 33, 2020.
  * [31] B. García-Archilla, J. Novo, and E. S. Titi.  Uniform in time error estimates for a finite element method applied to a downscaling data assimilation algorithm for the Navier-Stokes equations.  SIAM J. Numer. Anal., 58(1):410–429, 2020.
  * [32] M. Gardner, A. Larios, L. G. Rebholz, D. Vargun, and C. Zerfas.  Continuous data assimilation applied to a velocity-vorticity formulation of the 2D Navier-Stokes equations.  Electron Res. Arch., 29(3):2223–2247, 2021.
  * [33] M. Gesho, E. Olson, and E. S. Titi.  A computational study of a data assimilation algorithm for the two-dimensional Navier–Stokes equations.  Commun. Comput. Phys., 19(4):1094–1110, 2016.
  * [34] H. A. Ibdah, C. F. Mondaini, and E. S. Titi.  Fully discrete numerical schemes of a data assimilation algorithm: uniform-in-time error estimates.  IMA Journal of Numerical Analysis, 40(4):2584–2625, 2020.
  * [35] M. S. Jolly, V. R. Martinez, E. J. Olson, and E. S. Titi.  Continuous data assimilation with blurred-in-time measurements of the surface quasi-geostrophic equation.  Chin. Ann. Math. Ser. B, 40(5):721–764, 2019.
  * [36] M. S. Jolly, V. R. Martinez, and E. S. Titi.  A data assimilation algorithm for the subcritical surface quasi-geostrophic equation.  Adv. Nonlinear Stud., 17(1):167–192, 2017.
  * [37] M. S. Jolly, T. Sadigov, and E. S. Titi.  A determining form for the damped driven nonlinear Schrödinger equation—Fourier modes case.  J. Differential Equations, 258(8):2711–2744, 2015.
  * [38] A. Larios and Y. Pei.  Nonlinear continuous data assimilation.  (submitted) arXiv:1703.03546.
  * [39] A. Larios and Y. Pei.  Approximate continuous data assimilation of the 2D Navier–Stokes equations via the Voigt-regularization with observable data.  Evol. Equ. Control Theory, 9(3):733–751, 2020.
  * [40] A. Larios, L. G. Rebholz, and C. Zerfas.  Global in time stability and accuracy of IMEX-FEM data assimilation schemes for Navier-Stokes equations.  Computer Methods in Applied Mechanics and Engineering, 2018.
  * [41] A. Larios and C. Victor.  Improving convergence rates of continuous data assimilation for 2D Navier-Stokes using observations that are sparse in space and time.  (preprint).
  * [42] A. Larios and C. Victor.  Continuous data assimilation with a moving cluster of data points for a reaction diffusion equation: A computational study.  Commun. Comp. Phys., 29:1273–1298, 2021.
  * [43] E. Lunasin and E. S. Titi.  Finite determining parameters feedback control for distributed nonlinear dissipative systems—a computational study.  Evol. Equ. Control Theory, 6(4):535–557, 2017.
  * [44] P. A. Markowich, E. S. Titi, and S. Trabelsi.  Continuous data assimilation for the three-dimensional Brinkman-Forchheimer-extended Darcy model.  Nonlinearity, 29(4):1292–1328, 2016.
  * [45] C. F. Mondaini and E. S. Titi.  Uniform-in-time error estimates for the postprocessing Galerkin method applied to a data assimilation algorithm.  SIAM J. Numer. Anal., 56(1):78–110, 2018.
  * [46] E. Olson and E. S. Titi.  Determining modes for continuous data assimilation in 2D turbulence.  J. Statist. Phys., 113(5-6):799–840, 2003.  Progress in statistical hydrodynamics (Santa Fe, NM, 2002).
  * [47] E. Olson and E. S. Titi.  Determining modes and grashof number in 2D turbulence: a numerical case study.  Theor. Comp. Fluid Dyn., 22(5):327–339, 2008.
  * [48] B. Pachev, J. P. Whitehead, and S. A. McQuarrie.  Concurrent multi-parameter learning demonstrated on the Kuramoto-Sivashinsky equation.  2021\.
  * [49] Y. Pei.  Continuous data assimilation for the 3D primitive equations of the ocean.  Commun. Pure Appl. Anal., 18(2):643–661, 2019.
  * [50] L. G. Rebholz and C. Zerfas.  Simple and efficient continuous data assimilation of evolution equations via algebraic nudging.  Numer. Methods Partial Differ. Equations, pages 1–25, 2021.
  * [51] Spaceballs.  Mel Brooks (producer and director). Metro-Goldwyn-Mayer and Brooksfilms, 1987.
  * [52] R. Temam.  Navier–Stokes Equations and Nonlinear Functional Analysis, volume 66 of CBMS-NSF Regional Conference Series in Applied Mathematics.  Society for Industrial and Applied Mathematics (SIAM), Philadelphia, PA, second edition, 1995.
  * [53] C. Zerfas, L. G. Rebholz, M. Schneier, and T. Iliescu.  Continuous data assimilation reduced order models of fluid flow.  Comput. Methods Appl. Mech. Engrg., 357:112596, 18, 2019.


# The Quiet Eye Phenomenon in Minimally Invasive Surgery

Alaa Eldin Abdelaal1 , Rachelle Van Rumpt2, Sayem Nazmuz Zaman3, Irene Tong2,
Anthony Jarc4, Gary L. Gallia5, Masaru Ishii6, Gregory D. Hager7 , and
Septimiu E. Salcudean1 Manuscript received September 6, 2023. This work was
supported in part by the Natural Sciences and Engineering Research Council of
Canada (Discovery Grant), in part by the Canada Foundation for Innovation
(infrastructure and operating funds), in part by Intuitive Surgical (equipment
donation), in part by an Intuitive Foundation Clinical Research Grant, in part
by the C.A. Laszlo Chair in Biomedical Engineering held by Prof. Salcudean,
and in part by the Vanier Canada Graduate Scholarship and a Michael Smith
Foreign Study Supplement both held by Alaa Eldin Abdelaal.1A. E. Abdelaal is
with the Mechanical Engineering Department, Stanford University, Stanford,
USA. (email: abdelaal@stanford.edu)2 R. Van Rumpt, I. Tong and S. E. Salcudean
are with the Electrical and Computer Engineering Department, University of
British Columbia, 2332 Main Mall, Vancouver, BC Canada. 3S. N. Zaman is with
the Department of Physics and Astronomy, University of British Columbia, 325 -
6224 Agricultural Road, Vancouver, BC Canada.4A. Jarc is with Intuitive
Surgical, Atlanta, GA, USA.5G. L. Gallia is with Department of Neurosurgery,
Johns Hopkins University School of Medicine, Baltimore, MD, USA.6M. Ishii is
with Department of Otolaryngology – Head and Neck Surgery, Johns Hopkins
University School of Medicine, Baltimore, MD, USA7G. D. Hager is with
Department of Computer Science, Johns Hopkins University, Baltimore, MD 21218,
USA.

###### Abstract

In this paper, we report our discovery of a gaze behavior called Quiet Eye
(QE) in minimally invasive surgery. The QE behavior has been extensively
studied in sports training and has been associated with higher level of
expertise in multiple sports. We investigated the QE behavior in two
independently collected data sets of surgeons performing tasks in a sinus
surgery setting and a robotic surgery setting, respectively. Our results show
that the QE behavior is more likely to occur in successful task executions and
in performances of surgeons of high level of expertise. These results open the
door to use the QE behavior in both training and skill assessment in minimally
invasive surgery.

###### Index Terms:

Eye gaze Tracking, Minimally Invasive Surgery, Robot-Assisted Surgery,
Surgical Skill Assessment, Surgical Training

##  I Introduction

Studying how experts see their workspace when performing a task is one of the
main research topics in motor skill learning. In particular, researchers in
this area are interested in investigating where experts look in their
workspace, when they look at critical cues, for how long, and how this
behavior is coordinated with the body/hand motions of these experts. Answering
these questions can lead to interesting insights that can then be used to
train novices.

Indeed, gaze-based training methods have shown great success in many fields
including sports [1], law enforcement [2] and the military [3]. The general
idea of these methods is to train novices to adopt the gaze behaviors of
experts. The essence of these methods is to teach novices where and when to
look at locations of interest and, for how long. The underlying argument is
that the motor skills of novices can be improved if they adopt the “visual
behavior” of experts.

One promising gaze-based training method is called quiet eye (QE) training.
The term “quiet eye” refers to the final fixation right before a performer
carries out a critical movement to execute a task [4]. The period of this
final fixation is called the QE period. Research in sports shows that a longer
QE period is associated with a higher level of expertise [5]. This QE behavior
has been discovered in experts’ performances in targeting tasks in sports such
as basketball free throws [6] and golf putting [7]. One theory explaining such
behavior states that this long QE period is the period needed to change the
neural connections in the performer’s brain to absorb and process the
important visual information and plan the critical movement of the task before
actually carrying it out [8].

As many surgical tasks can be considered as targeting tasks, QE training has
the potential to be applied in surgical training, inspired by the strong
evidence of its effectiveness in other domains. Studies in this area have been
limited and are focused solely on open surgery settings. In this work, we
investigate the QE phenomenon in the context of minimally invasive surgery
(MIS). In particular, the contributions of this work are:

  * •

To investigate the existence of QE behavior in surgeons’ performances in two
minimally-invasive surgery (MIS) settings. Our investigation is conducted on
two independently collected data sets, one in a sinus surgery setting and the
other in a robotic surgery setting using the da Vinci Surgical robot.

  * •

To observe and report the changes in the QE behavior in successful and
unsuccessful tasks.

  * •

To observe and report the changes in the QE behavior between highly
experienced and less experienced surgeons.

These contributions provide the foundation to understand the QE behavior in
MIS and open the door for further research on the effect of applying QE
training in this setting.

##  II Related Work

Researchers in sports training have extensively studied the eye gaze of
experts as a proxy of how they allocate their attention. To achieve that, they
follow what is called the vision-in-action paradigm for data collection [6],
where they collect the gaze data of experts as they actually perform the task.
In addition, they also use cameras to capture synchronized views of the body
of the experts as well as views of what they see in their workspace. The
analysis of the collected data has been used to find interesting gaze patterns
in experts and study how such patterns differ from novices. Such an approach
has been successfully applied in many sports such as basketball [9], pistol
shooting [10], badminton [11] and golf [7].

A phenomenon called Quiet Eye (QE) was discovered by Joan Vickers when
analyzing the gaze behavior of experts in basketball [6] and it was identified
and associated with experts’ performances in many other sports later on as in
[5, 12, 13]. QE is defined as the final fixation (or tracking gaze) right
before a performer carries out a critical movement to execute a task [4]. The
period of this behavior is called the QE period.

Thus far, the QE phenomenon has been confirmed in three main task categories:
targeting tasks [14] (such as basketball free throws and billiards),
interceptive timing tasks [13] (such as responding to the serve in volleyball
and table tennis) and tactical tasks [15] (such as soccer defence and
offence). Adopting the QE behavior of experts has been shown to be effective
in training novices in these three task categories.

QE behavior can be identified by looking for the following measurable
characteristics in the collected data [16]:

  * •

It is a fixation (or tracking gaze) on a specific target in the workspace
which strictly means that the performer’s gaze remains within 3 degrees of
visual angle (or less) from the same target for at least 100 ms.

  * •

The beginning of the QE period (which is also called QE onset) begins right
before the critical movement of the task. For instance, if it is a targeting
task, then the QE period begins right before its aiming part.

  * •

The end of the QE period (or the QE offset) occurs when the fixation (or
tracking gaze) deviates from the location/object of interest by more than 3
degrees of visual angle for longer than 100 ms.

The successful use of the QE in sports encouraged its exploration in surgical
training. For example, Causer et al. [17] conducted a user study to assess the
efficiency of QE in open surgery setting in a knot tying task. In their study,
the trainees were divided into two groups, a QE training group and a
conventional training group. Their results show that the QE group performs
better than the other one in terms of completion time and efficiency of the
hand movements during the task. The results of another study [18] also show
that the QE training group performance is more robust to high levels of
pressure and anxiety compared with the other group. Through analyzing the gaze
behavior, the authors argue that QE training directs the trainees’ gaze
towards a single point of focus, which increases the effectiveness of their
attention allocation to the most important information for the task at hand.
This frees up some mental processing resources that enable QE trainees to
overcome high anxiety levels.

To the best of our knowledge, the QE phenomenon has neither been identified in
surgeons’ gaze behavior nor applied to surgical training in MIS settings. We
argue that many surgical tasks fall under the targeting task category.
Findings from sports research suggest that there is a very high chance of
finding the QE phenomenon in surgeons’ performances of tasks in this category
in our case. In addition, according to Newell’s Constraints-Led Model [19], a
change in the environment/setting is a significant factor to consider before
studying (and later on applying) a motor skill learning phenomenon to other
environments, which justifies the need to study the QE phenomenon in MIS
settings (in contrast to open surgery settings as in previous work).

In this work, we take the first step towards filling this gap by investigating
the existence of the QE behavior in surgeons’ performances in two different
MIS settings. In addition, we report our observations regarding how the QE
behavior changes with successful/unsuccessful task executions, and with
surgeons’ experience level. We believe that by doing so, this work provides a
deeper understanding of the QE behavior in MIS, which, in turn, can be
beneficial in applications such as training and skill assessment in MIS.

##  III Methods and Hypotheses

###  III-A Hypotheses

In this work we have two hypotheses as follows:

  * •

Hypothesis 1: QE behavior occurs more frequently in successful tasks compared
with unsuccessful ones. In addition, QE duration is longer in successful tasks
than in unsuccessful ones.

  * •

Hypothesis 2: QE behavior occurs more frequently in performances from highly
experienced surgeons compared with less experienced ones. In addition, QE
duration is longer in the former case than the latter one.

We verify the first hypothesis using a data set in a sinus surgery setting.
The second hypothesis is verified using another data set in a robotic surgery
setting using the da Vinci surgical robot.

###  III-B Sinus Surgery Data Set

The sinus surgery data set is a data set of a series of targeting tasks on a
cadaver model [20]. The data set was collected from 20 surgeons at the annual
resident endoscopic skull-base surgery course at Johns Hopkins University. The
subjects were asked to visualize and touch nine endonasal structures using an
endoscope and pointer. The experimental setup used for the data collection is
shown in Fig. 1. An electromagnetic tracker was used to track the positions of
the pointer and endoscope, and an eye gaze tracker was used to collect the
subject’s eye gaze position on the video monitor every 0.02 seconds. Videos of
what subjects saw were recorded directly using a video capture card situated
in a data collection computer. Temporal synchronization between all the
collected data units was achieved using time stamps assigned to each of these
units.

Figure 1: The experimental setup used for the sinus surgery data collection
[20]. This figure is used with permission.

Each video in the data set represents a targeting task by one surgeon, aiming
for a predefined target inside the sinus anatomy. Each video was assessed by
three independent assessors. Each assessor gave each video a task score of
either zero, if the task was not successful, or one if it was successful. This
makes the data set a suitable candidate to investigate the QE behavior and how
it changes with the successful execution of these targeting tasks.

The video annotations were carried out manually based on the QE three
measurable characteristics mentioned in Section II. In particular, we annotate
the target location in the first frame in each video and track it (along with
its surrounding region) in the subsequent frames using OpenCV. This gives us
the 2D pixel location of the target in each frame. The raw data contains the
2D pixel location of the surgeon’s eye gaze in each frame and its
corresponding time stamp. Using this information, we compute all the fixations
on the target location, within three degrees of visual angle, that last for at
least 100 ms. The three degrees of visual angle are represented on each frame
as a circle whose center is the target pixel location. The radius of this
circle is computed by projecting the three degrees of visual angle from the
surgeon’s eye onto the screen. To compute this radius in pixels, we use the
distance between the surgeon’s eye and screen, the size of the screen and its
resolution. All these values are known from the data collection process. A
fixation is detected as long as the surgeon’s eye gaze location is within this
circle for at least 100 ms. Using time stamps, we also compute the duration of
each detected fixation. To find out if QE occurs or not, we first manually
identify the frame number right before the start of the aiming part of the
targeting task, and its associated time stamp. Using this information, we
conclude that QE occurs in a video if and only if a fixation starts at or
before this time stamp and lasts after the start of the aiming part of the
task. The QE duration is the same as the duration of this fixation. The data
set was annotated by two independent annotators and the interclass correlation
between them is 1 for QE detection and 0.9903 for QE duration.

The data set has a total of 302 videos and we excluded 28 of them because they
were difficult to annotate due to the unstable visual feedback caused by
frequent camera movement. For the remaining 274 videos, we annotated each
video so that we can learn two things:

  * •

Whether the QE exists in the video. In this case, the video was assigned a
binary value (e.g., one if the QE occurs in the video and zero otherwise).

  * •

The QE duration in milliseconds, only if QE exists in the video.

A sample annotated frame from one of the videos where QE occurred is shown in
Fig. 2.

Figure 2: A sample annotated frame from one of the videos where QE occurred in
the sinus surgery data set. The blue point represents the target location, the
red point represents the surgeon’s eye gaze, and the green circle represents
the area within $3$ degrees of visual angle around the target (according to
the definition of QE). A QE green flag appears on the top right of the view
when the QE occurred in the video.

To verify the first hypothesis, we only considered the videos with unanimous
task scores (either zero or one) from the three assessors. This represents a
total of 187 videos from the entire data set.

###  III-C Robotic Surgery Data Set

The robotic surgery data set was collected from seven RAS surgeons operating
on the da Vinci Si Surgical System [21]. The surgeons performed porcine and
cadaver exercises. During each exercise, the estimated eye gaze for each
surgeon’s left and right eye was collected using EyeTech VT2 mini eye tracker
(EyeTech Digital Systems, Mesa, AZ), that was placed within the da Vinci Si
system. The surgeon’s point of gaze was then calculated as the mean of the
left and right eye locations. In addition, the instrument kinematics, system
events, and endoscope videos were collected directly from the da Vinci
platform, and were synchronized with the eye gaze data using the associated
time stamps.

Since we are interested in targeting tasks, we annotated the videos of all the
suturing exercises in the data set. This is because the suturing task has a
clear targeting component whenever the surgeon inserts the needle into the
tissue. The annotation was performed manually, similar to the annotation of
the sinus surgery data set. A sample annotated frame from one of the videos
where QE occurs is shown in Fig. 3

Figure 3: A sample annotated frame from one of the videos where QE occurred in
the robotic surgery data set. The blue point represents estimated gaze for the
surgeon’s right eye and the green point represents estimated gaze for the
surgeon’s left eye. The blue circle represents the area within 3 degrees of
visual angle around the target (according to the definition of QE). A QE green
flag appears on the top middle of the view when the QE occurred in the video.

In total, we had 33 videos of suturing exercises performed by two surgeons.
The first surgeon has approximately 20 years of surgical experience and the
second surgeon has less than 5 years of surgical experience. The remaining
surgeons in the data set did not perform any suturing exercises. The results
from these videos are used to verify our second hypothesis.

###  III-D Performance Metrics

We considered in our investigation two metrics: (i) the percentage of the
videos that have QE for each considered task score/experience level, and (ii)
the QE duration in these videos.

##  IV Results

###  IV-A Sinus Surgery Data Set Results

The results of the sinus surgery data set, as shown in Fig. 4 and Fig. 5, are
used to verify our first hypothesis. Results show that videos with a unanimous
task score of one have on average 22% more videos with QE compared with videos
with a task score of zero as shown in Fig. 4. Furthermore, the QE duration in
videos with a task score of one is on average 2.3 times longer than the QE
duration in videos with a task score of zero as shown in Fig. 5.

Figure 4: The percentage of the videos that have QE for each considered task
score. Figure 5: The QE duration for each considered task score.

We perform hypothesis testing using a Mann–Whitney U test on the above
results. The two results were statistically significant, with p $<$ 0.05 in
the two cases. These results verify our first hypothesis. They show that the
QE behavior occurs more frequently in successful tasks than unsuccessful ones.
In addition, the results show that QE duration is longer in successful tasks
than unsuccessful ones.

###  IV-B Robotic Surgery Data Set Results

The results of the robotic surgery data set are used to verify our second
hypothesis. The results show that 40% of the more experienced surgeon’s videos
have QE, compared with 0% of the less experienced surgeon’s videos. The
average QE duration in the more experienced surgeon’s videos is 867 $\pm$ 480
ms, compared with zero ms in the less experienced surgeon’s videos, since QE
was not found in any of them.

We conducted two hypothesis tests on the above results. A Chi-square test
(with $\alpha$ = 0.05) was used to assess whether the QE phenomenon occurs
more often with the more experienced surgeon than with the less experienced
one. The test was statistically significant with ${\chi}$2(1) = 6.864 and p
$<$ 0.05. Moreover, a Mann–Whitney U test was used to assess whether QE
duration is longer in the more experienced surgeon’s case than the less
experienced one. This test was also statistically significant with p $<$ 0.05.
These results verify our second hypothesis.

##  V Discussion

In this work, we report the discovery of the QE behavior in two independently
collected data sets in two MIS settings. Our results show that the QE behavior
is more likely to be observed in successful targeting tasks (compared with
unsuccessful ones) and in targeting tasks performed by more experienced
surgeons (compared with less experienced ones). In addition, the results show
that QE duration is significantly longer in these two cases. Besides, our
results show that the QE duration is potentially a stronger distinguishing
factor between different performances (based on the success of the performance
or the experience level of the surgeon) than the existence of the QE behavior
on its own. These results are consistent with previous findings in the sports
training literature [16].

One limitation, though, of this study is the small number of surgeons whose
data were used in our investigation. We used data from 20 surgeons in the
first data set and from two surgeons in the second data set. More data sets,
with more surgeons and more targeting tasks, are needed to reach more
conclusive results and investigate the QE behavior in a variety of other
targeting tasks in MIS.

Nevertheless, the potential impact of this study spans the areas of training,
skill assessment and human-robot interaction in MIS. For training, this work
opens the door to testing QE training [22] in MIS settings, with the evidence
it provides that QE behavior does exist in this setting. Since the
characteristics of the QE behavior are well defined, we think that it can be
relatively easy to integrate QE training into existing surgical training
methods. Furthermore, QE training has been shown to be robust against high
levels of anxiety and pressure [14], which can make it even more valuable for
surgical training in MIS.

The results of our study can be useful for surgical skill assessment in MIS.
Learning how the QE behavior changes as the surgeon’s skill/experience level
increases can open the door to monitor these changes and use them as a tool to
classify the skill levels of practicing surgeons based on their QE behavior.
Furthermore, it would be interesting to investigate the effect of adding the
QE behavior related features into existing machine learning based skill
assessment systems [23] on their classification performance. Moreover, the
tasks we considered in this study provide a clue of the family of the surgical
tasks where using the QE behavior can be useful for skill assessment.

Our study can lead to some important implications on the design of human-robot
interfaces in RAS. In particular, our study highlights the importance of
collecting and investigating eye gaze data in this setting. This provides
additional evidence of the potential value of integrating eye gaze tracking in
new and existing RAS platforms and simulation environments [24] [25].

##  VI Conclusion and Future Work

In this paper, we report our discovery of the QE phenomenon in MIS. We
investigated the existence and duration of the QE behavior in two
independently collected data sets in a sinus surgery setting and a robotic
surgery setting. In two targeting tasks, we observed that the QE behavior
occurs more frequently and for longer durations in successful task
performances (compared with unsuccessful ones) and in performances by
experienced surgeons (compared with less experienced ones). These observations
are consistent with similar findings in sports training, where the QE behavior
has been successfully used to improve the motor skills of novice trainees.

This work opens up several new lines of research in surgical training and
skill assessment in MIS. For instance, more investigations are needed to
identify more tasks where the QE behavior would occur. These tasks can then be
used to test the effect of integrating QE training into the traditional
training approaches of these tasks. The integration of the QE training has the
potential to improve the motor skill learning aspects of these tasks and can
potentially make the traditional training methods more robust to situations of
high anxiety and pressure. Furthermore, it would be interesting to include the
well-defined characteristics of the QE behavior as novel features in skill
assessment machine learning models and evaluate how they can improve their
performance.

## ACKNOWLEDGMENT

We would like to thank Maram Sakr and Tim Powers for their assistance with the
statistical analysis of the results of this work.

## References

  * [1] J. Causer, S. J. Bennett, P. S. Holmes, C. M. Janelle, and A. M. Williams, “Quiet eye duration and gun motion in elite shotgun shooting,” _Medicine and Science in Sports and Exercise_ , vol. 42, no. 8, pp. 1599–1608, 2010.
  * [2] J. N. Vickers and W. Lewinski, “Performing under pressure: Gaze control, decision making and shooting performance of elite and rookie police officers,” _Human Movement Science_ , vol. 31, no. 1, pp. 101–117, 2012\.
  * [3] P. Ward, D. Farrow, K. R. Harris, A. M. Williams, D. W. Eccles, and K. A. Ericsson, “Training perceptual-cognitive skills: Can sport psychology research inform military decision training?” _Military Psychology_ , vol. 20, no. sup1, pp. S71–S102, 2008.
  * [4] J. N. Vickers, “Advances in coupling perception and action: the quiet eye as a bidirectional link between gaze, attention, and action,” _Progress in Brain Research_ , vol. 174, pp. 279–288, 2009.
  * [5] A. M. Williams, R. N. Singer, and S. G. Frehlich, “Quiet eye duration, expertise, and task complexity in near and far aiming tasks,” _Journal of Motor Behavior_ , vol. 34, no. 2, pp. 197–207, 2002.
  * [6] J. N. Vickers, “Visual control when aiming at a far target.” _Journal of Experimental Psychology: Human Perception and Performance_ , vol. 22, no. 2, p. 342, 1996.
  * [7] ——, “Gaze control in putting,” _Perception_ , vol. 21, no. 1, pp. 117–132, 1992.
  * [8] D. T. Mann, S. A. Coombes, M. B. Mousseau, and C. M. Janelle, “Quiet eye and the bereitschaftspotential: visuomotor mechanisms of expert motor performance,” _Cognitive Processing_ , vol. 12, no. 3, pp. 223–234, 2011\.
  * [9] H. Ripoll, C. Bard, and J. Paillard, “Stabilization of head and eyes on target as a factor in successful basketball shooting,” _Human Movement Science_ , vol. 5, no. 1, pp. 47–58, 1986.
  * [10] H. Ripoll, J.-P. Papin, J.-Y. Guezennec, J.-P. Verdy, and M. Philip, “Analysis of visual scanning patterns of pistol shooters,” _Journal of Sports Sciences_ , vol. 3, no. 2, pp. 93–101, 1985.
  * [11] H. Ripoll and P. Fleurance, “What does keeping one’s eye on the ball mean?” _Ergonomics_ , vol. 31, no. 11, pp. 1647–1654, 1988.
  * [12] D. Panchuk, J. N. Vickers, and W. G. Hopkins, “Quiet eye predicts goaltender success in deflected ice hockey shots,” _European Journal of Sport Science_ , vol. 17, no. 1, pp. 93–99, 2017.
  * [13] G. Sun, L. Zhang, S. J. Vine, and M. R. Wilson, “The quiet eye provides preplanning and online control support for interceptive task performance,” _Journal of Sport and Exercise Psychology_ , vol. 38, no. 5, pp. 458–469, 2016.
  * [14] S. J. Vine, L. J. Moore, and M. R. Wilson, “Quiet eye training: The acquisition, refinement and resilient performance of targeting skills,” _European Journal of Sport Science_ , vol. 14, no. sup1, pp. S235–S242, 2014.
  * [15] A. M. Williams and K. Davids, “Visual search strategy, selective attention, and expertise in soccer,” _Research Quarterly for Exercise and Sport_ , vol. 69, no. 2, pp. 111–128, 1998.
  * [16] J. N. Vickers, _Perception, cognition, and decision training: The quiet eye in action_. Human Kinetics, 2007.
  * [17] J. Causer, A. Harvey, R. Snelgrove, G. Arsenault, and J. N. Vickers, “Quiet eye training improves surgical knot tying more than traditional technical training: A randomized controlled study,” _The American Journal of Surgery_ , vol. 208, no. 2, pp. 171–177, 2014.
  * [18] J. Causer, J. N. Vickers, R. Snelgrove, G. Arsenault, and A. Harvey, “Performing under pressure: quiet eye training improves surgical knot-tying performance,” _Surgery_ , vol. 156, no. 5, pp. 1089–1096, 2014.
  * [19] K. Newell, “Constraints on the development of coordination,” _Motor Development in Children: Aspects of Coordination and Control_ , 1986.
  * [20] N. Ahmidi, M. Ishii, G. Fichtinger, G. L. Gallia, and G. D. Hager, “An objective and automated method for assessing surgical skill in endoscopic sinus surgery using eye-tracking and tool-motion data,” in _International Forum of Allergy & Rhinology_, vol. 2, no. 6. Wiley Online Library, 2012, pp. 507–515.
  * [21] I. G. Tong, “Eye gaze tracking in surgical robotics,” Master’s thesis, University of British Columbia, 2017.
  * [22] J. Digmann, Z. Prouty, A. Klein, W. Castro, A. Lang, F. Fontana, and M. Mack, “Quiet eye: Practical applications in sport and physical education,” _Journal of Physical Education, Recreation & Dance_, vol. 89, no. 9, pp. 20–25, 2018.
  * [23] S. S. Vedula, M. Ishii, and G. D. Hager, “Objective assessment of surgical technical skill and competency in the operating room,” _Annual Review of Biomedical Engineering_ , vol. 19, p. 301, 2017.
  * [24] I. Tong, O. Mohareri, S. Tatasurya, C. Hennessey, and S. Salcudean, “A retrofit eye gaze tracker for the da vinci and its integration in task execution using the da vinci research kit,” in _Proc. of IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS’15)_ , 2015, pp. 2043–2050.
  * [25] M. Nathan, “The Senhance® Surgical System,” in _Robotic Surgery_. Springer, 2021, pp. 159–163.