# Detecting Worker Attention Lapses in Human-Robot Interaction: An Eye
Tracking and Multimodal Sensing Study

Zhuangzhuang Dai1, Jinha Park2, Aleksandra Kaszowska3, and Chen Li4 1Z. Dai is
with Dept. of Computer Science, Engineering and Physical Science, School of
Engineering and Applied Science, Aston University, B4 7ET, Birmingham, United
Kingdom<EMAIL_ADDRESS>Park is with the SDU Robotics, The Maersk Mc
Kinney Moller Institute, University of Southern Denmark, Campusvej 55, 5230
Odense, Denmark<EMAIL_ADDRESS>Kaszowska is with the Department of
Electronic Systems, Aalborg University, Fredrik Bajers Vej 7, B5-215, 9220,
Aalborg, Denmark<EMAIL_ADDRESS>Li is with the Department of Materials
and Production, Aalborg University, Fibigerstræde 16, 9220, Aalborg, Denmark
<EMAIL_ADDRESS>
###### Abstract

The advent of industrial robotics and autonomous systems endow human-robot
collaboration in a massive scale. However, current industrial robots are
restrained in co-working with human in close proximity due to inability of
interpreting human agents’ attention. Human attention study is non-trivial
since it involves multiple aspects of the mind: perception, memory, problem
solving, and consciousness. Human attention lapses are particularly
problematic and potentially catastrophic in industrial workplace, from
assembling electronics to operating machines. Attention is indeed complex and
cannot be easily measured with single-modality sensors. Eye state, head pose,
posture, and manifold environment stimulus could all play a part in attention
lapses. To this end, we propose a pipeline to annotate multimodal dataset of
human attention tracking, including eye tracking, fixation detection, third-
person surveillance camera, and sound. We produce a pilot dataset containing
two fully annotated phone assembly sequences in a realistic manufacturing
environment. We evaluate existing fatigue and drowsiness prediction methods
for attention lapse detection. Experimental results show that human attention
lapses in production scenarios are more subtle and imperceptible than well-
studied fatigue and drowsiness.

Keywords— Human attention monitoring, eye tracking, industrial robots, Human-
Robot Interaction

##  I INTRODUCTION

Human attention is a cognitive process that involves a combination of
physiological, psychological, and environmental attributes. Human attention
lapses, characterized by the loss of focus or distraction, can significantly
compromise task performance and bring about hazards for workers. Costs can be
unbearably high when failing to detect attention lapses in circumstances such
as assembling electronics and operating machines. In particular, the
detrimental impact of attention lapses can result in errors, safety
violations, and incurring significant impedance for industry to deploy
autonomous robotic systems in context of Industrial 5.0.

Attention is a state in which one’s cognitive resources are focused on certain
aspects of the environment, and the operator is ready to respond to
environmental stimuli. It can be inferred via physiological and behavioral
indicators but not a physical entity to be measured directly. Attention lapses
manifest widely across individuals due to different cognitive abilities,
emotional states, and environment variations. These altogether make attention
lapses difficult to monitor, nor to measure them accurately.

Figure 1: An example of distraction (looking at a mobile device) at workplace.
Since the worker retains similar gaze, head pose, and posture in (a) dedicated
and (b) distracted states, it is difficult to perceive such subtle difference
without eye tracking. Vision-based detector, e.g., OpenFace [1] (mid row),
fails in contrasting attention lapses to normal states.

In order to track attention, our intuition is that many sensor modalities
should work together to provide useful information. Given the multimodal
nature of attention, it can manifest in different ways, such as eye blinking,
looking away, moving head or arm, speaking, or signs of drowsiness [26]. In
light of thriving multi-modality sensing systems, the combination of data on
gaze, head pose, and audio can provide rich information for human attention
and attention lapse study. Measuring attention lapses multimodally and
automatically underlies a key aspect of promoting perception systems of social
robots as well as improving worker safety. Social robots will benefit
tremendously from better comprehending human’s attention and disengagement, so
as to make more appropriate interactions.

A multimodal sensing approach to attention lapse detection could assist in
understanding the mechanisms of attention and developing effective
interventions to improve human-robot collaboration. As can be seen from Fig.
1, a worker who looks in a correct direction with normal head pose and facial
expression could be either engaged in working or distracted. Subtle
differences in the worker’s focus make recognizing such attention lapse a
challenge. Without eye tracking to reveal human’s cognitive state it is almost
impossible to approach effective attention lapse detection. Nonetheless,
relying solely on gaze modality will compromise reliability when there is
occlusion of pupil or visual degradation.

To this end, we first curated a dataset of human operators conducting assembly
task in a realistic production environment. A Franka robot played an assistive
role in handing over components to the human agent [14]. A Pupil Core eye
tracker and a Microsoft Azure Kinect RGB-D camera with microphone are used for
data collection. This setup allows multimodally and unobtrusively tracking the
human operator’s gaze, egocentric world view, as well as head pose, body
posture, and environmental stimuli from a third-person perspective [5]. In
order to streamline data annotation and pre-processing, we developed a
pipeline to label distracted attention states frame-by-frame and visualizing
the timelines. We also set up Apriltags [13] in the environment for head pose
tracking. We used open-source tools [3, 12, 22] to perform eye state
classification (incl. fixation and blink) and fixated object detection. This
pilot dataset has been annotated frame-by-frame to mark onsets of human
attention lapses and in-operation states.

We also investigated the effect of applying existing fatigue and drowsiness
detection methods [2, 7, 19] on the pilot dataset. These methods have been
widely used in driving safety domain. Unfortunately, they not only fall short
in robustness for relying on single modality, but are hardly usable in real-
world industrial settings where attention lapses take manifold forms. The
results reveal a remarkable gap in accurate human attention lapse detection to
allow safe and trustworthy HRI for industry.

In this work, our contribution is threefold: (1) We collected two pilot data
sequences of multimodal human attention data in realistic industrial
environments for benchmarking of human attention lapses detection; (2) We
developed a pipeline for labelling multimodal sensory data, namely eye
tracking, surveillance camera, and auditory input; (3) We experimented with
existing fatigue and drowsiness prediction methods and identified a
significant gap of knowledge in human attention lapse detection.

The rest of this paper is organized as follows. Section 2 reviews existing
literature around attention lapse detection datasets, methods, and
applications. Our proposed multimodal annotation pipeline and benchmarking
datasets are expanded in details in Section 3. Evaluation outcomes with
existing methods are reported in Section 4. Section 5 summarizes future work.

##  II BACKGROUND

###  II-A Human Attention Lapse Study

Driving fatigue and drowsiness detection have been actively studied for its
oriented application in driver safety domain. PERCLOS [7] is one of the most
commonly used metrics for driving fatigue evaluation. An eye state tracking
approach to PERCLOS is proposed in [8] to estimate eyelid closure ratio with
respect to fatigue level. Although eye closure is agreed to be directly
correlated to fatigue, human attention lapse can be quite polymorphic.
Driver’s eye looking away from the normal direction could be a sign of
distraction or fatigue. Head pose [1, 22] which is often associated with eye
gaze may also indicate attention shifting.

Human attention lapses are of far more complex causes than fatigue and
drowsiness. Indeed, a person may be mind wandering whilst having eye fixation
at the right direction and objects. The fixation state of eyes may not
directly indicate focused or attention lapse but the states of eye are closely
correlated to human attention in general [23]. According to human cognition
and recognition studies [10], the following gaze patterns are strong
indicators of attention lapse: long fixation on a single point, looking away,
slow or irregular saccades, eye closure, and a high rate of blinking.
Attention lapse is related to environmental distractors (auditory and visual),
cumulative fatigue doing repetitive work, internal cognitive state such as
motivation. A single-modality approach to determine human attention detection
often falls short in accuracy and reliability [28].

Our hypothesis is that attention lapse is multimodal, that is, human’s
internal cognitive state, environmental stimuli, together with eye state,
facial expression, and posture collectively formulate trustworthy attention
lapse detection. Existing driving fatigue detectors cannot be generalized for
diverse applications [28, 15] in the sense that a driver sits in a confined
position with a relatively simple but effective metric of “looking ahead”. In
this paper, we look into an open and more challenging scenario of assembly
tasks in manufacturing.

###  II-B Existing Datasets and Methods

In the domain of driver fatigue research, detecting drowsiness and distraction
using multimodal behavioural features [20] allows applicable solutions to
enhance driver safety. Transferring this idea to HRI could mean a big step
forward for workers’ safety in automated production lines [4]. According to
[15], it is found the combination of eye feature (gaze, blinking, and fixation
states) and head pose achieve reliable estimations. Methods based on single-
modality turn out susceptible to occlusions and hard to generalize well. In
[2], the authors proposed handcrafted thresholds of $30^{\circ}$ nodding and
200-frame (about 6.7s) no change in gaze direction to detect drawsiness, which
can never produce satisfactory reliability. Yawning makes an effective
indicator of fatigues but becomes less useful in distraction detection [28].

Static human attention tracking within a fixed plain (e.g., looking at a flat
screen) has been well-studied, such as SNAG dataset [25] and human attention
dataset in image captioning [9]. Recently more focuses have been drawn on
studying human attention in action such as assembling a camping tent [23] and
driver monitoring [8].

Modern eye trackers achieve fairly high accuracy, precision, and effective
frequency [11] even wearers performing substantial movements. Nevertheless,
eye tracking in unrestrained settings, such as lost track of pupil and long
eye closure, will inevitably affect the data quality in accuracy and precision
[18]. Furthermore, a robust eye state tracking entails high-fidelity visual
monitoring from third-person views [6]. This deteriorates credibility of
predicting attention by eye tracking only.

It has been widely acknowledged environmental noises, such as those in
industrial workplace, have a profound impact on human cognition [24]. Auditory
stimuli are also common external stimuli that draw human attention and arouse
distraction [16]. We are thus motivated to collect a multimodal human
attention tracking dataset tailored for cooperation between human agents and
robot manipulators in industrial settings. As aforementioned, human’s eye
states, head pose, body posture, and sound stimuli shall all be synchronized
and recorded. The following sections expands our data collection campaign,
annotation pipeline, and evaluation results.

##  III METHOD

Figure 2: Visualization of the data collection environment and annotations.
(A)(D) show the assembly worktop defined by QR codes and its 2D projection
with gaze in post-hoc processing. Green dots are gaze projections; yellow
circle marks the fixation position. (B) shows the Pupil Core eye tracker. Head
pose tracking results are shown in (C).

We collected a pilot dataset of a human worker doing a dummy phone assembly
task with the aid of a pre-programmed robot manipulator. Multi-modality
sensors, namely an RGB-D camera with microphone (Azure Kinect DK) and an eye
tracker with egocentric world camera (Pupil Core), are deployed with multiple
artificial distractors injected during the experiment process. An automated
annotation pipeline is proposed to streamline the labelling of multiple
primitives for human attention lapse research. We also utilized pre-trained
object detectors [21] for fixated object recognition and pre-trained head pose
detector [22] for head pose calibration.

###  III-A Pilot Dataset

Figure 3: Annotated in-operation and distracted states in phone assembly sequence 002 compared with classic estimation methods. Blue bars represent onsets of the worker’s hands-on operation which implies greater hazards upon robotic intervention. Red bars represent onsets when the worker is distracted. Magenta bars indicate eye fixations within the predefined surface (i.e., worktop). Grey bars mark head movements with a high angular velocity. These events are annotated against temporal UTC timestamps of millisecond (msec) precision. Table I: Annotations in the pilot dataset Label | Type | Units
---|---|---
Eye state | fixation/blink/saccade | Classes and msec
Gaze | 2D position | Relative coordinates
Fixation | 2D position | Relative coordinates
Fixation object | Detected object class | Classes
Head pose | 6 degree-of-freedom pose | Pose
Fixation in AOI | 2D position and whether fixation is in AOI | Relative coordinates and True/False
Distractor |  noise/motion/phone and onset timestamps | Classes and msec
Distracted state | Start and end timestamps | msec
In-operation state | Start and end timestamps | msec

The data collection was conducted in Aalborg University 5G Smart Production
Lab [17]. It provides a realistic manufacturing environment to present real-
world industrial scenarios. A Franka robot has been pre-programmed to harvest
five components of a dummy phone, shown in Fig. 2(D), and hand them over by
releasing the components ahead of human operator’s right hand position. The
release takes place at slightly varying locations which requires human’s
attention to catch the components. Note these five dummy components (one front
cover, one back cover, one printed circuit board, and two fuses) are made of
plastic and would cause no harm if caught unsuccessfully. A software-based
stop command and an emergency button are placed next to the human participant
for safety. A human experimenter was asked to perform phone assembly tasks
while multimodal data of gaze, vision, speech, and environmental stimuli from
multiple angles were collected. During the task, the dummy phone parts placed
on a flat worktop will be picked up by the Franka robot and the human
participant was expected to put these parts together, as is shown in Fig.
2(A).

We experimented with a Pupil Core eye tracker as shown in Fig. 2(B). Pupil
Core offers binocular eye tracking at 200Hz (highest among mass marketed
wearable and mobile products) and a scene camera to shoot world view of
$155^{\circ}/85^{\circ}$ horizontal/vertical field of view with configurable
resolution and frame rate [12]. The eye cameras take infrared illuminated
images and detects the pupil area with a robust and efficient algorithm and
multiple noise filters (an average gaze detection accuracy of $0.6^{\circ}$ of
visual angle and a processing latency of 45ms). Pupil Core stands out for its
light-weight design, minimized visual obstruction, and ability to accommodate
varying facial geometries in comparison to SMI ETG2 60 or Tobii G-series [11].
More importantly, the open source software of Pupil Core allows customization
in data collection, processing, and labelling processes. Data streaming, blink
detection, and head pose tracking modules can be integrated as plugins at
ease. Eye tracking endows the ability of detecting introverted human attention
in a direct and non-obstructive means [13].

A Microsoft Azure Kinect DK with microphone array was used for a third-person
perspective recording. This kit contains an RGB-D camera with infrared
sensing, 7-microphone array, and integrated body pose tracking backends. We
used a Lenovo laptop with an Intel i7 processor and Ubuntu 18.04 operating
system installed for data collection. All sensors are synchronized by the host
laptop’s local clock.

QR codes are commonly used for head pose estimation in eye tracking practices.
Following the method proposed in [27], we deployed multiple Apriltags [13] in
the environment and utilized the post-hoc pose tracking algorithm of the Pupil
Core software for head pose estimation. We deployed eight Apriltags from the
tag36h11 family next to the assembly worktop as shown in Fig. 2(C,D). Three of
them are attached to the flat surface of the worktop to allow surface
abstraction and gaze tracking in Area of Interest (AOI).

###  III-B Annotation Pipiline

To achieve multimodal attention detection, we developed a data annotation
pipeline for efficient processing of the multimodal data. First, we recognized
a list of significant primitives in human attention study as detailed in Table
I. Specifically, the eye state, gaze position, and fixation position can be
extracted from Pupil Core’s software system outputs but require careful
calibration as errors are common in default detectors. The fixation object is
automatically annotated by a pre-trained object detector [21] with manual
corrections. Head pose and surface tracking are derived with the aid of eight
Apriltags deployed next to the assembly platform. Of our greatest interest,
the distracted states and in-operation states are manually labelled frame-to-
frame to identify subtle attention lapses. An example of the annotated
sequence 002 is shown in Fig. 3.

Note ensuring ad-hoc occurrence of real-world distractors in an assembly task
is critical for attention lapse study. We curated two pilot data sequences of
a human participant assembling a dummy phone in a realistic production
laboratory with manufacturing background noises, moving MiR robots, and
workers around. According to K. Holmqvist and R. Andersson [10], the following
distractors in industrial settings are identified:

  * •

Noise (background/sudden)

  * •

Co-workers (talk to “me”/talk to each other)

  * •

Malfunction (user mistake/equipment)

  * •

Fatigue (drowsiness/mind wandering)

  * •

Visual stimuli (environment/moving objects)

  * •

Multitasking

  * •

Motivation (task difficulty/repetitiveness/reward)

During the pilot data collection, we introduced three of the above distractors
due to constraints of the environment and safety concerns: clapping hands
(sudden noise), walking co-workers at the scene (visual distraction), and
looking at a personal smart phone (mind wandering). In the first clip
(Sequence 002), we asked the experimenter to be open-minded to attention
lapses and look along distractors. In the second clip (Sequence 002), the
experimenter was asked to stay as concentrated as possible in which the only
distractor is mind wandering. The onset of these events have been post-hoc
annotated by checking multiple camera views and auditory recordings as shown
in Fig. 4.

A FasterRCNN-based object detector [21] is leveraged to perform post-hoc
fixation object recognition. We applied the object detector on every frame and
used fixation point as a filter to only annotate object of attentive interest.
If the fixation is more than 15-pixel away from an object’s bound box the
detected object will be omitted as the fixation lands far apart from the
object. Errors are inevitable using such automated annotations. We further
validated and corrected wrong labels frame-by-frame.

##  IV EVALUATION

Figure 4: Visualization of audio distractors (1st row), eye tracker’s world
view (2nd row), surveillance RGB scene (3rd row), and depth scene (4th row).
The second frame (column) shows the worker being distracted by hand claps to
her left. The fourth frame (column) illustrates distraction of talking to co-
workers.

We experimented with existing single-modality methods on the pilot dataset.
Classic fatigue and drowsiness detection modalities include ratio of eye
closure state (PERCLOS) [7], blink frequency [19], gaze fixation state [10],
and head pose tracking [2]. We examined their performance in determining
attention lapse upon phone assembly sequences 002 and 003.

Table II: Detection Results on Phone Assembly Sequence 002. Method | Precision | Recall | F1 Score
---|---|---|---
Eye closure period (PERCLOS) | $35.71\%$ | $37.42\%$ | $0.37$
Eye blink frequency | $19.55\%$ | $13.16\%$ | $0.16$
Fixation within AOI | $8.70\%$ | $19.31\%$ | $0.12$
Head movement velocity | $13.11\%$ | $9.49\%$ | $0.11$
Table III: Detection Results on Phone Assembly Sequence 003. Method | Precision | Recall | F1 Score
---|---|---|---
Eye closure period (PERCLOS) | $0$ | $0$ | N/A
Eye blink frequency | $4.24\%$ | $100\%$ | $0.08$
Fixation within AOI | $3.15\%$ | $52.22\%$ | $0.06$
Head movement velocity | $0.85\%$ | $24.39\%$ | $0.02$

PERCLOS refers to the percentage of duration of closed-eye state within a
given time interval (30 seconds or 1 minute). For a fair comparison, we use EM
criteria, which represents 50% of eye closure rate, upon a 30s sliding window
to metric attention lapse on the phone assembly data sequences. According to
[19], eye blink frequency drops below 10 per minute when sleepy. We adopted
this threshold for blink frequency based drowsiness estimation in a 30s
sliding window.

Workers are expected to pay attention to an AOI, such as a worktop, when
focusing on specific tasks. This has been widely accepted as an important
attribute in attention detection. We defined a worktop area using Pupil Core’s
software as shown in Fig. 2(A). The entry and exits from this AOI can be
calculated as shown in Fig. 3 “On-surface”. I. Choi and Y. Kim [2] contended
nodding over $30^{\circ}$ in a short time implies drowsiness. We extend this
to head rotation around any axis for distraction detection. We specified the
short time equals $2$ seconds. Given densely deployed QR codes, pose-hoc head
poses can be reconstructed as seen in Fig. 2(C). Thus, we obtained distraction
states as shown in Fig. 3 “Head Move”.

Overall, poor precision, recall, and F1 scores are seen from pilot data
sequences as shown in Tables II and III. PERCLOS turns out the most reliable
attention lapse detector when open-minded to distraction. Yet its precision
and recall are far from satisfactory. Other single-modality methods
demonstrate extremely low coherence with true distraction states. In Sequence
003 where experimenter was mentally prepared, all fatigue and drowsiness
methods fail. These back up our judgement that attention lapse is multimodal
and complex, suggesting simple paradigms of monitoring eye blinks, fixation,
or head pose cannot be trusted. Additionally, single or few sensors for
detection may suffer in accuracy when occlusion or data discrepancy occur. For
instance, we found some long-lasting blinks, i.e., over 5s, in Pupil Core’s
algorithm for blink detection. This may be due to losing tracks of pupils. We
conclude from our pilot study that a multimodal and multi-sensory scheme for
worker attention lapse detection becomes imperative in this research area.

##  V FUTURE WORK

In this work, we addressed data primitives of a human attention lapse study
and created a pilot dataset to evaluate existing detection methods. The pilot
dataset collected falls short in size to support data-driven algorithms for
attention lapse detection, such as SVM and deep neural networks. We plan to
launch a large-scale data collection campaign with dozens of human
participants accomplishing various assembly tasks. Current experimental setup
supports one set of sensors on eye tracker and the other set in front of the
workspace. We will expand it with more sets of sensors from multiple angles in
the complete dataset.

We notice the frame rates of multimodal sensors differ. Eye cameras run at
200Hz, whereas, world camera and Azure Kinect DK run at 30Hz. Furthermore,
there is an offset between sound track and RGB-D video. We will elaborate the
data collection pipeline to achieve enhanced synchronisation. We also plan to
integrate threads of automatic body posture tracking and hand gesture
recognition into the annotation pipeline.

## ACKNOWLEDGMENT

This work was funded by 2022/23 Aston Pump Priming Scheme and AAU Bridging
Project “A Multimodal Attention Tracking In Human-robot Collaboration For
Manufacturing Tasks.” We thank the Aalborg 5G Smart Production Lab for
supporting our data collection campaign.

## References

  * [1] Tadas Baltrusaitis, Peter Robinson, and Louis-Philippe Morency.  Openface: An open source facial behavior analysis toolkit.  In WACV, pages 1–10. IEEE Computer Society, 2016.
  * [2] In-Ho Choi and Yong-Guk Kim.  Head pose and gaze direction tracking for detecting a drowsy driver.  In 2014 International Conference on Big Data and Smart Computing (BIGCOMP), pages 241–244, 2014.
  * [3] Zhuangzhuang Dai, Yuhang He, Vu Tran, Niki Trigoni, and Andrew Markham.  Deepaoanet: Learning angle of arrival from software defined radios with deep neural networks.  IEEE Access, 10:3164–3176, 2022.
  * [4] Zhuangzhuang Dai and Frank J.W. Podd.  A power-efficient ble augmented gnss approach to site-specific navigation.  In 2020 IEEE/ION Position, Location and Navigation Symposium (PLANS), pages 1305–1310, 2020.
  * [5] Zhuangzhuang Dai, Muhamad Risqi U. Saputra, Chris Xiaoxuan Lu, Vu Tran, L. N. S. Wijayasingha, M. Arif Rahman, John A. Stankovic, Andrew Markham, and Niki Trigoni.  Deep odometry systems on edge with ekf-lora backend for real-time indoor positioning.  In 2022 Workshop on Cyber Physical Systems for Emergency Response (CPS-ER), pages 1–6, 2022.
  * [6] Zhuangzhuang Dai, Vu Tran, Andrew Markham, Niki Trigoni, M. Arif Rahman, L. N. S. Wijayasingha, John Stankovic, and Chen Li.  Egocap and egoformer: First-person image captioning with context fusion.  In SSRN. Available at SSRN 4259901, 2022.
  * [7] David F. Dinges and Randolph C. Grace.  Perclos: A valid psychophysiological measure of alertness as assessed by psychomotor vigilance.  1998\.
  * [8] Xiang-Yu Gao, Yu-Fei Zhang, Wei-Long Zheng, and Bao-Liang Lu.  Evaluating driving fatigue detection algorithms using eye tracking glasses.  In 2015 7th International IEEE/EMBS Conference on Neural Engineering (NER), pages 767–770, 2015.
  * [9] Sen He, Hamed Rezazadegan Tavakoli, Ali Borji, and Nicolas Pugeault.  Human attention in image captioning: Dataset and analysis.  2019 IEEE/CVF International Conference on Computer Vision (ICCV), pages 8528–8537, 2019.
  * [10] Kenneth Holmqvist and Richard Andersson.  Eye-tracking: A comprehensive guide to methods, paradigms and measures.  11 2017.
  * [11] Ignace T. C. Hooge, Diederick C. Niehorster, Roy S. Hessels, Jeroen S. Benjamins, and Marcus Nyström.  How robust are wearable eye trackers to slow and fast head and body movements?  Behavior Research Methods, 2022.
  * [12] Moritz Kassner, William Patera, and Andreas Bulling.  Pupil: An open source platform for pervasive eye tracking and mobile gaze-based interaction.  In Proceedings of the 2014 ACM International Joint Conference on Pervasive and Ubiquitous Computing: Adjunct Publication, UbiComp ’14 Adjunct, page 1151–1160, New York, NY, USA, 2014. Association for Computing Machinery.
  * [13] Maximilian Krogius, Acshi Haggenmiller, and Edwin Olson.  Flexible layouts for fiducial tags.  In Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), October 2019.
  * [14] Chen Li, Jinha Park, Hahyeon Kim, and Dimitrios Chrysostomou.  How can i help you? an intelligent virtual assistant for industrial robots.  In Companion of the 2021 ACM/IEEE International Conference on Human-Robot Interaction, HRI ’21 Companion, page 220–224, New York, NY, USA, 2021. Association for Computing Machinery.
  * [15] Fan Liu, Delong Chen, Jun Zhou, and Feng Xu.  A review of driver fatigue detection and its advances on the use of rgb-d camera and deep learning.  Engineering Applications of Artificial Intelligence, 2022.
  * [16] Gholamreza Moradi, Leila Omidi, Shahram Vosoughi, Hossein Ebrahimi, Ahad Alizadeh, and Iraj Alimohammadi.  Effects of noise on selective attention: The role of introversion and extraversion.  Applied Acoustics, 146:213–217, 2019.
  * [17] Marco Nardello, Ole Madsen, and Charles Møller.  The smart production laboratory: A learning factory for industry 4.0 concepts.  09 2017.
  * [18] D. C. Niehorster, T. H. W. Cornelissen, K. Holmqvist, I. T. C. Hooge, and R. S. Hessels.  What to expect from your remote eye-tracker when participants are unrestrained.  Behavior research methods, page 213–227, 2018.
  * [19] Novie Theresia Br. Pasaribu, Agus Prijono, Ratnadewi Ratnadewi, Roy Pramono Adhie, and Joseph Felix.  Drowsiness detection according to the number of blinking eyes specified from eye aspect ratio value modification.  In Proceedings of the 1st International Conference on Life, Innovation, Change and Knowledge (ICLICK 2018), pages 171–174. Atlantis Press, 2019/07.
  * [20] Leo Pauly and Deepa Sankar.  Detection of drowsiness based on hog features and svm classifiers.  In 2015 IEEE International Conference on Research in Computational Intelligence and Communication Networks (ICRCICN), pages 181–186, 2015.
  * [21] Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun.  Faster r-cnn: Towards real-time object detection with region proposal networks.  In C. Cortes, N. Lawrence, D. Lee, M. Sugiyama, and R. Garnett, editors, Advances in Neural Information Processing Systems, volume 28. Curran Associates, Inc., 2015.
  * [22] Nataniel Ruiz, Eunji Chong, and James M. Rehg.  Fine-grained head pose estimation without keypoints.  In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, June 2018.
  * [23] Brian Sullivan, Casimir J. H. Ludwig, Dima Damen, Walterio Mayol-Cuevas, and Iain D. Gilchrist.  Look-ahead fixations during visuomotor behavior: Evidence from assembling a camping tent.  Journal of Vision, 21, 2021.
  * [24] Rhiannon Thompson, Rachel B. Smith, Yasmin Bou Karim, Chen Shen, Kayleigh Drummond, Chloe Teng, and Mireille B. Toledano.  Noise pollution and human cognition: An updated systematic review and meta-analysis of recent evidence.  Environment International, 158:106905, 2022.
  * [25] Preethi Vaidyanathan, Emily T. Prud’hommeaux, Jeff B. Pelz, and Cecilia O. Alm.  Snag: Spoken narratives and gaze dataset.  In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), page 132–137. Association for Computational Linguistics, 2018.
  * [26] Leonard Elia van Dyck, Roland Kwitt, Sebastian Jochen Denzler, and Walter Roland Gruber.  Comparing object recognition in humans and deep convolutional neural networks—an eye tracking study.  Frontiers in Neuroscience, 15, 2021.
  * [27] Yuheng Yan, Yiqiu Liang, Zihan Zhou, Bin Jiang, and Jian Xiao.  Fastqr: Fast pose estimation of objects based on multiple qr codes and monocular vision in mobile embedded devices.  Wireless Communications and Mobile Computing, 2021:1–9, 09 2021\.
  * [28] Cong Yang, Zhenyu Yang, Weiyu Li, and John See.  Fatigueview: A multi-camera video dataset for vision-based drowsiness detection.  IEEE Transactions on Intelligent Transportation Systems, 24(1):233–246, January 2023.  Funding Information: This work was supported in part by the Natural Science Foundation of the Jiangsu Higher Education Institutions of China under Grant 22KJB520008 and in part by the Halo Special Research Fund from Horizon Robotics under Grant PDT2020004. Publisher Copyright: © 2000-2011 IEEE.


# Seasonal and regional characterization of horizontal stirring in the global
ocean

Ismael Hernández-Carrasco,1 Cristóbal López,1∗
Emilio Hernández-García1 and Antonio Turiel,2

1Instituto de Física Interdisciplinar y Sistemas Complejos (CSIC-UIB)
07122 Palma de Mallorca, Spain
2Institut de Ciències del Mar, CSIC
Passeig Marítim de la Barceloneta 37-49, 08003 Barcelona, Spain

∗To whom correspondence should be addressed; E-mail<EMAIL_ADDRESS>
###### Abstract

Recent work on Lagrangian descriptors has shown that Lyapunov Exponents can be
applied to observed or simulated data to characterize the horizontal stirring
and transport properties of the oceanic flow. However, a more detailed
analysis of regional dependence and seasonal variability was still lacking. In
this paper, we analyze the near-surface velocity field obtained from the Ocean
general circulation model For the Earth Simulator (OFES) using Finite-Size
Lyapunov Exponents (FSLE). We have characterized regional and seasonal
variability. Our results show that horizontal stirring, as measured by FSLEs,
is seasonally-varying, with maximum values in Summer time. FSLEs also strongly
vary depending on the region: we have first characterized the stirring
properties of Northern and Southern Hemispheres, then the main oceanic basins
and currents. We have finally studied the relation between averages of FSLE
and some Eulerian descriptors such as Eddy Kinetic Energy (EKE) and vorticity
($\omega$) over the different regions.

##  1 Introduction

A detailed knowledge of the transport, dispersion, stirring and mixing
mechanisms of water masses across the global ocean is of crucial interest to
fully understand, for example, heat and tracer budgets, or the role of oceans
in climate regulation. There has been a recent strong activity in the study of
these processes from a Lagrangian perspective. Some works have addressed the
global variability of them using finite-time Lyapunov exponents (FTLEs)
computed from currents derived from satellite altimetry [4, 42]. These studies
quantify stirring intensity, and identify mesoscale eddies and other
Lagrangian Coherent Structures (LCSs). Furthermore, previous works [43]
pointed out relationships between Lagrangian and Eulerian quantifiers of
stirring/mixing activity (FTLEs and Eddy Kinetic Energy (EKE) or mean strain
rate).

Having in mind the implications for the distribution of biogeochemical
tracers, our goal is to extend the previous works to provide detailed seasonal
analysis and a comparative study between different ocean regions and different
scales: Earth’s hemispheres, ocean basins, and boundary currents. To this end
we use finite-size Lyapunov exponents (FSLEs). These quantities are related to
FTLEs since they also compute stretching and contraction time scales for
transport, but they depend on explicit spatial scales which are simple to
specify and to interpret in oceanographic contexts [10, 11, 20, 40]. In
particular we will focus on the impact on transport of mesoscale processes,
for which characteristic spatial scales as a function of latitude are well
known. We are also interested in checking the existence of relationships
between Lagrangian measures of horizontal stirring intensity, as given by
averages of finite-size Lyapunov exponents (FSLE), and other dynamic, Eulerian
quantities, such as EKE or vorticity. Such a functional relation does not need
to hold in general, but may be present when there is a connection between the
mechanisms giving rise to mesoscale turbulence (probably, baroclinic
instability) and horizontal stirring.

The paper is organized as follows. In Section 2 we describe the data and tools
used in this study. In section 3 we first present the geographical and
seasonal characterization of the horizontal stirring, and then we investigate
the relation of FSLE with EKE and vorticity. Finally, in the Conclusions we
present a summary and concluding remarks.

##  2 Data and Methods

Our dataset consists of an output from the Ocean general circulation model For
the Earth Simulator (OFES) [26, 25]. This is a near-global ocean model that
has been spun up for 50 years under climatological forcing taken from monthly
mean NCEP (United States National Centers for Environmental Prediction)
atmospheric data. After that period the OFES is forced by the daily mean NCEP
reanalysis for 48 years from 1950 to 1998. See [26] for additional details on
the forcing. The output of the model corresponds to daily data for the last 8
years. Horizontal angular resolution is the same in both the zonal, $\phi$,
and meridional, $\theta$, directions, with values of
$\Delta\theta=\Delta\phi=1/10^{\circ}$. The output has been interpolated to 54
vertical z-layers and has a temporal resolution of one day. The velocity
fields that we have used in this work correspond to the first two years, 1990
and 1991, of the output. Vertical displacements are unimportant during the
time scales we consider here so that, despite horizontal layers are not true
isopycnals, most fluid elements remain in their initial horizontal layer
during the time of our Lagrangian computation. Thus we use in our analysis
horizontal velocities in single horizontal layers. We refer to recent works
[28, 6] for Lyapunov analyses considering vertical displacements. Unless
explicitly stated, our calculations are for the second output layer, at $7.56$
m depth, which is representative of the surface motion but limits the effect
of direct wind drag (we have also studied the layer at $97$ m depth; results
on this layer are briefly shown in Fig. 3). See [26] and [25] for a thorough
evaluation of the model performance.

Among Lagrangian techniques used to quantify ocean transport and mixing, local
Lyapunov methods are being widely used. The idea in them is to look at the
dispersion of a pair of particles as they are transported by the flow. To
calculate FTLEs, pairs of particles infinitesimally close are released and
their separation after a finite time is accounted; for FSLEs [2] two finite
distances are fixed, and the time taken by pairs of particles to separate from
the smallest to the largest is computed. Both methods thus measure how water
parcels are stretched by the flow, and they also quantify pair dispersion. The
methods can also be tailored to reveal two complementary pieces of
information. On the one hand they provide time-scales for dispersion and
stirring process [1, 2, 9, 23, 10, 19, 30]. On the other, they are useful to
identify Lagrangian Coherent Structures (LCSs), persistent structures that
organize the fluid transport [15, 13, 8, 21, 22, 24, 14, 37, 4, 11, 40, 29].
This second capability arises because the largest Lyapunov values tend to
concentrate in space along characteristic lines which could often be
identified with the manifolds (stable and unstable) of hyperbolic trajectories
[15, 13, 14, 16, 37]. Since these manifolds are material lines that can not be
crossed by fluid elements, they strongly constrain and determine fluid motion,
acting then as LCSs that organize ocean transport on the horizontal. Thus,
eddies, fronts, avenues and barriers to transport, etc. can be conveniently
located by computing spatial Lyapunov fields. We note however that more
accurate characterization of LCSs can be done beyond Lyapunov methods [16],
that high Lyapunov values can correspond also to non-hyperbolic structures
with high shear [12], and that an important class of LCSs is associated to
small, and not to large values of the Lyapunov exponents [35, 5].

In the present work, however, we are more interested in obtaining the first
type of information, i.e. in extracting characteristic dispersion time-scales,
quantifying the intensity of stirring, for the different ocean regions and
seasons. In particular we want to focus on the transport process associated to
eddies and other mesoscale structures. Previous Lagrangian analyses of the
global ocean [4, 42] used FTLE to quantify such horizontal stirring. This
quantity depends on the integration time during which the pair of particles is
followed. FTLEs generally decrease as this integration time increases,
approaching the asymptotic value of the infinite-time Lyapunov exponent [42].
We find difficult to specify finite values of this integration time for which
easy-to-interpret results would be obtained across the different ocean
regions. But for the mesoscale processes on which we want to focus,
characteristic spatial scales are related to the Rossby Deformation Radius
(RDR), with easily defined values and latitudinal dependence (see below).
Thus, we use in this paper FSLEs as a convenient way to identify
characteristics of stirring by mesoscale processes. FSLE are also convenient
in finite ocean basins, where relevant spatial scales are also clearly imposed
[1, 7, 23]. As a quantifier of horizontal stirring, measuring the stretching
of water parcels, FSLEs give also information on the intensity of horizontal
mixing between water masses, although a complete correspondence between
stirring and mixing requires the consideration of diffusivity and of the
stretching directions [12].

More in detail, at a given point the FSLE (denoted by $\lambda$ in the
following) is obtained by computing the minimal time $\tau$ at which two fluid
particles, one centered on the point of study and the other initially
separated by a distance $\delta_{0}$, reach a final separation distance
$\delta_{f}$. At position x and time $t$, the FSLE is given by:
$\lambda(\textbf{x},t,\delta_{0},\delta_{f})=\tau^{-1}\ln(\delta_{f}/\delta_{0})$.
To estimate the minimal time $\tau$ we would need to integrate the
trajectories of all the points around the analyzed one and select the
trajectory which diverges the first. We can obtain a very good approximation
of $\tau$ by just considering the four trajectories defined by the closest
neighbors of the point in the regular grid of initial conditions at which we
have computed the FSLE; the spacing of this grid is taken equal to
$\delta_{0}$. The equations of motion that describe the horizontal evolution
of particle trajectories are

$\displaystyle\frac{d\phi}{dt}$ $\displaystyle=$
$\displaystyle\frac{u(\phi,\theta,t)}{R\cos{\theta}},$ (1)
$\displaystyle\frac{d\theta}{dt}$ $\displaystyle=$
$\displaystyle\frac{v(\phi,\theta,t)}{R},$ (2)

where $u$ and $v$ stand for the zonal and meridional components of the surface
velocity field coming from the OFES simulations; $R$ is the radius of the
Earth ($6400$ $km$), $\phi$ is longitude and $\theta$ latitude. Numerically we
proceed by integrating Eqs. (1) and (2) using a standard, fourth-order Runge-
Kutta scheme, with an integration time step $dt=6$ hours. Since information is
provided just in a discrete space-time grid, spatiotemporal interpolation of
the velocity data is required, that is performed by bilinear interpolation.
Initial conditions for which the prescribed final separation $\delta_{f}$ has
not been reached after integrating all the available times in the data set are
assigned a value $\lambda=0$. A possible way to introduce small-scale features
that are not resolved by our simulated velocity fields is by inclusion of
noise terms in the equations of motion (2). We have recently shown [20] that
the main mesoscale features are maintained when this eddy-diffusivity is taken
into account, though sub-mesoscale structures may change considerably. For
global scales we expect the effects of noise to be even more negligible.

The field of FSLEs thus depends on the choice of two length scales: the
initial separation $\delta_{0}$ (which coincides with the lattice spacing of
the FSLE grid and is fixed in our computations to the model resolution,
$\delta_{0}$=$1/10^{\circ}$) and the final separation $\delta_{f}$. As in
previous works in middle latitudes [10, 11, 20] we will focus on transport
processes arising from the mesoscale structures. In these studies $\delta_{f}$
was taken about $110km$, which is of the order of, but larger than, the
mesoscale size in middle latitudes. Note that $\delta_{f}$ should be a
decreasing function of the latitude, since mesoscale structures decrease in
size with Rossby Deformation Radius (RDR). We need not to exactly match RDR
but to guarantee that our choice of $\delta_{f}$ is similar but larger than
mesoscale lengths, and also that it is a smooth function to avoid inducing
artifacts. We have then chosen $\delta_{f}$ as $\delta_{f}=1.3|\cos\theta|$
degrees; other reasonable choices lead to similar results to those presented
here.

We compute the FSLEs by backwards time integration. In this way we quantify
the fluid deformation by past stirring. When computing LCSs this leads to
structures easier to interpret since they can be associated with the actual
shape of tracer filaments [21, 11]. However, given that forward and backward
exponents in incompressible flows are related by temporal shifts and spatial
distortions [17], and that we are interested in temporal and spatial averages
over relatively large scales, we do not expect significant differences when
using forward exponents to calculate the stirring quantifiers presented below.
This was explicitly checked in a similar framework in [10].

Lagrangian measurements have been shown to correlate well with several
Eulerian quantities at several scales [43, 42]. In particular it is pertinent
to correlate stirring with Eddy Kinetic Energy (EKE) since it is expected that
more energetic turbulent areas would also present stronger horizontal
stirring, mainly due to the spawning of eddies (see however [34, 33]). Given
an integration period $T$ long enough (for instance $T$= one year), the EKE
(per unit of mass) is given by: $EKE=\frac{1}{2}\left\langle u^{\prime
2}+v^{\prime 2}\right\rangle$, where $u^{\prime}$ and $v^{\prime}$ are the
instant deviations in zonal and meridional velocities from the average over
the period $T$, and the brackets denote average over that period. Another
Eulerian measurement used in this work is the surface relative vorticity,
given by $\omega=\frac{\partial v}{\partial x}-\frac{\partial u}{\partial y}$,
with positive (vs negative) $\omega$ associated to cyclonic (vs anticyclonic)
motion in the Northern Hemisphere (opposite signs in the Southern Hemisphere).
An additional Eulerian candidate to look for Lagrangian correspondences is the
local strain rate, but it has been shown [43, 42] to scale linearly with
$EKE^{1/2}$ and thus it will not be explicitly considered here.

Conditioned averages of $\lambda$ as a function of another variable $y$ (let
$y$ be EKE1/2 or $\omega$) introduced in Subsection 3.4 are obtained by
discretizing the allowed values of $y$ by binning; 100 bins were taken, each
one defining a range of values $(y_{n},y_{n+1})$ and represented by the
average value $\hat{y}_{n}=\frac{y_{n}+y_{n+1}}{2}$. So, for each discretized
value of $\hat{y}_{n}$ the average of all the values of $\lambda$ which occur
coupled with a value in $(y_{n},y_{n+1})$ is computed. The result is an
estimate of the conditioned average $\tilde{\lambda}(y)$ (which is a function
of $y$) at the points $\hat{y}_{n}$.

##  3 Results

###  3.1 Global horizontal stirring from FSLE

In Fig. 1 we present a map of FSLEs at a given time. Typical values are in the
order of $0.1-0.6$ $days^{-1}$, that correspond well to the horizontal
stirring times expected at the mesoscale, in the range of days/weeks. Spatial
structures, from filaments and mesoscale vortices to larger ones, are clearly
identified; see a representative zoom of the South Atlantic Ocean (Bottom of
Fig. 1), where the typical filamental structures originated by the horizontal
motions are evident.

Instantaneous maps of FSLEs have a significant signature of short-lived fast
processes and are adequate to extract LCSs, but we are more interested in
slower processes at larger scales. We have hence taken time averages of FSLEs
over different periods, in order to select the low-frequency, large-scale
signal. In this way we can easily characterize regions in the global ocean
with different horizontal stirring activity; areas with larger values of
averaged FSLEs are identified as zones with more persistent horizontal
stirring [10], as shown in Fig. 2a. As expected, we can observe that high
stirring values correspond to Western Boundary Currents (WBCs) and to the
Antarctic Circumpolar Current, while the rest of the ocean and the Eastern
Boundary Currents (EBCs) display significantly lower values.

###  3.2 Geographical characterization of horizontal stirring

A convenient quantity used to characterize stirring in a prescribed
geographical area $A$ was introduced by [10], which is simply the spatial
average of the FSLEs over that area at a given time, denoted by $<\lambda({\bf
x},t)>_{A}$. Time series of this quantity for the whole ocean and the Northern
and Southern hemispheres are shown in Fig. 3a. It is worth noting that the
stirring intensity is typically larger in the Northern Hemisphere than in the
Southern one.

Further information can be obtained by analyzing the FSLE Probability
Distribution Functions (PDFs). In Fig. 3b we present the PDFs for both
hemispheres and the whole ocean; the required histograms are built using
$\lambda$ values computed once every week during one year (52 snapshots) at
each point of the spatial FSLE grid in the area of interest. Each one of these
PDFs is broad and asymmetric, with a small mode $\lambda_{m}$ (i.e., the value
of $\lambda$ at which the probability attains its maximum) and a heavy tail.
Similarly to what was discussed by [43] and [42] for the FTLE case, these PDFs
are well described by Weibull distributions with appropriate values for the
defining parameters. We note that an explicit relationship between FTLE and
FSLE distributions was derived by [41], but we have not checked if our flow is
in the regime considered in that reference. The mode $\lambda_{m}$ for the
Southern Hemisphere is smaller than that of the Northern Hemisphere. Thus,
Northern Hemisphere is globally more active in terms of horizontal dispersion
than the Southern one. The same conclusions hold when looking at seasonally
averaged instead of annually averaged quantities (not shown).

Taking into account the observed differences between Northern and Southern
Hemispheres, we have repeated the same analyses over the main ocean basins in
a search for isolating the factors which could contribute to one or another
observed behaviors. In Fig. 3c we show the time evolution of $<\lambda>_{A}$
as computed over the six main ocean basins (North Atlantic, South Atlantic,
North Pacific, South Pacific, Indian Ocean and Southern Ocean), compared to
the one obtained over the global ocean. The Southern Ocean happens to be the
most active (in terms of horizontal stirring) because of the presence of the
Antarctic Circumpolar Current, followed by the Atlantic and Indian Oceans, and
finally the Pacific. We have also computed (Fig. 3d) PDFs of FSLE for the
different oceans. As before, we obtain broad, asymmetric PDFs with small modes
and heavy tails. The smallest mode $\lambda_{m}$ corresponds to the Southern
Pacific, meaning than there is less horizontal stirring activity in this
basin, in support of what is also visually evident in Fig. 3c. On the opposite
regime we observe that the largest FSLE values correspond to the Southern
Ocean. For the rest of oceans the PDFs are rather coincident with the whole
ocean PDF.

We have gone further to a smaller scale, by repeating the same analyses for
the main currents in the global ocean: Gulf Stream, Benguela, Kuroshio,
Mozambique, East Australian, California, Peru and Canary currents. As
evidenced by Fig. 3e there is a clear separation in two groups of currents in
terms of their horizontal stirring properties: the most active currents
(including Gulf Stream, Kuroshio, Mozambique and East Australian currents, all
of them WBCs) and the least active ones (including Benguela, California, Peru
and Canary Currents, which correspond to EBCs). The distinction remains in the
PDF analysis: we can clearly distinguish two groups of PDFs: a) narrow PDFs
highly peaked around a very small value of $\lambda$ (EBCs); b) PDFs peaking
at a slightly greater value of $\lambda$, but significantly broader (WBCs).
Since the PDFs of the WBCs are broader, large values of FSLEs are found more
frequently, i.e., more intense stirring occurs. This appears to be a
reflection of the well-known mechanism of Western Intensification by [39].
Also, the asymmetry and tails of the PDFs show that the FSLE field is
inhomogeneous and that there are regions with very different dispersion
properties. Following [3], asymmetry and heavy tails make the PDFs quite
different from the Gaussians expected under more homogeneous mixing. These
characteristics are then indications that chaotic motion plays a dominant role
versus turbulent, smaller scales, dynamics. That is, the large scale velocity
features control the dynamics, something that is also reflected in the
filamentary patterns of the LCS shown in Fig. 1.

###  3.3 Seasonal characterization of horizontal stirring

Horizontal stirring in the global ocean has a strong seasonal variability, as
shown in Fig. 3a. Maximum values of $<\lambda>_{A}$ in the Northern Hemisphere
are reached early in that hemisphere Summer, and minimum ones early in that
hemisphere Winter. The same happens for the Southern hemisphere related to its
Summer and Winter periods.

Seasonally averaged FSLEs in the whole ocean over the four seasons are shown
in Fig. 4. The spatial pattern is rather similar in all of them, and also
similar to the annually-averaged spatial distribution shown in Fig 2a. Higher
FSLE levels are found at the Gulf Stream and Kuroshio in the Northern
Hemisphere in Spring and Summer of that hemisphere. Analogously for the
Eastern Australia and Mozambique Currents in the Southern Hemisphere relative
to their own Spring and Summer time.

Following [44], to analyze which areas are more sensitive to seasonal changes,
we computed the standard deviation of the annual time series of FSLE (see Fig.
5). Larger values appear to correspond to the more energetic regions thus
showing a higher seasonal variability. More information about seasonal
variability of different oceanic regions can be obtained again from Fig. 3.
Time evolution of stirring in the North Atlantic and North Pacific, shown in
Fig. 3c, attains high values in Spring and Summer, and minimum ones in Winter.
Concerning the main currents, we found than values of stirring in Kuroshio,
Gulf Stream, East Australia, and Mozambique currents increase in Spring and
Summer and decrease in Winter (see Fig. 3e). This seasonal variability is also
present in EBCs but the amplitude of the changes is smaller than in WBCs.

The generic increase in mesoscale stirring in Summer time detected here with
Lyapunov methods has also been identified in previous works and several
locations [18, 32, 27, 31, 44] (in most of the cases from the EKE behavior
extracted from altimetric data). Although no consensus on a single mechanisms
seems to exist (see discussion in [44]) enhanced baroclinic instability has
been proposed in particular areas [32, 31], as well as reduced dissipation
during Summer [44].

We have also calculated longitudinal (zonal) averages of the time averages of
FSLE in Figs. 2a and 4. This is shown in Fig. 6 (top figure for the Northern
hemisphere and bottom figure for the Southern one). First of all, we see that
horizontal stirring has a general tendency to increase with latitude in both
hemispheres. One may wonder if this is a simple consequence of the decreasing
value of $\delta_{f}$ we take when increasing latitude. We have checked that
the same increasing tendency remains when the calculation is redone with a
constant $\delta_{f}$ over the whole globe (not shown), so that this trivial
effect is properly compensated by the factor $\ln(\delta_{f}/\delta_{0})$ in
the FSLE definition, and what we see in Fig. 6 is really a stronger stirring
at higher latitudes. Note that this type of dependence is more similar to the
equivalent sea surface slope variability, $K_{sl}$, calculated from altimetry
in [38] than to the raw zonal dependency of the EKE obtained in the same
paper. Since $K_{sl}$ is intended to represent Sea Surface Height variability
with the large scale components filtered out, we see again that our FSLE
calculation is capturing properly the mesoscale components of ocean stirring
observed by other means.

It is also clearly seen that latitudinal positions of local maxima of stirring
correspond to the main currents (e.g. Gulf Stream and Kuroshio around 35∘N;
Mozambique, Brazil and East Australia around 25∘S). The picture in Fig. 6
confirms that horizontal stirring is somehow higher in local Summer in mid-
latitudes, were the main currents are, for both hemispheres. At low and high
latitudes however the horizontal stirring is higher in local winter-time for
both hemispheres, which is particularly visible in the Northern Hemisphere at
high latitudes. A similar behavior was noted by [44] in the subpolar North
Pacific and part of the subpolar North Atlantic for EKE derived from
altimetry. Possible causes pointed there are barotropic instabilities or
direct wind forcing.

###  3.4 Lagrangian-Eulerian relations

Lagrangian measures such as FSLEs provide information on the cumulative effect
of flow at a given point, as it integrates the time-evolution of water parcels
arriving to that point. They are not directly related to instantaneous
measurements as those provided by Eulerian quantities such as EKE or
vorticity, unless some kind of dynamic equilibrium or ergodicity-type property
is established so that the time-integrated effect can be related to the
instantaneous spatial pattern (for instance, if the spatial arrangement of
eddies at a given time gives an idea about the typical time evolution of a
water parcel) or their averages. EKE gives information on the turbulent
component of the flow, which is associated to high eddy activity, while
relative vorticity $\omega$ takes into account the shear and the rotation of
the whole flow. Eventual establishment of such dynamic equilibrium would allow
to substitute in some instances time averages along trajectories by spatial
averages, so providing a useful tool for rapid diagnostics of sea state. Thus,
we will relate the Lagrangian stirring (as measured by the FSLEs) with an
instantaneous, Eulerian, state variable. Of course, the Lagrangian-Eulerian
relations will be useful only if the same, or only a few functional
relationships hold in different ocean regions. If the relation should be
recalculated for every study zone, the predictive power is completely lost.

We have thus explored the functional dependence of FLSEs with EKE and relative
vorticity. In Fig. 2 the time average of these three fields is shown.
Comparing FSLEs (Fig. 2a) and EKE (Fig. 2b), we see that high and low values
of these two quantities are generally localized in the same regions. There are
a few exceptions, such as the North Pacific Subtropical Countercurrent, which
despite being energetic [32] does not seem to produce enough pair dispersion
and stretching at the scales we are considering. It was already shown by [43]
and [42] that variations in horizontal stirring are closely related to
variations in mesoscale activity as measured by EKE. Note the similarity, with
also an analogous range of values, of the EKE plot in Fig. 2b), obtained from
a numerical model, to that of [42] (first figure), which is obtained from
altimetry data. In [43] a proportionality between the stretching rate (as
measured by FTLE) and $EKE^{1/4}$ was inferred for the Tasman Sea (a relation
was found but no fit was attempted in the global data set described in [42]).
In order to verify if a similar functional dependence between FSLE and EKE
could hold for our global scale dataset, we have computed different
conditioned averages (see Section 2), shown in Fig. 7: in the left panel we
present the conditioned average $\tilde{\lambda}(EKE)$, while in the right
panel $\tilde{\lambda}(\omega)$ is shown; both functions were derived from the
time averaged variables shown in Figure 2.

The smooth curve depicted in Fig. 7, left, is an indication of a well-defined
functional relationship between $\overline{\lambda}$ and $\overline{EKE}$,
similar to the ones found by [43] and [42] from altimeter data. Notice however
that the plot just gives conditioned averages, but the conditioned standard
deviation -which is a measure of randomness and fluctuations- is not
negligible. An idea of the scatter is given for selected areas in Fig. 8.
Considerably less compact relationships were obtained in the Mediterranean sea
[11]. Fig. 8 shows that very different dynamical regimes identified by
different values of $\lambda$ may correspond to the same level of EKE. As a
Lagrangian diagnostic, we believe that FSLE is more suitable to link
turbulence properties to tracer dynamics than Eulerian quantifiers such as
EKE. FSLEs provide complementary information since very energetic areas, with
large typical velocities, do not necessarily correspond to high stretching
regions. A paradigmatic example is a jet, or a shear flow, where small
dispersions may be found because of the absence of chaotic trajectories. A
functional relation between $\overline{\lambda}$ and $\overline{\omega}$ is
also obtained (Fig. 7, right), although it is much noisier and probably worse-
behaved. When particularizing for the different regions, we see that for EKE
the WBCs are all roughly associated with one particular functional relation
for the conditioned average $\overline{\lambda}$ while EBCs gather around a
different one. None of the two prototype Lagrangian-Eulerian relations fits
well to the relation $\lambda\propto EKE^{1/4}$ proposed for FTLE by [43] from
altimeter data in the Tasman sea. Data are too scarce to make a reliable
fitting for the conditioned average, in particular for the EBC. In Fig. 8 we
see that relations of the form $\lambda\propto EKE^{\alpha}$ could be
reasonably fitted to scatter plots of the data, with $\alpha$ larger than the
$0.25$ obtained in [43], specially for WBC were $\alpha$ is in the range
$(0.34,0.40)$. This quantitative difference of our results with [43] may rest
upon the fact that they considered just the Tasman Sea and we consider the
different oceans. Other sources for the difference could be that we are using
FSLE of velocity data from a numerical model, instead of FTLE from altimetry,
or that they use a grid of relatively low resolution $0.5^{\circ}\times
0.5^{\circ}$, while ours is $0.1^{\circ}\times 0.1^{\circ}$. Maybe their
coarser resolution is not enough to resolve filaments which are the most
relevant structures in our FSLE calculations. Despite this the qualitative
shape of the Lagrangian-Eulerian relations is similar to the previous works
[43, 42].

In order to analyze the ocean regions beyond boundary currents, we have also
computed the conditioned averages for the Equatorial Current and for a
$40^{\circ}$ longitude by $20^{\circ}$ latitude sub-region centered at
$245^{\circ}$ longitude and $-30^{\circ}$ latitude in the middle of the sub-
tropical gyre in the Pacific Ocean (and hence an area of scarce horizontal
stirring activity). We see (Fig. 7, left) that the EBC Lagrangian-Eulerian
relation is valid for these two areas. We have also verified that the
relations derived from annually-averaged quantities remain the same for
seasonal averages (not shown). The important point here is the occurrence of
just two different shapes for the EKE-FSLE relations across very different
ocean regions, which may make useful this type of parametrization of a
Lagrangian quantity in terms of an Eulerian one. For the relations of FSLE in
terms of relative vorticity, a distinction between WBC and EBC still exists
but the results are less clear and class separation is not as sharp as in the
case of EKE (see Fig. 7, right). For instance, Gulf Stream and Kuroshio,
despite being both WBC, do not seem to share the same Lagrangian-Eulerian
relation, which limits its usefulness.

##  4 Conclusions

In this paper we have studied the space and time variability of horizontal
stirring in the global ocean by means of FSLE analysis of the outputs of a
numerical model. Similarly to what has been done in previous works, FSLEs can
be taken as indicators of horizontal stirring. Being Lagrangian variables,
they integrate the evolution of water parcels and thus they are not completely
local quantities. We have taken averages to analyze two main time scales
(annual and seasonal) and three space scales (planetary scale, ocean scale and
horizontal boundary scale). Our velocity data were obtained by using
atmospheric forcing from NCEP. Structures and dynamics at small scales will be
probably more realistic if forcing with higher resolution observed winds, as
in [36]. But since we have not studied the first model layer which is directly
driven by wind, and we have focused on averages at relatively large time and
spatial scales, we do not expect much differences if using more detailed
forcing.

Horizontal stirring intensity tends to increase with latitude, probably as a
result of having higher planetary vorticity and stronger wind action at high
latitudes, or rather, as argued in [44] because of barotropic instabilities.
Certainly, new studies are required to evaluate these hypothesis. At a
planetary scale we observe a significantly different behavior in the Northern
hemisphere with respect to the Southern Hemisphere, the first being on average
more active in terms of horizontal stirring than the second one. This
difference can probably be explained by the greater relative areas of
subtropical gyres in the Southern Hemisphere with small stirring activity
inside them, which compensates in the averages the great activity of the
Antartic Circumpolar Current. At an ocean scale, we observe that the level of
stirring activity tends to decay as the size of subtropical gyres increases,
what is an indication that the most intense horizontal stirring takes place at
the geographical boundaries of ocean basins. For that reason, we have finally
analyzed the behavior of stirring at boundary scale, which is mainly related
to WBCs and EBCs. EBCs behave in a similar way to ocean interior in terms of
all the quantities we have computed, including the Lagrangian-Eulerian
relations. Thus, the main hot spots of horizontal stirring in the ocean are
WBC. The observed small mode in the global FSLE PDFs also indicates that
horizontal stirring is not very intense for the vast majority of the ocean,
but the heavy tails indicate the existence of large excursions at some
specific, stretched locations (e.g., inside the WBCs and other smaller scale
currents active enough to generate stirring). This type of uneven distribution
is characteristic of multifractal systems arising from large scale chaotic
advection, something that was discussed for oceanic FSLEs in [20].

Regarding seasonal variability, generally we observe stronger stirring during
each hemisphere’s Summer time. Medium and high latitudes behave however in the
opposite way: stirring is more active during the hemisphere Summer for medium
latitudes and during the hemisphere Winter for high latitudes. Medium
latitudes are strongly affected by the behavior of WBC, which experience
intensification of horizontal stirring during Summer [18, 32, 27, 31, 44]. As
commented before, high latitude Winter intense stirring could be the result of
a stronger action of wind during that period or of barotropic instabilities
[44], and dedicated studies are required to evaluate these hypothesis.

Finally, we have studied the connection between time-extended Lagrangian FSLEs
and instant Eulerian quantities such as EKE and relative vorticity. For the
case of EKE, the different ocean regions give rise to just two different
Lagrangian-Eulerian relations, associated to an intense or a weak stirring
regimes. The existence of these two regimes implies that pair dispersion and
stretching strength are larger in a class of ocean areas (represented by WBC)
than in another (e.g. EBC) at mesoscales, even when having the same EKE.

## Acknowledgments

I.H.-C., C.L. and E.H.-G. acknowledge support from MICINN and FEDER through
project FISICOS (FIS200760327); A. Turiel has received support from Interreg
TOSCA project (G-MED09-425) and Spanish MICINN project MIDAS-6
(AYA2010-22062-C05-01). The OFES simulation was conducted on the Earth
Simulator under the support of JAMSTEC. We thank Earth Simulator Center-
JAMSTECH team for providing these data.

## References

  * [1] V Artale, G Boffetta, A Celani, M Cencini, and A Vulpiani.  Dispersion of passive tracers in closed basins: Beyond the diffusion coefficient.  Phys. Fluids, 9:3162–3171, 1997.
  * [2] E Aurell, G Boffetta, A Crisanti, G Paladin, and A Vulpiani.  Predictability in the large: an extension of the Lyapunov exponent.  J. Phys. A, 30:1–26, 1997.
  * [3] F.J. Beron-Vera.  Mixing by low- and high-resolution surface geostrophic currents.  J. Geophys. Res., 115:C10027, 2010.
  * [4] F.J. Beron-Vera, M.J. Olascoaga, and G.J. Goni.  Oceanic mesoscale eddies as revealed by Lagrangian coherent structures.  Geophys. Res. Lett, 35:L12603, 2008.
  * [5] Francisco J Beron-Vera, María J Olascoaga, Michael G Brown, Huseyin Koçak, and Irina I Rypina.  Invariant-tori-like Lagrangian coherent structures in geophysical flows.  Chaos, 20(1):017514, 2010.
  * [6] J.H. Bettencourt, C. López, and E. Hernández-García.  Oceanic three-dimensional Lagrangian coherent structures: A study of a mesoscale eddy in the Benguela upwelling region.  Ocean Modell., to appear, 2012.
  * [7] G. Boffetta, A. Celani, M. Cencini, G. Lacorata, and A. Vulpiani.  Non-asymptotic properties of transport and mixing.  Chaos, 10:50–60, 2000.
  * [8] G. Boffetta, G. Lacorata, G. Redaelli, and A. Vulpiani.  Detecting barriers to transport: A review of different techniques.  Physica D, 159:58–70, 2001.
  * [9] G. Buffoni, P. Falco, A. Griffa, and E. Zambianchi.  Dispersion processes and residence times in a semi-enclosed basin with recirculating gyres: An application to the Tyrrhenian sea.  J. Geophys. Res.-Oceans, 102:18699–18713, 1997.
  * [10] F. d’Ovidio, V. Fernández, E. Hernández-García, and C. López.  Mixing structures in the Mediterranean sea from Finite-Size Lyapunov Exponents.  Geophys. Res. Lett., 31:L17203, 2004.
  * [11] F. d’Ovidio, J. Isern-Fontanet, C. López, E. Hernández-García, and E. García-Ladona.  Comparison between Eulerian diagnostics and Finite-Size Lyapunov Exponents computed from altimetry in the Algerian basin.  Deep-Sea Res. I, 56:15–31, 2009.
  * [12] Francesco d’Ovidio, Emily Shuckburgh, and Bernard Legras.  Local mixing events in the upper-troposphere and lower-stratosphere. Part I: detection with the Lyapunov diffusivity.  J. Atmos. Sci., 66(12):3678–3694, 2009.
  * [13] G. Haller.  Distinguished material surfaces and coherent structure in three-dimensional fluid flows.  Physica D, 149:248–277, 2001.
  * [14] G. Haller.  Lagrangian coherent structures from approximate velocity data.  Phys. Fluids, 14:1851–1861, 2002.
  * [15] G. Haller and G. Yuan.  Lagrangian coherent structures and mixing in two-dimensional turbulence.  Physica D, 147:352–370, 2000.
  * [16] George Haller.  A variational theory of hyperbolic Lagrangian coherent structures.  Physica D, 240(7):574 – 598, 2011.  Erratum and addendum: Physica D 241, 439–441 (2012).
  * [17] George Haller and Themistoklis Sapsis.  Lagrangian coherent structures and the smallest finite - time Lyapunov exponent.  Chaos, 21(2):023115, 2011.
  * [18] G Halliwell, D B Olson, and G Peng.  Stability of the Sargasso Sea subtropical frontal zone.  J. Phys. Oceanogr., 24(6):1166–1183, 1994.
  * [19] A. C. Haza, A. C. Poje, T. M. Özgökmen, and P. Martin.  Relative dispersion from a high-resolution coastal model of the Adriatic Sea.  Ocean Modell., 22:48–65, 2008.
  * [20] Ismael Hernández-Carrasco, Cristóbal López, Emilio Hernández-García, and Antonio Turiel.  How reliable are finite-size Lyapunov exponents for the assessment of ocean dynamics?  Ocean Modelling, 36(3-4):208 – 218, 2011.
  * [21] B. Joseph and B. Legras.  Relation between kinematic boundaries, stirring, and barriers for the Antartic polar vortex.  J. Atm. Sci., 59:1198–1212, 2002.
  * [22] T-Y. Koh and B. Legras.  Hyperbolic lines and the stratospheric polar vortex.  Chaos, 12(2):382–394, 2002.
  * [23] G Lacorata, E Aurell, and A Vulpiani.  Drifter dispersion in the Adriatic sea: Lagrangian data and chaotic model.  Ann. Geophysicae, 19:121–129, 2001.
  * [24] G. Lapeyre.  Characterization of finite-time Lyapunov exponents and vectors in two-dimensional turbulence.  Chaos, 12(3):688–698, 2002.
  * [25] Yukio Masumoto.  Sharing the results of a high-resolution ocean general circulation model under a multi-discipline framework - a review of OFES activities.  Ocean Dynamics, 60:633–652, 2010.
  * [26] Yukio Masumoto, Hideharu Sasaki, Takashi Kagimoto, Nobumasa Komori, Akio Ishida, Yoshikazu Sasai, Toru Miyama, Tatsuo Motoi, Humio Mitsudera, Keiko Takahashi, and et al.  A fifty-year eddy-resolving simulation of the world ocean - Preliminary outcomes of OFES (OGCM for the Earth Simulator).  J. of the Earth Simulator, 1(April):35–56, 2004.
  * [27] Rosemary Morrow, Aurore Brut, and Alexis Chaigneau.  Seasonal and interannual variations of the upper ocean energetics between tasmania and antarctica.  Deep-Sea Res. I, 50(3):339–356, 2003.
  * [28] T. M. Özgökmen, A. C. Poje, P. F. Fischer, and A. C. Haza.  Large eddy simulations of mixed layer instabilities and sampling strategies.  Ocean Modell., 39:311–331, 2011.
  * [29] Thomas Peacock and John Dabiri.  Introduction to Focus Issue: Lagrangian coherent structures.  Chaos, 20(1):017501, 2010.
  * [30] Andrew C Poje, Angelique C Haza, Tamay M Özgökmen, Marcello G Magaldi, and Zulema D Garraffo.  Resolution dependent relative dispersion statistics in a hierarchy of ocean models.  Ocean Modell., 31:36–50, 2010.
  * [31] B Qiu and S Chen.  Seasonal modulations in the eddy field of the South Pacific ocean.  J. Phys. Oceanogr., 34(7):1515–1527, 2004.
  * [32] Bo Qiu.  Seasonal eddy field modulation of the North Pacific Subtropical Countercurrent: TOPEX/Poseidon observations and theory.  J. Phys. Oceanogr., 29(10):2471–2486, 1999.
  * [33] V. Rossi, C. López, E. Hernández-García, J. Sudre, , V. Garçon, and Y. Morel.  Surface mixing and biological activity in the four Eastern Boundary upwelling systems.  Nonlin. Processes Geophys., 16:557–568, 2009.
  * [34] V. Rossi, C. López, J. Sudre, E. Hernández-García, and V. Garçon.  Comparative study of mixing and biological activity of the Benguela and Canary upwelling systems.  Geophys. Res. Lett., 35:L11602, 2008.
  * [35] I I Rypina, F J Beron-Vera, M G Brown, H Kocak, M J Olascoaga, and I A Udovydchenkov.  On the Lagrangian dynamics of atmospheric zonal jets and the permeability of the stratospheric polar vortex.  J. Atm. Sci., 33149:3595, 2006.
  * [36] Hideharu Sasaki, Yoshikazu Sasai, Masami Nonaka, Yukio Masumoto, and Shintaro Kawahara.  An eddy-resolving simulation of the quasi-global ocean driven by satellite-observed wind field –  preliminary outcomes from physical and biological fields -.  J. Earth Simulator, 6:35–49, 2006.
  * [37] Shawn C. Shadden, Francois Lekien, and Jerrold E. Marsden.  Definition and properties of Lagrangian coherent structures from finite-time Lyapunov exponents in two-dimensional aperiodic flows.  Physica D, 212(34):271 – 304, 2005.
  * [38] Detlef Stammer.  Global characteristics of ocean variability estimated from regional TOPEX/Poseidon altimeter measurements.  J. Phys. Oceanogr., 27(8):1743–1769, 1997.
  * [39] H Stommel.  The westward intensification of wind-driven ocean currents.  Trans. Amer. Geophys. Union, 29(2):202–206, 1948.
  * [40] E. Tew Kai, V. Rossi, J. Sudre, H. Weimerskirch, C. López, E. Hernández-García, F. Marsac, and V. Garçon.  Top marine predators track Lagrangian coherent structures.  Proc. Natl. Acad. Sci. U.S.A., 106(20):8245–8250, 2009.
  * [41] Alexandra Tzella and Peter H. Haynes.  Smooth and filamental structures in chaotically advected chemical fields.  Phys. Rev. E, 81:016322, Jan 2010.
  * [42] D. W. Waugh and E. R. Abraham.  Stirring in the global surface ocean.  Geophys. Res. Lett., 35:L20605, 2008.
  * [43] D. W. Waugh, E. R. Abraham, and M. M. Bowen.  Spatial variations of stirring in the surface ocean: A case of study of the Tasman sea.  J. Phys. Oceanogr., 36:526–542, 2006.
  * [44] Xiaoming Zhai, Richard J. Greatbatch, and Jan-Dirk Kohlmann.  On the seasonal variability of eddy kinetic energy in the Gulf Stream region.  Geophys. Res. Lett., 35:L24609, 2008.

Figure 1: Top: Snapshot of spatial distributions of FSLEs backward in time
corresponding to November 11, 1990 of the OFES output. Resolution is
$\delta_{0}=1/10^{\circ}$. Bottom: Zoom in the area of the box inside top
figure (South Atlantic Ocean). Coherent structures and vortices can be clearly
seen. The colorbar has units of $day^{-1}$.

Figure 2: a) Time average of the FSLEs in the Global Ocean. Geographical
regions of different stirring activity appear. The colorbar has units of
$day^{-1}$. b) Spatial distribution of annual $EKE^{1/2}$ (cm/s). c) Time
average of Relative Vorticity ($\omega$) in the Global Ocean. The color bar
has units of $day^{-1}$. In all the plots the averages are over the 52 weekly
maps computed from November 1st, 1990 to October 31th, 1991.

Figure 3: Left column: Temporal evolution (from November 1st, 1990 to October
31th, 1991) of the horizontal stirring (Spatial average of FSLEs). Right
column: PDFs of the FSLEs (histograms are built from the $\lambda$ values
contained at all locations of the 52 weekly maps computed for the second
simulation output year). Top: for both hemispheres and for the whole ocean.
Middle: for different oceanic regions. Bottom: for some main currents during
one simulation year. In addition to the results from the second surface layer
analyzed through the paper, panel a) shows also stirring intensity in a layer
close to 100m depth.

Figure 4: Time average of the FSLEs in the Global Ocean for the each season.
Spring: from March 22 to June 22. Summer: from June 22 to September 22.
Autumn: from September 22 to December 22. Winter: from December 22 to March
22. The colorbar has units of $day^{-1}$.

Figure 5: Standard deviation of weekly FSLE maps of one year. The colorbar has
units of $day^{-1}$

Figure 6: Cross-ocean zonal average of the annual, relative Summer and
relative Winter time average of FSLE maps from Fig 2a as a function of
latitude (expressed as absolute degrees from Equator to make both hemispheres
comparable). Top: Northern Hemisphere; bottom: Southern Hemisphere.

Figure 7: Left: Lagrangian-Eulerian relations. Left: the conditional average
$\tilde{\lambda}_{EKE}$ as a function of its corresponding annually averaged
(second year) $\overline{EKE}$ for different regions and currents. We clearly
observe two groups of relations FSLE-EKE. Right: same plot for the conditional
average $\tilde{\lambda}_{\omega}$ as a function of its corresponding annually
averaged (second year) $\overline{\omega}$. Although we observe also the same
two two groups of FSLE-$\omega$ relations, these functions are much noisier
and region-dependent.

Figure 8: Scatter plots showing temporally averaged FSLE values at different
spatial points in regions of Fig. 2a, and EKE values (as displayed in Fig. 2b)
at the same points. The regions displayed here are eight of the main currents.
Fittings of the type $y=cX^{b}$ are also displayed, where $y$ is the temporal
mean of FSLE and $X$ is $\textrm{EKE}^{1/2}$. Note that this implies
$<\textrm{FSLE}>=c~{}\textrm{EKE}^{\alpha}$ with $\alpha=b/2$.