(a)  two head motion trajectories in VR

(b)  model-predicted cumulative neck muscle contraction and discomfort levels
over time

Figure 1.  Predicting the neck muscle contraction and discomfort levels of VR
users. (a) A VR user chooses between two candidate head motion trajectories of
seemingly similar muscular workload for a visual task. (b) Our computational
model predicts the user’s potential neck muscle contraction level and thus
perceived neck muscle discomfort before the movements happen. 3D asset credits
to Mixall, Bizulka, RootMotion at Unity, and shockwavegamez01, joseVG at
Sketchfab.

# Toward Optimized VR/AR Ergonomics:
Modeling and Predicting User Neck Muscle Contraction

Yunxiang Zhang  New York UniversityUSA<EMAIL_ADDRESS>,  Kenneth Chen
New York UniversityUSA<EMAIL_ADDRESS>and  Qi Sun  New York UniversityUSA
<EMAIL_ADDRESS>
(2023)

###### Abstract.

Ergonomic efficiency is essential to the mass and prolonged adoption of VR/AR
experiences. While VR/AR head-mounted displays unlock users’ natural wide-
range head movements during viewing, their neck muscle comfort is inevitably
compromised by the added hardware weight. Unfortunately, little quantitative
knowledge for understanding and addressing such an issue is available so far.

Leveraging electromyography devices, we measure, model, and predict VR users’
neck muscle contraction levels (MCL) while they move their heads to interact
with the virtual environment. Specifically, by learning from collected
physiological data, we develop a bio-physically inspired computational model
to predict neck MCL under diverse head kinematic states. Beyond quantifying
the cumulative MCL of completed head movements, our model can also predict
potential MCL requirements with target head poses only. A series of objective
evaluations and user studies demonstrate its prediction accuracy and
generality, as well as its ability in reducing users’ neck discomfort by
optimizing the layout of visual targets. We hope this research will motivate
new ergonomic-centered designs for VR/AR and interactive graphics
applications. Source code is released at: https://github.com/NYU-ICL/xr-
ergonomics-neck-comfort.

Ergonomics, Electromyography, Head-Mounted Display

††journalyear: 2023††copyright: acmlicensed††conference: Special Interest
Group on Computer Graphics and Interactive Techniques Conference Conference
Proceedings; August 6–10, 2023; Los Angeles, CA, USA††booktitle: Special
Interest Group on Computer Graphics and Interactive Techniques Conference
Conference Proceedings (SIGGRAPH ’23 Conference Proceedings), August 6–10,
2023, Los Angeles, CA, USA††price: 15.00††doi: 10.1145/3588432.3591495††isbn:
979-8-4007-0159-7/23/08††ccs: Computing methodologies Virtual reality††ccs:
Computing methodologies Mixed / augmented reality††ccs: Computing
methodologies Neural networks††ccs: Human-centered computing Human computer
interaction (HCI)

##  1\. Introduction

VR/AR devices unlock natural viewing experiences via their uniquely wide-field
displays. With head tracking, users can move their heads to shift attention
and interact with peripheral content (Bahill et al., 1975; Monteiro et al.,
2021). However, their current head-mounted form factors incur non-trivial “in
vitro” weight and shift the head’s natural center of mass (Chen et al., 2021).
The resulting changes in neck muscle state and workload have been evidenced to
cause discomfort and injuries (Marklin Jr et al., 2022; Forde et al., 2011;
Chihara and Seo, 2018; Penumudi et al., 2020; Souchet et al., 2022). Despite
emerging evidence and concerns over such ergonomic side effects,
comprehensively assessing and optimizing VR users’ muscular comfort is still
in its infancy.

A major cause of ergonomic discomfort is muscle fatigue and stress (Lowe,
1996), especially from external weights, e.g., HMDs (Knight and Baber, 2004;
Chihara and Seo, 2018). Unlike optically trackable body movements, measuring
muscular activities is remarkably difficult. Besides indirect sensing such as
calorimetry (Holdy, 2004), biometrics from electromyography (EMG) sensors
reveal muscular status via its detected electric potential generated by muscle
fibers. In fact, the EMG signals directly correlate to muscle contraction
(Komi and Viitasalo, 1976). Therefore, extensive literature attempted to
understand our muscular functionalities during daily tasks, with face/gaze
(Manssuer et al., 2016), arm/hand (Zhang et al., 2022), and full-body (Brown
et al., 2021).

Recent attention has arisen to measure the influence of emerging usage of HMDs
(Chen et al., 2021). For instance, Chihara et al. (2018) measured and
associated the altered muscular contraction with ergonomic discomfort by
studying various viewing and interaction postures. However, surprisingly, we
still have little quantitative knowledge of the introduced ergonomic effects
before deploying a VR/AR application. Computationally forecasting muscle
contraction is the foundation toward the ultimate aim - systematically
optimizing visual content for ergonomically enhanced VR/AR.

We present a biophysically-inspired model to predict VR users’ neck muscular
contraction and thus potential ergonomic discomforts over time. The model is
applicable to both after (given a head trajectory) and before (given a target
position) users’ head movements. We first perform a physiological study in VR
to obtain EMG-sensed biometrics from characterized natural head movements. The
analysis reveals muscle contraction’s significant correlations with head poses
and motion patterns. Developed upon the data, our biophysical model first
predicts the instantaneous muscle contraction given a head pose and angular
acceleration. Then, by approximating representative trajectories
(Farshadmanesh et al., 2012), the model further extends to forecast potential
discomfort given only the target location and before the head movement occurs.

Our objective measurements and user studies demonstrate the model’s: 1)
prediction accuracy and generalizability with both post-hoc estimation and
pre-hoc prediction, 2) capability in optimizing visual target layouts to
reduce user-perceived muscular discomfort.

We hope this research will motivate new ergonomic-centered designs for VR/AR.
As a first step, our model serves as a quantitative metric for evaluating and
optimizing immersive applications, e.g., button layout in AR assistive tools
or target positions in VR gaming.

In summary, our main contributions include:

  * •

an EMG-sensed biometrical dataset of VR users’ neck muscle activity,
characterizing wide ranges of head movements,

  * •

a biophysically formulated and learned model that predicts muscular
contraction with head poses and movements,

  * •

an extended metric that forecasts the viewing-induced muscular efforts and
discomfort level given the target position,

  * •

demonstrations of the model’s effectiveness in enhancing users’ muscle comfort
via altering targets’ spatial layouts.

##  2\. Related Work

###  2.1. Ergonomics in VR/AR Interaction

Ergonomic efficiency is essential to the mass adoption of VR/AR. Despite
extensive efforts on designing more lightweight HMDs, their current form
factors still considerably alter users’ behaviors. Consequently, muscular
discomfort (Forde et al., 2011; Penumudi et al., 2020; Chihara and Seo, 2018;
Souchet et al., 2022), especially in the neck and shoulders (Kim and Shin,
2018; Marklin Jr et al., 2022), may be induced. Prior research has studied the
impacts of head-supported mass on neck muscle activities under various
application scenarios to better design dedicated hardware devices (Thuresson
et al., 2003, 2005; Rubine-Gatina et al., 2022; Le et al., 2021), such as
military helmets. Complementing advances in hardware designs, we focus on
modeling and predicting muscle activities under the VR/AR settings, where
visual stimuli are controllable and optimizable, to guide the design of
virtual content for better ergonomic comfort. Our approach shares a similar
mindset to (Ruiz et al., 2018; Li et al., 2020), i.e., task-dependent
optimization of virtual content.

(a)  neck muscle anatomy

(b)  pilot study setup, EMG sensors, and stimuli

(c)  EMG data processing to obtain normalized MCL

(d)  MCL during stationary viewing

(e)  $\Delta$MCL during dynamic viewing (head movements)

Figure 2.  Pilot study illustration and results. (a) illustrates the major
muscles controlling head movements, with highlighted EMG sensor attachment
regions. (b) shows our experimental setup with EMG sensors annotated. (c) The
top row shows an example raw EMG sequence (light green curve and right Y-axis)
and its corresponding normalized MCL (dark green curve and left Y-axis). The
bottom row shows the total MCL integrated across all $4$ channels. (d)
visualizes the user-aggregated MCL for stationary viewing. (e) shows the
movement-induced $\Delta$MCL for dynamic viewing. Each $\Delta$MCL, i.e.,
arrow, was computed by subtracting the MCL during stationary viewing at an
anchor head pose from the average MCL during head movement from that anchor
head pose (arrow tail) to a target head pose (arrow head). The arrow lengths
were scaled to $13\%$ for easier visualization. 3D asset credits to joseVG and
danielmclogan at Sketchfab.

###  2.2. Muscle Contraction during Movements

Muscle contraction level is a core biometrical indicator for studying
ergonomics (Dugan and Frontera, 2000). Unlike motions, which can be reliably
tracked by cameras, quantifying muscle activities is notably more challenging.
Existing work mainly exploits EMG to reveal muscle activities by detecting the
electric signals propagating in neurally-activated muscles (Merletti and
Parker, 2004; Criswell, 2010). In particular, elevated EMG readings indicate
stronger muscle contraction and elevated discomfort over time (Chesler and
Durfee, 1997; Dimitrova and Dimitrov, 2003; Cifrek et al., 2009; Vigotsky et
al., 2018). Recently, researchers have explored machine learning techniques to
infer muscle activities from spinal cord signals (Guo et al., 2018; Gok and
Sahin, 2019) and simulated human musculoskeletal animation data (Nakada et
al., 2018).

###  2.3. Learning from Electromyography Signals

Learning EMG signals has emerged to understand human muscular behaviors with
various applications (Ahsan et al., 2009; Atzori and Müller, 2015; Phinyomark
and Scheme, 2018), including human-machine interfaces (Moon et al., 2005;
Atzori et al., 2016; Xiong et al., 2021; Karolus et al., 2022) and VR/AR
(Tsuboi et al., 2017; Hirota et al., 2018; Pai et al., 2019; Lou et al.,
2019). EMG data has also been leveraged to enable various sensing tasks, such
as body movement (Jaramillo-Yánez et al., 2020; Baldacchino et al., 2018; Zhao
et al., 2020; Javaid et al., 2021; Du et al., 2017; Wei et al., 2019), hand
(Liu et al., 2021) and head (Barniv et al., 2005; Sugiarto et al., 2021)
tracking. The flexible and non-invasive design of the latest EMG creates new
possibilities for understanding our behaviors invisible to cameras. For
instance, estimating force (Becker et al., 2018; Zhang et al., 2022; Gailey et
al., 2017; Bardizbanian et al., 2020; Martínez et al., 2020; Wu et al., 2021),
prosthetic (Castellini and Van Der Smagt, 2009; Gulati et al., 2021) and gait
(Papagiannis et al., 2019; Nazmi et al., 2019) control. We aim to achieve the
inverse by predicting the EMG-measurable muscle status from human head
movements.

##  3\. Neck Muscle Contraction Level during Head Movements in VR

###  3.1. Neck Muscles Controlling Head Rotations

We aim to model and predict neck muscle contraction level (MCL). Human neck is
a highly flexible skeletal structure that allows the head to change its pitch,
yaw, and roll angles. As shown in Figure 2a, the major neck muscles include
sternocleidomastoid (SCM) and splenius capitis (SC) (Vasavada et al., 1998).
In particular, SCM/SC laterally rotate the head to the opposite/same side when
acting unilaterally and flex/extend the head when acting bilaterally.

###  3.2. Experiment Description

#### Participants and setup

We recruited $8$ participants (ages $23-31$, $3$ female). 4 of them have prior
experience with VR headsets before the study. All participants reported normal
neck muscle conditions. For each participant, we attached $4$ Delsys Trigno
wireless EMG sensors on the left/right SCM and SC muscles (see Figures 2a and
2b). We originally tested with $6$ EMG sensors by also including the upper
trapezius (UT) muscles. However, EMG sensors on the UT exhibited significantly
weaker signals compared to the others and were thus excluded from the
experiments. The EMG sensors detect the electric potential (in millivolts, mV)
generated by users’ muscle fibers at $2000$ Hz and stream the data to a PC
with $<1$ms latency. During the study, every participant wore an Oculus Quest
2 head-mounted display (HMD), remained seated, and performed a target reaching
task with visual stimuli. Their head poses were tracked by the HMD and
streamed to the same PC for time synchronization with EMG. The HMD provides
$1872\times 1856$ resolution per eye at $90$ FPS, and $98^{\circ}/104^{\circ}$
vertical/horizontal field of view (FoV).

#### Stimuli and tasks

As illustrated in Figure 2b, the stimuli were sequentially displayed pairs of
spheres ($3^{\circ}$ in the FoV), one colored yellow and the other red. The
yellow sphere indicates an “anchor” head pose $\mathbf{r}\triangleq(p,y)$,
where $p/y$ represent pitch and yaw angles. The red sphere indicates a
“target” head pose that is $\mathbf{\Delta\mathbf{r}}\triangleq({\Delta
p},{\Delta y})$ away from $\mathbf{r}$. While keeping their torso stationary,
participants were instructed to rotate their heads from the yellow anchor to
the red target. Once the user successfully fixated on the target for $2$
seconds consecutively (simulating stationary viewing), both spheres were
shifted to continue with the next trial. A line connecting the two spheres was
rendered to guide the user’s head rotation to the next target, designed to
eliminate potential errors for target searching. Please refer to our video for
an example of the study process.

#### Conditions

Across participants, the stimuli pairs appeared randomly with a pre-sampled
anchor set
$\mathcal{R}\triangleq\\{(\mathbf{r}_{i},\mathbf{\Delta\mathbf{r}}_{i}),i=1,\dots
N_{\mathcal{R}}\\}$. Given the comfortable range of human head rotations (Ruiz
et al., 2018), we sampled stimulus directions within a $60^{\circ}\times
100^{\circ}$ visual field. We tested and chose not to cover additional
vertical range to ensure participants’ neck comfort and avoid sprains. We
started with a preliminary test to determine the effective range of
$\mathbf{r}$. Our test with $3$ participants showed non-trivial EMG change for
$|y|>15^{\circ}$ and $|p|>5^{\circ}$. Therefore, using a $10^{\circ}$ step
size, we sampled 63 anchor head poses for $p\in\\{\pm 30^{\circ},\pm
20^{\circ},\pm 10^{\circ},0^{\circ}\\}$ and $y\in\\{\pm 50^{\circ},\pm
40^{\circ},\pm 30^{\circ},\pm 20^{\circ},0^{\circ}\\}$. Here, $-/+$ indicates
left/right or down/up from the head’s forward-facing direction. For each of
the $63$ anchor poses $\mathbf{r}$, $8$ surrounding targets giving varying
movement patterns $\mathbf{\Delta\mathbf{r}}$ were studied. They were selected
with ${\Delta p}\in\\{\pm 35^{\circ},0\\},{\Delta y}\in\\{\pm
25^{\circ},0\\}$. We discarded the conditions with target stimuli outside of
the $60^{\circ}\times 100^{\circ}$ range. That is, we ensured
$\mathbf{r}+\mathbf{\Delta\mathbf{r}}\in[-30^{\circ},+30^{\circ}]\times[-50^{\circ},+50^{\circ}]$.

#### Duration

We split the study into $7$ sessions (about 5 minutes each) with enforced
breaks in between to avoid posture drifting. Every session was monitored to
ensure the subject remained stationary below the neck. The study, including
hardware setup, pre-study instructions, warm-up session (30 discarded trials),
and breaks, took about $2.5$ hours per participant. In total, we collected
about 5 hours of time-synchronized motion-EMG paired data.

#### Data processing and analysis

We aim to model and optimize neck muscle contraction driving head movements.
However, raw EMG signals cannot be directly used because they exhibit: 1)
frequency-dependent sensory noise; 2) oscillations between negative and
positive values; 3) left-right asymmetry due to sensor positioning error
(Chihara and Seo, 2018; Lehman and McGill, 1999); 4) cross-user difference in
scale for the same head movement. Therefore, similar to prior literature
(Sommerich et al., 2000; Reaz et al., 2006), we performed a series of EMG
signal processing, including detrending, bandpass filtering, and
rectification, as well as inter-channel balancing, normalization, and
integration. Please refer to Supplement A for details. At each time frame, our
processing pipeline outputs a single normalized muscle contraction value,
integrated across processed EMG signals from all $4$ channels. Figure 2c
illustrates the EMG-to-MCL transformation with an example sequence.

###  3.3. Results

#### Stationary viewing ($|\mathbf{\Delta\mathbf{r}}|=0$)

Figure 2d shows the MCL when the head remains static. The average normalized
MCL was $.32\pm.12$. The head pose demanding the least MCL ($.17\pm.02$) was
$\mathbf{r}=(0,0)$, significantly lower than far-reaching poses. For instance,
when the target was located at $\mathbf{r}=(30^{\circ},50^{\circ})$, the MCL
was higher at $.68\pm.09$. A repeated measures ANOVA showed that both $p$ and
$y$ had a significant main effect on MCL ($F_{6,42}=1.17,p<.001$ for $p$,
$F_{8,56}=1.5,p<.001$ for $y$). A significant $p\times y$ interaction effect
was also observed ($F_{48,336}=7.4,p<.001$). In particular, higher absolute
values of yaw elevate the corresponding average MCL, from $.21\pm.09$ with
$y=0^{\circ}$ to $.47\pm.15$ with $|y|=50^{\circ}$. A Mann-Kendall (M.K.)
trend test showed a significant monotonic trend ($\tau=1.0,p<.05$). On the
other hand, the effect from pitch was asymmetric and non-monotonic. The
highest values of $p$, $p=+30^{\circ}/-30\circ$ induces MCL at
$.49\pm.16$/$.27\pm.09$. An M.K. trend test did not show a significant
monotonic trend of pitch angle’s effect on MCL ($\tau=-.6,p=.07$).

#### Dynamic viewing ($|\mathbf{\Delta\mathbf{r}}|>0$)

Figure 2e visualizes the $\Delta$MCL during head movements, which was computed
by subtracting the stationary MCL at an anchor from the average MCL during the
movement. Introducing movements (i.e., non-zero $\mathbf{\Delta\mathbf{r}}$)
significantly elevated MCL up to $31.21\%$ across all studied $\mathbf{r}$. In
addition to $\mathbf{r}$, movement pattern $\mathbf{\Delta\mathbf{r}}$ jointly
influences the observed MCL. A repeated measures ANOVA showed that ${\Delta
p}\times{\Delta y}$ has a significant main effect on MCL
($F_{3,21}=23.44,p<.001$). For each time frame, we further extracted the
angular acceleration in both directions
$\boldsymbol{\alpha}\triangleq(\boldsymbol{\alpha}_{p},\boldsymbol{\alpha}_{y})$.
Pearson correlation coefficients were computed to assess the relationship;
There were positive correlations between MCL and $|\boldsymbol{\alpha}_{p}|$
($r(12431)=.14,p<.001$), as well as MCL and $|\boldsymbol{\alpha}_{y}|$
($r(12431)=.038,p<.001$). The elevation rate depends on individual
$\mathbf{r}$. For example, the rate in yaw direction with movement starting at
$\mathbf{r}=(30^{\circ},50^{\circ})$ was $88.9\%$ higher than
$\mathbf{r}=(0^{\circ},0^{\circ})$.

###  3.4. Discussion

The analysis above leads us to several observations and motivations for
learning a computational model. First, despite individual participants’
variances in muscular strengths and sizes, the measured MCL shares consistent
trends for each condition, both during static viewing and dynamic movements.
Second, the head pose (yaw $y$ and pitch $p$) significantly influences MCL. In
particular, despite the left-right symmetry with yaw, the pitch angle exhibits
significantly asymmetric and non-monotonic effects on MCL. Lower pitch angles
(i.e., heads facing downward) tend to reduce MCL. Third, in dynamic scenarios,
increasing acceleration significantly elevates MCL. The elevation effect size
depends on both the corresponding starting head pose and movement direction.

##  4\. Method: Modeling and Predicting Neck Muscle Contraction Level

The analysis of EMG-motion paired data from our pilot study motivates us to
establish a computational model correlating head movements with neck muscle
contraction level (MCL). Note that while MCL can be measured using EMG
sensors, they are 1) tedious and costly to deploy; 2) insufficient to
_forecast_ MCL before a movement happens. Therefore, we first propose a bio-
physically inspired MCL estimation model with open functions to characterize
muscle-driven head motions in Section 4.1. Using our collected data, we then
fit the open functions with machine learning models to:

  1. (1)

estimate the MCL associated with a completed head movement, i.e., _after_ a
movement happens (Section 4.2);

  2. (2)

predict the MCL for a potential movement using target directions only, i.e.,
_before_ a movement happens (Section 4.3).

###  4.1. Bio-Physically Inspired MCL Model

Muscle-generated torque is proportional to MCL (Watanabe and Akima, 2009;
Clancy et al., 2011; Paquin and Power, 2018). Our neck muscles actively
generate the required amount of torque to enable head rotation at varying
speeds. Denoting this active torque as
$\mathbf{\mathcal{T}_{a}}\in\mathbb{R}^{2}$, we establish a mapping
$\mathcal{E}\left(\cdot\right)$ such that
$\text{MCL}=\mathcal{E}\left(\mathbf{\mathcal{T}_{a}}\right)$.

From our pilot study (Figure 2d), maintaining the head stationary at various
poses requires significantly different levels of MCL, and thus
$\mathbf{\mathcal{T}_{a}}$. To maintain stationary viewing, however, there
must be another pose-dependent torque that counterbalances
$\mathbf{\mathcal{T}_{a}}$. We term this underlying torque as passive torque
$\mathbf{\mathcal{T}_{p}}\in\mathbb{R}^{2}$. We hypothesize that
$\mathbf{\mathcal{T}_{p}}$ is induced by factors such as gravity and muscle
relaxation. Notably, $\mathbf{\mathcal{T}_{p}}$ exists during both stationary
and dynamic viewing conditions, but only depends on head poses, i.e.,
$\mathbf{\mathcal{T}_{p}}\triangleq\mathbf{\mathcal{T}_{p}}\left(\mathbf{r}\right)$.
By contrast, we proactively generate $\mathbf{\mathcal{T}_{a}}$ to perform
stationary viewing (compensating $\mathbf{\mathcal{T}_{p}}$) and change our
head pose at wish for dynamic viewing. Therefore, through the moment of
inertia $I\in\mathbb{R}$, $\mathbf{\mathcal{T}_{a}}$ correlates both with the
head pose $\mathbf{r}$ and angular acceleration $\boldsymbol{\alpha}$:

(1)
$\displaystyle\mathbf{\mathcal{T}_{p}}\left(\mathbf{r}\right)+\mathbf{\mathcal{T}_{a}}\left(\mathbf{r},\boldsymbol{\alpha}\right)$
$\displaystyle=I\times\boldsymbol{\alpha}.$

This further transforms the mapping $\mathcal{E}\left(\cdot\right)$ between
$\mathbf{\mathcal{T}_{a}}$ and MCL:

(2)
$\displaystyle\begin{split}\text{MCL}&=\left(\mathcal{E}\circ\mathbf{\mathcal{T}_{a}}\right)\left(\mathbf{r},\boldsymbol{\alpha}\right)=\mathcal{E}\left(I\times\boldsymbol{\alpha}-\mathbf{\mathcal{T}_{p}}\left(\mathbf{r}\right)\right)\\\
&\triangleq\mathcal{H}_{m}\left(\mathcal{E},\mathbf{\mathcal{T}_{p}},I,\mathbf{r},\boldsymbol{\alpha}\right)\mapsto\mathbb{R^{+}},\end{split}$

where $I$, $\mathbf{\mathcal{T}_{p}}\left(\cdot\right)$, and
$\mathcal{E}\left(\cdot\right)$ are the unknowns that map
$\mathbf{r}$/$\boldsymbol{\alpha}$ to MCL.

###  4.2. Estimating MCL with Complete Trajectories

(a)  illustration of MCLNet’s architectural design

(b)  model-predicted vs. hardware-measured neck MCL

Figure 3.  MCLNet illustration and example MCL estimation results. (a)
illustrates the architectural design of MCLNet for jointly learning $I$,
$\mathbf{\mathcal{T}_{p}}\left(\cdot\right)$, and
$\mathcal{E}\left(\cdot\right)$ from head motion and MCL paired data; (b)
shows the model-predicted vs. hardware-measured MCL for a sample sequence from
the test set.

The data from our pilot study provide a large set of time-synchronized head
movement trajectories and MCL sequences:

(3)
$\displaystyle\mathbf{r}_{t},\boldsymbol{\alpha}_{t}\mapsto\mathcal{H}_{m}\left(\mathcal{E},\mathbf{\mathcal{T}_{p}},I,\mathbf{r}_{t},\boldsymbol{\alpha}_{t}\right).$

Using these paired sequential data, we formulate an MCL regression problem and
optimize 1D CNN models with $L_{2}$ loss to jointly approximate the unknowns
$I$, $\mathbf{\mathcal{T}_{p}}\left(\cdot\right)$, and
$\mathcal{E}\left(\cdot\right)$. The complete model, named MCLNet, is
illustrated in Figure 3a.

Notably, a phenomenon called electro-mechanical delay exists between EMG
signals and muscular motions. Depending on individuals and muscle areas, the
delay can incur a temporal offset between the two modalities up to $100$ms
(Cavanagh and Komi, 1979). To accommodate this temporal inconsistency for
robust prediction, our model takes in motion sequences with T=$400$ms window
and predicts MCL for the central $200$ms interval, i.e., inputs cover
additional $100$ms outputs from the beginning and end. Given a sequence of
uniformly sampled head poses $\\{\mathbf{r}^{t}\\}_{t=1}^{T}$, we first
calculate the corresponding angular accelerations
$\\{\boldsymbol{\alpha}^{t}\\}_{t=1}^{T}$ through finite difference, then
execute our model at each $t$ to obtain the overlapping sequences of predicted
MCL. Figure 3b visualizes the prediction-measurement comparison for an example
trajectory taken from the test set. Visualized results for each subject are
shown in Figure 8.

So far, a core requirement of MCLNet is the prior knowledge of completed head
movement trajectories. However, to benefit real-life applications such as UX
design and cinematography, we shall reduce the potential discomfort before
deploying to users. To this end, we further extend our model to _forecast_ MCL
before a movement.

###  4.3. Predicting MCL with Target Head Poses

Given the starting and ending head poses
$\\{\mathbf{\mathbf{r}_{s}},\mathbf{\mathbf{r}_{e}}\\}$ of a uni-directional
head movement, the required MCL to travel between them is determined by the
actual movement trajectory. However, as evidenced by our analysis in Section
3.4, $\\{\mathbf{\mathbf{r}_{s}},\mathbf{\mathbf{r}_{e}}\\}$ alone carry
significant influence on the overall MCL. Therefore, using our collected data,
we regress a representative motion trajectory for each pair of
$\\{\mathbf{\mathbf{r}_{s}},\mathbf{\mathbf{r}_{e}}\\}$, to approximate the
temporal patterns of angular velocity $\boldsymbol{\omega}_{t}$:

(4)
$\displaystyle\boldsymbol{\omega}_{t}(\mathbf{\mathbf{r}_{s}},\mathbf{\mathbf{r}_{e}})\in\mathbb{R}^{2},\text{\,
s.t.
\,}\mathbf{\mathbf{r}_{s}}+\int_{t=t_{s}}^{t_{e}}\boldsymbol{\omega}_{t}(\mathbf{\mathbf{r}_{s}},\mathbf{\mathbf{r}_{e}})\mathrm{d}t=\mathbf{\mathbf{r}_{e}}.$

Motivated by prior literature studying the main sequence effect of head
movements (Zangemeister et al., 1981) and our observations of a single main
peak in each velocity profile (Figure 4), we perform a unimodal Gaussian
approximation for the angular velocity:

(5)
$\displaystyle\boldsymbol{\omega}_{t}^{i}(\mathbf{\mathbf{r}_{s}},\mathbf{\mathbf{r}_{e}})\triangleq
A^{i}(\mathbf{\mathbf{r}_{s}},\mathbf{\mathbf{r}_{e}})e^{-\frac{\left(t-\mu^{i}(\mathbf{\mathbf{r}_{s}},\mathbf{\mathbf{r}_{e}})\right)^{2}}{2\left(\sigma^{i}(\mathbf{\mathbf{r}_{s}},\mathbf{\mathbf{r}_{e}})\right)^{2}}},i\in\\{p,y\\}.$

Using our collected data, we formulate a trajectory regression problem and
optimize a Multi-Layer Perceptron (MLP) model, annotated as TrajectoryNet, to
predict $\\{A^{i},\mu^{i},\sigma^{i}\\}_{i\in\\{p,y\\}}$ given an arbitrary
pair of head poses $\\{\mathbf{\mathbf{r}_{s}},\mathbf{\mathbf{r}_{e}}\\}$.
Using predicted angular velocity curves, we can approximate the overall MCL:

(6)
$\displaystyle\mathrm{H}_{c}(\mathbf{\mathbf{r}_{s}},\mathbf{\mathbf{r}_{e}})=\int_{t=t_{s}}^{t_{e}}\mathcal{H}_{m}\left(\mathcal{E},\mathbf{\mathcal{T}_{p}},I,\mathbf{\mathbf{r}_{s}}+\int_{\bar{t}=t_{s}}^{t}\boldsymbol{\omega}_{\bar{t}}(\mathbf{\mathbf{r}_{s}},\mathbf{\mathbf{r}_{e}})\mathrm{d}\bar{t},\dot{\boldsymbol{\omega}}_{t}\right)\mathrm{d}t.$

Figure 4 compares the velocity curves collected from our users with
TrajectoryNet’s predictions over an example pair of
$\\{\mathbf{\mathbf{r}_{s}},\mathbf{\mathbf{r}_{e}}\\}$ taken from the test
set. Full implementation details for both MCLNet and TrajectoryNet can be
found in Supplement B.

Figure 4.  Predicting head motion trajectories with starting/ending head
poses. Light/dark curves show tracked/predicted angular velocities.
Orange/green curves show angular velocities in pitch/yaw directions.

##  5\. Evaluation

We present a series of objective measurements on our model’s performance in
predicting neck MCL, and a subjective psychophysical study to demonstrate how
the prediction reflects neck muscle discomfort. We first evaluate MCLNet’s
estimation accuracy with complete head motion trajectories in Section 5.1,
then extend to predict MCL _before_ head movements by incorporating
TrajectoryNet in Section 5.2. Through a user study, we demonstrate our
method’s potential in forecasting and reducing users’ neck workload for a more
comfortable VR experience in Section 5.3.

###  5.1. MCL Estimation: After Head Movements

#### Experimental setup

We leveraged our data from the conditions detailed in Section 3.2 to train our
model. Additionally, during the pilot study, we also collected two groups of
conditions to establish an evaluation dataset with unseen conditions. It
contains $4$ pitch and $4$ yaw angles, $p\in\\{\pm 25^{\circ},\pm
5^{\circ}\\},y\in\\{\pm 45^{\circ},\pm 15^{\circ}\\}$, resulting in a total of
$16$ anchor head poses. The same $8$ surrounding targets with travel angle
${\Delta p}\in\\{\pm 35^{\circ},0^{\circ}\\},{\Delta y}\in\\{\pm
25^{\circ},0^{\circ}\\}$ were introduced to each of them. Note that the
evaluation conditions were designed to contain no overlap with the training
set. Due to the extra long collection process and thus scheduling conflicts,
$6$ ($3$ female) of the $8$ participants completed the evaluation condition
session. We adopted their data for this experiment. Two quantitative metrics
were considered: Normalized Root-Mean-Square Error (NRMSE) and Normalized Mean
Absolute Error (NMAE). The metrics are applied to measure the error ratio
between the model-predicted and hardware-measured (by the same method detailed
in Supplement A) MCL; a lower error ratio indicates better model performance.

#### Results and discussion

MCLNet achieves an overall performance of $12.39\pm 4.74\%$ NRMSE and $9.54\pm
4.14\%$ NMAE across all 6 subjects and 16 anchor head poses. Figure 6
summarizes its subject-wise performance. Beyond the average accuracy, we
further measure the correlation. That is, whether the model can predict the
elevation/reduction of MCL consistently with the hardware measurement. We
leveraged Pearson’s and Spearman’s coefficients between the two conditions.
The results indicate a significant correlation between model predictions and
hardware measurements ($r(70066)=.62,p<.001$ and $r(70066)=.60,p<.001$). The
analyzes above validate our method’s effectiveness in estimating neck MCL when
head motion trajectories were known beforehand.

###  5.2. MCL Prediction: Before Head Movements

#### Experimental setup

For this experiment, we first used the head motion data from Section 5.1 to
optimize and validate TrajectoryNet (Section 4.3), then combined it with
MCLNet to predict MCL using target head poses only. Since our dataset consists
of sequences of stationary head pose followed by pose-changing movements, we
extracted the dynamic part to construct a dataset of starting/ending head
poses $\\{\mathbf{\mathbf{r}_{s}},\mathbf{\mathbf{r}_{e}}\\}$ paired with HMD-
tracked trajectories. The train-test data split from Section 5.1 was used for
evaluation.

#### Results and discussion

The performance of our MCL prediction framework, composed of trajectory
regression and MCL estimation, is shown in Figure 7. TrajectoryNet achieves an
overall NRMSE/NMAE of $3.54\pm 1.11\%$/$2.16\pm 0.65\%$ in pitch velocity and
$3.45\pm 0.98\%$/$2.01\pm 0.51\%$ in yaw. The overall MCL prediction
performance is $16.76\pm 6.05\%$ NRMSE and $14.71\pm 5.96\%$ NMAE. Pearson’s
and Spearman’s coefficients between the two conditions are
$r(70066)=.59,p<.001$ and $r(70066)=.57,p<.001$, indicating a significant
correlation. The results above demonstrate that our method can reliably
predict the potential MCL with only the target head poses, before the actual
movement occurs.

###  5.3. Predicting and Reducing Neck Discomfort

(a)  setting and stimuli

(b)  visualization of conditions

(c)  aggregated voting results

Figure 5.  Stimuli and results of our neck discomfort user study. (a) shows
the stimuli. (b) visualizes the visual targets’ angular distributions of $3$
example conditions. Color gradients indicate the temporal order of appearance.
All 3 conditions share the same total head rotations with full visual field
coverage. (c) summarizes the voting distribution of the $3$ comparisons on
which condition being more uncomfortable. Individual votes per condition are
detailed in Table 1. 3D asset credits to Mixall at Unity.

#### Participants and setup

We recruited $13$ participants (ages $20-35$, $6$ female). None of them were
aware of the hypothesis, the research, or the number of conditions. One
participant came in with a prior condition of neck injury and was excluded
during warm-up sessions. There was no overlap between these participants and
those from the pilot study in Section 3. We conducted the study using an
Oculus Quest 2 HMD (_without EMG_). During the study, participants observed
stimuli through the HMD and were instructed to remain seated while keeping
their torso stationary. The study took around $60$ minutes for each
participant, including breaks between sessions.

#### Stimuli

As shown in Figure 5a, we developed and experimented with a 3D balloon-popping
game with target acquisition. The stimuli were displayed as a sequence of red
balloon targets rendered in an amusement park scene. As an indicator, the
target color was changed to magenta upon a participant’s fixation. A line
pointing toward the next target was displayed to guide the user.

#### Tasks

The task was designed as two-alternative forced choice to avoid bias from
scaled rating, similar to prior works measuring muscular discomfort (Pinto et
al., 2021; Farid et al., 2018). During each session, the balloon targets were
displayed, one at a time, following a pre-defined scan path with $31$ targets
in different directions
$\mathcal{S}=\\{\mathbf{r}_{t},t=0,\dots,30,\mathbf{r}_{i}\in\mathcal{R}\\}$,
similar to the definition in Section 3.2. Participants were instructed to
rotate their heads to fixate on the balloon until it disappeared. To trigger
both dynamic and stationary head status, a 1-second fixation on each target
was enforced before the next one appeared at $\mathbf{r}_{t+1}$. Each session
contained a pair of two sequentially tested $\mathcal{S}$ that were generated
from $2$ out of $3$ different conditions, as detailed in the _conditions_
paragraph. After each session, the participants were instructed to use the
keyboard to indicate “which one of the two scan paths was more uncomfortable,
tiring, or difficult for your neck?”.

#### Conditions

Guided by our model, we designed $3$ conditions of progressively generated
scan paths. They were created to ensure an identical total head rotations and
similar spatial coverage but varied cumulative MCL. To ensure fair movements
across conditions, we randomly pre-partitioned a fixed amount of total head
rotations $900^{\circ}$ into $30$ steps. Then, at each step $t$, the next
target pose $\mathbf{r}_{t+1}$ was chosen from a set of candidate poses by
maximizing the corresponding score function of the selected condition:

  1. $\mathbf{MAX}$:

$\mathrm{C}(\mathbf{r}_{i\in[1,t+1]})+\mathrm{H}_{c}(\mathbf{r}_{t},\mathbf{r}_{t+1})$

  2. $\mathbf{RND}$:

$\mathrm{C}(\mathbf{r}_{i\in[1,t+1]})$

  3. $\mathbf{MIN}$:

$\mathrm{C}(\mathbf{r}_{i\in[1,t+1]})-\mathrm{H}_{c}(\mathbf{r}_{t},\mathbf{r}_{t+1})$

Here, $\mathrm{C}$ is a term to ensure full visual field coverage for
condition-wise fairness. On average, the ratio between the cumulative MCL of
the three conditions $\mathbf{MAX}$/$\mathbf{RND}$/$\mathbf{MIN}$ was $3.48$
vs. $1.95$ vs. $1.00$. Please refer to Supplement D for details on our
condition generation algorithm and Figure 5b for an example of each condition.
The three conditions generate $3$ different pairs for 2AFC comparisons, namely
$\mathbf{C_{1}}$: $\mathbf{MAX}$ vs. $\mathbf{MIN}$; $\mathbf{C_{2}}$:
$\mathbf{RND}$ vs. $\mathbf{MAX}$; $\mathbf{C_{3}}$: $\mathbf{MIN}$ vs.
$\mathbf{RND}$. We repeated the random pre-partition of total head rotations
and condition generation process to get 6 sets of
$\mathbf{MAX}$/$\mathbf{RND}$/$\mathbf{MIN}$ conditions for a total of 18
sessions. The appearance order of the 18 sessions was randomized and counter-
balanced across participants, same for the 2 conditions within each session,
to avoid bias.

#### Results

Table 1 in Supplement E and Figure 5c show individual votes and the summary
for each comparison, respectively. By aggregating all sessions,
$\mathbf{MAX}$/$\mathbf{RND}$/$\mathbf{MIN}$ were $86.1\%$/$50.7\%$/$13.1\%$
voted as being more uncomfortable in the related comparisons. Among all
comparisons, The difference was significantly higher than a random guess
($50\%$). By analyzing individual votes, a repeated measures ANOVA indicated
that the condition had a statistically significant effect on the votes
($F_{2,22}=89.46,p<.001$). Post-hoc pairwise $t$-tests with Holm adjustments
showed that the difference was significant among all $3$ comparisons ($p<.001$
for all conditions).

#### Discussion

The analysis above shows the significant difference in participants’
subjectively perceived discomfort levels among the three conditions. The
participant-rated discomfort levels, $\mathbf{MAX}>\mathbf{RND}>\mathbf{MIN}$,
also matched our model’s prediction ($\mathrm{H}_{c}$). Note that the
significant difference was not induced by head rotation angles which were
ensured to be identical via our progressive trajectory generation. These
results demonstrated our model’s capability of predicting a user’s neck
discomfort with target head poses only, i.e., before the head movement takes
place.

##  6\. Limitations and Future Work

This work considered the influence of yaw and pitch angles on MCL, but not the
roll dimension due to the challenges of precisely manipulating it with visual
stimuli and natural head movements. However, it may also contribute to MCL
(Keshner et al., 1989). Introducing alternative tasks, such as full body
movement (Imai et al., 2001), may enable controlling roll angles. We plan to
investigate the options concerning their effects on noise and movement
naturalness. Similar extensions include other muscle groups, such as shoulders
during interaction (Chihara and Seo, 2018).

In Section 4.3, we estimate the motion trajectory speed as a Gaussian
representation given a starting and ending head pose. Despite the
representativeness (Hage et al., 2019), the approximation may not fully
contain the individuals’ behavioral variances. We envision probabilistic
modeling and learning (Ghahramani, 2015) may further reveal the statistical
variances across users.

##  7\. Conclusion

Using EMG sensors, we present biometrically-measured data that reveals VR
users’ neck muscular contraction levels and thus potential discomfort. By
leveraging the data, we learn a computational model that quantitatively
predicts the MCL, both after and before a head movement occurs. We hope the
research to motivate new ergonomic and health-aware designs for VR/AR and
interactive computer graphics, toward answering essential questions such as
“will VR/AR devices induce additional ergonomic burdens on users if they
replace smartphones and monitors for everyday usage?”, “how do we
theoretically design more comfortable immersive displays and interfaces before
they are deployed?”. To this end, our model may be applied to ergonomic-aware
VR/AR interface optimization, immersive video editing, and beyond.

###### Acknowledgements.

This project is partially supported by the National Science Foundation grants
#2225861 and #2232817, and a DARPA PTG program.

## References

  * (1)
  * Ahsan et al. (2009) Md Rezwanul Ahsan, Muhammad I Ibrahimy, Othman O Khalifa, et al. 2009\.  EMG signal classification for human computer interaction: a review.  _European Journal of Scientific Research_ 33, 3 (2009), 480–501.
  * Atzori et al. (2016) Manfredo Atzori, Matteo Cognolato, and Henning Müller. 2016\.  Deep learning with convolutional neural networks applied to electromyography data: A resource for the classification of movements for prosthetic hands.  _Frontiers in neurorobotics_ 10 (2016), 9.
  * Atzori and Müller (2015) Manfredo Atzori and Henning Müller. 2015.  The Ninapro database: a resource for sEMG naturally controlled robotic hand prosthetics. In _2015 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC)_. IEEE, 7151–7154.
  * Bahill et al. (1975) A T Bahill, D Adler, and L Stark. 1975.  Most naturally occurring human saccades have magnitudes of 15 degrees or less.  _Investigative Ophthalmology & Visual Science_ 14, 6 (06 1975), 468–469.  arXiv:https://arvojournals.org/arvo/content_public/journal/iovs/933061/468.pdf
  * Baldacchino et al. (2018) Tara Baldacchino, William R Jacobs, Sean R Anderson, Keith Worden, and Jennifer Rowson. 2018\.  Simultaneous force regression and movement classification of fingers via surface EMG within a unified Bayesian framework.  _Frontiers in bioengineering and biotechnology_ 6 (2018), 13\.
  * Bardizbanian et al. (2020) Berj Bardizbanian, Jennifer Keating, Xinming Huang, and Edward A Clancy. 2020.  Estimating Individual and Combined Fingertip Forces From Forearm EMG During Constant-Pose, Force-Varying Tasks. In _2020 42nd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC)_. IEEE, 3134–3137.
  * Barniv et al. (2005) Yair Barniv, Mario Aguilar, and Erion Hasanbelliu. 2005\.  Using EMG to anticipate head motion for virtual-environment applications.  _IEEE Transactions on Biomedical Engineering_ 52, 6 (2005), 1078–1093.
  * Becker et al. (2018) Vincent Becker, Pietro Oldrati, Liliana Barrios, and Gábor Sörös. 2018.  Touchsense: classifying finger touches and measuring their force with an electromyography armband. In _Proceedings of the 2018 ACM International Symposium on Wearable Computers_. 1–8.
  * Brown et al. (2021) Geoffrey L Brown, Nidhi Seethapathi, and Manoj Srinivasan. 2021\.  A unified energy-optimality criterion predicts human navigation paths and speeds.  _Proceedings of the National Academy of Sciences_ 118, 29 (2021), e2020327118.
  * Castellini and Van Der Smagt (2009) Claudio Castellini and Patrick Van Der Smagt. 2009.  Surface EMG in advanced hand prosthetics.  _Biological cybernetics_ 100, 1 (2009), 35–47.
  * Cavanagh and Komi (1979) Peter R Cavanagh and Paavo V Komi. 1979.  Electromechanical delay in human skeletal muscle under concentric and eccentric contractions.  _European journal of applied physiology and occupational physiology_ 42, 3 (1979), 159–163.
  * Chen et al. (2021) Yumiao Chen, Xin Wang, and Huijia Xu. 2021.  Human factors/ergonomics evaluation for virtual reality headsets: a review.  _CCF Transactions on Pervasive Computing and Interaction_ 3, 2 (2021), 99–111.
  * Chesler and Durfee (1997) Naomi C Chesler and William K Durfee. 1997.  Surface EMG as a fatigue indicator during FES-induced isometric muscle contractions.  _Journal of Electromyography and Kinesiology_ 7, 1 (1997), 27–37.
  * Chihara and Seo (2018) Takanori Chihara and Akihiko Seo. 2018.  Evaluation of physical workload affected by mass and center of mass of head-mounted display.  _Applied ergonomics_ 68 (2018), 204–212.
  * Cifrek et al. (2009) Mario Cifrek, Vladimir Medved, Stanko Tonković, and Saša Ostojić. 2009.  Surface EMG based muscle fatigue evaluation in biomechanics.  _Clinical biomechanics_ 24, 4 (2009), 327–340.
  * Clancy et al. (2011) Edward A Clancy, Lukai Liu, Pu Liu, and Daniel V Zandt Moyer. 2011.  Identification of constant-posture EMG–torque relationship about the elbow using nonlinear dynamic models.  _IEEE Transactions on Biomedical Engineering_ 59, 1 (2011), 205–212.
  * Criswell (2010) Eleanor Criswell. 2010\.  _Cram’s introduction to surface electromyography_.  Jones & Bartlett Publishers.
  * Dimitrova and Dimitrov (2003) NA Dimitrova and GV Dimitrov. 2003.  Interpretation of EMG changes with fatigue: facts, pitfalls, and fallacies.  _Journal of Electromyography and Kinesiology_ 13, 1 (2003), 13–36.
  * Du et al. (2017) Yu Du, Yongkang Wong, Wenguang Jin, Wentao Wei, Yu Hu, Mohan S Kankanhalli, and Weidong Geng. 2017.  Semi-Supervised Learning for Surface EMG-based Gesture Recognition.. In _IJCAI_. 1624–1630.
  * Dugan and Frontera (2000) Sheila A Dugan and Walter R Frontera. 2000.  Muscle fatigue and muscle injury.  _Physical medicine and rehabilitation clinics of North America_ 11, 2 (2000), 385–403.
  * Farid et al. (2018) Bassem Farid, Paul Yielder, Michael Holmes, Heidi Haavik, and Bernadette A Murphy. 2018\.  Association of subclinical neck pain with altered multisensory integration at baseline and 4-week follow-up relative to asymptomatic controls.  _Journal of manipulative and physiological therapeutics_ 41, 2 (2018), 81–91.
  * Farshadmanesh et al. (2012) Farshad Farshadmanesh, Patrick Byrne, Hongying Wang, Brian D Corneil, and J Douglas Crawford. 2012.  Relationships between neck muscle electromyography and three-dimensional head kinematics during centrally induced torsional head perturbations.  _Journal of Neurophysiology_ 108, 11 (2012), 2867–2883.
  * Forde et al. (2011) Kelsey A Forde, Wayne J Albert, Michael F Harrison, J Patrick Neary, James Croll, and Jack P Callaghan. 2011.  Neck loads and posture exposure of helicopter pilots during simulated day and night flights.  _International Journal of Industrial Ergonomics_ 41, 2 (2011), 128–135.
  * Gailey et al. (2017) Alycia Gailey, Panagiotis Artemiadis, and Marco Santello. 2017\.  Proof of concept of an online EMG-based decoding of hand postures and individual digit forces for prosthetic hand control.  _Frontiers in neurology_ 8 (2017), 7.
  * Ghahramani (2015) Zoubin Ghahramani. 2015\.  Probabilistic machine learning and artificial intelligence.  _Nature_ 521, 7553 (2015), 452–459.
  * Gok and Sahin (2019) Sinan Gok and Mesut Sahin. 2019.  Prediction of forelimb EMGs and movement phases from corticospinal signals in the rat during the reach-to-pull task.  _International journal of neural systems_ 29, 07 (2019), 1950009\.
  * Gulati et al. (2021) Paras Gulati, Qin Hu, and S Farokh Atashzar. 2021.  Toward Deep Generalization of Peripheral EMG-Based Human-Robot Interfacing: A Hybrid Explainable Solution for NeuroRobotic Systems.  _IEEE Robotics and Automation Letters_ 6, 2 (2021), 2650–2657.
  * Guo et al. (2018) Yi Guo, Sinan Gok, and Mesut Sahin. 2018.  Convolutional networks outperform linear decoders in predicting EMG from spinal cord signals.  _Frontiers in neuroscience_ 12 (2018), 689.
  * Hage et al. (2019) Renaud Hage, Fabien Buisseret, Laurent Pitance, Jean-Michel Brismée, Christine Detrembleur, and Frédéric Dierick. 2019.  Head-neck rotational movements using DidRen laser test indicate children and seniors’ lower performance.  _PloS one_ 14, 7 (2019), e0219515.
  * Hirota et al. (2018) Mamoru Hirota, Ayumu Tsuboi, Masayuki Yokoyama, and Masao Yanagisawa. 2018.  Gesture recognition of air-tapping and its application to character input in VR space.  In _SIGGRAPH Asia 2018 Posters_. 1–2.
  * Holdy (2004) Kalman E Holdy. 2004\.  Monitoring energy metabolism with indirect calorimetry: instruments, interpretation, and clinical application.  _Nutrition in Clinical Practice_ 19, 5 (2004), 447–454.
  * Imai et al. (2001) Takao Imai, Steven T Moore, Theodore Raphan, and Bernard Cohen. 2001.  Interaction of the body, head, and eyes during walking and turning.  _Experimental brain research_ 136, 1 (2001), 1–18.
  * Jaramillo-Yánez et al. (2020) Andrés Jaramillo-Yánez, Marco E Benalcázar, and Elisa Mena-Maldonado. 2020.  Real-time hand gesture recognition using surface electromyography and machine learning: a systematic literature review.  _Sensors_ 20, 9 (2020), 2467.
  * Javaid et al. (2021) Haider Ali Javaid, Mohsin Islam Tiwana, Ahmed Alsanad, Javaid Iqbal, Muhammad Tanveer Riaz, Saeed Ahmad, and Faisal Abdulaziz Almisned. 2021.  Classification of Hand Movements Using MYO Armband on an Embedded Platform.  _Electronics_ 10, 11 (2021), 1322.
  * Karolus et al. (2022) Jakob Karolus, Simon Thanheiser, David Peterson, Nicolas Viot, Thomas Kosch, Albrecht Schmidt, and Paweł W Wozniak. 2022.  Imprecise but Fun: Playful Interaction Using Electromyography.  _Proceedings of the ACM on Human-Computer Interaction_ 6, MHCI (2022), 1–21.
  * Keshner et al. (1989) Emily A Keshner, D Campbell, RT Katz, and BW Peterson. 1989\.  Neck muscle activation patterns in humans during isometric head stabilization.  _Experimental Brain Research_ 75, 2 (1989), 335–344.
  * Kim and Shin (2018) Eunjee Kim and Gwanseob Shin. 2018.  Head rotation and muscle activity when conducting document editing tasks with a head-mounted display. In _Proceedings of the Human Factors and Ergonomics Society Annual Meeting_ , Vol. 62. SAGE Publications Sage CA: Los Angeles, CA, 952–955.
  * Knight and Baber (2004) James F Knight and Chris Baber. 2004.  Neck muscle activity and perceived pain and discomfort due to variations of head load and posture.  _Aviation, space, and environmental medicine_ 75, 2 (2004), 123–131.
  * Komi and Viitasalo (1976) Paavo V Komi and Jukka HT Viitasalo. 1976.  Signal characteristics of EMG at different levels of muscle tension.  _Acta Physiologica Scandinavica_ 96, 2 (1976), 267–276.
  * Le et al. (2021) Peter Le, Charles A Weisenbach, Emily HL Mills, Lanie Monforton, and Micah J Kinney. 2021\.  Exploring the interaction between head-supported mass, posture, and visual stress on neck muscle activation.  _Human Factors_ (2021), 00187208211019154\.
  * Lehman and McGill (1999) Gregory J Lehman and Stuart M McGill. 1999.  The importance of normalization in the interpretation of surface electromyography: a proof of principle.  _Journal of manipulative and physiological therapeutics_ 22, 7 (1999), 444–446.
  * Li et al. (2020) Wanwan Li, Biao Xie, Yongqi Zhang, Walter Meiss, Haikun Huang, and Lap-Fai Yu. 2020\.  Exertion-aware path generation.  _ACM Trans. Graph._ 39, 4 (2020), 115.
  * Liu et al. (2021) Yilin Liu, Shijia Zhang, and Mahanth Gowda. 2021.  NeuroPose: 3D Hand Pose Tracking using EMG Wearables. In _Proceedings of the Web Conference 2021_. 1471–1482.
  * Lou et al. (2019) Jianwen Lou, Yiming Wang, Charles Nduka, Mahyar Hamedi, Ifigeneia Mavridou, Fei-Yue Wang, and Hui Yu. 2019.  Realistic facial expression reconstruction for VR HMD users.  _IEEE Transactions on Multimedia_ 22, 3 (2019), 730–743.
  * Lowe (1996) Nancy K Lowe. 1996\.  The pain and discomfort of labor and birth.  _Journal of Obstetric, Gynecologic, & Neonatal Nursing_ 25, 1 (1996), 82–92.
  * Manssuer et al. (2016) Luis R Manssuer, Ralph Pawling, Amy E Hayes, and Steven P Tipper. 2016.  The role of emotion in learning trustworthiness from eye-gaze: Evidence from facial electromyography.  _Cognitive Neuroscience_ 7, 1-4 (2016), 82–102.
  * Marklin Jr et al. (2022) Richard W Marklin Jr, Ashley M Toll, Eric H Bauman, John J Simmins, John F LaDisa Jr, and Robert Cooper. 2022.  Do Head-Mounted Augmented Reality Devices Affect Muscle Activity and Eye Strain of Utility Workers Who Do Procedural Work? Studies of Operators and Manhole Workers.  _Human factors_ 64, 2 (2022), 305–323.
  * Martínez et al. (2020) Itzel Jared Rodríguez Martínez, Andrea Mannini, Francesco Clemente, and Christian Cipriani. 2020.  Online grasp force estimation from the transient EMG.  _IEEE Transactions on Neural Systems and Rehabilitation Engineering_ 28, 10 (2020), 2333–2341.
  * Merletti and Parker (2004) Roberto Merletti and Philip J Parker. 2004.  _Electromyography: physiology, engineering, and non-invasive applications_. Vol. 11.  John Wiley & Sons.
  * Monteiro et al. (2021) Pedro Monteiro, Guilherme Gonçalves, Hugo Coelho, Miguel Melo, and Maximino Bessa. 2021\.  Hands-free interaction in immersive virtual reality: A systematic review.  _IEEE Transactions on Visualization and Computer Graphics_ 27, 5 (2021), 2702–2713.
  * Moon et al. (2005) Inhyuk Moon, Myungjoon Lee, Junuk Chu, and Museong Mun. 2005\.  Wearable EMG-based HCI for electric-powered wheelchair users with motor disabilities. In _Proceedings of the 2005 IEEE International Conference on Robotics and Automation_. IEEE, 2649–2654.
  * Nakada et al. (2018) Masaki Nakada, Tao Zhou, Honglin Chen, Tomer Weiss, and Demetri Terzopoulos. 2018.  Deep learning of biomimetic sensorimotor control for biomechanical human animation.  _ACM Transactions on Graphics (TOG)_ 37, 4 (2018), 1–15.
  * Nazmi et al. (2019) Nurhazimah Nazmi, Mohd Azizi Abdul Rahman, Shin-Ichiroh Yamamoto, and Siti Anom Ahmad. 2019.  Walking gait event detection based on electromyography signals using artificial neural network.  _Biomedical Signal Processing and Control_ 47 (2019), 334–343.
  * Pai et al. (2019) Yun Suen Pai, Tilman Dingler, and Kai Kunze. 2019\.  Assessing hands-free interactions for VR using eye gaze and electromyography.  _Virtual Reality_ 23, 2 (2019), 119–131.
  * Papagiannis et al. (2019) Georgios I Papagiannis, Athanasios I Triantafyllou, Ilias M Roumpelakis, Frantzeska Zampeli, Pinioti Garyfallia Eleni, Panayiotis Koulouvaris, Elias C Papadopoulos, Panayiotis J Papagelopoulos, and George C Babis. 2019\.  Methodology of surface electromyography in gait analysis: review of the literature.  _Journal of medical engineering & technology_ 43, 1 (2019), 59–65.
  * Paquin and Power (2018) James Paquin and Geoffrey A Power. 2018.  History dependence of the EMG-torque relationship.  _Journal of Electromyography and Kinesiology_ 41 (2018), 109–115.
  * Penumudi et al. (2020) Sai Akhil Penumudi, Veera Aneesh Kuppam, Jeong Ho Kim, and Jaejin Hwang. 2020.  The effects of target location on musculoskeletal load, task performance, and subjective discomfort during virtual reality interactions.  _Applied ergonomics_ 84 (2020), 103010.
  * Phinyomark and Scheme (2018) Angkoon Phinyomark and Erik Scheme. 2018.  EMG pattern recognition in the era of big data and deep learning.  _Big Data and Cognitive Computing_ 2, 3 (2018), 21\.
  * Pinto et al. (2021) Eleana A Pinto, Dimitri ML van Ryckeghem, Ann Meulders, Diana M Torta, Amber Claus, and Stefaan Van Damme. 2021.  Motor action changes pain perception: a sensory attenuation paradigm in the context of pain.  _Pain_ 162, 7 (2021), 2060–2069.
  * Reaz et al. (2006) Mamun Bin Ibne Reaz, M Sazzad Hussain, and Faisal Mohd-Yasin. 2006.  Techniques of EMG signal analysis: detection, processing, classification and applications.  _Biological procedures online_ 8, 1 (2006), 11–35.
  * Rubine-Gatina et al. (2022) Simona Rubine-Gatina, Nadina Rimere, Zane Zundane, Alise Gulajeva, and Jelena Reste. 2022\.  Sternocleidomastoid muscle and head position: How to minimize muscle tension.  _IISE Transactions on Occupational Ergonomics and Human Factors_ (2022), 1–9.
  * Ruiz et al. (2018) Nataniel Ruiz, Eunji Chong, and James M Rehg. 2018\.  Fine-grained head pose estimation without keypoints. In _Proceedings of the IEEE conference on computer vision and pattern recognition workshops_. 2074–2083.
  * Sommerich et al. (2000) Carolyn M Sommerich, Sharon MB Joines, Veerle Hermans, and Samuel D Moon. 2000.  Use of surface electromyography to estimate neck muscle activity.  _Journal of Electromyography and kinesiology_ 10, 6 (2000), 377–398.
  * Souchet et al. (2022) Alexis D Souchet, Domitile Lourdeaux, Alain Pagani, and Lisa Rebenitsch. 2022.  A narrative review of immersive virtual reality’s ergonomics and risks at the workplace: cybersickness, visual fatigue, muscular fatigue, acute stress, and mental overload.  _Virtual Reality_ (2022), 1–32.
  * Sugiarto et al. (2021) Tommy Sugiarto, Chun-Lung Hsu, Chi-Tien Sun, Wei-Chun Hsu, Shu-Hao Ye, and Kuan-Ting Lu. 2021\.  Surface EMG vs. High-Density EMG: Tradeoff Between Performance and Usability for Head Orientation Prediction in VR Application.  _IEEE Access_ 9 (2021), 45418–45427.
  * Thuresson et al. (2005) Marcus Thuresson, Björn Äng, Jan Linder, and Karin Harms-Ringdahl. 2005.  Mechanical load and EMG activity in the neck induced by different head-worn equipment and neck postures.  _International journal of industrial ergonomics_ 35, 1 (2005), 13–18.
  * Thuresson et al. (2003) Marcus Thuresson, Jan Linder, Karin Harms-Ringdahl, et al. 2003\.  Neck muscle activity in helicopter pilots: effect of position and helmet-mounted equipment.  _Aviation, space, and environmental medicine_ 74, 5 (2003), 527–532.
  * Tsuboi et al. (2017) Ayumu Tsuboi, Mamoru Hirota, Junki Sato, Masayuki Yokoyama, and Masao Yanagisawa. 2017.  A proposal for wearable controller device and finger gesture recognition using surface electromyography.  In _SIGGRAPH Asia 2017 Posters_. 1–2.
  * Vasavada et al. (1998) Anita N Vasavada, Siping Li, and Scott L Delp. 1998.  Influence of muscle morphometry and moment arms on the moment-generating capacity of human neck muscles.  _Spine_ 23, 4 (1998), 412–422.
  * Vigotsky et al. (2018) Andrew D Vigotsky, Israel Halperin, Gregory J Lehman, Gabriel S Trajano, and Taian M Vieira. 2018\.  Interpreting signal amplitudes in surface electromyography studies in sport and rehabilitation sciences.  _Frontiers in physiology_ (2018), 985.
  * Watanabe and Akima (2009) Kohei Watanabe and Hiroshi Akima. 2009.  Normalized EMG to normalized torque relationship of vastus intermedius muscle during isometric knee extension.  _European journal of applied physiology_ 106, 5 (2009), 665–673.
  * Wei et al. (2019) Wentao Wei, Qingfeng Dai, Yongkang Wong, Yu Hu, Mohan Kankanhalli, and Weidong Geng. 2019\.  Surface-electromyography-based gesture recognition by multi-view deep learning.  _IEEE Transactions on Biomedical Engineering_ 66, 10 (2019), 2964–2973.
  * Wu et al. (2021) Changcheng Wu, Qingqing Cao, Fei Fei, Dehua Yang, Baoguo Xu, Guanglie Zhang, Hong Zeng, and Aiguo Song. 2021\.  Optimal strategy of sEMG feature and measurement position for grasp force estimation.  _PloS one_ 16, 3 (2021), e0247883.
  * Xiong et al. (2021) Dezhen Xiong, Daohui Zhang, Xingang Zhao, and Yiwen Zhao. 2021\.  Deep learning for EMG-based human-machine interaction: A review.  _IEEE/CAA Journal of Automatica Sinica_ 8, 3 (2021), 512–533.
  * Zangemeister et al. (1981) Wolfgang H Zangemeister, Ashby Jones, and Lawrence Stark. 1981\.  Dynamics of head movement trajectories: main sequence relationship.  _Experimental neurology_ 71, 1 (1981), 76–91.
  * Zhang et al. (2022) Yunxiang Zhang, Benjamin Liang, Boyuan Chen, Paul M Torrens, S Farokh Atashzar, Dahua Lin, and Qi Sun. 2022.  Force-aware interface via electromyography for natural VR/AR interaction.  _ACM Transactions on Graphics (TOG)_ 41, 6 (2022), 1–18.
  * Zhao et al. (2020) Yihui Zhao, Zhiqiang Zhang, Zhenhong Li, Zhixin Yang, Abbas A Dehghani-Sanij, and Shengquan Xie. 2020.  An EMG-driven musculoskeletal model for estimating continuous wrist motion.  _IEEE Transactions on Neural Systems and Rehabilitation Engineering_ 28, 12 (2020), 3113–3120.

Figure 6. Performance of MCLNet for neck MCL estimation when complete head
motion trajectories are known. Performance of MCLNet for neck MCL estimation
when complete head motion trajectories are known.

(a)  performance of TrajectoryNet for angular velocity prediction (pitch) with
target head poses only

(b)  performance of TrajectoryNet for angular velocity prediction (yaw) with
target head poses only

(c)  performance of MCLNet coupled with TrajectoryNet for neck MCL prediction
with target head poses only

Figure 7.  Performance of our neck MCL prediction method (MCLNet
$+$TrajectoryNet) with target head poses only.Our method can reliably predict
the potential neck MCL of a head movement before it takes place.

(a)  Subject 1 (b)  Subject 2

(c)  Subject 3 (d)  Subject 4

(e)  Subject 5 (f)  Subject 6

Figure 8.  Qualitative MCL prediction results.Model-predicted vs. hardware-
measured neck MCL for each of the 6 subjects who contributed evaluation data.
Each sequence shown is randomly sampled from that particular subject’s
evaluation data.