# A Simple and Optimal Sublinear Algorithm for Mean Estimation

Beatrice Bertolotti  Matteo Russo  Chris Schwiegelshohn

###### Abstract

We study the sublinear mean estimation problem. Specifically, we aim to output
a point minimizing the sum of squared Euclidean distances. We show that a
multiplicative $(1+\varepsilon)$ approximation can be found with probability
$1-\delta$ using $O(\varepsilon^{-1}\log\delta^{-1})$ many independent random
samples. We also provide a matching lower bound.

##  1 Introduction

An extremely simple algorithmic paradigm is to sample a subset of the input
independently and uniformly at random and solve a problem on the sample. Such
algorithms are a type of sublinear algorithms and form a staple of large data
analysis. The most important property of a sublinear algorithm is that the
query of sample complexity is as small as possible. For most problems it is
rare for a sublinear algorithm with only constant queries to produce
multiplicative guarantees. Remarkably, finding the multivariate mean of a
point set $A$ in Euclidean space is one of the cases where not only can we
obtain a multiplicative error with constant queries, but can do so to
arbitrary precision. Specifically, we are interested in the objective

$\min_{c\in\mathbb{R}^{d}}\sum_{p\in A}\|p-c\|^{2}.$

It is well known that the mean minimizes this expression. We say that $c$ is a
$(1+\varepsilon)$-approximate mean, if the cost of $c$ is within a
$(1+\varepsilon)$ factor of the cost of the optimal mean.

The exact sample complexity for obtaining approximate means is still an open
problem. Inaba et al. [8] showed that $O(\varepsilon^{-1})$ uniform samples
are enough to find a $(1+\varepsilon)$ approximate mean on expectation.
Unfortunately, obtaining a high probability guarantee, that is a success
probability of $1-\delta$ with this analysis requires
$O(\varepsilon^{-1}\delta^{-1})$ many samples. Cohen-Addad et al. [4] improved
the dependence on the probability of failure by achieving a sample complexity
of $\tilde{O}(\varepsilon^{-5}\log^{2}\delta^{-1})$, however the dependency on
the precision parameter $\varepsilon$ deteriorated. In the same paper, the
authors also gave a lower bound of $\Omega(\varepsilon^{-1})$ for any
sublinear algorithm succeeding with constant probability, showing that the
Inaba et al. [8] result was optimal in the constant success probability
regime. Recently and concurrently, Woodruff and Yasuda [13] achieved almost
best of both worlds trade-offs with a sample complexity of
$O(\varepsilon^{-1}(\log\varepsilon^{-1}+\log\delta^{-1})\log\delta^{-1})$.

Our contribution is an algorithm with optimal sample complexity. We show that
$\Theta(\varepsilon^{-1}\log\delta^{-1})$ independent uniform samples are both
necessary and sufficient.

###  1.1 Related Work

The related work for obtaining approximate sample means has been covered in
the introduction. An alternative way of quantifying an approximate mean by
outputting a point close to the true mean has been studied in [10, 9]. While
there is some overlap, as the distance to the true mean is an important
quantity in our setting as well, the sample complexity, as well as underlying
distributional assumptions are different.

In a more general setting, one could ask for approximate power means
minimizing the objective

$\min_{c\in\mathbb{R}^{d}}\sum_{p\in A}\|p-c\|^{z}.$

for some $z$. This problem was first posed by Cohen-Addad et al. [4] who
showed that $O(\varepsilon^{-z-5}\log^{2}\delta^{-1})$ samples were sufficient
for a high success probability while $\Omega(\varepsilon^{-z+1})$ samples were
necessary for constant success probability. Woodruff and Yasuda [13]
effectively closed this problem up to logarithmic dependencies with a sample
complexity of
$O(\varepsilon^{-z+1}(\log\varepsilon^{-1}+\log\delta^{-1})\log\delta^{-1})$.
Indeed, the
$O(\varepsilon^{-1}(\log\varepsilon^{-1}+\log\delta^{-1})\log\delta^{-1})$
bound also holds for $z\in(1,2]$, which is nearly optimal due to a matching
lower bound by [11]. Surprisingly, the case $z=1$, also known as the geometric
median, has a strong phrase transition, requiring $\Omega(\varepsilon^{-2})$
many samples, see [1, 12]. In terms of upper bounds, Cohen et al. [3] showed
that $O(\varepsilon^{-2})$ samples are sufficient for a success probability of
at least $1-\varepsilon$.

Finally, the special case $z\rightarrow\infty$ corresponds to the minimum
enclosing ball problem. The solution is highly susceptible to outliers, thus
any sublinear algorithm must either be allowed to discard a fraction of the
input, or rely on additive guarantees, see [6, 7] and [2] for examples.

##  2 Notation and Preliminaries

Given a vector $x\in\mathbb{R}^{d}$, we use
$\|x\|=\sqrt{\sum_{i=1}^{d}x_{i}^{2}}$ to denote the Euclidean norm. For a set
of points $A\subset\mathbb{R}^{d}$, we use $\mu(A):=\frac{1}{|A|}\sum_{p\in
A}p$ to denote the mean, using $\mu$ if $A$ is clear from the context.
Finally, we use $OPT=\min_{c\in\mathbb{R}^{d}}\sum_{p\in A}\|p-c\|^{2}$. We
will also use the following lemma.

######  Lemma 2.1 (High Dimensional Mean-Variance Decomposition, [8]).

For any set of points $A\subset\mathbb{R}^{d}$ and any $c\in\mathbb{R}^{d}$,
we have

$\sum_{p\in A}\|p-c\|^{2}=\sum_{p\in A}\|p-\mu\|^{2}+|A|\cdot\|\mu-c\|^{2}.$

An immediate consequence, which we will frequently use is that a candidate
solution $c$ is a $(1+\varepsilon)$ approximate mean if and only if
$\|\mu-c\|\leq\sqrt{\varepsilon\cdot\frac{OPT}{|A|}}$.

##  3 The Sublinear Algorithm

Given a set of points $S$ sampled uniformly at random from $A$, we call
$\mu(S)$ the empirical estimator of $\mu(A)$. It is easy to show that
$\mathbb{E}\left[\|\mu(A)-\mu(S)\|^{2}\right]=\frac{1}{|S|}\cdot\frac{OPT}{|A|}$.
Our main goal is to obtain a high probability bound, using the following
algorithm.

Algorithm 1

for $i=1,\ldots,b\log\frac{1}{\delta}$ do

Take a uniform sample $S_{i}$ with $|S_{i}|=\frac{4a}{\varepsilon}$ points and

Compute the sample mean $\hat{\mu}_{i}=\frac{1}{|S_{i}|}\cdot\sum_{p\in
S_{i}}p$

for each $\hat{\mu}_{i}$ do

Sort the remaining $\hat{\mu}_{j}$ by increasing distance, that is
$\hat{\mu}_{i}(1)=\hat{\mu}_{i}=\min_{j\in\\{1,\ldots
b\log\frac{1}{\delta}\\}}\|\hat{\mu}_{j}-\hat{\mu}_{i}\|$ and
$\hat{\mu}_{i}(b\log\frac{1}{\delta})=\max_{j\in\\{1,\ldots
b\log\frac{1}{\delta}\\}}\|\hat{\mu}_{j}-\hat{\mu}_{i}\|$.

Let $D_{i}:=\sum_{j=1}^{2/3\cdot
b\log\delta^{-1}}\|\hat{\mu}_{i}(j)-\hat{\mu}_{i}\|$

Output $\hat{\mu}_{i}$ with minimal $D_{i}$

######  Theorem 3.1.

There exists a sublinear algorithm sampling at most
$O(\varepsilon^{-1}\log\delta^{-1})$ points that outputs a
$(1+\varepsilon)$-approximate Euclidean mean with probability $1-\delta$. The
running time is $O((\varepsilon^{-1}+\log\delta^{-1})\cdot\log\delta^{-1})$.

###### Proof.

We first recall the expectation bound, before proving the high probability
bound. The former bound is arguably folklore, but included for completeness.
We have

$\displaystyle\mathbb{E}\left[\|\hat{\mu}_{i}-\mu\|^{2}\right]$
$\displaystyle=\mathbb{E}\left[\|\frac{1}{|S_{i}|}\sum_{p\in
S_{i}}p-\mu\|^{2}\right]=\frac{1}{|S_{i}|^{2}}\mathbb{E}\left[\|\sum_{p\in
S_{i}}(p-\mu)\|^{2}\right]$
$\displaystyle=\frac{1}{|S_{i}|^{2}}\left(\mathbb{E}\left[\sum_{p\in
S_{i}}\|p-\mu\|^{2}\right]+\mathbb{E}\left[\sum_{p\neq
q}(p-\mu)^{T}(q-\mu)\right]\right)=\frac{1}{|S_{i}|}\frac{OPT}{|A|}$

where the third equality follows from the law of cosines and the fourth
equality $\mathbb{E}[p-\mu]=0$.

We now move onto the high probability statement. By Markov’s inequality, with
probability at least $1/4$,

$\|\hat{\mu}_{i}-\mu\|^{2}\leq\frac{\varepsilon}{a}\cdot\frac{OPT}{n}.$ (1)

Denote by $F_{i}$ the event that the estimator $\mu_{i}$ does not satisfy Eq.
1. We have $\mathbb{E}[F_{i}]\leq 1/4$. Moreover, the total number of
estimators that do not satisfy Eq. 1 is
$M=\sum_{i=1}^{b\log\delta^{-1}}F_{i}$. Denote by the event $E$ that
$M<1/3\cdot b\log\delta^{-1}$ Due to the Chernoff bound, we have

$\displaystyle\mathbb{P}\left[M\geq 1/3\cdot
b\log\delta^{-1}\right]\leq\mathbb{P}\left[M\geq(1+1/12)\cdot\mathbb{E}[M]\right]\leq\text{exp}\left(-\frac{(1/12)^{2}\cdot
1/4\cdot b\log\delta^{-1}}{3}\right)$

which is less than $\delta$ for a sufficiently large value of $b$ (e.g.
$b>2000$ is sufficient).

A natural goal is to extract an estimate $\hat{\mu}_{i}$ that did not fail.
Unfortunately, we are not able to do so. However, we are able to extract and
estimator $\hat{\mu}_{i}$ that is close to an estimator that did not fail.
First, let $\hat{\mu}_{i}$ be an estimator that did not fail. Since at least
$2/3$ of the estimators did not fail, we know that $\hat{\mu}_{i}(2/3\cdot
b\log\delta^{-1})\leq\sqrt{\frac{\varepsilon}{a}\cdot\frac{OPT}{n}}$, which
likewise implies that $D_{i}\leq 2/3\cdot
b\log\delta^{-1}\cdot\sqrt{\frac{\varepsilon}{a}\cdot\frac{OPT}{n}}$. Now
suppose that $\hat{\mu}_{j}$ is the estimator returned by the algorithm.
Clearly, $D_{j}\leq 2/3\cdot
b\log\delta^{-1}\cdot\sqrt{\frac{\varepsilon}{a}\cdot\frac{OPT}{n}}$. Let
$succ(\hat{\mu}_{j})$ be the set of successful estimators among the $2/3\cdot
b\log\delta^{-1}$ closest to $\hat{\mu}_{j}$. Moreover, conditioned on event
$E$, at least $1/3\cdot b\log\delta^{-1}$ of the $2/3\cdot b\log\delta^{-1}$
estimators closest to $\hat{\mu}_{j}$ did not fail, that is
$|succ(\hat{\mu}_{j})|\geq 1/3\cdot b\log\delta^{-1}$. We therefore have

$\displaystyle\sum_{\hat{\mu}_{i}\in
succ(\hat{\mu}_{j})}\|\hat{\mu}_{j}-\hat{\mu}_{i}\|\leq 2/3\cdot
b\log\delta^{-1}\cdot\sqrt{\frac{\varepsilon}{a}\cdot\frac{OPT}{n}}$
$\displaystyle\Longrightarrow$ $\displaystyle\min_{\hat{\mu}_{i}\in
succ(\hat{\mu}_{j})}\|\hat{\mu}_{j}-\hat{\mu}_{i}\|\leq
2\sqrt{\frac{\varepsilon}{a}\cdot\frac{OPT}{n}}.$

Therefore

$\displaystyle\|\hat{\mu}_{j}-\mu\|$ $\displaystyle\leq\min_{\hat{\mu}_{i}\in
succ(\hat{\mu}_{j})}\|\hat{\mu}_{j}-\hat{\mu}_{i}\|+\|\hat{\mu}_{i}-\mu\|\leq
2\sqrt{\frac{\varepsilon}{a}\cdot\frac{OPT}{n}}+\sqrt{\frac{\varepsilon}{a}\cdot\frac{OPT}{n}}\leq\sqrt{9\cdot\frac{\varepsilon}{a}\cdot\frac{OPT}{n}}.$

Setting $a=9$ and using Lemma 2.1 then implies that $\mu_{j}$ is a
$(1+\varepsilon)$-approximation.

Finally, we give a brief remark on the running time. The time to compute the
$\hat{\mu}_{i}$ is $O(\varepsilon^{-1}\log\delta^{-1})$. The time to compute
$D_{i}$ is slightly more non-trivial. We query the distances between all
estimators in time $O(\log^{2}\delta^{-1})$. For each $\hat{\mu}_{i}$, we find
the $2/3$-farthest point via any fast rank selection algorithm, see for
example [5, Chapter 9.3], which can be implemented to run in time
$O(\log\delta^{-1})$. Following this, any single $D_{i}$ can be computed in
time $O(\log\delta^{-1})$, which amounts to $O(\log^{2}\delta^{-1})$ for all
$D_{i}$. ∎

##  4 The High Probability Lower Bound

The lower bound is largely an adaptation of the lower bound from [4], but
slightly refined to account for the high probability statement.

######  Theorem 4.1.

Any sublinear algorithm sampling that outputs a $(1+\varepsilon)$ approximate
Euclidean mean with probability at least $1-\delta$ must sample at least
$\Omega(\varepsilon^{-1}\log\delta^{-1})$ many points.

###### Proof.

We give two instances. The first instance places $n$ points at the $0$ and
$\varepsilon\cdot n$ points at $1$. Thus, the optimal mean is placed at
$\frac{\varepsilon}{(1+\varepsilon)}$. A routing calculation shows that the
optimal cost is $\frac{\varepsilon\cdot n}{1+\varepsilon}$, conversely
$\frac{OPT}{n}=\frac{\varepsilon}{1+\varepsilon}$. The second instance places
all points at $0$.

First we observe that a sublinear algorithm can distinguish between these two
instances as any approximate mean for the second instance must output $0$. If
we output $0$ for the first instance, we incur a cost of $\varepsilon\cdot
n=\frac{\varepsilon\cdot n(1+\varepsilon)}{1+\varepsilon}=OPT+\varepsilon\cdot
OPT$, hence any algorithm improving over a $(1+\varepsilon)$ approximation for
the former cannot output $0$. Thus, a necessary condition to distinguish
between the two instances is for the algorithm to sample at least one point at
$1$.

Suppose we sample $m$ points. Our goal is to show that for
$m\in\Omega(\varepsilon^{-1}\log\delta^{-1})$, the probability that all of the
sampled points are drawn from $0$ is less than $\delta$ for the first
instance. Let $X_{i}$ denote the indicator variable of the $i$th sampled
point. We have

$\displaystyle\delta\geq\mathbb{P}[\forall
X_{i}=0]=\mathbb{P}[X_{1}=0]^{m}=\left(\frac{1}{1+\varepsilon}\right)^{m}=e^{m\ln\frac{1}{1+\varepsilon}}=e^{-m\ln(1+\varepsilon)}>e^{-m\cdot\varepsilon/2},$

where the final inequality follows from the Mercator series
$\ln(1+\varepsilon)=\sum_{i=1}^{\infty}\left(\frac{\varepsilon^{i}}{i}\right)\cdot(-1)^{i+1}$.
Thus for any $m\leq\varepsilon/2\cdot\log\delta^{-1}$, we have the desired
failure probability. Conversely, $m\in\Omega(\varepsilon^{-1}\log\delta^{-1})$
for an algorithm to succeed. ∎

## Acknowledgements

Matteo Russo are partially supported by the ERC Advanced Grant 788893 AMDROMA
“Algorithmic and Mechanism Design Research in Online Markets”. Chris
Schwiegelshohn is supported by the Independent Research Fund Denmark (DFF)
under a Sapere Aude Research Leader grant No 1051-00106B.

## References

  * Chen and Derezinski [2021] X. Chen and M. Derezinski.  Query complexity of least absolute deviation regression via robust uniform convergence.  In M. Belkin and S. Kpotufe, editors, _Conference on Learning Theory, COLT 2021, 15-19 August 2021, Boulder, Colorado, USA_ , volume 134 of _Proceedings of Machine Learning Research_ , pages 1144–1179. PMLR, 2021.  URL http://proceedings.mlr.press/v134/chen21d.html.
  * Clarkson et al. [2012] K. L. Clarkson, E. Hazan, and D. P. Woodruff.  Sublinear optimization for machine learning.  _J. ACM_ , 59(5):23:1–23:49, 2012.  doi:10.1145/2371656.2371658.  URL https://doi.org/10.1145/2371656.2371658.
  * Cohen et al. [2016] M. B. Cohen, Y. T. Lee, G. L. Miller, J. Pachocki, and A. Sidford.  Geometric median in nearly linear time.  In D. Wichs and Y. Mansour, editors, _Proceedings of the 48th Annual ACM SIGACT Symposium on Theory of Computing, STOC 2016, Cambridge, MA, USA, June 18-21, 2016_ , pages 9–21. ACM, 2016.  doi:10.1145/2897518.2897647.  URL https://doi.org/10.1145/2897518.2897647.
  * Cohen-Addad et al. [2021] V. Cohen-Addad, D. Saulpic, and C. Schwiegelshohn.  Improved coresets and sublinear algorithms for power means in euclidean spaces.  In M. Ranzato, A. Beygelzimer, Y. N. Dauphin, P. Liang, and J. W. Vaughan, editors, _Advances in Neural Information Processing Systems 34: Annual Conference on Neural Information Processing Systems 2021, NeurIPS 2021, December 6-14, 2021, virtual_ , pages 21085–21098, 2021.  URL https://proceedings.neurips.cc/paper/2021/hash/b035d6563a2adac9f822940c145263ce-Abstract.html.
  * Cormen et al. [2009] T. H. Cormen, C. E. Leiserson, R. L. Rivest, and C. Stein.  _Introduction to Algorithms, 3rd Edition_.  MIT Press, 2009.
  * Ding [2020] H. Ding.  A sub-linear time framework for geometric optimization with outliers in high dimensions.  In _28th Annual European Symposium on Algorithms, ESA 2020, September 7-9, 2020, Pisa, Italy (Virtual Conference)_ , pages 38:1–38:21, 2020.  doi:10.4230/LIPIcs.ESA.2020.38.  URL https://doi.org/10.4230/LIPIcs.ESA.2020.38.
  * Ding [2021] H. Ding.  Stability yields sublinear time algorithms for geometric optimization in machine learning.  In P. Mutzel, R. Pagh, and G. Herman, editors, _29th Annual European Symposium on Algorithms, ESA 2021, September 6-8, 2021, Lisbon, Portugal (Virtual Conference)_ , volume 204 of _LIPIcs_ , pages 38:1–38:19. Schloss Dagstuhl - Leibniz-Zentrum für Informatik, 2021.  doi:10.4230/LIPICS.ESA.2021.38.  URL https://doi.org/10.4230/LIPIcs.ESA.2021.38.
  * Inaba et al. [1994] M. Inaba, N. Katoh, and H. Imai.  Applications of weighted voronoi diagrams and randomization to variance-based _k_ -clustering (extended abstract).  In _Proceedings of the Tenth Annual Symposium on Computational Geometry, Stony Brook, New York, USA, June 6-8, 1994_ , pages 332–339, 1994.  doi:10.1145/177424.178042.  URL https://doi.org/10.1145/177424.178042.
  * Lee and Valiant [2021] J. C. H. Lee and P. Valiant.  Optimal sub-gaussian mean estimation in $\mathbb{R}$.  In _62nd IEEE Annual Symposium on Foundations of Computer Science, FOCS 2021, Denver, CO, USA, February 7-10, 2022_ , pages 672–683. IEEE, 2021.  doi:10.1109/FOCS52979.2021.00071.  URL https://doi.org/10.1109/FOCS52979.2021.00071.
  * Lugosi and Mendelson [2019] G. Lugosi and S. Mendelson.  Mean estimation and regression under heavy-tailed distributions: A survey.  _Found. Comput. Math._ , 19(5):1145–1190, 2019.  doi:10.1007/s10208-019-09427-x.  URL https://doi.org/10.1007/s10208-019-09427-x.
  * Musco et al. [2022] C. Musco, C. Musco, D. P. Woodruff, and T. Yasuda.  Active linear regression for $\ell_{p}$ norms and beyond.  In _2022 IEEE 63rd Annual Symposium on Foundations of Computer Science (FOCS)_ , pages 744–753, 2022.  doi:10.1109/FOCS54457.2022.00076.
  * Parulekar et al. [2021] A. Parulekar, A. Parulekar, and E. Price.  L1 Regression with Lewis Weights Subsampling.  In M. Wootters and L. Sanità, editors, _Approximation, Randomization, and Combinatorial Optimization. Algorithms and Techniques (APPROX/RANDOM 2021)_ , volume 207 of _Leibniz International Proceedings in Informatics (LIPIcs)_ , pages 49:1–49:21, Dagstuhl, Germany, 2021. Schloss Dagstuhl – Leibniz-Zentrum für Informatik.  ISBN 978-3-95977-207-5.  doi:10.4230/LIPIcs.APPROX/RANDOM.2021.49.  URL https://drops.dagstuhl.de/entities/document/10.4230/LIPIcs.APPROX/RANDOM.2021.49.
  * Woodruff and Yasuda [2024] D. P. Woodruff and T. Yasuda.  Coresets for multiple $\ell_{p}$ regression, 2024.


# Learning to Abstract Visuomotor Mappings using Meta-Reinforcement Learning

Carlos A. Velazquez-Vargas 1 , Isaac Ray Christian 1, Jordan A. Taylor 1,2,
Sreejan Kumar2,3
1 Princeton University Dept of Psychology
2 Princeton Neuroscience Institute
3 NYU Center for Data Science

###### Abstract

We investigated the human capacity to acquire multiple visuomotor mappings for
de novo skills. Using a grid navigation paradigm, we tested whether contextual
cues implemented as different ”grid worlds”, allow participants to learn two
distinct key-mappings more efficiently. Our results indicate that when
contextual information is provided, task performance is significantly better.
The same held true for meta-reinforcement learning agents that differed in
whether or not they receive contextual information when performing the task.
We evaluated their accuracy in predicting human performance in the task and
analyzed their internal representations. The results indicate that contextual
cues allow the formation of separate representations in space and time when
using different visuomotor mappings, whereas the absence of them favors
sharing one representation. While both strategies can allow learning of
multiple visuomotor mappings, we showed contextual cues provide a
computational advantage in terms of how many mappings can be learned.

Keywords: motor learning, context learning, meta-learning, reinforcement
learning, navigation

## Introduction

There has been considerable interest in determining how contextual cues allow
the consolidation and retrieval of multiple visuomotor memories Howard .
(2013); Heald . (2018); Heald, Lengyel Wolpert (2023). While it has been shown
that arbitrary external contextual-cues – such as colors, sounds or shapes –
are effective in separating the task contingencies in a variety of domains
such as in classical conditioning Gershman (2017), episodic memory Pu . (2022)
and value-based decision making Bornstein Norman (2017), the same cues cannot
prevent catastrophic interference in visuomotor adaptation tasks Howard .
(2013) unless they are presented in close temporal proximity Avraham . (2022).

In contrast with these laboratory-based findings, we appear capable of storing
multiple mappings when using a variety of digital devices despite having
similar movements (e.g., video games use the same controller to play a racing
game and a first-person shooter). One interpretation of the above, is that a
great proportion of these mappings may not be the result of visuomotor
adaptation but of instead of de novo skill learning. When a skill is acquired
de novo, new control policies are created, rather than recalibrating existing
ones Krakauer . (2019).

In a typical de novo task, participants are required to learn arbitrary, and
usually non-intuitive associations between their movements and the outcomes to
achieve the task goals Wilterson Taylor (2021); Mosier . (2005); Velazquez-
Vargas Taylor (2023). Crucially, these tasks are known to involve brain
regions associated with declarative processes such as the hippocampus Wise
Murray (2000) and the prefrontal cortex AS. Fermin . (2016), also crucial for
context-dependent learning Heald, Wolpert Lengyel (2023). This important
distinction could allow de novo skill learning to be sensitive to contextual
cues that are efficient in domains such as episodic memory, unlike visuomotor
adaptation tasks which are known to involve a significant cerebellar-dependent
component Taylor . (2010)

Meta-learning Wang . (2018) can be a unifying computational framework that
formalizes de novo skill learning. In meta-learning, an agent (human or
artificial) has to learn a distribution of different but related tasks. For de
novo skill learning, the different tasks can use the same motor repertoire but
may map each motor movement to a different outcome. Within this framework, a
meta-learning agent can either learn a single abstract representation across
all tasks that accommodates all motor mappings or learn separate
representations for all the unique motor mappings seen across tasks, where
each representation can be bound to a particular external contextual cue
Musslick Cohen (2021). Meta-learning models are becoming increasingly more
relevant in cognitive modeling of behavioral phenomenon due to their ability
to implement Bayes-optimal learning algorithms on tasks for which Bayesian
inference is intractableBinz . (2023).

In the present study, we designed a grid navigation task to dissociate these
two hypotheses. One group of participants (the context group) performed a grid
navigation task A. Fermin . (2010); AS. Fermin . (2016); Velazquez-Vargas .
(2023) where they moved a cursor from start to target locations in two
“worlds” randomly interleaved over trials (Figure 1). Each grid-world was
associated with a unique key-mapping and had distinctive contextual cues. We
compared performance on this group with another group of participants (no-
context group) that experienced the trial-changes in key-mappings but not in
the external contextual cues –i.e., grid worlds.

To implement cognitive models for learning to navigate with different
visuomotor mappings, we trained two recurrent-based meta-learning agents using
the architecture from wang2018prefrontal, which is an LSTM agent trained with
reinforcement learning using an Actor-Critic framework Mnih . (2016). The
first model (context LSTM) incorporated external contextual information into
its input while the second one did not (no-context LSTM). We tested their
performance in the task and how well they predicted human participants’ data.
In addition, we examined how their internal representations gave rise to
different strategies to learn multiple key-mappings, giving us potential
insights and hypotheses for the neural representations of the participants.

We showed that humans and the meta-learners performed better in the task when
provided with contextual information. We also found that, in both conditions,
there is individual variability as to whether an individual acts more like the
context LSTM or the no context LSTM. Additionally, we found that when no
contextual information is provided, the internal representations of the LSTM
agent are highly correlated in space and time while using the different key-
mappings. However, these representations are less correlated when the LSTM
agent that has the external context input. This suggests that the former is
learning separate representations for different motor mappings whereas the
latter is binding different motor mapppings to the same representation.
Finally, we showed that the capacity to learn multiple key-mappings is
dependent not only on the presence of contextual cues, but also on the
complexity (number of hidden LSTM units) of the internal representations of
the model. Our model-based neural analyses and human behavior results give us
potential insight on how the brain can effortlessly learn multiple visuomotor
mappings.

## Methods

### Participants

Thirty two participants from Princeton University (13 males and 19 females,
mean age = 21.7, sd = 3.8) were recruited through the psychology subject pool.
The study was approved by the Institutional Review Board and all subjects
provided informed consent prior to doing the experiment.

### Task

The experimental task was programmed using HTML/CSS/Javascript and hosted on
Google Firebase. Visual stimuli were displayed on a 60 Hz Dell monitor using a
Dell OptiPlex 7050’a machine (Dell, Round Rock, Texas) running Windows 10
(Microsoft Co., Redmond, Washington). Participants’ responses were recorded
using a standard desktop keyboard. Before the start of the experiment, the
instructions were displayed on the screen.

The goal of the task was to move a cursor from start to target locations in a
$9\times 9$ grid environment using the least possible number of moves A.
Fermin . (2010); AS. Fermin . (2016); Velazquez-Vargas . (2023); Bera .
(2021). To do so, participants used three keyboard keys (J, F and K) that
moved the cursor to arbitrary and unknown adjacent locations (key-mapping):
left-up, left-down and right (Figure 1). The task consisted of 360 trials
where the start-target locations were randomly sampled out of eight possible
pairs. For each pair, the target was placed seven moves away from the starting
location of the cursor. If participants arrived at the target using the
minimum number of moves (optimal arrival), they would observe a happy emoji
face at the target location and hear a pleasant sound. If they arrived at the
target but not using the minimum number of moves (arrival), they would observe
a neutral face and hear a neutral sound. If they did not arrive at the target
in less than 10 s, the trial was terminated and participants would observe a
sad face and hear an unpleasant sound.

Importantly for this study, participants performed the tasks using two key-
mappings, which corresponded to different arrangements of the same moving
directions (see Figure 1), and which were randomly interleaved across trials.
Participants were not informed that there would be more than one key-mapping
to perform the task. In order to test the effectiveness of external contextual
cues in separating the learning of the visuomotor mappings, we split
participants into two experimental groups.

Figure 1: Experimental task Subject perform a grid navigation using different
key mappings randomly interleaved over trials. In the context group, the key-
mappings where deterministically signalled by a unique grid world, whereas in
the no-context group, participants used both mappings in the same world.

Context Group: In this group (n =16) participants performed the grid
navigation task in two different “worlds”, either in the ocean or in a farm.
Each world consisted of unique visual (background, cursor shape) and auditory
(ambience song and cursor sound when moving) information. Most importantly,
each world was deterministically associated with one of the key-mappings.

No-context Group: Participants in this group (n=16) performed the grid
navigation task in a single world while still experiencing the changes in the
key-mappings. At the beginning of the experiment, one of the worlds would be
selected and remained throughout the whole task.

### Models

Following previous work on meta reinforcement learning Wang . (2018); Kumar .
(2020), we used a Long short-term memory (LSTM) network trained using
Advantage Actor Critic (A2C; mnih2016asynchronous. The recurrent structure of
this agent, joint with a model-free learning mechanism, makes it suitable for
sequential tasks like ours, where trial-and-error, reward-based processes are
crucial to achieve the goals. The agents were implemented using Stable
baselines 3 package Raffin . (2019) with the Proximal Policy Optimization
algorithm (PPO). We set constraints of the hyperparameters of PPO so that the
algorithm becomes equivalent to A2C Huang . (2022). We used Optuna Akiba .
(2019) to tune the following hyperparameters: learning rate, discount factor
gamma, GAE lambda, number of steps, batch size, entropy coefficient, the value
function coefficient and the number of LSTM hidden units. After optimization,
we trained the agents for $2\times 10^{5}$ time steps, where they reached
asymptotic performance. The reward structure was defined as follows: every
move that did not lead to a terminal state (arrival to the goal) received
$-1$, except when colliding with the grid walls, in which case the agent
received a penalty of $-100$. If the agent reached the goal, it received a
reward of $+10$.

We trained two types of agents that differ chiefly in the input provided to
the LSTM network. The first one (context LSTM) received its current location
and target locations on the grid as well as a context vector indicating which
of the worlds was currently being observed. This agent faced the same
situation as participants in the context group where the key-mapping changes
were linked to changes in the contextual cues. On the other hand, the second
agent (no context LSTM) only received its current location and the target
location as input, mirroring the desing of the no-context group.

## Results

Figure 2: Performance of Humans and Meta-RL Agents (A). Mean performance of
humans over the episodes they completed, measured by proportion of optimal
arrivals. Shading represents 95% confidence intervals. Humans that were given
context cues learned the task better. (B). Overall performance differences
across different experimental groups in humans. Those in the context group did
significantly better. (C). Mean reward over time during agent learning for
both context and no-context agents. (D). Median reward after training. Agents
that were given a external contextual cue for the mapping as additional input
did significantly better. Figure 3: In both experimental groups, humans are
split between behaving like the context vs no context LSTM (A). Joint
scatterplot and histograms for likelihood of subjects’ actions under the
context LSTM model vs. performance of subjects on the task. In both
experimental groups (whether participants received external context input or
not), there are two clusters of participants — those whose actions are
explained well by the context LSTM model and those whose are not. (B). If we
examine the mean likelihood under both types of models (context vs no context
LSTMs), we see that participants whose actions aren’t as well explained by the
context LSTM model have significantly higher likelihood under the no context
LSTM model.

### Human and Model Behavior

As a metric of human performance in the context and no-context groups, we
provide the proportion of times they arrive at the target optimally – i.e.,
using the minimum number of key presses– across trials and in the entire task.
Based on the optimal arrivals to the target, we observed that participants in
both the context and no-context groups improved their performance over trials
(Figure 2). However, the context group performed significantly better overall
($p=0.03$). For the context LSTM and no-context LSTM agents, we provide the
average reward per episode at different evaluation points across training, as
well as at the end of training. The context LSTM and the no-context LSTM
agents showed the same pattern as participants, where both models improved
over the timesteps but the former performing overall better in the task
($p<0.001$).

To identify the similarity between subjects’ behavior and the context LSTM and
no-context LSTM agents, we calculated the likelihood of subjects’ responses
according to the fully trained agents. That is, we performed a forward pass on
the agents and obtained the likelihood that they would take the same action as
the human. The likelihood for each action was averaged across all timesteps
for each subject to produce one value indicating the average likelihood of a
subject’s response under the model. This analysis was done for both the
context and the no-context LSTM agents.

Figure 3A shows the average response likelihood for both experimental
conditions under the context LSTM agent, where two clear clusters were formed.
In Cluster 1, the context-LSTM agent was considerably better in predicting
participants’ responses than in Cluster 2, where it performed around chance
level. Following this analysis, we found that participants that were poorly
predicted by the context LSTM (Cluster 2) agent were instead significantly
better predicted by the no-context LSTM ($p<0.05$; Figure 3B). Likewise, for
participants in Cluster 1, the context LSTM agent better predicted
participants’ responses compared to the no-context LSTM agent.

These results indicate that regardless of the experimental group, some
participants where better predicted by the context model, suggesting that even
in the absence of contextual cues from the experiment, participants could have
relied on alternative cues to separate the representation under the different
key-mappings. A point we we will address in the Discussion section.

### Model Representational Similarity Analysis

Figure 4: Context agents have lower representational similarity under the
different key-mappings over space and time (A). Spatial RSA analysis. We
correlated LSTM hidden state representations of the same episodes under the
different key-mappings and showed the mean correlation across different
spatial locations in the $9\times 9$ grid. For context agents, this
correlation is much lower, presumably because the representations of different
key-mappings are more separated. (B). Temporal RSA analysis. We show how the
correlation changes over time by plotting, for each timepoint, the mean
correlation of LSTM hidden state representations under the different key-
mappings. For context agents, this correlation goes down over time whereas for
no-context agents, there is less change overtime. This suggests context agents
exhibit more dissimilar representational similarity under different key-
mappings overtime.

We employed multiple analyses (Figure 4) to examine differences in
representations between the cue and no-cue agent. Specifically, we first
investigated how each network represents the same trial and environment under
different motor contexts. To do this, we performed a Representational
Similarity Analysis (RSA; kriegeskorte2008representational on both a spatial
and temporal scale.

For the spatial RSA (Figure 4A), we evaluated the agents in $10^{4}$ episodes
and computed the average hidden state (LSTM units) at each location in the
$9\times 9$ grid, separating the hidden states according to the key-mapping
being used. For every grid state, we obtained the correlation coefficient
between the averaged hidden states when using key-mapping A and the averaged
hidden states when using key-mapping B. Lower correlations would indicate that
agents represent the grid space between different action mappings more
distinctively. In the context LSTM agent, the correlations were low across
most grid states, with many locations on the grid showing no correlation,
suggesting that spatial location is represented differently when using mapping
A and mapping B. In contrast, for the no-context LSTM agent, the RSA generally
depicts grid states with correlations that are significantly higher
($p<0.001$), when the different key-mappings are being used compared to the
context LSTM agent. The difference between the two suggests that the context
LSTM agent has a distinct representation for grid states that depends on its
current context, while the no-context LSTM model does not.

The temporal RSA (Figure 4B) provides complementary results showing that
context is represented differently between models not just in space, but also
in time. Specifically, we explored the similarity of hidden states of the
agents as they progressed towards the target. To do so, we evaluated the
agents in $10^{4}$ episodes and for every timestep we correlated the averaged
hidden states when using key-mapping A with the averaged hidden states when
using key-mapping B. In the no context LSTM agent, the representations on the
initial timestep are highly correlated. As the agent steps towards the goal,
representations become slightly less correlated, but for the most part remain
highly correlated across the episode. In the context LSTM agent, however,
representations under the different key-mappings drastically change with each
step in the episode. Representations begin similar on the first time step
(although less so compared to the no-context LSTM agent), but quickly drop to
no correlation at around the 6th step in the episode. This suggests that the
agent representing context increasingly leverages its context representation
as it nears the goal state.

Taken together, these results show two different representational strategies,
one where the agent attempts to learn distinct representations for each
context, and another where the agent learns the task without an explicit
representation of context.

### Model Representational Capacity Analysis

Figure 5: For context agents, scaling the LSTM capacity enables learning more
mappings For both context and non-context agents, we varied the number of
hidden units and exposed them to different number of mappings during training.
For context agents, increasing the number of LSTM units allows for learning of
more mappings until about 5. For no-context agents, varying the number of LSTM
units does not have as strong an effect.

Given the capacity limitations in memory, computational resources and time, we
would expect that humans exhibit constraints in the number of visuomotor
mappings they can learn. To explore this idea, we investigated the agents’
capacity to learn multiple key-mappings by varying the complexity of their
internal structure, i.e. number of hidden units in the LSTM. In order to do so
we exposed both the context LSTM and no-context LSTM agents to up to ten
different key-mappings and using the following number of hidden units:
$2,5,10,20,40,80,100,120,256$ or $512$. Each agent was trained for $200$k
timesteps followed by $100$ evaluation episodes from which we report the
averaged (normalized) reward.

Figure 5 displays performance of both models as the number of contexts and
representational capacity increases. Across both models, we see better
performance with more representational capacity. Between models, the model
that explicitly receives contextual cues performs better as contexts increase
than does the model that does not receive a cue. This is perhaps due to the
way in which context is represented between both models. In the context model,
each context is distinctly represented within the hidden layer, allowing for a
unique task representation for each context. In contrast, the no context model
does not distinctly represent each context. This supports work showing that
unique task representations are necessary as the environment becomes more
complex (i.e. when the number of contexts increase) and adds that explicit
cues aid in both the speed at which the task is learned and how it is
represented.

## Discussion

Humans posses a remarkable capacity to learn complex, and often arbitrary,
visuomotor mappings. For example, when learning to play video games humans are
able link the button presses from the controller with arbitrary actions in the
game. What is more, not only are they capable of learning such arbitrary
mappings and use them to achieve goals, but they can build an entire
repertoire of them. It is not uncommon to witness players seamlessly
transitioning between mappings for distinct gaming genres, be it from a first-
person shooter to a racing or football game.

In the current work, we have shown that the acquisition of distinct visuomotor
associations can be aided by external contextual cues. In particular,
participants’ that are provided with contextual information that deferentially
cues the key-mappings, performed significantly better than participants
without it. These results are supported by previous findings outside the motor
domain where external contextual cues allow subjects to discriminate between
different contingencies of the task Gershman (2017); Bornstein Norman (2017).
Similar cues, however, do not prevent catastrophic interference in visuomotor
adaptation Howard . (2013), except under specific circumstances Avraham .
(2022). We believe this effect arises due to the fundamental distinction
between adaptation and de novo learning, where the former recalibrates an
existing control policy and the second one builds one from scratch. This can
generate qualitative differences between the contextual cues the system
supporting each process are sensitive to. On the one hand, cerebellar
dependent recalibration processes could be more easily influenced by context
related to the body states or the world kinematics. However, de novo learning,
particularly in the domain of spatial navigation likely involving the
hippocampus and the prefrontal cortex AS. Fermin . (2016), could be more
sensitive to visuospatial contextual cues.

It is important to emphasize that while the context group outperformed the no-
context group, the latter still improved its performance over trials, arriving
optimally a greater proportion of times toward the end of the experiment
($p<0.05$). This improvement occurred in the absence of external contextual
cues, which could aid in differentiating the key-mappings. We believe this
result could be attributed to participant relying on a different set of cues:
movement-related ones. In particular, given the specific movement directions
that we used for the key-mappings (left-up, left-down and back), it is
possible to arrive at the target locations with multiple and equivalently
optimal trajectories which begin with a different action. For example, for a
target that is right above the cursor, pressing the key that moves right, or
left-up, would leave the cursor one move away from the target. Although this
move makes no difference in terms of performance, it does provide information
about the ongoing mapping, given that no key moves to the cursor to the same
direction. Therefore, participants could have used the first move of the
cursor to sample the ongoing key-mapping and adapt the subsequent moves
accordingly. In effect, this strategy would have provided participants with
contextual information similar to the context group and would have been able
to learn the two mappings separately. This is consistent with the fact that
some participants in the no-context group had a higher likelihood under the
context LSTM than the no context LSTM (Figure 3A). Alternatively, participants
could have learned a single of the two key-mappings, or an averaged version of
them, which over time would have allowed them to improve in the task, although
not to the same extent as the context group. Participants relying on the
latter strategies would be learning, in effect, a single key-mapping. Further
analysis would be needed to test these hypotheses.

In order to have a behavioral model for the presence or absence of contextual
cues while acquiring multiple mappings, while also providing insight into the
neural representations, we leveraged recent developments on meta-reinforcement
learning Wang . (2018); Huang . (2022); Kumar . (2020). Mirroring human
performance, we found that an LSTM agent that explicitly receives information
about its context outperforms an LSTM agent that does not. Moreover, we found
that participants that were poorly described by the context LSTM, either in
the context or no-context groups, were instead best described by an LSTM model
with no contextual information. Through implementing these models, we were
able to find that within both experimental groups, people vary in either
producing behavior consistent with a model learning separate vs shared
representations across different mappings (Fig. 3).

Based on the spatial and temporal RSA, we would expect to see differences
within neural representations of participants relying on contextual
information (either from the experiment or potentially from their own actions)
vs. individuals that don’t. However, due to the fact that the no context LSTM
explained some participants’ behavior better than the context LSTM (Figure 3),
it is conceivable that some participants held a single representation in the
presence of contextual cues, while others held different representations in
the absence of them. This may beg the question — why would a participant
choose to hold a single representation even when given contextual cues? The
fact that the number of contexts learned scales with the capacity of the model
(Figure 5) suggests that learning these separate representations uses more
cognitive resources. Subjects choosing to not learn separate representations
may be behaving rationally according to their cognitive resources Lieder
Griffiths (2020). From our capacity analysis of the networks in Figure 5, we
predict that these participants who don’t learn separate representations may
take longer to learn the same number of mappings than participants who take
advantage of the contextual cues to learn separate representations. Confirming
these predictions with brain imaging, and generally studying how participants
share vs. separate neural representations of visuomotor mappings to manage
time and computational resources, will be a rich line of future work.

## Acknowledgments

Carlos Velazquez-Vargas and Jordan Taylor were supported by the J. Insley
Blair Pyne Fund, Office of Naval Research, Cognitive Science Program and
Research Innovation Fund for New Ideas in the Natural Sciences at Princeton
University. Sreejan Kumar is supported by a Google PhD Fellowship.

## References

  * Akiba . (2019) akiba2019optunaAkiba, T., Sano, S., Yanase, T., Ohta, T. Koyama, M.  2019\.  Optuna: A next-generation hyperparameter optimization framework Optuna: A next-generation hyperparameter optimization framework.  Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining Proceedings of the 25th acm sigkdd international conference on knowledge discovery & data mining ( 2623–2631).
  * Avraham . (2022) avraham2022contextualAvraham, G., Taylor, JA., Breska, A., Ivry, RB. McDougle, SD.  2022\.  Contextual effects in sensorimotor adaptation adhere to associative learning rules Contextual effects in sensorimotor adaptation adhere to associative learning rules.  Elife11e75801.
  * Bera . (2021) bera2021motorBera, K., Shukla, A. Bapi, RS.  2021\.  Motor chunking in internally guided sequencing Motor chunking in internally guided sequencing.  Brain Sciences113292.
  * Binz . (2023) binz2023metaBinz, M., Dasgupta, I., Jagadish, AK., Botvinick, M., Wang, JX. Schulz, E.  2023\.  Meta-learned models of cognition Meta-learned models of cognition.  Behavioral and Brain Sciences1–38.
  * Bornstein Norman (2017) bornstein2017reinstatedBornstein, AM. Norman, KA.  2017\.  Reinstated episodic context guides sampling-based decisions for reward Reinstated episodic context guides sampling-based decisions for reward.  Nature neuroscience207997–1003.
  * A. Fermin . (2010) fermin2010evidenceFermin, A., Yoshida, T., Ito, M., Yoshimoto, J. Doya, K.  2010\.  Evidence for model-based action planning in a sequential finger movement task Evidence for model-based action planning in a sequential finger movement task.  Journal of motor behavior426371–379.
  * AS. Fermin . (2016) fermin2016modelFermin, AS., Yoshida, T., Yoshimoto, J., Ito, M., Tanaka, SC. Doya, K.  2016\.  Model-based action planning involves cortico-cerebellar and basal ganglia networks Model-based action planning involves cortico-cerebellar and basal ganglia networks.  Scientific reports6131378.
  * Gershman (2017) gershman2017contextGershman, SJ.  2017\.  Context-dependent learning and causal structure Context-dependent learning and causal structure.  Psychonomic bulletin & review24557–565.
  * Heald . (2018) heald2018increasingHeald, JB., Franklin, DW. Wolpert, DM.  2018\.  Increasing muscle co-contraction speeds up internal model acquisition during dynamic motor learning Increasing muscle co-contraction speeds up internal model acquisition during dynamic motor learning.  Scientific reports8116355.
  * Heald, Lengyel Wolpert (2023) heald2023contextualHeald, JB., Lengyel, M. Wolpert, DM.  2023\.  Contextual inference in learning and memory Contextual inference in learning and memory.  Trends in Cognitive Sciences.
  * Heald, Wolpert Lengyel (2023) heald2023computationalHeald, JB., Wolpert, DM. Lengyel, M.  2023\.  The Computational and Neural Bases of Context-Dependent Learning The computational and neural bases of context-dependent learning.  Annual Review of Neuroscience46.
  * Howard . (2013) howard2013effectHoward, IS., Wolpert, DM. Franklin, DW.  2013\.  The effect of contextual cues on the encoding of motor memories The effect of contextual cues on the encoding of motor memories.  Journal of neurophysiology109102632–2644.
  * Huang . (2022) huang2022a2cHuang, S., Kanervisto, A., Raffin, A., Wang, W., Ontañón, S. Dossa, RFJ.  2022\.  A2C is a special case of PPO A2c is a special case of ppo.  arXiv preprint arXiv:2205.09123.
  * Krakauer . (2019) krakauer2019motorKrakauer, JW., Hadjiosif, AM., Xu, J., Wong, AL. Haith, AM.  2019\.  Motor learning Motor learning.  Compr Physiol92613–663.
  * Kriegeskorte . (2008) kriegeskorte2008representationalKriegeskorte, N., Mur, M. Bandettini, PA.  2008\.  Representational similarity analysis-connecting the branches of systems neuroscience Representational similarity analysis-connecting the branches of systems neuroscience.  Frontiers in systems neuroscience4.
  * Kumar . (2020) kumar2020metaKumar, S., Dasgupta, I., Cohen, JD., Daw, ND. Griffiths, TL.  2020\.  Meta-learning of structured task distributions in humans and machines Meta-learning of structured task distributions in humans and machines.  arXiv preprint arXiv:2010.02317.
  * Lieder Griffiths (2020) lieder2020resourceLieder, F. Griffiths, TL.  2020\.  Resource-rational analysis: Understanding human cognition as the optimal use of limited computational resources Resource-rational analysis: Understanding human cognition as the optimal use of limited computational resources.  Behavioral and brain sciences43e1.
  * Mnih . (2016) mnih2016asynchronousMnih, V., Badia, AP., Mirza, M., Graves, A., Lillicrap, T., Harley, T.Kavukcuoglu, K.  2016\.  Asynchronous methods for deep reinforcement learning Asynchronous methods for deep reinforcement learning.  International conference on machine learning International conference on machine learning ( 1928–1937).
  * Mosier . (2005) mosier2005remappingMosier, KM., Scheidt, RA., Acosta, S. Mussa-Ivaldi, FA.  2005\.  Remapping hand movements in a novel geometrical environment Remapping hand movements in a novel geometrical environment.  Journal of neurophysiology9464362–4372.
  * Musslick Cohen (2021) musslick2021rationalizingMusslick, S. Cohen, JD.  2021\.  Rationalizing constraints on the capacity for cognitive control Rationalizing constraints on the capacity for cognitive control.  Trends in Cognitive Sciences259757–775.
  * Pu . (2022) pu2022eventPu, Y., Kong, XZ., Ranganath, C. Melloni, L.  2022\.  Event boundaries shape temporal organization of memory by resetting temporal context Event boundaries shape temporal organization of memory by resetting temporal context.  Nature communications131622.
  * Raffin . (2019) raffin2019stableRaffin, A., Hill, A., Ernestus, M., Gleave, A., Kanervisto, A. Dormann, N.  2019\.  Stable baselines3. Stable baselines3.
  * Taylor . (2010) taylor2010explicitTaylor, JA., Klemfuss, NM. Ivry, RB.  2010\.  An explicit strategy prevails when the cerebellum fails to compute movement errors An explicit strategy prevails when the cerebellum fails to compute movement errors.  The Cerebellum9580–586.
  * Velazquez-Vargas . (2023) velazquez2023learningVelazquez-Vargas, CA., Daw, ND. Taylor, JA.  2023\.  Learning generalizable visuomotor mappings for de novo skills Learning generalizable visuomotor mappings for de novo skills.  bioRxiv2023–07.
  * Velazquez-Vargas Taylor (2023) velazquez2023exploringVelazquez-Vargas, CA. Taylor, J.  2023\.  Exploring human learning and planning inn grid navigation with arbitrary mappings. Exploring human learning and planning inn grid navigation with arbitrary mappings.  Proceedings of the Annual Meeting of the Cognitive Science Society Proceedings of the annual meeting of the cognitive science society ( 45).
  * Wang . (2018) wang2018prefrontalWang, JX., Kurth-Nelson, Z., Kumaran, D., Tirumala, D., Soyer, H., Leibo, JZ.Botvinick, M.  2018\.  Prefrontal cortex as a meta-reinforcement learning system Prefrontal cortex as a meta-reinforcement learning system.  Nature neuroscience216860–868.
  * Wilterson Taylor (2021) wilterson2021implicitWilterson, SA. Taylor, JA.  2021\.  Implicit visuomotor adaptation remains limited after several days of training Implicit visuomotor adaptation remains limited after several days of training.  ENeuro84.
  * Wise Murray (2000) wise2000arbitraryWise, SP. Murray, EA.  2000\.  Arbitrary associations between antecedents and actions Arbitrary associations between antecedents and actions.  Trends in neurosciences236271–276.


#  $k^{-3}$ superfluid spectrum of highly curved interacting quantum vortices

Jeffrey Yepez  Air Force Research Laboratory, Hanscom Air Force Base,
Massachusetts 01731, USA

###### Abstract

Presented is a prediction, based on the Frenet-Serret differential geometry of
space curves, that the wave number dependence of the average kinetic energy
per unit length of two mutually interacting highly curved quantum vortex
scales as $k^{-3}$. The interacting quantum vortices are helical in shape,
supporting circularly polarized counter-propagating waves, with arbitrary
curvature and torsion. This power-law spectrum agrees with the high-$k$
spectrum found in precise quantum simulations of turbulent superfluidity with
tangle of highly curved and excited quantum vortices.

BEC superfluid, Frenet-Serret formulas, mutually interacting vortices,
$k^{-3}$ helical wave spectrum

###### pacs:

67.25.dk,67.25.dt,67.85.De

Superfluid turbulence is an intriguing low-temperature phenomenon with power-
law energy cascades that are undergoing active investigation. In this Letter
we consider the origin of a $k^{-3}$ power-law in the kinetic energy spectrum
at high-$k$ wave numbers ($\gtrapprox$ the inverse coherence length)
associated with highly curved quantum vortices. The simplest theory for a
superfluid condensate in the zero-temperature limit is

${\cal L}_{\text{\tiny
BEC}}=i\hbar\,\varphi^{\ast}\partial_{t}\varphi+\frac{\hbar^{2}}{2m}(\nabla\varphi^{\ast})\cdot\nabla\varphi+\mu\,\varphi^{\ast}\varphi-\varphi^{\ast}V_{\text{\tiny
H}}\,\varphi,\qquad$ (1)

where $\varphi(x)$ is a complex scalar field for the Bose-Einstein condensate
(BEC), $V_{\text{\tiny H}}$ is a local self-consistent Hartree potential, and
$\mu$ is the chemical potential. Minimizing the action $\int d^{4}x\,{\cal
L}_{\text{\tiny BEC}}$ leads to the Euler-Lagrange equation, a nonlinear
Schroedinger equation when $V_{\text{\tiny
H}}(|\varphi|^{2})=\frac{1}{2}g|\varphi|^{2}$, where $g$ is the real-valued
coupling strength of the nonlinear interaction, known as the Gross-Pitaevskii
equation (GPE)Gross (1963); Pitaevskii (1961)
$i\hbar\partial_{t}\varphi=-\frac{\hbar^{2}}{2m}\nabla^{2}\varphi+(g|\varphi|^{2}-\mu)\,\varphi.$
It captures complex vortex-vortex interplay (nucleation, emission and
absorption of vortex rings, and reconnection) as well as quantum Kelvin wave
mode (kelvon) production on the vortices and sound modes (phonons) that escape
into the bulk region of the quantum fluid.

To determine a steady-state solution of the quantum vortex in a superfluid,
with a BEC wave function denoted by $\varphi_{\text{\tiny v}}$, one solves the
time-independent GPE: $-\xi^{2}\nabla^{2}\varphi_{\text{\tiny
v}}+\left(\frac{g}{\mu}\,|\varphi_{\text{\tiny
v}}|^{2}-1\right)\varphi_{\text{\tiny v}}=0$ with healing length
$\xi\equiv\hbar/\sqrt{2m\mu}$. A solution for the background condensate wave
function of a single rectilinear quantum vortex (with vorticity along
$\hat{\bm{z}}$) is found by separation of variables in polar coordinates.
Inserting $\varphi_{\text{\tiny v}}(r,\vartheta,z)=\phi_{\text{\tiny
v}}(r)Z_{\text{\tiny v}}(z)e^{in\vartheta}$ into time-independent GPE with
$g/\mu=\xi^{2}$ gives the following equations with a separation constant
$k_{\parallel}^{2}$:

$\frac{d^{2}Z_{\text{\tiny v}}(z)}{dx^{2}}+k_{\parallel}^{2}Z_{\text{\tiny
v}}(z)=0,\vspace{-.10in}$ (2a) $\frac{d^{2}\phi_{\text{\tiny
v}}(r)}{dr^{2}}+\frac{1}{r}\frac{d\phi_{\text{\tiny
v}}(r)}{dr}-\frac{n^{2}}{r^{2}}\phi_{\text{\tiny
v}}(r)+\left(a-\phi_{\text{\tiny v}}(r)^{2}\right)\phi_{\text{\tiny v}}(r)=0,$
(2b)

where $a\equiv\xi^{-2}-k_{\parallel}^{2}$. Equation (2a) admits sinusoidal
solutions and (2b) can be solved for integer winding number $n$. For the
simplest $n=1$ case, the Padé approximant $\phi_{\text{\tiny
v}}(r)=\sqrt{a}\sqrt{\frac{11ar^{2}(12+ar^{2})}{384+ar^{2}(128+11ar^{2})}}$
solves the time-independent GPE with errors at ${\cal
O}\left[(r\sqrt{a})^{7}\right]$. Notice that $\phi_{\text{\tiny
v}}(r)\rightarrow\sqrt{a}$ and $r\rightarrow\infty$, and thus the nonlinear
term in (2b) vanishes in the bulk.

The rectilinear quantum vortex solution of the GPE is $\varphi_{\text{\tiny
v}}(x)=\phi_{\text{\tiny
v}}(r)\left[Z_{\circ}^{+}e^{i(\omega_{\parallel}t+k_{\parallel}z)}+Z_{\circ}^{-}e^{i(\omega_{\parallel}t-k_{\parallel}z)}\right],$
with a parabolic dispersion relation
$\hbar\omega_{\parallel}=\frac{\hbar^{2}k_{\parallel}^{2}}{2m}$. For a vortex
line along the $\hat{\bm{z}}$-direction in cylindrical coordinates with unit
winding number, the irrotational part of the superfluid velocity has a
divergent (perpendicular) part characteristic of inviscid flow and an
advective (parallel) part characteristic of rigid translation

$\displaystyle\bm{v}\equiv\frac{\hbar}{m}\nabla\left(\vartheta\pm
k_{\parallel}z\right)=\frac{\hbar}{mr}\hat{\bm{\vartheta}}\pm\frac{\hbar
k_{\parallel}}{m}\hat{\bm{z}}.$ (3)

From this velocity field, we know that the circulation is quantized,
$\kappa\equiv\oint d\bm{l}\cdot\bm{v}=\frac{h}{m}.$ From Stokes’ theorem, we
have $\int d\bm{S}\cdot\nabla\times\bm{v}=\int
d\bm{S}\cdot\bm{\omega}=\frac{h}{m}.$ For a rectilinear
$\hat{\bm{z}}$-directed quantum vortex, the real part of the vorticity is
pinned at the vortex center
$\bm{\omega}=\frac{h}{m}\delta^{(2)}(r)\hat{\bm{z}}.$ For a vortex filament of
any shape, say a curve ${\cal C}$, the velocity field in general may be
written as

$\bm{v}(\bm{r})=\frac{\hbar}{2m}\oint_{\cal
C}\frac{d\bm{s}^{\prime}\times(\bm{r}-\bm{s}^{\prime})}{|\bm{r}-\bm{s}^{\prime}|^{3}},$
(4)

where $d\bm{s}^{\prime}$ is the differential length along the vortex filament,
$\bm{s}^{\prime}$ is the parametrization of ${\cal C}$, $\bm{r}$ is the field
point, and $d\bm{s}^{\prime}$ is the differential line element at the vortex
center and parallel to the vorticitySchwarz (1985). The Biot-Savart formula
(4) reduces to $\bm{v}=\hat{\bm{\vartheta}}\,{\hbar}/{(mr)}$ for the case of
an infinite rectilinear quantum vortex positioned along the center of a
cylindrical coordinate system with $\bm{s}^{\prime}={\bm{z}}$,
$d\bm{s}^{\prime}=dz\,\hat{\bm{z}}$, and
$d\bm{s}^{\prime}\times(\hat{\bm{r}}-\hat{\bm{s}}^{\prime})=\hat{\bm{\vartheta}}\,dz\,|z|/|\bm{r}-\bm{s}^{\prime}|$.

Using the Madelung transformationMadelung (1927)
$\varphi=\sqrt{\rho}\,e^{iS/\hbar}$, (1) can be written in terms of the
conjugate fluid variables (the action $S$ and number density $\rho$) as
follows: ${\cal L}_{\text{\tiny
BEC}}=-\rho\,\partial_{t}S-\rho\left[\frac{\left(\nabla
S\right)^{2}}{2m}+\frac{\hbar^{2}}{2m}\left(\frac{\nabla\rho}{2\rho}\right)^{2}-\mu+V_{\text{\tiny
H}}(\rho)\right].$ The quantity in square brackets is identified with a
Hamiltonian energy functional. Bohm originally made this identification while
considering quantum flow in a spatially-dependent linear external potential
$V(\bm{x})$ Bohm (1952). For a BEC, the nonlinear internal potential energy is
$E_{\text{\tiny int}}(\rho)\equiv V_{\text{\tiny H}}(\rho)-\mu.$ A
semiclassical energy functional is thus $H_{\text{\tiny
BEC}}=\frac{\left(\nabla
S\right)^{2}}{2m}+\frac{\hbar^{2}}{2m}\left(\frac{\nabla\rho}{2\rho}\right)^{2}+E_{\text{\tiny
int}}(\rho).$ The average energy is a statistical volume integral
$\overline{E}=\int d^{3}x\,\rho\,H_{\text{\tiny BEC}}$ with $\rho$ taken as
the effective probability distribution $\overline{E}=\int
d^{3}x\,\rho\left[\frac{\left(\nabla
S\right)^{2}}{2m}+\frac{\hbar^{2}}{2m}\left(\frac{\nabla\rho}{2\rho}\right)^{2}+E_{\text{\tiny
int}}(\rho)\right].$ $\overline{E}$ is a conserved quantity. Here we are
interested in the first term $\overline{E}_{\text{kin}}^{\text{cl}}=\int
d^{3}x\,\rho\frac{\left(\nabla S\right)^{2}}{2m}=\int
d^{3}x\,\rho(x)\,\left(\frac{1}{2}m\bm{v}(x)^{2}\right)$, where the de Broglie
relation $m\bm{v}=\nabla S$ is used. The average classical kinetic energy per
unit length $L$ of a single linear quantum vortex is ${\cal
E}\equiv\overline{E}_{\text{kin}}^{\text{cl}}/L$:

${\cal E}=m\rho_{\circ}\int_{r_{c}}^{r_{b}}2\pi
rdr\,\frac{v_{\vartheta}^{2}}{2}\stackrel{{\scriptstyle(\ref{irrotational_part_of_superfluid_velocity_field_c})}}{{=}}\frac{\rho_{\circ}h^{2}}{4\pi
m}\int_{r_{c}}^{r_{b}}\frac{dr}{r}=m\rho_{\circ}\frac{\kappa_{\circ}^{2}}{4\pi}\log\frac{r_{b}}{r_{c}},$
(5)

where $\kappa_{\circ}\equiv h/m$ is the quantum of circulation and
$\rho_{\circ}$ is the constant background number density of the condensate,
$r_{b}$ is a regularizing parameter associated with the size of the vessel
containing a single quantum vortex and $r_{c}$ is an effective cutoff
parameter to the divergent angular velocity field.111$r_{c}$ may be chosen so
that the integral for ${\cal E}$, with a cutoff that avoids the singularity at
the origin, is equivalent to the original nonsingular integral with no cutoff.
So, $r_{c}$ is technically not a cutoff parameter per se because the original
integral is nonsingular. Instead, $r_{c}$ is merely a matching parameter
useful for replacing a nonsingular but difficult integrand with an
analytically simpler one. All the expressions of the average classical kinetic
energy per unit length in (5) are equivalent and apply to a nearly straight
(high-tension and low-curvature) vortex.

Consider the original treatment by Fetter for nearly-rectilinear vortices
Fetter (1967). The initial (single vortex) equilibrium states are parallel
linear filaments with an arc length parametrization given by the following
vectors $\bm{R}_{1}^{(0)}=(\bm{r}_{1},z_{1})$ and
$\bm{R}_{2}^{(0)}=(\bm{r}_{2},z_{2}).$ The consequent deformed state due to
the mutual interaction of the vortices is parametrized by

$\bm{R}_{1}=(\bm{r}_{1}+\bm{u}_{1}(z_{1}),z_{1})\qquad\qquad\bm{R}_{2}=(\bm{r}_{2}+\bm{u}_{2}(z_{2}),z_{2}),$
(6)

where $\bm{u}_{1}$ and $\bm{u}_{2}$ are treated as small amplitude
perturbations in the radial directions with respect to the initial unperturbed
filamentary lines. Each vortex filament is a stretched helix, approximating a
nearly straight line parallel to the $\hat{\bm{z}}$-axis.

The first step is to calculate the fluctuation in position of a vortex element
at the first vortex due to the presence of the second vortex. The fluid
velocity at the first vortex located at $\bm{R}_{1}$ caused by the second
vortex at $\bm{R}_{2}$ is given by the Biot-Savart law (4)

$\bm{v}(\bm{R}_{1})=\frac{\kappa_{\circ}}{4\pi}\oint_{{\cal
C}_{2}}\frac{d\bm{s}_{2}\times(\bm{R}_{1}-\bm{R}_{2})}{|\bm{R}_{1}-\bm{R}_{2}|^{3}}.$
(7)

With $i=1,2$ denoting the vortices, the differential arc length is
$d\bm{s}_{i}=\frac{d\bm{s}_{i}}{dz_{i}}dz_{i}=dz_{i}\left(\hat{\bm{z}}+\frac{d\bm{u}_{i}(z_{i})}{dz_{i}}\right).$
Then, according to (7), the fluctuation of the position of the vortex element
originally at $\bm{R}^{(0)}_{1}$ is

$\displaystyle\frac{d\bm{u}_{1}}{dt}\\!\\!\\!$
$\displaystyle\stackrel{{\scriptstyle(\ref{deformed_R_s})}}{{=}}$
$\displaystyle\\!\\!\\!\frac{\kappa_{\circ}}{4\pi}\int
dz_{2}\frac{\left(\hat{\bm{z}}+\frac{d\bm{u}_{2}}{dz_{2}}\right)\times\left(\bm{r}_{12}+\hat{\bm{z}}z_{12}+\bm{u}_{12}\right)}{|\bm{r}_{12}+\hat{\bm{z}}z_{12}+\bm{u}_{12}|^{3}},$
where $\bm{r}_{12}\equiv\bm{r}_{1}-\bm{r}_{2}$, $z_{12}\equiv z_{1}-z_{2}$,
and $\bm{u}_{12}\equiv\bm{u}_{1}-\bm{u}_{2}$. Also, defining
$\bm{R}_{12}^{(0)}\equiv\bm{r}_{12}+\hat{\bm{z}}z_{12}$ and making use of the
Taylor expansion
$\frac{1}{|\bm{R}_{12}^{(0)}+\bm{u}_{12}|^{3}}=\frac{1}{|\bm{R}_{12}^{(0)}|^{3}}-\frac{3\bm{u}_{12}\cdot\bm{R}_{12}^{(0)}}{|\bm{R}_{12}^{(0)}|^{5}}+\cdots,$
we can write a leading order expansion of the position fluctuation (velocity
of the first vortex element at $z_{1}$ due to the presence of the second
vortex) $\displaystyle\frac{d\bm{u}_{1}}{dt}\\!\\!\\!$
$\displaystyle\stackrel{{\scriptstyle(\ref{velocity_fluctuation_c})}}{{=}}$
$\displaystyle\\!\\!\\!\frac{\kappa_{\circ}}{4\pi}\hat{\bm{z}}\times\int
dz_{2}\Big{[}\frac{\bm{r}_{12}+\bm{u}_{12}-z_{12}\frac{d\bm{u}_{2}}{dz_{2}}}{|\bm{R}_{12}^{(0)}|^{3}}$
(8b) $\displaystyle\hskip
54.2025pt-\frac{3\,\bm{r}_{12}\cdot\bm{u}_{12}}{|\bm{R}_{12}^{(0)}|^{5}}\bm{r}_{12}+\cdots\Big{]}.$

Making the analogy to the mutual inductance of two line currents, the Neumann
formula can be used to calculate the interaction energy of two vortices

$E_{12}=m\rho_{\circ}\int_{r_{c}}d^{3}x\,\frac{\bm{v}(x)^{2}}{2}\stackrel{{\scriptstyle(\ref{quantum_vortex_Biot_Savart_formula})}}{{=}}\frac{m\rho\kappa_{\circ}^{2}}{4\pi}\oint_{{\cal
C}_{1}}\oint_{{\cal C}_{2}}\frac{d\bm{s}_{1}\cdot
d\bm{s}_{2}}{|\bm{R}_{1}-\bm{R}_{2}|}.$ (9)

The next step is to develop an expansion for the interaction energy
$E_{12}=\frac{m\rho\kappa_{\circ}^{2}}{4\pi}\int\int
dz_{1}dz_{2}\frac{\left(1+\frac{d\bm{u}_{1}}{dz_{1}}\cdot\frac{d\bm{u}_{2}}{dz_{2}}+\cdots\right)}{|\bm{R}_{12}^{(0)}+\bm{u}_{12}|},\qquad$
where the cross-terms vanish because, in the reference frame at the original
center of the unperturbed $i$th vortex line along $\hat{\bm{z}}$, the motion
of the perturbed filament is in the polar direction
$\frac{d\bm{u}_{i}}{dz_{i}}/|\frac{d\bm{u}_{1}}{dz_{1}}|\approx\pm\hat{\bm{\vartheta}}$.
Employing Taylor’s theorem, the denominator is expanded to second order
$\frac{1}{|\bm{R}_{12}^{(0)}+\bm{u}_{12}|}=\frac{1}{|\bm{R}_{12}^{(0)}|}-\frac{\bm{r}_{12}\cdot\bm{u}_{12}}{|\bm{R}_{12}^{(0)}|^{3}}+\frac{3(\bm{r}_{12}\cdot\bm{u}_{12})^{2}}{2|\bm{R}_{12}^{(0)}|^{5}}-\frac{(\bm{u}_{12})^{2}}{2|\bm{R}_{12}^{(0)}|^{3}}+\cdots,$
so in turn the interaction energy expansion becomes

$\displaystyle E_{12}\\!\\!\\!$ $\displaystyle=$
$\displaystyle\\!\\!\\!\frac{m\rho\kappa_{\circ}^{2}}{4\pi}\int\int
dz_{1}dz_{2}\Big{[}\frac{1}{|\bm{R}_{12}^{(0)}|}-\frac{\bm{r}_{12}\cdot\bm{u}_{12}-\frac{1}{2}(\bm{u}_{12})^{2}}{|\bm{R}_{12}^{(0)}|^{3}}$
(10)
$\displaystyle+\frac{z_{12}}{|\bm{R}_{12}^{(0)}|^{3}}\frac{d\bm{u}_{2}}{dz_{2}}\cdot\bm{u}_{1}+\frac{\frac{3}{2}(\bm{r}_{12}\cdot\bm{u}_{12})^{2}}{|\bm{R}_{12}^{(0)}|^{5}}+\cdots\Big{]},$

where in (10) the term obtained by integrated by parts was rewritten as
$-\frac{d}{dz_{1}}\frac{1}{|\bm{R}_{12}^{(0)}|}=\frac{\hat{\bm{z}}\cdot\bm{R}_{12}^{(0)}}{|\bm{R}_{12}^{(0)}|^{3}}=\frac{z_{12}}{|\bm{R}_{12}^{(0)}|^{3}}$.
Hence, it is straightforward to calculate the variation of the mutual
interaction energy with respect to a fluctuation at $z_{1}$ of the center of
the first vortex line

$\begin{split}\frac{\delta
E_{12}}{\delta\bm{u}_{1}(z_{1})}=&\frac{m\rho\kappa_{\circ}^{2}}{4\pi}\int
dz_{2}\Big{[}-\frac{\bm{r}_{12}+\bm{u}_{12}-z_{12}\frac{d\bm{u}_{2}}{dz_{2}}}{|\bm{R}_{12}^{(0)}|^{3}}\\\
&+\frac{3\,\bm{r}_{12}\cdot\bm{u}_{12}}{|\bm{R}_{12}^{(0)}|^{5}}\bm{r}_{12}+\cdots\Big{]}.\end{split}$
(11)

Comparing this result with the previous result (8b) yields the useful relation

$m\rho\kappa_{\circ}\frac{d\bm{u}_{1}(z_{1})}{dt}=-\hat{\bm{z}}\times\frac{\delta
E_{12}}{\delta\bm{u}_{1}(z_{1})}.$ (12)

The mutual interaction energy part of the condensate energy arising from a
perturbed quantum vortex of length $L=\oint_{\cal C}\sqrt{dz^{2}+d\bm{u}^{2}}$
is primarily due to its bending, assuming a sufficient separation distance
exists between the vortices so that vortex-vortex straining has no low-order
effect. Therefore, we have

$\displaystyle\frac{\delta E_{12}}{\delta\bm{u}_{1}(z_{1})}$
$\displaystyle\stackrel{{\scriptstyle(\ref{Neumann_mutual_interaction_formula})}}{{\stackrel{{\scriptstyle(\ref{average_classical_kinetic_energy_per_unit_length})}}{{=}}}}$
$\displaystyle{\cal E}\frac{\delta L}{\delta\bm{u}_{1}(z_{1})}=-{\cal
E}\,\frac{d^{2}\bm{u}_{1}(z_{1})}{dz_{1}^{2}}+\cdots$ (13a)

Inserting this result into (12), yields

$m\rho\kappa_{\circ}\frac{d\bm{u}_{1}(z_{1})}{dt}={\cal
E}\hat{\bm{z}}\times\frac{d^{2}\bm{u}_{1}(z_{1})}{dz_{1}^{2}}.$ (14)

Having completed our review of Fetter’s treatment, let us consider a high-
curvature generalization of (14).

The Frenet-Serret formulas of multivariable calculus concerning the geometry
of curves describe the kinematic properties of a particle at position $\bm{R}$
moving along a continuous and differentiable curve $\cal C$ (the particle’s
trajectory or world line) embedded in three-dimensional Euclidean space
$\mathbb{R}^{3}$

$\displaystyle\frac{d\hat{\bm{t}}}{ds}$ $\displaystyle=$
$\displaystyle\kappa\hat{\bm{n}}\qquad\quad\frac{d\hat{\bm{n}}}{ds}=-\kappa\hat{\bm{t}}+\tau\hat{\bm{b}}\qquad\quad\frac{d\hat{\bm{b}}}{ds}=-\tau\hat{\bm{n}},\qquad$
(15)

where $s$ is the arc length parameter along $\cal C$, $\hat{\bm{t}}$ is the
unit tangent to $\cal C$ at the point $\bm{R}$, $\hat{\bm{n}}$ is the unit
normal perpendicular to $\hat{\bm{t}}$ at $\bm{R}$, $\hat{\bm{b}}$ is the unit
bi-normal perpendicular to both $\hat{\bm{t}}$ and $\hat{\bm{n}}$, and
$\kappa$ is the curvature and $\tau$ is the torsion of $\cal C$ at $\bm{R}$.
Given a fixed curve $\cal C$, one constructs a local Frenet-Serret frame as
follows

$\displaystyle\hat{\bm{t}}\equiv\frac{\bm{R}^{\prime}(s)}{|\bm{R}^{\prime}(s)|}\qquad\hat{\bm{n}}\equiv\frac{\hat{\bm{t}}^{\prime}(s)}{|\hat{\bm{t}}^{\prime}(s)|}\qquad\hat{\bm{b}}\equiv\hat{\bm{t}}\times\hat{\bm{n}},$
(16)

where the prime indicates differentiation with respect to $s$. So
$\hat{\bm{n}}$ points along the direction of the derivative of $\hat{\bm{t}}$
with respect to the arc length parameter of the curve and equating (15) with
(16), the curvature is $\kappa=|\hat{\bm{t}}^{\prime}(s)|.$ Therefore, the
unit vectors $\hat{\bm{t}}$, $\hat{\bm{n}}$, and $\hat{\bm{b}}$ serve as an
orthogonal coordinate system centered at $\bm{R}$, a local reference frame
that moves with the particle. For example, specifying the points in
$\mathbb{R}^{3}$ with the cylindrical coordinates $(r,\vartheta,z)$, if $\cal
C$ is a helix with its axis along $r=0$

$\displaystyle\bm{R}(s)$ $\displaystyle=$ $\displaystyle\left(\mathfrak{r}\cos
kz,\mathfrak{r}\sin kz,0\middle)+\middle(0,0,\mathfrak{h}kz\right)$ (17a)
$\displaystyle=$ $\displaystyle\bm{u}(kz)+\begin{cases}0+{\cal
O}\left(\frac{\mathfrak{h}}{\mathfrak{r}}\right)&\mathfrak{h}\ll\mathfrak{r},\\\
s\hat{\bm{z}}-{\cal
O}\left(\frac{\mathfrak{r}^{2}}{\mathfrak{h}^{2}}\right)&\mathfrak{r}\ll\mathfrak{h}\end{cases}$
(17b)

with $kz\equiv s/\sqrt{\mathfrak{r}^{2}+\mathfrak{h}^{2}}$, then the Frenet-
Serret frame is

$\displaystyle\hat{\bm{t}}$ $\displaystyle=$
$\displaystyle\frac{\mathfrak{r}\hat{\bm{\vartheta}}+\mathfrak{h}\hat{\bm{z}}}{\sqrt{\mathfrak{r}^{2}+\mathfrak{h}^{2}}}\qquad\hat{\bm{n}}=-\hat{\bm{r}}\qquad\hat{\bm{b}}=\frac{-\mathfrak{h}\hat{\bm{\vartheta}}+\mathfrak{r}\hat{\bm{z}}}{\sqrt{\mathfrak{r}^{2}+\mathfrak{h}^{2}}}$
(18)

and the local curvature and torsion of ${\cal C}$ are

$\displaystyle\kappa=\frac{\mathfrak{r}}{\mathfrak{r}^{2}+\mathfrak{h}^{2}}\qquad\qquad\tau=\frac{\mathfrak{h}}{\mathfrak{r}^{2}+\mathfrak{h}^{2}}.$
(19)

In the limit $\mathfrak{h}\ll\mathfrak{r}$, the curve is a compressed helix
where each cycle approximates a circle with curvature $1/\mathfrak{r}$. In the
opposite limit $\mathfrak{h}\gg\mathfrak{r}$, the curve is a stretched helix
(approximating a straight line) with torsion $1/\mathfrak{h}$.

We may reconsider the case of a perturbation to a rectilinear quantum vortex:
a quantum vortex supporting a small amplitude circularly-polarized plane
Kelvin wave counterrotating in a sense opposite to the vorticity direction of
the unperturbed vortex line. The solution we seek is based on the general
Frenet-Serret formulas.222 To make additional contact with the previous
literature, consider the limit of small curvature with the perturbed vortex is
a stretched helix. The local frame, which we may choose to fix at $z_{1}$
centered on the first quantum vortex, is
$\hat{\bm{t}}\equiv\frac{\bm{u}_{1}^{\prime}(z_{1})}{|\bm{u}_{1}^{\prime}(z_{1})|}=\hat{\bm{z}}+\cdots$,
$\hat{\bm{n}}\equiv\frac{\bm{u}_{1}^{\prime\prime}(z_{1})}{|\bm{u}_{1}^{\prime\prime}(z_{1})|}=-\hat{\bm{r}}+\cdots,$
and
$\hat{\bm{b}}\equiv\hat{\bm{t}}\times\hat{\bm{n}}=-\hat{\bm{\vartheta}}+\cdots$,
where the arc length is parametrized by $s\approx z_{1}$. The mutual
interaction of the vortices causes them to bend into the filamentary shape of
a rotating helix—a circularly polarized Kelvin wave. Equation (14) may be
rewritten as
$\dot{\bm{u}}\stackrel{{\scriptstyle(\ref{Frenet_Serret_construction_a})}}{{=}}\frac{{\cal
E}}{m\rho\kappa_{\circ}}\frac{\bm{u}^{\prime}}{|\bm{u}^{\prime}|}\times\bm{u}^{\prime\prime},$
In a parametrization with $|\bm{u}^{\prime}|=1$, this is the local induction
approximation (LIA) used for quantum turbulence simulations Schwarz (1985).
Since the second derivative of the radial perturbation of the quantum vortex
center is
$\bm{u}_{1}^{\prime\prime}\stackrel{{\scriptstyle(\ref{particle_trajectory})}}{{=}}-k^{2}\bm{u}_{1},$
(14) takes the form of an undamped Bloch equation

$\frac{d\bm{u}_{1}(z_{1})}{dt}=\frac{{\cal
E}k^{2}}{m\rho\kappa_{\circ}}\,\bm{u}_{1}(z_{1})\times\hat{\bm{s}}_{2}.$ (20)

The radial displacement $\bm{u}_{1}$ behaves like the magnetization vector of
a nuclear spin precessing about a background magnetic field along
$\hat{\bm{s}}_{2}\approx\hat{z}$. Thus, two segments of the mutually
interacting quantum vortices behave like coupled nuclear spins.

The helix rotates in time as a sinusoidal perturbation
$\bm{u}_{1}\sim\mathfrak{r}_{1}\,e^{i(kz-\omega t)}$, so (14) may be written
as

$\displaystyle i\omega\bm{u}_{1}(z_{1})$ $\displaystyle=$
$\displaystyle\frac{{\cal
E}(k)k^{2}}{m\rho\kappa_{\circ}}\hat{\bm{z}}\times\bm{u}_{1}(z_{1}).$ (21)

Then with $\bm{u}_{1}=(x,y)$, we have $i\omega(x,y)=\frac{{\cal
E}(k)k^{2}}{m\rho\kappa_{\circ}}(-y,x),$ or in matrix form
$\scriptsize\begin{pmatrix}i\omega&\frac{{\cal
E}k^{2}}{m\rho\kappa_{\circ}}\\\ -\frac{{\cal
E}k^{2}}{m\rho\kappa_{\circ}}&i\omega\end{pmatrix}\begin{pmatrix}x\\\
y\end{pmatrix}=\begin{pmatrix}0\\\ 0\end{pmatrix}.$ The well known solution by
setting the determinant to zero, for $r_{b}=1/k$, is the Kelvin wave
dispersion relation

$\displaystyle\omega_{\text{\tiny K}}$ $\displaystyle=$
$\displaystyle\frac{{\cal
E}(k)k^{2}}{m\rho\kappa_{\circ}}\stackrel{{\scriptstyle(\ref{average_classical_kinetic_energy_per_unit_length})}}{{=}}\frac{\kappa_{\circ}k^{2}}{4\pi}\log\frac{1}{r_{c}k},$
(22)

valid in the limit where $\mathfrak{r}/\mathfrak{h}$ is a small quantity with
${\cal C}$ nearly a line. Now, with a locally helical-shaped quantum vortex
parametrized by (17), we need not restrict $\mathfrak{r}/\mathfrak{h}$ to be a
small parameter. Instead the only generic constraint imposed upon the shape of
the perturbed quantum vortex is that it be locally parametrized by (17a). We
will not loose the generality of two interacting quantum vortices of arbitrary
shape by restricting their mutually interacting segments to helices.

Since $|\bm{R}^{\prime}(s)|=1$,
$\hat{\bm{t}}\stackrel{{\scriptstyle(\ref{Frenet_Serret_construction_a})}}{{=}}\bm{R}^{\prime}(s)$
and in turn (15) is

$\frac{d^{2}\bm{R}(s)}{ds^{2}}\stackrel{{\scriptstyle(\ref{curvature_and_torsion_of_helix})}}{{=}}\frac{\mathfrak{r}}{\mathfrak{r}^{2}+\mathfrak{h}^{2}}\hat{\bm{n}}.$
(23)

So, allowing for high curvature, (16) can be written as

$\displaystyle\hat{\bm{b}}$
$\displaystyle\stackrel{{\scriptstyle(\ref{curvature_equation})}}{{=}}$
$\displaystyle\frac{\mathfrak{r}^{2}+\mathfrak{h}^{2}}{\mathfrak{r}}\hat{\bm{t}}\times\frac{d^{2}\bm{R}(s)}{ds^{2}}$
(24a)
$\displaystyle\stackrel{{\scriptstyle(\ref{particle_trajectory_a})}}{{=}}$
$\displaystyle\frac{1}{\mathfrak{r}}\,\hat{\bm{t}}\times\bm{R}(s)$ (24b)
$\displaystyle-\mathfrak{h}\hat{\bm{\vartheta}}+\mathfrak{r}\hat{\bm{z}}$
$\displaystyle\stackrel{{\scriptstyle(\ref{Frenet_Serret_frame})}}{{=}}$
$\displaystyle\left(\hat{\bm{\vartheta}}+\frac{\mathfrak{h}}{\mathfrak{r}}\hat{\bm{z}}\right)\times\bm{R}(s).$
(24c)

We are now considering a pair of parallel vortex segments, arbitrary helices
equal in magnitude and spin, circling around each other whereby each segment
moves by the influence of the velocity field of the other. Since the velocity
field of a vortex segment (with a circularly-polarized radial perturbation) is
anti-parallel to the bi-normal unit vector of a helical curve (i.e.,
$\bm{v}\propto-\hat{\bm{b}}$), this velocity field may be generally expressed
in the Frenet-Serret frame as

$\displaystyle\bm{v}$
$\displaystyle\stackrel{{\scriptstyle(\ref{Frenet_Serret_frame})}}{{=}}$
$\displaystyle\omega_{\circ}(\mathfrak{h}\hat{\bm{\vartheta}}-\mathfrak{r}\hat{\bm{z}}).$
(25)

Next, let us determine the radius $\mathfrak{r}$ and the stretching
$\mathfrak{h}$ of the helix directly in terms of the available physical
parameters describing the mutually interacting quantum vortices, such as the
separation distance between the vortices $2R_{12}^{(0)}$. Let us denote the
orbital radius at $R_{\circ}\equiv R_{12}^{(0)}$. Thus, the magnitude of the
polar velocity component is $v_{\vartheta}=\frac{\hbar}{m(2R_{\circ})}.$ If
$\bm{v}_{\vartheta}=\bm{\omega}_{\circ}\times\bm{R}_{\circ}$, where
$\bm{\omega}_{\circ}=\omega_{\circ}\hat{\bm{z}}$ is the effective orbital
“cyclotron” spin vector, then the orbital angular frequency of the vortex pair
is
$\omega_{\circ}=\frac{v_{\vartheta}}{R_{\circ}}=\frac{\hbar}{2mR_{\circ}^{2}}=\frac{\kappa_{\circ}}{4\pi
R_{\circ}^{2}}.$ The quantum vortex solution (3) with a plane-wave phonon mode
counter-propagating to $\bm{\omega}_{\circ}$ along the $-\hat{\bm{z}}$
direction with wave number $k=k_{\parallel}$ is concomitant to the quantum
vortex kelvon mode

$\displaystyle\bm{v}$
$\displaystyle\stackrel{{\scriptstyle(\ref{v_field})}}{{=}}$
$\displaystyle\frac{\hbar}{2mR_{\circ}}\hat{\bm{\vartheta}}-\frac{\hbar
k}{m}\hat{\bm{z}}=\omega_{\circ}\left(R_{\circ}\hat{\bm{\vartheta}}-2R_{\circ}^{2}k\hat{\bm{z}}\right).$
(26)

Equating (26) to (25), the helical parameters $\mathfrak{h}$ and
$\mathfrak{r}$ are thus analytically determined to be $\mathfrak{h}=R_{\circ}$
and $\mathfrak{r}=2R_{\circ}^{2}k.$ In turn, the curvature and torsion of the
helix are
$\kappa\stackrel{{\scriptstyle(\ref{curvature_and_torsion_of_helix})}}{{=}}\frac{2k_{\parallel}}{1+4k^{2}R_{\circ}^{2}}$
and
$\tau\stackrel{{\scriptstyle(\ref{curvature_and_torsion_of_helix})}}{{=}}\frac{1}{R_{\circ}}\,\frac{1}{1+4k^{2}R_{\circ}^{2}}.$
One observes that the maximum curvature $\kappa_{\text{max}}=k$ occurs when
$k=1/(2R_{\circ})$ and the maximum torsion $\tau_{\text{max}}=1/R_{\circ}$
occurs when $k=0$. Both the curvature and torsion vanish in the limit of
infinitely separated straight-line vortices. Inserting (25) into (24c), one
finds a precise self-consistent equation governing the dynamics of a quantum
vortex with large-amplitude helical wave with arbitrary wave number $k$ in
mutual interaction with another quantum vortex

$\dot{\bm{R}}=-\omega_{\circ}\left(\hat{\bm{\vartheta}}+\frac{\hat{\bm{z}}}{2R_{\circ}k}\right)\times\bm{R}(s).$
(27a) For analytical continuation to Fetter’s treatment in the high-tension
limit $\mathfrak{h}\gg\mathfrak{r}$, the fluctuation of (27a) obeys
$\dot{\bm{u}}\stackrel{{\scriptstyle(\ref{particle_trajectory_c})}}{{=}}-\frac{\omega_{\circ}}{2R_{\circ}k}\hat{\bm{z}}\times\bm{u}.$
(27b)

Equating (27b) to the Bloch equation (20) provides a way to determine the wave
number dependence ${\cal E}={\cal E}(k)$:

${\cal
E}(k)=\frac{m\rho\kappa_{\circ}\omega_{\circ}}{2R_{\circ}k^{3}}=\left(\frac{m\rho\kappa_{\circ}^{2}}{8\pi
R_{\circ}^{3}}\right)\,k^{-3}.$ (28)

A helical wave triggered by a $k$-mode axial phonon counter-propagates along
the segment undergoing mutual interaction. Only the inital axial phonon
dispersion relation $\omega=\frac{\kappa_{\circ}}{4\pi}k^{2}$ is consonant
with the dispersion relation for $\omega_{\text{\tiny K}}$ given by (22) for a
semiclassical Kelvin wave.

Equation (28) is an analytical prediction, based on the Frenet-Serret
differential geometry of space curves, that the helical wave spectrum scales
as $k^{-3}$ for highly curved quantum vortices. This power-law was found in
precise quantum simulations of superfluid turbulence supporting highly curved
vorticesYepez et al. (2009). A $k^{-3}$ spectrum also arises from a single
rectilinear quantum vortex Nore et al. (1997), and so it has been recently
suggested that this straight vortex power-law underlies the high-$k$ part of a
turbulent superfluid spectrum Krstulovic and Brachet (2009). The analysis
presented here suggests that (28), not (5) originally discovered by Lord
Kelvin in 1880 nor the Fourier spectrum of a single rectilinear quantum
vortex, may be responsible for the high-$k$ spectrum because of the effect of
mutually interacting highly-curved quantum vortices characteristic of
superfluid turbulence.

## References

  * Gross (1963) E. P. Gross, J.Math.Phys. 4, 195 (1963).
  * Pitaevskii (1961) L. P. Pitaevskii, Soviet Phys. JETP 13, 451 (1961).
  * Schwarz (1985) K. W. Schwarz, Phys. Rev. B 31, 5782 (1985).
  * Madelung (1927) E. Madelung, Zeit. F. Phys. 40, 322 (1927).
  * Bohm (1952) D. Bohm, Phys. Rev. 85, 166 (1952).
  * Fetter (1967) A. Fetter, Physical Review 162, 143 (1967).
  * Yepez et al. (2009) J. Yepez, G. Vahala, L. Vahala, and M. Soe, Phys. Rev. Lett. 103, (2009).
  * Nore et al. (1997) C. Nore, M. Abid, and M. E. Brachet, Phys. Fluids 9, 2644 (1997).
  * Krstulovic and Brachet (2009) G. Krstulovic and M. Brachet, arXiv:0911.1749 (2009).