# Regularly oscillating mappings between metric spaces and a theorem of Hardy
and Littlewood

Marijan Marković  Faculty of Sciences and MathematicsUniversity of
MontenegroDžordža Vašingtona bb81000 PodgoricaMontenegro
<EMAIL_ADDRESS>
###### Abstract.

This paper is motivated by the classical theorem due to Hardy and Littlewood
which concerns analytic mappings on the unit disk and relates the growth of
the derivative with the Hölder continuity. We obtain a version of this result
in a very general setting – for regularly oscillating mappings on a metric
space equipped with a weight, which is a continuous and positive function,
with values in another metric space. As a consequence, we derive the Hardy and
Littlewood theorem for analytic mappings on the unit ball of a normed space.

###### Key words and phrases:

Hardy and Littlewood theorem, regularly oscillating mappings, the distance
function, uniform domains, mappings between metric spaces, analytic mappings
between normed spaces, Lipschitz type spaces, Bloch type spaces

###### 2020 Mathematics Subject Classification:

Primary 26A16, 51F30, 46E50; Secondary 46G20, 30L15, 54E45

##  1\. Introduction

Let $\mathbb{D}\subseteq\mathbb{C}$ be the unit disk, and $\alpha\in(0,1]$.
The classical Hardy and Littlewood theorem [8] states that an analytic mapping
$f$ on $\mathbb{D}$ satisfies

(1.1) $|f^{\prime}(z)|\leq C_{f}(1-|z|)^{\alpha-1},\quad z\in\mathbb{D},$

if and only if it is Hölder continuous, i.e.,

(1.2) $|f(z)-f(w)|\leq\ C^{\prime}_{f}|z-w|^{\alpha},\quad z,w\in\mathbb{D},$

where $C_{f}$ and $C^{\prime}_{f}$ are constants (may depend only on the
mapping $f$). This theorem may be found in the Duren book [5, Theorem 5.1],
where the given proof is based on the Cauchy integral formula. The estimates
of the form $C^{\prime}_{f}\leq AC_{f}$ and $C_{f}\leq BC^{\prime}_{f}$, where
$A,B>0$ are absolute constants, are also known.

This classical result has counterparts for other mapping classes. The theorem
has a version for quasi-conformal mappings on uniform domains in
$\mathbb{R}^{n}$ obtained by Astala and Gehring [1]. An analog for quasi-
regular mappings on Lipα-extension domains is given in the Nolder paper [14].

Gehring and Martio [6] obtained a generalization of one part of the Hardy and
Littlewood theorem for (not necessarily analytic) mappings on uniform domains,
and somewhat later on Lipα-extension domains [7]. In their work for (analytic)
mappings on a domain $D\subseteq\mathbb{C}$, the condition (1.1) is replaced
by

$|f^{\prime}(z)|\leq C_{f}d(z,\partial D)^{\alpha-1},\quad z\in D,$

where $d(z,\partial D)$, is the distance function, i.e.,

$d(z,B)=\inf_{x\in B}|x-z|,\quad z\in\mathbb{C},\quad
B\subseteq\mathbb{C},B\neq\emptyset.$

Immediately after that, Lappalainen in his Ph.D. thesis [11] gave a
generalization of the same part of the Hardy and Littlewood theorem for
mappings on Lipφ-extension domains, where $\varphi$ is a majorant. A function
$\varphi\in C[0,\infty)$ is called the majorant if it has the following
properties:

(1) $\varphi(0)=0$; $\varphi(t)>0$, $t\in(0,\infty)$;

(2) $\varphi$ is an increasing function on $(0,\infty)$;

(3) $\varphi^{\prime}$ is a decreasing function on $(0,\infty)$.

These properties imply the following four (more important):

(a) $\varphi^{\prime}(t)\leq\frac{\varphi(t)}{t}$, $t\in(0,\infty)$;

(b) $\frac{\varphi(t)}{t}$ is decreasing;

(c) $\varphi(ct)\leq c\varphi(t)$, $t\in(0,\infty)$, $c>1$;

(d) $\varphi(t+s)\leq\varphi(t)+\varphi(s)$, $t,s\in(0,\infty)$.

A domain $D$ is called a Lipφ-extension domain [11] if there exists a constant
$M$ such that for every $x,y\in D$ there exists a rectifiable curve
$\gamma_{x,y}\subseteq D$ with ends at $x$ and $y$ such that

(1.3) $\int_{\gamma_{x,y}}\frac{\varphi(d(z,\partial D))}{d(z,\partial
D)}|dz|\leq M\varphi(|x-y|).$

For the standard majorant $\varphi(t)=\varphi_{\alpha}(t)=t^{\alpha}$,
$\alpha\in(0,1]$, one says that $D$ is a Lipα-extension domain (the class of
domains introduced previously by Gehring and Martio [7]).

The variant of the Hardy and Littlewood theorem obtained by Lappalainen [11,
Theorem 7.3] states that if for an analytic mapping on a Lipφ-extension domain
$D\subseteq\mathbb{C}$ holds

(1.4) $|f^{\prime}(z)|\leq C_{f}\frac{\varphi(d(z,\partial D))}{d(z,\partial
D)},\quad z\in D,$

then $f$ satisfies

(1.5) $|f(z)-f(w)|\leq C^{\prime}_{f}\varphi(|z-w|),\quad z,w\in D,$

where $C_{f}$ and $C^{\prime}_{f}$ are constants.

The first condition displayed above is local, and the second one is of the
global nature. In this paper we firstly examine, in the context of metric
spaces, when does a similar local condition imply the corresponding global
one. The condition (1.4) includes the distance function, which we shall
replace by a weight, that is a positive and continuous function, on a metric
space. Then we consider the converse question, i.e., what can locally be said
about a mapping which satisfy the condition (1.5) adapted for metric spaces.
We shall generalize separately the two parts of the classical theorem. As a
consequence of our main results, we are able to derive the Hardy and
Littlewood theorem for analytic mappings on the unit ball of a normed space
with values in an another normed space.

##  2\. Preliminaries

This section contains preliminaries on mapping classes we shall encounter in
the rest of this paper, as well as some facts concerning the weighted distance
on a metric space.

As we have already said, an everywhere positive and continuous function on a
metric space is called the weight function. Below we introduce the class of
regularly oscillating mappings between metric spaces with a weight on the
first one.

Let $(X,d_{X})$ and $(Y,d_{Y})$ be metric spaces. For a mapping $f:X\to Y$ we
define

$D^{\ast}f(x)=\limsup_{y\to x}\frac{d_{Y}(f(x),f(y))}{d_{X}(x,y)},\quad x\in
X;$

we set $D^{\ast}f(x)=0$ for an isolated $x\in X$. The mapping class
$D^{\ast}(X,Y)$ contains $f:X\to Y$ such that $D^{\ast}f(x)$ is finite for
every $x\in X$.

Assume moreover that $w$ be a weight on $X$. A mapping $f\in D^{\ast}(X,Y)$
which is bounded on every open ball $B_{X}(x,r)$, $x\in X$, $0<r<w(x)$, is
said to be regularly oscillating if there exists a constant $K=K_{f}$ such
that

(2.1) $D^{\ast}f(x)\leq\frac{K}{r}{\sup_{y\in
B_{X}(x,r)}d_{Y}(f(x),f(y))},\quad x\in X,\quad 0<r<w(x).$

The corresponding mapping class is denoted by $\mathrm{RO}_{w}^{K}(X,Y)$.

In case of differentiable mapping between normed spaces $D^{\ast}f(x)$
coincides with the operator norm of the differential $Df(x)$. A mapping
$f:D\to Y$, where $D\subseteq X$ is a domain in a normed space $X$ (over
$\mathbb{R}$ or $\mathbb{C}$), and $Y$ is another normed space, is Fréchet
differentiable at $x\in D$, if there exists a continuous linear mapping
$Df(x):X\to Y$, called Fréchet differential, such that in an open ball
$B_{X}(x,r)=\\{y\in X:\|x-y\|_{X}<r\\}\subseteq D$ there holds

$f(y)-f(x)=Df(x)(y-x)+\alpha(y-x),\quad y\in B_{X}(x,r),$

where $\alpha:B_{X}(0,r)\to Y$ satisfies $\lim_{h\to
0}\frac{\|\alpha(h)\|_{Y}}{\|h\|_{X}}=0$. We denote by

$\|Df(x)\|=\sup_{\|\zeta\|_{X}=1}\|Df(x)\zeta\|_{Y}$

the operator norm of $Df(x):X\to Y$. The proof that $D^{\ast}f(x)$ is equal to
the norm of Frechét differential $Df(x):X\to Y$ may be found in [12].
Therefore, we have

$\|Df(x)\|=\limsup_{y\to x}\frac{\|f(x)-f(y)\|_{Y}}{\|x-y\|_{X}}.$

If $X$ and $Y$ are normed spaces over the field $\mathbb{C}$, then a mapping
$f:D\to Y$ is said to be analytic on a domain $D\subseteq X$ if it is Fréchet
differentiable everywhere in $D$. The class of regularly oscillating mapping
on a domain in a normed with respect to the distance function of the domain
contains the class of bounded analytic mappings on $D$. This may be easily
seen by using the Schwarz lemma. For the sake of completeness we prove this in
the lemma below.

######  Lemma 2.1.

Every bounded analytic mapping $f:D\to Y$, where $D$ is domain a normed space
$X$, and $Y$ is another normed space, belongs to the class
$\mathrm{RO}^{1}_{d}(D,Y)$, where $d$ is the distance function on $D$, i.e.,
$d(z)=d(z,\partial D)=\inf_{w\in\partial D}\|z-w\|$, $z\in D$.

###### Proof.

Let $z\in D$. For $r<d(z,\partial D)$ the mapping
$g(\zeta)=\frac{1}{M}f(z+r\zeta)$, where $M=\sup_{w\in B(z,r)}\|f(w)\|_{Y}$,
is analytic on $B_{X}(0,1)$ and bounded by $1$. By the Schwarz lemma there
holds $\|Dg(0)\|\leq 1$. Since $Dg(0)=\frac{r}{M}Df(z)$, we have

$\|Df(z)\|\leq\frac{M}{r}=\frac{1}{r}\sup_{w\in B_{X}(z,r)}\|f(w)\|_{Y}.$

If we replace above $f$ with $f-C$, where $C=f(z)$, we obtain

$\|Df(z)\|\leq\frac{1}{r}\sup_{w\in B_{X}(z,r)}\|f(w)-f(z)\|_{Y},\quad z\in
D,\quad 0<r<d(z,\partial D),$

which proves that $f$ belongs to the class $\mathrm{RO}^{1}_{d}(D,Y)$. ∎

######  Remark 2.2.

Based on [15] and [16] we give below some examples of classes of regularly
oscillating functions on a domain in $\mathbb{R}^{n}$ with respect to the
distance function of the domain as the weight function.

The class $\mathrm{OC}^{2}(D)$, where $D\subseteq\mathbb{R}^{n}$ is a domain,
contains functions $f\in C^{2}(D)$ which satisfy the inequality

$|\Delta f(z)|\leq C_{f}\sup_{w\in B(z,r)}|\nabla f(w)|,\quad z\in D,\quad
0<r<d(z,\partial D),$

where $C_{f}$ is a constant. Every function in this class is regularly
oscillating with respect to the distance function of the domain $D$. The class
$\mathrm{OC}^{2}(D)$ contains many others well known classes of function, such
as harmonic, hyperbolically harmonic, and polyharmonic class of functions on
the domain $D$.

On the other hand, for a domain $D\subseteq\mathbb{R}^{n}$ one may relax the
definition of regularly oscillating mappings in order to include more classes.
It may be asked that the condition (2.1) holds for a locally Lipschitz mapping
on $D$ and for almost every $z\in D$. Then the class of regularly oscillating
mappings with respect to the distance function on $D$ include, for example,
the class of convex functions.

We introduce now the mapping classes we need in this work.

For a non-negative function $\varphi$ on $[0,\infty)$, which will be called
the majorant in this context, we introduce the mapping class
$\Lambda_{\varphi}(X,Y)$ (a Lipschitz type class) in the following way: A
mapping $f:X\to Y$ belongs to this class if there exists a constant $C_{f}$
such that

$d_{Y}(f(x),f(y))\leq C_{f}\varphi(d_{X}(x,y)),\quad x,y\in X.$

The infimum of all $C_{f}$ is called the norm of $f$ in the class
$\Lambda_{\varphi}(X,Y)$, and denoted by $\|f\|_{\Lambda_{\varphi}(X,Y)}$. In
the case $\varphi=\varphi_{\alpha}$, $\alpha\in(0,1]$, we write
$\Lambda^{\alpha}(X,Y)$ instead of $\Lambda_{\varphi_{\alpha}}(X,Y)$.

For a weight $w$ on the metric space $(X,d_{X})$ the class
$\mathrm{B}_{w}(X,Y)$ (a Bloch-type class) is consisted of mappings $f\in
D^{\ast}(X,Y)$ for which there exists a constant $C_{f}$ such that

$D^{\ast}f(z)\leq C_{f}w(z),\quad z\in X.$

The least possible constant $C_{f}$ is defined to be the norm of $f$ in this
class, and denoted by $\|f\|_{\mathrm{B}_{w}(X,Y)}$.

Notice that a mapping $f:X\to Y$ belongs to $\mathrm{B}_{1}(X,Y)$ if and only
if $D^{\ast}f$ is bounded on $X$.

Finally, for the sake of completeness we include here some facts concerning
the weighted distance on a metric space. We refer to [2] for the detailed
exposition; see also [12] for a short one.

A continuous mapping $\gamma:[a,b]\to X$, where $[a,b]\subseteq\mathbb{R}$ is
a segment, is called the curve in a metric space $(X,d_{X})$ (we identify the
curve $\gamma$ with its image $\gamma([a,b])\subseteq X$). We say that the
curve $\gamma$ connects $\gamma(a)$ and $\gamma(b)$, or that $\gamma(a)$ and
$\gamma(b)$ are endpoints of the curve $\gamma$. The length of the curve
$\gamma$ is

$\ell(\gamma)=\sup\sum_{i=1}^{n}d_{X}(\gamma(t_{i-1}),\gamma(t_{i})),$

where the supremum is taken over all partition $a=t_{0}<t_{1}<\dots<t_{n}=b$,
$n\in\mathbb{N}$, of $[a,b]$. A curve is rectifiable if its length is finite.
The family of rectifiable curves with endpoints at $x$ and $y$ is denoted by
$\Gamma({x,y})$. A metric space $X$ is rectifiable connected if $\Gamma(x,y)$
is not empty for every $x,y\in X$. Obviously, for $\gamma\in\Gamma_{x,y}$ we
have $d_{X}(x,y)\leq\ell(\gamma)$. If $\gamma:[a,b]\to X$ is a rectifiable
curve in a metric space $X$, and $[c,d]\subseteq[a,b]$, then the (rectifiable)
curve $\gamma:[c,d]\to X$ is denoted by $\gamma_{[c,d]}$. Clearly, we have
$\ell(\gamma)=\ell(\gamma_{[a,c]})+\ell(\gamma_{[c,b]})$, $c\in[a,b]$.

The integral of a continuous function $f$ on $X$ over a rectifiable curve
$\gamma\subseteq X$ is denoted by $\int_{\gamma}f$. It is the number which
satisfies: For every $\varepsilon>0$ there exists $\delta>0$ such that

$\left|\int_{\gamma}f-\sum_{i=1}^{n}f(\gamma(s_{i}))\ell(\gamma|_{[t_{i-1},t_{i}]})\right|<\varepsilon$

for every partition of $t_{0},t_{1},\dots,t_{n}$ of $[a,b]$ with $\max_{1\leq
i\leq n}|t_{i}-t_{i-1}|<\delta$, and every sequence $s_{1},s_{2},\dots,s_{n}$
such that $t_{i-1}\leq s_{i}\leq t_{i}$, $1\leq i\leq n$.

Let $w$ be a weight on a metric space $X$. The weighted distance on the
(rectifiable connected) metric space $X$ is introduced by

$d_{w}(x,y)=\inf_{\gamma\in\Gamma_{x,y}}\int_{\gamma}w,\quad x,y\in X.$

In particulary for $w\equiv 1$ we have $\int_{\gamma}1=\ell(\gamma)$ and
$d_{1}(x,y)=\inf_{\gamma\in\Gamma_{x,y}}\ell(\gamma)$, $x,y\in X$. The
distance $d_{1}$ is in some cases equal to $d_{X}$. For example, this occurs
if $X$ is a convex domain in a normed space. However, in general, we have
$d_{X}\leq d_{1}$. In case of domains in $\mathbb{R}^{n}$, $d_{1}$ is known as
the inner distance. In case when the weight function is reciprocal to the
distance function of a domain, for the weighted distance we have the known
quasi-hyperbolic distance on the domain introduced by Gehring and Palka.

######  Remark 2.3.

The topology of $(X,d_{X})$ may be the same as the topology of the metric
space $(X,d_{w})$. Let $(X,d_{X})$ be a metric space with the following local
property

(2.2) $(\exists M>0)(\forall x\in X)(\exists r>0)(\forall y\in
B_{X}(x,r))(\exists\gamma\in\Gamma_{x,y})(\ell(\gamma_{x,y})\leq Md(x,y)).$

For a weight $w$ on $X$ we shall prove

(2.3) $w(x)\leq\liminf_{y\to x}\frac{d_{w}(x,y)}{d_{X}(x,y)}\leq\limsup_{y\to
x}\frac{d_{w}(x,y)}{d_{X}(x,y)}\leq Mw(x),\quad x\in X,$

which is sufficient for the equality of the two topologies.

Indeed, since $w$ is continuous, for $\varepsilon>0$ there exists an open ball
$B_{X}(x,r)$ such that

$0<w(x)-\varepsilon<w(y)<w(x)+\varepsilon,\quad y\in B_{X}(x,r).$

We may replace $r$ with a smaller number such that (2.2) is satisfied. Now, we
have

$d_{w}(x,y)\leq\int_{\gamma_{xy}}\omega\leq(w(x)+\varepsilon)\ell(\gamma_{xy})\leq(w(x)+\varepsilon)Md_{X}(x,y).$

On the other hand, if $\gamma$ is among curves that connect $x$ and $y$, then

$d_{w}(x,y)=\inf_{\gamma}\int_{\gamma}w\geq(w(x)-\varepsilon)d_{X}(x,y).$

Therefore,

$w(x)-\varepsilon\leq\frac{d_{w}(x,y)}{d_{X}(x,y)}\leq
M(w(x)+\varepsilon),\quad y\in B_{X}(x,r),$

which implies

(2.4) $w(x)-\varepsilon\leq\liminf_{y\to
x}\frac{d_{w}(x,y)}{d_{X}(x,y)}\leq\limsup_{y\to
x}\frac{d_{w}(x,y)}{d_{X}(x,y)}\leq M(w(x)+\varepsilon),\quad x\in X.$

Letting $\varepsilon\to 0$, we obtain (2.3).

The property (2.3) occurs on domains in normed spaces, since one may chose
$\gamma_{x,y}(t)=(1-t)x+ty$, $t\in[0,1]$. It is easy to show that
$\gamma_{x,y}$ is a rectifiable curve, and $\ell(\gamma_{x,y})=\|x-y\|$.

For similar results on the coincidence of topologies we refer to [18].

##  3\. Main results

This section contains our main results. At the end of the paper, based on our
approach, we prove a version of the Hardy and Littlewood theorem for analytic
mappings between normed spaces.

The following theorem generalize the one part of the classical Hardy and
Littlewood theorem. It contains an integral condition which connects the
weight on a metric space with the ordinary metric through the majorant. This
condition is a generalization of (1.3) which concerns domains in
$\mathbb{R}^{n}$, and it simply means that the weighted distance may be
controlled by the ordinary one.

######  Theorem 3.1.

Let $(X,d_{X})$ and $(Y,d_{Y})$ be metric spaces, $w$ a weight on $X$, and
$\varphi$ a non-negative function on $[0,\infty)$. Assume the following
condition:

(3.1) $(\exists M>0)(\forall x,y\in
X)(\exists\gamma\in\Gamma(x,y))\int_{\gamma}w\leq M\varphi(d_{X}(x,y)).$

Then we have

$\|f\|_{\Lambda_{\varphi}(X,Y)}\leq M\|f\|_{\mathrm{B}_{w}(X,Y)},\quad
f\in\mathrm{B}_{w}(X,Y).$

In particular, if $\varphi$ is continuous in $0$ and $\varphi(0)=0$, then
every $f\in\mathrm{B}_{w}(X,Y)$ has a continuous extension on the minimal
completion of the metric space $X$, if $Y$ is a complete metric space.

In the proof we use the following lemma which gives the estimate of the length
of the image of a rectifiable curve. It may be found with a proof in [12].

######  Lemma 3.2 (see [12]).

Let $X$ and $Y$ be metric spaces, $\gamma\subseteq X$ a rectifiable curve, and
$f\in\mathrm{B}_{1}(X,Y)$. Then the curve $f\circ\gamma\subseteq Y$ is also
rectifiable, and we have

$\ell(f\circ\gamma)\leq\|f\|_{\mathrm{B}_{1}(X,Y)}\ell(\gamma).$

###### Proof of Theorem 3.1.

Let $f\in\mathrm{B}_{w}(X,Y)$ satisfies

(3.2) $D^{\ast}f(z)\leq C_{f}w(z),\quad z\in X,$

where $C_{f}$ is a constant. We shall show that $f$ belongs to the class
$\Lambda_{\varphi}(X,Y)$, and

$d_{Y}(f(x),f(y))\leq{C}^{\prime}_{f}\varphi(d_{X}(x,y)),\quad x,y\in X,$

where ${C}^{\prime}_{f}=MC_{f}$. This clearly implies the statement in our
theorem.

Since of (3.2) the quantity

$B_{f}=\|f\|_{\mathrm{B}_{w}(X,Y)}=\sup_{z\in X}{w(z)}^{-1}D^{\ast}f(z)$

is finite. Note that $D^{\ast}f$ is bounded on every compact subset of $X$,
which also follows from (3.2) (since $w$ is continuous). Therefore,
$f\in\mathrm{B}_{1}(K,Y)$, where $K\subseteq X$ is a compact set.

Let $\gamma:[a,b]\to X$ be a rectifiable curve such that $\gamma(a)=x$ and
$\gamma(b)=y$ (it follows from (3.1) that the set $\Gamma(x,y)$ is not empty).
The image curve $f\circ\gamma$ connects $f(x)$ with $f(y)$, and by Lemma 3.2
this curve is also a rectifiable one. Therefore,
$\ell(f\circ\gamma)=\int_{f\circ\gamma}{1}$ is finite (which means that the
set $\Gamma(f(x),f(y))$ is also non-empty). We now proceed to estimate the
last term.

Our first aim is to prove that the mapping $f$ is Lipschitz with respect to
the weighted distance on the first metric space, and the inner distance on the
second metric space, i.e.,

(3.3) $d_{1}(f(x),f(y))\leq B_{f}{d_{w}(x,y)},\quad x,y\in X.$

Let $\varepsilon>0$ be arbitrary. Since $w(z)$ is continuous, and since a
curve is a compact set, we may found a partition $a=t_{0}<t_{1}<\dots<t_{n}=b$
of $[a,b]$ with $\max_{1\leq i\leq n}|t_{i-1}-t_{i}|<\varepsilon$, such that

$|w(\gamma(t))-w(\gamma(t_{i-1}))|<B^{-1}_{f}\varepsilon,\quad
t\in[t_{i-1},t_{i}],\quad 1\leq i\leq n.$

Then we have

$w(\gamma(t))<w(\gamma(t_{i-1}))+B^{-1}_{f}\varepsilon,$

so it follows

$D^{\ast}f(\gamma(t))\leq
B_{f}w(\gamma(t))<B_{f}w(\gamma(t_{i-1}))+\varepsilon.$

By Lemma 3.2 we have

$\ell(f\circ\gamma|_{[t_{i-1},t_{i}]})\leq(B_{f}w(\gamma(t_{i-1}))+\varepsilon)\ell(\gamma|_{[t_{i-1},t_{i}]}),$

therefore for the corresponding integral sum of $\int_{f\circ\gamma}1$ we
derive the following estimate

$\begin{split}\sum_{i=1}^{n}1\cdot\ell(f\circ\gamma|_{[t_{i-1},t_{i}]})&\leq\sum_{i=1}^{n}(B_{f}w(\gamma(t_{i-1}))+\varepsilon)\ell(\gamma|_{[t_{i-1},t_{i}]})\\\
&\leq\sum_{i=1}^{n}B_{f}w(\gamma(t_{i-1}))\ell(\gamma|_{[t_{i-1},t_{i}]})+\varepsilon\sum_{i=1}^{n}\ell(\gamma|_{[t_{i-1},t_{i}]})\\\
&\leq\sum_{i=1}^{n}B_{f}w(\gamma(t_{i-1}))\ell(\gamma|_{[t_{i-1},t_{i}]})+\varepsilon\sum_{i=1}^{n}\ell(\gamma|_{[t_{i-1},t_{i}]})\\\
&\leq
B_{f}\sum_{i=1}^{n}{w(\gamma(t_{i-1}))}\ell(\gamma|_{[t_{i-1},t_{i}]})+\varepsilon\ell(\gamma).\end{split}$

Letting $\varepsilon\to 0$ above, we obtain

$\int_{f\circ\gamma}1\leq
B_{f}\int_{\gamma}w\Longrightarrow\inf_{\gamma\in\Gamma(x,y)}\int_{f\circ\gamma}1\leq
B_{f}d_{w}(x,y),$

so it follows

$\begin{split}d_{1}(f(x),f(y))&=\inf_{\delta\in\Gamma(f(x),f(y))}\int_{\delta}{1}\\\
&\leq\inf_{\gamma\in\Gamma(x,y)}\int_{f\circ\gamma}1\\\ &\leq
B_{f}d_{w}(x,y),\end{split}$

which proves (3.3).

We shall now prove that $f$ belongs to the class $\Lambda_{\varphi}(X,Y)$. The
condition (3.1) implies $d_{w}\leq M(\varphi\circ d_{X})$. Since $d_{Y}\leq
d_{1}$ and $B_{f}\leq C_{f}$, for every $x,y\in X$ we have

$\begin{split}d_{Y}(f(x),f(y))&\leq d_{1}(f(x),f(y))\\\ &\leq
C_{f}d_{w}(x,y)\\\ &\leq MC_{f}\varphi(d_{X}(x,y)),\end{split}$

which is the inequality we aimed to prove. ∎

The next two theorems are converse of the preceding one but for regularly
oscillating mappings with respect to the weight on the first metric space and
some conditions on the majorant. In the first one theorem we have very strong
assumptions on the majorant in order that we obtain the control of the B-norm.
In the second one we relax the conditions, but we have certain assumption on
the first metric space along with the weight. However, in the last case we are
not able to obtain any control of the norm of a mapping in $\mathrm{B}$-space
via the norm of the same mapping in $\Lambda$-space. In the proof of the last
mentioned result we use an idea of Stegbuchner [17].

######  Theorem 3.3.

Let $(X,d_{X})$ and $(Y,d_{Y})$ be metric spaces, and let a function
$\varphi\in C^{1}(0,\infty)$ be nonnegative on $[0,\infty)$ with the following
property: There exists a constant $A>0$ such that

(3.4) $\frac{\varphi(t)}{t}<A\varphi^{\prime}(t),\quad t\in(0,\infty).$

Then we have

$\|f\|_{\mathrm{B}_{\varphi^{\prime}\circ w}(X,Y)}\leq
AK\|f\|_{\Lambda_{\varphi}(X,Y)},\quad
f\in\Lambda_{\varphi}(X,Y)\cap\mathrm{RO}^{K}_{w}(X,Y)$

for every $K>0$ and every weight $w$ on the metric space $X$.

###### Proof.

Assume that $f\in\mathrm{RO}^{K}_{w}(X,Y)$ belongs to
$\Lambda_{\varphi}(X,Y)$, and let us denote
$C_{f}=\|f\|_{\Lambda_{\varphi}(X,Y)}$. Then we have

$d_{Y}(f(x),f(y))\leq C_{f}\varphi(d_{X}(x,y)),\quad x,y\in X.$

We shall show that $f\in B_{\varphi^{\prime}\circ w}(X,Y)$ and

$D^{\ast}f(z)\leq AKC_{f}\varphi^{\prime}(w(z)),\quad z\in X,$

from which follows the conclusion of our theorem.

Denote $M=AKC_{f}$, and assume contrary, i.e., that there exists $z\in X$ such
that

(3.5) $D^{\ast}f(z)>M\varphi^{\prime}(w(z)).$

Since $f\in\mathrm{RO}^{K}_{w}(X,Y)$, we have

$D^{\ast}f(z)\leq\frac{K}{r}\sup_{x\in B(z,r)}d_{Y}(f(z),f(x)),\quad
0<r<w(z),$

which implies

$\sup_{x\in
B(z,r)}d_{Y}(f(z),f(x))\geq\frac{r}{K}D^{\ast}f(z)>\frac{rM}{K}\varphi^{\prime}(w(z)).$

On the other hand, since $f\in\Lambda_{\varphi}(X,Y)$ and since $\varphi$ is
increasing, we obtain

$\begin{split}\sup_{x\in B(z,r)}d_{Y}(f(z),f(x))\leq C\sup_{x\in
B(z,r)}\varphi(d_{X}(z,x))\leq C\varphi(r),\quad 0<r<w(z).\end{split}$

Therefore, for every $r\in(0,w(z))$ the following inequality holds

$\frac{rM}{K}\varphi^{\prime}(w(z))<\sup_{x\in B(z,r)}d_{Y}(f(z),f(x))\leq
C\varphi(r).$

It follows that we must have

$A\varphi^{\prime}(w(z))<\frac{\varphi(r)}{r},\quad r\in(0,w(z)).$

However, according to our assumption on $\varphi$, this is not possible.
Indeed, by the condition (3.4) we have

$\frac{\varphi(w(z))}{w(z)}<A\varphi^{\prime}(w(z)).$

Now, since of continuity of $\varphi^{\prime}$, we obtain that if $r<w(z)$ is
sufficiently close to $w(z)$ the following inequality holds

$\frac{\varphi(r)}{r}<A\varphi^{\prime}(w(z)).$

Since we reached a contradiction, we conclude that (3.5) cannot hold for any
$z\in X$. This proves the inequality in our theorem. ∎

######  Corollary 3.4.

Let $(X,d_{X})$ and $(Y,d_{Y})$ be metric spaces, $\alpha\in(0,1)$, and $w$ a
weight on $X$. Assume that

(3.6) $(\exists M>0)(\forall x,y\in
X)(\exists\gamma\in\Gamma(x,y))\int_{\gamma}w^{\alpha-1}\leq
Md_{X}(x,y)^{\alpha}.$

Then we have

$\mathrm{RO}_{w}^{K}(X,Y)\cap\Lambda^{\alpha}(X,Y)=\mathrm{RO}_{w}^{K}(X,Y)\cap\mathrm{B}_{w^{\alpha-1}}(X,Y),$

Moreover, a regularly oscillating mapping $f\in\mathrm{RO}_{w}^{K}(X,Y)$
satisfies

$D^{\ast}f(z)\leq C_{f}w(z)^{\alpha-1},\quad z\in X,$

if and only if it satisfies

$d_{Y}(f(x),f(y))\leq C^{\prime}_{f}d_{X}(x,y)^{\alpha},\quad x,y\in X,$

where $C_{f}$ and $C^{\prime}_{f}$ are constants. If the first inequality
holds, then we can take $C^{\prime}_{f}\leq MC_{f}$. If the second inequality
holds, then the first one holds with
$C_{f}\leq\frac{2K}{\alpha}C^{\prime}_{f}$.

###### Proof.

We have just to apply Theorem 3.1 and Theorem 3.3 for the majorant
$\varphi_{\alpha}$ and the constant $A=\frac{2}{\alpha}$. ∎

As a consequence of the above result we have the following statement for
analytic mappings between normed spaces. We note that the classical theorem of
Hardy and Littlewood remains with the same constants estimate, if we replace
analytic mapping on the unit disk with analytic mapping on the unit ball in a
normed space with values in another normed space.

######  Corollary 3.5.

Let $f$ be an analytic mapping on $B_{X}(0,1)$, which is the unit ball in a
normed space $X$, with the range in another normed space $Y$. Then we have

$\|Df(z)\|\leq C_{f}(1-\|z\|_{X})^{\alpha-1},\quad z\in B_{X}(0,1),$

if and only if $f$ satisfies the Hölder condition

$\|f(z)-f(w)\|_{Y}\leq C^{\prime}_{f}\|z-w\|_{X}^{\alpha},\quad z,w\in
B_{X}(0,1),$

where $C_{f}$ and $C^{\prime}_{f}$ are constants. Moreover, there holds the
estimates $C^{\prime}_{f}\leq\frac{4}{\alpha}C_{f}$ and
$C_{f}\leq\frac{2}{\alpha}C^{\prime}_{f}$.

###### Proof.

That the unit ball $B_{X}(0,1)$ in the Hilbert space $X$ satisfies the
condition of the preceding theorem with the constant $M=\frac{4}{\alpha}$, may
be proved as in the work of Gehring and Martio [7] at the page 205. For
$x,y\in B_{X}(0,1)$, as in the cited work, one finds a curve
$\gamma\in\Gamma(x,y)$ such that

$\int_{\gamma}d^{\alpha-1}\leq\frac{\pi}{\alpha 2^{\alpha}}\|x-y\|^{\alpha}.$

where $d$ is the distance function (from the boundary of $B_{X}(0,1)$). The
constant which appears above is $\leq\frac{4}{\alpha}$, so it follows the
first inequality between constants. For the reverse estimate we only have to
note that $f$ belongs to the class $\mathrm{RO}^{1}_{d}(B_{X}(0,1),Y)$ and
apply the preceding corollary. ∎

######  Theorem 3.6.

Let $(X,d_{X})$ be a $\sigma$-compact metric space, $w$ a weight on $X$ such
that for every $\varepsilon>0$ there exists a compact set $K=K_{\varepsilon}$
with

$w(z)<\varepsilon,\quad z\in X\backslash K,$

and $\varphi\in C[0,\infty)$ continuously differentiable and concave on
$(0,\infty)$, with $\varphi(0)=0$ and $\varphi^{\prime}(t)>0$,
$t\in(0,\infty)$. If a regularly oscillating mapping $f:X\to Y$, where
$(Y,d_{Y})$ is another metric space, belongs to $\Lambda_{\varphi}(X,Y)$, then
there exists a constant $C$ such that

$D^{\ast}f(z)\leq C\varphi^{\prime}(w(z)),\quad z\in X,$

i.e., $f\in B_{\varphi^{\prime}\circ w}$.

###### Proof.

Let us denote

$Q(z)=\frac{D^{\ast}f(z)}{\varphi^{\prime}(w(z))},\quad z\in X.$

We shall firstly show that $Q$ is bounded on every compact set $K_{1}\subseteq
X$.

Assume contrary, i.e., that a compact set $K_{1}$ contains a sequence
$(z_{n})_{n\in\mathbb{N}}$ such that $(Q(z_{n}))_{n\in\mathbb{N}}$ diverge to
infinity. We can select a convergent subsequence of
$(z_{n})_{n\in\mathbb{N}}$. Denote it in the same way, and let it converge to
$z\in K_{1}$. Let $r=\frac{w(z)}{2}$. If an integer $n$ is enough big, we have
$z_{n}\in B(z,r)$. Since $\lim_{n\to\infty}w(z_{n})=w(z)$, we have

$\lim_{n\to\infty}\varphi^{\prime}(w(z_{n}))=\varphi^{\prime}(w(z))>\frac{\varphi^{\prime}(w(z))}{2}.$

Therefore, if $n$ is enough big, there holds
$\varphi^{\prime}(w(z_{n}))>\frac{\varphi^{\prime}(w(z))}{2}$. Let us denote

$r_{n}=\min\left\\{\frac{w(z_{n})}{2},\frac{r}{2}\right\\}=\min\left\\{\frac{w(z_{n})}{2},\frac{w(z)}{4}\right\\}\geq\frac{m}{4},$

where $m=\min_{\zeta\in K}w(\zeta)$. Since $f\in\mathrm{RO}^{K}_{w}(X,Y)$, we
have

$\begin{split}Q(z_{n})&=\frac{D^{\ast}f(z_{n})}{\varphi^{\prime}(w(z_{n}))}\leq\frac{D^{\ast}f(z_{n})}{\frac{\varphi^{\prime}(w(z))}{2}}\\\
&\leq\frac{2}{{\varphi^{\prime}(w(z))}}\frac{K}{r_{n}}\sup_{x\in
B(z_{n},r_{n})}d_{Y}(f(x),f(z_{n}))\\\
&\leq\frac{2K}{\varphi^{\prime}(w(z))}\frac{1}{r_{n}}\sup_{x\in
B(z_{n},r_{n})}(d_{Y}(f(x),f(z))+d_{Y}(f(z),f(z_{n})))\\\
&\leq\frac{2K}{\varphi^{\prime}(w(z))}\frac{1}{\frac{m}{4}}2\sup_{y\in{B}(z,r)}d_{Y}(f(y),f(z)),\end{split}$

which is a contradiction, since the sequence $(Q(z_{n}))_{n\in\mathbb{N}}$ is
unbounded.

If $X$ is a compact metric space, we may finish the proof at this moment. In
the sequel we suppose that this is not the case, and let
$X=\bigcup_{n\in\mathbb{N}}K_{n}$, where $(K_{n})_{n\in\mathbb{N}}$ is a
strictly increasing sequence of compact sets in the sense that $K_{1}\subset
K_{2}\subset\cdots\subset K_{n}\subset K_{n+1}\subset\cdots$

Assume that there exists a sequence $(z_{n})_{n\in\mathbb{N}}$ in $X$ such
that $\lim_{n\to\infty}Q(z_{n})=\infty$, and for the sake of simplicity, let
us denote

$Q_{n}=Q(z_{n})=\frac{D^{\ast}f(z_{n})}{\varphi^{\prime}(w(z_{n}))},\quad
n\in\mathbb{N}.$

Since $Q(z)$ is bounded on every compact subset of $X$, the set $K_{m}$,
$m\in\mathbb{N}$, may contain only a finite members of the sequence
$(z_{n})_{n\in\mathbb{N}}$, so according to our assumption on the weight $w$,
we must have $\lim_{n\to\infty}w(z_{n})=0$.

We will construct inductively a subsequence $(z_{n_{k}})_{k\in\mathbb{N}}$ of
$(z_{n})_{n\in\mathbb{N}}$: Let $n_{1}$ be such that $Q_{n_{1}}\geq 4$. Assume
that we have found $n_{k}$. Denote $\delta_{k}=\frac{w(z_{n_{k}})}{2}$. Let
$\delta_{k+1}^{\ast}\in(0,\delta_{k})$ be the unique number such that

$\varphi(\delta_{k+1}^{\ast})=\frac{\varphi(\delta_{k})}{2}$

(which exists, since $\varphi$ is continuous and increasing on $[0,\infty)$).
Now, let $n_{k+1}>n_{k}$ be such that $0<w(z_{n_{k+1}})<\delta_{k+1}$ (since
the sequence $(w(z_{n}))_{n\in\mathbb{N}}$ converge to $0$, it is possible to
find a such member), and $Q_{n_{k+1}}\geq 2^{k+2}$ (recall that
$(Q_{n})_{n\in\mathbb{N}}$ diverge to $\infty$).

For the sake of simplicity let the subsequence $(z_{n_{k}})_{k\in\mathbb{N}}$
be denote by $(z_{k})_{k\in\mathbb{N}}$. By the construction of the sequence
$(z_{k})_{k\in\mathbb{N}}$ we have

$D^{\ast}f(z_{k})\geq
2^{k+1}\varphi^{\prime}(w(z_{k})),\quad\varphi(\delta_{k+1})\leq\frac{\varphi(\delta_{k})}{2},\quad
k\in\mathbb{N}.$

Since $f$ belongs to the class $\mathrm{RO}_{w}^{K}(X,Y)$, and since
$\delta_{k}<w(z_{k})$, we have

$D^{\ast}f(z_{k})\leq\frac{K}{\delta_{k}}\sup_{x\in
B(z_{k},\delta_{k})}d_{Y}(f(x),f(z_{k})).$

Since $\varphi$ is concave on $(0,\infty)$, we have

$\varphi^{\prime}(t)\geq\frac{\varphi(s)-\varphi(t)}{s-t},\quad s>t>0.$

Therefore,

$\begin{split}\sup_{x\in
B(z_{k},\delta_{k})}d_{Y}(f(x),f(z_{k}))&\geq\frac{\delta_{k}}{K}D^{\ast}f(z_{k})\\\
&\geq\frac{\delta_{k}}{K}2^{k+1}\varphi^{\prime}(\delta_{k})\\\
&\geq\frac{\delta_{k}}{K}2^{k+1}\frac{\varphi(\delta_{k})-\varphi(\delta_{k+1})}{\delta_{k}-\delta_{k+1}}\\\
&\geq\frac{2^{k+1}}{K}\frac{\delta_{k}}{\delta_{k}-\delta_{k+1}}(\varphi(\delta_{k})-\varphi(\delta_{k+1}))\\\
&\geq\frac{2^{k+1}}{K}(\varphi(\delta_{k})-\varphi(\delta_{k+1}))\\\
&=\frac{2^{k+1}}{K}(\varphi(\delta_{k})-\frac{1}{2}\varphi(\delta_{k}))\\\
&=\frac{2^{k}}{K}\varphi(\delta_{k}).\end{split}$

However, if $k$ is enough large integer, the just obtained inequality is
opposite to the following one.

Since $f\in\Lambda_{\varphi}(X,Y)$ (recall that $\varphi$ is an increasing
function) we have

$\begin{split}\sup_{x\in
B(z_{k},\delta_{k})}d_{Y}(f(x),f(z_{k}))&\leq\|f\|_{\Lambda_{\varphi}(X,Y)}\sup_{x\in
B(z_{k},\delta_{k})}\varphi(d_{X}(x,z_{k}))\\\
&\leq\|f\|_{\Lambda_{\varphi}(X,Y)}\varphi(\delta_{k}).\end{split}$

We have just reached a contradiction. Therefore, the sequence
$(z_{n})_{n\in\mathbb{N}}$ for which $(Q(z_{n}))_{n\in\mathbb{N}}$ diverge
does not exist. ∎

### Acknowledgements

The author is thankful to the referee of this paper for useful advices, for
carefully reading, and finding several errors in the first version of the
manuscript.

## References

  * [1] K. Astala, F.W. Gehring, Quasiconformal Analogues of Theorems of Koebe and Hardy–Littlewood, Michigan Mathematical Journal 32 (1985), 99–107.
  * [2] A. Björn, J. Björn, Nonlinear Potential Theory on Metric Spaces, EMS Tracts in Mathematics 17, 2011
  * [3] O. Blasco, P. Galindo, A. Miralles, Bloch Functions on the Unit Ball of an Infinite Dimensional Hilbert Space, Journal of Functional Analysis 267 (2014), 1188–1204.
  * [4] Ch.-H. Chu, H. Hamada, T. Honda, G. Kohr, Bloch Functions on Bounded Symmetric Domains, Journal of Functional Analysis 272 (2017), 2412–2441.
  * [5] P.L. Duren, Theory of $H^{p}$ Spaces, Pure and Applied Mathematics, Vol. 38, Academic Press, New York, 1970.
  * [6] F.W. Gehring, O. Martio, Quasi-disks and the Hardy–Littlewood Property, Complex Variables 2 (1983), 67–78.
  * [7] F.W. Gehring, O. Martio, Lipschitz Classes and Quasi-conformal Mappings, Annales Academiæ Scientiarum Fennicæ 10 (1985), 203–219.
  * [8] G.H. Hardy, J.E. Littlewood, Some Properties of Fractional Integrals, II, Mathematische Zeitschrift 34 (1932), 403–439.
  * [9] D. Kalaj, Schwarz lemma for Holomorphic Mappings in the Unit Ball, Glasgow Mathematical Journal 60 (2018), 219–224.
  * [10] P. Koskela, V. Manojlović, Quasi-Nearly Subharmonic Functions and Quasiconformal Mappings, Potential Analysis 37 (2012), 187–196.
  * [11] V. Lappalainen, On Liph-extension Domains, Annales Academiæ Scientiarum Fennicæ Dissertationes 56 (1985), 1–52.
  * [12] M. Marković, Representations for the Bloch Type Semi-norm of Fréchet Differentiable Mappings, Journal of Geometric Analysis 31 (2021), 7947–7967.
  * [13] C.A. Nolder, D.M. Oberlin, Moduli of Continuity and a Hardy–Littlewood Theorem, Complex Analysis, Joensuu 1987, Lecture Notes in Mathematics, Vol. 1351, Springer, Berlin, 1988, 265–272.
  * [14] C.A. Nolder, A Quasiregular Analogue of a Theorem of Hardy and Littlewood, Transactions of the American Mathematical Society 331 (1992), 215–226.
  * [15] M. Pavlović, J. Riihentaus, Classes of Quasi-nearly Subharmonic Functions, Potential Analysis 29 (2008), 89–104.
  * [16] M. Pavlović, Function Classes on the Unit Disc (second edition), De Gruyter Studies in Mathematics, volumen 52, De Gruyter, Berlin, 2019.
  * [17] H. Stegbuchner, On Some Extensions of a Theorem of Hardy and Littlewood, Annales Academiæ Scientiarum Fennicæ 7 (1982), 113–117.
  * [18] K. Zhu, Distances and Banach Spaces of Holomorphic Functions on Complex Domains, Journal of London Mathematical Society 49 (1994), 163–182.


# Accurate Eye Tracking from Dense 3D Surface Reconstructions using Single-
Shot Deflectometry

Jiazhang Wang1  Tianfu Wang2  Bingjie Xu3
Oliver Cossairt1,3  and Florian Willomitzer4  1 Department of Electrical and
Computer Engineering, Northwestern University, Evanston, IL, 60208
2Department of Computer Science, ETH Zürich, Zürich, Switzerland, 8092
3Department of Computer Science, Northwestern University, Evanston, IL, 60208
4Wyant College of Optical Sciences, University of Arizona, Tuscon, AZ, 85721
<EMAIL_ADDRESS>
###### Abstract

Eye-tracking plays a crucial role in the development of virtual reality
devices, neuroscience research, and psychology. Despite its significance in
numerous applications, achieving an accurate, robust, and fast eye-tracking
solution remains a considerable challenge for current state-of-the-art
methods. While existing reflection-based techniques (e.g., "glint tracking")
are considered the most accurate, their performance is limited by their
reliance on sparse 3D surface data acquired solely from the cornea surface.

In this paper, we rethink the way how specular reflections can be used for eye
tracking: We propose a novel method for accurate and fast evaluation of the
gaze direction that exploits teachings from single-shot phase-measuring-
deflectometry (PMD). In contrast to state-of-the-art reflection-based methods,
our method acquires dense 3D surface information of both cornea and sclera
within only one single camera frame (single-shot). Improvements in acquired
reflection surface points (“glints”) of factors $>3300\times$ are easily
achievable. We show the feasibility of our approach with experimentally
evaluated gaze errors of only $\leq 0.25^{\circ}$, demonstrating a significant
improvement over the current state-of-the-art.

††journal: 3dim††articletype: Research Article

##  1 Introduction

Eye tracking is an enabling technology that is widely involved in different
fields of application. In virtual/augmented reality (VR/AR), it enables
important functions, such as foveated rendering, compensation of the vergence-
accommodation conflict, or interactions with the user interface virtual
avatars[1]. In clinical applications, it helps with the diagnosis of disorders
[2], or facilitates improved surgical training[3]. Moreover, eye tracking is
frequently used in psychology research [4] and cognitive neuroscience [5].
Many of the applications mentioned above utilize commercially available eye-
trackers. Although the reported error metrics for these devices vary, their
best reported error margin mostly lies in the $0.5^{\circ}\sim 1^{\circ}$
range [1]. While this number sounds impressive at first glance, it still
leaves room for improvement for some applications. For example, as VR/AR
headsets might reach the targeted “retinal resolution” in the future [6], a
$1^{\circ}$ error would correspond to $>60$ pix, which could still pose
challenges when the navigation through the user interface should happen purely
gaze-based with “mouse precision”. At larger standoff distances of e.g. $15m$,
a $1^{\circ}$ gazing error leads to a lateral localization error of $\sim
25cm$, which is already too large, e.g., to exactly determine a focused
product in a supermarket shelf or to interact with a virtual avatar in a
larger crowd.

Current state-of-the-art optical eye tracking approaches can be divided into
two groups: image-based methods [7, 8, 9, 10, 11] and reflection-based
methods[12, 13, 14, 15, 16, 17]. Image-based methods detect features such as
the pupil, iris, veins, or limbus in 2D images of the eye and use them to
estimate the gaze direction. This can be done, e.g., by exploiting geometrical
relations in the captured eye image (iris center, pupil eccentricity,
etc.)[10, 9, 18, 8], or via deep learning[11, 19, 20, 21]. The density of 2D
image features is, however, relatively sparse for all cases, which potentially
limits the quality of the reconstruction. Moreover, the transparent property
of the cornea makes it difficult for image-based methods to extract accurate
features in these regions (e.g., from the underlying iris) since the
refraction of light at the cornea surface is different at different viewing
angles[22].

In contrast, state-of-the-art reflection-based methods directly or indirectly
measure the 3D coordinates of the cornea surface and use the respective
information to calculate the gaze. ’Glint tracking’ is the most prominent
method in this class[16, 12, 23, 14]: Around 2-12 infrared point light sources
are spatially arranged in front of the eye (e.g., in a circle) to be reflected
by the cornea surface. Combined with other measured 2D features, such as the
pupil center, the gaze direction can be estimated. For those methods, it can
be intuitively understood that more acquired 3D surface information about the
eye generally leads to higher gaze reconstruction accuracy [14]. However,
current methods do not acquire more than $\sim 12$ sparse surface points. This
relatively low number of data points is caused by the conceptual design of the
respective glint tracking method: Its susceptibility to ambiguities makes it
difficult to increase the number of point sources.

In this paper, we improve the accuracy of reflection-based eye tracking by
radically increasing the number of measured eye surface points from $\sim 12$
to over $40,000$. We pair this significant increase in acquired information
with novel sophisticated algorithms that calculate the gaze direction fast and
computationally efficient. Our method to acquire 3D information about the eye
surface is based on phase measuring deflectometry (PMD)[24]. PMD was
originally invented as an optical metrology method for the dense 3D
reconstruction of specular objects, like lenses or astronomical mirrors [25,
26, 27, 28, 29]. Modern well-calibrated PMD setups can reach sub-micron
accuracy using only inexpensive off-the-shelf equipment (display and
camera)[24, 27, 30, 31]. For that reason, PMD is well known in the metrology
community as the big competitor to highly accurate optical interferometry
systems. Recently PMD has been used, e.g., for cultural heritage analysis[32,
33], or to measure the topography of the human cornea[34] for medical
diagnosis. Its special design for specular surfaces makes PMD an ideal
technique for eye surface measurements which yields several advantages
compared to techniques designed to measure matte surfaces, such as fringe
projection profilometry[35].

In a PMD setup, an off-the-shelf display showing a sinusoidal pattern serves
as the illumination source. The reflection of the display pattern over the
object surface is observed with a camera. Each pixel on the display can be
seen as an independent single-point light source that can be used to acquire
information about a surface point. For example, a fairly "low resolution"
$1000\times 1000~{}pix$ display paired with a "low resolution" $1Mpix$ camera
can, in theory, already acquire $1Mio$ independent 3D surface points. Although
these very high numbers are not achievable in practice due to geometrical
constraints, our prototype setup experimentally demonstrates measurements with
more than 40,000 acquired surface points. Compared to glint tracking with 12
points light sources, this is an improvement in acquired surface points of
$3300\times$, which results in experimentally evaluated gaze errors between
$0.12^{\circ}$ and $0.25^{\circ}$ at a precision below $0.1^{\circ}$ (see Tab
1).

Besides accuracy and precision, another important factor in eye tracking is
speed. Standard PMD principles normally rely on temporal phase shifting, which
requires a temporal sequence of multiple camera images to calculate a single
3D view [36]. For fast object movements (like those of human eyes), such a
temporal phase-shifting procedure is commonly not motion-robust and would lead
to motion artifacts in the final result. For this reason, we exploit a method
to measure the complete 3D information about the human eye in single-shot. Our
implemented approach is based on 2D continuous wavelet transforms for phase
measurement [37]. Due to its single-shot character, it allows for fully
motion-robust measurements, regardless of the moving speed of the eye. We
summarize the contributions of our paper as follows:

  * •

We introduce a new reflection-based method for eye tracking that is fast,
accurate, and precise. Our method exploits dense 3D surface information of the
eye (such as surface shape and normals) to estimate the gaze. It does not rely
on secondary eye features (such as pupil diameter or veins) and does not
require a rendered "eye base model".

  * •

We develop a novel eye-tracking prototype setup based on single-shot stereo
deflectometry [37] to capture dense and precise surface information within
each camera frame. To the best of our knowledge, this is the first time that
deflectometry has been used for eye tracking.

  * •

We introduce a novel calibration approach for our prototype system. The
approach stands out by its flexibility and ease of use. Its potential to
improve other deflectometry systems goes far beyond the application scope of
this paper.

  * •

We demonstrate the feasibility of our approach with quantitative evaluations
of real experiments and simulations. Our experiments showcase evaluated gaze
errors $\leq 0.25^{\circ}$ with a precision of the calculated gaze $\leq
0.1^{\circ}$. These results demonstrate a significant improvement over the
current state of the art.

Figure 1: Deflectometry based eye-tracking (a) Schematic of our single-shot
deflectometry-based gaze estimation approach: Two cameras observe the specular
reflection of a display over the eye surface. Eye surface shape and normal
information is calculated from the deformation of the pattern in the camera
images. (b) Tracing the measured surface normals back towards the eye delivers
cornea and sclera center, which are used to define the eye’s optical axis and
gazing directing (simulated result shown for better explicability). (c) The
feasibility of the proposed method is demonstrated with real-world
measurements of realistic eye models and human eyes in-vivo.

##  2 Methods

Our proposed eye-tracking method leverages accurate and dense 3D measurements
of the eye surface. We employ a specifically tailored deflectometry setup to
measure the shape and normal map of the eye surface: Instead of using sparse
point lights ("glints"), we illuminate the eye with an extended display. Each
pixel on the display serves as an individual point light source, and the
observed reflection of the display over the eye surface covers both the cornea
and sclera region of the eye (see Fig.1). To solve the inherent normal-depth
ambiguity[24] and to further enlarge the measured surface area (see Fig.1), we
implement a second camera ("stereo-deflectometry"). To facilitate the
potential for low latency measurements, we use a single-shot deflectometry
approach that exploits one single cross-sinusoid pattern (Fig.1.(a) and (c))
rather than classical phase-shifting deflectometry that requires multiple
sequentially captured patterns. After retrieving the depth map and normal map
of the eye surface, we calculate the gaze direction as follows: Given the
assumption that the shape of the cornea and sclera can be represented by two
spherical surfaces with different radii, one can back-trace the measured
surface normals towards the center of the eye where they aggregate at two
points: The center of the cornea sphere and the center of the sclera sphere.
The gazing vector is the vector that connects these two points (see Fig.1.(b)
and (c)).

In the following, we will provide a detailed explanation of our applied
procedures outlined above. We will start with classical phase-shifting
deflectometry and single-shot deflectometry, followed by our method for
iterative eye surface reconstruction(section 2.1) and gaze estimation (section
2.2). Eventually, we will introduce a novel system calibration method that
significantly improves the quality of our measurements and simplifies the
calibration process (section 2.3).

###  2.1 Deflectometry: Sequential captures vs Single-shot

Deflectometry is an established reflection-based method to reconstruct the
surface of specular objects. In a classical deflectometry setup (Fig.2.(a)), a
camera observes a display over the specular object surface. Depending on the
shape of the object surface, the image of the known display pattern is
deformed in the camera image. From this deformation, 3D information about the
object can be calculated. The deformation of the pattern is commonly
quantified by identifying so-called corresponding points between the display
image and the camera image. Eventually, rays are traced between a display
pixel and its corresponding camera pixel to calculate the surface normals (and
later the shape via integration). As each display pixel emits light in all
directions, one single pair of corresponding points leads to many possible
combinations of surface normal and surface height. A common solution to this
“depth-normal ambiguity problem”[36] is to add a second camera and turn the
system into a “stereo deflectometry” system[24].

A crucial step in the deflectometry process is to obtain the correspondence
between the display and camera pixels. The common procedure is to encode the
display pixel positions in a pattern, and eventually decode them on the camera
chip by analyzing the pattern. This is commonly done over the phase of
displayed sinusoidal intensity patterns of known frequency.

####  2.1.1 Correspondence evaluation in classical phase measuring
deflectometry (PMD)

In PMD, the phase values for correspondence evaluation are obtained by
temporally shifting vertical and horizontal sinusoidal patterns on the
display. Patterns with horizontal stripes (as shown in Fig. 2.(e)) encode the
vertical display coordinate $y_{D}$, while patterns with vertical stripes
encode the horizontal display coordinate $x_{D}$. The reflection of the
pattern over the object surface is imaged with the camera. The intensity in
each camera pixel $(x_{c},y_{c})$ can be expressed as

$I(x_{c},y_{c})=A(x_{c},y_{c})+B(x_{c},y_{c})\cdot
cos(\phi_{y}(x_{c},y_{c})+\Delta\phi)~{}~{}.$ (1)

$A(x_{c},y_{c})$ and $B(x_{c},y_{c})$ are terms that contain information about
the unknown reflectivity of the respective surface point, the bias
illumination, and the unknown background illumination at this point.
$\phi_{y}(x_{c},y_{c})$ is the vertical phase value that is needed to
establish correspondence. In classical PMD, $\phi_{y}(x_{c},y_{c})$ is
retrieved via temporal phase shifting, e.g., by sequentially changing
$\Delta\phi$ to $0,\frac{\pi}{2},\pi,\frac{3\pi}{2}$ and acquiring an image
$I_{k}(k=1,2,3,4)$ for each of those phase shifts (“four-phaseshift-method")
(see Fig. 2.(e) and (f)) [24, 36]. Eventually, $\phi_{y}(x_{c},y_{c})$ is
calculated by

$\phi_{y}(x_{c},y_{c})=atan(\frac{I_{4}-I_{2}}{I_{3}-I_{1}})~{}~{}.$ (2)

Figure 2.(g) shows the retrieved vertical phase map $\phi_{y}$ in the camera
image, which uniquely encodes the corresponding vertical display coordinates
(Fig.2.(h)) after unwrapping. The horizontal phase map $\phi_{x}$ and the
horizontal display coordinates are retrieved in an analog fashion by phase-
shifting vertical sinusoidal patterns. Eventually, the surface normal is
calculated from the obtained correspondence between display pixels and camera
pixels via ray tracing (see Fig. 2.(a)), where height-normal ambiguities are
resolved by exploiting information from the second camera, as described in
sec. 2.1.3.

Multi-phase shift procedures like the discussed four-phaseshift-method provide
high-quality results but have one severe drawback for the purpose of this
paper: Several sequentially captured camera images (8 in this example) are
required to capture one single 3D model. During this sequence, the object is
not allowed to move. As the human eye is in constant fast motion, this method
is not feasible for our purpose in practice. For this reason, we utilize a
procedure to facilitate high-quality deflectometry measurements in single-
shot.

Figure 2:  Single-shot and multi-shot deflectometry for eye surface
measurement. (a) Schematic of our setup: a display with a known pattern
illuminates the specular object. The camera observes the deformation of the
pattern after reflection. (b) Single-shot deflectometry raw data: Realistic
eye model with reflection of cross-sinusoid pattern. (c) and (d) Respective
retrieved phase map and unwrapped display-camera correspondence map. (e)
Multi-shot deflectometry raw data for comparison: Realistic eye model with
reflection of a sinoidal pattern with horizontal stripes. (f) The four-
phaseshifting method can be used to retrieve the phase map (g) and eventually
the display-camera correspondence map (h) after unwrapping.

####  2.1.2 Correspondence evaluation via single-shot deflectometry

To obtain the display-camera correspondence information in single-shot, the
unidirectional sinusoidal pattern on the display is replaced with a crossed
sinusoidal pattern, i.e., one sinusoid in the horizontal direction, overlaid
with a sinusoid in vertical direction (see Fig. 2.(b) or pattern schematic in
Fig. 1.(a)). In analogy to Eq. 1, the intensity in each camera pixel
$(x_{c},y_{c})$ can be expressed by the term

$I(x_{c},y_{c})=A(x_{c},y_{c})+B(x_{c},y_{c})[cos(\phi_{x}(x_{c},y_{c}))+cos(\phi_{y}(x_{c},y_{c}))]~{}~{}~{},$
(3)

which now contains both the horizontal and vertical phase values
$\phi_{y}(x_{c},y_{c})$, $\phi_{x}(x_{c},y_{c})$, needed to establish
correspondence. A common method to retrieve these phase values in single-shot
exploits single-sideband demodulation in Fourier space[38, 39, 34, 40].
However, as the frequency evaluation with a Fourier transform is a global
operation, the respective method suffers limitations of the allowed maximal
object bandwidth. A violation of these limitations leads to artifacts in the
reconstruction. In our case, these artifacts are caused by the high surface
frequencies at the limbus region of the eye. For this reason, we apply a
recent method to locally retrieve the phase value in single-shot [37]. The
approach is based on 2D continuous wavelet transforms, and is able to locally
evaluate the phase of the crossed fringe pattern. This works by convolving the
camera image with a morlet wavelet $\Psi_{s,u_{x},u_{y}}$ that can be varied
in scale and translation:

$\Psi_{s,u_{x},u_{y}}(x_{c},y_{c})=\frac{1}{s\sqrt{\pi f_{b}}}\exp(2\pi
f_{0}(\frac{x_{c}-u_{x}}{s})i-\frac{1}{f_{b}}((\frac{x_{c}-u_{x}}{s})^{2}+(\frac{y_{c}-u_{y}}{s})^{2}))$
(4)

Here $s$ is the scale parameter, $u_{x},u_{y}$ are the translation parameters
in both directions, $f_{0}$ is the center frequency of the wavelet and $f_{b}$
is the bandwidth. As the object shape does not only influence the local
frequency of the pattern in the camera image, but also its direction, the
wavelet additionally needs to be rotated to find the maximal wavelet modules:

$\theta_{m},s_{m}=\mathop{\arg\max}_{\theta,s\in\mathbf{R}}\lvert\iint_{\mathbf{R^{2}}}I(x_{c},y_{c})\Psi_{\theta,s,u_{x},u_{y}}(x_{c},y_{c})dx_{c}dy_{c}\lvert$
(5)

Eventually, the horizontal phase value $\phi_{x}(x_{c},y_{c})$ is determined
by the phase angle of the complex valued wavelet transform. For the vertical
phase value $\phi_{y}(x_{c},y_{c})$, the wavelet is rotated by $90^{\circ}$
and the process is repeated. The retrieved vertical phase map
$\phi_{y}(x_{c},y_{c})$ is shown in Fig.2(c) and the respective correspondence
map is shown in Fig.2(d).

####  2.1.3 Surface normal and shape reconstruction

After the correspondence between camera image and display has been obtained,
the eye surface can be reconstructed. As shown in Fig. 2.(a), the surface
normal can be obtained by tracing rays between two corresponding points over
the specular object surface. However, this mapping is not unique. If the
object surface changes its distance to the camera along the ray of sight, the
same pair of corresponding points would lead to a different surface normal.
This problem is commonly referred to as the ‘depth-normal-ambiguity problem’
[36]. We solve this problem by exploiting information from a second camera
that is added to the system (“stereo deflectometry”) [24]. The basic idea:
Each camera has its own set of possible surface normal-height combinations,
but both sets only match for one combination, which is the true height and
normal of the surface.

Calculating every single surface point via stereo deflectometry is
computationally expensive, which could restrict the potential future real-time
implementation of our system. Moreover, only those surface points can be
reconstructed that reflect a display pixel back in both cameras, which limits
the effective coverage (the total surface area that can be measured) of the
method. To speed up the reconstruction and increase the effective coverage, we
only use stereo deflectometry to reconstruct surface normal and height at a
few sparse "anchor points" of the surface. Eventually, we obtain a dense
surface reconstruction by iterative slope integration [41]: A first estimate
of the surface height map is initialized as best fit surface through all
anchor points. We calculate the first iteration of the surface normal map
“with respect to” this initial surface height map. This means that the normal
in each pixel is calculated by using the respective height value of the
initial surface height map. Eventually, the next iteration of the surface
height map is calculated by integrating the calculated normal map via zonal
reconstruction (see Supplementary Material and [42] for more information). A
second iteration of the normal map is then calculated with respect to the
updated height map, and so on. This process is repeated until the calculated
iterations of surface height map and normal map converge, which typically
happens after 2-3 iterations. The result is a highly accurate representation
of the measured eye surface (including surface normals), which only requires
minimal overlap of measured surface points between both cameras. For example,
the cornea surface can be reconstructed from camera 1, while the sclera
surface can be reconstructed from camera 2 (see Fig.6.(b)$\sim$(d)).

###  2.2 Gaze estimation

The gaze direction is estimated from the obtained dense eye surface 3D
reconstruction. Our calculation is based on the common assumption that cornea
and sclera can be approximated by two spheres with different radii (see
Fig.3.(b))[16, 15, 43, 44]. The cornea center, sclera center, and pupil center
are located on the optical axis of the eye. In each camera pixel of our
effective measurement field, we have acquired information about the height and
the normal vector of the respective surface point. We calculate the cornea
center and sclera center by tracing the aquired surface normals back inside
cornea and sclera. For a perfect measurement and for a perfectly spherical
cornea and sclera, those normals intersect exactly at the center of cornea and
sclera (see simulated measurement and evaluation in Fig. 1.(b) and Fig.
3.(b)). In practice, i.e., for real “imperfect” measurements, we detect the
closest points of all backtraced normals to determine the center points and
eventually interpolate the optical axis, while simultaneously rejecting
outliers. Those outliers can be caused by noise on the captured data or may be
connected to a normal that originates from a surface part with rapid slope
variation, such as the limbus region (the region where the cornea meets the
sclera). We emphasize that, strictly spoken, the retrieved optical axis of the
eye is not the visual axis that defines the gazing direction. As the fovea is
the spot on the retina that is responsible for the sharpest vision, the actual
visual axis slightly differs from the optical axis by an angle $\kappa$. In
real-world eye tracking applications, this angle $\kappa$ can be easily
calibrated, e.g., by observing a moving point on a computer display [15, 17,
23]. As such a $\kappa$-angle calibration process is outside the scope of this
work, we here assume the angle $\kappa$ to be a known constant angular offset
for the remainder of this paper, meaning that we can directly calculate the
gaze direction from the reconstructed optical axis. For this reason, we will
further refer to “calculate the gazing direction” in our explanations;
although strictly spoken, we calculate the optical axis of the eye.

Figure 3: Gaze estimation (a) We assume that both cornea and sclera surface
are spherical in shape. The optical axis and gaze direction differ by the
fixed angle $\kappa$, which is assumed to be known for the purpose of ths
paper. (b) Our method calculates the optical axis of the eye by connecting
cornea center and sclera center. The respective center points are obtained by
tracing back the measured surface normal towards the middle of the eye
(simulated result shown for better explicability).

###  2.3 Easy and flexible deflectometry system calibration method

The accuracy of deflectometry measurements heavily relies on a high quality
system calibration [24, 36, 45, 46], which includes the intrinsic and
extrinsic calibration of all involved cameras, as well as the extrinsic
calibration of the entire camera-display system. Zhang’s method[47] is well-
established for solving camera intrinsic and extrinsic calibration, where a
fixed board with a known pattern (e.g., a checkerboard pattern) is utilized as
calibration target. In practice, the manufacturing quality of the checkerboard
highly impacts the calibration result, especially for small field-of-view
(FoV) applications.

During the extrinsic calibration, the relative pose of both cameras and
display is calibrated. As the cameras do not image the display directly,
state-of-the-art deflectometry calibration methods use a planar mirror to
reflect the display towards the camera (Fig.4.(a)). Each camera sees a virtual
image of the display, that can be used to calculate the real display position.
The required pose of the mirror can be obtained from markers that are placed
on the mirror and observed by each camera. For a highly accurate pose
estimation, the markers should be densely distributed across the mirror
surface. However, this would occlude parts of the reflected display pattern,
leaving specific regions of the display uncalibrated. For this reason, the
markers are normally placed at the edge of the mirror, which leads to less
accurate calibration results. Moreover, each marker still occupies space in
the FoV of each camera which is lost for the actual surface measurement(see
Fig. 4.(b)).

Figure 4: Novel method for calibration of deflectometry setups (a) and (b)
Shortcomings of the "classical" method[33, 46] The display is observed over an
auxiliary mirror, whose position in space is determined by attached markers.
For precise pose estimation, the markers need to cover a large part of the
available FoV, which occludes the display and/or restricts the effective
measurement field. (c) and (d) Novel calibration method: A specular phone
screen replaces the mirror. In a first step, a dense marker pattern is
displayed on the phone screen, which is used to determine the screen’s
position in space (c). Eventually, the screen is switched off functions as a
mirror that reflects the display and hence allows for a full-field calibration
of the whole display without occlusions (d).

The key idea of our novel calibration method(see Fig 4.(c)(d)) is to replace
the static marker-mirror by a switchable specular screen. In the “on-state”,
the screen displays a known calibration pattern (e.g., a checkerboard pattern)
that densely fills the entire FoV of both cameras. We evaluate the accurate
pose of the screen in space from the respective camera images. Eventually, the
screen switches to “off-state”: No pattern is displayed and the screen acts as
mirror to observe the deflectometry display. This method allows to calibrate
the display position without any marker occlusion while simultaneouly
exploiting the camera’s FoV in a most efficient way, which ultimately leads to
measurements at much higher accuracy compared to conventional calibration
methods[46].

##  3 Results

We demonstrate the feasibility of our approach in real experiments. We first
quantify the quality of our surface and normal reconstruction method by
measuring a spherical bearing ball, which roughly has the same radius as a
human eye. Eventually, we evaluate the mean relative gazing errors for fixed
rotation positions of an realistic real-world eye model. The evaluated errors
are all $\leq 0.25^{\circ}$, with precision values $<0.1^{\circ}$. Moreover,
we perform a qualitative gaze direction estimation with our method on a real
human eye in-vivo. The following subsections describe our experiments and
results in detail. A further evaluation of our method within the framework of
a realistic simulation can be found in the supplementary material.

###  3.1 Quantitative evaluation of the surface reconstruction quality

Figure 5: Quantitative evaluation of surface reconstruction quality. (a) The
system accuracy is evaluated by measuring a spherical bearing ball with the
known radius (close to the radius of a real eye). (b) Reconstructed ball
surface with measured normal vectors. The calculated radius is 11.92mm, with
the ground truth radius being 12mm. The standard deviation of the distance of
the calculated sphere center to all normals is only $7\mu m$

As described above, our method relies on a high-quality deflectometric surface
reconstruction for a reliable gaze estimation. For this reason, we first
separately evaluate the quality of our surface reconstruction before
evaluating the gaze estimation. As measurement object we use a metallic
bearing ball with a well-defined radius that is subject to very little
manufacturing tolerances (McMaster-Carr 1598K39, $R=12mm\pm 0.00125mm$ ). Our
prototype setup to perform all measurements is shown in Fig. 6.(a). We use two
machine vision cameras (Model: flea3 fl3-u3-13s2c) equipped with 9 mm
objectives that face the object, which is illuminated by an iPhone 12 pro
display (resolution $2532\times 1170$ pix). The system is calibrated using the
novel calibration method described above.

We use the single-shot stereo-deflectometry method with iterative surface
integration (detailed in section 2.1) to measure and reconstruct the spherical
surface of the bearing ball. We capture a total of $56,140$ surface points and
with respective surface normals in only one single-shot. An exemplary image
from camera 1 is shown in Fig. 5.(a). The respective surface reconstruction
(including normals) is shown in Fig. 5.(b). To quantify the quality of our
reconstruction we calculate two metrics: The radius of the reconstructed
surface and the precision at which the back-traced normals meet at one point
(the center of the sphere). The radius of our reconstructed surface is
calculated to $R_{m}=11.92mm$, which deviates only $80\mu m$ from the
specified radius of the bearing ball. To evaluate the precision of the
calculated surface normals, we back-trace all surface normals towards the
center of the sphere and estimate the sphere center by calculating their best-
fit intersection point. Eventually the precision $\sigma_{m}$ is calculated
via the standard deviation of the absolute distance of the backtraced normals
to the calculated center. We obtain a standard deviation of only
$\sigma_{m}=7\mu m$.

###  3.2 Quantitative evaluation of the gaze estimation

Rotation position $a$ |  -4∘ |  -2∘ |  0∘ |  2∘ |  4∘
---|---|---|---|---|---
Mean relative error $\epsilon_{0}$ |  0.14∘ |  0.25∘ |  0∘ |  0.13∘ |  0.12∘
Precision $\sigma_{a}$ |  0.08∘ |  0.06∘ |  0.04∘ |  0.07∘ |  0.02∘
Table 1: Evaluation of the mean relative gaze error and the respective
precision values for a real-world measurement of a realistic eye model at
different gaze directions.

Quantifying the performance of our gaze estimation with high reliability
requires the design of a systematic experiment. Evaluations of gazing
directions using real human subjects can be too unreliable for our purposes,
mainly for two reasons: (1) one possibility of such an evaluation might
require a third-party eye tracker to deliver the ground truth for our
measurement. The estimation error of this third-party eye tracker might be
higher than the error of our method, which would lead to wrong conclusions.
(2) another possible procedure requires the subject to look at a certain
signal on the screen (e.g., a moving point) to calculate the ground truth
gazing direction[15, 48]. Those methods require very accurate calibration and
rely on the assumption that the human subject is not moving during the entire
duration of the experiment and also really looks at the signal for most of the
time, which introduces another source of error.

For these reasons, we designed an experiment that largely eliminates different
sources of “third-party" errors, and focuses only on the accuracy of our
system. Instead of a human subject, we use a realistic real-world eye model
(with distinct cornea and sclera) that is mounted on a high-precision rotation
stage. An image of our eye model mounted in our setup is shown in Fig. 6.(a).
We rotate the eye model to several positions of our stage to accurately change
the gazing direction by known angles. We emphasize that the ground truth
gazing direction for the eye model on the stage is still unknown and can also
not be evaluated in a reliable way (see above). For this reason, we define a
“default” gazing direction by rotating the eye model to $0^{\circ}$ on the
rotation stage, and eventually measure the gaze directions as relative angles
w.r.t. the default gaze direction. For our evaluation, we rotate the stage to
different rotation positions $a$, comprised of $-4^{\circ}$, $-2^{\circ}$,
$0^{\circ}$, $2^{\circ}$, and $4^{\circ}$. We capture a single-shot
measurement at one position and rotate to the next position until we have
acquired a total of 20 measurements at each position. On average, we capture
$42,673$ surface points with respective surface normals in each single-shot
measurement at each rotation position. We emphasize that we never capture two
consecutive measurements without having rotated the eye model. For each
measurement at rotation position $a$ we calculate the relative gazing vector
w.r.t. to the “default” $0^{\circ}$ position. Eventually, we calculate the
mean relative error $\epsilon_{0^{\circ}}$ at each rotation angle $a$ as
follows:

$\epsilon_{0^{\circ}}=||\bar{\theta_{a}}-\bar{\theta_{0^{\circ}}}|-|a||$ (6)

Here $\bar{\theta_{a}}$ is the mean evaluated gazing angle of 20 performed
measurements at rotation angle $a$, and $\bar{\theta_{0^{\circ}}}$ is the mean
evaluated gazing angle of 20 performed measurements at rotation angle
${0^{\circ}}$ ( the “default” gazing angle). The results of this evaluation
are summarized in Tab. 1. The mean relative gazing error
$\epsilon_{0^{\circ}}$ is $\leq 0.25^{\circ}$ for all measurements with most
errors even being below $0.15^{\circ}$.

In a second step, we evaluate the precision $\sigma_{a}$ of our method by
calculating the standard deviation of our evaluated gazing angles at each
rotation position $a$:

$\sigma_{a}=\sqrt{\frac{\sum^{n}_{i=1}(\theta_{a,i}-\bar{\theta_{a}})^{2}}{n}}$
(7)

Figure 6: Quantitative real-world experiments on a realistic eye model.(a)
Prototype setup and eye model mounted on a rotation stage. (b) and (c)
Captured eye model images, overlayed with the respective evaluation result.
The cornea and sclera are reconstructed from the images of Camera 1 and Camera
2, respectively, to ensure maximal possible coverage. Red arrows are sampled
reconstructed surface normals. Blue arrows are back-traced normal vectors that
intersect at the cornea and sclera center. (d) Combining both evaluation
results in one coordinate system allows us to determine the eye’s optical axis
by connecting the calculated cornea and sclera center. Shown is the image of
camera 1 overlaid with the final evaluation result.

$\theta_{a,i}$ is the $i^{th}$ measurement at rotation position $a$, and
$n=20$ in this experiment. As also shown in Tab. 1 the evaluated gazing
precision $\sigma_{a}$ is $\leq 0.08^{\circ}$ for all measurements.

We note again that it is generally hard to compare the quantitative evaluation
of different eye tracking methods, as different papers use different error
metrics and evaluation procedures. According to [1], it can be said in general
that the best reported errors of current eye tracking methods vary within the
$0.5^{\circ}\sim 1^{\circ}$ range[1], which attests our method a significant
improvement over the current state of the art.

###  3.3 Qualitative measurement on a real human eye in-vivo

In-vivo measurements on real human eyes differ from systematic measurements of
a model eye under lab conditions: The human eye surface might be (partially)
occluded by the eyelids or lashes. Also, the human sclera surface is, in
reality, not as specular as the cornea surface and diffusely scatters parts of
the incoming light, which leads to a contrast reduction in the imaged
pattern[49]. Moreover, the sclera surface shows “little bumps” wherever a
blood vessel is located close to the surface, which represents a significant
deviation from its overall spherical shape.

To demonstrate that our method can still work under these conditions, we
perform a qualitative gaze evaluation on a real human eye in-vivo. The result
is shown in Fig. 7. The pattern is clearly visible on both sclera and cornea
and we can obtain a dense deflectometric surface and normal reconstruction.
When tracing back the normals towards the sclera and cornea center, the
“bumpy” sclera surface parts as well as the highly curved limbus region,
produce normals that do not intersect cornea or sclera close to their center.
As discussed, we treat those normals as “outliers” in our evaluation process,
and reject them via a least square optimization process.

The reconstruction after outlier rejection is shown in Fig. 7.(a)and(b). The
backtraced normals nicely intersect at two distinct center points (see Fig.
7.(c)). When plotting the reconstructed gazing vector in the coordinate system
of camera 1 and overlaying both images (Fig. 7.(c)), it can be seen that the
vector passes through the pupil center and points in the direction of the
optical axis. This means that, within the boundaries of a qualitative
evaluation, our method produces reasonable results on real human eyes in-vivo.
As discussed above, a quantitative evaluation on real human eyes that is free
from third-party error sources will require substantial experimental effort
paired with extensive user studies on multiple human subjects, which is
outside the scope of the contributions of this paper.

Figure 7: Qualitative gaze estimation on real human eye in-vivo (a) and (b)
Camera images of Camera 1 and Camera 2, overlaid with the respective cornea
and sclera reconstructions. (c) Final evaluation result of the gaze direction.
The calculated gaze vector passes through the pupil center and points in the
visually correct direction.

##  4 Summary, discussion, and outlook

This paper introduced a novel method for reflection-based eye racking that
exploits dense 3D eye surface reconstructions from single-shot deflectometry
measurements. State-of-the-art reflection-based methods (e.g., glint tracking)
only sample the eye surface at $\sim 10-15$ points. In our real-world gaze
evaluations shown in sec. 3.2, we acquire >40,000 points of the eye surface in
each single-shot measurement, which leads to a calculated mean relative gaze
error $\leq 0.25^{\circ}$, at a precision $<0.1^{\circ}$. Moreover, we showed
a first qualitative measurement on a real human eye in-vivo, which
demonstrates the feasibility of our method for real-world applications.

Besides its single-shot ability and high accuracy, our method has additional
interesting attributes that will potentially become important for future
application scenarios: It does not rely on 2D eye texture features such as
iris images or pupil center, whose exact position is prone to errors due to
refraction of light rays at the cornea surface. Moreover, we only make minimal
model assumptions about the eye shape. Although we motivate our method by the
justified assumption that cornea and sclera are spherical in shape, this
assumption is, strictly spoken, not required. It would already be sufficient
for our method if the cornea and sclera are rotationally symmetric. In this
case, all backtraced normals would meet along the optical axis of the eye,
which can then be used to calculate the gaze.

Another interesting feature comes from the 3D imaging capability of
deflectometry: The “byproduct” of our gaze estimation is a very accurate
measurement of the eye surface. As described in[34], this surface measurement
can be used to quantify vision impairments. In a potential VR/AR headset
setting, this information can be used to correct vision impairments of the
user ”on the fly", meaning that the user would not be required to wear contact
lenses or glasses, and a previous calibration is not required.

In our future research, we will seek to further improve our method by
replacing certain steps in our pipeline with data-driven approaches, and also
by combining it with certain aspects of our inverse rendering-based
technique[50, 51]. Moreover, we estimate that increasing the pattern coverage
on the eye surface can also significantly contribute to the even higher
accuracy of our method. We hope that our introduced method will become part of
a new wave of accurate and fast eye-tracking methods that enable unprecedented
features in VR/AR development and drastically improves prediction accuracy in
clinical use and psychology research.

## References

  * [1] I. B. Adhanom, P. MacNeilage, and E. Folmer, “Eye tracking in virtual reality: a broad review of applications and challenges,” Virtual Reality pp. 1–24 (2023).
  * [2] R. Clark, J. Blundell, M. J. Dunn, J. T. Erichsen, M. E. Giardini, I. Gottlob, C. Harris, H. Lee, L. Mcilreavy, A. Olson _et al._ , “The potential and value of objective eye tracking in the ophthalmology clinic,” Eye 33, 1200–1202 (2019).
  * [3] N. Merali, D. Veeramootoo, and S. Singh, “Eye-tracking technology in surgical training,” Journal of Investigative Surgery 32, 587–593 (2019).
  * [4] R.-M. Rahal and S. Fiedler, “Understanding cognitive and affective mechanisms in social psychology through eye-tracking,” Journal of Experimental Social Psychology 85, 103842 (2019).
  * [5] R. S. Hessels and I. T. Hooge, “Eye tracking in developmental cognitive neuroscience–the good, the bad and the ugly,” Developmental cognitive neuroscience 40, 100710 (2019).
  * [6] Y. Zhao, D. Lindberg, B. Cleary, O. Mercier, R. Mcclelland, E. Penner, Y.-J. Lin, J. Majors, and D. Lanman, “Retinal-resolution varifocal vr,” in _ACM SIGGRAPH 2023 Emerging Technologies,_ (2023), pp. 1–3.
  * [7] K. Nishino and S. K. Nayar, “The world in an eye [eye image interpretation],” in _Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004._ , vol. 1 (IEEE, 2004), pp. I–I.
  * [8] F. Lu, T. Okabe, Y. Sugano, and Y. Sato, “A head pose-free approach for appearance-based gaze estimation.” in _BMVC,_ (2011), pp. 1–11.
  * [9] F. Lu, Y. Sugano, T. Okabe, and Y. Sato, “Inferring human gaze from appearance via adaptive linear regression,” in _2011 International Conference on Computer Vision,_ (IEEE, 2011), pp. 153–160.
  * [10] J. Li and S. Li, “Gaze estimation from color image based on the eye model with known head pose,” IEEE Transactions on Human-Machine Systems 46, 414–423 (2015).
  * [11] N. Valliappan, N. Dai, E. Steinberg, J. He, K. Rogers, V. Ramachandran, P. Xu, M. Shojaeizadeh, L. Guo, K. Kohlhoff _et al._ , “Accelerating eye movement research via accurate and affordable smartphone eye tracking,” Nature communications 11, 1–12 (2020).
  * [12] F. L. Coutinho and C. H. Morimoto, “Free head motion eye gaze tracking using a single camera and multiple light sources,” in _2006 19th Brazilian Symposium on Computer Graphics and Image Processing,_ (IEEE, 2006), pp. 171–178.
  * [13] C. A. Hennessey and P. D. Lawrence, “Improving the accuracy and reliability of remote system-calibration-free eye-gaze tracking,” IEEE transactions on biomedical engineering 56, 1891–1900 (2009).
  * [14] C. Mestre, J. Gautier, and J. Pujol, “Robust eye tracking based on multiple corneal reflections for clinical applications,” Journal of biomedical optics 23, 035001 (2018).
  * [15] J. Liu, J. Chi, and S. Fan, “A method for accurate 3d gaze estimation with a single camera and two collinear light sources,” IEEE Transactions on Instrumentation and Measurement (2022).
  * [16] D. Beymer and M. Flickner, “Eye gaze tracking using an active stereo head,” in _2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings._ , vol. 2 (IEEE, 2003), pp. II–451.
  * [17] S.-W. Shih and J. Liu, “A novel approach to 3-d gaze tracking using stereo cameras,” IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics) 34, 234–245 (2004).
  * [18] K.-H. Tan, D. J. Kriegman, and N. Ahuja, “Appearance-based eye gaze estimation,” in _Sixth IEEE Workshop on Applications of Computer Vision, 2002.(WACV 2002). Proceedings._ , (IEEE, 2002), pp. 191–195.
  * [19] S. Baluja and D. Pomerleau, “Non-intrusive gaze tracking using artificial neural networks,” Advances in Neural Information Processing Systems 6 (1993).
  * [20] K. Krafka, A. Khosla, P. Kellnhofer, H. Kannan, S. Bhandarkar, W. Matusik, and A. Torralba, “Eye tracking for everyone,” in _Proceedings of the IEEE conference on computer vision and pattern recognition,_ (2016), pp. 2176–2184.
  * [21] W. Zhu and H. Deng, “Monocular free-head 3d gaze tracking with deep learning and geometry constraints,” in _Proceedings of the IEEE International Conference on Computer Vision,_ (2017), pp. 3143–3152.
  * [22] S. Patel and L. Tutchenko, “The refractive index of the human cornea: A review,” Contact Lens and Anterior Eye 42, 575–580 (2019).
  * [23] Z. Zhu and Q. Ji, “Novel eye gaze tracking techniques under natural head movement,” IEEE Transactions on biomedical engineering 54, 2246–2260 (2007).
  * [24] M. C. Knauer, J. Kaminski, and G. Häusler, “Phase measuring deflectometry: a new approach to measure specular free-form surfaces,” in _Optical Metrology in Production Engineering,_ vol. 5457 (SPIE, 2004), pp. 366–376.
  * [25] M. C. Knauer, C. Richter, P. Vogt, and G. Häusler, “Measuring the refractive power with deflectometry in transmission,” DGaO Proc (2008).
  * [26] S. Höfer, J. Burke, and M. Heizmann, “Infrared deflectometry for the inspection of diffusely specular surfaces,” Advanced Optical Technologies 5, 377–387 (2016).
  * [27] C. Faber, E. Olesch, R. Krobot, and G. Häusler, “Deflectometry challenges interferometry: the competition gets tougher!” in _Interferometry XVI: techniques and analysis,_ vol. 8493 (SPIE, 2012), pp. 232–246.
  * [28] E. Olesch, G. Häusler, A. Wörnlein, F. Stinzing, and C. van Eldik, “Deflectometric measurement of large mirrors,” Advanced Optical Technologies 3, 335–343 (2014).
  * [29] G. Häusler, C. Faber, E. Olesch, and S. Ettl, “Deflectometry vs. interferometry,” in _Optical measurement systems for industrial inspection VIII,_ vol. 8788 (SPIE, 2013), pp. 367–377.
  * [30] P. Su, S. Wang, M. Khreishi, Y. Wang, T. Su, P. Zhou, R. E. Parks, K. Law, M. Rascon, T. Zobrist _et al._ , “Scots: a reverse hartmann test with high dynamic range for giant magellan telescope primary mirror segments,” in _Modern Technologies in Space-and Ground-based Telescopes and Instrumentation II,_ vol. 8450 (SPIE, 2012), pp. 332–340.
  * [31] I. Trumper, H. Choi, and D. W. Kim, “Instantaneous phase shifting deflectometry,” Optics express 24, 27993–28007 (2016).
  * [32] F. Willomitzer, C.-K. Yeh, V. Gupta, W. Spies, F. Schiffers, A. Katsaggelos, M. Walton, and O. Cossairt, “Hand-guided qualitative deflectometry with a mobile device,” Optics express 28, 9027–9038 (2020).
  * [33] Y. Li, C.-K. Yeh, B. Xu, F. Schiffers, M. Walton, J. Tumblin, A. Katsaggelos, F. Willomitzer, and O. Cossairt, “A low-cost solution for 3d reconstruction of large-scale specular objects,” in _Computational Optical Sensing and Imaging,_ (Optica Publishing Group, 2021), pp. CW4H–3.
  * [34] H. Liang, E. Olesch, Z. Yang, and G. Häusler, “Single-shot phase-measuring deflectometry for cornea measurement,” Advanced Optical Technologies 5, 433–438 (2016).
  * [35] Y. Zheng, Q. Chao, Y. An, S. Hirsh, and A. Fix, “Fringe projection-based single-shot 3d eye tracking using deep learning and computer graphics,” in _Optical Architectures for Displays and Sensing in Augmented, Virtual, and Mixed Reality (AR, VR, MR) IV,_ vol. 12449 (SPIE, 2023), pp. 265–275.
  * [36] L. Huang, M. Idir, C. Zuo, and A. Asundi, “Review of phase measuring deflectometry,” Optics and Lasers in Engineering 107, 247–257 (2018).
  * [37] H. Liang, T. Sauer, and C. Faber, “Using wavelet transform to evaluate single-shot phase measuring deflectometry data,” in _Applications of Digital Image Processing XLIII,_ vol. 11510 (SPIE, 2020), pp. 404–410.
  * [38] M. Takeda and K. Mutoh, “Fourier transform profilometry for the automatic measurement of 3-d object shapes,” Applied optics 22, 3977–3982 (1983).
  * [39] X. Su and W. Chen, “Fourier transform profilometry:: a review,” Optics and lasers in Engineering 35, 263–284 (2001).
  * [40] M. Ballester, H. Wang, J. Li, O. Cossairt, and F. Willomitzer, “Single-shot tof sensing with sub-mm precision using conventional cmos sensors,” arXiv preprint arXiv:2212.00928 (2022).
  * [41] E. Slogsnat and K.-H. Brenner, “Non-stereoscopic method for deflectometric measurement of reflecting surfaces,” DGaO-Proceedings (Online-Zeitschrift der Deutschen Gesellschaft für angewandte Optik e. V.), ISSN pp. 1614–8436 (2009).
  * [42] L. Huang, J. Xue, B. Gao, C. Zuo, and M. Idir, “Zonal wavefront reconstruction in quadrilateral geometry for phase measuring deflectometry,” Applied optics 56, 5139–5144 (2017).
  * [43] J. Wang, B. Xu, T. Wang, W. J. Lee, M. Walton, N. Matsuda, O. Cossairt, and F. Willomitzer, “Vr eye-tracking using deflectometry,” in _Computational Optical Sensing and Imaging,_ (Optica Publishing Group, 2021), pp. CF2E–3.
  * [44] J. Wang, T. Wang, B. Xu, O. Cossairt, and F. Willomitzer, “Accurate and fast vr eye-tracking using deflectometric information,” in _Computational Optical Sensing and Imaging,_ (Optica Publishing Group, 2023), p. CTh2B.5.
  * [45] H. Ren, F. Gao, and X. Jiang, “Iterative optimization calibration method for stereo deflectometry,” Optics express 23, 22060–22068 (2015).
  * [46] J. Wang, B. Xu, O. Cossairt, and F. Willomitzer, “Easy and flexible calibration approach for deflectometry-based vr eye-tracking systems,” in _Computational Optical Sensing and Imaging,_ (Optica Publishing Group, 2022), pp. CTh5C–1.
  * [47] Z. Zhang, “A flexible new technique for camera calibration,” IEEE Transactions on pattern analysis and machine intelligence 22, 1330–1334 (2000).
  * [48] A. N. Angelopoulos, J. N. Martel, A. P. Kohli, J. Conradt, and G. Wetzstein, “Event-based near-eye gaze tracking beyond 10,000 hz,” IEEE Transactions on Visualization and Computer Graphics 27, 2577–2586 (2021).
  * [49] T. Wang, F. Schiffers, F. Willomitzer, and O. Cossairt, “A mitsuba-based study on trade-offs between projection and reflection based systems in structured-light 3d imaging,” in _OSA Imaging and Applied Optics Congress 2021 (3D, COSI, DH, ISA, pcAOP),_ (Optica Publishing Group, 2021), p. CW4H.2.
  * [50] T. Wang, J. Wang, O. Cossairt, and F. Willomitzer, “Optimization-based eye tracking using deflectometric information,” arXiv preprint arXiv:2303.04997 (2023).
  * [51] O. Cossairt, F. Willomitzer, C.-K. Yeh, and M. Walton, “Low-budget 3d scanning and material estimation using pytorch3d,” in _2020 54th Asilomar Conference on Signals, Systems, and Computers,_ (IEEE, 2020), pp. 1316–1317.