# Anisotropic Special Relativity

Hassan Ganjitabar<EMAIL_ADDRESS>School of Engineering,
Newcastle University, Newcastle upon Tyne, NE1 7RU, UK

###### Abstract

Anisotropic Special Relativity (ASR) is the relativistic theory of nature with
a preferred direction in space-time. By relaxing the full-isotropy constraint
on space-time to the preference of one direction, we get a perturbative
modification of the Minkowski metric as
$\mathscr{g}_{\mu\nu}=\eta_{\mu\nu}+b\epsilon_{\mu\nu}$ leading to an
extension to the geometrical objects such as line element. The symmetry group
in ASR is obtained to be the space-time translations together with the group
of 2-dimensional complex special linear transformations, $SL(2,C)$, generated
by a complexification of $A=K_{x}+K_{y}$ and $B=J_{y}-J_{x}$ and $C=K_{z}$, in
the case of $z$-direction preference. A procedure to make the anisotropic
quantum field theory is provided where the Lorentz invariant Lagrangians need
to be replaced with their ASR version in which the inner product of any pair
of covariant/contravariant indices is mediated by the anisotropic metric
$\mathscr{g}_{\mu\nu}$.

PACS numbers

11.10.-z, 03.30.+p, 11.30.Cp

###### pacs:

Valid PACS appear here

##  I Introduction

One of the widely accepted fundamental theories of physics is Special
Relativity (SR) in which Quantum Field Theory (QFT) and Standard Model (SM) of
the universe are grounded. In SR, some quantities, like the line element
${ds}^{2}=(dt^{2}-dx^{2}-dy^{2}-dz^{2})$, are invariant under the Lorentz
group transformations. Despite the very strict restrictions on departure from
Lorentz symmetry [1, 2, 3], the Lorentz Invariance Violation (LIV) idea [4, 5]
is still popular. One of the scenarios for LIV is Standard Model Extension
(SME) [6] in which the Lorentz violation is allowed by adding non-relativistic
terms to the Lagrangian. On the other side, there are theories wherein the LIV
does not require the complete breakdown of relativistic symmetry. The examples
of latter are perturbation approach of Coleman and Glashow [7] and Very
Special Relativity (VSR) proposed by Cohen and Glashow [8].

There are two more theories allowing for LIV, Finslerian structure of the
space-time [9] and Non-Commutative Quantum Field Theory [10], in context of
which VSR can be realised [11, 12]. In fact, a deformation of VSR symmetry can
be seen in the case of a Finsler space-time in which the fundamental metric
$g$ can be obtained from a function
$F={(\eta_{\mu\nu}dx^{\mu}dx^{\nu})}^{(1-b)/2}{(n_{\alpha}dx^{\alpha})}^{b}$
using $g_{\mu\nu}=\frac{1}{2}{\partial_{\mu}\partial_{\nu}}F^{2}$, where
$\eta_{\mu\nu}$ is the Minkowski metric of SR, $n^{\alpha}=(1,0,0,1)$ is a
null vector and $b$ is a constant parameter. VSR suggests a subgroup of the
Lorentz group, SIM(2), in addition to the space-time translations to be the
maximal symmetry of nature wherein one of the three spatial directions is
preferred. The anisotropy of space is one of the possible scenarios for LIV,
whose upper bound has already been investigated by analysing the cosmological
data [13] as well as performing terrestrial experiments [14, 15].

On the track of building an anisotropic version of SR, Bogoslovsky suggested
the Finslerian line element
$ds={(\eta_{\mu\nu}dx^{\mu}dx^{\nu})}^{(1-b)/2}{(n_{\alpha}dx^{\alpha})}^{b}$
which is not SR invariant but VSR invariant [16, 17]. In this letter, we ask
if there may be any other possible form of line element that is invariant
under a reduced Lorentz symmetry but not under the transformations of the full
Lorentz group?

##  II Results and Discussion

We obtain such a line element by easing the isotropy constraint on the
Minkowski metric, but considering a preferred direction, $n^{\mu}$, in space-
time instead. In an anisotropic theory we need non-zero off-diagonal elements
in the metric. But, the only part of the Minkowski metric we are promoting
here is its $x-y$ block because we want $n^{\mu}$ to be yet invariant. The
promotion of the $x-y$ block must be done in a symmetric way as we are not
looking for a full isotropy breakdown; the isotropy in x-y plane is still
reserved. A modification of Minkowski metric as

$\eta_{\mu\nu}\rightarrow\mathscr{g}_{\mu\nu}=\left(\begin{tabular}[]{cccc}$1$&$0$&$0$&$0$\\\
$0$&$-1+b$&$-b$&$0$\\\ $0$&$-b$&$-1+b$&$0$\\\ $0$&$0$&$0$&$-1$\\\
\end{tabular}\right)$ (1)

is showing such a desirable anisotropy in space-time with the $t$\- and
$z$-elements being left as same as before, leading to a preferred direction
determined by the null vector $n^{\mu}=(1,0,0,1)$. By Anisotropic Special
Relativity (ASR) we mean a relativistic theory in which the inner product of a
pair of covariant/contravariant indices is mediated by the anisotropic metric
$\mathscr{g}$.

Applying metric (1) gives our version of Lorentz violating line element as

${ds}^{2}=(dt^{2}-dx^{2}-dy^{2}-dz^{2})+b{(dx-dy)}^{2}$ (2)

which can be considered as a perturbation of the Lorentz invariant line
element with $b$ factor to be a constant parameter controlling the
perturbation. Obviously, setting $b=0$ turns the situation back to the full
Lorentz symmetry. The metric $\mathscr{g}$ itself can be also written as a
perturbation of the Minkowski metric

$\mathscr{g}_{\mu\nu}=\eta_{\mu\nu}+b\epsilon_{\mu\nu}$ (3)

where

$\epsilon_{\mu\nu}=\left(\begin{tabular}[]{cccc}$0$&$0$&$0$&$0$\\\
$0$&$1$&$-1$&$0$\\\ $0$&$-1$&$1$&$0$\\\ $0$&$0$&$0$&$0$\\\
\end{tabular}\right),$ (4)

or can be obtained through the Finsler approach using a function of the form

$F=\sqrt{(\eta_{\mu\nu}+b\epsilon_{\mu\nu})dx^{\mu}dx^{\nu}}$ (5)

Moreover, it can be checked that the metric introduced above is compatible
with the $SL(2,C)$ symmetry group. Starting from the line element invariance
principal
${\mathscr{g}^{\prime}}_{\mu\nu}{dx^{\prime}}^{\mu}{dx^{\prime}}^{\nu}=\mathscr{g}_{\mu\nu}dx^{\mu}dx^{\nu}$
and considering $\Lambda$ as the most general transformation of the theory
(excluding the space-time translations), one can write
$\Lambda^{T}(\Lambda\mathscr{g}\Lambda^{T})\Lambda=\mathscr{g}$ in the matrix
form. Defining $\Lambda=e^{\omega}$, and in terms of infinitesimal
transformation, $\Lambda\simeq\mathds{1}+\omega$, we have
$(\omega^{T}+\omega)\mathscr{g}(\omega^{T}+\omega)=\mathscr{g}$ suggesting
$\omega$ as

$\omega=\left(\begin{tabular}[]{cccc}$0$&$\alpha$&$\alpha$&$\xi$\\\
$\alpha$&$0$&$\omega_{12}$&$\omega_{13}$\\\
$\alpha$&$-\omega_{12}$&$0$&$\omega_{23}$\\\
$\xi$&$-\omega_{13}$&$-\omega_{23}$&$0$\\\ \end{tabular}\right)$ (6)

where $\alpha$ and $\xi$ are two constant parameter. The metric $\mathscr{g}$
given by (1) has been used in the last step. Eq. (6) can be written as
$\omega=-i\alpha(K_{x}+K_{y})-i\xi
K_{z}+i\omega_{23}J_{x}-i\omega_{13}J_{y}+i\omega_{12}J_{z}$ where $K_{i}$ and
$J_{i}$ are respectively the boost and rotation generators of the Lorentz
group. The appearance of $K_{x}+K_{y}$ and its commutation with $K_{z}$
constraints the parameters $\omega_{13}$ and $\omega_{23}$ to be equal, say
$\omega_{13}=\omega_{23}=\beta$, and sets $\omega_{12}=0$ in order to keep the
algebra closed. Therefore, we can rewrite (6) as $\omega=-i\alpha A-i\beta
B-i\xi C$ with the generators $A=K_{x}+K_{y}$, $B=J_{y}-J_{x}$ and $C=K_{z}$
satisfying the commutator relations $[A,B]=i2C$, $[A,C]=-iB$ and $[B,C]=iA$. A
complexification of these generators in the form of

$G^{+}=\frac{A+iB}{\sqrt{2}},\,\,\,G^{-}=\frac{A-iB}{\sqrt{2}},\,\,\,X=2C$ (7)

gives the below algebra

$[G^{+},G^{-}]=X,\,\,\,[X,G^{\pm}]=\pm 2G^{\pm}$ (8)

which is the algebra of the group of complex special linear transformation in
two dimension, $SL(2,C)$.

Since $SL(2,C)$ has no abelian minimal subgroup, we can expect non-trivial
irreducible representations in ASR. As a matter of fact, $SL(2,C)$ is
isomorphic to the entire Lorentz group, SO(1,3), which in turn means that the
representations of ASR not only do exist but also are the same as those of the
full Lorentz group.

As another consequence of our metric $\mathscr{g}$ given by (1), we can infer
to introducing a Lorentz violating term into the fermionic Lagrangian to
explain the neutrino mass in a natural way without violating the lepton number
or adding sterile right-handed neutrinos. This has been discussed for the
first time in [18, 19]; however, we ask if we can add a different Lorentz
violating term as an implication of our metric $\mathscr{g}$ which preserve
the features. We consider the initially massless Majorana Lagrangian of
neutrino and modify it just by applying metric $\mathscr{g}_{\mu\nu}$ to lower
the index of one of the contravariant vectors e.i.
$\mathcal{L}^{ASR}=i\bar{\nu}_{M}\gamma^{\alpha}\mathscr{g}_{\alpha\beta}\partial^{\beta}\nu_{M}$
leading to

$\mathcal{L}^{ASR}=\mathcal{L}^{SR}+ib\bar{\nu}_{M}(\gamma^{1}-\gamma^{2})(\partial^{1}-\partial^{2})\nu_{M}$
(9)

where $\mathcal{L}^{SR}$ is the massless Majorana Lagrangian of neutrino in SR
given by
$i\bar{\nu}_{M}(\gamma^{0}\partial^{0}-\gamma^{1}\partial^{1}-\gamma^{2}\partial^{2}-\gamma^{3}\partial^{3})\nu_{M}$
with $\nu_{M}$ to be the four-component Majorana neutrino and
$\gamma^{\alpha}$, $\alpha=0-3$, to be the Dirac matrices. Eq. (9) gives the
dispersion relation as
${(m^{2})}^{ASR}=(p_{\alpha}p^{\alpha})^{SR}+b(p_{x}-p_{y})^{2}$ for which
$(p_{\alpha}p^{\alpha})^{SR}={(m^{2})}^{SR}$ is assumed to be zero; even in
such a case, it is still possible for $(m^{2})^{ASR}$ to be not zero due to
the appearance of the perturbative term $b(p_{x}-p_{y})^{2}$. However, for the
neutrinos moving parallel to the $z$ direction or, more generally, those
carrying the same momentum in $x$\- and $y$-direction, $p_{x}=p_{y}$, the
corrected mass yet remains zero. In total, the neutrino mass in anisotropic
space-time will depend on the direction of its movement which in turn means
that the neutrino flavour oscillation allowed by anisotropic theories like ASR
and VSR [20] could be direction-dependent as well.

It can also be noticed that the field theory (FT) obtained in ASR remains
local in contrast to the non-local FT emerging from the correction term
introduced by Cohen and Glashow in VSR [18]. The term they suggested to
correct the equations of motion with is
$\frac{m_{\nu}^{2}\eta_{\alpha\beta}\gamma^{\alpha}n^{\beta}}{2n.\partial}\nu_{M}$
where the appearance of $\partial$ operator in the denominator makes the
theory non-local. However, ASR-FT remains local as only the lower orders of
the derivative operator appear in the Lagrangian.

In general, the mass of any field in ASR can be modified just similar to the
line element in Eq. (2) i.e.

${(m^{2})}^{SR}\,\,\rightarrow\,\,{(m^{2})}^{ASR}={(m^{2})}^{SR}+b(p_{x}-p_{y})^{2}$
(10)

where $m^{SR}$ is the mass of the field in SR given by
$(E^{2}-{p_{x}}^{2}-{p_{y}}^{2}-{p_{z}}^{2})$ with $E$ and $p^{i}$, $i=1-3$,
to be respectively the energy and momentum of the field.

In ASR, we just need to lower the indices of any contravariant vector or
tensor with the anisotropic metric $\mathscr{g}$. This gives the inner product
of two arbitrary vectors $v^{\mu}$ and $u^{\mu}$ as

${(v_{\mu}u^{\mu})}^{ASR}={(v_{\mu}u^{\mu})}^{SR}+b(v^{1}-v^{2})(u^{1}-u^{2})$
(11)

where ${(v_{\mu}u^{\mu})}^{SR}$ is the inner product of the vectors in SR. In
the case that at least one of the two vectors is parallel to the preferred
direction ($v^{1}=v^{2}=0$ or $u^{1}=u^{2}=0$) or its $x$\- and $y$-component
are equal ($v^{1}=v^{2}$ or $u^{1}=u^{2}$), the inner product reduces down to
its SR version.

In order to make an anisotropic quantum field theory, one can promote any
Lagrangian by replacing the isotropic inner product of any pair of
covariant/contravariant indices with its anisotropic version. For example, the
Dirac Lagrangian for the fermions can be modified to

$\mathcal{L}_{D}^{ASR}=\mathcal{L}_{D}^{SR}+ib\bar{\psi}(\gamma^{1}-\gamma^{2})(\partial^{1}-\partial^{2})\psi$
(12)

a consequence of which is the extension of the dispersion relation as (10).
Similarly, in the electrodynamics Lagrangian
$\mathcal{L}_{E}^{SR}=-\frac{1}{4}F_{\mu\nu}F^{\mu\nu}$ we just need to lower
the indices using $\mathscr{g}$ i.e.
$\mathcal{L}_{E}^{ASR}=-\frac{1}{4}\mathscr{g}_{\mu\alpha}\mathscr{g}_{\nu\beta}F^{\alpha\beta}F^{\mu\nu}$
resulting in equations of motion as

$(\partial_{\mu}F^{\mu\nu})^{SR}+b\epsilon_{\mu\alpha}\partial^{\alpha}F^{\mu\nu}=0$
(13)

from which the anisotropically modified Maxwell equations can be derived.

The shape of gauge symmetry in ASR remains the same as in SR

$\left\\{\begin{tabular}[]{c}$\psi\,\,\rightarrow\,\,e^{iq\vartheta}\psi$\\\
$\partial_{\mu}\,\,\rightarrow\,\,D_{\mu}=\partial_{\mu}-iqA_{\mu}$\\\
$A_{\mu}\,\,\rightarrow A_{\mu}+\partial_{\mu}\vartheta$\end{tabular}\right.$
(14)

with the difference that, here, all the covariant indices are made from the
contravariant ones by applying the metric $\mathscr{g}$. For instance, the ASR
version of $\partial_{\mu}$ and $A_{\mu}$ are

$\left\\{\begin{tabular}[]{c}${\partial_{\mu}}^{ASR}={\partial_{\mu}}^{SR}+b(0,\partial^{1}-\partial^{2},\partial^{2}-\partial^{1},0)$\\\
${A_{\mu}}^{ASR}={A_{\mu}}^{SR}+b(0,A^{1}-A^{2},A^{2}-A^{1},0)$\end{tabular}\right.$
(15)

##  III Conclusion

In this letter, we proposed an anisotropic relativistic theory, ASR, as an
alternative to the special theory of relativity. ASR can be formalised by
replacing the Minkowski metric with a perturbative anisotropic metric in which
the space-time isotropy constraint has been replaced with the preference of a
direction determined by the null vector $n^{\mu}$. On the one hand, the
partial breakdown of Lorentz symmetry is allowed while on the other hand, the
symmetry group $SL(2,C)$ guarantees the irreducible representations of the
theory to be just the same as those of SR. Despite a direction-dependent
modification to the dispersion relation, the QFT built up in ASR remains
local.

As one of the interesting applications of ASR, we can infer to redesigning of
neutrino oscillation experiments. ASR suggests the neutrino mass, hence the
neutrino oscillation, to be direction dependent; so, designing a neutrino
oscillation experiment in which the direction of propagation of neutrinos can
sweep different spatial directions could be of special interest. In addition,
as the transformation of ASR, presented by $SL(2,C)$ group, are direction
dependent, we expect most of the relativistic phenomena like length
contraction, time dilation, Doppler effect and aberration of light, etc. to be
direction dependent too. A possible origin for the space-time anisotropy could
be the anisotropic mass/energy distribution in the universe, which can be
reflected in cosmic microwave background. This means that different locations
in the universe could be expected to experience different level or direction
of anisotropy. In such a case, a universal version of ASR can be achieved by
promoting the perturbative parameter $b$ from a constant to a field.

## References

  * [1] V. A. Kostelecky and N. Russell, “Data tables for lorentz and cpt violation,” Reviews of Modern Physics, vol. 83, no. 1, pp. 11–32, 2011.
  * [2] D. Mattingly, “Modern tests of lorentz invariance,” Living Reviews in Relativity, vol. 8, no. 1, p. 5, 2005.
  * [3] S. Liberati, “Tests of lorentz invariance: a 2013 update,” Classical and Quantum Gravity, vol. 30, no. 13, p. 133001, 2013.
  * [4] D. A. Kirzhnits and V. A. Chechin, “The ultra-high-energy cosmic rays and a possible generalization of the relativistic theory,” Yadern Fiz, vol. 15, no. 5, pp. 1051–1059, 1972.
  * [5] T. G. Pavlopoulos, “Breakdown of lorentz invariance,” Physical Review, vol. 159, no. 5, pp. 1106–1110, 1967.
  * [6] D. Colladay and V. A. Kostelecký, “Lorentz-violating extension of the standard model,” Physical Review D, vol. 58, no. 11, p. 116002, 1998.
  * [7] S. Coleman and S. L. Glashow, “High-energy tests of lorentz invariance,” Physical Review D, vol. 59, no. 11, p. 116008, 1999.
  * [8] A. G. Cohen and S. L. Glashow, “Very special relativity,” Physical Review Letters, vol. 97, no. 2, p. 021601, 2006.
  * [9] P. Antonelli, R. Ingarden, and M. Matsumoto, The Theory of Sprays and Finsler Spaces with Applications in Physics and Biology.  1993\.
  * [10] M. R. Douglas and N. A. Nekrasov, “Noncommutative field theory,” Rev. Mod. Phys., vol. 73, pp. 977–1029, Nov 2001.
  * [11] L. Zhang and X. Xue, “The finsler type of space-time realization of deformed very special relativity,” 2012.
  * [12] M. M. Sheikh-Jabbari and A. Tureanu, “Realization of cohen-glashow very special relativity on noncommutative space-time,” Physical Review Letters, vol. 101, no. 26, p. 261601, 2008.
  * [13] E. Komatsu, J. Dunkley, M. R. Nolta, C. L. Bennett, B. Gold, G. Hinshaw, N. Jarosik, D. Larson, M. Limon, L. Page, D. N. Spergel, M. Halpern, R. S. Hill, A. Kogut, S. S. Meyer, G. S. Tucker, J. L. Weiland, E. Wollack, and E. L. Wright, “Five-year wilkinson microwave anisotropy probe observations: Cosmological interpretation,” The Astrophysical Journal Supplement Series, vol. 180, no. 2, pp. 330–376, 2009.
  * [14] C. Eisele, A. Y. Nevsky, and S. Schiller, “Laboratory test of the isotropy of light propagation at the ${10}^{-17}$ level,” Physical Review Letters, vol. 103, no. 9, p. 090401, 2009.
  * [15] B. Altschul, “Bounding isotropic lorentz violation using synchrotron losses at lep,” Physical Review D, vol. 80, no. 9, p. 091901, 2009.
  * [16] G. Y. Bogoslovsky, “A special-relativistic theory of the locally anisotropic space-time,” Il Nuovo Cimento B (1971-1996), vol. 40, no. 1, pp. 99–115, 1977.
  * [17] G. Y. Bogoslovsky, “A special-relativistic theory of the locally anisotropic space-time,” Il Nuovo Cimento B (1971-1996), vol. 40, no. 1, pp. 116–134, 1977.
  * [18] A. Cohen and S. Glashow, “A lorentz-violating origin of neutrino mass?,” 2006\.
  * [19] A. E. Bernardini, “Dirac neutrino mass from the beta-decay end point modified by the dynamics of a lorentz-violating equation of motion,” Physical Review D, vol. 75, no. 9, p. 097901, 2007.
  * [20] J. Alfaro, P. González, and R. Ávila, “Electroweak standard model with very special relativity,” Physical Review D, vol. 91, no. 10, p. 105007, 2015\.


# Strong Laws of Large Numbers for Generalizations of Fréchet Mean Sets

Christof Schötz

###### Abstract

A Fréchet mean of a random variable $Y$ with values in a metric space
$(\mathcal{Q},d)$ is an element of the metric space that minimizes
$q\mapsto\mathbb{E}[d(Y,q)^{2}]$. This minimizer may be non-unique. We study
strong laws of large numbers for sets of generalized Fréchet means. Following
generalizations are considered: the minimizers of
$\mathbb{E}[d(Y,q)^{\alpha}]$ for $\alpha>0$, the minimizers of
$\mathbb{E}[H(d(Y,q))]$ for integrals $H$ of non-decreasing functions, and the
minimizers of $\mathbb{E}[\mathfrak{c}(Y,q)]$ for a quite unrestricted class
of cost functions $\mathfrak{c}$. We show convergence of empirical versions of
these sets in outer limit and in one-sided Hausdorff distance. The derived
results require only minimal assumptions.

##  1 Fréchet Mean Sets

For a random variable $Y$ with values in $\mathbb{R}^{s}$ and
$\mathbb{E}[\|Y\|^{2}]<\infty$, it holds

$\mathbb{E}[Y]=\operatorname*{arg\,min}_{q\in\mathbb{R}^{s}}\mathbb{E}[d(Y,q)^{2}]\,,$

where $d(x,y)=\|x-y\|$ is the Euclidean distance. We can also write

$\mathbb{E}[Y]=\operatorname*{arg\,min}_{q\in\mathbb{R}^{s}}\mathbb{E}[d(Y,q)^{2}-d(Y,0)^{2}]\,,$

as we just add a constant term. In the latter equation, we only require $Y$ to
be once integrable, $\mathbb{E}[\|Y\|]<\infty$, instead of twice as
$|\|y-q\|^{2}-\|y\|^{2}|\leq 2\|y\|\|q\|+\|q\|^{2}$.

The concept of Fréchet mean, proposed in [Fréchet, 1948], builds upon this
minimizing property of the Euclidean mean to generalize the expected value to
random variables with values in a metric space. Let $(\mathcal{Q},d)$ be a
metric space. As a shorthand we may write $\overline{q,\\!p}$ instead of
$d(q,p)$. Let $Y$ be a random variable with values in $\mathcal{Q}$. Fix an
arbitrary element $o\in\mathcal{Q}$. The _Fréchet mean set_ of $Y$ is
$M=\operatorname*{arg\,min}_{q\in\mathcal{Q}}\mathbb{E}[\overline{Y,\\!q}^{2}-\overline{Y,\\!o}^{2}]$
assuming the expectations exist. This definition does not depend on $o$. The
reason for subtracting $\overline{Y,\\!o}^{2}$ is the same as in the Euclidean
case: We need to make less moment assumptions to obtain a meaningful value:
The triangle inequality implies

$\left|\overline{y,\\!q}^{2}-\overline{y,\\!o}^{2}\right|\leq\overline{o,\\!q}\left(\overline{o,\\!q}+2\overline{y,\\!o}\right)$

for all $y,q,o\in\mathcal{Q}$. Thus, if
$\mathbb{E}[\overline{Y,\\!o}]<\infty$, then
$\mathbb{E}[\overline{Y,\\!q}^{2}-\overline{Y,\\!o}^{2}]<\infty$ for all
$q\in\mathcal{Q}$.

In Euclidean spaces and other Hadamard spaces (metric spaces with nonpositive
curvature), the Fréchet mean is always unique [Sturm, 2003, Proposition 4.3].
This is not true in general. On the circle, a uniform distribution on two
antipodal points has two Fréchet means. For a deeper analysis of Fréchet means
on the circle, see [Hotz and Huckemann, 2015]. Similarly, Fréchet means on
many positively curved spaces like (hyper-)spheres may not be unique. For the
metric space $\mathcal{Q}=\mathbb{R}$ with $d(q,p)=\sqrt{\left|q-p\right|}$,
the Fréchet mean set is the set of medians, which may also be non-unique.
These examples underline the importance of considering sets of minimizers in a
general theory instead of assuming uniqueness.

The notion of Fréchet mean can be generalized to cases where the cost function
to be minimized is not a squared metric, e.g. [Huckemann, 2011]. We will not
explicitly write down measurablity conditions, but silently demand that all
spaces have the necessary measurable structure and all functions are
measurable when necessary. Let $(\mathcal{Q},d)$ be a metric space and
$\mathcal{Y}$ be a set. Let
$\mathfrak{c}\colon\mathcal{Y}\times\mathcal{Q}\to\mathbb{R}$ be a function.
Let $Y$ be a random variable with values in $\mathcal{Y}$. Let
$M:=\operatorname*{arg\,min}_{q\in\mathcal{Q}}\mathbb{E}[\mathfrak{c}(Y,q)]$
assuming the expectations exist. In this context, $\mathfrak{c}$ is called
cost function, $\mathcal{Y}$ is called data space, $\mathcal{Q}$ is called
descriptor space, $q\mapsto\mathbb{E}[\mathfrak{c}(Y,q)]$ is called objective
function (or Fréchet function), and $M$ is called generalized Fréchet mean set
or $\mathfrak{c}$-Fréchet mean set.

This general scenario contains the setting of general M-estimation. It
includes many important statistical frameworks like maximum likelihood
estimation, where $\mathcal{Q}=\Theta$ parameterizes a family of densities
$(f_{\vartheta})_{\vartheta\in\Theta}$ on $\mathcal{Y}=\mathbb{R}^{p}$ and
$\mathfrak{c}(x,\vartheta)=-\log f_{\vartheta}(x)$, or linear regression,
where $\mathcal{Q}=\mathbb{R}^{s+1}$,
$\mathcal{Y}=(\\{1\\}\times\mathbb{R}^{s})\times\mathbb{R}$,
$\mathfrak{c}((x,y),\beta)=(y-\beta\\!^{\top}\\!x)^{2}$. It also includes
nonstandard settings, e.g. [Huckemann, 2011], where geodesics in $\mathcal{Q}$
are fitted to points in $\mathcal{Y}$.

Fix an arbitrary element $o\in\mathcal{Q}$. We will use cost functions
$\mathfrak{c}(y,q)=H(\overline{y,\\!q})-H(\overline{y,\\!o})$, where
$H(x)=\int_{0}^{x}h(t)\mathrm{d}t$ for a non-decreasing function $h$, and
$\mathfrak{c}(y,q)=\overline{y,\\!q}^{\alpha}-\overline{y,\\!o}^{\alpha}$ with
$\alpha>0$. In both cases the set of minimizers does not depend on $o$. We
call the minimizers of the former cost function $H$-Fréchet means. In the
latter case, we call the minimizers power Fréchet means or $\alpha$-Fréchet
means. We can interpret the different exponents $\alpha=2$, $\alpha=1$,
$\alpha\to 0$, $\alpha\to\infty$ as mean, median, mode, and circumcenter (or
mid-range), respectively, see [MacQueen, 1967]. The minimizers for $\alpha=1$
are sometimes called Fréchet median, e.g. [Arnaudon et al., 2013]. If
$\mathcal{Q}$ is a Banach space, then they are called geometric or spatial
median, e.g. [Kemperman, 1987]. $H$-Fréchet means serve as a generalization of
$\alpha$-Fréchet means for $\alpha>1$ as well as an intermediate result for
proving strong laws of large numbers for $\alpha$-Fréchet mean sets with
$\alpha\in(0,1]$.

For a function $f\colon\mathcal{Q}\to\mathbb{R}$ and $\epsilon\geq 0$, define

$\epsilon\text{-}\operatorname*{arg\,min}_{q\in\mathcal{Q}}f(q):=\\{q\in\mathcal{Q}\nonscript\,|\allowbreak\nonscript\,\mathopen{}f(q)\leq\epsilon+\inf_{q\in\mathcal{Q}}f(q)\\}\,.$

Let $Y_{1},\dots,Y_{n}$ be independent random variables with the same
distribution as $Y$. Choose
$(\epsilon_{n})_{n\in\mathbb{N}}\subseteq[0,\infty)$ with
$\epsilon_{n}\xrightarrow{n\to\infty}0$. Let
$M_{n}:=\epsilon_{n}\text{-}\operatorname*{arg\,min}_{q\in\mathcal{Q}}\frac{1}{n}\sum_{i=1}^{n}\mathfrak{c}(Y_{i},q)$.
Our goal is to show almost sure convergence of elements in $M_{n}$ to elements
in $M$.

######  Remark 1.1.

Considering sets of elements that minimize the objective only up to
$\epsilon_{n}$ makes the results more relevant to applications in which
Fréchet mean sets are approximated numerically. Furthermore, it may allow us
to find more elements of $M$ in the limit of $M_{n}$ than for
$\epsilon_{n}=0$, as discussed in 2.5 below and appendix A.

There are different possibilities of how a convergence of sets $M_{n}$ to a
set $M$ can be described.

######  Definition 1.2.

Let $(\mathcal{Q},d)$ be a metric space.

  1. (i)

Let $(B_{n})_{n\in\mathbb{N}}$ with $B_{n}\subseteq\mathcal{Q}$ for all
$n\in\mathbb{N}$. Then the _outer limit_ of $(B_{n})_{n\in\mathbb{N}}$ is

$\operatorname*{lim\,\overline{sup}}_{n\to\infty}B_{n}:=\bigcap_{n\in\mathbb{N}}\overline{\bigcup_{k\geq
n}B_{k}}\,,$

where $\overline{B}$ denotes the closure of the set $B$.

  2. (ii)

The _one-sided Hausdorff distance_ between $B,B^{\prime}\subseteq\mathcal{Q}$
is

$d_{\subseteq}(B,B^{\prime}):=\sup_{x\in B}\inf_{x^{\prime}\in
B^{\prime}}d(x,x^{\prime})\,.$

  3. (iii)

The _Hausdorff distance_ between $B,B^{\prime}\subseteq\mathcal{Q}$ is

$d_{\mathsf{H}}(B,B^{\prime}):=\max(d_{\subseteq}(B,B^{\prime}),d_{\subseteq}(B^{\prime},B))\,.$

######  Remark 1.3.

  1. (i)

The outer limit is the set of all points of accumulation of all sequences
$(x_{n})_{n\in\mathbb{N}}$ with $x_{n}\in B_{n}$. We may write
$\operatorname*{lim\,\overline{sup}}_{n\to\infty}B_{n}=\\{q\in\mathcal{Q}\nonscript\,|\allowbreak\nonscript\,\mathopen{}\liminf_{n\to\infty}d(B_{n},q)=0\\}$,
where $d(B,q):=\inf_{p\in B}d(p,q)$ for a subset $B\subseteq\mathcal{Q}$. The
inner limit is dual to the outer limit. It is defined as
$\operatorname*{innerlim}_{n\to\infty}B_{n}:=\\{q\in\mathcal{Q}\nonscript\,|\allowbreak\nonscript\,\mathopen{}\limsup_{n\to\infty}d(B_{n},q)=0\\}$.
Clearly,
$\operatorname*{innerlim}_{n\to\infty}B_{n}\subseteq\operatorname*{lim\,\overline{sup}}_{n\to\infty}B_{n}$.
Thus, results of the form
$\operatorname*{lim\,\overline{sup}}_{n\to\infty}B_{n}\subseteq B$, which we
show below, are stronger than
$\operatorname*{innerlim}_{n\to\infty}B_{n}\subseteq B$.

  2. (ii)

It holds $d_{\subseteq}(B,B^{\prime})=0$ if and only if
$B\subseteq\overline{B^{\prime}}$, but $d_{\mathsf{H}}(B,B^{\prime})=0$ if and
only if $\overline{B}=\overline{B}^{\prime}$. The function $d_{\mathsf{H}}$ is
a metric on the set of closed and bounded subsets of $\mathcal{Q}$.

  3. (iii)

Elements from a sequence of sets might have sub-sequences that have no point
of accumulation and are bounded away from the outer limit of the sequence of
sets. That cannot happen with the one-sided Hausdorff limit. Here, every sub-
sequence is eventually arbitrarily close to the limiting set. As an example,
the outer limit of the sequence of sets $\\{0,n\\}$, $n\in\mathbb{N}$ on the
Euclidean real line is $\\{0\\}$, but
$d_{\subseteq}(\\{0,n\\},\\{0\\})\xrightarrow{n\to\infty}\infty$. Aside from
an element with diverging distance ($n$ in the example), another cause for the
two limits to not align may be non-compactness of bounded sets: Consider the
space $\ell^{2}$ of all sequences
$(x_{k})_{k\in\mathbb{N}}\subseteq\mathbb{R}$ with
$\sum_{k=1}^{\infty}x_{k}^{2}<\infty$ with distance
$d((x_{k})_{k\in\mathbb{N}},(y_{k})_{k\in\mathbb{N}})=(\sum_{k=1}^{\infty}(x_{k}-y_{k})^{2})^{\frac{1}{2}}$.
Let $\underline{0}\in\ell^{2}$ be the sequence with all entries equal to 0.
Let $e^{n}:=(e^{n}_{k})_{k\in\mathbb{N}}\in\ell^{2}$ with $e_{n}^{n}=1$ and
$e^{n}_{k}=0$ for all $k\neq n$. Then
$d_{\subseteq}(\\{\underline{0},e_{n}\\},\\{\underline{0}\\})=1$ for all
$n\in\mathbb{N}$, but
$\operatorname*{lim\,\overline{sup}}_{n\to\infty}\\{\underline{0},e_{n}\\}=\\{\underline{0}\\}$.

We will state conditions so that
$\operatorname*{lim\,\overline{sup}}_{n\to\infty}M_{n}\subseteq M$ almost
surely or $d_{\subseteq}(M_{n},M)\xrightarrow{n\to\infty}_{\mathsf{a.s.}}0$,
where the index $\mathsf{a.s.}$ indicates almost sure convergence. It is not
easily possible to show
$d_{\mathsf{H}}(M_{n},M)\xrightarrow{n\to\infty}_{\mathsf{a.s.}}0$ if $M$ is
not a singleton, as discussed in 2.5 below and appendix A. These limit
theorems may be called strong laws of large numbers of the Fréchet mean set or
(strong) consistency of the empirical Fréchet mean set. Notably, in [Evans and
Jaffe, 2020] the connection to convergence in the sense of topology is made:
If the set of closed subsets of $\mathcal{Q}$ is equipped with the Kuratowski
upper topology, a sequence of closed subsets $(B_{n})_{n\in\mathbb{N}}$
converges to a closed subset $B$ if and only if
$\operatorname*{lim\,\overline{sup}}_{n\to\infty}B_{n}\subseteq B$. If the set
of nonempty compact subsets of $\mathcal{Q}$ is equipped with the Hausdorff
upper topology, a sequence of nonempty compact subsets
$(B_{n})_{n\in\mathbb{N}}$ converges to a nonempty compact subset $B$ if and
only if $d_{\subseteq}(B_{n},B)\xrightarrow{n\to\infty}0$.

[Ziezold, 1977] shows a strong law in outer limit for Fréchet mean sets with a
second moment condition. [Sverdrup-Thygeson, 1981] shows a strong law in outer
limit for power Fréchet mean sets in compact spaces. [Bhattacharya and
Patrangenaru, 2003] shows almost sure convergence of Fréchet mean sets in one-
sided Hausdorff distance with a second moment condition. The independent
parallel work [Evans and Jaffe, 2020] shows strong laws in outer limit and
one-sided Hausdorff distance for $\alpha$-Fréchet mean sets requiring
$\mathbb{E}[\overline{Y,\\!o}^{\alpha}]<\infty$, which is a second moment
condition for the Fréchet mean. In contrast, we show strong laws of large
numbers for power Fréchet mean sets in outer limit and in one-sided Hausdorff
distance with less moment assumptions: For power $\alpha>1$, we require
$\mathbb{E}[\overline{Y,\\!o}^{\alpha-1}]<\infty$, and for $\alpha\in(0,1]$ no
moment assumption is made, see 5.1 and 5.2. Thus, $\alpha$-Fréchet means may
be of interest in robust statistics. [Huckemann, 2011] shows almost sure
convergence in one-side Hausdorff distance as well as in outer limit for
generalized Fréchet means. Our results for $\mathfrak{c}$-Fréchet means
require slightly less strict assumptions, see Theorem 3.2 and Theorem 3.5,
which make them applicable in a larger class of settings and allows us to
derive our results for $H$\- and $\alpha$-Fréchet means with minimal moment
assumptions. Results in [Artstein and Wets, 1995, Korf and Wets, 2001, Choirat
et al., 2003] imply strong laws and ergodic theorems in outer limit for
generalized Fréchet means. We recite parts of these results to state Theorem
3.2. Furthermore, we show strong laws of large numbers for $H$-Fréchet means
sets in outer limit, 4.3, and one-sided Hausdorff distance, 4.4. When $M$ is
singleton a quantitative version (rates of convergence) of the results
presented in this article is given in [Schötz, 2019].

Before we consider the probabilistic setting, we present theory on convergence
of minimizing sets for deterministic functions in section 2, where we
partially follow [Rockafellar and Wets, 1998]. Thereafter, we derive strong
laws of large numbers for $\mathfrak{c}$-Fréchet mean sets in section 3, for
$H$-Fréchet mean sets in section 4, and for $\alpha$-Fréchet mean sets in
section 5. Appendix A uses the median as a simple example to illustrate some
peculiarities when dealing with sets of Fréchet means. All strong laws in the
main part of this article build upon [Korf and Wets, 2001, Theorem 1.1] – a
deep convergence result for functions $\mathcal{Q}\to\mathbb{R}$. In appendix
B, we show a different route to a strong law in one-side Hausdorff distance.
This is illustrative, but requires slightly stricter assumptions. In appendix
C some auxiliary results are stated and proven.

##  2 Convergence of Minimizer Sets of Deterministic Functions

Let $(\mathcal{Q},d)$ be a metric space. The diameter of a set
$B\subseteq\mathcal{Q}$ is defined as
$\operatorname{\mathsf{diam}}(B)=\sup_{q,p\in B}d(q,p)$. The following notion
of convergence of functions will be useful to infer a convergence results of
their minimizers.

######  Definition 2.1.

Let $f,f_{n}\colon\mathcal{Q}\to\mathbb{R}$, $n\in\mathbb{N}$. The sequence
$(f_{n})_{n\in\mathbb{N}}$ _epi-converges_ to $f$ _at_ $x\in\mathcal{Q}$ if
and only if

$\displaystyle\forall(x_{n})_{n\in\mathbb{N}}\subseteq\mathcal{Q},x_{n}\to
x\colon$ $\displaystyle\liminf_{n\to\infty}f_{n}(x_{n})\geq
f(x)\qquad\text{and}$
$\displaystyle\exists(y_{n})_{n\in\mathbb{N}}\subseteq\mathcal{Q},y_{n}\to
x\colon$ $\displaystyle\limsup_{n\to\infty}f_{n}(y_{n})\leq f(x)\,.$

The sequence $(f_{n})_{n\in\mathbb{N}}$ _epi-converges_ to $f$ if and only if
it epi-converges at all $x\in\mathcal{Q}$. We then write
$f_{n}\xrightarrow{n\to\infty}_{\mathsf{epi}}f$.

We introduce some short notation. Let $f\colon\mathcal{Q}\to\mathbb{R}$ and
$\epsilon\geq 0$. Denote $\inf f=\inf_{x\in\mathcal{Q}}f(x)$,
$\operatorname*{arg\,min}f=\\{x\in\mathcal{Q}\nonscript\,|\allowbreak\nonscript\,\mathopen{}f(x)=\inf
f\\}$,
$\epsilon\text{-}\operatorname*{arg\,min}f=\\{x\in\mathcal{Q}\nonscript\,|\allowbreak\nonscript\,\mathopen{}f(x)\leq\epsilon+\inf
f\\}$. Let $\delta>0$ and $x_{0}\in\mathcal{Q}$. Denote
$\operatorname{\mathrm{B}}_{\delta}(x_{0})=\\{x\in\mathcal{Q}\nonscript\,|\allowbreak\nonscript\,\mathopen{}d(x,x_{0})<\delta\\}$.
Furthermore, $f$ is called _lower semi-continuous_ if and only if
$\liminf_{x\to x_{0}}f(x)\geq f(x_{0})$ for all $x_{0}\in\mathcal{Q}$.

To state convergence results for minimizing sets of deterministic functions,
we need one final definition.

######  Definition 2.2.

A sequence $(B_{n})_{n\in\mathbb{N}}$ of sets $B_{n}\subseteq\mathcal{Q}$ is
called _eventually precompact_ if and only if there is $n\in\mathbb{N}$ such
that the set $\bigcup_{k=n}^{\infty}B_{k}$ is precompact, i.e. its closure is
compact.

The first theorem of this section relates epi-convergence of functions to
convergence of their sets of minimizers in outer limit.

######  Theorem 2.3.

Let $f,f_{n}\colon\mathcal{Q}\to\mathbb{R}$. Let
$(\epsilon_{n})_{n\in\mathbb{N}}\subseteq[0,\infty)$ with
$\epsilon_{n}\xrightarrow{n\to\infty}0$. Assume
$f_{n}\xrightarrow{n\to\infty}_{\mathsf{epi}}f$. Then

$\operatorname*{lim\,\overline{sup}}_{n\to\infty}\,\epsilon_{n}\text{-}\operatorname*{arg\,min}f_{n}\subseteq\operatorname*{arg\,min}f$

and

$\limsup_{n\to\infty}\inf f_{n}\leq\inf f\,.$

Large parts of this theorem can be found e.g., in [Rockafellar and Wets, 1998,
chapter 7]. To make this article more self-contained, we give a proof here.

###### Proof.

Let
$x\in\operatorname*{lim\,\overline{sup}}_{n\to\infty}\,\epsilon_{n}\text{-}\operatorname*{arg\,min}f_{n}$.
Then there is a sequence
$x_{n}\in\epsilon_{n}\text{-}\operatorname*{arg\,min}f_{n}$ with a subsequence
converging to $x$, i.e., $x_{n_{i}}\xrightarrow{i\to\infty}x$, where
$n_{i}\xrightarrow{i\to\infty}\infty$. Let $y\in\mathcal{Q}$ be arbitrary. As
$f_{n}\xrightarrow{n\to\infty}_{\mathsf{epi}}f$, there is a sequence
$(y_{n})_{n\in\mathbb{N}}\subseteq\mathcal{Q}$ with
$y_{n}\xrightarrow{n\to\infty}y$ and $\limsup_{n\to\infty}f_{n}(y_{n})\leq
f(y)$. It holds $f_{n_{i}}(x_{n_{i}})\leq\epsilon_{n_{i}}+\inf
f_{n_{i}}\leq\epsilon_{n_{i}}+f_{n_{i}}(y_{n_{i}})$. Thus, by the definition
of epi-convergence and $\epsilon_{n}\xrightarrow{n\to\infty}0$, we obtain

$f(x)\leq\liminf_{i\to\infty}f_{n_{i}}(x_{n_{i}})\leq\liminf_{i\to\infty}\left(\epsilon_{n_{i}}+f_{n_{i}}(y_{n_{i}})\right)\leq\limsup_{i\to\infty}f_{n_{i}}(y_{n_{i}})\leq
f(y)\,.$

Thus, $x\in\operatorname*{arg\,min}f$. Next, we turn to the inequality of the
infima. For $\epsilon>0$ choose an arbitrary
$x\in\epsilon\text{-}\operatorname*{arg\,min}f$. There is a sequence
$(y_{n})_{n\in\mathbb{N}}\subseteq\mathcal{Q}$ with
$y_{n}\xrightarrow{n\to\infty}x$ and
$f_{n}(y_{n})\xrightarrow{n\to\infty}f(x)$. Thus,

$\limsup_{n\to\infty}\inf f_{n}\leq\limsup_{n\to\infty}f_{n}(y_{n})\leq\inf
f+\epsilon\,.$

∎

It is illustrative to compare this result with Theorem B.2, which shows that a
stronger notion of convergence for functions – convergences uniformly on
bounded sets – yields convergence of sets of minimizers in one-sided Hausdorff
distance, which is a stronger notion of convergence of sets as the next
theorem shows.

######  Theorem 2.4.

Let $(B_{n})_{n\in\mathbb{N}}$ with $B_{n}\subseteq\mathcal{Q}$ for all
$n\in\mathbb{N}$. Let $B\subseteq\mathcal{Q}$.

  1. (i)

If $d_{\subseteq}(B_{n},B)\xrightarrow{n\to\infty}0$ then
$\operatorname*{lim\,\overline{sup}}_{n\to\infty}\,B_{n}\subseteq\overline{B}$.

  2. (ii)

Assume $(B_{n})_{n\in\mathbb{N}}$ is eventually precompact. If
$\operatorname*{lim\,\overline{sup}}_{n\to\infty}\,B_{n}\subseteq\overline{B}$
then $d_{\subseteq}(B_{n},B)\xrightarrow{n\to\infty}0$.

###### Proof.

  1. (i)

Assume $d_{\subseteq}(B_{n},B)\xrightarrow{n\to\infty}0$. Let
$x_{\infty}\in\operatorname*{lim\,\overline{sup}}_{n\to\infty}\,B_{n}$, i.e.,
there is a sequence $(x_{n_{k}})_{k\in\mathbb{N}}\subseteq\mathcal{Q}$ with
$n_{1}<n_{2}<\dots$ and $x_{n_{k}}\in B_{n_{k}}$ such that
$x_{n_{k}}\xrightarrow{k\to\infty}x_{\infty}$. Thus,

$\inf_{x\in B}d(x_{\infty},x)\leq d(x_{\infty},x_{n_{k}})+\inf_{x\in
B}d(x_{n_{k}},x)\xrightarrow{k\to\infty}0\,.$

This shows $\inf_{x\in B}d(x_{\infty},x)=0$. Hence,

$\operatorname*{lim\,\overline{sup}}_{n\to\infty}\,B_{n}\subseteq\\{x_{\infty}\in\mathcal{Q}\nonscript\,|\allowbreak\nonscript\,\mathopen{}\inf_{x\in
B}d(x_{\infty},x)=0\\}=\overline{B}\,.$

  2. (ii)

Assume
$\operatorname*{lim\,\overline{sup}}_{n\to\infty}\,B_{n}\subseteq\overline{B}$.
Further assume the existence of $\epsilon>0$ and a sequence
$(x_{n_{k}})_{k\in\mathbb{N}}\subseteq\mathcal{Q}$ with $n_{1}<n_{2}<\dots$
and $x_{n_{k}}\in B_{n_{k}}$ such that $\inf_{x\in
B}d(x_{n_{k}},x)\geq\epsilon$. As $(B_{n_{k}})_{k\in\mathbb{N}}$ is eventually
precompact, the sequence $(x_{n_{k}})_{k\in\mathbb{N}}$ has an accumulation
point $x_{\infty}$ in $\overline{\bigcup_{k\geq k_{0}}B_{n_{k}}}$ for some
$k_{0}\in\mathbb{N}$ with $\inf_{x\in B}d(x_{\infty},x)\geq\epsilon$. In
particular, $x_{\infty}\not\in\overline{B}$, which contradicts the first
assumption in the proof. Thus, a sequence $(x_{n_{k}})_{k\in\mathbb{N}}$ with
these properties cannot exist, which implies
$d_{\subseteq}(B_{n},B)\xrightarrow{n\to\infty}0$.

∎

Note that the argument for the second part is essentially the same as in
[Huckemann, 2011, proof of Theorem A.4].

######  Remark 2.5.

Together Theorem 2.3 and Theorem 2.4 may yield convergence of minimizers in
one-sided Hausdorff distance. But even if
$d_{\subseteq}(\epsilon_{n}\text{-}\operatorname*{arg\,min}f_{n},\operatorname*{arg\,min}f)\xrightarrow{n\to\infty}0$,
$d_{\mathsf{H}}(\epsilon_{n}\text{-}\operatorname*{arg\,min}f_{n},\operatorname*{arg\,min}f)$
does not necessarily vanish unless $\operatorname*{arg\,min}f$ is a singleton.
Similarly, for an arbitrary sequence $\epsilon_{n}\xrightarrow{n\to\infty}0$,
the outer limit of $\epsilon_{n}\text{-}\operatorname*{arg\,min}f_{n}$ may be
a strict subset of $\operatorname*{arg\,min}f$. But according to [Rockafellar
and Wets, 1998, Theorem 7.31 (c)], there exists a sequence
$(\epsilon_{n})_{n\in\mathbb{N}}$ with $\epsilon_{n}\xrightarrow{n\to\infty}0$
slow enough such that
$\operatorname*{lim\,\overline{sup}}_{n\to\infty}\epsilon_{n}\text{-}\operatorname*{arg\,min}f_{n}=\operatorname*{arg\,min}f$.
An explicit example of this phenomenon is presented in appendix A.

##  3 Strong Laws for $\mathfrak{c}$-Fréchet Mean Sets

Let $(\mathcal{Q},d)$ be a metric space, the descriptor space. Let
$\mathcal{Y}$ be a set, the data space. Let
$\mathfrak{c}\colon\mathcal{Y}\times\mathcal{Q}\to\mathbb{R}$ be a function,
the cost function. Let $(\Omega,\Sigma,\mathbb{P})$ be a probability space
that is silently underlying all random variables in this section. Let $Y$ be a
random variable with values in $\mathcal{Y}$. Denote the
$\mathfrak{c}$-Fréchet mean set of $Y$ as
$M=\operatorname*{arg\,min}_{q\in\mathcal{Q}}\mathbb{E}[\mathfrak{c}(Y,q)]$.
Let $Y_{1},\dots,Y_{n}$ be independent random variables with the same
distribution as $Y$. Choose
$(\epsilon_{n})_{n\in\mathbb{N}}\subseteq[0,\infty)$ with
$\epsilon_{n}\xrightarrow{n\to\infty}0$. Set
$M_{n}=\epsilon_{n}\text{-}\operatorname*{arg\,min}_{q\in\mathcal{Q}}\frac{1}{n}\sum_{i=1}^{n}\mathfrak{c}(Y_{i},q)$.

######  Assumptions 3.1.

  * •

Polish: $(\mathcal{Q},d)$ is separable and complete.

  * •

LowerSemiContinuity: $q\mapsto\mathfrak{c}(y,q)$ is lower semi-continuous.

  * •

Integrable:
$\mathbb{E}[\left\nonscript\;|\nonscript\;\mathfrak{c}(Y,q)\right\nonscript\;|\nonscript\;]<\infty$
for all $q\in\mathcal{Q}$.

  * •

IntegrableInf: $\mathbb{E}[\inf_{q\in\mathcal{Q}}\mathfrak{c}(Y,q)]>-\infty$.

######  Theorem 3.2.

Assume Polish, LowerSemiContinuity, Integrable, and IntegrableInf. Then,
almost surely,

$\operatorname*{lim\,\overline{sup}}_{n\to\infty}\,M_{n}\subseteq M\,.$

###### Proof.

Define $F(q)=\mathbb{E}[\mathfrak{c}(Y,q)]$,
$F_{n}(q)=\frac{1}{n}\sum_{i=1}^{n}\mathfrak{c}(Y_{i},q)$. By Integrable,
$F(q)<\infty$. [Korf and Wets, 2001, Theorem 1.1] states that
$F_{n}\xrightarrow{n\to\infty}_{\mathsf{epi}}F$ almost surely if Polish,
LowerSemiContinuity, and IntegrableInf are true. Theorem 2.3 then implies
$\operatorname*{lim\,\overline{sup}}_{n\to\infty}\,M_{n}\subseteq M$ almost
surely. ∎

######  Assumptions 3.3.

  * •

HeineBorel: Every closed bounded set in $\mathcal{Q}$ is compact.

  * •

SampleHeineBorel: Almost surely following is true: There is
$N_{0}\in\mathbb{N}$ such that every closed and bounded subset of
$\bigcup_{n\geq N_{0}}M_{n}$ is compact.

  * •

UpperBound: $\mathbb{E}[\sup_{q\in
B}\left\nonscript\;|\nonscript\;\mathfrak{c}(Y,q)\right\nonscript\;|\nonscript\;]<\infty$
for all bounded sets $B\subseteq\mathcal{Q}$.

  * •

LowerBound: There are $o\in\mathcal{Q}$,
$\psi^{+},\psi^{-}\colon[0,\infty)\to[0,\infty)$,
$\mathfrak{a}^{+},\mathfrak{a}^{-}\in(0,\infty)$, and
$\sigma(Y_{1},\dots,Y_{n})$-measurable random variables
$\mathfrak{a}^{+}_{n},\mathfrak{a}^{-}_{n}\in[0,\infty)$ such that

$\displaystyle\mathfrak{a}^{+}\psi^{+}(\overline{q,\\!o})-\mathfrak{a}^{-}\psi^{-}(\overline{q,\\!o})$
$\displaystyle\leq\mathbb{E}[\mathfrak{c}(Y,q)]\,,$
$\displaystyle\mathfrak{a}^{+}_{n}\psi^{+}(\overline{q,\\!o})-\mathfrak{a}^{-}_{n}\psi^{-}(\overline{q,\\!o})$
$\displaystyle\leq\frac{1}{n}\sum_{i=1}^{n}\mathfrak{c}(Y_{i},q)$

for all $q\in\mathcal{Q}$. Furthermore,
$\mathfrak{a}^{+}_{n}\xrightarrow{n\to\infty}_{\mathsf{a.s.}}\mathfrak{a}^{+}$
and
$\mathfrak{a}^{-}_{n}\xrightarrow{n\to\infty}_{\mathsf{a.s.}}\mathfrak{a}^{-}$.
Lastly,
$\psi^{+}(\delta)/\psi^{-}(\delta)\xrightarrow{\delta\to\infty}\infty$.

######  Remark 3.4.

  * •

Following implications hold:

HeineBorel $\displaystyle\Rightarrow\textsc{Polish}\,,$ HeineBorel
$\displaystyle\Rightarrow\textsc{SampleHeineBorel}\,,$ UpperBound
$\displaystyle\Rightarrow\textsc{Integrable}\,.$

  * •

On HeineBorel: A space enjoying this property is also called boundedly compact
or proper metric space. The Euclidean spaces $\mathbb{R}^{s}$, finite
dimensional Riemannian manifolds, as well as $\mathcal{C}^{\infty}(U)$ for
open subsets $U\subseteq\mathbb{R}^{s}$ fulfill Heine–Borel [Edwards, 1995,
section 8.4.7]. See [Williamson and Janos, 1987] for a construction of further
spaces where Heine–Borel is true.

  * •

On SampleHeineBorel and infinite dimension: If $M_{n}=\\{m_{n}\\}$ and
$M=\\{m\\}$ are singleton sets and $m_{n}\xrightarrow{n\to\infty}m$ almost
surely, then SampleHeineBorel holds. It is less strict than HeineBorel: In
separable Hilbert spaces of infinite dimension HeineBorel does not hold. But
with the metric $d$ induced by the inner product and $\mathfrak{c}=d^{2}$,
$\mathfrak{c}$-Fréchet means are unique and equal to the usual notion of mean.
Furthermore, strong laws of large numbers in Hilbert spaces are well-known,
see e.g. [Kawabe, 1986]. Thus, SampleHeineBorel is true. Let it be noted that
proving SampleHeineBorel in a space where HeineBorel is false may be of
similar difficulty as showing convergence of Fréchet means directly. In the
case of infinite dimensional Banach spaces, results on strong laws of large
numbers for a different notion of mean – the Bochner integral – are well
established, see e.g. [Hoffmann-Jørgensen and Pisier, 1976].

  * •

On LowerBound: We illustrate this condition in the linear regression setting
with $\mathcal{Q}=\mathbb{R}^{s+1}$,
$\mathcal{Y}=(\\{1\\}\times\mathbb{R}^{s})\times\mathbb{R}$,
$\mathfrak{c}((x,y),\beta)=(y-\beta\\!^{\top}\\!x)^{2}-y^{2}=-2\beta\\!^{\top}\\!xy+\beta\\!^{\top}\\!xx\\!^{\top}\\!\beta$.
Let $(X,Y)$ be random variables with values in $\mathcal{Y}$. Let
$(X_{1},Y_{1}),\dots,(X_{n},Y_{n})$ be independent with the same distribution
as $(X,Y)$. We can set $o=0\in\mathbb{R}^{s+1}$,
$\mathfrak{a}^{+}=\lambda_{\mathsf{min}}(\mathbb{E}[XX\\!^{\top}\\!])$, where
$\lambda_{\mathsf{min}}$ denotes the smallest eigenvalue,
$\mathfrak{a}^{-}=2\|\mathbb{E}[XY]\|$,
$\mathfrak{a}^{+}_{n}=\lambda_{\mathsf{min}}(\frac{1}{n}\sum_{i=1}^{n}X_{i}X_{i}\\!^{\top}\\!)$,
$\mathfrak{a}^{-}_{n}=2\|\frac{1}{n}\sum_{i=1}^{n}X_{i}Y_{i}\|$,
$\psi^{+}(\delta)=\delta^{2}$ and $\psi^{-}(\delta)=\delta$. If
$\lambda_{\mathsf{min}}(\mathbb{E}[XX\\!^{\top}\\!])>0$, the largest
eigenvalue $\lambda_{\mathsf{max}}(\mathbb{E}[XX\\!^{\top}\\!])<\infty$, and
$\mathbb{E}[\|XY\|]<\infty$, all conditions are fulfilled.

For a further application of LowerBound, see the proof of 4.4 in the next
section.

######  Theorem 3.5.

Assume Polish, LowerSemiContinuity, IntegrableInf, SampleHeineBorel,
UpperBound, and LowerBound. Then

$d_{\subseteq}(M_{n},M)\xrightarrow{n\to\infty}_{\mathsf{a.s.}}0\,.$

###### Proof.

The proof consists of following steps:

  1. 1.

Apply Theorem 3.2.

  2. 2.

Reduction to a bounded set.

  3. 3.

Show that $M_{n}$ is eventually precompact almost surely.

  4. 4.

Apply Theorem 2.4.

Step 1. Polish, LowerSemiContinuity, and IntegrableInf are assumptions.
UpperBound implies Integrable. Thus, Theorem 3.2 yields
$\operatorname*{lim\,\overline{sup}}_{n\to\infty}\,M_{n}\subseteq M$ almost
surely.

Step 2. Define $F(q)=\mathbb{E}[\mathfrak{c}(Y,q)]$,
$F_{n}(q)=\frac{1}{n}\sum_{i=1}^{n}\mathfrak{c}(Y_{i},q)$. We want to show
that there is a bounded set $B_{1}\subseteq\mathcal{Q}$ such that $F(q)\geq
F(m)+1$ and $F_{n}(q)\geq F_{n}(m)+1$ for all $q\in\mathcal{Q}\setminus B_{1}$
and $m\in M$. If $\mathcal{Q}$ is bounded, we can take $B_{1}=\mathcal{Q}$.
Assume $\mathcal{Q}$ is not bounded.

Let $m\in M$. By UpperBound, $F(m)<\infty$. Let $o\in\mathcal{Q}$ from
LowerBound. Due to LowerBound,
$F(q)\geq\mathfrak{a}^{+}\psi^{+}(\delta)-\mathfrak{a}^{-}\psi^{-}(\delta)\geq
F(m)+2$ for all
$q\in\mathcal{Q}\setminus\operatorname{\mathrm{B}}_{\delta}(o)$ and $\delta$
large enough. This holds for all $m\in M$ as $F(m)$ does not change with $m$.
We set $B_{1}=\operatorname{\mathrm{B}}_{\delta}(o)$. For $F_{n}$, it holds
$F_{n}(m)\xrightarrow{n\to\infty}_{\mathsf{a.s.}}F(m)$ and
$\inf_{q\in\mathcal{Q}\setminus
B_{1}}F_{n}(q)\geq\mathfrak{a}^{+}_{n}\psi^{+}(\delta)-\mathfrak{a}^{-}_{n}\psi^{-}(\delta)$
with
$\mathfrak{a}^{+}_{n}\xrightarrow{n\to\infty}_{\mathsf{a.s.}}\mathfrak{a}^{+}$
and
$\mathfrak{a}^{-}_{n}\xrightarrow{n\to\infty}_{\mathsf{a.s.}}\mathfrak{a}^{-}$.
Thus, there is a random variable $N_{1}$ such that almost surely $F_{n}(q)\geq
F_{n}(m)+1$ for all $n\geq N_{1}$, $q\in\mathcal{Q}\setminus B_{1}$, and $m\in
M$.

Step 3. Take $N_{0}$ from SampleHeineBorel. Choose
$N_{2}\geq\max(N_{0},N_{1})$ such that $\epsilon_{n}<1$ for all $n\geq N_{2}$.
Then $M_{n}\subseteq B_{1}$ for all $n\geq N_{2}$. Thus, $\bigcup_{n\geq
N_{2}}M_{n}$ is bounded and – due to SampleHeineBorel, Polish, and C.5 –
precompact almost surely.

Step 4. Finally, step 1 and 3 together with Theorem 2.4 yield
$d_{\subseteq}(M_{n},M)\xrightarrow{n\to\infty}_{\mathsf{a.s.}}0$. ∎

##  4 Strong Laws for $H$-Fréchet Mean Sets

Let $(\mathcal{Q},d)$ be a metric space. Let $(\Omega,\Sigma,\mathbb{P})$ be a
probability space that is silently underlying all random variables in this
section. Let $Y$ be a random variable with values in $\mathcal{Q}$. Let
$h\colon[0,\infty)\to[0,\infty)$ be a non-decreasing function. Define
$H\colon[0,\infty)\to[0,\infty),x\mapsto\int_{0}^{x}h(t)\mathrm{d}t$. Fix an
arbitrary element $o\in\mathcal{Q}$. Denote the $H$-Fréchet mean set of $Y$ as
$M=\operatorname*{arg\,min}_{q\in\mathcal{Q}}\mathbb{E}[H(\overline{Y,\\!q})-H(\overline{Y,\\!o})]$.
Let $Y_{1},\dots,Y_{n}$ be independent random variables with the same
distribution as $Y$. Choose
$(\epsilon_{n})_{n\in\mathbb{N}}\subseteq[0,\infty)$ with
$\epsilon_{n}\xrightarrow{n\to\infty}0$. Set
$M_{n}=\epsilon_{n}\text{-}\operatorname*{arg\,min}_{q\in\mathcal{Q}}\frac{1}{n}\sum_{i=1}^{n}(H(\overline{Y_{i},\\!q})-H(\overline{Y_{i},\\!o}))$.

######  Assumptions 4.1.

  * •

InfiniteIncrease: $h(x)\xrightarrow{x\to\infty}\infty$.

  * •

Additivity: There is $b\in[1,\infty)$ such that $h(2x)\leq bh(x)$ for all
$x\geq 0$.

  * •

$h$-Moment: $\mathbb{E}[h(\overline{Y,\\!o})]<\infty$.

######  Remark 4.2.

  * •

On Additivity: This implies $h(x+y)\leq b(h(x)+h(y))$ for all $x,y\geq 0$, see
C.2 (appendix). If $h$ is concave, Additivity holds with $b=2$ and we even
have $h(x+y)\leq h(x)+h(y)$. This condition is not very restrictive, but it
excludes functions that grow exponentially.

######  Corollary 4.3.

Assume Polish, Additivity, and $h$-Moment. Then, almost surely,

$\operatorname*{lim\,\overline{sup}}_{n\to\infty}\,M_{n}\subseteq M\,.$

###### Proof.

We check the conditions of Theorem 3.2. Polish is an assumption.
LowerSemiContinuity is fulfilled as $(q,p)\mapsto d(q,p)$ and $x\mapsto H(x)$
are continuous. For Integrable, we note that $H$ is non-decreasing and apply
C.2 (i),

$\displaystyle\left|H(\overline{y,\\!q})-H(\overline{y,\\!o})\right|$
$\displaystyle\leq\left|\overline{y,\\!q}-\overline{y,\\!o}\right|h\\!\left(\max(\overline{y,\\!q},\overline{y,\\!o})\right)$
$\displaystyle\leq\overline{q,\\!o}\,h(\overline{q,\\!o}+\overline{y,\\!o})$
$\displaystyle\leq
b\,\overline{q,\\!o}\left(h(\overline{q,\\!o})+h(\overline{y,\\!o})\right)\,,$

where the last inequality follows from C.2 (ii) using Additivity. Thus,
$h$-Moment implies Integrable. To show IntegrableInf, we note that $H$ is non-
decreasing and apply C.2 (iii),

$\displaystyle H(\overline{y,\\!q})-H(\overline{y,\\!o})$ $\displaystyle\geq
H(\left|\overline{y,\\!o}-\overline{q,\\!o}\right|)-H(\overline{y,\\!o})$
$\displaystyle\geq
b^{-1}H(\overline{q,\\!o})-2\,\overline{q,\\!o}\,h(\overline{y,\\!o})$

due to Additivity. Furthermore,
$H(\delta)=\int_{0}^{\delta}h(x)\mathrm{d}x\geq\frac{1}{2}\delta
h(\frac{1}{2}\delta)$. With that, $h$-Moment implies IntegrableInf. Thus,
Theorem 3.2 can be applied. ∎

######  Corollary 4.4.

Assume SampleHeineBorel, Polish, Additivity, InfiniteIncrease, and $h$-Moment.
Then

$d_{\subseteq}(M_{n},M)\xrightarrow{n\to\infty}_{\mathsf{a.s.}}0\,.$

###### Proof.

We check the conditions of Theorem 3.5. SampleHeineBorel and Polish are
assumptions of the corollary. LowerSemiContinuity and IntegrableInf are shown
in the proof of 4.3. Following that proof, we find, due to Additivity,

$\displaystyle\left|H(\overline{y,\\!q})-H(\overline{y,\\!o})\right|$
$\displaystyle\leq
b\,\overline{q,\\!o}\left(h(\overline{q,\\!o})+h(\overline{y,\\!o})\right)\,,$
$\displaystyle H(\overline{y,\\!q})-H(\overline{y,\\!o})$ $\displaystyle\geq
b^{-1}H(\overline{q,\\!o})-2\,\overline{q,\\!o}\,h(\overline{y,\\!o})\,,$
$\displaystyle H(\delta)$ $\displaystyle\geq\frac{1}{2}\delta
h\\!\left(\frac{1}{2}\delta\right)\,.$

The first inequality together with $h$-Moment implies UpperBound. For
LowerBound, we use the second inequality. We set
$\psi^{+}(\delta)=b^{-1}H(\delta)$, $\psi^{-}(\delta)=2\delta$,
$\mathfrak{a}^{+}=\mathfrak{a}_{n}^{+}=1$,
$\mathfrak{a}^{-}=\mathbb{E}[h(\overline{Y,\\!o})]$, and
$\mathfrak{a}^{-}_{n}=\frac{1}{n}\sum_{i=1}^{n}h(\overline{Y_{i},\\!o})$ with
$\mathfrak{a}^{-}_{n}\xrightarrow{n\to\infty}_{\mathsf{a.s.}}\mathfrak{a}^{-}$
due to $h$-Moment. Because of the third inequality,
$\psi^{+}(\delta)/\psi^{-}(\delta)\geq\frac{1}{4}b^{-1}h(\frac{1}{2}\delta)\xrightarrow{\delta\to\infty}\infty$
by InfiniteIncrease. ∎

##  5 Strong Laws for $\alpha$-Fréchet Mean Sets

Let $(\mathcal{Q},d)$ be a metric space. Let $(\Omega,\Sigma,\mathbb{P})$ be a
probability space that is silently underlying all random variables in this
section. Let $Y$ be a random variable with values in $\mathcal{Y}$. Let
$\alpha>0$. Fix an arbitrary element $o\in\mathcal{Q}$. Denote the
$\alpha$-Fréchet mean set of $Y$ as
$M=\operatorname*{arg\,min}_{q\in\mathcal{Q}}\mathbb{E}[\overline{Y,\\!q}^{\alpha}-\overline{Y,\\!o}^{\alpha}]$.
Let $Y_{1},\dots,Y_{n}$ be independent random variables with the same
distribution as $Y$. Choose
$(\epsilon_{n})_{n\in\mathbb{N}}\subseteq[0,\infty)$ with
$\epsilon_{n}\xrightarrow{n\to\infty}0$. Set
$M_{n}=\epsilon_{n}\text{-}\operatorname*{arg\,min}_{q\in\mathcal{Q}}\frac{1}{n}\sum_{i=1}^{n}(\overline{Y_{i},\\!q}^{\alpha}-\overline{Y_{i},\\!o}^{\alpha})$.

######  Corollary 5.1.

Let $\alpha>1$. Assume $\mathbb{E}[\overline{Y,\\!o}^{\alpha-1}]<\infty$ and
Polish.

  1. (i)

Then $\operatorname*{lim\,\overline{sup}}_{n\to\infty}\,M_{n}\subseteq M$
almost surely.

  2. (ii)

Additionally, assume SampleHeineBorel. Then
$d_{\subseteq}(M_{n},M)\xrightarrow{n\to\infty}_{\mathsf{a.s.}}0$.

###### Proof.

Set $h(x)=\alpha^{-1}x^{\alpha-1}$. This function is non-decreasing, fulfills
Additivity with $b=2^{\alpha-1}$ and InfiniteIncrease. Due to
$\mathbb{E}[\overline{Y,\\!o}^{\alpha-1}]<\infty$, $h$-Moment is fulfilled.
Furthermore, $H(x)=x^{\alpha}$. Thus, 4.4 and 4.3 imply the claims. ∎

######  Corollary 5.2.

Let $\alpha\in(0,1]$. Assume Polish.

  1. (i)

Then $\operatorname*{lim\,\overline{sup}}_{n\to\infty}\,M_{n}\subseteq M$
almost surely.

  2. (ii)

Additionally, assume SampleHeineBorel. Then
$d_{\subseteq}(M_{n},M)\xrightarrow{n\to\infty}_{\mathsf{a.s.}}0$.

###### Proof.

First, consider the case $\alpha=1$. Apply C.3 (appendix) on
$\overline{Y,\\!o}$ to obtain a function $h\colon[0,\infty)\to[0,\infty)$
which is strictly increasing, continuous, concave, fulfills InfiniteIncrease,
and $\mathbb{E}[h(\overline{Y,\\!o})]<\infty$. Concavity implies Additivity
with $b=1$. As its derivative is strictly increasing,
$H(x)=\int_{0}^{x}h(t)\mathrm{d}t$ is convex and strictly increasing. Thus,
$H$ has an inverse $H^{-1}$ and $H^{-1}$ is concave. This implies that
$d_{H}(q,p)=H^{-1}(\overline{q,\\!p})$ is a metric.

As $H^{-1}$ is concave, there are $u_{0},u_{1}\in[0,\infty)$ such that
$H^{-1}(x)\leq u_{0}+u_{1}x$ for all $x\geq 0$. As $h$ is concave, there are
$v_{0},v_{1}\in[0,\infty)$ such that $h(u_{0}+u_{1}x)\leq v_{0}+v_{1}h(x)$ for
all $x\geq 0$. Thus,
$\mathbb{E}[h(d_{H}(Y,o))]=\mathbb{E}[h(H^{-1}(\overline{Y,\\!o}))]\leq
v_{0}+v_{1}\mathbb{E}[h(\overline{Y,\\!o})]<\infty$. Hence, $h$-Moment is true
for the metric $d_{H}$.

Moreover, Polish and HeineBorel-type properties of $(\mathcal{Q},d)$ are
preserved in $(\mathcal{Q},d_{H})$, as $H^{-1}$ is strictly increasing,
concave, and continuous, with $H^{-1}(0)=0$ and
$H^{-1}(\delta)\xrightarrow{\delta\to\infty}\infty$, and thus, the properties
boundedness, compactness, separability, and completeness coincide for $d$ and
$d_{H}$. Applying 4.3 and 4.4 on the minimizers of
$\mathbb{E}[H(d_{H}(Y,q))-H(d_{H}(Y,o))]=\mathbb{E}[\overline{Y,\\!q}-\overline{Y,\\!o}]$
now yields the claims for $\alpha=1$.

For $\alpha\in(0,1)$ just note, that $\tilde{d}(q,p)=d(q,p)^{\alpha}$ is a
metric, which preserves Polish and HeineBorel-type properties, and apply the
result for $\alpha=1$ on $\tilde{d}$. ∎

######  Remark 5.3.

For convergence, we need the $(\alpha-1)$-moment to be finite in the case of
$\alpha\geq 1$. [Schötz, 2019, Corollary 5] shows that in metric spaces with
nonnegative curvature the typical parametric rate of convergence
$n^{-\frac{1}{2}}$ is obtained for $\alpha$-Fréchet means assuming the
$2(\alpha-1)$-moment to be finite in the case of $\alpha\in[1,2]$ under some
further conditions.

##  Appendix A Example: The Set of Medians

Let $s\in\mathbb{N}$. Consider the metric space $(\mathbb{R}^{s},d_{1})$,
where $d_{1}(q,p)=\|q-p\|_{1}=\sum_{j=1}^{s}\left|q_{j}-p_{j}\right|$. The
power Fréchet mean with $\alpha=1$ in this space is equivalent to the standard
($\alpha=2$) Fréchet mean in $(\mathbb{R}^{s},d_{1}^{\frac{1}{2}})$. For $s=1$
it is equal to the median. Let $Y=(Y^{1},\dots,Y^{s})$ be a random vector in
$\mathbb{R}^{s}$ such that
$\mathbb{P}(Y^{k}=0)=\mathbb{P}(Y^{k}=1)=\frac{1}{2}$ for $k=1,\dots,s$ and
$Y^{1},\dots,Y^{s}$ are independent. Let $Y_{1},Y_{2},\dots$ be independent
and identically distributed copies of $Y$. Let
$M=\operatorname*{arg\,min}_{q\in\mathbb{R}^{s}}\mathbb{E}[d_{1}(Y,q)]$ be the
Fréchet mean set of $Y$ and
$M_{n}=\epsilon_{n}\text{-}\operatorname*{arg\,min}_{q\in\mathbb{R}^{s}}\frac{1}{n}\sum_{i=1}^{n}d_{1}(Y_{i},q)$
its sample version.

###  A.1 No Convergence in Hausdorff Distance

First consider the case $s=1$ and $\epsilon_{n}=0$. As $s=1$,
$M=\operatorname*{arg\,min}_{q\in\mathbb{R}}\mathbb{E}[\left\nonscript\;|\nonscript\;Y-q\right\nonscript\;|\nonscript\;]$
is the median of $Y$, which is $M=[0,1]$ as
$2\mathbb{E}[\left\nonscript\;|\nonscript\;Y-q\right\nonscript\;|\nonscript\;]=\left|1-q\right|+\left|q\right|$
achieves its minimal value $1$ precisely for all $q\in[0,1]$. Define
$p_{n}:=\frac{1}{n}\sum_{i=1}^{n}Y_{i}$. Then the empirical objective function
is $F_{n}(q):=p_{n}\left|1-q\right|+(1-p_{n})\left|q\right|$, i.e., the sample
Fréchet mean set is $M_{n}=\operatorname*{arg\,min}_{q\in\mathbb{R}}F_{n}(q)$.
If $n$ is odd, then either $M_{n}=\\{0\\}$ or $M_{n}=\\{1\\}$ holds. The same
is true for an even value of $n$ except when $p_{n}=\frac{1}{2}$, in which
case $M_{n}=[0,1]$. Thus, $d_{\subseteq}(M_{n},M)=0$, but
$d_{\mathsf{H}}(M_{n},M)$ does not converge almost surely.

###  A.2 The Outer Limit as a Strict Subset

Next, we keep $\epsilon_{n}=0$, but consider the value of
$\operatorname*{lim\,\overline{sup}}_{n\to\infty}M_{n}$ in a multi-dimensional
setting, i.e., $s\in\mathbb{N}$, as this yields a potentially surprising
result: By C.4, $M$ is just the Cartesian product of the median sets in each
dimension, i.e., $M=[0,1]^{s}$ (this is not to be confused with the geometric
median, which is the Fréchet mean with respect to the square root of the
Euclidean norm). Similarly, $M_{n}=\bigtimes_{k=1}^{s}M_{n}^{k}$ decomposes
into the sample Fréchet mean sets $M_{n}^{k}$ of each dimension $k=1,\dots,s$.
It holds $M_{n}^{k}=[0,1]$ if and only if the respective value of
$p_{n}^{k}:=\frac{1}{n}\sum_{i=1}^{n}Y_{i}^{k}$ is equal to $\frac{1}{2}$,
i.e., if and only if the symmetric simple random walk
$S_{n}^{k}:=\sum_{i=1}^{n}(2Y_{i}^{k}-1)$ hits $0$. Let
$N=\\#\\{n\in\mathbb{N}\nonscript\,|\allowbreak\nonscript\,\mathopen{}S_{n}^{1}=\dots=S_{n}^{s}=0\\}$.
Let $A\subseteq\mathbb{R}$ and $B_{n}\subseteq\mathbb{R}$ for all
$n\in\mathbb{N}$. If $A\subseteq B_{n}$ for infinitely many $n$, then
$A\subseteq\operatorname*{lim\,\overline{sup}}_{n\to\infty}B_{n}$. Thus, we
want to know whether $N$ is finite or infinite. This is answered by Pólya’s
Recurrence Theorem [Pólya, 1921]. It implies that for $s\in\\{1,2\\}$,
$N=\infty$ almost surely. Furthermore, if $s\geq 3$, then $N<\infty$ almost
surely. To find which points are not element of the outer limit of sample
Fréchet mean sets, note following fact: For an open subset
$A\subseteq\mathbb{R}$, if $A\subseteq\mathbb{R}\setminus B_{n}$ for all but
finitely many $n$, then
$A\subseteq\mathbb{R}\setminus\operatorname*{lim\,\overline{sup}}_{n\to\infty}B_{n}$.
We conclude, that in general a vector $x\in[0,1]^{s}$ is an element of
$\operatorname*{lim\,\overline{sup}}_{n\to\infty}M_{n}$ if and only if at most
two entries are not in $\\{0,1\\}$, i.e., almost surely

$\operatorname*{lim\,\overline{sup}}_{n\to\infty}M_{n}=\\{(x_{1},\dots,x_{s})\in[0,1]^{s}\nonscript\,|\allowbreak\nonscript\,\mathopen{}\\#\\{k\in\\{1,\dots,s\\}|x_{k}\in(0,1)\\}\leq
2\\}\,.$

Thus, $\operatorname*{lim\,\overline{sup}}_{n\to\infty}M_{n}=M$ for
$s\in\\{1,2\\}$ and
$\operatorname*{lim\,\overline{sup}}_{n\to\infty}M_{n}\subsetneq M$ for $s\geq
3$.

###  A.3 Convergence in Hausdorff Distance

Lastly, we use the setting $s=1$ and $\epsilon_{n}\in[0,\infty)$, where we
want to find $\epsilon_{n}$ such that $[0,1]\subseteq M_{n}$. At least one of
$0$ and $1$ is a minimizer of
$F_{n}(q)=p_{n}\left|1-q\right|+(1-p_{n})\left|q\right|$ and the $F_{n}(q)$ is
linear on $[0,1]$. Thus, $[0,1]\subseteq M_{n}$ if and only if
$\epsilon_{n}\geq|F_{n}(0)-F_{n}(1)|$. This is equivalent to
$|p_{n}-\frac{1}{2}|<\frac{1}{2}\epsilon_{n}$. By Markov’s inequality

$\mathbb{P}\mathopen{}\left(\left|p_{n}-\frac{1}{2}\right|\geq\frac{1}{2}\epsilon_{n}\right)\mathclose{}\leq\frac{n^{-3}\mathbb{E}\left[(Y-\frac{1}{2})^{4}\right]}{2^{-4}\epsilon_{n}^{4}}\,.$

For $\epsilon_{n}=n^{-\frac{1}{4}}$, we obtain

$\sum_{n=1}^{\infty}\mathbb{P}\mathopen{}\left(\left|p_{n}-\frac{1}{2}\right|\geq\frac{1}{2}\epsilon_{n}\right)\mathclose{}\leq\sum_{n=1}^{\infty}n^{-2}<\infty\,.$

The Borel–Cantelli lemma implies that almost surely and for all $n$ large
enough, $|p_{n}-\frac{1}{2}|<\frac{1}{2}\epsilon_{n}$ and thus,
$[0,1]\subseteq M_{n}$. Together with 5.2, we obtain
$d_{\mathsf{H}}(M_{n},M)\xrightarrow{n\to\infty}_{\mathsf{a.s.}}0$.

##  Appendix B Alternative Route to One-Sided Hausdorff Convergence

In this section, we show an alternative proof of a strong law of large numbers
for generalized Fréchet mean sets in one-sided Hausdorff distance. Although
the final result, Theorem B.5, is weaker than Theorem 3.5, it is very
illustrative to follow this line of proof: In contrast to the arguments in the
main part of the article, it does not rely on the powerful result [Korf and
Wets, 2001, Theorem 1.1], which seems to be rather complex to prove. Instead
our reasoning here is simpler and more self-contained. Furthermore, a
comparison between convergence in outer limit and in one-sided Hausdorff
distance seems more natural in view of the deterministic results Theorem 2.3
and Theorem B.2, and the stochastic results Theorem 3.2 and Theorem B.5.

###  B.1 Convergence of Minimizer Sets of Deterministic Functions

Let $(\mathcal{Q},d)$ be a metric space. For $A\subseteq\mathcal{Q}$ and
$\delta>0$, denote $\operatorname{\mathrm{B}}_{\delta}(A)=\bigcup_{x\in
A}\operatorname{\mathrm{B}}_{\delta}(x)$.

######  Definition B.1.

  1. (i)

Let $f,f_{n}\colon\mathcal{Q}\to\mathbb{R}$, $n\in\mathbb{N}$. The sequence
$(f_{n})_{n\in\mathbb{N}}$ _converges_ to $f$ _uniformly on bounded sets_ if
and only if for every $B\subseteq\mathcal{Q}$ with
$\operatorname{\mathsf{diam}}(B)<\infty$,

$\lim_{n\to\infty}\sup_{x\in B}\left|f_{n}(x)-f(x)\right|=0\,.$

We then write $f_{n}\xrightarrow{n\to\infty}_{\mathsf{ubs}}f$.

  2. (ii)

A sequence $(B_{n})_{n\in\mathbb{N}}$ of sets $B_{n}\subseteq\mathcal{Q}$ is
called _eventually bounded_ if and only if

$\limsup_{n\to\infty}\operatorname{\mathsf{diam}}\\!\left(\bigcup_{k=n}^{\infty}B_{k}\right)<\infty\,.$

  3. (iii)

A function $f$ has _approachable minimizers_ if and only if for all
$\epsilon>0$ there is a $\delta>0$ such that
$\delta\text{-}\operatorname*{arg\,min}f\subseteq
B_{\epsilon}(\operatorname*{arg\,min}f)$.

The last definition directly implies that
$d_{\subseteq}(\delta\text{-}\operatorname*{arg\,min}f,\operatorname*{arg\,min}f)\xrightarrow{\delta\to
0}0$ is equivalent to $f$ having approachable minimizers. Furthermore, if $f$
has approachable minimizers, then $\operatorname*{arg\,min}f\neq\emptyset$, as
for every $\delta>0$ the set $\delta\text{-}\operatorname*{arg\,min}f$ is non-
empty, but $\operatorname{\mathrm{B}}_{\epsilon}(\emptyset)=\emptyset$.

######  Theorem B.2.

Let $f,f_{n}\colon\mathcal{Q}\to\mathbb{R}$. Let
$(\epsilon_{n})_{n\in\mathbb{N}}\subseteq[0,\infty)$ with
$\epsilon_{n}\xrightarrow{n\to\infty}0$. Assume $f$ has approachable
minimizers, $f_{n}\xrightarrow{n\to\infty}_{\mathsf{ubs}}f$, and
$(\epsilon_{n}\text{-}\operatorname*{arg\,min}f_{n})_{n\in\mathbb{N}}$ is
eventually bounded. Then

$d_{\subseteq}(\epsilon_{n}\text{-}\operatorname*{arg\,min}f_{n},\operatorname*{arg\,min}f)\xrightarrow{n\to\infty}0$

and

$\inf f_{n}\xrightarrow{n\to\infty}\inf f\,.$

###### Proof.

Let $\epsilon>0$. As $f$ has approachable minimizers, there is $\delta>0$ such
that
$(3\delta)\text{-}\operatorname*{arg\,min}f\subseteq\operatorname{\mathrm{B}}_{\epsilon}(\operatorname*{arg\,min}f)$;
also $\operatorname*{arg\,min}f\neq\emptyset$. Let
$y\in\operatorname*{arg\,min}f$. As $f_{n}(y)\xrightarrow{n\to\infty}f(y)$,
there is $n_{1}\in\mathbb{N}$ such that $\inf f_{n}\leq\inf f+\delta$ for all
$n\geq n_{1}$. As $\epsilon_{n}\xrightarrow{n\to\infty}0$, there is
$n_{2}\in\mathbb{N}$ such that $\epsilon_{n}\leq\delta$ for all $n\geq n_{2}$.
As $(\epsilon_{n}\text{-}\operatorname*{arg\,min}f_{n})_{n\in\mathbb{N}}$ is
eventually bounded, there is $n_{3}\in\mathbb{N}$ such that
$\operatorname{\mathsf{diam}}(B)<\infty$ for $B=\bigcup_{n\geq
n_{3}}\epsilon_{n}\text{-}\operatorname*{arg\,min}f_{n}$. As
$f_{n}\xrightarrow{n\to\infty}_{\mathsf{ubs}}f$ there is $n_{4}$ such that
$\sup_{x\in B}\left|f_{n}(x)-f(x)\right|\leq\delta$. Let
$n\geq\max(n_{1},n_{2},n_{3},n_{4})$ and
$x\in\epsilon_{n}\text{-}\operatorname*{arg\,min}f_{n}$. Then

$f(x)\leq f_{n}(x)+\delta\leq\inf f_{n}+2\delta\leq\inf f+3\delta\,.$

Thus, $x\in(3\delta)\text{-}\operatorname*{arg\,min}f$. By the choice of
$\epsilon$ and $\delta$, we obtain
$\epsilon_{n}\text{-}\operatorname*{arg\,min}f_{n}\subseteq\operatorname{\mathrm{B}}_{\epsilon}(\operatorname*{arg\,min}f)$
or equivalently
$d_{\subseteq}(\epsilon_{n}\text{-}\operatorname*{arg\,min}f_{n},\operatorname*{arg\,min}f)\leq\epsilon$.

Finally, we show the convergence of the infima. We already know $\inf
f_{n}\leq\inf f+\epsilon$ for all $\epsilon>0$ and $n$ large enough. If $\inf
f_{n}\xrightarrow{n\to\infty}\inf f$ does not hold, there is a sequence
$x_{n}\in\epsilon_{n}\text{-}\operatorname*{arg\,min}f_{n}$ and $\epsilon>0$
such that $f_{n}(x_{n})<\inf f-\epsilon$ for all $n$ large enough. As before,
because of eventual boundedness and uniform convergence on bounded sets, we
have
$\sup_{k\in\mathbb{N}}\left|f_{n}(x_{k})-f(x_{k})\right|\xrightarrow{n\to\infty}0$.
Therefore, for all $\epsilon>0$ we have $f(x_{n})\leq f_{n}(x_{n})+\epsilon$
for $n$ large enough, which contradicts $f_{n}(x_{n})<\inf f-\epsilon$. ∎

In the following, we construct examples to show that none of the conditions
for one-sided Hausdorff convergence can be dropped.

######  Example B.3.

  1. (i)

Let $f,f_{n}\colon\mathbb{N}_{0}\to\mathbb{R}$,
$f_{n}=1-\mathds{1}_{\\!\left\\{0,n\right\\}}$,
$f=1-\mathds{1}_{\\!\left\\{0\right\\}}$, $d(i,j)=1$ for $i\neq j$. It holds
that $f$ is continuous and has approachable minimizers, and the sequence of
nonempty sets $\operatorname*{arg\,min}f_{n}=\left\\{0,n\right\\}$ is
eventually bounded, as $\operatorname{\mathsf{diam}}(A)\leq 1$ for every
$A\subseteq\mathbb{N}_{0}$. Furthermore, $f_{n}$ converges to $f$ uniformly on
compact sets, which are exactly the finite subsets of $\mathbb{N}_{0}$, but
not uniformly on bounded sets like $\mathbb{N}_{0}$ itself. There is a
subsequence of minimizers $x_{n}=n\in\operatorname*{arg\,min}f_{n}$ that is
always bounded away from $0$, the minimizer of $f$. This shows that uniform
convergence on compact sets (instead of bounded sets) is not enough.

  2. (ii)

As above, let $f,f_{n}\colon\mathbb{N}_{0}\to\mathbb{R}$,
$f_{n}=1-\mathds{1}_{\\!\left\\{0,n\right\\}}$,
$f=1-\mathds{1}_{\\!\left\\{0\right\\}}$, but define $d(i,j)=|i-j|$. It holds
that $f$ is continuous and has approachable minimizers, and
$f_{n}\xrightarrow{n\to\infty}_{\mathsf{ubs}}f$, but the sequence of nonempty
sets $\operatorname*{arg\,min}f_{n}=\left\\{0,n\right\\}$ is not eventually
bounded. Again, there is a subsequence of minimizers
$x_{n}=n\in\operatorname*{arg\,min}f_{n}$ that is always bounded away from
$0$, the minimizer of $f$. This shows that eventual boundedness of minimizer
sets cannot be dropped.

  3. (iii)

Let $f,f_{n}\colon\mathbb{N}_{0}\to\mathbb{R}$, $f(0)=0$, $f(i)=\frac{1}{i}$,
$f_{n}(i)=f(i)\mathds{1}_{\\!\mathopen{}\left\\{i<n\right\\}\mathclose{}}$,
and set $d(i,j)=1$ for $i\neq j$. It holds that $f$ is continuous, but $f$
does not have approachable minimizers. The sequence of nonempty sets
$\operatorname*{arg\,min}f_{n}=\left\\{0,n,n+1,\dots\right\\}$ is eventually
bounded and $f_{n}\xrightarrow{n\to\infty}_{\mathsf{ubs}}f$. There is a
subsequence of minimizers $x_{n}=n\in\operatorname*{arg\,min}f_{n}$ that is
always bounded away from $0$, the minimizer of $f$. This shows that
approachability of minimizers of $f$ cannot be dropped.

###  B.2 Strong Laws for $\mathfrak{c}$-Fréchet Mean Sets

Let $(\mathcal{Q},d)$ be a metric space, the descriptor space. Let
$\mathcal{Y}$ be a set, the data space. Let
$\mathfrak{c}\colon\mathcal{Y}\times\mathcal{Q}\to\mathbb{R}$ be a function,
the cost function. Let $(\Omega,\Sigma,\mathbb{P})$ be a probability space
that is silently underlying all random variables in this section. Let $Y$ be a
random variable with values in $\mathcal{Y}$. Denote the
$\mathfrak{c}$-Fréchet mean set of $Y$ as
$M=\operatorname*{arg\,min}_{q\in\mathcal{Q}}\mathbb{E}[\mathfrak{c}(Y,q)]$.
Let $Y_{1},\dots,Y_{n}$ be independent random variables with the same
distribution as $Y$. Choose
$(\epsilon_{n})_{n\in\mathbb{N}}\subseteq[0,\infty)$ with
$\epsilon_{n}\xrightarrow{n\to\infty}0$. Set
$M_{n}=\epsilon_{n}\text{-}\operatorname*{arg\,min}_{q\in\mathcal{Q}}\frac{1}{n}\sum_{i=1}^{n}\mathfrak{c}(Y_{i},q)$.

######  Assumptions B.4.

  * •

Continuity: The function $q\mapsto\mathfrak{c}(Y,q)$ is continuous almost
surely.

######  Theorem B.5.

Assume HeineBorel, Continuity, UpperBound, and LowerBound. Then

$d_{\subseteq}(M_{n},M)\xrightarrow{n\to\infty}_{\mathsf{a.s.}}0\,.$

###### Proof.

Define $F(q)=\mathbb{E}[\mathfrak{c}(Y,q)]$,
$F_{n}(q)=\frac{1}{n}\sum_{i=1}^{n}\mathfrak{c}(Y_{i},q)$. The proof consists
of following steps:

  1. 1.

Show that $F_{n}\xrightarrow{n\to\infty}_{\mathsf{ubs}}F$ almost surely.

  2. 2.

Reduction to a bounded set.

  3. 3.

Show that $F$ has approachable minimizers.

  4. 4.

Show that $M_{n}$ is eventually bounded.

  5. 5.

Apply Theorem B.2.

Step 1. To show uniform convergence on bounded sets, we will use the uniform
law of large numbers, Theorem C.1 below. Let $B\subseteq\mathcal{Q}$ be a
bounded set. By HeineBorel, $\overline{B}$ is compact. By Continuity,
$q\mapsto\mathfrak{c}(Y,q)$ is almost surely continuous. By UpperBound,
$\mathbb{E}[\sup_{q\in
B}\left\nonscript\;|\nonscript\;\mathfrak{c}(Y,q)\right\nonscript\;|\nonscript\;]<\infty$.
Thus, Theorem C.1 implies that $q\mapsto F(q)$ is continuous and

$\sup_{q\in
B}\left|F_{n}(q)-F(q)\right|\xrightarrow{n\to\infty}_{\mathsf{a.s.}}0\,.$

Fix an arbitrary element $o\in\mathcal{Q}$. For all bounded sets $B$, there is
$\delta\in\mathbb{N}$ such that
$B\subseteq\operatorname{\mathrm{B}}_{\delta}(o)$. By the previous
considerations, uniform convergence holds almost surely for all
$(\operatorname{\mathrm{B}}_{\delta}(o))_{\delta\in\mathbb{N}}$. Thus,
$F_{n}\xrightarrow{n\to\infty}_{\mathsf{ubs}}F$ almost surely.

Step 2. Find $B_{1}\subseteq\mathcal{Q}$ and a random variable
$N_{1}\in\mathbb{N}$ as in step 2 in the proof of Theorem 3.5.

Step 3. Clearly, $M\subseteq B_{1}$ is bounded. Furthermore, for all
$\epsilon>0$ small enough the set
$D_{\epsilon}=\overline{B_{1}\setminus\operatorname{\mathrm{B}}_{\epsilon}(M)}$
is not empty (if it is, increase $\delta$), does not contain any element of
$M$ and, by HeineBorel, is compact. Thus, the continuous function $q\mapsto
F(q)$ attains its infimum on $D_{\epsilon}$ where $\inf_{q\in
D_{\epsilon}}F(q)>\inf_{q\in\mathcal{Q}}F(q)$. Take
$\zeta=\min(1,\frac{1}{2}(\inf_{q\in
D_{\epsilon}}F(q)-\inf_{q\in\mathcal{Q}}F(q)))$. Then
$\zeta\text{-}\operatorname*{arg\,min}_{q\in\mathcal{Q}}F(q)\subseteq\operatorname{\mathrm{B}}_{\epsilon}(M)$,
i.e., $F$ has approachable minimizers.

Step 4. For $\epsilon_{n}<1$ and $n\geq N_{1}$, it holds $M_{n}\subseteq
B_{1}$. Thus, $(M_{n})_{n\in\mathbb{N}}$ is eventually bounded almost surely.

Step 5. Finally, Theorem B.2 implies
$d_{\subseteq}(M_{n},M)\xrightarrow{n\to\infty}_{\mathsf{a.s.}}0$. ∎

##  Appendix C Auxiliary Results

There are many versions of uniform laws of large numbers in the literature. We
state and prove one version that is tailored to our needs.

######  Theorem C.1.

Let $(\mathcal{Y},\Sigma_{\mathcal{Y}})$ be a measurable space and $Y$ be a
random variable with values in $\mathcal{Y}$. Let $Y_{1},\dots,Y_{n}$ be
independent and have the same distribution as $Y$. Let $(\mathcal{Q},d)$ be a
metric space and $B\subseteq\mathcal{Q}$ compact. Let
$f\colon\mathcal{Y}\times B\to\mathbb{R}$ be such that $q\mapsto f(Y,q)$ is
almost surely continuous. Assume there is a random variable $Z$ such that
$\left|f(Y,q)\right|\leq Z$ for all $q\in B$ with $\mathbb{E}[Z]<\infty$. Then
$q\mapsto\mathbb{E}[f(Y,q)]$ is continuous and

$\sup_{q\in
B}\left|\frac{1}{n}\sum_{i=1}^{n}f(Y_{i},q)-\mathbb{E}[f(Y,q)]\right|\xrightarrow{n\to\infty}_{\mathsf{a.s.}}0\,.$

###### Proof.

Let $\epsilon>0$. As $B$ is compact, there is a finite set
$\left\\{q_{1},\dots,q_{k}\right\\}\subseteq\mathcal{Q}$ such that
$B\subseteq\bigcup_{\ell=1}^{k}\operatorname{\mathrm{B}}_{\epsilon}(q_{\ell})$.
We split the supremum,

$\displaystyle\sup_{q\in
B}\left|\frac{1}{n}\sum_{i=1}^{n}f(Y_{i},q)-\mathbb{E}[f(Y,q)]\right|$
$\displaystyle\leq\sup_{\ell\in\left\\{1,\dots,k\right\\}}\sup_{q\in\operatorname{\mathrm{B}}_{\epsilon}(q_{\ell})}\left|\frac{1}{n}\sum_{i=1}^{n}\left(f(Y_{i},q)-f(Y_{i},q_{\ell})\right)-\mathbb{E}[f(Y,q)-f(Y,q_{\ell})]\right|$
$\displaystyle\quad+\sup_{\ell\in\left\\{1,\dots,k\right\\}}\left|\frac{1}{n}\sum_{i=1}^{n}f(Y_{i},q_{\ell})-\mathbb{E}[f(Y,q_{\ell})]\right|\,.$

For the second summand, by the standard strong law of large numbers applied to
each $\ell\in\\{1,\dots,k\\}$ with $\mathbb{E}[Z]<\infty$,

$\displaystyle\sup_{\ell\in\left\\{1,\dots,k\right\\}}\left|\frac{1}{n}\sum_{i=1}^{n}f(Y_{i},q_{\ell})-\mathbb{E}[f(Y,q_{\ell})]\right|\xrightarrow{n\to\infty}_{\mathsf{a.s.}}0\,.$

For the first summand,

$\displaystyle\sup_{\ell\in\left\\{1,\dots,k\right\\}}\sup_{q\in\operatorname{\mathrm{B}}_{\epsilon}(q_{\ell})}\left|\frac{1}{n}\sum_{i=1}^{n}\left(f(Y_{i},q)-f(Y_{i},q_{\ell})\right)-\mathbb{E}[f(Y,q)-f(Y,q_{\ell})]\right|$
$\displaystyle\leq\frac{1}{n}\sum_{i=1}^{n}\sup_{q,p\in
B,\,\overline{q,p}\leq\epsilon}\left|f(Y_{i},q)-f(Y_{i},p)\right|+\mathbb{E}\left[\sup_{q,p\in
B,\,\overline{q,p}\leq\epsilon}\left\nonscript\;\middle|\nonscript\;f(Y,q)-f(Y,p)\right\nonscript\;\middle|\nonscript\;\right]\,.$

By the standard strong law of large numbers with $\mathbb{E}[Z]<\infty$,

$\displaystyle\frac{1}{n}\sum_{i=1}^{n}\sup_{q,p\in
B,\,\overline{q,p}\leq\epsilon}\left|f(Y_{i},q)-f(Y_{i},p)\right|$
$\displaystyle\xrightarrow{n\to\infty}_{\mathsf{a.s.}}\mathbb{E}\left[\sup_{q,p\in
B,\,\overline{q,p}\leq\epsilon}\left\nonscript\;\middle|\nonscript\;f(Y,q)-f(Y,p)\right\nonscript\;\middle|\nonscript\;\right]\,.$

Thus,

$\mathbb{P}\mathopen{}\left(\limsup_{n\to\infty}\sup_{q\in
B}\left|\frac{1}{n}\sum_{i=1}^{n}f(Y_{i},q)-\mathbb{E}[f(Y,q)]\right|\leq
a_{\epsilon}\right)\mathclose{}=1\,,$ (1)

where $a_{\epsilon}=2\mathbb{E}\left[\sup_{q,p\in
B,\,\overline{q,p}\leq\epsilon}\left\nonscript\;\middle|\nonscript\;f(Y,q)-f(Y,p)\right\nonscript\;\middle|\nonscript\;\right]$.
As $q\mapsto f(Y,q)$ is almost surely continuous and $B$ is compact, $q\mapsto
f(Y,q)$ is almost surely uniformly continuous, i.e., for all $\delta>0$ there
is $\varepsilon(\delta,Y)>0$ such that $\left|f(Y,q)-f(Y,p)\right|\leq\delta$
for all $\overline{q,\\!p}\leq\varepsilon(\delta,Y)$. As
$\mathbb{E}[Z]<\infty$, we can use dominated convergence to obtain

$\displaystyle\lim_{\epsilon\searrow 0}\mathbb{E}\left[\sup_{q,p\in
B,\,\overline{q,p}\leq\epsilon}\left\nonscript\;\middle|\nonscript\;f(Y,p)-f(Y,p)\right\nonscript\;\middle|\nonscript\;\right]$
$\displaystyle=\mathbb{E}\left[\lim_{\epsilon\searrow 0}\sup_{q,p\in
B,\,\overline{q,p}\leq\epsilon}\left\nonscript\;\middle|\nonscript\;f(Y,p)-f(Y,p)\right\nonscript\;\middle|\nonscript\;\right]=0\,.$

Thus, $a_{\epsilon}\xrightarrow{\epsilon\searrow 0}0$. Together with (1), this
implies

$\sup_{q\in
B}\left|\frac{1}{n}\sum_{i=1}^{n}f(Y_{i},q)-\mathbb{E}[f(Y,q)]\right|\xrightarrow{n\to\infty}_{\mathsf{a.s.}}0\,.$

We have also shown that $q\mapsto\mathbb{E}[f(Y,q)]$ is continuous, as
$\left|\mathbb{E}[f(Y,q)]-\mathbb{E}[f(Y,p)]\right|\leq a_{\overline{q,p}}$. ∎

######  Lemma C.2.

Let $h\colon[0,\infty)\to[0,\infty)$ be a non-decreasing function. Define
$H\colon[0,\infty)\to[0,\infty),x\mapsto\int_{0}^{x}h(t)\mathrm{d}t$. Let
$x,y\geq 0$. Then

  1. (i)

$\left|H(x)-H(y)\right|\leq\left|x-y\right|h(\max(x,y))$.

Assume, there is $b\in[1,\infty)$ such that $h(2u)\leq bh(u)$ for all $u\geq
0$. Then

  1. (ii)

$\frac{1}{2}h(x)+\frac{1}{2}h(y)\leq h(x+y)\leq b\left(h(x)+h(y)\right)$,

  2. (iii)

$H(\left|x-y\right|)-H(x)\geq b^{-1}H(y)-2yh(x)$.

###### Proof.

  1. (i)

This is a direct consequence of the mean value theorem.

  2. (ii)

As $h$ is non-decreasing, $\max(h(x),h(y))\leq h(x+y)\leq\max(h(2x),h(2y))$.
By the definition of $b$ and with $\frac{1}{2}(u+v)\leq\max(u,v)\leq u+v$ for
$u,v\geq 0$ the claim follows.

  3. (iii)

First, consider the case $x\geq y$. Define
$f(x,y)=H(x-y)-H(x)-b^{-1}H(y)+2yh(x)$. We want to show $f(x,y)\geq 0$. The
derivative of $f$ with respect to $y$ is

$\partial_{y}f(x,y)=-h(x-y)-b^{-1}h(y)+2h(x)\,.$

By applying the first inequality of (ii) to $h(x)=h((x-y)+y)$, we obtain
$\partial_{y}f(x,y)\geq 0$ as $b^{-1}\leq 1$. Hence, $f(x,y)\geq f(x,0)=0$, as
$H(y)=0$.

Now, consider the case $x\leq y$. Set $g(x,y)=H(y-x)-H(x)-b^{-1}H(y)+2yh(x)$,
which yields

$\partial_{y}g(x,y)=h(y-x)-b^{-1}h(y)+2h(x)\,.$

By applying the second inequality of (ii) to $h(y)=h((y-x)+x)$, we obtain
$\partial_{y}g(x,y)\geq 0$ as $b^{-1}\leq 1$. Thus, $g(x,y)\geq
g(x,x)=-(1+b^{-1})H(x)+2xh(x)$ as $H(0)=0$. By the definition of $H$, as $h$
is non-decreasing, $H(x)\leq xh(x)$. Hence, $g(x,y)\geq 0$ as $1+b^{-1}\leq
2$.

Together, we have shown $H(\left|x-y\right|)-H(x)-b^{-1}H(y)+2yh(x)\geq 0$ for
all $x,y\geq 0$.

∎

######  Lemma C.3.

Let $X$ be a random variable with values in $[0,\infty)$. Then there is a
strictly increasing, continuous, and concave function
$h\colon[0,\infty)\to[0,\infty)$ with
$h(\delta)\xrightarrow{\delta\to\infty}\infty$ such that
$\mathbb{E}[h(X)]<\infty$.

###### Proof.

If there is $K>0$ such that $\mathbb{P}(X<K)=1$ take $h(x)=x$. Now, assume
that $X$ is not almost surely bounded. We first construct a non-decreasing
function $\tilde{h}\colon[0,\infty)\to[0,\infty)$ such that
$\tilde{h}(x)\xrightarrow{x\to\infty}\infty$ with
$\mathbb{E}[\tilde{h}(X)]<\infty$. Then we construct a function $h$ from
$\tilde{h}$ with all desired properties.

Let $F$ be the distribution function of $X$, $F(x)=\mathbb{P}(X\leq x)$. Let
$z_{1}=0$ and $z_{n+1}=\inf\mathopen{}\left\\{x\geq
z_{n}+1\,\big{|}\,1-F(x)\leq\frac{1}{n}\right\\}\mathclose{}$. As
$F(x)\xrightarrow{x\to\infty}1$, $z_{n}<\infty$. Furthermore,
$z_{n+1}-z_{n}\geq 1$. Moreover, as $X$ is not almost surely bounded,
$1-F(x)>0$ for all $x\geq 0$. Set

$\displaystyle g(x)$
$\displaystyle=\sum_{n=1}^{\infty}(z_{n+1}-z_{n})^{-1}n^{-2}\mathds{1}_{[z_{n},z_{n+1})}(x)\,,$
$\displaystyle\tilde{h}(x)$
$\displaystyle=\int_{0}^{x}\frac{g(t)}{1-F(t)}\mathrm{d}t\,.$

Then

$\displaystyle\lim_{x\to\infty}\tilde{h}(x)=\int_{0}^{\infty}\frac{g(t)}{1-F(t)}\mathrm{d}t\geq\sum_{n=1}^{\infty}n^{-1}=\infty\,.$

Moreover, $\tilde{h}(x)$ is strictly increasing, as $g(t)\geq 0$ and
$1-F(t)\geq 0$. The function $\tilde{h}$ is continuously differentiable
everywhere except at point $z_{n}$, $n\in\mathbb{N}$. Thus,

$\displaystyle\mathbb{E}[\tilde{h}(X)]$
$\displaystyle=\int_{0}^{\infty}\mathbb{P}\mathopen{}\left(\tilde{h}(X)>t\right)\mathclose{}\mathrm{d}t$
$\displaystyle=\int_{0}^{\infty}\mathbb{P}\mathopen{}\left(X>\tilde{h}^{-1}(t)\right)\mathclose{}\mathrm{d}t$
$\displaystyle=\int_{0}^{\infty}\tilde{h}^{\prime}(t)\mathbb{P}\mathopen{}\left(X>t\right)\mathclose{}\mathrm{d}t$
$\displaystyle=\int_{0}^{\infty}g(t)\mathrm{d}t$
$\displaystyle=\sum_{n=1}^{\infty}n^{-2}<\infty\,.$

Let $a_{0}=1$, $x_{0}=0$, $x_{n+1}=\inf\mathopen{}\left\\{x\geq
x_{n}+a_{n}^{-1}\,\big{|}\,\tilde{h}(x)\geq n+1\right\\}\mathclose{}$ and
$a_{n+1}=(x_{n+1}-x_{n})^{-1}$. Let $h\colon[0,\infty)\to[0,\infty)$ be the
linear interpolation of $(x_{n},n)_{n\in\mathbb{N}_{0}}$. As
$\tilde{h}(x)\xrightarrow{x\to\infty}\infty$, all $x_{n}$ are finite. Hence,
$h(x)\xrightarrow{x\to\infty}\infty$. Because of $a_{n}>0$, $h$ is strictly
increasing. Furthermore, $a_{n+1}\leq a_{n}$ as $x_{n+1}\geq
x_{n}+a_{n}^{-1}$. As $h$ is continuous and $a_{n}$ is the derivative of $h$
in the interval $(x_{n},x_{n+1})$, $h$ is concave. Lastly,
$h(x)\leq\tilde{h}(x)+1$. Thus, $\mathbb{E}[h(X)]<\infty$. ∎

######  Lemma C.4 (Fréchet means in product spaces).

Let $K\in\mathbb{N}$. Let
$(\mathcal{Q}_{1},d_{1}),\dots,(\mathcal{Q}_{K},d_{K})$ be metric spaces. Let
$\alpha\geq 1$. Set $\mathcal{Q}:=\bigtimes_{k=1}^{K}\mathcal{Q}_{k}$ and
$d\colon\mathcal{Q}\times\mathcal{Q}\to[0,\infty)$,
$d(q,p):=(\sum_{k=1}^{K}d_{k}(q_{k},p_{k})^{\alpha})^{\frac{1}{\alpha}}$. Then
$(\mathcal{Q},d)$ is a metric space. Let $Y=(Y^{1},\dots,Y^{K})$ be a tuple of
random variables such that $Y^{k}$ has values in $\mathcal{Q}_{k}$ and
$\mathbb{E}[d(Y,o)^{\alpha-1}]<\infty$ for an element $o\in\mathcal{Q}$. Let
$M$ be the $\alpha$-Fréchet mean set of $Y$ in $(\mathcal{Q},d)$, and $M^{k}$
be the $\alpha$-Fréchet mean set of $Y^{k}$ in $(\mathcal{Q}_{k},d_{k})$. Then
$M$ is the Cartesian product of the sets $M^{k}$, i.e.,

$M=\bigtimes_{k=1}^{K}M^{k}\,.$

C.4 can be proven by straight forward calculations.

######  Lemma C.5.

Let $(\mathcal{Q},d)$ be a complete metric space and $A\subseteq\mathcal{Q}$.
Assume that all closed subsets $B\subseteq A$ are compact. Then $\overline{A}$
is compact.

###### Proof.

Let $(a_{k})_{k\in\mathbb{N}}\subseteq\overline{A}$. We show that
$(a_{k})_{k\in\mathbb{N}}$ has a converging subsequence with limit
$a\in\overline{A}$, which implies compactness of $\overline{A}$.

Let $\delta_{n}=2^{-n}$. Define
$B_{n}:=\mathcal{Q}\setminus\operatorname{\mathrm{B}}_{\delta_{n}}(\mathcal{Q}\setminus
A)$. The sets $B_{n}$ are closed and subsets of $A$. Thus, they are compact.
Let $b_{k}^{n}\in\operatorname*{arg\,min}_{b\in B_{n}}d(b,a_{k})$. Such an
element exists as $B_{n}$ is compact. Furthermore,
$d(b_{k}^{n},a_{k})\leq\delta_{n}$. Define the subindex sequences
$(k(n,\ell))_{\ell\in\mathbb{N}}\subseteq\mathbb{N}$ such that
$k(0,\ell)=\ell$ and $(b^{n}_{k(n,\ell)})_{\ell\in\mathbb{N}}$ is a converging
subsequence of $(b^{n}_{k(n-1,\ell)})_{\ell\in\mathbb{N}}$ with limit
$b^{n}_{k(n,\ell)}\xrightarrow{\ell\to\infty}b^{n}_{\infty}$ and
$d(b^{n}_{k(n,\ell)},b^{n}_{\infty})\leq\delta_{\ell}$. By the triangle
inequality $d(b^{n_{1}}_{k},b^{n_{2}}_{k})\leq
d(b^{n_{1}}_{k},a_{k})+d(b^{n_{2}}_{k},a_{k})\leq\delta_{n_{1}}+\delta_{n_{2}}$.
Thus,
$d(b^{n_{1}}_{\infty},b^{n_{2}}_{\infty})\leq\delta_{n_{1}}+\delta_{n_{2}}$,
which makes $(b^{n}_{\infty})_{n\in\mathbb{N}}$ a Cauchy-sequence. Define $a$
as its limit, i.e., $b^{n}_{\infty}\xrightarrow{n\to\infty}a$. As
$b^{n}_{\infty}\in B_{n}\subseteq\overline{A}$, also $a\in\overline{A}$.
Finally, the triangle inequality yields

$d(a_{k(n,n)},a)\leq
d(a_{k(n,n)},b^{n}_{k(n,n)})+d(b^{n}_{k(n,n)},b^{n}_{\infty})+d(b^{n}_{\infty},a)\xrightarrow{n\to\infty}0\,,$

i.e., $(a_{k(n,n)})_{n\in\mathbb{N}}$ is subsequence of
$(a_{k})_{k\in\mathbb{N}}$ which converges in $\overline{A}$. ∎

## References

  * [Arnaudon et al., 2013] Arnaudon, M., Barbaresco, F., and Yang, L. (2013).  Medians and means in Riemannian geometry: existence, uniqueness and computation.  In Matrix information geometry, pages 169–197. Springer, Heidelberg.
  * [Artstein and Wets, 1995] Artstein, Z. and Wets, R. J.-B. (1995).  Consistency of minimizers and the SLLN for stochastic programs.  J. Convex Anal., 2(1-2):1–17.
  * [Bhattacharya and Patrangenaru, 2003] Bhattacharya, R. and Patrangenaru, V. (2003).  Large sample theory of intrinsic and extrinsic sample means on manifolds. I.  Ann. Statist., 31(1):1–29.
  * [Choirat et al., 2003] Choirat, C., Hess, C., and Seri, R. (2003).  A functional version of the birkhoff ergodic theorem for a normal integrand: A variational approach.  Ann. Probab., 31(1):63–92.
  * [Edwards, 1995] Edwards, R. E. (1995).  Functional analysis.  Dover Publications, Inc., New York.  Theory and applications, Corrected reprint of the 1965 original.
  * [Evans and Jaffe, 2020] Evans, S. N. and Jaffe, A. Q. (2020).  Strong laws of large numbers for Fréchet means.  arXiv preprint.
  * [Fréchet, 1948] Fréchet, M. (1948).  Les éléments aléatoires de nature quelconque dans un espace distancié.  Ann. Inst. H. Poincaré, 10:215–310.
  * [Hoffmann-Jørgensen and Pisier, 1976] Hoffmann-Jørgensen, J. and Pisier, G. (1976).  The law of large numbers and the central limit theorem in Banach spaces.  Ann. Probability, 4(4):587–599.
  * [Hotz and Huckemann, 2015] Hotz, T. and Huckemann, S. (2015).  Intrinsic means on the circle: uniqueness, locus and asymptotics.  Ann. Inst. Statist. Math., 67(1):177–193.
  * [Huckemann, 2011] Huckemann, S. F. (2011).  Intrinsic inference on the mean geodesic of planar shapes and tree discrimination by leaf growth.  Ann. Statist., 39(2):1098–1124.
  * [Kawabe, 1986] Kawabe, J. (1986).  Characterization of Hilbert spaces by the strong law of large numbers.  J. Multivariate Anal., 20(1):155–160.
  * [Kemperman, 1987] Kemperman, J. H. B. (1987).  The median of a finite measure on a Banach space.  In Statistical data analysis based on the $L_{1}$-norm and related methods (Neuchâtel, 1987), pages 217–230. North-Holland, Amsterdam.
  * [Korf and Wets, 2001] Korf, L. A. and Wets, R. J.-B. (2001).  Random lsc functions: An ergodic theorem.  Mathematics of Operations Research, 26(2):421–445.
  * [MacQueen, 1967] MacQueen, J. (1967).  Some methods for classification and analysis of multivariate observations.  In Proceedings of the Fifth Berkeley Symposium on Mathematical Statistics and Probability, Volume 1: Statistics, pages 281–297, Berkeley, Calif. University of California Press.
  * [Pólya, 1921] Pólya, G. (1921).  Über eine Aufgabe der Wahrscheinlichkeitsrechnung betreffend die Irrfahrt im Straßennetz.  Math. Ann., 84(1-2):149–160.
  * [Rockafellar and Wets, 1998] Rockafellar, R. and Wets, R. J.-B. (1998).  Variational Analysis.  Springer Verlag, Heidelberg, Berlin, New York.
  * [Schötz, 2019] Schötz, C. (2019).  Convergence rates for the generalized Fréchet mean via the quadruple inequality.  Electron. J. Stat., 13(2):4280–4345.
  * [Sturm, 2003] Sturm, K.-T. (2003).  Probability measures on metric spaces of nonpositive curvature.  In Heat kernels and analysis on manifolds, graphs, and metric spaces (Paris, 2002), volume 338 of Contemp. Math., pages 357–390. Amer. Math. Soc., Providence, RI.
  * [Sverdrup-Thygeson, 1981] Sverdrup-Thygeson, H. (1981).  Strong law of large numbers for measures of central tendency and dispersion of random variables in compact metric spaces.  Ann. Statist., 9(1):141–145.
  * [Williamson and Janos, 1987] Williamson, R. and Janos, L. (1987).  Constructing metrics with the Heine-Borel property.  Proc. Amer. Math. Soc., 100(3):567–573.
  * [Ziezold, 1977] Ziezold, H. (1977).  On expected figures and a strong law of large numbers for random elements in quasi-metric spaces.  In Transactions of the Seventh Prague Conference on Information Theory, Statistical Decision Functions, Random Processes and of the Eighth European Meeting of Statisticians (Tech. Univ. Prague, Prague, 1974), Vol. A, pages 591–602.