# Absolute Eigenvalues-Based Covariance Matrix Estimation for a Sparse Array

Kaushallya Adhikari This work was supported by the U.S. Office of Naval
Research under Grant N00014-20-1-2820. The author is with the University of
Rhode Island. e-mail<EMAIL_ADDRESS>
###### Abstract

The ensemble covariance matrix of a wide sense stationary signal spatially
sampled by a full linear array is positive semi-definite and Toeplitz.
However, the direct augmented covariance matrix of an augmentable sparse array
is Toeplitz but not positive semi-definite, resulting in negative eigenvalues
that pose inherent challenges in its applications, including model order
estimation and source localization. The positive eigenvalues-based covariance
matrix for augmentable sparse arrays is robust but the matrix is unobtainable
when all noise eigenvalues of the direct augmented matrix are negative, which
is a possible case. To address this problem, we propose a robust covariance
matrix for augmentable sparse arrays that leverages both positive and negative
noise eigenvalues. The proposed covariance matrix estimate can be used in
conjunction with subspace based algorithms and adaptive beamformers to yield
accurate signal direction estimates.

###### Index Terms:

Covariance matrix, DOA estimation, positive semi-definite, sparse array,
Toeplitz

##  I Introduction

Estimation of directions of arrival (DOAs) of wide-sense stationary (WSS)
signals using the data sampled by an array of sensors is an important task in
many fields such as sonar, radar, seismology, radio astronomy, and
neurophysiology. The data sampled by an $L$-element uniform linear array (ULA)
that has an intersensor spacing of $\lambda/2,$ where $\lambda$ is the
wavelength of the WSS signals impinging on the array, can be used to resolve
up to $L-1$ DOAs [1, 2]. There exist augmentable sparse arrays for which the
number of resolvable DOAs exceeds the number of sensors. An $L_{S}$-element
augmentable sparse array that has an aperture of $(L_{F}-1)\lambda/2$ can
resolve $L_{A}-1$ DOAs where $L_{F}\geq L_{A}\geq L_{S}$. Sparse arrays where
$L_{A}=L_{F}$ are called fully augmentable arrays and sparse arrays where
$L_{S}<L_{A}<L_{F}$ are called partially augmentable arrays. Minimum redundant
arrays [3] and nested arrays [4] are examples of fully augmentable sparse
arrays. Semi-coprime arrays [5] are examples of partially augmentable sparse
arrays. Coprime arrays [6] may be partially or fully augmentable based on the
design parameters.

The accuracy and resolution of DOA estimates for any array also depends on the
estimation algorithm. Conventional beamforming-based algorithms such as
product processing and min processing have lower resolution than algorithms
that are based on estimates of second order statistics (signal covariance
matrix) such as multiple signal classification (MUSIC) [7] or minimum variance
distortionless response (MVDR) beamformer [8, 9, 10, 11, 12, 13, 14, 15, 16]
for augmentable sparse arrays. This work focuses on the algorithm that uses
signal covariance estimates.

The performance of a DOA estimation algorithm such as MUSIC and MVDR depends
on the accuracy of the covariance matrix estimate. The ensemble covariance
matrix associated with an impinging WSS signal possesses two important
properties: positive semi-definite and Toeplitz structure. Many covariance
matrix estimates found in the literature do not possess one or both of these
properties. For instance, the sample covariance matrix (SCM) of a full ULA (to
be defined in Section II), which is widely used in DOA estimation, is positive
semi-definite but is not guaranteed to be Toeplitz. The direct augmented
matrix (DAM) of a sparse array (to be defined in Section II) is Toeplitz but
is not guaranteed to be semi-definite [17, 18]. For sparse arrays, Abramovich
_et al._ have presented an algorithm for estimating a positive definite
Toeplitz covariance matrix, called positive eigenvalues-based matrix (PEM)
[17]. This algorithm generates reasonable covariance matrix estimates when the
corresponding DAM has at least one non-negative noise eigenvalue. However,
when the DAM’s noise eigenvalues are all negative, this algorithm is
inapplicable. We present a new algorithm to estimate covariance matrix for
sparse arrays that works even when all noise eigenvalues of the DAM are
negative. Morever, when the proposed covariance matrix is used in conjunction
with subspace based methods such as MUSIC [7] or adaptive algorithms such as
MVDR beamformer [19], the DOA estimates are more accurate or commensurate with
the PEM.

##  II Problem Formulation

Consider an $L_{S}$-sensor linear array oriented along the positive $z$-axis.
The sensor locations along the $z$-axis are $d_{1}=0,$ $d_{2}\lambda/2$,
$d_{3}\lambda/2$, $...$, $d_{L_{S}}\lambda/2=(L_{F}-1)\lambda/2$, where
$d_{2}$, $d_{3}$, $...$, $d_{L_{S}}$ are assumed to be positive integers. Note
that a ULA is a special case of this $L_{S}$-sensor array where $d_{2}=1$,
$d_{3}=2$, $...$, $d_{L_{S}}=L_{S}-1.$ If the data sampled by the array
consist of $Q$ WSS uncorrelated planewaves in white Gaussian noise, the
received data vector at time $t$ is given by

$\mathbf{x}(t)=\sum_{i=1}^{Q}\mathbf{v}(\theta_{i})s_{i}(t)+\mathbf{n}(t),$
(1)

where $s_{i}(t)$ is the complex amplitude of the $i^{th}$ signal,
$\mathbf{v}(\theta_{i})$ is the array manifold vector corresponding to the
$i^{th}$ signal, $\theta_{i}$ is the $i^{th}$ planewave angle measured from
the positive $z$-axis, and $\mathbf{n}(t)$ is the $L_{S}$-element noise
vector. We assume that the variables $s_{i}(t)$ and $\mathbf{n}(t)$ are zero-
mean proper complex Gaussian random variable and random vector, respectively.
The $k^{th}$ element of the array manifold vector $\mathbf{v}(\theta_{i})$ is
$\exp(j\pi u_{i}d_{k}),$ where $u_{i}=\cos(\theta_{i})$ is the direction
cosine.

The SCM corresponding to the data vector in (1) is
$\mathbf{x}(t)\mathbf{x}(t)^{H}$. Since the SCM is the outer product of a
vector with itself, it is guaranteed to be positive semi-definite. If there
are $T$ snapshots sampled at times $t=t_{1},$ $t_{2},$ $...,$ $t_{T},$
respectively, the SCM is given by the average of the data vectors’ outer
products,
$\boldsymbol{\mathcal{S}}=\sum_{k=1}^{T}\mathbf{x}(t_{k})\mathbf{x}^{H}(t_{k})/T$.
The dimension of $\boldsymbol{\mathcal{S}}$ is $L_{S}\times L_{S}.$ To obtain
a Toeplitz estimate, $\boldsymbol{\mathcal{T}}$, we can apply redundancy
averaging, which entails replacing every element of $\boldsymbol{\mathcal{S}}$
by the average of the elements along the diagonal in which it resides. Since
the dimension of $\boldsymbol{\mathcal{T}}$ is $L_{S}\times L_{S}$, it can be
used to estimate up to only $L_{S}-1$ DOAs. Sparse arrays such as MRAs,
coprime arrays, and nested arrays are popular for their ability to resolve
more DOAs than the number of sensors, which cannot be realized with
$\boldsymbol{\mathcal{S}}$ or $\boldsymbol{\mathcal{T}}.$ To utilize all
degrees of freedom offered by a sparse array, consider the direct augmented
matrix (DAM) which is obtained as described below:

  * •

Step 1: Find the possible covariance estimate at each possible lag as
$r_{(0)}[k]=y[k]\star y^{*}[-k]$, where the symbol $\star$ denotes discrete
convolution and $y[k]$ is the measurement made by the sensor at
$z=k\lambda/2.$ If there is no sensor present at $z=k\lambda/2,$ $y[k]$ is
considered $0.$ The covariance function $r_{(0)}[k]$ is defined for lags in
the range $-(L_{F}-1)\leq k\leq L_{F}-1.$ However, some of the values of
$r_{(0)}[k]$ are $0$ in the range $-(L_{F}-1)\leq k\leq L_{F}-1$ if the sparse
array is only partially augmentable.

  * •

Step 2: Extract the hole-free covariance function estimate, $r_{(1)}[k]$ by
truncating $r_{(0)}[k]$. This step is not needed in a fully augmentable sparse
array. The function $r_{(1)}[k]$ is defined for lags in the range
$-(L_{A}-1)\leq k\leq L_{A}-1.$

  * •

Step 3: Remove the redundancy in $r_{(1)}[k]$ by discarding the values
corresponding to negative lags. We will refer to the new covariance function
as $r_{(2)}[k]$, which is defined for lags in the range $0\leq k\leq L_{A}-1.$

  * •

Step 4: Remove the bias in $r_{(2)}[k]$ to obtain $r_{(3)}[k]$ as
$r_{(2)}[k]/c[k]$, where $c[k]$ is the hole-free coarray of the sparse array
corresponding to positive lags.

  * •

Step 5: The DAM is obtained by forming a Toeplitz matrix that has the elements
$r_{(3)}[k]$ for $k=0,$ $1,$ $...,$ $L_{A}-1$ along its first row. This matrix
is denoted by $\boldsymbol{\mathcal{T}}_{DAM}$ in the sequel.

The DAM can be directly used with eigenanalysis based DOA estimation
algorithms such as MUSIC [20, 21]. However, the DAM is not guaranteed to be
positive semi-definite [17, 18].

##  III Positive Eigenvalues-Based Matrix

An iterative algorithm to estimate a positive definite covariance matrix of
dimension $L_{A}\times L_{A}$ for a sparse array is described in [17]. This
algorithm uses the alternating convex projection method in [22] to compute an
augmented covariance matrix that is both Toeplitz and positive definite. The
algorithm keeps alternating between enforcing Toeplitz property and positive
definiteness until the algorithm converges to a matrix that is both Toeplitz
and positive definite. We summarize the essential components of their
algorithm below:

  * •

Step 1: Set $\boldsymbol{\mathcal{T}}_{DAM}$ as the initial estimate of the
covariance matrix: $\hat{\mathbf{T}}^{(0)}=\boldsymbol{\mathcal{T}}_{DAM}$.

  * •

Step 2: If $\hat{\lambda}_{1},$ $\hat{\lambda}_{2},$ $...,$
$\hat{\lambda}_{L_{A}}$ are the eigenvalues of $\hat{\mathbf{T}}^{(0)}$ in
descending order, form the positive definite Hermitian matrix
$\hat{\mathbf{H}}$ corresponding to $\hat{\mathbf{T}}^{(0)}$ as

$\hat{\mathbf{H}}=\bar{\lambda}_{n}\mathbf{I}_{L_{A}}+\sum_{j=1}^{Q}\Bigl{(}\hat{\lambda}_{j}-\bar{\lambda}_{n}\Bigr{)}\mathbf{v}_{j}\mathbf{v}_{j}^{H},$
(2)

where

$\bar{\lambda}_{n}=\frac{1}{L_{A}-Q}\sum_{j=Q+1}^{L_{A}}\hat{\lambda}_{j}$ (3)

is the average of the noise eigenvalues and $\mathbf{I}_{L_{A}}$ is the
$L_{A}\times L_{A}$ identity matrix.

  * •

Step 3: Given the positive definite matrix $\hat{\mathbf{H}}$, form the
Toeplitz matrix $\hat{\mathbf{T}}^{(1)}$ by applying redundancy averaging.

  * •

Step 4: Let $\hat{\lambda}_{1},$ $\hat{\lambda}_{2},$ $...,$
$\hat{\lambda}_{L_{A}}$ be the eigenvalues of $\hat{\mathbf{T}}^{(1)}$ in
descending order. If
$\frac{\hat{\lambda}_{Q+1}-\hat{\lambda}_{L_{A}}}{\hat{\lambda}_{L_{A}}}\geq\epsilon<<1,$
where $\epsilon$ is a small positive number, then replace
$\hat{\mathbf{T}}^{(0)}$ in Step 1 by $\hat{\mathbf{T}}^{(1)}$ and repeat Step
2, 3, and 4.

This algorithm converges when the largest noise eigenvalue is approximately
equal to the smallest noise eigenvalue. [17] mentions that some of the noise
eigenvalues of the DAM might be negative. To tackle this problem, [17]
suggests discarding the negative noise eigenvalues and modifying (3) as

$\bar{\lambda}_{n}=\frac{1}{\sum_{j=Q+1}^{L_{A}}I[j]}\hskip
5.69046pt\sum_{j=Q+1}^{L_{A}}\hat{\lambda}_{j}I[j],$ (4)

where $I[j]=1$ if $\hat{\lambda}_{j}\geq 0$ and $I[j]=1$ if
$\hat{\lambda}_{j}<0.$

###  III-A Degenerative Case for PEM

One important possibility that has not been addressed in [17] is that every
noise eigenvalue of the DAM of a sparse matrix may be negative. The algorithm
in [17] is not applicable when all noise eigenvalues are negative because it
relies on having a positive subset of the noise eigenvalues to obtain
$\bar{\lambda}_{n}$ in (4). To illustrate this point, we consider a fully
augmentable sparse array depicted in Fig. 1. It is a coprime array formed by
interleaving Subarray 1 that has $4$ sensors and an intersensor spacing of
$3\lambda/2$ and Subarray 2 that has $5$ sensors and an intersensor spacing of
$2\lambda/2$. The number of sensors in the array is $7$ and the aperture is
$9\lambda/2.$ The corresponding DAM is of dimension $10\times 10.$ Consider a
scenario with two planewave sources, each of SNR $25$ dB. The source direction
cosines are selected as $u_{1}=0.2165\frac{4}{10}$ and
$u_{2}=-0.2165\frac{4}{10}$ so that the source locations are half-power
beamwidth apart for a full ULA with $9\lambda/2$ aperture [1, p. 999]. We used
$25$ snapshots to compute the DAM and find the corresponding eigenvalues. Note
that the number of negative eigenvalues differ from trial to trial. The
eigenvalues of the DAM were found to be $2603.9$, $294.9$, $-10.2$, $-6.9$,
$-6$, $-3.7$, $-3.1$, $-2.1$, $-1.7$, and $-1.2$ in one of the trials. The
first two eigenvalues are signal plus noise eigenvalues and are positive. The
remaining $8$ eigenvalues are noise eigenvalues and all values are negative. A
set of all negative noise eigenvalues poses an inherent challenge in
covariance matrix estimation. In the next section, we propose a covariance
matrix estimate to address this challenge.

Figure 1: A fully augmentable coprime array formed by interleaving Subarray 1
that has $4$ sensors and an intersensor spacing of $3\lambda/2$ and Subarray 2
that has $5$ sensors and an intersensor spacing of $2\lambda/2$.

##  IV Absolute Eigenvalues-Based Matrix

Figure 2: Comparison of normalized histograms (approximated PDFs) of the
minimum positive noise eigenvalue ($min(\hat{\lambda}^{(+)})$) and the minimum
absolute value of negative noise eigenvalue ($min(|\hat{\lambda}^{(-)}|)$) at
two different SNR values.

When the noise eigenvalues of the DAM are all negative, it renders the
alternating convex projection method of covariance estimation inapplicable. We
propose a method to estimate the augmented covariance matrix that leverages
negative noise eigenvalues instead of discarding them. The proposed covariance
matrix estimate is referred to as absolute eigenvalues-based matrix (AEM). To
motivate utilization of negative eigenvalues, consider the coprime array in
Fig. 1 in the two planewave scenario described in Section III-A. We
approximated the probability density functions (PDFs) of two random variables:
(1) minimum positive eigenvalue, $min(\lambda^{(+)})$, of the DAM; and (2)
minimum of the absolute values of the negative eigenvalues,
$min(|\lambda^{(-)}|)$. These PDFs were obtained using $1,000,000$
realizations of the DAM, with each realization consisting of $10$ snapshots.
The PDFs are depicted in Fig. 2 for $0$ dB and $10$ dB SNR. The close
resemblance between the PDFs of $min(\lambda^{(+)})$ and
$min(|\lambda^{(-)}|)$ suggests that negative noise eigenvalues can be used in
improving the value of $\bar{\lambda}_{n}$. Therefore, given the eigenvalues
of the DAM in descending order of their magnitudes
$|\hat{\lambda}_{1}|\geq|\hat{\lambda}_{2}|\geq,...,\geq|\hat{\lambda}_{L_{A}}|$,
we propose calculating average noise eigenvalues as

$\bar{\lambda}_{n}=\frac{1}{L_{A}-Q}\sum_{j=Q+1}^{L_{A}}|\hat{\lambda}_{j}|.$
(5)

The AEM is then formed as

$\boldsymbol{\mathcal{R}}=\sum_{j=1}^{Q}\hat{\lambda}_{j}\mathbf{v}_{j}\mathbf{v}_{j}^{H}+\sum_{j=Q+1}^{L_{A}}\bar{\lambda}_{n}\mathbf{v}_{j}\mathbf{v}_{j}^{H},$
(6)

where $\mathbf{v}_{j}$ is the eigenvector corresponding to $|\lambda_{j}|$.
Since each term in the two sums of (6) is a vector’s outer product with
itself, scaled by a non-negative scalar, $\boldsymbol{\mathcal{R}}$ is
guaranteed to be positive semi-definite. This approach does not fail when all
noise eigenvalues are negative. Moreover, when only some of the noise
eigenvalues are negative, using (5) provides an estimate of
$\bar{\lambda}_{n}$ that has less variance since it is obtained by averaging
over more realizations compared to (4). Note that $\boldsymbol{\mathcal{R}}$
in (6) is not guaranteed to be Toeplitz. We can alternate between Toeplitz and
positive semi-definite estimates following the approach used in the PEM
computation. However, our simulation results in Section V show that
$\boldsymbol{\mathcal{R}}$ in (6) is robust and does not warrant further
improvement with an iterative method.

##  V Results

Figure 3: Comparison of RMSEs for MUSIC using PEM (top panel) and AEM (bottom
panel).

Figure 4: Comparison of RMSEs for MVDR using PEM (top panel) and AEM (bottom
panel).

Figure 5: Comparison of RMSEs in DOA for MUSIC (top panel) and MVDR (bottom
panel) when AEM (red lines) and PEM (blue lines) are used as covariance matrix
estimates.

This section compares the DOA estimation performance of MUSIC and MVDR using
the covariance matrices PEM and AEM. We use the array in Fig. 1 in the two
uncorrelated planewave scenario described in Section III-A. For fair
comparison between the PEM and AEM based DOA estimation, we used the same
$1000$ datasets, discarding any dataset that consisted of all negative noise
eigenvalues since the PEM cannot be evaluated for such datasets.

Fig. 3 compares the root mean squared error (RMSE) in dB scale for DOA
estimation with MUSIC using PEM and AEM over a range of SNRs and numbers of
snapshots. The top panel corresponds to the PEM-based MUSIC and the bottom
panel corresponds to the AEM-based MUSIC. Comparing the two panels, we can
infer that there is negligible difference between AEM and PEM based MUSIC, in
cases when PEM is not degenerative. Similarly, Fig. 4 compares the RMSE with
MVDR using PEM (top panel) and AEM (bottom panel). For MVDR also, there is no
noticeable difference in the two panels. In Fig. 5, we plot several cuts of
Fig. 3 and Fig. 4. We consider three different numbers of snapshots: $5$ (less
than the number of sensors), $10$ (equal to the number of sensors), and $100$
(greater than the number of sensors). The solid linestyle with asterisk marker
indicates $5$ snapshots, the dash-dot linestyle with square markers indicates
$10$ snapshots, the dotted line with circle marker indicates $100$ snapshots;
and the red and blue colors indicate AEM and PEM, respectively. From these
RMSE plots, we can infer that in the cases when PEM is evaluable, the AEM-
based estimation is either as accurate as or more accurate than the PEM-based
estimation. This is true for both MUSIC and MVDR, which demonstrates that AEM
is more robust than PEM.

##  VI Conclusion

We presented a robust positive semi-definite covariance matrix, AEM, for
sparse arrays. The proposed approach does not degenerate when all eigenvalues
of the DAM are negative. As evidenced by the simulations, the AEM is more
robust than the competing covariance matrix estimate, PEM.

## References

  * [1] H. Van Trees, _Optimum Array Processing (Detection, Estimation and Modulation Theory, Part IV)_. John Wiley and Sons, Inc., New York, 2002.
  * [2] D. Johnson and D. Dudgeon, _Array Signal Processing: Concepts and Techniques_. Simon & Schuster, 1992.
  * [3] A. Moffet, “Minimum-redundancy linear arrays,” _IRE Trans. Antennas Propag._ , vol. 16, pp. 172–175, March 1968.
  * [4] P. Pal and P. Vaidyanathan, “Nested arrays: A novel approach to array processing with enhanced degrees of freedom,” _IEEE Transactions on Signal Processing_ , vol. 58, no. 8, pp. 4167 –4181, August 2010.
  * [5] K. Adhikari, “Beamforming with semi-coprime arrays,” _The Journal of the Acoustical Society of America_ , vol. 145, no. 5, pp. 2841–2850, 2019. [Online]. Available: https://doi.org/10.1121/1.5100281
  * [6] P. Vaidyanathan and P. Pal, “Sparse sensing with co-prime samplers and arrays,” _IEEE Transactions on Signal Processing_ , vol. 59, no. 2, pp. 573–586, February 2011.
  * [7] R. Schmidt, “Multiple emitter location and signal parameter estimation,” _IEEE Transactions on Antennas and Propagation_ , vol. 34, no. 3, pp. 276–280, March 1986.
  * [8] K. Adhikari, J. Buck, and K. Wage, “Extending coprime sensor arrays to achieve the peak side lobe height of a full uniform linear array,” _EURASIP Journal on Advances in Signal Processing_ , vol. 2014, no. 1, p. 148, Sep 2014\. [Online]. Available: https://doi.org/10.1186/1687-6180-2014-148
  * [9] G. Di Martino and A. Iodice, “Coprime synthetic aperture radar (copsar): A new acquisition mode for maritime surveillance,” _IEEE Transactions on Geoscience and Remote Sensing_ , vol. 53, no. 6, pp. 3110–3123, June 2015.
  * [10] K. Adhikari and B. Drozdenko, “Design and statistical analysis of tapered coprime and nested arrays for the min processor,” _IEEE Access_ , vol. 7, pp. 139 601–139 615, 2019.
  * [11] Y. Liu and J. R. Buck, “Gaussian source detection and spatial spectral estimation using a coprime sensor array with the min processor,” _IEEE Transactions on Signal Processing_ , vol. 66, no. 1, pp. 186–199, Jan 2018.
  * [12] K. Adhikari and J. Buck, “Gaussian signal detection with product arrays,” _IEEE Access_ , vol. 7, pp. 36 256–36 266, 2020.
  * [13] K. Adhikari, J. Buck, and K. Wage, “Beamforming with extended co-prime sensor arrays,” _2013 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)_ , pp. 4183–4186, May 2013.
  * [14] V. Chavali, K. Wage, and J. Buck, “Multiplicative and min processing of experimental passive sonar data from thinned arrays,” _journal = The Journal of the Acoustical Society of America_ , vol. 144, no. 6, pp. 3262–3274, December 2018.
  * [15] K. Adhikari and J. Buck, “Gaussian signal detection by coprime sensor arrays,” _2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)_ , pp. 2379–2383, April 2015.
  * [16] I. M. Rooney and J. R. Buck, “Spatial power spectral density estimation using a welch coprime sensor array processor,” _The Journal of the Acoustical Society of America_ , vol. 145, no. 4, pp. 2350–2362, 2019. [Online]. Available: https://doi.org/10.1121/1.5097572
  * [17] Y. I. Abramovich, D. A. Gray, A. Y. Gorokhov, and N. K. Spencer, “Positive-definite toeplitz completion in doa estimation for nonuniform linear antenna arrays. i. fully augmentable arrays,” _IEEE Transactions on Signal Processing_ , vol. 46, no. 9, pp. 2458–2471, 1998.
  * [18] K. Adhikari and J. Buck, “Spatial spectral estimation with product processing of a pair of colinear arrays,” _IEEE Transactions on Signal Processing_ , vol. 65, no. 9, pp. 2389–2401, May 2017.
  * [19] J. Capon, “High-resolution frequency-wavenumber spectrum analysis,” _Proceedings of the IEEE_ , vol. 57, no. 8, pp. 1408–1418, Aug 1969.
  * [20] K. Adhikari and B. Drozdenko, “Symmetry-imposed rectangular coprime and nested arrays for direction of arrival estimation with multiple signal classification,” _IEEE Access_ , vol. 7, pp. 153 217–153 229, 2019.
  * [21] ——, “Comparison of music variants for sparse arrays,” _2019 IEEE National Aerospace and Electronics Conference (NAECON)_ , pp. 398–405, 2019.
  * [22] K. M. Grigoriadis, A. E. Frazho, and R. E. Skelton, “Application of alternating convex projection methods for computation of positive toeplitz matrices,” _IEEE Transactions on Signal Processing_ , vol. 42, no. 7, pp. 1873–1875, 1994.


††thanks: These authors contributed equally to this work††thanks: These
authors contributed equally to this work

# One-Dimensional Edge Contact to Encapsulated MoS2 with a Superconductor

A. Seredinski  School of Sciences and Humanities, Wentworth Institute of
Technology, Boston, MA 02115  Department of Physics, Duke University, Durham,
NC, 27708  E.G. Arnault  Department of Physics, Duke University, Durham, NC,
27708  V.Z. Costa  Department of Physics and Astronomy, San Francisco State
University, San Francisco, CA 94132  L. Zhao  Department of Physics, Duke
University, Durham, NC, 27708  T.F.Q. Larson  Department of Physics, Duke
University, Durham, NC, 27708  K. Watanabe  National Institute for Materials
Science, Namiki 1-1, Tsukuba, Ibaraki 305-0044, Japan  T. Taniguchi  National
Institute for Materials Science, Namiki 1-1, Tsukuba, Ibaraki 305-0044, Japan
F. Amet  Department of Physics and Astronomy, Appalachian State University,
Boone, NC 28607  A.K.M. Newaz  Department of Physics and Astronomy, San
Francisco State University, San Francisco, CA 94132  G. Finkelstein
Department of Physics, Duke University, Durham, NC, 27708

###### Abstract

Establishing ohmic contact to van der Waals semiconductors such as MoS2 is
crucial to unlocking their full potential in next-generation electronic
devices. Encapsulation of few layer MoS2 with hBN preserves the material’s
electronic properties but makes electrical contacts more challenging. Progress
toward high quality edge contact to encapsulated MoS2 has been recently
reported. Here, we evaluate a contact methodology using sputtered MoRe, a Type
II superconductor with a relatively high critical field and temperature
commonly used to induce superconductivity in graphene. We find that the
contact transparency is poor and that the devices do not support a measurable
supercurrent down to 3 Kelvin, which has ramifications for future fabrication
recipes.

††preprint: AIP/123-QED

Soon after the isolation of monolayer graphene, it was found that mono- and
few-layer crystals could be isolated from transition metal dichalcogenides
(TMDs) Novoselov  _et al._ (2005). TMDs host an array of interesting phenomena
including superconductivity, charge density waves, and quantum spin Hall
states Manzeli  _et al._ (2017). Among the library of TMDs, molybdenym
disulfide (MoS2) has attracted attention due to its layer-dependent band
structure Mak  _et al._ (2010); Lee  _et al._ (2010), high mobility
Radisavljevic  _et al._ (2011); Kim  _et al._ (2012); Baugher  _et al._
(2013), large spin-orbit interaction Zhu, Cheng, and Schwingenschlögl (2011);
Xiao  _et al._ (2012); Kośmider, González, and Fernández-Rossier (2013), and
gate-induced superconductivity Ye  _et al._ (2012); Taniguchi  _et al._
(2012); Lu  _et al._ (2015); Costanzo  _et al._ (2016). Encapsulation of MoS2
with hexagonal boron nitride (hBN) both protects it from atmosphere and
separates it from sources of disorder Lee  _et al._ (2015); Cao  _et al._
(2015). However, due to Schottky barriers, a readily formed oxide layer, and
the fabrication challenges that come along with encapsulation, ohmic contact
to hBN/MoS2/hBN heterostructures has proven difficult.

Figure 1: (a) Optical image of the first sample. The black outline shows the
location of the encapsulated MoS2. Scale bar 5 $\mu$m. (b) Schematic side view
of the one-dimensional edge contact between the encapsulated MoS2 and the
sputtered MoRe (not to scale).

Low temperature ohmic contact of normal metals to encapsulated MoS2 has been
achieved through workfunction engineeringCui  _et al._ (2017) as well as
intervening graphene layers Lee  _et al._ (2015); Cui  _et al._ (2015).
Recently, progress has been made in one-dimensional edge contact to MoS2 with
normal metal through in situ Ar+ sputtering Jain  _et al._ (2019); Cheng  _et
al._ (2019). It would be highly desirable to develop superconducting edge
contact to MoS2, which could enable the study of the Josephson junction
physics taking advantage of MoS2’s spin-orbit and spin-valley couplings.

In this work we make one-dimensional edge contact to encapsulated MoS2 using
molybdenum-rhenium (MoRe), a Type II superconductor known to form high
transparency contact to MoS2 for a 2D interface Island  _et al._ (2016). We
utilize a recipe known to make ohmic edge contacts to hBN-encapsulated
graphene Calado  _et al._ (2015); Borzenets  _et al._ (2016). Our measurements
show low transparency contact to MoS2 that is improved neither by Ar+
sputtering pre-treatment of the contact interfaces nor by annealing. These
results indicate the probable presence of interfacial tunnel barriers. This
result may prove informative for groups developing hybrid samples made of van
der Waals heterostructures with superconducting contacts.

We study two MoS2 devices encapsulated within hBN. Both samples are contacted
by several MoRe electrodes, which define a series of Josephson junctions of
different lengths. The first device uses bilayer MoS2, while the second device
uses monolayer MoS2. Figure 1 shows an optical image of the first device as
well as a schematic view of the one-dimensional edge contact between the MoS2
and MoRe, created via reactive ion etching and sputtering. The second device
underwent an in situ Ar+ sputtering pre-treatment immediately before MoRe
deposition.

Figure 2: (a) Gate voltage dependence of the $I-V$ characteristics in a 200 nm
long, 5 $\mu$m wide junction on the first device ($J_{1}$). Inset: resistance
at high $V_{SD}$ for each gate voltage. The junction is seen to be highly
resistive across applied gate and bias voltages, and no signs of
superconducting behavior are visible. (b) $I-V$ curves for junctions $J_{1-3}$
of the first sample at $V_{BG}=42$ V. There is no significant difference
between the 200 nm and 500 nm long junctions, indicating that the current is
limited by the contacts. Inset: top-down schematic of the sample with
$J_{1-3}$ labeled.

Both van der Waals heterostructures were assembled from mechanically
exfoliated flakes using a dry transfer technique utilizing a polyethylene
terephthalate stamp. Polymer residue was removed by immersion in
dichloromethane for one hour at 70 ∘C followed by several hours at room
temperature.

The one-dimensional interface between the MoS2 and the MoRe was prepared via
standard electron-beam lithography techniques, reactive ion etching (RIE), and
sputtering. RIE consisted of three steps, all carried out with a process
pressure of 10-1 Torr. First, a ten second CHF3 / O2 (10:1 flow rate ratio)
step removed leftover e-beam resist (PMMA) residue from the top surface of the
heterostructure. This was followed by a ten second SF6 process to etch through
the top hBN. Finally, a ten second CF4 step was used to etch the MoS2 in the
contact region. While a CF4 etch is a typical process for MoS2, SF6 may itself
be sufficient Jain  _et al._ (2019). In order to limit the device’s exposure
to atmosphere, and so the formation of MoOx along the interface, the device
was not removed from the system and imaged between these steps.

The devices had minimal exposure to air before being transferred to the
sputtering system. The second sample was treated with Ar+ sputtering before
metal deposition to refresh the contact interface. The chamber was pumped to a
pressure of $\sim 10^{-8}$ Torr and 100 nm of MoRe (50-50% by weight) was
sputtered on both devices. To minimize processing, the Josephson junctions
were not shaped with further etching, so the flakes of MoS2 continue beyond
the boundaries of the junctions. This is visible in Figure 1a, which shows an
optical image of the first device.

The samples are cooled in a closed-cycle cryocooler with a base temperature of
3 K. Unless otherwise noted, a voltage $V_{applied}$ is applied to the
junction in series with a protective $R_{S}=$10 M$\Omega$ resistor. The drain
current, $I_{D}$ is measured, and the source-drain voltage is calculated as
$V_{SD}=V_{applied}-R_{S}I_{D}$; as a result the curves in Figures 2 and 3
have different horizontal extent.

Figure 2a shows the effects of electrostatic gating on the $I-V$ curves of a
200 nm long and 5 $\mu$m wide junction made on the first device. The gate
voltage ($V_{BG}$) increases the Fermi level in the MoS2, causing it to
approach the conduction band. We observe that for increasing $V_{BG}$, the
threshold of $V_{SD}$ required to achieve a linear slope decreases. Figure 2b
demonstrates the $I-V$ curves measured for three junctions of different length
at the maximal gate voltage of 42 V. (See the schematic in the inset: $J_{1}$
is 200 nm long, and $J_{2,3}$ are 500 nm long.) It is clear that 1) the curves
show no significant length dependence, indicating that the current is limited
by the contact barriers; and 2) the measurements are consistent between the
three junctions, indicating uniform properties of the contacts. These initial
measurements are consistent with the presence of barriers (likely Schottky
barriers) at the interfaces Jain  _et al._ (2019). At the highest gate voltage
(42 V) the resistance is 2.4 M$\Omega$, corresponding to the the contact
resistance of $R_{c}\approx$ 6 M$\Omega$$\cdot$$\mu$m.

Due to this high contact resistance, we next anneal the sample at 200∘C for 17
hours in a vacuum of $10^{-6}$ mbar. Annealing processes have been shown to
decrease contact resistance in similar devices. This may be due to a host of
phenomena which change the bonding or structure at the interface Jain  _et
al._ (2019). In this study, the annealing resulted in higher contact
resistance, with an increase of as much as 40% at high bias and $V_{BG}=42$ V.
This decrease in contact quality may be due to the MoRe reflowing away from
the contact edge, as seen in gold junctions without an additional metal
sticking layer Jain  _et al._ (2019).

Figure 3: Temperature dependencies measured in the 200 nm long, 5 $\mu$m wide
junction in the first device. (a) Post-anneal $I-V$ characteristics. (b) Low
bias ($V_{SD}=0.05$ V) resistance $R$, plotted in linear and (inset) log
scale, which shows $R$ decaying with temperature. $V_{BG}=42$ V throughout.
(c) $\ln(I_{D})$ vs $(V_{SD}/\mathrm{Volt})^{1/2}$ plot of the same data
showing an approximately linear relationship in the intermediate temperature
range. This is consistent with thermionic transport across the contact
interfaces.

We study the behavior of the junction as a function of temperature to gain
insight into the poor contact quality. Figure 3a plots the $I-V$
characteristics of the same junction from 3 to 290 K. A clear reduction in
low-bias resistance spanning more than a decade is seen as the temperature
rises (Figure 3b). Such behavior is consistent with thermionic transport
across a barrier. This interpretation is supported by an approximately linear
relation between the log of the current and the square root of the bias
voltage in the device (Figure 3c) as expected, e.g., for a triangular Schottky
barrier Sze and Ng (2006). This relation breaks down for low bias voltages at
higher temperatures.

Due to the contact characteristics of this device, we study a second device
utilizing Ar+ sputtering immediately prior to the deposition of the MoRe
contacts, focusing on a 500 nm long and 5 $\mu$m wide junction. Despite this
change in deposition parameters and an overnight anneal at 300∘C in $10^{-6}$
mbar, this second device also displays high contact resistances at low
temperature. Utilizing a direct voltage biasing scheme without a 10 M$\Omega$
series resistor, we measure gate sweeps for different $V_{SD}$ (Figure 4).
Even at the highest applied VSD=5 V, the currents supported by the junction
are orders of magnitude lower than comparable or longer junctions made with
both top contacts Liu  _et al._ (2015); Smithe  _et al._ (2016) and high
quality normal metal edge contacts Jain  _et al._ (2019); Cheng  _et al._
(2019).

Figure 4: Current vs $V_{BG}$ sweeps measured in a 500 nm long by 5 $\mu$m
wide junction in the second device following the annealing, which show the
induced highly resistive behavior. The three curves correspond to $V_{SD}=$
1.5, 3, and 5 V. Inset: the same data in log scale.

In summary, we tested a methodology for making one-dimensional edge contact to
encapsulated MoS2 with MoRe, and found high contact resistances on the order
of M$\Omega\cdot\mu$m. This contact was not improved by annealing at 200-300
∘C. In situ Ar+ sputtering of the interface before the deposition of MoRe also
did not improve the contact quality. We conclude that the presence of tunnel
barriers limits the performance of these devices. The lack of length
dependence, consistency between different junctions, insensitivity to Ar+ pre-
cleaning, and the lack of improvement upon annealing all point to the presence
of intrinsic Schottky barriers at the interfaces.

Higher transparency contacts may be achieved in the future by replacing MoRe
with superconductors having a significantly higher or lower work function.
Nevertheless, the current contact recipe could support the use of MoS2 in more
complex superconducting heterostructures. Namely, TMDs, including MoS2Safeer
_et al._ (2019), are already used to induce the spin-orbit coupling in
graphene Wang  _et al._ (2016); Island  _et al._ (2019). One can extend these
studies to Josephson junctions by making superconducting contacts that would
selectively contact the graphene but not the TMD layer. In this context, our
work establishes an order of magnitude estimate for the (very small) current
expected to be shunted through an MoS2 layer in such a complex van der Waals
heterostructure.

###### Acknowledgements.

A.S., E.G.A, T.F.L., L.Z., and G.F. acknowledge support by the Office of Basic
Energy Sciences, U.S. Department of Energy, under Award de-sc0002765. V.Z.C.
and A.K.M.N. acknowledge support from the National Science Foundation Grant
ECCS-1708907 and Department of Defense Award (ID: 72495RTREP). K.W. and
T.T.acknowledge the Elemental Strategy Initiative conducted by the MEXT, Japan
and the CREST (JPMJCR15F3), JST. F.A. acknowledges the ARO under Award
W911NF-16-1-0132. This work was performed in part at the Duke University
Shared Materials Instrumentation Facility (SMIF), a member of the North
Carolina Research Triangle Nanotechnology Network (RTNN), which is supported
by the National Science Foundation (Grant ECCS-1542015) as part of the
National Nanotechnology Coordinated Infrastructure (NNCI).

## References

  * Novoselov  _et al._ (2005) K. S. Novoselov, D. Jiang, F. Schedin, T. J. Booth, V. V. Khotkevich, S. V. Morozov, and A. K. Geim, PNAS 102, 10451 (2005).
  * Manzeli  _et al._ (2017) S. Manzeli, D. Ovchinnikov, D. Pasquier, O. V. Yazyev, and A. Kis, Nature Reviews Materials 2, 17033 (2017).
  * Mak  _et al._ (2010) K. F. Mak, C. Lee, J. Hone, J. Shan, and T. F. Heinz, Physical Review Letters 105, 136805 (2010).
  * Lee  _et al._ (2010) C. Lee, H. Yan, L. E. Brus, T. F. Heinz, J. Hone, and S. Ryu, ACS Nano 4, 2695 (2010).
  * Radisavljevic  _et al._ (2011) B. Radisavljevic, A. Radenovic, J. Brivio, V. Giacometti, and A. Kis, Nature Nanotechnology 6, 147 (2011).
  * Kim  _et al._ (2012) S. Kim, A. Konar, W.-S. Hwang, J. H. Lee, J. Lee, J. Yang, C. Jung, H. Kim, J.-B. Yoo, J.-Y. Choi, Y. W. Jin, S. Y. Lee, D. Jena, W. Choi, and K. Kim, Nature Communications 3, 1011 (2012).
  * Baugher  _et al._ (2013) B. W. H. Baugher, H. O. H. Churchill, Y. Yang, and P. Jarillo-Herrero, Nano Letters 13, 4212 (2013).
  * Zhu, Cheng, and Schwingenschlögl (2011) Z. Y. Zhu, Y. C. Cheng, and U. Schwingenschlögl, Physical Review B 84, 153402 (2011).
  * Xiao  _et al._ (2012) D. Xiao, G.-B. Liu, W. Feng, X. Xu, and W. Yao, Physical Review Letters 108, 196802 (2012).
  * Kośmider, González, and Fernández-Rossier (2013) K. Kośmider, J. W. González, and J. Fernández-Rossier, Physical Review B 88, 245436 (2013).
  * Ye  _et al._ (2012) J. T. Ye, Y. J. Zhang, R. Akashi, M. S. Bahramy, R. Arita, and Y. Iwasa, Science 338, 1193 (2012).
  * Taniguchi  _et al._ (2012) K. Taniguchi, A. Matsumoto, H. Shimotani, and H. Takagi, Applied Physics Letters 101, 042603 (2012).
  * Lu  _et al._ (2015) J. M. Lu, O. Zheliuk, I. Leermakers, N. F. Q. Yuan, U. Zeitler, K. T. Law, and J. T. Ye, Science 350, 1353 (2015).
  * Costanzo  _et al._ (2016) D. Costanzo, S. Jo, H. Berger, and A. F. Morpurgo, Nature Nanotechnology 11, 339 (2016).
  * Lee  _et al._ (2015) G.-H. Lee, X. Cui, Y. D. Kim, G. Arefe, X. Zhang, C.-H. Lee, F. Ye, K. Watanabe, T. Taniguchi, P. Kim, and J. Hone, ACS Nano 9, 7019 (2015).
  * Cao  _et al._ (2015) Y. Cao, A. Mishchenko, G. L. Yu, E. Khestanova, A. P. Rooney, E. Prestat, A. V. Kretinin, P. Blake, M. B. Shalom, C. Woods, J. Chapman, G. Balakrishnan, I. V. Grigorieva, K. S. Novoselov, B. A. Piot, M. Potemski, K. Watanabe, T. Taniguchi, S. J. Haigh, A. K. Geim, and R. V. Gorbachev, Nano Letters 15, 4914 (2015).
  * Cui  _et al._ (2017) X. Cui, E.-M. Shih, L. A. Jauregui, S. H. Chae, Y. D. Kim, B. Li, D. Seo, K. Pistunova, J. Yin, J.-H. Park, H.-J. Choi, Y. H. Lee, K. Watanabe, T. Taniguchi, P. Kim, C. R. Dean, and J. C. Hone, Nano Letters 17, 4781 (2017).
  * Cui  _et al._ (2015) X. Cui, G.-H. Lee, Y. D. Kim, G. Arefe, P. Y. Huang, C.-H. Lee, D. A. Chenet, X. Zhang, L. Wang, F. Ye, F. Pizzocchero, B. S. Jessen, K. Watanabe, T. Taniguchi, D. A. Muller, T. Low, P. Kim, and J. Hone, Nature Nanotechnology 10, 534 (2015).
  * Jain  _et al._ (2019) A. Jain, Á. Szabó, M. Parzefall, E. Bonvin, T. Taniguchi, K. Watanabe, P. Bharadwaj, M. Luisier, and L. Novotny, Nano Letters 19, 6914 (2019).
  * Cheng  _et al._ (2019) Z. Cheng, Y. Yu, S. Singh, K. Price, S. G. Noyce, Y.-C. Lin, L. Cao, and A. D. Franklin, Nano Letters 19, 5077 (2019).
  * Island  _et al._ (2016) J. O. Island, G. A. Steele, H. S. J. van der Zant, and A. Castellanos-Gomez, 2D Materials 3, 031002 (2016).
  * Calado  _et al._ (2015) V. E. Calado, S. Goswami, G. Nanda, M. Diez, A. R. Akhmerov, K. Watanabe, T. Taniguchi, T. M. Klapwijk, and L. M. K. Vandersypen, Nature nanotechnology 10, 761 (2015).
  * Borzenets  _et al._ (2016) I. V. Borzenets, F. Amet, C. T. Ke, A. W. Draelos, M. T. Wei, A. Seredinski, K. Watanabe, T. Taniguchi, Y. Bomze, M. Yamamoto, S. Tarucha, and G. Finkelstein, Physical Review Letters 117, 237002 (2016).
  * Sze and Ng (2006) S. M. Sze and K. K. Ng,  _Physics of Semiconductor Devices_ (John Wiley & Sons, Ltd., 2006).
  * Liu  _et al._ (2015) W. Liu, D. Sarkar, J. Kang, W. Cao, and K. Banerjee, ACS Nano 9, 7904 (2015).
  * Smithe  _et al._ (2016) K. K. H. Smithe, C. D. English, S. V. Suryavanshi, and E. Pop, 2D Materials 4, 011009 (2016).
  * Safeer  _et al._ (2019) C. K. Safeer, J. Ingla-Aynés, F. Herling, J. H. Garcia, M. Vila, N. Ontoso, M. R. Calvo, S. Roche, L. E. Hueso, and F. Casanova, Nano Letters 19, 1074 (2019).
  * Wang  _et al._ (2016) Z. Wang, D.-K. Ki, J. Y. Khoo, D. Mauro, H. Berger, L. S. Levitov, and A. F. Morpurgo, Physical Review X 6, 041020 (2016).
  * Island  _et al._ (2019) J. O. Island, X. Cui, C. Lewandowski, J. Y. Khoo, E. M. Spanton, H. Zhou, D. Rhodes, J. C. Hone, T. Taniguchi, K. Watanabe, L. S. Levitov, M. P. Zaletel, and A. F. Young, Nature 571, 85 (2019).


&=\max\\{0,\tau-y\cdot\phi(\tilde{x})\\}-\max\\{0,\tau-y\cdot\phi(x)\\}\\\
&\geq
y\cdot\phi(x-\tilde{x})=\delta\phi(\tilde{v})\geq(L_{\phi}-\epsilon)d(\tilde{z},z);\end{array}$

if $\tau-y\cdot\phi(x)<0$, we have
$d(\tilde{z},z)=\left\llbracket\left(2(y\cdot\phi(x)-\tau)/\epsilon+\delta\right)y\tilde{v}\right\rrbracket=2(y\cdot\phi(x)-\tau)/\epsilon+\delta\geq\delta$
and

$\begin{array}[]{lllll}&\max\\{0,\tau-\psi(\tilde{z})\\}-\max\\{\tau-\psi(z)\\}\\\
&=\max\\{0,\tau-\tilde{y}\cdot\phi(\tilde{x})\\}-\max\\{0,\tau-y\cdot\phi(x)\\}\\\
&\geq\tau-y\cdot\phi(x)+\phi(\tilde{v})d(\tilde{z},z)\\\
&\geq-\frac{\epsilon}{2}d(\tilde{z},z)+\phi(\tilde{v})d(\tilde{z},z)\\\
&\geq(L_{\phi}-\epsilon)d(\tilde{z},z).\end{array}$

This means that $\max\\{0,\tau-\psi\\}$ satisfies Assumptions (A2) at
$\mathcal{Z}$ for any $\delta>0$. Next, we turn to Assumption (B). Fix $r>1$,
$\delta>0$, a dataset $\mathcal{Z}_{N}\subset\mathcal{Z}$ and the
corresponding empirical distribution $\mathbb{P}_{N}$. For any
$\hat{z}\in\mathcal{Z}_{N}$, we consider the following cases.

  * •

If $\mathrm{E}_{\mathbb{P}_{N}}[\max\left\\{0,\tau-\phi(Z)\right\\}^{r}]=0$,
(A2) and (B) are equivalent.

  * •

If $\mathrm{E}_{\mathbb{P}_{N}}[\max\left\\{0,\tau-\phi(Z)\right\\}^{r}]\neq
0$ and $\psi(\hat{z})\geq\tau$, one can choose $\tilde{z}=\hat{z}$ such that
(B) holds.

  * •

If $\mathrm{E}_{\mathbb{P}_{N}}[\max\left\\{0,\tau-\phi(Z)\right\\}^{r}]\neq
0$ and $\psi(\hat{z})<\tau$. For any $\sigma\geq\delta$, let
$\tilde{z}=(\tilde{x},\tilde{y})\in\mathcal{Z}$ be defined as
$\tilde{x}=\hat{x}-\hat{y}\sigma\tilde{v}$ and $\tilde{y}=\hat{y}$. Then we
have

$\begin{array}[]{ll}\psi(\tilde{z})=\tilde{y}\cdot\phi(\tilde{x})&=\hat{y}\cdot\phi(\hat{x})-\sigma\phi(\tilde{v})\\\
&=\psi(\hat{z})-\sigma\phi(\tilde{v})<\tau.\end{array}$

Moreover, one can see that
$d(\tilde{z},\hat{z})=\left\llbracket\tilde{x}-\hat{x}\right\rrbracket=\left\llbracket\hat{y}\sigma\tilde{v}\right\rrbracket=\sigma$
and

$\begin{array}[]{ll}&\max\left\\{0,\tau-\psi(\tilde{z})\right\\}-\max\left\\{0,\tau-\psi(\hat{z})\right\\}\\\
&=\hat{y}\cdot\phi(\hat{x}-\tilde{x})=\sigma\phi(\tilde{v})\geq(L_{\phi}-\epsilon)d(\tilde{z},\hat{z}).\end{array}$

This means that $\max\\{0,\tau-\phi\\}$ satisfies Assumptions (B) at
$\mathcal{Z}$ for any $\delta>0$.

###  A.4 Proof of Proposition 4.4

As the proof of this proposition is similar to that of Proposition 4.2, we
only sketch it. For any $z^{\prime},z\in\mathcal{Z}$, we can see that

$\begin{array}[]{llll}&\left|\psi(z^{\prime})-\psi(z)\right|=\left|h(y^{\prime}\cdot\phi(x^{\prime}))-h(y\cdot\phi(x))\right|\\\
&\leq L_{h}\left|\phi(y^{\prime}\cdot x^{\prime}-y\cdot x)\right|\\\ &\leq
L_{h}L_{\phi}\left\llbracket y^{\prime}\cdot x^{\prime}-y\cdot
x\right\rrbracket\\\ &\leq L_{h}L_{\phi}\left(\left\llbracket
x^{\prime}-x\right\rrbracket+{\boldsymbol{\delta}}_{\\{0\\}}(y^{\prime}-y)\right)=L_{h}L_{\phi}d(z^{\prime},z).\end{array}$

Hence, $\psi$ is $(L_{h}L_{\phi},d)$-Lipschitz at $\mathcal{Z}$.

Next, suppose that $L_{\phi}>0$. Then we can see that $\psi$ satisfies
Assumption (A1) at $\mathcal{Z}$ with $\delta$. For any
$\hat{z}=(\hat{x},\hat{y})\in\mathcal{Z}$, denote
$t_{0}\coloneqq\hat{y}\cdot\phi\left(\hat{x}\right)$, then we have
$\psi(\hat{z})=h(t_{0})$. Let $0<\epsilon<L_{h}L_{\phi}$. As in the proof of
Proposition 4.1, there exists $\tilde{v}\in\mathcal{V}$ such that
$\left\llbracket\tilde{v}\right\rrbracket=1$ and
$0<L_{\phi}-\frac{\epsilon}{2L_{h}}<\phi(\tilde{v})\leq L_{\phi}$. By
Assumption (H2) and the Lipschitz property of $h$, there exists
$\tilde{t}\in\mathbb{R}$ such that $|\tilde{t}|\geq L_{\phi}\delta$ and

$h(\tilde{t}+t_{0})-h(t_{0})\geq\left(L_{h}-\frac{\epsilon}{2\phi(\tilde{v})}\right)|\tilde{t}|.$

Define
$\tilde{z}\coloneqq\left(\hat{x}+\tilde{t}\hat{y}\tilde{v}/\phi(\tilde{v}),\hat{y}\right)$,
then we have

$d\left(\tilde{z},\hat{z}\right)=\left\llbracket\frac{\tilde{t}\hat{y}}{\phi(\tilde{v})}\tilde{v}\right\rrbracket=\frac{|\tilde{t}|}{\phi(\tilde{v})}\geq\frac{|\tilde{t}|}{L_{\phi}}\geq\delta,$

and

$\begin{array}[]{lllll}&\psi(\tilde{z})-\psi(\hat{z})=h\left(\hat{y}\cdot\phi\left(\hat{x}+\frac{\tilde{t}\hat{y}}{\phi(\tilde{v})}\tilde{v}\right)\right)-h(\hat{y}\cdot\phi(\hat{x}))\\\
&=h(t_{0}+\tilde{t})-h(t_{0})\\\
&\geq\left(L_{h}-\frac{\epsilon}{2\phi(\tilde{v})}\right)|\tilde{t}|=\left(L_{h}\phi(\tilde{v})-\frac{\epsilon}{2}\right)\frac{|\tilde{t}|}{\phi(\tilde{v})}\\\
&\geq\left(L_{h}\left(L_{\phi}-\frac{\epsilon}{2L_{h}}\right)-\frac{\epsilon}{2}\right)\frac{|\tilde{t}|}{\phi(\tilde{v})}\\\
&=(L_{\phi}L_{h}-\epsilon)d(\tilde{z},\hat{z}).\end{array}$

Therefore, $\psi$ satisfies Assumption (A2) at $\mathcal{Z}$ with the given
$\delta$. This completes the proof.

##  Appendix B Proof of examples

We start with the following technical lemma, which will be used later in the
proof of Examples 3.1 and 3.2.

######  Lemma B.1.

Let $T\in(0,\infty]$ and $h\colon(0,T)\rightarrow\mathbb{R}$ be a convex,
continuously differentiable function. Given any $\hat{t}\in(0,T)$, let
$\mathcal{H}_{h,\hat{t}}\colon(0,T)\rightarrow\mathbb{R}$ defined as

$\mathcal{H}_{h,\hat{t}}(t)=\begin{cases}\frac{h(t)-h(\hat{t})}{t-\hat{t}}&\text{if
}t\neq\hat{t},\\\ \nabla h(\hat{t})&\text{otherwise.}\end{cases}$

Then we have that $\mathcal{H}_{h,\hat{t}}$ is continuous and non-decreasing
on $(0,T)$. Moreover,

$\sup_{t\in(0,T)}\left|\mathcal{H}_{h,\hat{t}}(t)\right|=\max\left\\{\left|\mathcal{H}^{+}_{h,\hat{t}}(0)\right|,\left|\mathcal{H}^{-}_{h,\hat{t}}(T)\right|\right\\}.$

where $\mathcal{H}^{+}_{h,\hat{t}}(0)\coloneqq\lim_{t\downarrow
0}\mathcal{H}_{h,\hat{t}}(t)$ and
$\mathcal{H}^{-}_{h,\hat{t}}(T)\coloneqq\lim_{t\uparrow
T}\mathcal{H}_{h,\hat{t}}(t)$.

Proof. Since $h$ is continuously differentiable, we can see that
$\mathcal{H}_{h,\hat{t}}$ is continuous on $(0,T)$. For any $t\in(0,T)$ such
that $t\neq\hat{t}$, we can see

$\nabla\mathcal{H}_{h,\hat{t}}(t)=\frac{(t-\hat{t})\nabla
h(t)-h(t)+h(\hat{t})}{(t-\hat{t})^{2}}.$

Note that $h$ is convex, hence $h(\hat{t})\geq h(t)+(\hat{t}-t)\nabla h(t)$
for any $t\in(0,T)$, and thus $\nabla\mathcal{H}_{h,\hat{t}}(t)\geq 0$ for any
$t\in(0,T)\setminus\\{\hat{t}\\}$. Therefore, $\mathcal{H}_{h,\hat{t}}$ is
non-decreasing on $(0,T)$, and the remaining conclusion follows. $\square$

###  B.1 Proof of Example 3.1

(a) Note that $\nabla h(t)=\log(t)-\log(1-t)$ for any $t\in(0,1)$, we can
easily see that $h$ is convex and continuously differentiable. Moreover, we
can see that

$\sup_{t^{\prime},t\in(0,1),t^{\prime}\neq
t}\frac{\left|h(t^{\prime})-h(t)\right|}{\left|t^{\prime}-t\right|}\geq\lim_{t\rightarrow
0}\left|\nabla h(t)\right|=\infty.$

Thus $h$ is not globally Lipschitz on $(0,1)$.

(b) For any $z^{\prime}\in(0,1)$ such that $z^{\prime}\neq\hat{z}$, we have
$\beta z^{\prime},\beta\hat{z}\in(0,1)$ and

$\begin{array}[]{lll}&\left|\psi_{\beta}(z^{\prime})-\psi_{\beta}(\hat{z})\right|=\left|h(\beta
z^{\prime})-h(\beta\hat{z})\right|\\\
&=\beta|z^{\prime}-\hat{z}|\left|\frac{h(\beta
z^{\prime})-h(\beta\hat{z})}{\beta z^{\prime}-\beta\hat{z}}\right|\\\
&\leq\beta|z^{\prime}-\hat{z}|\sup_{t\in(0,1)}\left|\mathcal{H}_{h,\beta\hat{z}}(t)\right|,\end{array}$

where $\mathcal{H}_{h,\beta\hat{z}}$ is defined as in Lemma B.1. According to
Lemma B.1, we have that $\mathcal{H}_{h,\beta\hat{t}}$ is continuous and non-
decreasing on $(0,1)$. Moreover, we can see that

$\begin{array}[]{llll}&\mathcal{H}^{+}_{h,\beta\hat{z}}(0)=\frac{h(\beta\hat{z})-\lim_{t\downarrow
0}h(t)}{\beta\hat{z}}=\frac{h(\beta\hat{z})}{\beta\hat{z}}\\\
&\qquad=\frac{\beta\hat{z}\log(\beta\hat{z})+(1-\beta\hat{z})\log(1-\beta\hat{z})}{\beta\hat{z}}<0,\\\
&\mathcal{H}^{-}_{h,\beta\hat{z}}(1)=\frac{\lim_{t\uparrow
1}h(t)-h(\beta\hat{z})}{1-\beta\hat{z}}=\frac{-h(\beta\hat{z})}{1-\beta\hat{z}}\\\
&\qquad=\frac{-\beta\hat{z}\log(\beta\hat{z})-(1-\beta\hat{z})\log(1-\beta\hat{z})}{1-\beta\hat{z}}>0.\end{array}$

Together with Lemma B.1, we have that

$\begin{array}[]{ll}\beta\sup_{t\in(0,1)}\left|\mathcal{H}_{h,\beta\hat{z}}(t)\right|&=\beta\max\left\\{-\mathcal{H}^{+}_{h,\beta\hat{t}}(0),\mathcal{H}^{-}_{h,\beta\hat{t}}(1)\right\\}\\\
&=-\beta\mathcal{H}^{+}_{h,\beta\hat{z}}(0)=L_{\beta}^{\\{\hat{z}\\}},\end{array}$

where the second equality follows from the fact that
$\hat{z}\in(0,\frac{1}{2}]$. Thus, $\psi_{\beta}$ is
$(L_{\beta}^{\\{\hat{z}\\}},d)$-Lipschitz at $\\{\hat{z}\\}$.

(b1) Suppose $0<\delta<\hat{z}$. For any $\epsilon>0$, since
$\mathcal{H}_{h,\beta\hat{z}}$ is continuous, non-decreasing on $(0,1)$, and
also satisfies $\mathcal{H}^{+}_{h,\beta\hat{z}}(0)<0$, there exists
$\tilde{z}\in(0,\hat{z}-\delta)$ such that

$\mathcal{H}_{h,\beta\hat{z}}(\beta\tilde{z})<0\text{ and
}0\leq\mathcal{H}_{h,\beta\hat{z}}(\beta\tilde{z})-\mathcal{H}^{+}_{h,\beta\hat{z}}(0)<\frac{\epsilon}{\beta}.$

Then
$d(\tilde{z},\hat{z})=\left|\tilde{z}-\hat{z}\right|=\hat{z}-\tilde{z}>\delta$
and

$\begin{array}[]{lll}&\psi_{\beta}(\tilde{z})-\psi_{\beta}(\hat{z})=-\beta(\hat{z}-\tilde{z})\frac{h(\beta\tilde{z})-h(\beta\hat{z})}{\beta\tilde{z}-\beta\hat{z}}\\\
&=-\beta d(\tilde{z},\hat{z})\mathcal{H}_{h,\beta\hat{z}}(\beta\tilde{z})\\\
&\geq\beta
d(\tilde{z},\hat{z})\left(-\mathcal{H}^{+}_{h,\beta\hat{z}}(0)-\frac{\epsilon}{\beta}\right)=(L_{\beta}^{\\{\hat{z}\\}}-\epsilon)d(\tilde{z},\hat{z}).\end{array}$

Hence, Assumption (A2) is satisfied. By Theorem 3.2, we have that

$\sup_{\mathbb{P}\colon\mathcal{W}_{d,1}(\mathbb{P},\mathbb{P}_{N})\leq\delta}\mathrm{E}_{\mathbb{P}}[\ell(Z;\beta)]=\ell(\hat{z};\beta)+L_{\beta}^{\\{\hat{z}\\}}\delta.$

(b2) Suppose $\delta\geq\hat{z}$. Note that $h$ is upper bounded by 0. Thus,
we have that

$\sup_{\mathbb{P}\colon\mathcal{W}_{d,1}(\mathbb{P},\mathbb{P}_{N})\leq\delta}\mathrm{E}_{\mathbb{P}}[\ell(Z;\beta)]\leq
0.$

On the other hand, if we choose
$\tilde{\mathbb{P}}_{k}\coloneqq{\boldsymbol{\chi}}_{\\{\tilde{z}_{k}\\}}$
with $\tilde{z}_{k}=\frac{1}{k}$ for each $k>1$, then by Lemma 3.1, we have
$\mathcal{W}_{d,1}(\tilde{\mathbb{P}}_{k},\mathbb{P}_{N})=\mathcal{W}_{d,1}({\boldsymbol{\chi}}_{\\{\tilde{z}_{k}\\}},{\boldsymbol{\chi}}_{\\{\hat{z}\\}})=d(\tilde{z}_{k},\hat{z})=\hat{z}-\frac{1}{k}<\delta$
for sufficiently large $k$. Moreover, it holds that

$\mathrm{E}_{\tilde{\mathbb{P}}_{k}}[\ell(Z;\beta)]=\ell(\tilde{z}_{k};\beta)=h(\beta/k)\rightarrow
0$

as $k\rightarrow\infty$. Therefore, we have

$\sup_{\mathbb{P}\colon\mathcal{W}_{d,1}(\mathbb{P},\mathbb{P}_{N})\leq\delta}\mathrm{E}_{\mathbb{P}}[\ell(Z;\beta)]=0.$

This completes the proof.

###  B.2 Proof of Example 3.2

(a) For any $z\in\mathbb{R}$, we have $d(z,\hat{z})=\|z\|_{{\mathbb{R}^{n}}}$
and

$\begin{array}[]{lll}&|\psi_{\beta}(z)-\psi_{\beta}(\hat{z})|=\left|h(\langle\beta,z\rangle)-\frac{1}{2}\right|\\\
&=\begin{cases}\frac{1}{2}&\text{if }\left|\langle\beta,z\rangle\right|\geq
1,\\\
\frac{1}{2}\left|\langle\beta,z\rangle\right|&\text{otherwise}\end{cases}\\\
&\leq\frac{1}{2}\left|\langle\beta,z\rangle\right|\leq\frac{\|\beta\|_{\mathbb{R}^{n},*}}{2}\|z\|_{{\mathbb{R}^{n}}}.\end{array}$

Therefore, we can see that $\psi_{\beta}$ is
$\left(\frac{\|\beta\|_{\mathbb{R}^{n},*}}{2},d\right)$-Lipschitz at
$\\{\hat{z}\\}$.

(a1) Suppose $0<\delta\leq\frac{1}{\vartheta_{2}}$. For any
$0<\epsilon<\frac{\|\beta\|_{\mathbb{R}^{n},*}}{2}$, let
$\tilde{z}\coloneqq\frac{1}{\|\beta\|_{{\mathbb{R}^{n},*}}}\alpha_{\beta}$,
then we have that
$d(\tilde{z},\hat{z})=\|\tilde{z}-\hat{z}\|_{{\mathbb{R}^{n}}}=\frac{1}{\|\beta\|_{{\mathbb{R}^{n},*}}}\geq\frac{1}{\vartheta_{2}}\geq\delta$
and

$\begin{array}[]{ll}&\psi_{\beta}(\tilde{z})-\psi_{\beta}(\hat{z})=h(1)-h\left(0\right)=\frac{1}{2}\\\
&>\left(\frac{\|\beta\|_{\mathbb{R}^{n},*}}{2}-\epsilon\right)\frac{1}{\|\beta\|_{\mathbb{R}^{n},*}}=\left(\frac{\|\beta\|_{\mathbb{R}^{n},*}}{2}-\epsilon\right)d(\tilde{z},\hat{z}),\end{array}$

which means that $\psi_{\beta}$ satisfies Assumption (A2) at $\\{\hat{z}\\}$
for $0<\delta\leq\frac{1}{\vartheta_{2}}$. Therefore, according to Theorem
3.2, we have

$\begin{array}[]{ll}&\sup_{\mathbb{P}\colon\mathcal{W}_{d,1}(\mathbb{P},\mathbb{P}_{N})\leq\delta}\mathrm{E}_{\mathbb{P}}[\ell(Z;\beta)]=\mathrm{E}_{\mathbb{P}_{N}}[\ell(Z;\beta)]+\frac{\|\beta\|_{\mathbb{R}^{n},*}}{2}\delta\\\
&=\ell(\hat{z};\beta)+\frac{\|\beta\|_{\mathbb{R}^{n},*}}{2}\delta,\quad
0<\delta\leq\frac{1}{\vartheta_{2}}.\end{array}$

(a2) Suppose $\delta\geq\frac{1}{\vartheta_{1}}$. We have that for any
$\mathbb{P}\in\mathcal{P}(\mathcal{Z})$,

$\mathrm{E}_{\mathbb{P}}[\ell(Z;\beta)]=\int_{\mathbb{R}^{n}}h(\langle\beta,z\rangle)\mathrm{d}\mathbb{P}(z)\leq
1\int_{\mathbb{R}^{n}}\mathrm{d}\mathbb{P}(z)=1.$

Hence,
$\sup_{\mathbb{P}\colon\mathcal{W}_{d,1}(\mathbb{P},\mathbb{P}_{N})\leq\delta}\mathrm{E}_{\mathbb{P}}[\ell(Z;\beta)]\leq
1$. On the other hand, if we choose
$\tilde{\mathbb{P}}\coloneqq{\boldsymbol{\chi}}_{\\{\tilde{z}\\}}$ with
$\tilde{z}=\frac{1}{\|\beta\|_{\mathbb{R}^{n},*}}\alpha_{\beta}$, then by
Lemma 3.1, we have
$\mathcal{W}_{d,1}(\tilde{\mathbb{P}},\mathbb{P}_{N})=\mathcal{W}_{d,1}({\boldsymbol{\chi}}_{\\{\tilde{z}\\}},{\boldsymbol{\chi}}_{\\{\hat{z}\\}})=d(\tilde{z},\hat{z})=\frac{1}{\|\beta\|_{\mathbb{R}^{n},*}}\leq\frac{1}{\vartheta_{1}}\leq\delta$.
Moreover, it holds that

$\mathrm{E}_{\tilde{\mathbb{P}}}[\ell(Z;\beta)]=\ell(\tilde{z};\beta)=h(1)=1,$

which means

$\sup_{\mathbb{P}\colon\mathcal{W}_{d,1}(\mathbb{P},\mathbb{P}_{N})\leq\delta}\mathrm{E}_{\mathbb{P}}[\ell(Z;\beta)]=1=\ell(\hat{z};\beta)+\frac{1}{2}.$

(b) For any $z\in\mathbb{R}^{n}$, we can see that
$d(z,\bar{z})=\left\|z+\frac{3}{\vartheta}\alpha_{\beta}\right\|_{\mathbb{R}^{n}}$
and

$\begin{array}[]{ll}&|\psi_{\beta}(z)-\psi_{\beta}(\bar{z})|=\left|h(\langle\beta,z\rangle)-h(-3)\right|=\left|h(\langle\beta,z\rangle)\right|\\\
&=\begin{cases}0&\text{if }\langle\beta,z\rangle\leq-1,\\\
\frac{\langle\beta,z\rangle+1}{2}&\text{if }-1<\langle\beta,z\rangle<1,\\\
1&\text{otherwise},\end{cases}\end{array}$

which further implies that

$\begin{array}[]{lll}&\left|\psi_{\beta}(z)-\psi_{\beta}(\bar{z})\right|\leq\frac{1}{4}\left|\langle\beta,z\rangle+3\right|\\\
&=\frac{1}{4}\left|\langle\beta,z\rangle+\langle\frac{3}{\vartheta}\alpha_{\beta},\beta\rangle\right|\\\
&\leq\frac{\|\beta\|_{\mathbb{R}^{n},*}}{4}\left\|z+\frac{3}{\vartheta}\alpha_{\beta}\right\|_{\mathbb{R}^{n}}=\frac{\vartheta}{4}\left\|z+\frac{3}{\vartheta}\alpha_{\beta}\right\|_{\mathbb{R}^{n}}=\frac{\vartheta}{4}d(z,\bar{z}).\end{array}$

Therefore, we can see that $\psi_{\beta}$ is
$(\frac{\vartheta}{4},d)$-Lipschitz at $\\{\bar{z}\\}$.

(b1) Suppose $0<\delta\leq\frac{4}{\vartheta}$. For any
$0<\epsilon<\frac{\vartheta}{4}$, let
$\tilde{z}\coloneqq\frac{1}{\vartheta}\alpha_{\beta}$, then
$d(\tilde{z},\bar{z})=\|\tilde{z}-\bar{z}\|_{\mathbb{R}^{n}}=\frac{4}{\vartheta}\geq\delta$
and

$\begin{array}[]{ll}\psi_{\beta}(\tilde{z})-\psi_{\beta}(\bar{z})&=h(1)-h\left(-3\right)=1\\\
&>\left(\frac{\vartheta}{4}-\epsilon\right)\frac{4}{\vartheta}=\left(\frac{\vartheta}{4}-\epsilon\right)d(\tilde{z},\bar{z}),\end{array}$

which means that $\psi_{\beta}$ satisfies Assumption (A2) at $\\{\bar{z}\\}$
for $0<\delta\leq\frac{4}{\vartheta}$. Therefore, according to Theorem 3.2, we
have

$\begin{array}[]{ll}&\sup_{\mathbb{P}\colon\mathcal{W}_{d,1}(\mathbb{P},\mathbb{P}_{N})\leq\delta}\mathrm{E}_{\mathbb{P}}[\ell(Z;\beta)]=\mathrm{E}_{\mathbb{P}_{N}}[\ell(Z;\beta)]+\frac{\vartheta}{4}\delta\\\
&=\ell(\bar{z};\beta)+\frac{\vartheta}{4}\delta,\quad
0<\delta\leq\frac{4}{\vartheta}.\end{array}$

(b2) Suppose $\delta\geq\frac{4}{\vartheta}$. Similar to (b1), we have
$\sup_{\mathbb{P}\colon\mathcal{W}_{d,1}(\mathbb{P},\mathbb{P}_{N})\leq\delta}\mathrm{E}_{\mathbb{P}}[\ell(Z;\beta)]\leq
1$. On the other hand, if we choose
$\tilde{\mathbb{P}}\coloneqq{\boldsymbol{\chi}}_{\\{\tilde{z}\\}}$ with
$\tilde{z}=\frac{1}{\vartheta}\alpha_{\beta}$, then by Lemma 3.1, we have
$\mathcal{W}_{d,1}(\tilde{\mathbb{P}},\mathbb{P}_{N})=d(\tilde{z},\bar{z})=\frac{4}{\vartheta}\leq\delta$.
Moreover, it holds that
$\mathrm{E}_{\tilde{\mathbb{P}}}[\ell(Z;\beta)]=\ell(\tilde{z};\beta)=h(1)=1$.
Therefore,

$\sup_{\mathbb{P}\colon\mathcal{W}_{d,1}(\mathbb{P},\mathbb{P}_{N})\leq\delta}\mathrm{E}_{\mathbb{P}}[\ell(Z;\beta)]=1=\ell(\bar{z};\beta)+1.$

This completes the proof.

###  B.3 Proof of Example 4.1

It is obvious that the function $\phi\colon z=(x,y)\mapsto
y-\langle\beta,x\rangle$ is linear on $\mathcal{Z}$. We are going to apply
Proposition 4.1 and Corollary 4.1 to draw the conclusions. Here we will check
the conditions in Proposition 4.1 and Corollary 4.1 case by case.

(i) Let $\left\llbracket(x,y)\right\rrbracket=\|[x;y]\|_{\mathbb{R}^{n+1}}$,
then it can be seen that $\left\llbracket\cdot\right\rrbracket$ is absolutely
homogeneous on $\mathcal{Z}$. Then
$d((x^{\prime},y^{\prime}),(x,y))=\left\llbracket(x^{\prime}-x,y^{\prime}-y)\right\rrbracket$
and
$\left\llbracket\cdot\right\rrbracket^{-1}(0)=\\{{\boldsymbol{0}}_{\mathbb{R}^{n+1}}\\}\subseteq\phi^{-1}(0)$.
In addition,

$\begin{array}[]{ll}L_{\phi}&=\sup_{x\in\mathbb{R}^{n},y\in\mathbb{R}}\left\\{|y-\langle\beta,x\rangle|\mid\|[x;y]\|_{\mathbb{R}^{n+1}}=1\right\\}\\\
&=\|[-\beta;1]\|_{\mathbb{R}^{n+1},*}<\infty.\end{array}$

(ii) Let
$\left\llbracket(x,y)\right\rrbracket=\|x\|_{\mathbb{R}^{n}}+{\boldsymbol{\delta}}_{\\{0\\}}(y)$,
then $\left\llbracket\cdot\right\rrbracket$ is absolutely homogeneous,
$d((x^{\prime},y^{\prime}),(x,y))=\left\llbracket(x^{\prime}-x,y^{\prime}-y)\right\rrbracket$
and
$\left\llbracket\cdot\right\rrbracket^{-1}(0)=\\{{\boldsymbol{0}}_{\mathbb{R}^{n+1}}\\}\subseteq\phi^{-1}(0)$.
In addition,

$\begin{array}[]{ll}L_{\phi}&=\sup_{x\in\mathbb{R}^{n},y\in\mathbb{R}}\left\\{\left|y-\langle\beta,x\rangle\right|\mid\|x\|_{\mathbb{R}^{n}}=1,y=0\right\\}\\\
&=\|\beta\|_{\mathbb{R}^{n},*}<\infty.\end{array}$

(iii) Let
$\left\llbracket(x,y)\right\rrbracket=\|x_{\mathcal{I}}\|_{\mathbb{R}^{|\mathcal{I}|}}+{\boldsymbol{\delta}}_{\\{{\bf
0}_{\mathbb{R}^{|\mathcal{I}^{c}|+1}}\\}}([x_{\mathcal{I}^{c}};y])$, then we
have that $\left\llbracket\cdot\right\rrbracket$ is absolutely homogeneous,
$d((x^{\prime},y^{\prime}),(x,y))=\left\llbracket(x^{\prime}-x,y^{\prime}-y)\right\rrbracket$
and
$\left\llbracket\cdot\right\rrbracket^{-1}(0)=\\{{\boldsymbol{0}}_{\mathbb{R}^{n+1}}\\}\subseteq\phi^{-1}(0)$.
In addition,

$\begin{array}[]{ll}L_{\phi}&=\sup\limits_{\begin{subarray}{c}x\in\mathbb{R}^{n}\\\
y\in\mathbb{R}\end{subarray}}\left\\{\left|y-\langle\beta,x\rangle\right|\mid\left\|x_{\mathcal{I}}\right\|_{\mathbb{R}^{|\mathcal{I}|}}=1,x_{\mathcal{I}^{c}}=0,y=0\right\\}\\\
&=\left\|\beta_{\mathcal{I}}\right\|_{\mathbb{R}^{|\mathcal{I}|},*}<\infty.\end{array}$

(iv) Let
$\left\llbracket(x,y)\right\rrbracket=\inf_{\bar{x}\in\mathbb{R}^{s}}\left\\{\|\bar{x}\|_{\mathbb{R}^{s}}\mid
B^{T}\bar{x}=x\right\\}+{\boldsymbol{\delta}}_{\\{0\\}}(y)$, then we have that
$d((x^{\prime},y^{\prime}),(x,y))=\left\llbracket(x^{\prime}-x,y^{\prime}-y)\right\rrbracket$.
It follows from Chu et al. (2022, Proposition 2(a)) that
$\left\llbracket\cdot\right\rrbracket$ is a norm on
$\mathtt{Range}(B^{T})\times\\{0\\}$ and infinite otherwise, hence it is
absolutely homogeneous. Moreover, we can see that
$\left\llbracket\cdot\right\rrbracket^{-1}(0)=\\{{\boldsymbol{0}}_{\mathbb{R}^{n+1}}\\}\subseteq\phi^{-1}(0)$.
In addition, it follows from Chu et al. (2022, Proposition 2(c)) that

$\begin{array}[]{llll}&L_{\phi}\\\
&=\sup\limits_{\begin{subarray}{c}x\in\mathbb{R}^{n}\\\
y\in\mathbb{R}\end{subarray}}\left\\{\left|y-\langle\beta,x\rangle\right|\middle|\inf\limits_{\bar{x}\in\mathbb{R}^{s}}\left\\{\|\bar{x}\|_{\mathbb{R}^{s}}\mid
B^{T}\bar{x}=x\right\\}+{\boldsymbol{\delta}}_{\\{0\\}}(y)=1\right\\}\\\
&=\sup\limits_{x\in\mathbb{R}^{n}}\left\\{\left|\langle\beta,x\rangle\right|\mid\inf\limits_{\bar{x}\in\mathbb{R}^{s}}\left\\{\|\bar{x}\|_{\mathbb{R}^{s}}\mid
B^{T}\bar{x}=x\right\\}=1\right\\}\\\
&=\|B\beta\|_{\mathbb{R}^{s},*}<\infty.\end{array}$

The desired conclusions then follow from Proposition 4.1 and Corollary 4.1.

###  B.4 Proof of Example 4.2

It is obvious that the function $\phi$ defined in (a) or (b) is linear on
$\mathcal{Z}$. In addition, let
$\left\llbracket(x,y)\right\rrbracket\coloneqq\left(\int_{0}^{1}|x(t)|^{2}\mathrm{d}t\right)^{1/2}+{\boldsymbol{\delta}}_{\\{0\\}}(y)$,
then we can see that $\left\llbracket\cdot\right\rrbracket$ is absolutely
homogeneous on $\mathcal{Z}$,
$d((x^{\prime},y^{\prime}),(x,y))=\left\llbracket(x^{\prime}-x,y^{\prime}-y)\right\rrbracket$
and

$\begin{array}[]{lll}&\left\llbracket\cdot\right\rrbracket^{-1}(0)\\\
&=\left\\{(x,y)\in\mathfrak{L}^{2}[0,1]\times\mathbb{R}:\left(\int\limits_{0}^{1}|x(t)|^{2}\mathrm{d}t\right)^{1/2}=0,y=0\right\\}\\\
&\subseteq\phi^{-1}(0).\end{array}$

The desired conclusions follows from Proposition 4.1 and Corollary 4.1 since
we have

$\begin{array}[]{lll}&\sup\limits_{\begin{subarray}{c}x\in\mathfrak{L}^{2}[0,1]\\\
y\in\mathbb{R}\end{subarray}}\left\\{\left|y-\int\limits_{0}^{1}x(t)\beta(t)\mathrm{d}t\right|\colon\left(\int\limits_{0}^{1}|x(t)|^{2}\mathrm{d}t\right)^{1/2}+{\boldsymbol{\delta}}_{\\{0\\}}(y)=1\right\\}\\\
&=\sup\limits_{x\in\mathfrak{L}^{2}[0,1]}\left\\{\left|\int\limits_{0}^{1}x(t)\beta(t)\mathrm{d}t\right|\
\colon\
\left(\int\limits_{0}^{1}|x(t)|^{2}\mathrm{d}t\right)^{1/2}=1\right\\}\\\
&=\left(\int\limits_{0}^{1}\left|\beta(t)\right|^{2}\mathrm{d}t\right)^{1/2},\end{array}$

and

$\begin{array}[]{ll}&\sup\limits_{\begin{subarray}{c}x\in\mathfrak{L}^{2}[0,1]\\\
y\in\mathbb{R}\end{subarray}}\left\\{\begin{matrix}\left|y-\int_{0}^{1}x(t)\sum_{j=1}^{n}\beta_{j}{\boldsymbol{g}}_{j}(t)\mathrm{d}t\right|:\\\
\left(\int_{0}^{1}|x(t)|^{2}\mathrm{d}t\right)^{1/2}+{\boldsymbol{\delta}}_{\\{0\\}}(y)=1\end{matrix}\right\\}\\\
&=\left(\int_{0}^{1}\left|\sum_{j=1}^{n}\beta_{j}{\boldsymbol{g}}_{j}(t)\right|^{2}\mathrm{d}t\right)^{1/2}.\end{array}$

This completes the proof.

###  B.5 Proof of Example 4.3

(a) We first see that

$h^{\prime}(t)=\tanh t=\frac{e^{2t}-1}{e^{2t}+1},$

which means that $h$ is globally $1$-Lipschitz on $\mathbb{R}$ and for any
$t_{0}\in\mathbb{R}$, we have

$\begin{array}[]{lll}&\lim_{k\rightarrow\infty}\frac{h(k+t_{0})-h(t_{0})}{k}\\\
&=\lim_{k\rightarrow\infty}\frac{\log(\cosh(k+t_{0}))-\log(\cosh(t_{0}))}{k}\\\
&=\lim_{k\rightarrow\infty}\tanh(k+t_{0})=1.\end{array}$

This is to say, $h$ defined in (a) satisfies Assumptions (H1-H2) with
$L_{h}=1$. Then the rest of the proof which involves finding $L_{\phi}$ is
similar to Example 4.1. We omit the details here. Then our conclusion follows
from Corollary 4.2.

(b) We have that $h^{\prime}(t)=\min\\{1,\max\\{-1,t\\}\\}$, which means that
$h$ is globally $1$-Lipschitz on $\mathbb{R}$. For any $t_{0}\in\mathbb{R}$,
we have

$\lim_{k\rightarrow\infty}\frac{h(k+t_{0})-h(t_{0})}{k}=\lim_{k\rightarrow\infty}\frac{k-h(t_{0})}{k}=1,$

which means that $h$ defined in (b) satisfies Assumptions (H1-H2) with
$L_{h}=1$. The rest of the proof is similar to that of (a).

(c) It is easy to see that $h$ defined in (c) is globally $1$-Lipschitz on
$\mathbb{R}$. Moreover, for any $t_{0}\in\mathbb{R}$, we can see that

$\lim_{k\rightarrow\infty}\frac{h(-k+t_{0})-h(t_{0})}{k}=\lim_{k\rightarrow\infty}\frac{k-t_{0}-h(t_{0})}{k}=1,$

which means that $h$ defined in (c) satisfies Assumptions (H1-H2) with
$L_{h}=1$. The rest of the proof is similar to that of (a).

###  B.6 Proof of Example 4.4

We first show that $\psi_{\beta}(z):=\langle[\beta;1],z\rangle^{2}$ for any
$z\in\mathbb{R}^{n+1}$ satisfies Assumption (A1) with
$L_{\beta}\coloneqq\|\beta\|_{2}^{2}+1$. For any
$z^{\prime},z\in\mathbb{R}^{n+1}$, we have

$\begin{array}[]{llll}&\left|\psi_{\beta}(z^{\prime})-\psi_{\beta}(z)\right|=\left|\langle[\beta;1],z^{\prime}\rangle^{2}-\langle[\beta;1],z\rangle^{2}\right|\\\
&=\left|\langle[\beta;1],z^{\prime}-z\rangle\right|\left|\langle[\beta;1],z^{\prime}+z\rangle\right|\\\
&\leq\|[\beta;1]\|_{2}\|z^{\prime}-z\|_{2}\cdot\|[\beta;1]\|_{2}\|z^{\prime}+z\|_{2}\\\
&=\left(\|\beta\|_{2}^{2}+1\right)d(z^{\prime},z).\end{array}$

Hence, $\psi_{\beta}$ is $(L_{\beta},d)$-Lipschitz at $\mathbb{R}^{n+1}$.

Next, we show that $\psi_{\beta}$ satisfies Assumption (A2). For any
$z\in\mathbb{R}^{n+1}$ and $k>0$, let $\tilde{z}\coloneqq z+k\Delta$ with
$\Delta\coloneqq\frac{[\beta;1]}{\|[\beta;1]\|_{2}}$, then
$\|\tilde{z}-z\|_{2}=\|k\Delta\|_{2}=k,\|\tilde{z}+z\|_{2}=\|2z+k\Delta\|_{2}$
and

$\begin{array}[]{ll}d(\tilde{z},z)&=\|\tilde{z}-z\|_{2}\|\tilde{z}+z\|_{2}=k\|2z+k\Delta\|_{2}\\\
&\geq k|k-2\|z\|_{2}|\rightarrow\infty,\end{array}$

as $k\rightarrow\infty$. On the other hand, we have

$\frac{\left|\langle\Delta,\tilde{z}+z\rangle\right|}{\|\tilde{z}+z\|_{2}}\leq\|\Delta\|_{2}=1,$

and

$\begin{array}[]{ll}\frac{\langle\Delta,\tilde{z}+z\rangle}{\|\tilde{z}+z\|_{2}}&=\frac{\langle\Delta,2z+k\Delta\rangle}{\|2z+k\Delta\|_{2}}\\\
&=\frac{\sum_{i=1}^{n+1}\Delta_{i}\left(2z_{i}/k+\Delta_{i}\right)}{\sqrt{\sum_{i=1}^{n+1}\left(2z_{i}/k+\Delta_{i}\right)^{2}}}\rightarrow
1,\end{array}$

as $k\rightarrow\infty$. Thus, for the given $\delta$ and any
$0<\epsilon<L_{\beta}$, there exists a positive integer $k_{\epsilon}$ such
that for $\tilde{z}_{\epsilon}=z+k_{\epsilon}\Delta$, one has

$\begin{array}[]{ll}d(\tilde{z}_{\epsilon},z)&=k_{\epsilon}\left\|\tilde{z}_{\epsilon}+z\right\|_{2}\geq\delta,\\\
\frac{\langle\Delta,\tilde{z}_{\epsilon}+z\rangle}{\|\tilde{z}_{\epsilon}+z\|_{2}}&\geq
1-\frac{\epsilon}{\|[\beta;1]\|_{2}^{2}}.\end{array}$

This implies that

$\begin{array}[]{llll}&\psi_{\beta}(\tilde{z}_{\epsilon})-\psi_{\beta}(z)=\langle[\beta;1],\tilde{z}_{\epsilon}-z\rangle\cdot\langle[\beta;1],\tilde{z}_{\epsilon}+z\rangle\\\
&=\|[\beta;1]\|_{2}^{2}\langle\Delta,k_{\epsilon}\Delta\rangle\langle\Delta,\tilde{z}_{\epsilon}+z\rangle\\\
&\geq\|[\beta;1]\|_{2}^{2}k_{\epsilon}\left(1-\frac{\epsilon}{\|[\beta;1]\|_{2}^{2}}\right)\left\|\tilde{z}_{\epsilon}+z\right\|_{2}\\\
&=\left(\|[\beta;1]\|_{2}^{2}-\epsilon\right)k_{\epsilon}\left\|\tilde{z}_{\epsilon}+z\right\|_{2}=\left(L_{\beta}-\epsilon\right)d(\tilde{z}_{\epsilon},z).\end{array}$

Therefore, it satisfies Assumption (A2). By Theorem 3.2, we have the required
result.

###  B.7 Proof of Example 4.5

It is obvious that $\phi(\cdot):=\langle\beta,\cdot\rangle$ is linear on
$\mathbb{R}^{n}$. Let $\left\llbracket
x\right\rrbracket=\|x\|_{\mathbb{R}^{n}}$, then we can see that
$\left\llbracket\cdot\right\rrbracket$ is absolutely homogeneous,
$d((x^{\prime},y^{\prime}),(x,y))=\left\llbracket
x^{\prime}-x\right\rrbracket+{\boldsymbol{\delta}}_{\\{0\\}}(y^{\prime}-y)$
and
$\left\llbracket\cdot\right\rrbracket^{-1}(0)=\\{{\boldsymbol{0}}_{\mathbb{R}^{n}}\\}\subseteq\phi^{-1}(0)$.
In addition,

$\begin{array}[]{ll}&L_{\phi}=\sup_{x\in\mathbb{R}^{n},y\in\mathbb{R}}\left\\{\left|y-\langle\beta,x\rangle\right|\mid\|x\|_{\mathbb{R}^{n}}=1,y=0\right\\}\\\
&=\sup_{x\in\mathbb{R}^{n}}\left\\{|\langle\beta,x\rangle|\mid\|x\|_{\mathbb{R}^{n}}=1\right\\}=\|\beta\|_{\mathbb{R}^{n},*}<\infty.\end{array}$

According to Corollary 4.3, we have the conclusions.

###  B.8 Proof of Example 4.6

According to Corollary 4.4 and the proof to Example 4.1, it suffices to show
that each $h$ defined in this example satisfies Assumption (H1-H2) with
$L_{h}=1$. Next, we discuss them one by one.

(a) For $h(t)=\log(1+\exp(-t))$, it can be seen that

$h^{\prime}(t)=-\frac{\exp(-t)}{1+\exp(-t)}=-\frac{1}{1+\exp(t)},$

which indicates that $h$ is globally $1$-Lipschitz on $\mathbb{R}$. Moreover,
given any $t_{0}\in\mathbb{R}$, we have

$\begin{array}[]{lll}&\lim_{k\rightarrow\infty}\frac{h(-k+t_{0})-h(t_{0})}{k}\\\
&=\lim_{k\rightarrow\infty}\frac{\log(1+\exp(k-t_{0}))-h(t_{0})}{k}\\\
&=\lim_{k\rightarrow\infty}\frac{\exp(k-t_{0})}{1+\exp(k-t_{0})}=1.\end{array}$

(b) Given

$h(t)=\left\\{\begin{array}[]{lll}0&\text{if }t\geq 1\\\
\frac{1}{2}(1-t)^{2}&\text{if }0<t<1\\\
\frac{1}{2}-t&\text{otherwise.}\end{array}\right.$

It can be easily seen that $h$ is globally $1$-Lipschitz on $\mathbb{R}$. In
addition, for any $t_{0}\in\mathbb{R}$, we have

$\begin{array}[]{ll}&\lim_{k\rightarrow\infty}\frac{h(-k+t_{0})-h(t_{0})}{k}\\\
&=\lim_{k\rightarrow\infty}\frac{\frac{1}{2}+k-t_{0}-h(t_{0})}{k}=1.\end{array}$

(c) Let $h$ be defined as

$h(t)=\left\\{\begin{array}[]{lll}1-t&\text{if }t\geq 1\\\
-\tau_{1}(1-t)&\text{if }-\tau_{2}<t<0\\\
\tau_{1}\tau_{2}&\text{otherwise.}\end{array}\right.$

The piecewise linear function $h$ is obviously globally $1$-Lipschitz on
$\mathbb{R}$. For any $t_{0}\in\mathbb{R}$, we can see that

$\begin{array}[]{ll}&\lim_{k\rightarrow\infty}\frac{h(-k+t_{0})-h(t_{0})}{k}\\\
&=\lim_{k\rightarrow\infty}\frac{1+k-t_{0}-h(t_{0})}{k}=1.\end{array}$

This completes the proof.

###  B.9 Proof of Example 5.1

According to Example 4.1 , for any $t\in\mathbb{R}$, we know that

$\begin{array}[]{ll}&\sup_{\mathbb{P}\colon\mathcal{W}_{d,1}(\mathbb{P},\mathbb{P}_{N})\leq\delta}\mathrm{E}_{\mathbb{P}}[\left(\left|Y-\langle\beta,X\rangle\right|-t\right)_{+}]\\\
&=\mathrm{E}_{\mathbb{P}_{N}}[\left(\left|Y-\langle\beta,X\rangle\right|-t\right)_{+}]+\|[-\beta;1]\|_{\mathbb{R}^{n+1},*}\delta.\end{array}$

Define the function $G:\mathbb{R}^{n}\times\mathbb{R}\rightarrow\mathbb{R}$ as

$G(Z)=\left|Y-\langle\beta,X\rangle\right|,$

for any $Z=(X,Y)\in\mathbb{R}^{n}\times\mathbb{R}$. Then we can see that $G$
is $(\|[-\beta;1]\|_{\mathbb{R}^{n+1},*},d)$-Lipschitz at
$\mathbb{R}^{n}\times\mathbb{R}$. Therefore, the conclusion follows from
Corollary 5.1(a).

###  B.10 Proof of Example 5.2

From Corollary 4.3, we can see that for any $t\in\mathbb{R}$,

$\begin{array}[]{ll}&\sup_{\mathbb{P}\colon\mathcal{W}_{d,1}(\mathbb{P},\mathbb{P}_{N})\leq\delta}\mathrm{E}_{\mathbb{P}}[\left(-Y\cdot\langle\beta,X\rangle-t\right)_{+}]\\\
&=\mathrm{E}_{\mathbb{P}_{N}}[\left(-Y\cdot\langle\beta,X\rangle-t\right)_{+}]+\|\beta\|_{\mathbb{R}^{n},*}\delta.\end{array}$

Let $G:\mathbb{R}^{n}\times\mathbb{R}\rightarrow\mathbb{R}$ be defined as

$G(Z)=-Y\cdot\langle\beta,X\rangle,\quad\mbox{for any
}Z=(X,Y)\in\mathbb{R}^{n}\times\mathbb{R}.$

It can be seen that $G(\cdot)$ is $(\|\beta\|_{\mathbb{R}^{n},*},d)$-Lipschitz
at $\mathbb{R}^{n}\times\mathbb{R}$. The conclusion then follows from
Corollary 5.1(a).

###  B.11 Proof of Example 5.3

The conclusion follows directly from Corollary 4.1 and Corollary 5.1(b), as
the function $G:\mathbb{R}^{n}\rightarrow\mathbb{R}$ defined as

$G(Z)=\langle\beta,X\rangle,\quad\mbox{for any }Z\in\mathbb{R}^{n},$

is $(\|\beta\|_{\mathbb{R}^{n},*},d)$-Lipschitz at $\mathbb{R}^{n}$.

##  Appendix C A weaker version of Theorem 3.2 with relaxed assumptions

######  Theorem C.1.

Let $\mathcal{Z}_{N}\coloneqq\\{Z^{(1)},\dots,Z^{(N)}\\}\subset\mathcal{Z}$ be
a given dataset and
$\mathbb{P}_{N}\coloneqq\sum_{i=1}^{N}\mu_{i}{\boldsymbol{\chi}}_{\\{Z^{(i)}\\}}\in\mathcal{P}(\mathcal{Z})$
be the corresponding empirical distribution. In addition, let $d(\cdot,\cdot)$
be a cost function on $\mathcal{Z}\times\mathcal{Z}$ and $\delta\in(0,\infty)$
be a scalar. Suppose the loss function
$\ell:\mathcal{Z}\times\mathcal{B}\rightarrow\mathbb{R}$ takes the form as

$\ell\colon(z;\beta)\mapsto\psi_{\beta}(z),$

where the function $\psi_{\beta}\colon\mathcal{Z}\rightarrow\mathbb{R}$
satisfies the following assumptions:

  1. (C1)

$\psi_{\beta}$ is $(L_{\beta}^{\\{Z^{(i)}\\}},d)$-Lipschitz at $\\{Z^{(i)}\\}$
with $L_{\beta}^{\\{Z^{(i)}\\}}\in(0,\infty)$ for each $1\leq i\leq N$;

  2. (C2)

for any $\epsilon\in(0,\min_{i}L_{\beta}^{\\{Z^{(i)}\\}})$ and each
$Z^{(i)}\in\mathcal{Z}_{N}$, there exists
$\tilde{Z}^{(i)}_{\epsilon}\in\mathcal{Z}$ such that $\delta\leq
d(\tilde{Z}^{(i)}_{\epsilon},Z^{(i)})<\infty$ and

$\psi_{\beta}(\tilde{Z}^{(i)}_{\epsilon})-\psi_{\beta}(Z^{(i)})\geq(L_{\beta}^{\\{Z^{(i)}\\}}-\epsilon)d(\tilde{Z}^{(i)}_{\epsilon},Z^{(i)}).$

Then we have that
$\widehat{\mathcal{L}}\leq\mathcal{S}\leq\widehat{\mathcal{U}}$, where

$\begin{array}[]{ll}&\widehat{\mathcal{L}}=\mathrm{E}_{\mathbb{P}_{N}}[\ell(Z;\beta)]+\sum_{i=1}^{N}\mu_{i}L_{\beta}^{\\{Z^{(i)}\\}}\delta,\\\
&\widehat{\mathcal{U}}=\mathrm{E}_{\mathbb{P}_{N}}[\ell(Z;\beta)]+\max_{i=1,\dots,N}L_{\beta}^{\\{Z^{(i)}\\}}\delta,\end{array}$

which means that

$\begin{array}[]{lll}&\mathrm{E}_{\mathbb{P}_{N}}[\ell(Z;\beta)]+\sum_{i=1}^{N}\mu_{i}L_{\beta}^{\\{Z^{(i)}\\}}\delta\\\
&\leq\sup_{\mathbb{P}\colon\mathcal{W}_{d,1}(\mathbb{P},\mathbb{P}_{N})\leq\delta}\mathrm{E}_{\mathbb{P}}[\ell(Z;\beta)]\\\
&\leq\mathrm{E}_{\mathbb{P}_{N}}[\ell(Z;\beta)]+\max_{i=1,\dots,N}L_{\beta}^{\\{Z^{(i)}\\}}\delta.\end{array}$

Proof. Since $\psi_{\beta}$ is $(L_{\beta}^{\\{Z^{(i)}\\}},d)$-Lipschitz at
$\\{Z^{(i)}\\}$ for each $1\leq i\leq N$, it implies that $\psi_{\beta}$ is
$(L_{\beta}^{\mathcal{Z}_{N}},d)$-Lipschitz at $\mathcal{Z}_{N}$ with

$L_{\beta}^{\mathcal{Z}_{N}}=\max_{i=1,\dots,N}L_{\beta}^{\\{Z^{(i)}\\}}.$

By Theorem 3.1, by letting
$\mathcal{L}_{i}=\sup_{\mathbb{P}\in\mathcal{P}(\mathcal{Z})}\left\\{\mathrm{E}_{\mathbb{P}}[\ell(Z;\beta)]\middle|\mathcal{W}_{d,1}(\mathbb{P},{\boldsymbol{\chi}}_{\\{Z^{(i)}\\}})\leq\delta\right\\}$,
$i=1,\cdots,N$, we have that

$\begin{array}[]{ll}&\sum_{i=1}^{N}\mu_{i}\mathcal{L}_{i}\leq\mathcal{S}=\sup_{\mathbb{P}\colon\mathcal{W}_{d,1}(\mathbb{P},\mathbb{P}_{N})\leq\delta}\mathrm{E}_{\mathbb{P}}[\ell(Z;\beta)]\\\
&\leq\mathrm{E}_{\mathbb{P}_{N}}[\ell(Z;\beta)]+L_{\beta}^{\mathcal{Z}_{N}}\delta=\widehat{\mathcal{U}}.\end{array}$

Then by applying Theorem 3.2 for each $\mathcal{L}_{i}$, we can see that

$\mathcal{L}_{i}=\ell(Z^{(i)};\beta)+L_{\beta}^{\\{Z^{(i)}\\}}\delta,$

which means that
$\sum_{i=1}^{N}\mu_{i}\mathcal{L}_{i}=\sum_{i=1}^{N}\mu_{i}\ell(Z^{(i)};\beta)+\sum_{i=1}^{N}\mu_{i}L_{\beta}^{\\{Z^{(i)}\\}}\delta=\widehat{\mathcal{L}}$.
This completes the proof. $\square$

## References

  * An and Gao (2021) An Y, Gao R (2021) Generalization bounds for (Wasserstein) robust optimization. _Advances in Neural Information Processing Systems_ , volume 34, 10382–10392.
  * Bawa (1975) Bawa VS (1975) Optimal rules for ordering uncertain prospects. _Journal of Financial Economics_ 2(1):95–121.
  * Belloni et al. (2011) Belloni A, Chernozhukov V, Wang L (2011) Square-root lasso: Pivotal recovery of sparse signals via conic programming. _Biometrika_ 98(4):791–806.
  * Bertsimas et al. (2018) Bertsimas D, Gupta V, Kallus N (2018) Robust sample average approximation. _Mathematical Programming_ 171:217–282.
  * Bickel et al. (2009) Bickel PJ, Ritov Y, Tsybakov AB (2009) Simultaneous analysis of Lasso and Dantzig selector. _The Annals of Statistics_ 37(4):1705–1732.
  * Blanchet et al. (2019) Blanchet J, Kang Y, Murthy K (2019) Robust Wasserstein profile inference and applications to machine learning. _Journal of Applied Probability_ 56(3):830–857.
  * Blanchet and Murthy (2019) Blanchet J, Murthy K (2019) Quantifying distributional model risk via optimal transport. _Mathematics of Operations Research_ 44(2):565–600.
  * Bunea et al. (2013) Bunea F, Lederer J, She Y (2013) The group square-root lasso: Theoretical properties and fast algorithms. _IEEE Transactions on Information Theory_ 60(2):1313–1325.
  * Cai and Yuan (2012) Cai TT, Yuan M (2012) Minimax and adaptive prediction for functional linear regression. _Journal of the American Statistical Association_ 107(499):1201–1216.
  * Cardot et al. (1999) Cardot H, Ferraty F, Sarda P (1999) Functional linear model. _Statistics & Probability Letters_ 45(1):11–22.
  * Chen et al. (2011) Chen L, He S, Zhang S (2011) Tight bounds for some risk measures, with applications to robust portfolio selection. _Operations Research_ 59(4):847–865.
  * Chu et al. (2022) Chu HT, Toh KC, Zhang Y (2022) On regularized square-root regression problems: Distributionally robust interpretation and fast computations. _Journal of Machine Learning Research_ 23(1):13885–13923.
  * Cohn (2013) Cohn DL (2013) _Measure Theory_ , volume 5 (Springer).
  * Collobert (2004) Collobert R (2004) Large scale machine learning. Technical report, Université de Paris VI.
  * Delage and Ye (2010) Delage E, Ye Y (2010) Distributionally robust optimization under moment uncertainty with application to data-driven problems. _Operations Research_ 58(3):595–612.
  * Drucker et al. (1996) Drucker H, Burges CJ, Kaufman L, Smola A, Vapnik V (1996) Support vector regression machines. _Advances in Neural Information Processing Systems_ , volume 9, 155–161.
  * Erdoğan and Iyengar (2006) Erdoğan E, Iyengar G (2006) Ambiguous chance constrained problems and robust optimization. _Mathematical Programming_ 107:37–61.
  * Feng et al. (2014) Feng J, Xu H, Mannor S, Yan S (2014) Robust logistic regression and classification. _Advances in Neural Information Processing Systems_ , volume 27.
  * Fishburn (1977) Fishburn PC (1977) Mean-risk analysis with risk associated with below-target returns. _The American Economic Review_ 67(2):116–126.
  * Gao (2022) Gao R (2022) Finite-sample guarantees for Wasserstein distributionally robust optimization: Breaking the curse of dimensionality. _Operations Research_ .
  * Gao et al. (2022) Gao R, Chen X, Kleywegt AJ (2022) Wasserstein distributionally robust optimization and variation regularization. _Operations Research_ .
  * Gao and Kleywegt (2023) Gao R, Kleywegt A (2023) Distributionally robust stochastic optimization with Wasserstein distance. _Mathematics of Operations Research_ 48(2):603–655.
  * Gibbs and Su (2002) Gibbs AL, Su FE (2002) On choosing and bounding probability metrics. _International Statistical Review_ 70(3):419–435.
  * Goh and Sim (2010) Goh J, Sim M (2010) Distributionally robust optimization and its tractable approximations. _Operations Research_ 58(4-part-1):902–917.
  * Hoerl and Kennard (1970) Hoerl AE, Kennard RW (1970) Ridge regression: Biased estimation for nonorthogonal problems. _Technometrics_ 12(1):55–67.
  * Horel (1962) Horel A (1962) Application of ridge analysis to regression problems. _Chemical Engineering Progress_ 58:54–59.
  * Howard et al. (2019) Howard A, Sandler M, Chu G, Chen LC, Chen B, Tan M, Wang W, Zhu Y, Pang R, Vasudevan V, Le QV, Adam H (2019) Searching for mobilenetv3. _Proceedings of the IEEE/CVF International Conference on Computer Vision_ , 1314–1324.
  * Hu and Hong (2013) Hu Z, Hong LJ (2013) Kullback-Leibler divergence constrained distributionally robust optimization. _Available at Optimization Online_ 1(2):9.
  * Huber (1973) Huber PJ (1973) Robust regression: Asymptotics, conjectures and monte carlo. _The Annals of Statistics_ 799–821.
  * Hurtik et al. (2022) Hurtik P, Tomasiello S, Hula J, Hynar D (2022) Binary cross-entropy with dynamical clipping. _Neural Computing and Applications_ 34(14):12029–12041.
  * Jiang et al. (2021) Jiang H, Luo S, Dong Y (2021) Simultaneous feature selection and clustering based on square root optimization. _European Journal of Operational Research_ 289(1):214–231.
  * Jiang and Guan (2016) Jiang R, Guan Y (2016) Data-driven chance constrained stochastic program. _Mathematical Programming_ 158(1-2):291–327.
  * Koenker and Bassett Jr (1978) Koenker R, Bassett Jr G (1978) Regression quantiles. _Econometrica: Journal of the Econometric Society_ 33–50.
  * Koenker and Hallock (2001) Koenker R, Hallock KF (2001) Quantile regression. _Journal of Economic Perspectives_ 15(4):143–156.
  * Krokhmal (2007) Krokhmal PA (2007) Higher moment coherent risk measures. _Quantitative Finance_ 7(4):373–387.
  * Kuhn et al. (2019) Kuhn D, Esfahani P, Nguyen V, Shafieezadeh-Abadeh S (2019) Wasserstein distributionally robust optimization: Theory and applications in machine learning. Operations Research & Management Science In The Age Of Analytics. pp. 130-166
  * Lee et al. (2005) Lee YJ, Hsieh WF, Huang CM (2005) $\epsilon$-SSVR: A smooth support vector machine for $\epsilon$-insensitive regression. _IEEE Transactions on Knowledge and Data Engineering_ 17(05):678–685.
  * Li et al. (2022) Li J, Lin S, Blanchet J, Nguyen V (2022) Tikhonov Regularization is Optimal Transport Robust under Martingale Constraints. _Advances In Neural Information Processing Systems_. 35:17677-17689
  * Li et al. (2018a) Li X, Sun DF, Toh KC (2018a) On efficiently solving the subproblems of a level-set method for fused Lasso problems. _SIAM Journal on Optimization_ 28(2):1842–1866
  * Li et al. (2018b) Li X, Sun DF, Toh KC (2018b) A highly efficient semismooth Newton augmented Lagrangian method for solving Lasso problems. _SIAM Journal on Optimization_ 28(1): 433–458
  * Lounici et al. (2011) Lounici K, Pontil M, Van De Geer S, Tsybakov AB (2011) Oracle inequalities and optimal inference under group sparsity. _The Annals of Statistics_ 39(4):2164 – 2204.
  * Luo et al. (2019) Luo Z, Sun DF, Toh KC, Xiu N (2019) Solving the OSCAR and SLOPE models using a semismooth Newton-based augmented Lagrangian method. Journal of Machine Learning Research, 20(106):1–25
  * Mohajerin Esfahani and Kuhn (2018) Mohajerin Esfahani P, Kuhn D (2018) Data-driven distributionally robust optimization using the Wasserstein metric: Performance guarantees and tractable reformulations. _Mathematical Programming_ 171(1-2):115–166.
  * Montiel Olea et al. (2023) Montiel Olea JL, Rush C, Velez A, Wiesel J (2023) The out-of-sample prediction error of the square-root-LASSO and related estimators. _arXiv preprint arXiv:2211.07608_.
  * Peyré and Cuturi (2017) Peyré G, Cuturi M (2017) Computational optimal transport. _Center for Research in Economics and Statistics Working Papers_ (2017-86).
  * Plan and Vershynin (2012) Plan Y, Vershynin R (2012) Robust 1-bit compressed sensing and sparse logistic regression: A convex programming approach. _IEEE Transactions on Information Theory_ 59(1):482–494.
  * Ramsay and Dalzell (1991) Ramsay JO, Dalzell C (1991) Some tools for functional data analysis. _Journal of the Royal Statistical Society: Series B (Methodological)_ 53(3):539–561.
  * Ramsay and Silverman (2005) Ramsay JO, Silverman BW (2005) _Functional Data Analysis_. Springer Series in Statistics (New York: Springer), 2nd edition, ISBN 978-0-387-40080-8.
  * Schölkopf et al. (1998) Schölkopf B, Bartlett P, Smola A, Williamson R (1998) Support vector regression with automatic accuracy control. _ICANN 98: Proceedings of the 8th International Conference on Artificial Neural Networks_ , 111–116 (Springer).
  * Schölkopf et al. (2000) Schölkopf B, Smola AJ, Williamson RC, Bartlett PL (2000) New support vector algorithms. _Neural Computation_ 12(5):1207–1245.
  * Scott (2012) Scott C (2012) Calibrated asymmetric surrogate losses. _Electronic Journal of Statistics_ 6(none):958 – 992.
  * Shafieezadeh-Abadeh et al. (2023) Shafieezadeh-Abadeh S, Aolaritei L, Dörfler F, Kuhn D (2023) New perspectives on regularization and computation in optimal transport-based distributionally robust optimization. _ArXiv Preprint ArXiv:2303.03900_.
  * Shafieezadeh-Abadeh et al. (2019) Shafieezadeh-Abadeh S, Kuhn D, Esfahani PM (2019) Regularization via mass transportation. _Journal of Machine Learning Research_ 20(103):1–68.
  * Shafieezadeh-Abadeh et al. (2015) Shafieezadeh-Abadeh S, Mohajerin Esfahani PM, Kuhn D (2015) Distributionally robust logistic regression. _Advances in Neural Information Processing Systems_ , volume 28.
  * Shen et al. (2017) Shen X, Niu L, Qi Z, Tian Y (2017) Support vector machine classifier with truncated pinball loss. _Pattern Recognition_ 68:199–210.
  * Sion (1958) Sion M (1958) On general minimax theorems. _Pacific Journal of Mathematics_ 8(1):171–176.
  * Stucky and Van De Geer (2017) Stucky B, Van De Geer S (2017) Sharp oracle inequalities for square root regularization. _Journal of Machine Learning Research_ 18(67):1–29.
  * Tang et al. (2020) Tang P, Wang C, Sun DF, Toh KC (2020) A sparse semismooth Newton based proximal majorization-minimization algorithm for nonconvex square-root-loss regression problem. The Journal Of Machine Learning Research. 21, 9253-9290
  * Tong and Ng (2018) Tong H, Ng M (2018) Analysis of regularized least squares for functional linear regression model. _Journal of Complexity_ 49:85–94.
  * Vapnik and Chervonenkis (2015) Vapnik VN, Chervonenkis AY (2015) On the uniform convergence of relative frequencies of events to their probabilities. _Measures of Complexity: Festschrift for Alexey Chervonenkis_ , 11–30 (Springer).
  * Villani (2009) Villani C (2009) _Optimal Transport: Old and New_ , volume 338 (Springer).
  * Wang et al. (2016) Wang JL, Chiou JM, Müller HG (2016) Functional data analysis. _Annual Review of Statistics and its Application_ 3:257–295.
  * Wang et al. (2020) Wang Q, Ma Y, Zhao K, Tian Y (2020) A comprehensive survey of loss functions in machine learning. _Annals of Data Science_ 1–26.
  * Wiesemann et al. (2014) Wiesemann W, Kuhn D, Sim M (2014) Distributionally robust convex optimization. _Operations Research_ 62(6):1358–1376.
  * Wu et al. (2022) Wu Q, Li JYM, Mao T (2022) On generalization and regularization via Wasserstein distributionally robust optimization. _arXiv preprint arXiv:2212.05716_.
  * Yi-de et al. (2004) Yi-de M, Qing L, Zhi-Bai Q (2004) Automated image segmentation using improved pcnn model based on cross-entropy. _Proceedings of 2004 International Symposium on Intelligent Multimedia, Video and Speech Processing, 2004._ , 743–746 (IEEE).
  * Zhang et al. (2022) Zhang L, Yang J, Gao R (2022) A simple and general duality proof for Wasserstein distributionally robust optimization. _arXiv preprint arXiv:2205.00362_.
  * Zhang et al. (2020) Zhang Y, Zhang N, Sun DF, Toh KC (2020) An efficient Hessian based algorithm for solving large-scale sparse group Lasso problems. Mathematical Programming, 179(1): 223–263
  * Zhen et al. (2023) Zhen J, Kuhn D, Wiesemann W (2023) A unified theory of robust and distributionally robust optimization via the primal-worst-equals-dual-best principle. _Operations Research_.