Tight Concentration Inequality for Sub-Weibull RVs with GBO norms Tight
Concentration Inequality for Sub-Weibull Random Variables with Generalized
Bernstien Orlicz norms Heejong Bong 111Department of Statistics & Data
Science, Carnegie Mellon University, Pittsburgh, Pennsylvania, United States
of America. hbong<EMAIL_ADDRESS>and Arun Kumar Kuchibhotla
11footnotemark: 1 concentration inequality; sub-Weibull random variables;
generalized Bernstein Orlicz norm; heavy-tailed distribution; Paley-Zygmund
inequality 60G50; 60E15 60B20; 62E22 February 25, 2023 2302.03850 0 2020 0
10.1214/YY-TN Recent development in high-dimensional statistical inference has
necessitated concentration inequalities for a broader range of random
variables. We focus on sub-Weibull random variables, which extend sub-Gaussian
or sub-exponential random variables to allow heavy-tailed distributions. This
paper presents concentration inequalities for independent sub-Weibull random
variables with finite Generalized Bernstein-Orlicz norms, providing
generalized Bernstein’s inequalities and Rosenthal-type moment bounds. The
tightness of the proposed bounds is shown through lower bounds of the
concentration inequalities obtained via the Paley-Zygmund inequality. The
results are applied to a graphical model inference problem, improving previous
sample complexity bounds.

##  1 Introduction

Concentration inequalities provide essential tools for theoretical analyses of
statistical inference. Since Sergei Bernstein introduced the Cramér-Cernoff
technique in 1927, remarkable development has been made for concentration
inequalities. Some of the most popular establishments include Bernstein’s
inequality, Hoeffding’s inequality [11], and Bennet’s inequality [2]. Those
inequalities result from the Cramér-Cernoff technique applied to the moment-
generating functions (MGFs) of the random variables. Those inequalities result
in exponential tail bounds for sums of independent random variables, which
form a foundation of theoretical analyses in high-dimensional statistics. The
classical forms of the inequalities were established on bounded random
variables, but later, their assumptions were relaxed to the sub-Gaussian and
sub-exponential conditions (see Sections 2.6-2.8 in [18] and Section 2.1 in
[19] for thorough reviews.) More recent investigations verified the optimality
of the inequalities. For example, both Hoeffding’s and Bennett’s inequalities
cannot be improved for sums of Bernoulli random variables (Lemma 4.7 in [3];
Example 2.4 in [15]). Recently, Zhang and Zhou [21] showed the general
tightness of the MGF-based Cramér-Chernoff bounds.

The recent development in high-dimensional statistical inference has
necessitated concentration inequalities for a broader range of random
variables, including those which do not have finite MGF in any neighborhood of
$t=0$. For example, in the context of inference in graphical models (e.g.,
Chapter 2 of [5] and Section 3 of this paper), the leading error term is the
bias due to the uncertainty in covariance matrix estimation. The uncertainty
is quantified as the fourth power of the observed data. If the data are
Gaussian, the MGF of the fourth power is not finite at $t\neq 0$, so the MGF-
based concentration inequalities are not applicable. In this sense, we say the
bias term is heavy-tailed. To bind heavy-tailed random variables, we need
concentration inequalities for generalized tail probability characteristics.
Here we focus on sub-Weibull random variables, which have exponential tails
like sub-Gaussian or sub-Exponential random variables. A random variable $X$
is said to be sub-Weibull of order $\alpha>0$ or sub-Weibull$(\alpha)$ with
parameters $\nu$ and $L$ if

$\mathbb{P}[\lvert X\rvert\geq
t]\leq\begin{cases}2\exp\left(-t^{2}/\nu^{2}\right),&0\leq t<\nu
L^{-\alpha/(2-\alpha)},\\\
2\exp\left(-t^{\alpha}/(L\nu)^{\alpha}\right),&t\geq\nu
L^{-\alpha/(2-\alpha)}.\end{cases}$ (1)

A sub-Weibull random variable $X$ has a sub-Gaussian tail for small $t$, where
the order $\alpha$ determines the size of tail at large $t$. For $\alpha=2$
and $1$, sub-Weibull$(\alpha)$ generalizes sub-Gaussian and sub-exponential,
respectively [20]. If $\alpha$ is smaller than $1$, sub-Weibull$(\alpha)$
random variables are heavy-tailed. In particular, the fourth power bias term
in our example is sub-Weibull$(1/2)$.

An equivalent way of configuring the sub-Weibull tail behaviors is by Orlicz
norms. Given a non-decreasing function $g:[0,\infty)\rightarrow[0,\infty)$,
the $g$-Orlicz norm of a random variable $X$ is defined by

$\lVert X\rVert_{g}:=\inf\\{\eta>0:\mathbb{E}[g(\lvert X\rvert/\eta)]\leq
1\\}.$

The $g$-Orlicz norm representation is equivalent to the tail bound

$\mathbb{P}[\lvert X\rvert\geq t]\leq\frac{2}{g\left(t/\lVert
X\rVert_{g}\right)+1},~{}\forall t\geq 0.$

Because Eq. 1 implies $\mathbb{P}[\lvert X\rvert\geq t]\leq
2\exp(-\min\\{\frac{t^{2}}{\nu^{2}},\frac{t^{\alpha}}{(\lambda\nu)^{\alpha}}\\})$,
the tail bound leads to the Generalized Bernstein-Orlicz (GBO) norm of [12],
the Orlicz norm with respect to
$\phi_{\alpha,L}(x):=\exp(\min\\{x^{2},(x/L)^{\alpha}\\})-1$. We note that the
parameter $\nu$ corresponds to the size of $\lVert X\rVert_{\phi_{\alpha,L}}$.
We denote the collection of mean zero random variables with $\lVert
X\rVert_{\phi_{\alpha,L}}\leq\nu$ by $\Phi_{\alpha,L}(\nu)$.

In [12], Kuchibhotla and Chakrabortty provided a generalized version of
Bernstein’s inequality for sub-Weibull random variables and its application to
high-dimensional statistical theory when sub-Weibull$(\alpha)$ was defined
with $\psi_{\alpha}$-Orlicz norm, where $\psi_{\alpha}(x)=\exp(x^{\alpha})-1$.
We note that random variable $X$ with finite $\psi_{\alpha}$-Orlicz norm has
an unmixed tail bound, $\mathbb{P}[|X|\geq t]\leq 2\exp(-{t^{\alpha}}/{\lVert
X\rVert_{\psi_{\alpha}}^{\alpha}})$ so that $X$ is sub-Weibull$(\alpha)$
(i.e., satisfies Eq. 1) with parameters $\nu=\lVert X\rVert_{\psi_{\alpha}}/L$
and $L$ for any $L>0$. To exemplify the advantages offered by the granularity
of the GBO norm, consider the tail bound implied by Theorem 3.1 of [12]: If
$X_{1},\dots,X_{n}$ are independent mean zero random variables with $\lVert
X_{i}\rVert_{\psi_{\alpha}}<\infty$, then for any vector
$a=(a_{1},\dots,a_{n})\in\mathbb{R}^{n}$,

$\mathbb{P}\left[\left\lvert\sum_{i=1}^{n}a_{i}X_{i}\right\rvert\geq
C(\alpha)(\lVert b\rVert_{2}\sqrt{t}+\lVert
b\rVert_{\beta(\alpha)}t^{1/\alpha})\right]\leq 2e^{-t},$

where $b=(a_{1}\lVert X_{1}\rVert_{\psi_{\alpha}},\dots,a_{n}\lVert
X_{n}\rVert_{\psi_{\alpha}})$, $\beta(\alpha)=\infty$ when $\alpha\leq 1$,
$\beta(\alpha)=\alpha/(\alpha-1)$ when $\alpha>1$, and $C(\alpha)$ is a
constant dependent on $\alpha$. The tail bound is a mixture of sub-Gaussian
(when $t$ is small) and sub-Weibull$(\alpha)$ (when $t$ is large) tails and is
better represented in [12] by the GBO norm, rather than the $\psi_{\alpha}$
norm. If $\psi_{\alpha}$-Orlicz norm were used, the tightness of the tail at
large $t$ should be compromised by upper bounding the sub-Gaussian tail at
small $t$. This is due to a better granularity of the sub-Weibull space
represented by the GBO norm than by $\psi_{\alpha}$-Orlicz norm. To have a
finite $\psi_{\alpha}$-Orlicz norm, random variable $X$ should satisfy Eq. 1
with every value of $L>0$, with $\nu=\|X\|_{\psi_{\alpha}}/L$. In contrary,
the GBO norm has one more degree of freedom with parameter to further
granulate the space of sub-Weibull$(\alpha)$ random variables.

In this paper, we extend the results of [12] to sub-Weibull$(\alpha,L)$ random
variables with finite Generalized Bernstein-Orlicz (GBO) norms. The added
granularity of the specification for the summands $X_{1},\dots,X_{n}$ benefits
the tightness of theoretical analyses in many statistical applications; we
discuss an example in Section 3. We consider concentration inequalities in
moments, the GBO norms, and tail probability bounds for weighted sums of
independent $\text{sub-Weibull}(\alpha,L)$ random variables. To bound the
moments, we follow the sequence Orlicz norm argument devised by Latała in
[13], which we provide as preliminary theoretical results in Section 5.1. The
resulting moment bound follows the form of Rosenthal’s inequality (see Theorem
2.1). From the moment bounds, we induce the upper bounds for the GBO norms and
tail probabilities, using the arguments in [12]. The resulting tail
probability bounds provide generalized Bernstein’s inequalities for
$\text{sub-Weibull}(\alpha,L)$ random variables. We also show the tightness of
the proposed bounds by obtaining the lower bounds of the concentration
inequalities in the worst case. The lower bound proof follows the Paley-
Zygmund inequality technique in the corollary of [8] and Theorem 2.1 of [9].

Our theoretical contribution focuses on the case of $\alpha\leq 1$. Tail
probabilities of $\text{sub-Weibull}(\alpha,L)$ random variables with order
$\alpha>1$ are bounded by a mixture of logarithmically concave (log-concave)
tails. For this case, the tight moment and tail probability bounds are readily
available by [8], which we also include in this paper for completeness. We
refer to Theorem 1 of [10] for log-convex tail cases and Examples 3.2 and 3.3
in [13] for the sequence Orlicz norm technique establishing the tight moment
bounds in the two references.

##  2 Main Results

Let $X_{i}$ be independent mean zero sub-Weibull$(\alpha,L_{i})$ random
variables for $i=1,\dots,n$. Our objective is to bound the size of a weighted
sum $X^{*}:=\sum_{i=1}^{n}a_{i}X_{i}$ for
$a=(a_{1},\dots,a_{n})\in\mathbb{R}^{n}$. Because $\lVert
X_{i}/c\rVert_{\phi_{\alpha,L_{i}}}=\lVert
X_{i}\rVert_{\phi_{\alpha,L_{i}}}/c$ for $c>0$, by adjusting the value of
$a_{i}$, we assume $\lVert X_{i}\rVert_{\phi_{\alpha,L_{i}}}\leq 1$ and
$a_{i}\geq 0$. We denote the collection of such random variables by
$\Phi_{\alpha,L_{i}}(1)$. More generally, for $\eta>0$,

$\Phi_{\alpha,L}(\eta):=\\{X:\mathbb{E}[X]=0,\lVert
X\rVert_{\phi_{\alpha,L}}\leq\eta\\}.$

For $L=(L_{1},\dots,L_{n})$, we denote the collection $(X_{1},\dots,X_{n})$ of
random variables satisfying the above conditions by

$\mathcal{X}_{\alpha,L}=\\{(X_{1},\dots,X_{n}):\mathbb{E}[X_{i}]=0,~{}X_{i}\in\Phi_{\alpha,L_{i}}(1),~{}\text{and}~{}X_{1},\ldots,X_{n}\text{
are independent}\\}.$ (2)

Also, because the order of the random variables does not change the sum, we
will assume $(a_{i}\overline{L}_{i})_{i=1}^{n}$ is non-decreasing, where
$\overline{L}=(\max\\{1,L_{i}\\})_{i=1}^{n}$. That is,
$a_{1}\overline{L}_{1}\geq\dots\geq a_{n}\overline{L}_{n}\geq 0$.

Among the collection $\mathcal{X}_{\alpha,L}$, there are random variables that
take a special place; these are random variables that attain the worst case
upper bounds. Let $Z_{1},Z_{2},\ldots,Z_{n}$ be independent random variables
satisfying

$\mathbb{P}[\lvert Z_{i}\rvert\geq
t]=\exp(-\min\\{t^{2},(t/L_{i})^{\alpha}\\}),~{}\forall t\geq
0,~{}i=1,\dots,n,$ (3)

and $Z^{*}:=\sum_{i=1}^{n}a_{i}Z_{i}$. We note that
$\min\\{\sqrt{2},2^{1/\alpha}\\}\leq\lVert
Z_{i}\rVert_{\phi_{\alpha,L_{i}}}\leq\max\\{\sqrt{3},3^{1/\alpha}\\}$; see
Section 6 for the proof. Also, the proofs for the main results are provided in
Section 5.2.

Notation. The binary operator $\odot$ between two vectors indicates the
element-wise multiplication, i.e., for two vectors $a,b\in\mathbb{R}^{n}$,
$a\odot b=(a_{i}b_{i})_{i=1}^{n}$. In the following results, $C(\dots)$ is a
constant with implicit dependency to the parameters in the parentheses, whose
value changes across lines. We omit the parentheses for absolute constants
with no dependency on problem parameters and denote them by $C$.

###  2.1 Optimal Moment Bound

First, we provide a tight upper bound for the moments of $X^{*}$. Following
the proof of Theorem 3.1 in [12], we show that $\lVert X^{*}\rVert_{p}$ is
upper bounded by $C(\alpha)\lVert Z^{*}\rVert_{p}$. On the other hand,
$Z_{i}/\max\\{\sqrt{3},3^{1/\alpha}\\}\in\Phi_{\alpha,L_{i}}(1)$, so $\lVert
Z^{*}/\max\\{\sqrt{3},3^{1/\alpha}\\}\rVert_{p}$ lower bounds
$\sup_{X_{i}\in\Phi_{\alpha,L_{i}}(1)}\lVert X^{*}\rVert_{p}$. Therefore, the
tight moment bounds for $Z^{*}$ are sufficient for deriving those for $X^{*}$.
For $\alpha\leq 1$, the tail probability $\mathbb{P}[\lvert Z_{i}\rvert\geq
t]$ is log-concave for small $t$ and logarithmically convex (log-convex) for
large $t$, which are addressed in Examples 3.2 and 3.3 of [13], respectively.
For $\alpha\geq 1$, the tail probability is bounded by two log-concave
functions. The tight moment concentration inequality for log-concave-tailed
random variables is available in [8]. We use a slightly generalized result in
Example 3.2 of [13], which didn’t require i.i.d. assumption made in [8]. The
resulting moment bound is given as follows.

######  Theorem 2.1.

Fix $\alpha\geq 0$ and non-negative real numbers $L_{1},\ldots,L_{n}$. Then
for $p\geq 1$,

$\sup_{(X_{1},\dots,X_{n})\in\mathcal{X}_{\alpha,L}}\lVert
X^{*}\rVert_{p}~{}\leq~{}C(\alpha)\max\left\\{\sqrt{p}\lVert
a\odot\overline{L}\rVert_{2},\;p^{1/\alpha}\lVert(a_{i}L_{i}:i\leq
p)\rVert_{\beta}\right\\},$ (4)

where $\beta=\frac{\alpha}{\alpha-1}$ for $\alpha>1$ and $\infty$ for
$\alpha\leq 1$. Furthermore, the upper bound is tight in terms of that for
$p\geq 1$,

$\sup_{(X_{1},\dots,X_{n})\in\mathcal{X}_{\alpha,L}}\lVert
X^{*}\rVert_{p}~{}\geq~{}\frac{1}{C(\alpha)}\max\left\\{\sqrt{p}\lVert
a\odot\overline{L}\rVert_{2},\;p^{1/\alpha}\lVert(a_{i}L_{i}:i\leq
p)\rVert_{\beta}\right\\}.$ (5)

The bound is in the form of the Rosenthal inequalities. We focus on the
dependency of the constants on order $p$ of the moments, while the dependency
on $\alpha$ is given implicitly. The appearance of $\overline{L}$ is
anticipated because sub-Weibull$(\alpha,L)$ random variables have a mixture of
sub-Gaussian and sub-Weibull$(\alpha)$ tail probability bounds. The tail
behavior of $X^{*}$ follows both sub-Gaussian and sub-Weibull$(\alpha)$
concentration inequalities. Based on [12], the sub-Gaussian concentration
inequality induces a $p$-moment upper bound $C(\alpha)\sqrt{p}\lVert
a\rVert_{2}$, whereas the sub-Weibull$(\alpha)$ version yields
$C(\alpha)\max\\{\sqrt{p}\lVert a\odot
L\rVert_{2},p^{1/\alpha}\lVert(a_{i}L_{i}:i\leq p)\rVert_{\beta}\\}$. Taking
their maximum, we have $\overline{L}$ term inside the
$\lVert\cdot\rVert_{2}$-norm. This term is confirmed to be essential by the
lower bound in Eq. 5, which demonstrates the tightness of the upper bound. We
note that the supremum is attained (up to a constant) when
$X_{i}=Z_{i}/\max\\{\sqrt{3},3^{1/\alpha}\\}$.

######  Remark 2.2.

Suppose that $X_{i}$ are sub-Weibull$(\alpha)$ random variables with $\lVert
X_{i}\rVert_{\psi_{\alpha}}\leq 1$ and that $\lvert
a_{1}\rvert\geq\dots\geq\lvert a_{n}\rvert$. By the definitions of the
$\psi_{\alpha}$\- and $\phi_{\alpha,L}$-Orlicz norms,

$\lVert X_{i}\rVert_{\psi_{\alpha}}\leq 1\Rightarrow\lVert
L_{i}X_{i}\rVert_{\phi_{\alpha,L_{i}}}\leq 1,$

for any $L_{i}>0$. Applying Eq. 4 to
$X^{*}=\sum_{i=1}^{n}a_{i}X_{i}=\sum_{i=1}^{n}\frac{a_{i}}{L_{i}}(L_{i}X_{i})$,
we obtain

$\lVert X^{*}\rVert_{p}\leq
C(\alpha)\max\left\\{\sqrt{p}\lVert(a_{i}\max\\{1,1/L_{i}\\}:i\in[n])\rVert,p^{1/\alpha}\lVert(a_{i}:i\leq
p)\rVert_{\beta}\right\\}.$ (6)

Because Eq. 6 holds for every $L_{i}>0$, we have a concentration inequality
for sub-Weibull$(\alpha)$ random variables with finite $\psi_{\alpha}$ norms.

$\lVert X^{*}\rVert_{p}\leq C(\alpha)\max\left\\{\sqrt{p}\lVert
a\rVert_{2},p^{1/\alpha}\lVert(a_{i}:i\leq p)\rVert_{\beta}\right\\}.$ (7)

This upper bound is better than those in the proof of Theorem 3.1 in [12]. In
[12], the moment bound was written only in terms of $\|a\|_{\beta}$ and did
not involve $\lVert(a_{i}:i\leq p)\rVert_{\beta}$. Because

$\|(a_{i}:\,1\leq i\leq p)\|_{\beta}=\begin{cases}\|(a_{i}:\,1\leq i\leq
p)\|_{\infty}=\|a\|_{\infty}=\|a\|_{\beta},&\mbox{if }\alpha\leq 1,\\\
\|(a_{i}:\,1\leq i\leq p)\|_{\beta}\leq\|a\|_{\beta},&\mbox{if
}\alpha>1,\end{cases}$

we get that the moment upper bound in [12] was suboptimal for $\alpha>1$.

###  2.2 Optimal GBO Norm Bound

Next, we induce GBO norm bounds for $X^{*}$ from the moment bound results in
Section 2.1. We use the equivalence between the moment growth and the GBO
norm, provided in Proposition A.4 of [12]. The resulting GBO norm bound is
given as follows.

######  Theorem 2.3.

There exists a constant $C(\alpha)$ depending only on $\alpha$ such that with
$L^{*}:=C(\alpha)\lVert a\odot L\rVert_{\beta}/\lVert
a\odot\overline{L}\rVert_{2}$,

$\sup_{(X_{1},\dots,X_{n})\in\mathcal{X}_{\alpha,L}}\lVert
X^{*}\rVert_{\phi_{\alpha,L^{*}}}~{}\leq~{}C(\alpha)\lVert
a\odot\overline{L}\rVert_{2},$ (8)

where $\beta=\frac{\alpha}{\alpha-1}$ for $\alpha>1$ and $\infty$ for
$\alpha\leq 1$. Furthermore, the upper bound is tight in terms of that

$\sup_{(X_{1},\dots,X_{n})\in\mathcal{X}_{\alpha,L}}\lVert
X^{*}\rVert_{\phi_{\alpha,L^{*}}}~{}\geq~{}\frac{1}{C(\alpha)}\lVert
a\odot\overline{L}\rVert_{2}.$ (9)

The supremum in Eq. 9 is attained (up to constants) when
$X_{i}=Z_{i}/\max\\{\sqrt{3},3^{1/\alpha}\\}\in\Phi_{\alpha,L_{i}}(1)$.

###  2.3 Optimal Tail Probability Bound

Last, we induce the tail probability bounds for $X^{*}$ from the moment bound
in Section 2.1. The upper bound follows from Chebyshev’s inequality, and the
lower bound from the Paley-Zygmund inequality. The proof follows Corollary of
[8] and Theorem 2.1 of [9].

######  Theorem 2.4.

Let $K(t)=\sup\\{p:\sqrt{p}\lVert
a\odot\overline{L}\rVert_{2}+p^{1/\alpha}\lVert(a_{i}L_{i}:i\leq
p)\rVert_{\beta}\leq t\\}$, where $\beta=\frac{\alpha}{\alpha-1}$ for
$\alpha>1$ and $\infty$ for $\alpha\leq 1$. Then, for all $\alpha>0$, there
exists a constant $C(\alpha)$ such that for all $t\geq\lVert
a\odot\overline{L}\rVert_{2}+\lVert a\odot L\rVert_{\infty}$,

$\sup_{(X_{1},\dots,X_{n})\in\mathcal{X}_{\alpha,L}}\mathbb{P}\left[\left\lvert
X^{*}\right\rvert\geq
t\right]~{}\leq~{}\exp\left(-\frac{1}{C(\alpha)}K(t)\right).$ (10)

Furthermore, the upper bound is tight in terms of that for $t\geq\lVert
a\odot\overline{L}\rVert_{2}+\lVert a\odot L\rVert_{\infty}$,

$\sup_{(X_{1},\dots,X_{n})\in\mathcal{X}_{\alpha,L}}\mathbb{P}\left[\left\lvert
X^{*}\right\rvert\geq t\right]~{}\geq~{}\exp\left(-C(\alpha)K(t)\right).$ (11)

Alternatively, for $t\geq 0$,

$\begin{split}&\sup_{(X_{1},\dots,X_{n})\in\mathcal{X}_{\alpha,L}}\mathbb{P}\left[\left\lvert
X^{*}\right\rvert\geq
t\right]~{}\leq~{}2\exp\left(-\frac{1}{C(\alpha)}\min\left\\{\frac{t^{2}}{\lVert
a\odot\overline{L}\rVert_{2}^{2}},\frac{t^{\alpha}}{\lVert a\odot
L\rVert_{\beta}^{\alpha}}\right\\}\right),\end{split}$ (12)

and the upper bound is tight if $\alpha\leq 1$: for $t\geq 0$,

$\begin{split}&\sup_{(X_{1},\dots,X_{n})\in\mathcal{X}_{\alpha,L}}\mathbb{P}\left[\left\lvert
X^{*}\right\rvert\geq
t\right]~{}\geq~{}\frac{1}{C(\alpha)}\exp\left(-C(\alpha)\min\left\\{\frac{t^{2}}{\lVert
a\odot\overline{L}\rVert_{2}^{2}},\frac{t^{\alpha}}{\lVert a\odot
L\rVert_{\beta}^{\alpha}}\right\\}\right).\end{split}$ (13)

Inequalities Eq. 10 and Eq. 11 provide an optimal tail bounds for all
$\alpha>0$ and all $t\geq\|a\odot\overline{L}\|_{2}+\|a\odot L\|_{\infty}$.
Unfortunately, these tail bounds involve $K(t)$ which is not available in
closed form because the moment bound includes $(a_{i}L_{i}:1\leq i\leq p)$.
Inequality Eq. 12 provides a simple tail upper bound for all $\alpha>0$. While
this upper bound may not be optimal for all $\alpha>0$, it is a tight bound
when $\alpha\leq 1$. This tightness comes from the fact that $\lVert a\odot
L\rVert_{\infty}=\lVert(a_{i}L_{i}:i\leq p)\rVert_{\beta}=\lVert a\odot
L\rVert_{\beta}$ for $\alpha\leq 1$.

##  3 Application

In this section, we apply the techniques and results developed in previous
sections to derive dimension/sample complexity in an application related to
covariance matrix estimation. Suppose we observe $m$ sets of $p$-dimensional
random vectors and estimate the covariance matrix within each sample set.
Namely, for $l\in[m]$, $X^{(l)}_{1},\dots,X^{(l)}_{n}$ are independent random
vectors with mean zero and covariance matrix $\Sigma^{(l)}$. A straightforward
estimator of $\Sigma^{(l)}$ is the empirical covariance matrix
$\widehat{\Sigma}^{(l)}=\frac{1}{n}\sum_{k=1}^{n}X^{(l)}_{k}X^{(l)\top}_{k}$.
We attempt to identify the shared covariance structure among
$\\{\Sigma^{(l)}:l\in[m]\\}$ based on $\\{\widehat{\Sigma}^{(l)}:l\in[m]\\}$.
An important challenge in statistical inference is estimating heterogeneity
across the $m$ estimators. For example, in Chapter 2 of [5], the author
developed and studied a method of identifying the common graphical structure
across multiple sets of matrix-variate random samples. Their theoretical
analyses configured the leading term of the inference error to be
$\sup_{i,j\in[p]}\sum_{l=1}^{m}(\widehat{\Sigma}_{ij}^{(l)}-\Sigma_{ij}^{(l)})^{2}$,
which evaluates the variability of the estimation error in
$\widehat{\Sigma}^{(l)}$ across $l\in[m]$. It is desired to configure the
tight concentration inequality for the error term.

For simplicity of the analysis, we assume each entry of the random vectors is
sub-Gaussian: $\lVert X^{(l)}_{k,i}\rVert_{\psi_{2}}\leq 1$ for $i\in[p]$,
$k\in[n]$ and $l\in[m]$. Recall that $\lVert\cdot\rVert_{\psi_{\alpha}}$ is
the Orlicz norm with $\psi_{\alpha}(x)=\exp(x^{\alpha})-1$. Based on
Proposition D.2 of [12] and the centering lemma (Exercise 2.7.10, [18]),
$X_{k,i}^{(l)}X_{k,j}^{(l)}$ is sub-exponential: $\lVert
X_{k,i}^{(l)}X_{k,j}^{(l)}-\Sigma_{ij}^{(l)}\rVert_{\psi_{1}}\leq C$ for some
universal constant $C>0$, and Theorem 3.1 of [12] implies that

$\left\lVert\widehat{\Sigma}_{ij}^{(l)}-\Sigma_{ij}^{(l)}\right\rVert_{\phi_{1,\frac{C}{\sqrt{n}}}}=\left\lVert\frac{1}{n}\sum_{k=1}^{n}X_{k,i}^{(l)}X_{k,j}^{(l)}-\Sigma_{ij}^{(l)}\right\rVert_{\phi_{1,\frac{C}{\sqrt{n}}}}\leq\frac{C}{\sqrt{n}}.$
(14)

It is easy to check that $(\widehat{\Sigma}_{ij}^{(l)}-\Sigma_{ij}^{(l)})^{2}$
has a mixture sub-exponential and sub-Weibull$(1/2)$ tails: namely,

$\mathbb{P}[(\widehat{\Sigma}_{ij}^{(l)}-\Sigma_{ij}^{(l)})^{2}\geq t]\leq
2\exp\left(-C\min\\{nt,n\sqrt{t}\\}\right).$ (15)

Based on this observation, we obtain Theorem 3.1 for the tail probability
bound of $\sum_{l=1}^{m}(\widehat{\Sigma}_{ij}^{(l)}-\Sigma_{ij}^{(l)})^{2}$,
using the main results in Section 2. See Section 5.3 for the proof details. It
is important to note that the results in Section 2 are not readily applicable
(particularly, the lower bounds) because the tail in Eq. 15 is a mixture of
exponential and sub-Weibull$(1/2)$ and not a mixture of Gaussian and sub-
Weibull($1/2$) as required by the GBO norm.

######  Theorem 3.1.

For $\widehat{\Sigma}$ satisfying the aforementioned conditions,

$\displaystyle\mathbb{P}\left[\sum_{l=1}^{m}(\widehat{\Sigma}_{ij}^{(l)}-\Sigma_{ij}^{(l)})^{2}\geq
t+C\frac{m}{n}\right]\leq
4\exp\left(-C\min\left\\{\frac{n^{2}t^{2}}{m},nt,n\sqrt{t}\right\\}\right),~{}\forall
t>0,$ (16)

or equivalently,

$\sum_{l=1}^{m}(\widehat{\Sigma}_{ij}^{(l)}-\Sigma_{ij}^{(l)})^{2}\leq
C\left(\frac{m+\sqrt{m\nu}+\nu}{n}+\frac{\nu^{2}}{n^{2}}\right),$

with probability at least $1-e^{-\nu}$ for any $\nu>0$. On the other hand,
suppose that there exists $C>0$ such that
$\mathbb{P}[\lvert\widehat{\Sigma}_{ij}^{(l)}-\Sigma_{ij}^{(l)}\rvert\geq
t]{\geq}\exp(-C\min\\{nt,nt^{2}\\}),~{}\forall t>0$. Then,

$\displaystyle\mathbb{P}\left[\sum_{l=1}^{m}(\widehat{\Sigma}_{ij}^{(l)}-\Sigma_{ij}^{(l)})^{2}\geq
t+C\frac{m}{n}\right]$
$\displaystyle\geq\frac{1}{C}\exp\left(-C\min\left\\{\frac{n^{2}t^{2}}{m},nt,n\sqrt{t}\right\\}\right),~{}\forall
t>0.$ (17)

The required sample complexity for the convergence of the leading error term
to $0$ in probability is ${m+\log(q)}=o({n})$, which is better than
${m+\log(q)^{2}}=o({n})$ in Lemma B.2.5 of [5]. Moreover, Eq. 17 shows the
tightness of the upper bound. The condition
$\mathbb{P}[\lvert\widehat{\Sigma}_{ij}^{(l)}-\Sigma_{ij}^{(l)}\rvert\geq
t]{\geq}\exp(-C\min\\{nt,nt^{2}\\}),~{}\forall t>0$ is feasible in covariance
estimation. For example, suppose that $X_{k}^{(l)}$ is a multivariate Gaussian
random vector with the identity covariance matrix. Then, for any $i\in[p]$,
$\widehat{\Sigma}_{ii}^{(l)}$ is a chi-squared distribution random variable
with degree of freedom $n$. In this case, the lower bound for the tail
probability lower bound satisfies the condition. We refer to Corollary 3 of
[21] for the tail probability lower bound of chi-squared random variables.

##  4 Discussion

In this paper, we presented concentration inequalities in moments, the GBO
norms, and tail probability bounds for weighted sums of independent
$\text{sub-Weibull}(\alpha,L)$ random variables. The resulting moment bound
was in form of Rosenthal’s inequalities, and the tail probability bounds
provided generalized Bernstein’s inequalities for $\text{sub-
Weibull}(\alpha,L)$ random variables. We also showed the tightness of the
proposed bounds by obtaining the lower bounds of the concentration
inequalities in the worst case.

We applied the main results to a graphical model inference problem. Although
the summands of interest were not sub-Weibull and had a mixture of sub-
exponential and sub-Weibull$({1}/{2}$) tails instead, we were still able to
provide a tight bound for the inference error, using the main results. This
improved the sample complexity in the original work (Chapter 2, [5]) for the
convergence of the leading error term to $0$ in probability. Likewise, the
developed concentration inequalities will be useful in binding the size of
summed random variables when the random variables have mixture tails, which
U-statistics often entail with (e.g., see [7] and [1]).

There are several interesting future directions based on this work. Firstly,
our bounds are derived for random variables under the condition of the GBO
norm. Often in applications, it would be useful to take into account the
variance of the random variables involved. In [12], the authors provided upper
bounds for the tails and moments under the additional condition of variance.
To our knowledge, lower bounds on the tail are not available for sums of
independent sub-Weibull random variables with an additional condition on
variance. Secondly, mixture of different tails appear naturally for
$U$-statistics as shown in [7], but optimal tail bounds are not available
unless the kernel is assumed to be bounded. It would be interesting to develop
a parallel framework for $U$-statistics with general decay on the kernel.

##  5 Proofs

###  5.1 Preliminary Lemmas

Following the arguments in [13], we define

$\displaystyle\varphi_{p}(x)$ $\displaystyle:=\frac{\lvert
1+x\rvert^{p}+\lvert 1-x\rvert^{p}}{2},$ $\displaystyle\phi_{p}(X)$
$\displaystyle:=\mathbb{E}[\varphi_{p}(X)],$

for $x\in\mathbb{R}$ and a random variable $X$. We define the following Orlicz
norm for a sequence of symmetric random variables $X_{1},\dots,X_{n}$:

$\lvert\kern-1.07639pt\lvert\kern-1.07639pt\lvert(X_{i})\rvert\kern-1.07639pt\rvert\kern-1.07639pt\rvert_{p}:=\inf\\{\eta>0:\sum_{i}\log(\phi_{p}(X_{i}/\eta))<p\\}.$
(18)

Due to technical issues in the forthcoming proofs, we use a different
definition of sequence Orlicz norm than that in [13], which was given by
$\inf\\{\eta>0:\sum_{i}\log(\phi_{p}(X_{i}/\eta))\leq p\\}$. We note that
$\phi_{p}(X/\eta)$ is strictly decreasing for $\eta\in(0,\infty)$ unless $X=0$
almost surely, so the resulting value of
$\lvert\kern-1.07639pt\lvert\kern-1.07639pt\lvert(X_{i})\rvert\kern-1.07639pt\rvert\kern-1.07639pt\rvert_{p}$
does not change. The following lemma references to [13]. We note that the
lemma holds for non-integer $p$.

######  Lemma 5.1 ($p$-th moment bound; Theorem 2, [13]).

Let $X_{1},\dots,X_{n}$ be a sequence of independent symmetric random
variables, and $p\geq 2$. then,

$\frac{e-1}{2e^{2}}\lvert\kern-1.07639pt\lvert\kern-1.07639pt\lvert(X_{i})\rvert\kern-1.07639pt\rvert\kern-1.07639pt\rvert_{p}\leq\lVert
X_{1}+\dots+X_{n}\rVert_{p}\leq
e\lvert\kern-1.07639pt\lvert\kern-1.07639pt\lvert(X_{i})\rvert\kern-1.07639pt\rvert\kern-1.07639pt\rvert_{p}.$

In [13], the authors demonstrated the use of Lemma 5.1 in linear sums of iid
symmetric random variables when the random variables have logarithmically
concave or convex tails. (See the examples in Sections 3.2 and 3.3 therein.)
Although $Z_{i}$ does not have such tails, we can use the results in [13]
based on the relationship between $Z_{i}$ and the following random variable.
Let $Y_{i}$ be an iid symmetric sub-Gaussian random variable satisfying

$\mathbb{P}[\lvert Y_{i}\rvert\geq t]=e^{-t^{2}},~{}~{}\forall t\geq
0,~{}~{}i=1,\dots,n.$ (19)

Then, $Y_{i}$ relates to $Z_{i}$ through the following lemma:

######  Lemma 5.2.

$\lvert Z_{i}\rvert\overset{d}{=}\max\\{\lvert Y_{i}\rvert,L_{i}\lvert
Y_{i}\rvert^{2/\alpha}\\}.$ (20)

See Section 5.4 for the proof.

###  5.2 Proofs for Main Results

######  Proof 5.3 (Proof of Theorem 2.1).

For the upper bound in Eq. 4, we follow the argument of [12] using the
symmetrization inequality (Proposition 6.3 of [14]) and Theorem 1.3.1 of [6].
If we set $\mu_{i}=\max\\{\sqrt{\log 2},L_{i}(\log 2)^{1/\alpha}\\}$,

$\displaystyle\mathbb{P}[\lvert X_{i}\rvert\geq t]\leq
2\exp(-\min\\{t^{2},(t/L_{i})^{\alpha}\\})$ (21)
$\displaystyle\Longrightarrow\mathbb{P}[\max\\{\lvert
X_{i}\rvert-\mu_{i},0\\}\geq t]\leq\exp(-\min\\{t^{2},(t/L_{i})^{\alpha}\\}).$

Because $\mu_{i}\leq C(\alpha)\overline{L}_{i}$, for $p\geq 1$,

$\displaystyle\lVert X^{*}\rVert_{p}$ $\displaystyle\leq
2\left\lVert\sum_{i=1}^{n}\varepsilon_{i}a_{i}\max\\{\lvert
X_{i}\rvert-\mu_{i},0\\}\right\rVert_{p}+2\sqrt{p}\lVert a\odot\mu\rVert_{2}$
$\displaystyle\leq 2\left\lVert Z^{*}\right\rVert_{p}+C(\alpha)\sqrt{p}\lVert
a\odot\overline{L}\rVert_{2}.$

For the lower bound in Eq. 5,
$Z_{i}/\max\\{\sqrt{3},3^{1/\alpha}\\}\in\Phi_{\alpha,L_{i}}(1)$, so $\lVert
Z^{*}/\max\\{\sqrt{3},3^{1/\alpha}\\}\rVert_{p}$ lowerbounds
$\sup_{X_{i}\in\Phi_{\alpha,L_{i}}(1)}\lVert X^{*}\rVert_{p}$. Therefore, it
suffices to derive the tight moment bound for $Z^{*}$. We consider the cases
of $\alpha>1$ and $\alpha\leq 1$, separately.

Cases of $\alpha>1$:  Let $Z^{\prime}_{i}=Z_{i}/\overline{L}_{i}$ and
$a^{\prime}_{i}=a_{i}\overline{L}_{i}$ so that $\mathbb{P}[\lvert
Z^{\prime}_{i}\rvert\leq 1]=e^{-1}$. We define
$N_{i}(t)=-\log(\mathbb{P}[\lvert Z^{\prime}_{i}\rvert\geq
t])=\min\\{N_{i,1}(t),N_{i,2}(t)\\}$, where
$N_{i,1}(t)=\overline{L}_{i}^{2}t^{2}$, and
$N_{i,2}(t)=(\frac{\overline{L}_{i}}{L_{i}}t)^{\alpha}$. We also define the
dual functions $N_{i}^{*}(t)$, $N_{i,1}^{*}(t)$ and $N_{i,2}^{*}(t)$ of
$N_{i}$, $N_{i,1}$ and $N_{i,2}$:

$N_{i}^{*}(t)=\sup\\{st-N_{i}(s):s>0\\}\enskip\text{and}\enskip
N_{i,k}^{(t)}=\sup\\{st-N_{i}(s):s>0\\}\text{ for }k\in\\{1,2\\}.$

In particular, $N_{i,1}^{*}(t)=\frac{t^{2}}{4\overline{L}_{i}^{2}}$ and
$N_{i,2}^{*}(t)=(\alpha-1)(\frac{L_{i}t}{\overline{L}_{i}\alpha})^{\alpha/(\alpha-1)}$.
Because they are suprema of linear (i.e., convex) functions, $N_{i}$,
$N_{i,1}$ and $N_{i,2}$ are convex. We claim that
$N_{i}^{*}(t)=\max\\{N_{i,1}^{*}(t),N_{i,2}^{*}(t)\\}$. The claim is justified
by

$\displaystyle N_{i}^{*}(t)$
$\displaystyle=\sup\left\\{st-\min\left\\{N_{i,1}(t),N_{i,2}(t)\right\\}\right\\}=\sup\left\\{st-
N_{i,k}(t):k=1,2\right\\}$ $\displaystyle=\max\\{\sup\\{st-
N_{i,1}(t)\\},\sup\\{st-
N_{i,2}(t)\\}\\}=\max\\{N_{i,1}^{*}(t),N_{i,2}^{*}(t)\\}.$

Although $N_{i}(t)$ is not convex, $N_{i}(st)\leq sN_{i}(t)$ for any $t>0$ and
$s\in[0,1]$. So Theorem 1 of [8] is applicable. (See Remark 4 therein.) Here
we state a slightly generalized version that does not require i.i.d.
assumption made in [8] and provide a standalone proof in Section 5.4.

######  Lemma 5.4 (A generalized version of Theorem 1, [8]).

Let $Z_{1},\dots,Z_{n}$ be random variables with $\mathbb{P}[\lvert
Z_{i}\rvert\geq t]=e^{-N_{i}(t)}$ and $a_{1},\dots,a_{n}$ be a nonincreasing
sequence of nonnegative real numbers. For each $i$, we define

$\kappa_{i}=\inf\left\\{c>0:\int_{s}^{\infty}e^{-N_{i}(t)}dt\leq\int_{s}^{\infty}e^{-t/c}dt,\text{
for all }s>0\right\\}$

and assume that $N_{i}(1)=1$ and $N_{i}(st)\leq sN_{i}(t),\forall t>0,\forall
s\in[0,1]$. For each $1\leq p<\infty$,

$\displaystyle C_{1}\Big{(}\inf\Big{\\{}t>0:{\textstyle\sum}_{i\leq
p}N_{i}^{*}(pa_{i}/t)\leq
p\Big{\\}}+\sqrt{p}\big{(}{\textstyle\sum}_{i>p}a_{i}^{2}\big{)}^{1/2}\Big{)}$
$\displaystyle\leq\lVert{\textstyle\sum}_{i=1}^{n}a_{i}Z_{i}\rVert_{p}\leq
C_{2}\Big{(}\inf\Big{\\{}t>0:{\textstyle\sum}_{i\leq p}N_{i}^{*}(pa_{i}/t)\leq
p\Big{\\}}+\sqrt{p}\big{(}{\textstyle\sum}_{i>p}a_{i}^{2}\big{)}^{1/2}\Big{)},$

where $C_{1}=\min\\{\min\\{\frac{\mathbb{E}[\lvert
Z_{i}\rvert]}{4}:i\in[n]\\},\frac{1}{4e}\\}$ and
$C_{2}=3+2\max\\{\kappa_{i}:i\in[n]\\}$.

We apply Lemma 5.4 to
${\textstyle\sum}_{i=1}^{n}a^{\prime}_{i}Z^{\prime}_{i}$. First, we note that
$C_{1}$ and $C_{2}$ are constants with only dependency on $\alpha$, because
$\lvert Z^{\prime}_{i}\rvert\geq\min\\{Y_{i},Y_{i}^{2/\alpha}\\}$ and
$N_{i}(t)\geq\min\\{x^{2},x^{\alpha}\\}$. Thus, Lemma 5.4 implies that

$\lVert
Z^{*}\rVert_{p}=\lVert{\textstyle\sum}_{i=1}^{p}a^{\prime}_{i}Z^{\prime}_{i}\rVert_{p}\sim\inf\Big{\\{}t>0:{\textstyle\sum}_{i\leq
p}N_{i}^{*}(pa_{i}\overline{L}_{i}/t)\leq
p\Big{\\}}+\sqrt{p}\big{(}{\textstyle\sum}_{i>p}a_{i}^{2}\overline{L}_{i}^{2}\big{)}^{1/2},$

as $a^{\prime}_{i}=a_{i}\overline{L}_{i}$. Based on

$N_{i}^{*}(pa_{i}\overline{L}_{i}/t)=\max\left\\{\frac{p^{2}}{4t^{2}}a_{i}^{2},(\alpha-1)\left(\frac{pa_{i}L_{i}}{\alpha
t}\right)^{\beta}\right\\},$

we obtain

$\displaystyle\lVert Z^{*}\rVert_{p}$
$\displaystyle\asymp\inf\Big{\\{}t>0:\frac{p^{2}}{4t^{2}}{\textstyle\sum}_{i\leq
p}a_{i}^{2}+C(\alpha)\left(\frac{p}{t}\right)^{\beta}{\textstyle\sum}_{i\leq
p}(a_{i}L_{i})^{\beta}\leq p\Big{\\}}$
$\displaystyle\quad+\sqrt{p}\big{(}{\textstyle\sum}_{i>p}a_{i}^{2}\overline{L}_{i}^{2}\big{)}^{1/2}$
$\displaystyle\asymp\sqrt{p}\left({\textstyle\sum}_{i\leq
p}a_{i}^{2}\right)^{1/2}+p^{1/\alpha}\left({\textstyle\sum}_{i\leq
p}(a_{i}L_{i})^{\beta}\right)^{1/\beta}+\sqrt{p}\big{(}{\textstyle\sum}_{i>p}a_{i}^{2}\overline{L}_{i}^{2}\big{)}^{1/2},$

where $\beta=\frac{\alpha}{\alpha-1}$ and $\asymp$ indicates the equivalence
with respect to constants depending only on $\alpha$, i.e., $x\asymp
y\Leftrightarrow\frac{1}{C(\alpha)}x\leq y\leq C(\alpha)x$ for any
$x,y\in\mathbb{R}$. The desired result comes from the fact that

$\sqrt{p}\left({\textstyle\sum}_{i\leq
p}a_{i}^{2}\right)^{1/2}+p^{1/\alpha}\left({\textstyle\sum}_{i\leq
p}(a_{i}L_{i})^{\beta}\right)^{1/\beta}\sim\sqrt{p}\left({\textstyle\sum}_{i\leq
p}a_{i}^{2}\overline{L}_{i}^{2}\right)^{1/2}+p^{1/\alpha}\left({\textstyle\sum}_{i\leq
p}(a_{i}L_{i})^{\beta}\right)^{1/\beta},$

for $\alpha\leq 2$.

Cases of $\alpha\leq 1$:  Using the relationship between $Z_{i}$ and $Y_{i}$
in Lemma 5.2, we obtain Lemma 5.5. Because $Y_{i}$ and $\lvert
Y_{i}\rvert^{2/\alpha}$ have log-concave and convex tails, they fall into
Examples 3.2 and 3.3 of [13], respectively.

######  Lemma 5.5.

For $i=1,\dots,n$ and $\eta>0$,

$\displaystyle
p\min\left\\{1,\max\left\\{\frac{\overline{L}_{i}^{2}}{\eta^{2}}p,\frac{1}{C(\alpha)}\frac{L_{i}^{p}}{e^{2p}\eta^{p}}\left(\frac{p}{\alpha
e}\right)^{\frac{p}{\alpha}}\right\\}\right\\}$ (22)
$\displaystyle\leq\log\phi_{p}\left(\frac{Z_{i}}{\eta}\right)\leq
C(\alpha)~{}p~{}\max\left\\{\frac{\overline{L}_{i}^{2}}{\eta^{2}}p,\frac{e^{2p}L_{i}^{p}}{\eta^{p}}\left(\frac{p}{\alpha
e}\right)^{\frac{p}{\alpha}}\right\\},$

where $\overline{L}=(\max\\{1,L_{i}\\})_{i=1,\dots,n}$.

For each $i\in[n]$, we replace $\eta$ in the above lemma with $\eta/a_{i}$ to
obtain

$\displaystyle
p\min\left\\{1,\frac{1}{C(\alpha)}\max\left\\{\frac{(a_{i}\overline{L}_{i})^{2}}{\eta^{2}}p,\frac{(a_{i}L_{i})^{p}}{e^{2p}\eta^{p}}\left(\frac{p}{\alpha
e}\right)^{\frac{p}{\alpha}}\right\\}\right\\}$
$\displaystyle\leq\log\phi_{p}\left(\frac{a_{i}Z_{i}}{\eta}\right)\leq
C(\alpha)~{}p~{}\max\left\\{\frac{(a_{i}\overline{L}_{i})^{2}}{\eta^{2}}p,\frac{e^{2p}(a_{i}L_{i})^{p}}{\eta^{p}}\left(\frac{p}{\alpha
e}\right)^{\frac{p}{\alpha}}\right\\}.$

By the summation across $i\in[n]$, we bound
$\sum_{i=1}^{n}\log\phi_{p}\left(\frac{a_{i}Z_{i}}{\eta}\right)$: for
$\eta>0$,

$\displaystyle
p\sum_{i=1}^{n}\min\left\\{1,\frac{1}{C(\alpha)}\max\left\\{\frac{(a_{i}\overline{L}_{i})^{2}}{\eta^{2}}p,\frac{(a_{i}L_{i})^{p}}{e^{2p}\eta^{p}}\left(\frac{p}{\alpha
e}\right)^{\frac{p}{\alpha}}\right\\}\right\\}$
$\displaystyle\leq\sum_{i=1}^{n}\log\phi_{p}\left(\frac{a_{i}Z_{i}}{\eta}\right)\leq
C(\alpha)~{}p~{}\sum_{i=1}^{n}\max\left\\{\frac{(a_{i}\overline{L}_{i})^{2}}{\eta^{2}}p,\frac{e^{2p}(a_{i}L_{i})^{p}}{\eta^{p}}\left(\frac{p}{\alpha
e}\right)^{\frac{p}{\alpha}}\right\\}.$

For the upperbound,

$\displaystyle
C(\alpha)~{}p~{}\sum_{i=1}^{n}\max\left\\{\frac{(a_{i}\overline{L}_{i})^{2}}{\eta^{2}}p,\frac{e^{2p}(a_{i}L_{i})^{p}}{\eta^{p}}\left(\frac{p}{\alpha
e}\right)^{\frac{p}{\alpha}}\right\\}$ $\displaystyle\leq
C(\alpha)~{}p~{}\sum_{i=1}^{n}\left(\frac{(a_{i}\overline{L}_{i})^{2}}{\eta^{2}}p+\frac{e^{2p}(a_{i}L_{i})^{p}}{\eta^{p}}\left(\frac{p}{\alpha
e}\right)^{\frac{p}{\alpha}}\right)$ $\displaystyle\leq
2C(\alpha)~{}p~{}\max\left\\{\sum_{i=1}^{n}\frac{(a_{i}\overline{L}_{i})^{2}}{\eta^{2}}p,\sum_{i=1}^{n}\frac{e^{2p}(a_{i}L_{i})^{p}}{\eta^{p}}\left(\frac{p}{\alpha
e}\right)^{\frac{p}{\alpha}}\right\\}$
$\displaystyle=2C(\alpha)~{}p~{}\max\left\\{\frac{\lVert
a\odot\overline{L}\rVert_{2}^{2}}{\eta^{2}}p,\frac{e^{2p}\lVert a\odot
L\rVert_{p}^{p}}{\eta^{p}}\left(\frac{p}{\alpha
e}\right)^{\frac{p}{\alpha}}\right\\}.$

For the lowerbound, if
$\frac{1}{C(\alpha)}\max\left\\{\frac{(a_{i}\overline{L}_{i})^{2}}{\eta^{2}}p,\frac{(a_{i}L_{i})^{p}}{e^{2p}\eta^{p}}\left(\frac{p}{\alpha
e}\right)^{\frac{p}{\alpha}}\right\\}\leq 1$ for all $i\in[n]$, then

$\displaystyle
p\sum_{i=1}^{n}\min\left\\{1,\frac{1}{C(\alpha)}\max\left\\{\frac{(a_{i}\overline{L}_{i})^{2}}{\eta^{2}}p,\frac{(a_{i}L_{i})^{p}}{e^{2p}\eta^{p}}\left(\frac{p}{\alpha
e}\right)^{\frac{p}{\alpha}}\right\\}\right\\}$
$\displaystyle=p\sum_{i=1}^{n}\frac{1}{C(\alpha)}\max\left\\{\frac{(a_{i}\overline{L}_{i})^{2}}{\eta^{2}}p,\frac{(a_{i}L_{i})^{p}}{e^{2p}\eta^{p}}\left(\frac{p}{\alpha
e}\right)^{\frac{p}{\alpha}}\right\\}$
$\displaystyle\geq\frac{p}{C(\alpha)}\max\left\\{\sum_{i=1}^{n}\frac{(a_{i}\overline{L}_{i})^{2}}{\eta^{2}}p,\sum_{i=1}^{n}\frac{(a_{i}L_{i})^{p}}{e^{2p}\eta^{p}}\left(\frac{p}{\alpha
e}\right)^{\frac{p}{\alpha}}\right\\}$
$\displaystyle=\frac{p}{C(\alpha)}\max\left\\{\frac{\lVert
a\odot\overline{L}\rVert_{2}^{2}}{\eta^{2}}p,\frac{e^{2p}\lVert a\odot
L\rVert_{p}^{p}}{\eta^{p}}\left(\frac{p}{\alpha
e}\right)^{\frac{p}{\alpha}}\right\\}.$

On the other hand, without loss of generality, if
$\frac{1}{C(\alpha)}\max\left\\{\frac{(a_{1}\overline{L}_{1})^{2}}{\eta^{2}}p,\frac{(a_{1}L_{1})^{p}}{e^{2p}\eta^{p}}\left(\frac{p}{\alpha
e}\right)^{\frac{p}{\alpha}}\right\\}>1$, then

$\displaystyle
p\sum_{i=1}^{n}\min\left\\{1,\frac{1}{C(\alpha)}\max\left\\{\frac{(a_{i}\overline{L}_{i})^{2}}{\eta^{2}}p,\frac{(a_{i}L_{i})^{p}}{e^{2p}\eta^{p}}\left(\frac{p}{\alpha
e}\right)^{\frac{p}{\alpha}}\right\\}\right\\}$ $\displaystyle\geq
p\min\left\\{1,\frac{1}{C(\alpha)}\max\left\\{\frac{(a_{1}\overline{L}_{1})^{2}}{\eta^{2}}p,\frac{(a_{1}L_{1})^{p}}{e^{2p}\eta^{p}}\left(\frac{p}{\alpha
e}\right)^{\frac{p}{\alpha}}\right\\}\right\\}$ $\displaystyle=p.$

In sum,

$\displaystyle p\min\left\\{1,\frac{1}{C(\alpha)}\max\left\\{\frac{\lVert
a\odot\overline{L}\rVert_{2}^{2}}{\eta^{2}}p,\frac{\lVert a\odot
L\rVert_{p}^{p}}{e^{2p}\eta^{p}}\left(\frac{p}{\alpha
e}\right)^{\frac{p}{\alpha}}\right\\}\right\\}$ (23)
$\displaystyle\leq\sum_{i=1}^{n}\log\phi_{p}\left(\frac{a_{i}Z_{i}}{\eta}\right)\leq
C(\alpha)~{}p~{}\max\left\\{\frac{\lVert
a\odot\overline{L}\rVert_{2}^{2}}{\eta^{2}}p,\frac{e^{2p}\lVert a\odot
L\rVert_{p}^{p}}{\eta^{p}}\left(\frac{p}{\alpha
e}\right)^{\frac{p}{\alpha}}\right\\},$

where $a\odot L=(a_{i}L_{i})_{i=1,\dots,n}$.

Now we give a tight bound for
$\lvert\kern-1.07639pt\lvert\kern-1.07639pt\lvert(a_{i}Z_{i})\rvert\kern-1.07639pt\rvert\kern-1.07639pt\rvert_{p}$
according to its definition in Eq. 18. For the upperbound, suppose that
$\eta_{0}=C_{1}(\alpha)\max\left\\{\sqrt{p}\lVert
a\odot\overline{L}\rVert_{2},p^{1/\alpha}\lVert a\odot L\rVert_{p}\right\\}$.
By the upperbound in LABEL:eq:log_phi_sum_bound,

$\displaystyle\sum_{i=1}^{n}\log\phi_{p}\left(\frac{a_{i}Z_{i}}{\eta_{0}}\right)\leq
C(\alpha)~{}p~{}\max\left\\{\frac{\lVert
a\odot\overline{L}\rVert_{2}^{2}}{\eta_{0}^{2}}p,\frac{e^{2p}\lVert a\odot
L\rVert_{p}^{p}}{\eta_{0}^{p}}\left(\frac{p}{\alpha
e}\right)^{\frac{p}{\alpha}}\right\\}$ $\displaystyle\leq
C(\alpha)~{}p~{}\max\left\\{\frac{\lVert
a\odot\overline{L}\rVert_{2}^{2}}{(C_{1}(\alpha)\sqrt{p}\lVert
a\odot\overline{L}\rVert_{2})^{2}}p,\frac{e^{2p}\lVert a\odot
L\rVert_{p}^{p}}{(C_{1}(\alpha)p^{1/\alpha}\lVert a\odot
L\rVert_{p})^{p}}\left(\frac{p}{\alpha e}\right)^{\frac{p}{\alpha}}\right\\}$
$\displaystyle\leq
C(\alpha)~{}p~{}\max\left\\{\frac{1}{C_{1}(\alpha)^{2}},\frac{e^{2p}}{C_{1}(\alpha)^{p}}\left(\frac{1}{\alpha
e}\right)^{\frac{p}{\alpha}}\right\\}.$

Thus, for
$C_{1}(\alpha)\geq\sqrt{2\max\\{C(\alpha),1\\}}\max\left\\{1,\frac{e^{2}}{(\alpha
e)^{1/\alpha}}\right\\}$,
$\sum_{i=1}^{n}\log\phi_{p}\left(\frac{a_{i}Z_{i}}{\eta_{0}}\right)\leq\frac{p}{2}$,
and

$\lvert\kern-1.07639pt\lvert\kern-1.07639pt\lvert(X_{i})\rvert\kern-1.07639pt\rvert\kern-1.07639pt\rvert_{p}\leq\eta_{0}=C_{1}(\alpha)\max\left\\{\sqrt{p}\lVert
a\odot\overline{L}\rVert_{2},p^{1/\alpha}\lVert a\odot L\rVert_{p}\right\\}.$

For the lowerbound, suppose that
$\eta_{0}=\frac{1}{C_{2}(\alpha)}\max\left\\{\sqrt{p}\lVert
a\odot\overline{L}\rVert_{2},p^{1/\alpha}\lVert a\odot L\rVert_{p}\right\\}$.
If $\sqrt{p}\lVert a\odot\overline{L}\rVert_{2}\leq p^{1/\alpha}\lVert a\odot
L\rVert_{p}$, by the lowerbound in LABEL:eq:log_phi_sum_bound,

$\displaystyle\sum_{i=1}^{n}\log\phi_{p}\left(\frac{a_{i}Z_{i}}{\eta_{0}}\right)\geq
p\min\left\\{1,\frac{1}{C(\alpha)}\max\left\\{\frac{\lVert
a\odot\overline{L}\rVert_{2}^{2}}{\eta_{0}^{2}}p,\frac{\lVert a\odot
L\rVert_{p}^{p}}{e^{2p}\eta_{0}^{p}}\left(\frac{p}{\alpha
e}\right)^{\frac{p}{\alpha}}\right\\}\right\\}$ $\displaystyle\geq
p\min\left\\{1,\frac{1}{C(\alpha)}\frac{C_{2}(\alpha)^{p}\lVert a\odot
L\rVert_{p}^{p}}{e^{2p}(p^{1/\alpha}\lVert a\odot
L\rVert_{p})^{p}}\left(\frac{p}{\alpha
e}\right)^{\frac{p}{\alpha}}\right\\}\geq
p\min\left\\{1,\frac{C_{2}(\alpha)^{p}}{C(\alpha)}\left(\frac{1}{\alpha
e^{2\alpha+1}}\right)^{\frac{p}{\alpha}}\right\\}.$

If $\sqrt{p}\lVert a\odot\overline{L}\rVert_{2}\geq p^{1/\alpha}\lVert a\odot
L\rVert_{p}$,

$\displaystyle\sum_{i=1}^{n}\log\phi_{p}\left(\frac{a_{i}Z_{i}}{\eta_{0}}\right)\geq
p\min\left\\{1,\frac{1}{C(\alpha)}\max\left\\{\frac{\lVert
a\odot\overline{L}\rVert_{2}^{2}}{\eta_{0}^{2}}p,\frac{\lVert a\odot
L\rVert_{p}^{p}}{e^{2p}\eta_{0}^{p}}\left(\frac{p}{\alpha
e}\right)^{\frac{p}{\alpha}}\right\\}\right\\}$ $\displaystyle\geq
p\min\left\\{1,\frac{1}{C(\alpha)}\frac{C_{2}(\alpha)^{2}\lVert
a\odot\overline{L}\rVert_{2}^{2}}{(\sqrt{p}\lVert
a\odot\overline{L}\rVert_{2})^{2}}p\right\\}\geq
p\min\left\\{1,\frac{C_{2}(\alpha)^{2}}{C(\alpha)}\right\\}.$

In sum,

$\displaystyle\sum_{i=1}^{n}\log\phi_{p}\left(\frac{a_{i}Z_{i}}{\eta}\right)\geq
p\min\left\\{1,\frac{C_{2}(\alpha)^{2}}{C(\alpha)},\frac{C_{2}(\alpha)^{p}}{C(\alpha)}\left(\frac{1}{\alpha
e^{2\alpha+1}}\right)^{\frac{p}{\alpha}}\right\\}\geq p,$

for $C_{2}(\alpha)\geq\sqrt{\max\\{C(\alpha),1\\}}(\alpha
e^{2\alpha+1})^{1/\alpha}$. Because
$\sum_{i=1}^{n}\log\phi_{p}\left(\frac{a_{i}Z_{i}}{\eta}\right)$ strictly
decreases for $\eta\in(0,\infty)$,

$\lvert\kern-1.07639pt\lvert\kern-1.07639pt\lvert(a_{i}Z_{i})\rvert\kern-1.07639pt\rvert\kern-1.07639pt\rvert_{p}\geq\eta_{0}=\frac{1}{C_{2}(\alpha)}\max\left\\{\sqrt{p}\lVert
a\odot\overline{L}\rVert_{2},p^{1/\alpha}\lVert a\odot L\rVert_{p}\right\\}.$

The resulting bound for
$\lvert\kern-1.07639pt\lvert\kern-1.07639pt\lvert(X_{i})\rvert\kern-1.07639pt\rvert\kern-1.07639pt\rvert_{p}$
is

$\displaystyle\frac{1}{C(\alpha)}\max\left\\{\sqrt{p}\lVert
a\odot\overline{L}\rVert_{2},p^{1/\alpha}\lVert a\odot L\rVert_{p}\right\\}$
$\displaystyle\leq\lvert\kern-1.07639pt\lvert\kern-1.07639pt\lvert(a_{i}Z_{i})\rvert\kern-1.07639pt\rvert\kern-1.07639pt\rvert_{p}\leq
C(\alpha)\max\left\\{\sqrt{p}\lVert
a\odot\overline{L}\rVert_{2},p^{1/\alpha}\lVert a\odot L\rVert_{p}\right\\}.$

Then, Lemma 5.1 derives

$\displaystyle\frac{1}{C(\alpha)}\max\left\\{\sqrt{p}\lVert
a\odot\overline{L}\rVert_{2},p^{1/\alpha}\lVert a\odot L\rVert_{p}\right\\}$
$\displaystyle\leq\lVert Z^{*}\rVert_{p}\leq
C(\alpha)\max\left\\{\sqrt{p}\lVert
a\odot\overline{L}\rVert_{2},p^{1/\alpha}\lVert a\odot L\rVert_{p}\right\\},$

for $p\geq 2$, which implies the lowerbound in Eq. 5. We bound $\lVert a\odot
L\rVert_{p}$ in terms of $\lVert a\odot L\rVert_{2}$ and $\lVert a\odot
L\rVert_{\infty}$ following the argument in Corollary 1.2 in [4]. In the
proof, [4] showed that for $p\geq 2$,

$\lVert a\odot L\rVert_{\infty}\leq\lVert a\odot
L\rVert_{p}\leq\frac{e^{(2-\alpha)/\alpha}}{p^{1/\alpha}}(\sqrt{p}\lVert
a\odot L\rVert_{2}+p^{1/\alpha}\lVert a\odot L\rVert_{\infty}).$

Plugging it in Eqs. 5 and 4, we obtain the desired results for $p\geq 2$. Now
the remaining cases are for $1\leq p<2$. For the upper bound,

$\displaystyle\lVert Z^{*}\rVert_{p}$ $\displaystyle\leq\lVert
Z^{*}\rVert_{2}\leq C(\alpha)\max\left\\{\sqrt{2}\lVert
a\odot\overline{L}\rVert_{2},2^{1/\alpha}\lVert a\odot
L\rVert_{\infty}\right\\}$ $\displaystyle\leq
C(\alpha)\max\left\\{\sqrt{p}\lVert
a\odot\overline{L}\rVert_{2},p^{1/\alpha}\lVert a\odot
L\rVert_{\infty}\right\\}.$

On the other hand, because

$\displaystyle\lVert
Z^{*}\rVert_{2}\geq\frac{1}{C(\alpha)}\max\left\\{\sqrt{2}\lVert
a\odot\overline{L}\rVert_{2},2^{1/\alpha}\lVert a\odot
L\rVert_{\infty}\right\\}$ $\displaystyle\lVert Z^{*}\rVert_{4}\leq
C(\alpha)\max\left\\{2\lVert a\odot\overline{L}\rVert_{2},4^{1/\alpha}\lVert
a\odot L\rVert_{\infty}\right\\},$

by the Paley-Zygmund inequality,

$\displaystyle\mathbb{P}\left[\lvert Z^{*}\rvert\geq\frac{1}{2}\lVert
Z^{*}\rVert_{2}\right]$ $\displaystyle\geq\left(1-\frac{1}{4}\frac{\lVert
Z^{*}\rVert_{2}^{2}}{\lVert Z^{*}\rVert_{2}^{2}}\right)^{2}\frac{\lVert
Z^{*}\rVert_{2}^{4}}{\lVert Z^{*}\rVert_{4}^{4}}$
$\displaystyle\geq\frac{3}{4}\frac{1}{C(\alpha)}\left(\frac{\max\left\\{\sqrt{2}\lVert
a\odot\overline{L}\rVert_{2},2^{1/\alpha}\lVert a\odot
L\rVert_{\infty}\right\\}}{\max\left\\{2\lVert
a\odot\overline{L}\rVert_{2},4^{1/\alpha}\lVert a\odot
L\rVert_{\infty}\right\\}}\right)^{4}$
$\displaystyle\geq\frac{1}{C(\alpha)}\left(\max\left\\{\frac{\sqrt{2}\lVert
a\odot\overline{L}\rVert_{2}}{2\lVert
a\odot\overline{L}\rVert_{2}},\frac{2^{1/\alpha}\lVert a\odot
L\rVert_{\infty}}{4^{1/\alpha}\lVert a\odot
L\rVert_{\infty}}\right\\}\right)^{4}$ $\displaystyle\geq\frac{1}{C(\alpha)}.$

Hence, for $1\leq p<2$, by the Markov inequality,

$\displaystyle\lVert Z^{*}\rVert_{p}$ $\displaystyle\geq\lVert
Z^{*}\rVert_{1}\geq\frac{1}{2}\lVert
Z^{*}\rVert_{2}\cdot\mathbb{P}\left[\lvert Z^{*}\rvert\geq\frac{1}{2}\lVert
Z^{*}\rVert_{2}\right]$ $\displaystyle\geq\frac{1}{2C(\alpha)}\lVert
Z^{*}\rVert_{2}$ $\displaystyle\geq\frac{1}{C(\alpha)}\max\left\\{\lVert
a\odot\overline{L}\rVert_{2},\lVert a\odot L\rVert_{\infty}\right\\}$
$\displaystyle\geq\frac{1}{C(\alpha)}\max\left\\{\sqrt{p}\lVert
a\odot\overline{L}\rVert_{2},p^{1/\alpha}\lVert a\odot
L\rVert_{\infty}\right\\}.$

######  Proof 5.6 (Proof of Theorem 2.3).

For the upperbound, we use the moment bound in Theorem 2.1 using Proposition
C.1 in [12]:

######  Lemma 5.7 (Proposition C.1, [12]).

If $\lVert X\rVert_{p}\leq C_{1}\sqrt{p}+C_{2}p^{1/\alpha}$, holds for $p\geq
1$ and some constants $C_{1}$ and $C_{2}$, then $\lVert
X\rVert_{\phi_{\alpha,L}}\leq 2eC_{1}$ where
$L:=4^{1/\alpha}\frac{C_{2}}{2C_{1}}$.

Based on Theorem 2.1,

$\lVert X^{*}\rVert_{\phi_{\alpha,L^{*}}}\leq C(\alpha)\lVert
a\odot\overline{L}\rVert_{2}$

where $L^{*}=C(\alpha)\lVert a\odot L\rVert_{\beta}/\lVert
a\odot\overline{L}\rVert_{2}$. For the lowerbound, we use Proposition A.4 in
[12]:

######  Lemma 5.8 (Proposition A.4, [12]).

For any random variable $X$,

$\frac{1}{C(\alpha)}\sup_{p\geq 1}\frac{\lVert
X\rVert_{p}}{\sqrt{p}+Lp^{1/\alpha}}\leq\lVert X\rVert_{\phi_{\alpha,L}}\leq
C(\alpha)\sup_{p\geq 1}\frac{\lVert X\rVert_{p}}{\sqrt{p}+Lp^{1/\alpha}}.$

Because Eq. 5 implies that

$\displaystyle\sup_{p\geq 1}\frac{\lVert
Z^{*}\rVert_{p}}{\sqrt{p}+L^{*}p^{1/\alpha}}$ $\displaystyle\geq\frac{\lVert
Z^{*}\rVert_{n}}{\sqrt{n}+L^{*}n^{1/\alpha}}$
$\displaystyle\geq\frac{1}{C(\alpha)}\frac{\max\left\\{\sqrt{n}\lVert
a\odot\overline{L}\rVert_{2},n^{1/\alpha}\lVert a\odot
L\rVert_{\beta}\right\\}}{\sqrt{n}+L^{*}n^{1/\alpha}}$
$\displaystyle\geq\frac{1}{C(\alpha)}\lVert a\odot\overline{L}\rVert_{2},$

the desired lower bound $\lVert
Z^{*}\rVert_{\phi_{\alpha,L^{*}}}\geq\frac{1}{C(\alpha)}\lVert
a\odot\overline{L}\rVert_{2}$ is achieved.

######  Proof 5.9 (Proof of Theorem 2.4).

For Eqs. 10 and 11, see the proof of Corollary in [8]. Eq. 12 is the result of
Chernoff’s inequliaty on the GBO norm upperbound in Eq. 8. For Eq. 13, we use
the Paley-Zygmund inequality [17] based on the tight moment bound established
in Theorem 2.1. For $p\geq 2$,

$\displaystyle\mathbb{P}\left[\lvert Z^{*}\rvert\geq\frac{1}{2}\lVert
Z^{*}\rVert_{p}\right]$ $\displaystyle=\mathbb{P}\left[\lvert
Z^{*}\rvert^{p}\geq\frac{1}{2^{p}}\lVert Z^{*}\rVert_{p}^{p}\right]$
$\displaystyle\geq\left(1-\frac{1}{2^{p}}\right)^{2}\frac{\lVert
Z^{*}\rVert_{p}^{2p}}{\lVert Z^{*}\rVert_{2p}^{2p}}$
$\displaystyle\geq\frac{1}{2C(\alpha)^{p}}\frac{\max\\{p^{p}\lVert
a\odot\overline{L}\rVert_{2}^{2p},p^{2p/\alpha}\lVert a\odot
L\rVert_{p}^{2p}\\}}{\max\\{(2p)^{p}\lVert
a\odot\overline{L}\rVert_{2}^{2p},(2p)^{2p/\alpha}\lVert a\odot
L\rVert_{2p}^{2p}\\}}$
$\displaystyle\geq\frac{1}{2C(\alpha)^{p}}\max\left\\{\frac{p^{p}\lVert
a\odot\overline{L}\rVert_{2}^{2p}}{(2p)^{p}\lVert
a\odot\overline{L}\rVert_{2}^{2p}},\frac{p^{2p/\alpha}\lVert a\odot
L\rVert_{p}^{2p}}{(2p)^{2p/\alpha}\lVert a\odot L\rVert_{2p}^{2p}}\right\\}$
$\displaystyle\geq\frac{1}{2}\exp(-C(\alpha)~{}p).$

Plugging the moment lowerbound (Eq. 5) in,

$\mathbb{P}\left[\lvert
Z^{*}\rvert\geq\frac{1}{C(\alpha)}\max\\{\sqrt{p}\lVert
a\odot\overline{L}\rVert_{2},p^{1/\alpha}\lVert a\odot
L\rVert_{\infty}\\}\right]\geq\frac{1}{2}\exp(-C(\alpha)~{}p).$

With $p=\min\left\\{\left(\frac{2C(\alpha)t}{\lVert
a\odot\overline{L}\rVert_{2}}\right)^{2},\left(\frac{2C(\alpha)t}{\lVert
a\odot L\rVert_{\infty}}\right)^{\alpha}\right\\}$, we obtain the desired
lowerbound for $t\geq\frac{1}{2}\lVert Z^{*}\rVert_{2}$. Last, for
$t<\frac{1}{2}\lVert Z^{*}\rVert_{2}$, we use the Paley-Zygmund inequality on
the second moment bound:

$\displaystyle\mathbb{P}\left[\lvert Z^{*}\rvert\geq t\right]$
$\displaystyle\geq\left(1-\frac{t^{2}}{\lVert
Z^{*}\rVert_{2}^{2}}\right)^{2}\frac{\lVert Z^{*}\rVert_{2}^{4}}{\lVert
Z^{*}\rVert_{4}^{4}}$
$\displaystyle\geq\frac{1}{C(\alpha)}\exp\left(-\frac{2t^{2}}{\lVert
Z^{*}\rVert_{2}^{2}}\right)$
$\displaystyle\geq\frac{1}{C(\alpha)}\exp\left(-\frac{1}{C(\alpha)}\min\left\\{\left(\frac{2t}{\lVert
a\odot\overline{L}\rVert_{2}}\right)^{2},\left(\frac{2t}{\lVert a\odot
L\rVert_{\infty}}\right)^{\alpha}\right\\}\right).$

###  5.3 Proof for Application

######  Proof 5.10 (Proof of Theorem 3.1).

Following the same argument in the proof of Theorem 2.1 (see Eq. 21),

$\displaystyle\mathbb{P}[\max\\{(\widehat{\Sigma}_{ij}^{(l)}-\Sigma_{ij}^{(l)})^{2}-\mu,0\\}\geq
t]$ $\displaystyle\leq\exp(-C\min\\{nt,n\sqrt{t}\\}),$

where $\mu=\frac{C\log 2}{n}$. We note that based on the same argument in
Lemma 5.2,

$\displaystyle\mathbb{P}\left[C\max\left\\{\frac{Y^{(l)2}}{n},\frac{Y^{(l)4}}{n^{2}}\right\\}\geq
t\right]$ $\displaystyle=\exp(-C\min\\{nt,n\sqrt{t}\\}),$

where $Y^{(l)}$ is a sub-Gaussian random variable with the same survival
function as in Eq. 19, and therefore,
$(\widehat{\Sigma}_{ij}^{(l)}-\Sigma_{ij}^{(l)})^{2}\overset{d}{\leq}C\max\left\\{\frac{Y^{(l)2}}{n},\frac{Y^{(l)4}}{n^{2}}\right\\}+\mu\leq
C\frac{Y^{(l)2}}{n}+C\frac{Y^{(l)4}}{n^{2}}+\mu$. Because every term in both
hand sides is almost surely positive for every $l\in[m]$,

$\displaystyle\mathbb{P}\left[\sum_{l=1}^{m}(\widehat{\Sigma}_{ij}^{(l)}-\Sigma_{ij}^{(l)})^{2}\geq
t+C\frac{m}{n}\right]$
$\displaystyle\leq\mathbb{P}\left[C\sum_{l=1}^{m}\frac{Y^{(l)2}}{n}+C\sum_{l=1}^{m}\frac{Y^{(l)4}}{n^{2}}\geq
t\right]$
$\displaystyle\leq\mathbb{P}\left[C\sum_{l=1}^{m}\frac{Y^{(l)2}}{n}\geq\frac{t}{2}\right]+\mathbb{P}\left[C\sum_{l=1}^{m}\frac{Y^{(l)4}}{n^{2}}\geq\frac{t}{2}\right].$

Based on our observation that $\lVert Y^{(l)2}\rVert_{\psi_{1}},\lVert
Y^{(l)4}\rVert_{\psi_{1/2}}\leq C$, Theorem 3.1 of [12] derives

$\left\lVert\sum_{l=1}^{m}\frac{Y^{(l)2}}{n}\right\rVert_{\phi_{1,\frac{1}{\sqrt{m}}}}\leq
C\frac{\sqrt{m}}{n},\quad\left\lVert\sum_{l=1}^{m}\frac{Y^{(l)4}}{n^{2}}\right\rVert_{\phi_{\frac{1}{2},\frac{1}{\sqrt{m}}}}\leq
C\frac{\sqrt{m}}{n^{2}}.$

In sum,

$\displaystyle\mathbb{P}\left[\sum_{l=1}^{m}(\widehat{\Sigma}_{ij}^{(l)}-\Sigma_{ij}^{(l)})^{2}\geq
t+C\frac{m}{n}\right]\leq\mathbb{P}\left[C\sum_{l=1}^{m}\frac{Y^{(l)2}}{n}\geq\frac{t}{2}\right]+\mathbb{P}\left[C\sum_{l=1}^{m}\frac{Y^{(l)4}}{n^{2}}\geq\frac{t}{2}\right]$
(24) $\displaystyle\leq
2\exp\left(-C\min\left\\{\frac{n^{2}t^{2}}{m},nt\right\\}\right)+2\exp\left(-C\min\left\\{\frac{n^{4}t^{2}}{m},n\sqrt{t}\right\\}\right)$
$\displaystyle\leq
4\exp\left(-C\min\left\\{\frac{n^{2}t^{2}}{m},nt,n\sqrt{t}\right\\}\right),$

and therefore,
$\sum_{l=1}^{m}(\widehat{\Sigma}_{ij}^{(l)}-\Sigma_{ij}^{(l)})^{2}\leq
C\left(\frac{m+\sqrt{m\nu}+\nu}{n}+\frac{\nu^{2}}{n^{2}}\right)$ with
probability at least $1-e^{-\nu}$. Thus, if $\frac{\log q}{n}\leq C$, for
$\nu=C\log(qn)$,

$\sup_{i,j\in[q]}\sum_{l=1}^{m}(\widehat{\Sigma}_{ij}^{(l)}-\Sigma_{ij}^{(l)})^{2}\leq
C\frac{m+\log(qn)}{n},$

with probability at least $1-\frac{1}{n}$.

Now we prove the lower bound. Suppose that
$\mathbb{P}[\lvert\widehat{\Sigma}_{ij}^{(l)}-\Sigma_{ij}^{(l)}\rvert\geq
t]{\geq}\exp(-C\min\\{nt,nt^{2}\\})$ for some $C>0$. Then,

$\displaystyle\sum_{l=1}^{m}(\widehat{\Sigma}_{ij}^{(l)}-\Sigma_{ij}^{(l)})^{2}$
$\displaystyle\overset{d}{\geq}C\sum_{l=1}^{m}\max\left\\{\frac{Y^{(l)2}}{n},\frac{Y^{(l)4}}{n^{2}}\right\\}$
$\displaystyle\geq
C\max\left\\{\sum_{l=1}^{m}\frac{Y^{(l)2}}{n},\sum_{l=1}^{m}\frac{Y^{(l)4}}{n^{2}}\right\\},$

which means that

$\displaystyle\mathbb{P}\left[\sum_{l=1}^{m}(\widehat{\Sigma}_{ij}^{(l)}-\Sigma_{ij}^{(l)})^{2}\geq
t\right]$
$\displaystyle\geq\mathbb{P}\left[C\max\left\\{\sum_{l=1}^{m}\frac{Y^{(l)2}}{n},\sum_{l=1}^{m}\frac{Y^{(l)4}}{n^{2}}\right\\}\geq
t\right]$
$\displaystyle\geq\max\left\\{\mathbb{P}\left[C\sum_{l=1}^{m}\frac{Y^{(l)2}}{n}\geq
t\right],\mathbb{P}\left[C\sum_{l=1}^{m}\frac{Y^{(l)4}}{n^{2}}\geq
t\right]\right\\}.$

Applying Theorem 2.4 to each of
$\mathbb{P}\left[C\sum_{l=1}^{m}\frac{Y^{(l)2}}{n}\geq t\right]$ and
$\mathbb{P}\left[C\sum_{l=1}^{m}\frac{Y^{(l)4}}{n^{2}}\geq t\right]$, we prove
Eq. 17.

###  5.4 Proof of Lemmas

######  Proof 5.11 (Proof of Lemma 5.2).

Because $\alpha\leq 1$, for $y\in\mathbb{R}$, $L_{i}y^{2/\alpha}\geq
y\Longleftrightarrow y\geq L_{i}^{-\frac{\alpha}{2-\alpha}}$. Hence,

$\mathbb{P}[\max\\{\lvert Y_{i}\rvert,L_{i}\lvert
Y_{i}\rvert^{2/\alpha}\\}\geq t]=\mathbb{P}[t\leq\lvert Y_{i}\rvert\leq
L_{i}^{-\frac{\alpha}{2-\alpha}}]+\mathbb{P}[L_{i}\lvert
Y_{i}\rvert^{2/\alpha}\geq t,\lvert
Y_{i}\rvert>L_{i}^{-\frac{\alpha}{2-\alpha}}].$

If $t\leq L_{i}^{-\frac{\alpha}{2-\alpha}}$, then

$\displaystyle\mathbb{P}[L_{i}\lvert Y_{i}\rvert^{2/\alpha}\geq t,\lvert
Y_{i}\rvert>L_{i}^{-\frac{\alpha}{2-\alpha}}]$
$\displaystyle=\mathbb{P}[\lvert Y_{i}\rvert\geq(t/L_{i})^{\alpha/2},\lvert
Y_{i}\rvert>L_{i}^{-\frac{\alpha}{2-\alpha}}]$
$\displaystyle=\mathbb{P}[\lvert
Y_{i}\rvert\geq\max\\{(t/L_{i})^{\alpha/2},L_{i}^{-\frac{\alpha}{2-\alpha}}\\}]$
$\displaystyle=\mathbb{P}[\lvert Y_{i}\rvert\geq
L_{i}^{-\frac{\alpha}{2-\alpha}}],$ $\displaystyle\mathbb{P}[\max\\{\lvert
Y_{i}\rvert,L_{i}\lvert Y_{i}\rvert^{2/\alpha}\\}\geq t]$
$\displaystyle=\mathbb{P}[t\leq\lvert Y_{i}\rvert\leq
L_{i}^{-\frac{\alpha}{2-\alpha}}]+\mathbb{P}[\lvert Y_{i}\rvert\geq
L_{i}^{-\frac{\alpha}{2-\alpha}}]$ $\displaystyle=\mathbb{P}[\lvert
Y_{i}\rvert\geq t]=e^{-t^{2}}.$

If $t>L_{i}^{-\frac{\alpha}{2-\alpha}}$, then $\mathbb{P}[\lvert
Y_{i}\rvert\geq t,\lvert Y_{i}\rvert\leq L_{i}^{-\frac{\alpha}{2-\alpha}}]=0$,

$\displaystyle\mathbb{P}[L_{i}\lvert Y_{i}\rvert^{2/\alpha}\geq t,\lvert
Y_{i}\rvert>L_{i}^{-\frac{\alpha}{2-\alpha}}]$
$\displaystyle=\mathbb{P}[\lvert
Y_{i}\rvert\geq\max\\{(t/L_{i})^{\alpha/2},L_{i}^{-\frac{\alpha}{2-\alpha}}\\}]$
$\displaystyle=\mathbb{P}[\lvert Y_{i}\rvert\geq(t/L_{i})^{\alpha/2}],$
$\displaystyle\mathbb{P}[\max\\{\lvert Y_{i}\rvert,L_{i}\lvert
Y_{i}\rvert^{2/\alpha}\\}\geq t]$ $\displaystyle=\mathbb{P}[t\leq\lvert
Y_{i}\rvert\leq L_{i}^{-\frac{\alpha}{2-\alpha}}]+\mathbb{P}[\lvert
Y_{i}\rvert\geq L_{i}^{-\frac{\alpha}{2-\alpha}}]$
$\displaystyle=\mathbb{P}[\lvert
Y_{i}\rvert\geq(t/L_{i})^{\alpha/2}]=e^{-(t/L_{i})^{\alpha}}.$

In sum,

$\displaystyle\mathbb{P}[\max\\{\lvert Y_{i}\rvert,L_{i}\lvert
Y_{i}\rvert^{2/\alpha}\\}\geq t]$
$\displaystyle=\begin{cases}e^{-t^{2}},&t\leq
L_{i}^{-\frac{\alpha}{2-\alpha}}\\\
e^{-(t/L_{i})^{\alpha}},&t>L_{i}^{-\frac{\alpha}{2-\alpha}}\end{cases}$
$\displaystyle=\exp(-\min\\{t^{2},(t/L_{i})^{\alpha}\\}),$

which is the same as the survival function of $X_{i}$.

######  Proof 5.12 (Proof of Lemma 5.4).

By Karamata’s inequality (Proposition B.4.a in Chapter 16 of [16]) and the
definition of $\kappa_{i}$,

$\displaystyle\mathbb{E}\left[\left\lvert{\textstyle\sum}_{i>p}a_{i}Z_{i}\right\rvert^{p}\right]^{1/p}$
$\displaystyle\leq\mathbb{E}\left[\left\lvert{\textstyle\sum}_{i>p}\kappa_{i}a_{i}\epsilon_{i}Y_{i}^{2}\right\rvert^{p}\right]^{1/p}$
(25)
$\displaystyle\leq\max\\{\kappa_{i}:i\in[n]\\}\mathbb{E}\left[\left\lvert{\textstyle\sum}_{i>p}a_{i}\epsilon_{i}Y_{i}^{2}\right\rvert^{p}\right]^{1/p}$
$\displaystyle\leq\max\\{\kappa_{i}:i\in[n]\\}\max\\{\sqrt{p}({\textstyle\sum}_{i>p}a_{i}^{2})^{1/2},pa_{\lceil
p\rceil}\\},$

where $\epsilon_{i}$ is a Rademacher random variable, and $\lceil p\rceil$ is
the smallest integer larger than $p$. If we assume that
$\inf\Big{\\{}t>0:{\textstyle\sum}_{i\leq p}N_{i}^{*}(pa_{i}/t)\leq
p\Big{\\}}+\sqrt{p}\big{(}{\textstyle\sum}_{i>p}a_{i}^{2}\big{)}^{1/2}\leq 1$,
then

$\inf\Big{\\{}t>0:{\textstyle\sum}_{i\leq p}N_{i}^{*}(pa_{i}/t)\leq
p\Big{\\}}\leq 1,$

and by recalling the definition of $N_{i}^{*}(t)=\sup\\{st-N_{i}(s):s>0\\}$,

$1\geq\frac{1}{p}{\textstyle\sum}_{i\leq p}N_{i}^{*}(pa_{i})\geq\frac{\lfloor
p\rfloor}{p}N_{\lfloor p\rfloor}^{*}(pa_{\lfloor p\rfloor})\geq\lfloor
p\rfloor\left(a_{\lfloor p\rfloor}-\frac{N_{\lfloor
p\rfloor}(1)}{p}\right)=\lfloor p\rfloor\left(a_{\lfloor
p\rfloor}-\frac{1}{p}\right).$

Hence, $a_{i}\leq\frac{2}{p}$ for all $i>p$, and therefore Eq. 25 results in

$\displaystyle\mathbb{E}\left[\left\lvert{\textstyle\sum}_{i>p}a_{i}Z_{i}\right\rvert^{p}\right]^{1/p}\leq
2\max\\{\kappa_{i}:i\in[n]\\}.$

By the homogeneity of the $p$-norm,

$\displaystyle\mathbb{E}\left[\left\lvert{\textstyle\sum}_{i>p}a_{i}Z_{i}\right\rvert^{p}\right]^{1/p}$
$\displaystyle\leq
2\max\\{\kappa_{i}:i\in[n]\\}\left(\inf\Big{\\{}t>0:{\textstyle\sum}_{i\leq
p}N_{i}^{*}(pa_{i}/t)\leq
p\Big{\\}}+\sqrt{p}\big{(}{\textstyle\sum}_{i>p}a_{i}^{2}\big{)}^{1/2}\right).$

On the other hand, because $Z_{i}\sim\epsilon_{i}N_{i}^{-1}(\lvert
Y_{i}^{2}\rvert)$,

$\mathbb{E}\left[{\textstyle\sum}_{i\leq
p}a_{i}Z_{i}\right]=\mathbb{E}\left[{\textstyle\sum}_{i\leq
p}a_{i}\epsilon_{i}N_{i}^{-1}(\lvert Y_{i}^{2}\rvert)\right].$

Due to the definition of $N_{i}^{*}$, for any $t,s>0$, $ptN_{i}^{-1}(s)\leq
N_{i}^{*}(pt)+s$. If $\inf\Big{\\{}t>0:{\textstyle\sum}_{i\leq
p}N_{i}^{*}(pa_{i}/t)\leq p\Big{\\}}\leq 1$, by the contraction principle
(Lemma 4.6, [14]),

$\displaystyle\mathbb{E}\left[\left\lvert{\textstyle\sum}_{i\leq
p}a_{i}Z_{i}\right\rvert^{p}\right]^{1/p}$
$\displaystyle\leq\mathbb{E}\left[\left\lvert\frac{1}{p}{\textstyle\sum}_{i\leq
p}\epsilon_{i}(N_{i}^{*}(pa_{i})+\lvert
Y_{i}^{2}\rvert)\right\rvert^{p}\right]^{1/p}$
$\displaystyle\leq\frac{1}{p}\left({\textstyle\sum}_{i\leq
p}N_{i}^{*}(pa_{i})+\mathbb{E}\left[\left\lvert{\textstyle\sum}_{i\leq
p}\lvert Y_{i}^{2}\rvert\right\rvert^{p}\right]^{1/p}\right)$
$\displaystyle\leq 3.$

By the homogeneity of the $p$-norm, we get
$\mathbb{E}\left[\left\lvert{\textstyle\sum}_{i\leq
p}a_{i}Z_{i}\right\rvert^{p}\right]^{1/p}\leq
3\inf\Big{\\{}t>0:{\textstyle\sum}_{i\leq p}N_{i}^{*}(pa_{i}/t)\leq
p\Big{\\}}$. This proves the right-hand side.

For the left-hand side, let $b_{1},\dots,b_{n}\geq 0$ be such that
${\textstyle\sum}_{i=1}^{n}N_{i}(b_{i})=p$ and

${\textstyle\sum}_{i\leq p}a_{i}b_{i}=\sup\left\\{{\textstyle\sum}_{i\leq
p}a_{i}b_{i}:{\textstyle\sum}_{i\leq p}N_{i}(b_{i})\leq p\right\\}.$

Then,

$\displaystyle\mathbb{E}\left[\left\lvert{\textstyle\sum}_{i=1}^{n}a_{i}Z_{i}\right\rvert^{p}\right]^{1/p}$
$\displaystyle\geq\mathbb{E}\left[\left\lvert{\textstyle\sum}_{i\leq
p}a_{i}Z_{i}\right\rvert^{p}\right]^{1/p}$
$\displaystyle\geq{\textstyle\sum}_{i\leq
p}a_{i}b_{i}\cdot{\textstyle\prod}_{i\leq p}\mathbb{P}[Z_{i}\geq b_{i}]^{1/p}$
$\displaystyle={\textstyle\sum}_{i\leq p}a_{i}b_{i}\cdot 2^{-\lfloor
p\rfloor/p}\exp\left(-\frac{1}{p}{\textstyle\sum}_{i\leq
s}N_{i}(b_{i})\right)$ $\displaystyle=\frac{2^{-\lfloor
p\rfloor/p}}{e}\sup\left\\{{\textstyle\sum}_{i\leq
p}a_{i}b_{i}:{\textstyle\sum}_{i\leq p}N_{i}(b_{i})\leq p\right\\}.$

Because the duality between $N_{i}$ and $N_{i}^{*}$ implies

$\sup\left\\{{\textstyle\sum}_{i\leq p}a_{i}b_{i}:{\textstyle\sum}_{i\leq
p}N_{i}(b_{i})\leq p\right\\}\geq\inf\Big{\\{}t>0:{\textstyle\sum}_{i\leq
p}N_{i}^{*}(pa_{i}/t)\leq p\Big{\\}},$

the moment ${\textstyle\sum}_{i=1}^{n}a_{i}Z_{i}$ satisfies

$\displaystyle\mathbb{E}\left[\left\lvert{\textstyle\sum}_{i=1}^{n}a_{i}Z_{i}\right\rvert^{p}\right]^{1/p}\geq\frac{1}{2e}\inf\Big{\\{}t>0:{\textstyle\sum}_{i\leq
p}N_{i}^{*}(pa_{i}/t)\leq p\Big{\\}}.$

By Jensen’s inequality,

$\mathbb{E}\left[\left\lvert{\textstyle\sum}_{i=1}^{n}a_{i}Z_{i}\right\rvert^{p}\right]^{1/p}\geq\mathbb{E}\left[\left\lvert{\textstyle\sum}_{i=1}^{n}a_{i}\epsilon_{i}\mathbb{E}[\lvert
Z_{i}\rvert]\right\rvert^{p}\right]^{1/p}\geq\min\\{\mathbb{E}[\lvert
Z_{i}\rvert]:i\in[n]\\}\sqrt{p}\left({\textstyle\sum}_{i>p}a_{i}^{2}\right)^{1/2}.$

######  Proof 5.13 (Proof of Lemma 5.5).

Because $\varphi_{p}$ is symmetric, positive in $\mathbb{R}$, and monotonic in
$[0,\infty)$,

$\displaystyle\phi_{p}\left(\frac{Z_{i}}{\eta}\right)$
$\displaystyle=\int_{0}^{\infty}\mathbb{P}\left[\varphi_{p}\left(\frac{Z_{i}}{\eta}\right)\geq
x\right]dx$
$\displaystyle=1+\int_{0}^{\infty}\frac{1}{\eta}\varphi_{p}^{\prime}\left(\frac{t}{\eta}\right)\mathbb{P}[\lvert
Z_{i}\rvert\geq t]dt$
$\displaystyle=1+\int_{0}^{\infty}\frac{1}{\eta}\varphi_{p}^{\prime}\left(\frac{t}{\eta}\right)\max\\{\mathbb{P}[\lvert
Y_{i}\rvert\geq t],\mathbb{P}[L_{i}\lvert Y_{i}\rvert^{2/\alpha}\geq t]\\}dt.$

For the lowerbound,

$\displaystyle\phi_{p}\left(\frac{Z_{i}}{\eta}\right)$
$\displaystyle=1+\int_{0}^{\infty}\frac{1}{\eta}\varphi_{p}^{\prime}\left(\frac{t}{\eta}\right)\max\\{\mathbb{P}[\lvert
Y_{i}\rvert\geq t],\mathbb{P}[L_{i}\lvert Y_{i}\rvert^{2/\alpha}\geq t]\\}dt$
(26) $\displaystyle\geq
1+\max\left\\{\int_{0}^{\infty}\frac{1}{\eta}\varphi_{p}^{\prime}\left(\frac{t}{\eta}\right)\mathbb{P}[\lvert
Y_{i}\rvert\geq
t]dt,\int_{0}^{\infty}\frac{1}{\eta}\varphi_{p}^{\prime}\left(\frac{t}{\eta}\right)\mathbb{P}[L_{i}\lvert
Y_{i}\rvert^{2/\alpha}\geq t]dt\right\\}$
$\displaystyle=\max\left\\{\phi_{p}\left(\frac{Y_{i}}{\eta}\right),\phi_{p}\left(\frac{L_{i}Y_{i}^{2/\alpha}}{\eta}\right)\right\\}.$

For the upperbound,

$\displaystyle\phi_{p}\left(\frac{Z_{i}}{\eta}\right)$
$\displaystyle=1+\int_{0}^{\infty}\frac{1}{\eta}\varphi_{p}^{\prime}\left(\frac{t}{\eta}\right)\max\\{\mathbb{P}[\lvert
Y_{i}\rvert\geq t],\mathbb{P}[L_{i}\lvert Y_{i}\rvert^{2/\alpha}\geq t]\\}dt$
(27) $\displaystyle\leq
1+\int_{0}^{\infty}\frac{1}{\eta}\varphi_{p}^{\prime}\left(\frac{t}{\eta}\right)(\mathbb{P}[\lvert
Y_{i}\rvert\geq t]+\mathbb{P}[L_{i}\lvert Y_{i}\rvert^{2/\alpha}\geq t])dt$
$\displaystyle\leq
1+2\max\left\\{\int_{0}^{\infty}\frac{1}{\eta}\varphi_{p}^{\prime}\left(\frac{t}{\eta}\right)\mathbb{P}[\lvert
Y_{i}\rvert\geq
t]dt,\int_{0}^{\infty}\frac{1}{\eta}\varphi_{p}^{\prime}\left(\frac{t}{\eta}\right)\mathbb{P}[L_{i}\lvert
Y_{i}\rvert^{2/\alpha}\geq t]dt\right\\}$
$\displaystyle=2\max\left\\{\phi_{p}\left(\frac{Y_{i}}{\eta}\right),\phi_{p}\left(\frac{L_{i}Y_{i}^{2/\alpha}}{\eta}\right)\right\\}-1$
$\displaystyle\leq\exp\left(2\max\left\\{\log\phi_{p}\left(\frac{Y_{i}}{\eta}\right),\log\phi_{p}\left(\frac{L_{i}Y_{i}^{2/\alpha}}{\eta}\right)\right\\}\right).$

Because $Y_{i}$ is a symmetric random variable with a logarithmically convex
tail, we follow the example in Section 3.2 of [13] to obtain

$p\min\left\\{1,\frac{p}{\eta^{2}e^{6}}\right\\}\leq\log\phi_{p}\left(\frac{Y_{i}}{\eta}\right)\leq\frac{16p^{2}}{\eta^{2}}.$
(28)

On the other hand, because $Y_{i}^{2/\alpha}$ has a logarithmically concave
tail, the example in Section 3.3 of [13] gives

$\displaystyle
p\min\left\\{1,\max\left\\{\frac{L_{i}^{p}}{e^{2p}\eta^{p}}\lVert
Y_{i}^{2/\alpha}\rVert_{p}^{p},\frac{pL_{i}^{2}}{e^{4}\eta^{2}}\lVert
Y_{i}^{2/\alpha}\rVert_{2}^{2}\right\\}\right\\}$ (29)
$\displaystyle\leq\log\phi_{p}\left(\frac{L_{i}Y_{i}^{2/\alpha}}{\eta}\right)\leq
p\max\left\\{\frac{e^{2p}L_{i}^{p}}{\eta^{p}}\lVert
Y_{i}^{2/\alpha}\rVert_{p}^{p},\frac{e^{4}pL_{i}^{2}}{\eta^{2}}\lVert
Y_{i}^{2/\alpha}\rVert_{2}^{2}\right\\}.$

The moments of $Y_{i}$ are given by the integration by parts,

$\displaystyle\lVert
Y_{i}\rVert_{p}^{p}=\int_{0}^{\infty}pt^{p-1}\exp(-t^{2})dt=\frac{p}{2}\int_{0}^{\infty}s^{\frac{p}{2}-1}\exp(-s)ds=\Gamma\left(\frac{p}{2}+1\right),$

where Stirling’s approximation derives

$\lVert
Y_{i}\rVert_{p}^{p}=\Gamma\left(\frac{p}{2}+1\right)=\sqrt{\frac{2\pi}{p}}\left(\frac{p}{2e}\right)^{\frac{p}{2}}\left(1+O\left(\frac{2}{p}\right)\right).$
(30)

Plugging Eqs. 28, 29 and 30 to Eqs. 26 and 27, we obtain

$\displaystyle
p\min\left\\{1,\max\left\\{\frac{p}{\eta^{2}e^{6}},\frac{pL_{i}^{2}}{e^{4}\eta^{2}}\sqrt{\frac{\alpha\pi}{2}}\left(\frac{2}{\alpha
e}\right)^{2/\alpha},\frac{L_{i}^{p}}{e^{2p}\eta^{p}}\sqrt{\frac{\alpha\pi}{p}}\left(\frac{p}{\alpha
e}\right)^{p/\alpha}\right\\}\right\\}$
$\displaystyle\leq\log\phi_{p}\left(\frac{Z_{i}}{\eta}\right)\leq
2p\max\left\\{\frac{16p}{\eta^{2}},\frac{e^{4}pL_{i}^{2}}{\eta^{2}}\sqrt{\frac{\alpha\pi}{2}}\left(\frac{2}{\alpha
e}\right)^{2/\alpha},\frac{e^{2p}L_{i}^{p}}{\eta^{p}}\sqrt{\frac{\alpha\pi}{p}}\left(\frac{p}{\alpha
e}\right)^{p/\alpha}\right\\}.$

##  6 GBO norm of $Z_{i}$

Suppose that a symmetric random variable $Z$’s survival function is

$\mathbb{P}[\lvert Z\rvert\geq t]=\exp(-\min\\{t^{2},(t/L)^{\alpha}\\}).$

Then, for $\eta>0$,

$\displaystyle\mathbb{E}\left[\phi_{\alpha,L}\left(\frac{\lvert
Z\rvert}{\eta}\right)\right]$
$\displaystyle=\int_{0}^{\infty}\mathbb{P}\left[\phi_{\alpha,L}\left(\frac{\lvert
Z\rvert}{\eta}\right)\geq t\right]dt$
$\displaystyle=\int_{1}^{\infty}\mathbb{P}\left[\exp\left(\min\left\\{\frac{\lvert
Z\rvert^{2}}{\eta^{2}},\left(\frac{\lvert Z\rvert}{\eta
L}\right)^{\alpha}\right\\}\right)\geq t\right]dt$
$\displaystyle=\int_{0}^{\infty}\mathbb{P}\left[\min\left\\{\frac{\lvert
Z\rvert^{2}}{\eta^{2}},\left(\frac{\lvert Z\rvert}{\eta
L}\right)^{\alpha}\right\\}\geq s\right]e^{s}ds$
$\displaystyle=\int_{0}^{L^{2\alpha/(\alpha-2)}}\mathbb{P}\left[\frac{\lvert
Z\rvert^{2}}{\eta^{2}}\geq
s\right]e^{s}ds+\int_{L^{2\alpha/(\alpha-2)}}^{\infty}\mathbb{P}\left[\left(\frac{\lvert
Z\rvert}{\eta L}\right)^{\alpha}\geq s\right]e^{s}ds$
$\displaystyle=\int_{0}^{L^{2\alpha/(\alpha-2)}}\mathbb{P}\left[\lvert
Z\rvert\geq\eta\sqrt{s}\right]e^{s}ds+\int_{L^{2\alpha/(\alpha-2)}}^{\infty}\mathbb{P}\left[\lvert
Z\rvert\geq\eta Ls^{1/\alpha}\right]e^{s}ds.$

Because $\mathbb{P}[\lvert Z\rvert\geq
t]=\max\\{\exp(-t^{2}),\exp(-(t/L)^{\alpha})\\}$,

$\displaystyle\mathbb{E}\left[\phi_{\alpha,L}\left(\frac{\lvert
Z\rvert}{\eta}\right)\right]$
$\displaystyle=\int_{0}^{L^{2\alpha/(\alpha-2)}}\mathbb{P}\left[\lvert
Z\rvert\geq\eta\sqrt{s}\right]e^{s}ds+\int_{L^{2\alpha/(\alpha-2)}}^{\infty}\mathbb{P}\left[\lvert
Z\rvert\geq\eta Ls^{1/\alpha}\right]e^{s}ds$
$\displaystyle\geq\int_{0}^{L^{2\alpha/(\alpha-2)}}\exp(-\eta^{2}s)e^{s}ds+\int_{L^{2\alpha/(\alpha-2)}}^{\infty}\exp(-\eta^{\alpha}s)e^{s}ds$
$\displaystyle\geq\int_{0}^{\infty}\exp(-\max\\{\eta^{2},\eta^{\alpha}\\}s)e^{s}ds$
$\displaystyle\geq\begin{cases}\infty,&0<\eta\leq 1,\\\
\frac{1}{\max\\{\eta^{2},\eta^{\alpha}\\}-1},&1<\eta.\end{cases}$

Hence,

$\lVert
Z\rVert_{\phi_{\alpha,L}}=\inf\\{\eta>0:\mathbb{E}[\phi_{\alpha,L}(\lvert
Z\rvert/\eta)]\leq 1\\}\geq\min\\{\sqrt{2},2^{1/\alpha}\\}.$

On the other hand, based on Lemma 5.2,

$\displaystyle\mathbb{E}\left[\phi_{\alpha,L}\left(\frac{\lvert
Z\rvert}{\eta}\right)\right]$
$\displaystyle\leq\mathbb{E}\left[\phi_{\alpha,L}\left(\frac{\max\\{\lvert
Y\rvert,L\lvert Y\rvert^{2/\alpha}\\}}{\eta}\right)\right]$
$\displaystyle\leq\mathbb{E}\left[\phi_{\alpha,L}\left(\frac{\lvert
Y\rvert}{\eta}\right)\right]+\mathbb{E}\left[\phi_{\alpha,L}\left(\frac{L\lvert
Y\rvert^{2/\alpha}}{\eta}\right)\right],$

where $Y$ is a symmetric sub-Gaussian random variable satisfying
$\mathbb{P}[\lvert Y\rvert\geq t]=e^{-t^{2}}$. Because
$\phi_{\alpha,L}(t)=\exp(\min\\{t^{2},(t/L)^{\alpha}\\})-1=\min\\{\exp(t^{2})-1,\exp((t/L)^{\alpha})-1\\}=\min\\{\psi_{2}(t),\psi_{\alpha}(t/L)\\}$,

$\displaystyle\mathbb{E}\left[\phi_{\alpha,L}\left(\frac{\lvert
Z\rvert}{\eta}\right)\right]$
$\displaystyle\leq\mathbb{E}\left[\phi_{\alpha,L}\left(\frac{\lvert
Y\rvert}{\eta}\right)\right]+\mathbb{E}\left[\phi_{\alpha,L}\left(\frac{L\lvert
Y\rvert^{2/\alpha}}{\eta}\right)\right]$
$\displaystyle\leq\mathbb{E}\left[\psi_{2}\left(\frac{\lvert
Y\rvert}{\eta}\right)\right]+\mathbb{E}\left[\psi_{\alpha}\left(\frac{\lvert
Y\rvert^{2/\alpha}}{\eta}\right)\right].$

For any $\alpha>0$,

$\displaystyle\mathbb{E}[\psi_{\alpha}(Y^{2/\alpha}/\eta)]$
$\displaystyle=\mathbb{E}[\exp(Y_{i}^{2}/\eta^{\alpha})-1]$
$\displaystyle=\int_{0}^{\infty}\mathbb{P}[\exp(Y^{2}/\eta^{\alpha})-1\geq
s]ds$
$\displaystyle=\int_{0}^{\infty}\mathbb{P}[Y\geq\eta^{\alpha/2}\sqrt{\log(1+s)}]ds$
$\displaystyle=\int_{0}^{\infty}\frac{2t}{\eta^{\alpha}}\mathbb{P}[Y\geq
t]\exp(t^{2}/\eta^{\alpha})dt$
$\displaystyle=\int_{0}^{\infty}\frac{2t}{\eta^{\alpha}}\exp(-(1-1/\eta^{\alpha})t^{2})dt$
$\displaystyle=\frac{1}{\eta^{\alpha}},$

where $s=\exp(t^{2}/\eta^{\alpha})-1$. Therefore,

$\mathbb{E}\left[\phi_{\alpha,L}\left(\frac{\lvert
Z\rvert}{\eta}\right)\right]\leq\frac{1}{\eta^{2}-1}+\frac{1}{\eta^{\alpha}-1}\leq\frac{2}{\min\\{\eta^{2},\eta^{\alpha}\\}-1},$

and

$\lVert
Z\rVert_{\phi_{\alpha,L}}=\inf\\{\eta>0:\mathbb{E}[\phi_{\alpha,L}(\lvert
Z\rvert/\eta)]\leq 1\\}\leq\max\\{\sqrt{3},3^{1/\alpha}\\}.$

## References

  * [1] M. Bakhshizadeh, _Exponential tail bounds and large deviation principle for heavy-tailed u-statistics_ , arXiv preprint arXiv:2301.11563 (2023).
  * [2] G. Bennett, _Upper bounds on the moments and probability inequalities for the sum of independent, bounded random variables_ , Biometrika 52 (1965), 559–569. 205358
  * [3] V. Bentkus, _On Hoeffding’s inequalities_ , Ann. Probab. 32 (2004), no. 2, 1650–1673. 2060313
  * [4] R. Bogucki, _Suprema of canonical Weibull processes_ , Statist. Probab. Lett. 107 (2015), 253–263. 3412784
  * [5] H. Bong, _Discovery of functional predictivity across brain regions from local field potentials_ , Ph.D. thesis, Carnegie Mellon University, 2022.
  * [6] V. H. de la Peña, _From dependence to complete independence: the decoupling approach_ , Fourth Symposium on Probability Theory and Stochastic Processes (Spanish) (Guanajuato, 1996), Aportaciones Mat. Notas Investigación, vol. 12, Soc. Mat. Mexicana, México, 1996, pp. 37–48. 1653754
  * [7] E. Giné, R. Latała, and J. Zinn, _Exponential and moment inequalities for $U$-statistics_, High dimensional probability, II (Seattle, WA, 1999), Progr. Probab., vol. 47, Birkhäuser Boston, Boston, MA, 2000, pp. 13–38. 1857312
  * [8] E. D. Gluskin and S. Kwapień, _Tail and moment estimates for sums of independent random variables with logarithmically concave tails_ , Studia Math. 114 (1995), no. 3, 303–309. 1338834
  * [9] P. Hitczenko and S. J. Montgomery-Smith, _A note on sums of independent random variables_ , Advances in stochastic inequalities (Atlanta, GA, 1997), Contemp. Math., vol. 234, Amer. Math. Soc., Providence, RI, 1999, pp. 69–73. 1694763
  * [10] P. Hitczenko, S. J. Montgomery-Smith, and K. Oleszkiewicz, _Moment inequalities for sums of certain independent symmetric random variables_ , Studia Math. 123 (1997), no. 1, 15–42. 1438303
  * [11] W. Hoeffding, _Probability inequalities for sums of bounded random variables_ , J. Amer. Statist. Assoc. 58 (1963), 13–30. 144363
  * [12] A. K. Kuchibhotla and A. Chakrabortty, _Moving beyond sub-Gaussianity in high-dimensional statistics: applications in covariance estimation and linear regression_ , Inf. Inference 11 (2022), no. 4, 1389–1456. 4526326
  * [13] R. Latała, _Estimation of moments of sums of independent real random variables_ , Ann. Probab. 25 (1997), no. 3, 1502–1513. 1457628
  * [14] M. Ledoux and M. Talagrand, _Probability in Banach spaces_ , Classics in Mathematics, Springer-Verlag, Berlin, 2011, Isoperimetry and processes, Reprint of the 1991 edition. 2814399
  * [15] P. Major, _Tail behaviour of multiple random integrals and $U$-statistics_, Probab. Surv. 2 (2005), 448–505. 2203678
  * [16] A. W. Marshall, I. Olkin, and B. C. Arnold, _Inequalities: theory of majorization and its applications_ , second ed., Springer Series in Statistics, Springer, New York, 2011. 2759813
  * [17] R. E. A. C. Paley and A. Zygmund, _A note on analytic functions in the unit circle_ , Mathematical Proceedings of the Cambridge Philosophical Society, vol. 28, Cambridge University Press, 1932, pp. 266–272.
  * [18] R. Vershynin, _High-dimensional probability_ , Cambridge Series in Statistical and Probabilistic Mathematics, vol. 47, Cambridge University Press, Cambridge, 2018, An introduction with applications in data science, With a foreword by Sara van de Geer. 3837109
  * [19] M. J. Wainwright, _High-dimensional statistics_ , Cambridge Series in Statistical and Probabilistic Mathematics, vol. 48, Cambridge University Press, Cambridge, 2019, A non-asymptotic viewpoint. 3967104
  * [20] J. A. Wellner, _The Bennett-Orlicz norm_ , Sankhya A 79 (2017), no. 2, 355–383. 3707425
  * [21] A. R. Zhang and Y. Zhou, _On the non-asymptotic and sharp lower tail bounds of random variables_ , Stat 9 (2020), e314, 11. 4193419


# Norm Inequalities Related to Heinz and Logarithmic Means

Guanghua Shi
School of Mathematical Sciences  Yangzhou University  Yangzhou  Jiangsu  China
<EMAIL_ADDRESS>
###### Abstract

In this paper, we got some refinements of the norm inequalities related to the
Heinz mean and logarithmic mean.

AMS classification: 47A63; 94A17

Keywords: Heinz mean, Logarithmic mean, Positive function, Unitarily invariant
norm

##  1 Introduction

There are several means that interpolate between the geometric and arithmetic
means. For instance, the Heinz mean $H_{t}(a,b),$ defined by

$H_{t}(a,b)=\frac{a^{1-t}b^{t}+a^{t}b^{1-t}}{2}\quad\mbox{for}\quad 0\leq
t\leq 1.$

In 1993, Bhatia-Davis [2] obtained that if $A,B$ and $X$ are $n\times n$
matrices with $A,B$ positive semidefinite, then for every unitarily invariant
norm
${\left|\kern-1.07639pt\left|\kern-1.07639pt\left|\cdot\right|\kern-1.07639pt\right|\kern-1.07639pt\right|},$

$\displaystyle{\left|\kern-1.07639pt\left|\kern-1.07639pt\left|A^{\frac{1}{2}}XB^{\frac{1}{2}}\right|\kern-1.07639pt\right|\kern-1.07639pt\right|}\leq\frac{1}{2}{\left|\kern-1.07639pt\left|\kern-1.07639pt\left|A^{1-t}XB^{t}+A^{t}XB^{1-t}\right|\kern-1.07639pt\right|\kern-1.07639pt\right|}\leq\frac{1}{2}{\left|\kern-1.07639pt\left|\kern-1.07639pt\left|AX+XB\right|\kern-1.07639pt\right|\kern-1.07639pt\right|}.$
(1.1)

The logarithmic mean $L(a,b),$ defined by

$L(a,b)=\frac{a-b}{\log a-\log b}=\int_{0}^{1}a^{t}b^{1-t}dt,$

also interpolates the geometric and arithmetic means. In 1999, Hiai-Kosaki [5]
proved the following inequality

$\displaystyle{\left|\kern-1.07639pt\left|\kern-1.07639pt\left|A^{\frac{1}{2}}XB^{\frac{1}{2}}\right|\kern-1.07639pt\right|\kern-1.07639pt\right|}\leq{\left|\kern-1.07639pt\left|\kern-1.07639pt\left|\int_{0}^{1}A^{v}XB^{1-v}dv\right|\kern-1.07639pt\right|\kern-1.07639pt\right|}\leq\frac{1}{2}{\left|\kern-1.07639pt\left|\kern-1.07639pt\left|AX+XB\right|\kern-1.07639pt\right|\kern-1.07639pt\right|}.$
(1.2)

Moreover, in 2006, Drissi [4] proved that the following Heinz-logarithmic
inequality

$\displaystyle{\left|\kern-1.07639pt\left|\kern-1.07639pt\left|A^{1-t}XB^{t}+A^{t}XB^{1-t}\right|\kern-1.07639pt\right|\kern-1.07639pt\right|}\leq
2{\left|\kern-1.07639pt\left|\kern-1.07639pt\left|\int_{0}^{1}A^{v}XB^{1-v}dv\right|\kern-1.07639pt\right|\kern-1.07639pt\right|}$
(1.3)

holds for $\frac{1}{4}\leq t\leq\frac{3}{4}.$

A complex-valued function $\varphi$ on $\mathbb{R}$ is said to be positive
definite if the matrix $[\varphi(x_{i}-x_{j})]$ is positive semidefinite for
all choices of real numbers $x_{1},x_{2},\ldots,x_{n},$ and $n=1,2,\ldots.$
Set $M(a,b)$ and $N(a,b)$ are two symmetric homogeneous means on
$(0,\infty)\times(0,\infty).$ $M$ is said to strongly dominate $N,$ denoted by
$M<<N,$ if and only if the matrix

$\left[\frac{M(\lambda_{i},\lambda_{j})}{N(\lambda_{i},\lambda_{j})}\right]_{i,j=1,\ldots,n}$

is positive semidefinite for any size $n$ and
$\lambda_{1},\ldots,\lambda_{n}>0.$ Drissi [4] proved that for $a,b\geq 0,$
$H_{t}(a,b)<<L(a,b)$ if and only if $\frac{1}{4}\leq t\leq\frac{3}{4}.$ In
general, the inequality $M<<N$ could be stronger than the Löwner’s order
inequality $M\leq N.$

For more operator or norm inequalities related to the Heinz mean and
logarithmic mean we refer the readers to [8, 9, 7] and the references therein.

##  2 Main results

######  Lemma 2.1.

For $\sinh x$ and $\cosh x$, we have

  1. (i)

If $|\beta|>|\alpha|>0,$ then the function $\frac{\cosh\alpha x}{\cosh\beta
x}$ is positive definite.

  2. (ii)

If $|\beta|>|\alpha|>0$ with $\alpha,\beta$ the same sign, then
$\frac{\sinh\alpha x}{\sinh\beta x}$ is positive definite.

  3. (iii)

If $\beta>0$ and $|\alpha|<\beta/2,$ then $\frac{\beta x\cosh\alpha
x}{\sinh\beta x}$ is positive definite.

###### Proof.

We follow a similar argument as in Chapter 5 of [1]. From the product
representations in p. 147-148 of [1],

$\displaystyle\frac{sinhx}{x}=\prod_{k=1}^{\infty}\left(1+\frac{x^{2}}{k^{2}\pi^{2}}\right),\qquad\cosh
x=\prod_{k=0}^{\infty}\left(1+\frac{4x^{2}}{(2k+1)^{2}\pi^{2}}\right),$ (2.1)

we have

$\displaystyle\frac{\sinh\alpha x}{\sinh\beta
x}=\frac{\alpha}{\beta}\prod_{k=1}^{\infty}\frac{1+\alpha^{2}x^{2}/k^{2}\pi^{2}}{1+\beta^{2}x^{2}/k^{2}\pi^{2}},\qquad\frac{\cosh\alpha
x}{\cosh\beta
x}=\prod_{k=0}^{\infty}\frac{1+4\alpha^{2}x^{2}/(2k+1)^{2}\pi^{2}}{1+4\beta^{2}x^{2}/(2k+1)^{2}\pi^{2}}.$
(2.2)

Each factor in the product is of the form

$\frac{1+a^{2}x^{2}}{1+b^{2}x^{2}}=\frac{a^{2}}{b^{2}}+\frac{1-a^{2}/b^{2}}{1+b^{2}x^{2}},\quad
b^{2}>a^{2}.$

Since $1/(1+b^{2}x^{2})$ is positive definite [1, 5.2.7], it follows that the
functions $\cosh\alpha x/\cosh\beta x$ is positive definite for
$|\beta|>|\alpha|>0,$ and $\sinh\alpha x/\sinh\beta x$ is positive definite
for $|\beta|>|\alpha|>0$ with $\alpha,\beta$ the same sign.

Since

$\frac{\beta x\cosh\alpha x}{\sinh\beta
x}=\frac{\frac{\beta}{2}x}{\sinh\frac{\beta}{2}x}\cdot\frac{\cosh\alpha
x}{\cosh\frac{\beta}{2}x}$

and $x/\sinh x$ is positive definite [1, 5.2.9], it follows from the above
argument that when $\frac{\beta}{2}>|\alpha|$ the function $\frac{\beta
x\cosh\alpha x}{\sinh\beta x}$ is positive definite. ∎

Now, we define

$L_{s}(a,b)=\frac{a^{1-s}b^{s}-a^{s}b^{1-s}}{(1-2s)(\log a-\log
b)}=\frac{1}{1-2s}\int_{s}^{1-s}a^{v}b^{1-v}dv$

for $a,b>0,$ and $0\leq s<\frac{1}{2}.$ When $s=0,$ it is the logarithmic
mean. So we can call it the generalized logarithmic mean. And we also have
$\lim_{s\rightarrow{\frac{1}{2}}}L_{s}(a,b)=a^{\frac{1}{2}}b^{\frac{1}{2}}.$

######  Theorem 2.2.

For Heinz mean and the generalized logarithmic mean, we have

  1. (i)

If $0\leq s<1/2,$ and $|1-2t|<\frac{1-2s}{2},$ then $H_{t}(a,b)<<L_{s}(a,b).$

  2. (ii)

If $|1-2t|<|1-2s|,$ then $H_{t}(a,b)<<H_{s}(a,b).$

  3. (iii)

If $0\leq s<t<1/2,$ or $1\geq s>t>1/2,$ then $L_{t}(a,b)<<L_{s}(a,b).$

###### Proof.

By definition, $H_{t}(a,b)<<L_{s}(a,b)$ if

$[y_{i,j}]=\left[\frac{H_{t}(\lambda_{i},\lambda_{j})}{L_{s}(\lambda_{i},\lambda_{j})}\right]_{i,j=1,\ldots,n}$

is positive semidefinite. Set $\lambda_{i}=e^{x_{i}}$ and
$\lambda_{j}=e^{x_{j}},$ with $x_{i},x_{j}\in\mathbb{R}.$ Then

$y_{i,j}=(1-2s)\frac{(\frac{x_{i}-x_{j}}{2})(e^{(1-2t)\frac{x_{i}-x_{j}}{2}}+e^{(1-2t)\frac{x_{j}-x_{i}}{2}})}{e^{(1-2s)\frac{x_{i}-x_{j}}{2}}-e^{(1-2s)\frac{x_{j}-x_{i}}{2}}}$

Thus the matrix $[y_{i,j}]$ is congruent to one with entries

$\frac{\beta(\frac{x_{i}-x_{j}}{2})\cosh(\alpha(\frac{x_{i}-x_{j}}{2}))}{\sinh(\beta(\frac{x_{i}-x_{j}}{2}))},$

where $\alpha=1-2t,\beta=1-2s.$ Hence $[y_{i,j}]$ is positive semidefinite if
and only if the function $\frac{\beta x\cosh\alpha x}{\sinh\beta x}$ is
positive definite, which by lemma 2.1 is correct.

Similarly, we have $H_{t}(a,b)<<H_{s}(a,b)$ if $\frac{\cosh\alpha
x}{\cosh\beta x}$ is positive definite, and $L_{t}(a,b)<<L_{s}(a,b)$ if
$\frac{\sinh\alpha x}{\sinh\beta x}$ is positive definite. ∎

######  Theorem 2.3.

Let $A,B$ be any positive matrices. Then for any matrix $X$ and for $0\leq
s<1/2$ and $|1-2t|<(1-2s)/2,$ we have

$\displaystyle{\left|\kern-1.07639pt\left|\kern-1.07639pt\left|A^{1-t}XB^{t}+A^{t}XB^{1-t}\right|\kern-1.07639pt\right|\kern-1.07639pt\right|}\leq\frac{2}{1-2s}{\left|\kern-1.07639pt\left|\kern-1.07639pt\left|\int_{s}^{1-s}A^{v}XB^{1-v}dv\right|\kern-1.07639pt\right|\kern-1.07639pt\right|}.$
(2.3)

###### Proof.

Firstly, we assume $A=B.$ Since
${\left|\kern-1.07639pt\left|\kern-1.07639pt\left|\cdot\right|\kern-1.07639pt\right|\kern-1.07639pt\right|}$
is unitarily invariant, we may suppose $A$ is diagonal with entries
$\lambda_{1},\ldots,\lambda_{n}.$ Then we have

$A^{1-t}XA^{t}+A^{t}XA^{1-t}=Y\circ(\int_{s}^{1-s}A^{v}XA^{1-v}dv),$

where $Y$ is the matrix with entries

$y_{i,j}=\frac{\lambda_{i}^{t}\lambda_{j}^{1-t}+\lambda_{i}^{1-t}\lambda_{j}^{t}}{\frac{\lambda_{i}^{1-s}\lambda_{j}^{s}-\lambda_{i}^{s}\lambda_{j}^{1-s}}{\log\lambda_{i}-\log\lambda_{j}}}=\frac{2H_{t}(\lambda_{i},\lambda_{j})}{(1-2s)L_{s}(\lambda_{i},\lambda_{j})}$

A well-known result related to the Schur multiplier norm [6, Theorem 5.5.18,
5.5.19] says that if $Y$ is any positive semidefinite matrix, then for all
matrix $X,$

$\displaystyle{\left|\kern-1.07639pt\left|\kern-1.07639pt\left|Y\circ
X\right|\kern-1.07639pt\right|\kern-1.07639pt\right|}\leq\max_{i}{y_{ii}}{\left|\kern-1.07639pt\left|\kern-1.07639pt\left|X\right|\kern-1.07639pt\right|\kern-1.07639pt\right|}$
(2.4)

for every unitarily invariant norm. By Theorem 2.2, $Y$ is positive
semidefinite. Applying $(2.4),$ we have

$\displaystyle{\left|\kern-1.07639pt\left|\kern-1.07639pt\left|A^{1-t}XA^{t}+A^{t}XA^{1-t}\right|\kern-1.07639pt\right|\kern-1.07639pt\right|}\leq\frac{2}{1-2s}{\left|\kern-1.07639pt\left|\kern-1.07639pt\left|\int_{s}^{1-s}A^{v}XA^{1-v}dv\right|\kern-1.07639pt\right|\kern-1.07639pt\right|}.$
(2.5)

Now replacing $A$ and $X$ in the inequality $(2.5)$ by the $2$ by $2$ matrices
$\left(\begin{array}[]{cc}A&0\\\ 0&B\\\ \end{array}\right)$ and
$\left(\begin{array}[]{cc}0&X\\\ 0&0\\\ \end{array}\right).$ This gives the
desired inequality $(2.3).$ ∎

When $s=0,$ we get Drissi’s result $(1.3).$ Moreover, when $s=0,t=1/2,$ we get
the first inequality of $(1.2).$

######  Theorem 2.4.

Let $A,B$ be any positive matrices. Then for any matrix $X$ and for $0\leq
s<1/2,$ we have

$\displaystyle{\left|\kern-1.07639pt\left|\kern-1.07639pt\left|\int_{s}^{1-s}A^{v}XB^{1-v}dv\right|\kern-1.07639pt\right|\kern-1.07639pt\right|}\leq\frac{1-2s}{2}{\left|\kern-1.07639pt\left|\kern-1.07639pt\left|A^{1-s}XB^{s}+A^{s}XB^{1-s}\right|\kern-1.07639pt\right|\kern-1.07639pt\right|}.$
(2.6)

###### Proof.

Suppose $A$ is diagonal with entries $\lambda_{1},\ldots,\lambda_{n}.$ Then we
have

$\int_{s}^{1-s}A^{v}XA^{1-v}dv=Y\circ(A^{1-s}XA^{s}+A^{s}XA^{1-s}),$

where $Y$ is the matrix with entries

$y_{i,j}=\frac{\frac{\lambda_{i}^{1-s}\lambda_{j}^{s}-\lambda_{i}^{s}\lambda_{j}^{1-s}}{\log\lambda_{i}-\log\lambda_{j}}}{\lambda_{i}^{s}\lambda_{j}^{1-s}+\lambda_{i}^{1-s}\lambda_{j}^{s}}=\frac{(1-2s)L_{s}(\lambda_{i},\lambda_{j})}{2H_{s}(\lambda_{i},\lambda_{j})}.$

By a similar argument as in the proof of Theorem 2.2, we know that $Y$ is
positive definite if and only if

$\frac{\beta}{2}\frac{\sinh\beta x}{\beta x\cosh\beta
x}=\frac{\beta}{2}\frac{\tanh\beta x}{\beta x}$

is positive definite for $\beta=1-2s>0,$ because $\tanh x/x$ is positive
definite (see Bhatia [1, 5.2.11]). Thus Applying $(2.4)$ we have

$\displaystyle{\left|\kern-1.07639pt\left|\kern-1.07639pt\left|\int_{s}^{1-s}A^{v}XA^{1-v}dv\right|\kern-1.07639pt\right|\kern-1.07639pt\right|}\leq\frac{1-2s}{2}{\left|\kern-1.07639pt\left|\kern-1.07639pt\left|A^{1-s}XA^{s}+A^{s}XA^{1-s}\right|\kern-1.07639pt\right|\kern-1.07639pt\right|}.$
(2.7)

Hence the desired result follows. ∎

When $s=0,$ we get the second inequality of $(1.2).$

######  Theorem 2.5.

Let $A,B$ be any positive matrices. Then for any matrix $X$ and for
$|1-2t|<|1-2s|$ with $1-2t,1-2s$ the same sign, we have

$\displaystyle{\left|\kern-1.07639pt\left|\kern-1.07639pt\left|A^{1-t}XB^{t}-A^{t}XB^{1-t}\right|\kern-1.07639pt\right|\kern-1.07639pt\right|}\leq\left|\frac{1-2t}{1-2s}\right|{\left|\kern-1.07639pt\left|\kern-1.07639pt\left|A^{1-s}XB^{s}-A^{s}XB^{1-s}\right|\kern-1.07639pt\right|\kern-1.07639pt\right|}.$
(2.8)

###### Proof.

Suppose $A$ is diagonal with entries $\lambda_{1},\ldots,\lambda_{n}.$ Then we
have

$A^{1-t}XA^{t}-A^{t}XA^{1-t}=Y\circ(A^{1-s}XA^{s}-A^{s}XA^{1-s}),$

where $Y$ is the matrix with entries

$y_{i,j}=\frac{\lambda_{i}^{1-t}\lambda_{j}^{t}-\lambda_{i}^{t}\lambda_{j}^{1-t}}{\lambda_{i}^{1-s}\lambda_{j}^{s}-\lambda_{i}^{s}\lambda_{j}^{1-s}}$

Put $\lambda_{i}=e^{x_{i}}$ and $\lambda_{j}=e^{x_{j}},$ with
$x_{i},x_{j}\in\mathbb{R}.$ Then $Y$ is congruent to the matrix with entries

$\frac{\sinh(\alpha\frac{x_{i}-x_{j}}{2})}{\sinh(\beta\frac{x_{i}-x_{j}}{2})}$

where $\alpha=1-2t,\beta=1-2s.$ Since $(\sinh\alpha x)/(\sinh\beta x)$ is
positive for $|\beta|>|\alpha|>0$ with $\alpha,\beta$ the same sign, it
follows that $Y$ is positive definite. Applying the inequality $(2.4),$ we
have

$\displaystyle{\left|\kern-1.07639pt\left|\kern-1.07639pt\left|A^{1-t}XA^{t}-A^{t}XA^{1-t}\right|\kern-1.07639pt\right|\kern-1.07639pt\right|}\leq\left|\frac{1-2t}{1-2s}\right|{\left|\kern-1.07639pt\left|\kern-1.07639pt\left|A^{1-s}XA^{s}-A^{s}XA^{1-s}\right|\kern-1.07639pt\right|\kern-1.07639pt\right|}.$
(2.9)

Hence the desired result follows. ∎

Set $s=0$ and $s=1,$ and combine the conclusions, we have the following
inequality proved by Bhatia-Davis [3],

######  Corollary 2.6.

Let $A,B$ be any positive matrices. Then for any matrix $X$ and for $0\leq
t\leq 1$ we have

$\displaystyle{\left|\kern-1.07639pt\left|\kern-1.07639pt\left|A^{1-t}XB^{t}-A^{t}XB^{1-t}\right|\kern-1.07639pt\right|\kern-1.07639pt\right|}\leq|1-2t|{\left|\kern-1.07639pt\left|\kern-1.07639pt\left|AX-
XB\right|\kern-1.07639pt\right|\kern-1.07639pt\right|}.$ (2.10)

### Acknowledgments

The author acknowledges support from National Natural Science Foundation of
China, Grant No: 12001477, and the Natural Science Foundation of Jiangsu
Province for Youth, Grant No: BK20190874.

## References

  * [1] R. Bhatia, Positive Definite Matrcies. Princeton University Press, Princeton (2007).
  * [2] R. Bhatia and C. Davis, More matrix forms of the arithmetic-geometric mean inequality, SIAM J. Matrix Anal. Appl., 14 (1993), 132-136.
  * [3] R. Bhatia and C. Davis, A Cauchy-Schwarz inequality for operators with applications, Linear Algebra Appl., 223/224 (1995), 119-129.
  * [4] D. Drissi, Sharp inequalities for some operator means, SIAM J. Matrix Anal. Appl., 28(3) (2006), 822-828.
  * [5] F. Hiai and H. Kosaki, Means for matrices and comparison of their norms, Indiana Univ. Math. J., 48 (1999), 899-936.
  * [6] R. A. Horn and C. R. Johnson, Topics in Matrix Analysis, Cambridge University Press, Cambridge, UK, 1991.
  * [7] Y. Kapil and M. Singh, Contractive maps on operator ideals and norm inequalities. Linear Algebra Appl., 459 (2014), 475-492.
  * [8] H. Kosaki, Arithmetic-geometric mean and related inequalities for operators, J. Funct. Anal., 156 (1998), 429-451.
  * [9] X. Zhan, Inequalities for unitarily invariant norms, SIAM J. Matrix Anal. Appl., 20 (1999), 466-470.