remarkRemark hypothesisHypothesis claimClaim Tensor Multivariate Trace
Inequalities and Their ApplicationsShih Yu Chang

# Tensor Multivariate Trace Inequalities and Their Applications

Shih Yu Chang Department of Applied Data Science, San Jose State University,
San Jose, CA 95192, USA<EMAIL_ADDRESS>
###### Abstract

We prove several trace inequalities that extend the Araki–Lieb–Thirring (ALT)
inequality, Golden–Thompson(GT) inequality and logarithmic trace inequality to
arbitrary many tensors. Our approaches rely on complex interpolation theory as
well as asymptotic spectral pinching, providing a transparent mechanism to
treat generic tensor multivariate trace inequalities. As an example
application of our tensor extension of the Golden–Thompson inequality, we give
the tail bound for the independent sum of tensors. Such bound will play a
fundamental role in high-dimensional probability and statistical data
analysis.

###### keywords:

Tensor, Multivariate, Trace, Golden–Thompson Inequality, Araki–Lieb–Thirring
Inequality

15A69,46B28,47H60,47A30

##  1 Introduction

Trace inequalities are mathematical relations between different multivariate
trace functionals involving linear operators. These relations are
straightforward equalities if the involved linear operators commute, however,
they can be difficult to prove when the non-commuting linear operators are
involved [4].

One of the most important trace inequalities is the famous Golden-Thompson
inequality [7]. For any two Hermitian matrices $\mathbf{H}_{1}$ and
$\mathbf{H}_{2}$, we have

(1) $\displaystyle\mathrm{Tr}\exp(\mathbf{H}_{1}+\mathbf{H}_{2})$
$\displaystyle\leq$
$\displaystyle\mathrm{Tr}\exp(\mathbf{H}_{1})\exp(\mathbf{H}_{2}).$

It is easy to see that the Eq. (1) becomes an identity if two Hermitian
matrices $\mathbf{H}_{1}$ and $\mathbf{H}_{2}$ are commute. The inequality in
Eq. (1) has been generalized to several situations. For example, it has been
demonstrated that it remains valid by replacing the trace with any unitarily
invariant norm [13, 22]. The Golden-Thompson inequality has been applied to
many various fields ranging from quantum information processing [15, 16],
statistical physics [24, 27], and random matrix theory [1, 25].

The Golden-Thompson inequality can be seens as a limiting case of the more
general Araki–Lieb–Thirring (ALT) inequality [3, 17]. For any two positive
semi-definite matrices $\mathbf{A}_{1}$ and $\mathbf{A}_{2}$ with $r\in(0,1]$
and $q>0$, ALT states that

(2)
$\displaystyle\mathrm{Tr}\left(\mathbf{A}_{1}^{\frac{r}{2}}\mathbf{A}_{2}^{r}\mathbf{A}_{1}^{\frac{r}{2}}\right)^{\frac{q}{r}}\leq\mathrm{Tr}\left(\mathbf{A}_{1}^{\frac{1}{2}}\mathbf{A}_{2}\mathbf{A}_{1}^{\frac{1}{2}}\right)^{q}.$

The Golden-Thompson inequality for Schatten $p$-norm is obtained by the Lie-
Trotter product formula by taking limit $r\rightarrow 0$. The ALT inequality
has also been expanded to various directions [2, 12, 26].

The following theorem is about logarithmic trace inequality which can be used
to bound quantum information divergence [2, 8]. For any two positive semi-
definite matrices $\mathbf{A}_{1}$ and $\mathbf{A}_{2}$ with $r\in(0,1]$ and
$p>0$, logarithmic trace inequality for matrix is

(3)
$\displaystyle\frac{1}{p}\mathrm{Tr}\mathbf{A}_{1}\log\mathbf{A}^{\frac{p}{2}}_{2}\mathbf{A}^{p}_{1}\mathbf{A}^{\frac{p}{2}}_{2}\leq\mathrm{Tr}\mathbf{A}_{1}(\log\mathbf{A}_{1}+\log\mathbf{A}_{2})\leq\frac{1}{p}\mathrm{Tr}\mathbf{A}_{1}\log\mathbf{A}^{\frac{p}{2}}_{1}\mathbf{A}^{p}_{2}\mathbf{A}^{\frac{p}{2}}_{1}.$

The paper is organized as follows. Preliminaries of tensors are given in
Section 2. In Section 3, the method of pinching and complex interpolation
theory will be introduced. Three useful matrix-based trace inequalities are
extended to multivariate tensors in Section 4. The new Golden-Thompson
inequality is applied to provide tail bounds for sums of random tensors in
Section 5. Finally, the conclusions are given in Section Section 6.

##  2 Tensors Preliminaries

Essential terminologies regarding tensors will be introduced in this section.
Throughout this paper, we denote scalars by lower-case letters (e.g., $d$,
$e$, $f$, $\ldots$), vectors by boldfaced lower-case letters (e.g.,
$\mathbf{d}$, $\mathbf{e}$, $\mathbf{f}$, $\ldots$), matrices by boldfaced
capitalized letters (e.g., $\mathbf{D}$, $\mathbf{E}$, $\mathbf{F}$,
$\ldots$), and tensors by calligraphic letters (e.g., $\mathcal{D}$,
$\mathcal{E}$, $\mathcal{F}$, $\ldots$), respectively. Tensors are referred to
as multiarrays of values which can be deemed high-dimensional generalizations
from vectors and matrices. Given a positive integer $N$, let
$[N]\stackrel{{\scriptstyle\text{\tiny def}}}{{=}}{1,2,\cdots,N}$. An _order-
$N$ tensor_ (or _$N$ -th order tensor_) is represented by
$\mathcal{A}\stackrel{{\scriptstyle\text{\tiny
def}}}{{=}}(a_{i_{1},i_{2},\cdots,i_{N}})$, where $1\leq i_{j}\leq I_{j}$ for
$j\in[N]$, is a multidimensional array containing $I_{1}\times
I_{2}\times\cdots\times I_{N}$ entries. Let
$\mathbb{C}^{I_{1}\times\cdots\times I_{N}}$ and
$\mathbb{R}^{I_{1}\times\cdots\times I_{N}}$ be the sets of order-$N$
$I_{1}\times\cdots\times I_{N}$ tensors over the complex field $\mathbb{C}$
and the real field $\mathbb{R}$, respectively. For example,
$\mathcal{A}\in\mathbb{C}^{I_{1}\times\cdots\times I_{N}}$ is an order-$N$
multiarray, where the first, second, …, and $N$-th orders have $I_{1}$,
$I_{2}$, …, and $I_{N}$ entries, respectively. Thus, each entry of
$\mathcal{A}$ can be represented by $a_{i_{1},\cdots,i_{N}}$. For example,
when $N=4$, $\mathcal{A}\in\mathbb{C}^{I_{1}\times I_{2}\times I_{3}\times
I_{4}}$ is a fourth-order tensor containing entries
$a_{i_{1},i_{2},i_{3},i_{4}}$, where $i_{j}\in[I_{j}]$ for $j\in[4]$.

Without loss of generality, one can partition the dimensions of a tensor into
two groups, say $M$ and $N$ dimensions, separately. Therefore, for two
order-($M$+$N$) tensors: $\mathcal{A}\stackrel{{\scriptstyle\text{\tiny
def}}}{{=}}(a_{i_{1},\cdots,i_{M},j_{1},\cdots,j_{N}})\in\mathbb{C}^{I_{1}\times\cdots\times
I_{M}\times J_{1}\times\cdots\times J_{N}}$ and
$\mathcal{B}\stackrel{{\scriptstyle\text{\tiny
def}}}{{=}}(b_{i_{1},\cdots,i_{M},j_{1},\cdots,j_{N}})\in\mathbb{C}^{I_{1}\times\cdots\times
I_{M}\times J_{1}\times\cdots\times J_{N}}$, according to [14], the _tensor
addition_
$\mathcal{A}+\mathcal{B}\in\mathbb{C}^{I_{1}\times\cdots\times I_{M}\times
J_{1}\times\cdots\times J_{N}}$ is given by

(4)
$\displaystyle(\mathcal{A}+\mathcal{B})_{i_{1},\cdots,i_{M},j_{1}\times\cdots\times
j_{N}}$ $\displaystyle\stackrel{{\scriptstyle\text{\tiny def}}}{{=}}$
$\displaystyle a_{i_{1},\cdots,i_{M},j_{1}\times\cdots\times j_{N}}$
$\displaystyle+b_{i_{1},\cdots,i_{M},j_{1}\times\cdots\times j_{N}}.$

On the other hand, for tensors
$\mathcal{A}=(a_{i_{1},\cdots,i_{M},j_{1},\cdots,j_{N}})\in\mathbb{C}^{I_{1}\times\cdots\times
I_{M}\times J_{1}\times\cdots\times J_{N}}$ and
$\mathcal{B}=(b_{j_{1},\cdots,j_{N},k_{1},\cdots,k_{L}})\in\mathbb{C}^{J_{1}\times\cdots\times
J_{N}\times K_{1}\times\cdots\times K_{L}}$, according to [14], the _Einstein
product_ (or simply referred to as _tensor product_ in this work)
$\mathcal{A}\star_{N}\mathcal{B}\in\mathbb{C}^{I_{1}\times\cdots\times
I_{M}\times K_{1}\times\cdots\times K_{L}}$ is given by

(5)
$\displaystyle(\mathcal{A}\star_{N}\mathcal{B})_{i_{1},\cdots,i_{M},k_{1}\times\cdots\times
k_{L}}\stackrel{{\scriptstyle\text{\tiny def}}}{{=}}$
$\displaystyle\sum\limits_{j_{1},\cdots,j_{N}}a_{i_{1},\cdots,i_{M},j_{1},\cdots,j_{N}}b_{j_{1},\cdots,j_{N},k_{1},\cdots,k_{L}}.$

This tensor product will be reduced to the standard matrix multiplication as
$L$ $=$ $M$ $=$ $N$ $=$ $1$. Other simplified situations can also be extended
as _tensor–vector product_ ($M>1$, $N=1$, and $L=0$) and _tensor–matrix
product_ ($M>1$ and $N=L=1$). In analogy to matrix analysis, we define some
typical tensors and elementary tensor-operations as follows.

######  Definition 2.1.

A tensor whose entries are all zero is called a _zero tensor_ , denoted by
$\mathcal{O}$.

######  Definition 2.2.

An _identity tensor_ $\mathcal{I}\in\mathbb{C}^{I_{1}\times\cdots\times
I_{N}\times J_{1}\times\cdots\times J_{N}}$ is defined by

(6) $\displaystyle(\mathcal{I})_{i_{1}\times\cdots\times i_{N}\times
j_{1}\times\cdots\times j_{N}}\stackrel{{\scriptstyle\text{\tiny
def}}}{{=}}\prod_{k=1}^{N}\delta_{i_{k},j_{k}},$

where $\delta_{i_{k},j_{k}}\stackrel{{\scriptstyle\text{\tiny def}}}{{=}}1$ if
$i_{k}=j_{k}$; otherwise
$\delta_{i_{k},j_{k}}\stackrel{{\scriptstyle\text{\tiny def}}}{{=}}0$.

In order to define the _Hermitian_ tensor, the _conjugate transpose operation_
(or _Hermitian adjoint_) of a tensor is specified as follows.

######  Definition 2.3.

Given a tensor $\mathcal{A}\stackrel{{\scriptstyle\text{\tiny
def}}}{{=}}(a_{i_{1},\cdots,i_{M},j_{1},\cdots,j_{N}})\in\mathbb{C}^{I_{1}\times\cdots\times
I_{M}\times J_{1}\times\cdots\times J_{N}}$, its conjugate transpose, denoted
by $\mathcal{A}^{H}$, is defined as

(7)
$\displaystyle(\mathcal{A}^{H})_{j_{1},\cdots,j_{N},i_{1},\cdots,i_{M}}\stackrel{{\scriptstyle\text{\tiny
def}}}{{=}}\overline{a_{i_{1},\cdots,i_{M},j_{1},\cdots,j_{N}}},$

where the overline notion indicates the complex conjugate of the complex
number $a_{i_{1},\cdots,i_{M},j_{1},\cdots,j_{N}}$. If a tensor $\mathcal{A}$
satisfying $\mathcal{A}^{H}=\mathcal{A}$, then $\mathcal{A}$ is a _Hermitian
tensor_.

######  Definition 2.4.

Given a tensor $\mathcal{A}\stackrel{{\scriptstyle\text{\tiny
def}}}{{=}}(a_{i_{1},\cdots,i_{M},j_{1},\cdots,j_{M}})\in\mathbb{C}^{I_{1}\times\cdots\times
I_{M}\times J_{1}\times\cdots\times J_{M}}$, if

(8)
$\displaystyle\mathcal{A}^{H}\star_{M}\mathcal{A}=\mathcal{A}\star_{M}\mathcal{A}^{H}=\mathcal{I}\in\mathbb{C}^{I_{1}\times\cdots\times
I_{M}\times J_{1}\times\cdots\times J_{M}},$

then $\mathcal{A}$ is a _unitary tensor_.

######  Definition 2.5.

Given a _square tensor_ $\mathcal{A}\in\mathbb{C}^{I_{1}\times\cdots\times
I_{M}\times I_{1}\times\cdots\times I_{M}}$, if there exists
$\mathcal{X}\in\mathbb{C}^{I_{1}\times\cdots\times I_{M}\times
I_{1}\times\cdots\times I_{M}}$ such that

(9)
$\displaystyle\mathcal{A}\star_{M}\mathcal{X}=\mathcal{X}\star_{M}\mathcal{A}=\mathcal{I},$

then $\mathcal{X}$ is the _inverse_ of $\mathcal{A}$. We usually write
$\mathcal{X}\stackrel{{\scriptstyle\text{\tiny def}}}{{=}}\mathcal{A}^{-1}$
thereby.

We also list other necessary tensor operations here. The _trace_ of a tensor
is equivalent to the summation of all diagonal entries such that

(10) $\displaystyle\mathrm{Tr}(\mathcal{A})\stackrel{{\scriptstyle\text{\tiny
def}}}{{=}}\sum\limits_{1\leq i_{j}\leq
I_{j},j\in[N]}\mathcal{A}_{i_{1},\cdots,i_{M},i_{1},\cdots,i_{M}}.$

The _inner product_ of two tensors $\mathcal{A}$,
$\mathcal{B}\in\mathbb{C}^{I_{1}\times\cdots\times I_{M}\times
J_{1}\times\cdots\times J_{N}}$ is given by

(11)
$\displaystyle\langle\mathcal{A},\mathcal{B}\rangle\stackrel{{\scriptstyle\text{\tiny
def}}}{{=}}\mathrm{Tr}(\mathcal{A}^{H}\star_{M}\mathcal{B}).$

According to Eq. (11), the _Frobenius norm_ of a tensor $\mathcal{A}$ is
defined by

(12)
$\displaystyle\left\|\mathcal{A}\right\|\stackrel{{\scriptstyle\text{\tiny
def}}}{{=}}\sqrt{\langle\mathcal{A},\mathcal{A}\rangle}.$

######  Definition 2.6.

Given a square tensor $\mathcal{A}\in\mathbb{C}^{I_{1}\times\cdots\times
I_{M}\times I_{1}\times\cdots\times I_{M}}$, the _tensor exponential_ of the
tensor $\mathcal{A}$ is defined as

(13) $\displaystyle e^{\mathcal{A}}\stackrel{{\scriptstyle\text{\tiny
def}}}{{=}}\sum\limits_{k=0}^{\infty}\frac{\mathcal{A}^{k}}{k!},$

where $\mathcal{A}^{0}$ is defined as the identity tensor
$\mathcal{I}\in\mathbb{C}^{I_{1}\times\cdots\times I_{M}\times
I_{1}\times\cdots\times I_{M}}$ and
$\mathcal{A}^{k}=\underbrace{\mathcal{A}\star_{M}\mathcal{A}\star_{M}\dots\star_{M}\mathcal{A}}_{\mbox{$k$
terms of $\mathcal{A}$}}$.

Given a tensor $\mathcal{B}$, the tensor $\mathcal{A}$ is said to be a _tensor
logarithm_ of $\mathcal{B}$ if $e^{\mathcal{A}}=\mathcal{B}$

Following definitions are about the Kronecker product and the sum of two
tensors.

######  Definition 2.7.

Given two tensors $\mathcal{A}\in\mathbb{C}^{I_{1}\times\cdots\times
I_{M}\times J_{1}\times\cdots\times J_{N}}$ and
$\mathcal{B}\in\mathbb{C}^{K_{1}\times\cdots\times K_{P}\times
L_{1}\times\cdots\times L_{Q}}$, we define the Kronecker product of two
tensors $\mathcal{A}\otimes\mathcal{B}$ as

(14)
$\displaystyle\mathcal{A}\otimes\mathcal{B}\stackrel{{\scriptstyle\text{\tiny
def}}}{{=}}(a_{i_{1},\cdots,i_{M},j_{1},\cdots,j_{N}}\mathcal{B})_{i_{1},\cdots,i_{M},j_{1},\cdots,j_{N}}.$

######  Definition 2.8.

Given two square tensors $\mathcal{A}\in\mathbb{C}^{I_{1}\times\cdots\times
I_{M}\times I_{1}\times\cdots\times I_{M}}$ and
$\mathcal{B}\in\mathbb{C}^{J_{1}\times\cdots\times J_{P}\times
J_{1}\times\cdots\times J_{P}}$, we define the Kronecker sum of two tensors
$\mathcal{A}\oplus\mathcal{B}$ as

(15)
$\displaystyle\mathcal{A}\oplus\mathcal{B}\stackrel{{\scriptstyle\text{\tiny
def}}}{{=}}\mathcal{A}\otimes\mathcal{I}_{J_{1}\times\cdots\times
J_{P}}+\mathcal{I}_{I_{1}\times\cdots\times I_{M}}\otimes\mathcal{B},$

where $\mathcal{I}_{I_{1}\times\cdots\times
I_{M}}\in\mathbb{C}^{I_{1}\times\cdots\times I_{M}\times
I_{1}\times\cdots\times I_{M}}$ and $\mathcal{I}_{J_{1}\times\cdots\times
J_{P}}\in\mathbb{C}^{J_{1}\times\cdots\times J_{P}\times
J_{1}\times\cdots\times J_{P}}$ are identity tensors.

We require the following two lemmas about Kroneecker product which will be
used for later proof in Theorem 3.8.

######  Lemma 2.9.

Given tensors $\mathcal{A}_{1}$ and $\mathcal{A}_{2}$ which act on spaces
$S_{1}$ and $S_{2}$, respectively, we have following identities :

(16) $\displaystyle\mathrm{Tr}(\mathcal{A}_{1}\otimes\mathcal{A}_{2})$
$\displaystyle=$
$\displaystyle\mathrm{Tr}(\mathcal{A}_{1})\mathrm{Tr}(\mathcal{A}_{2});$

and

(17) $\displaystyle\exp(\mathcal{A}_{1})\otimes\exp(\mathcal{A}_{2})$
$\displaystyle=$
$\displaystyle\exp(\mathcal{A}_{1}\otimes\mathcal{I}_{S_{2}}+\mathcal{I}_{S_{1}}\otimes\mathcal{A}_{2}),$

where $\mathcal{I}_{S_{1}}$ and $\mathcal{I}_{S_{2}}$ are identity tensors
acts on spaces $S_{1}$ and $S_{2}$, respectively.

Proof:

We prove Eq. (16) first. Suppose the tensor
$\mathcal{A}_{1}\in\mathbb{C}^{I_{1}\times\cdots\times I_{M}\times
I_{1}\times\cdots\times I_{M}}$, then its entries will be
$(a_{i_{1},\cdots,i_{M},j_{1},\cdots,j_{M}})$. By definition of the Kronecker
product, we have

$\displaystyle\mathrm{Tr}(\mathcal{A}_{1}\otimes\mathcal{A}_{2})=\sum\limits_{i_{1},\cdots,i_{M}}\mathrm{Tr}(a_{i_{1},\cdots,i_{M},i_{1},\cdots,i_{M}}\mathcal{A}_{2})$
(18)
$\displaystyle=\sum\limits_{i_{1},\cdots,i_{M}}a_{i_{1},\cdots,i_{M},i_{1},\cdots,i_{M}}\mathrm{Tr}(\mathcal{A}_{2})=\mathrm{Tr}(\mathcal{A}_{1})\mathrm{Tr}(\mathcal{A}_{2}).$

Next, we will verify the relation provided by Eq. (17). Because we have

(19)
$\displaystyle\exp(\mathcal{A}_{1})\otimes\exp(\mathcal{A}_{2})=_{1}\exp(\mathcal{A}_{1}\oplus\mathcal{A}_{2})=\exp(\mathcal{A}_{1}\otimes\mathcal{I}_{S_{2}}+\mathcal{I}_{S_{1}}\otimes\mathcal{A}_{2}),$

where the equality $=_{1}$ comes from Theorems 2 and 3 in [18], and the last
equality is provided by Definition 2.8. $\hfill\Box$

######  Lemma 2.10.

Given positive tensors $\mathcal{A}_{1}$ and $\mathcal{A}_{2}$ which act on
spaces $S_{1}$ and $S_{2}$, respectively, we have following identity :

(20) $\displaystyle\log(\mathcal{A}_{1}\otimes\mathcal{A}_{2})$
$\displaystyle=$
$\displaystyle(\log\mathcal{A}_{1})\otimes\mathcal{I}_{S_{2}}+\mathcal{I}_{S_{1}}\otimes(\log\mathcal{A}_{2}),$

where $\mathcal{I}_{S_{1}}$ and $\mathcal{I}_{S_{2}}$ are identity tensors
acts on spaces $S_{1}$ and $S_{2}$, respectively.

Proof:

From the relation (17) and set $\mathcal{B}_{1}=\log(\mathcal{A}_{1})$,
$\mathcal{B}_{2}=\log(\mathcal{A}_{2})$ , we have

$\displaystyle\exp(\mathcal{B}_{1})\otimes\exp(\mathcal{B}_{2})$
$\displaystyle=$
$\displaystyle\exp\left(\mathcal{B}_{1}\otimes\mathcal{I}_{S_{2}}+\mathcal{I}_{S_{1}}\otimes\mathcal{B}_{2}\right)$
(21) $\displaystyle\Leftrightarrow\hskip
31.2982pt\mathcal{A}_{1}\otimes\mathcal{A}_{2}$ $\displaystyle=$
$\displaystyle\exp\left(\log(\mathcal{A}_{1})\otimes\mathcal{I}_{S_{2}}+\mathcal{I}_{S_{1}}\otimes\log(\mathcal{A}_{2})\right)$

By taking $\log$ at both sides, we have desired result

(22) $\displaystyle\log(\mathcal{A}_{1}\otimes\mathcal{A}_{2})$
$\displaystyle=$
$\displaystyle(\log\mathcal{A}_{1})\otimes\mathcal{I}_{S_{2}}+\mathcal{I}_{S_{1}}\otimes(\log\mathcal{A}_{2}).$

$\hfill\Box$

##  3 Tools for Hermitian Tensors

In this section, we will introduce two main techniques used to prove
multivariate trace inequalities for tensors. Spectrum pinching method is
discussed in Section 3.1, and complex interpolation theory is presented in
Section 3.2.

###  3.1 Pinching Map

The purpose for studying the pinching method arises from the following
problem: Given two Hermitian tensors $\mathcal{H}_{1}$ and $\mathcal{H}_{2}$
that do not commute. Does there exist a method to transform one of the two
tensors such that they commute without completely destroying the structure of
the original tensor? The spectral pinching method is a tool to resolve this
problem. Before discussing this method in detail we have to introduce the
pinching map.

Given a Hermitian tensor $\mathcal{H}\in\mathbb{C}^{I_{1}\times\cdots\times
I_{M}\times I_{1}\times\cdots\times I_{M}}$, we have spectral decomposition as
[19]:

(23) $\displaystyle\mathcal{H}$ $\displaystyle=$
$\displaystyle\sum\limits_{\lambda\in\mbox{sp}(\mathcal{H})}\lambda\mathcal{U}_{\lambda},$

where $\lambda\in\mbox{sp}(\mathcal{H})\in\mathbb{R}$ and
$\mathcal{U}_{\lambda}\in\mathbb{C}^{I_{1}\times\cdots\times I_{M}\times
I_{1}\times\cdots\times I_{M}}$ are mutually orthogonal tensors. The
_pinching map_ with respect to $\mathcal{H}$ is defined as

(24)
$\displaystyle\mathfrak{P}_{\mathcal{H}}:\mathcal{X}\rightarrow\sum\limits_{\lambda\in\mbox{sp}(\mathcal{H})}\mathcal{U}_{\lambda}\star_{M}\mathcal{X}\star_{M}\mathcal{U}_{\lambda},$

where $\mathcal{X}\in\mathbb{C}^{I_{1}\times\cdots\times I_{M}\times
I_{1}\times\cdots\times I_{M}}$ is a Hermitian tensor. The pinching map
possesses various nice properties that will be discussed at this section. For
example, $\mathfrak{P}_{\mathcal{H}}(\mathcal{X})$ always commutes with
$\mathcal{H}$ for any nonnegative tensor $\mathcal{X}$. Two lemmas are
introduced first which will be used to prove several useful properties about
pinching maps.

######  Lemma 3.1.

Let the tensor $\mathcal{A}\in\mathbb{C}^{I_{1}\times\cdots\times I_{M}}$,
where $|I_{i}|=N$ for $i\in[M]$. The number of distinct eigenvalues of
$\mathcal{A}^{\otimes m}$ , where $\otimes$ is the Kronecker product defined
in Definition 2.7, grows polynomially with $m$.

Proof: Let us use the symbol $\mbox{sp}(\mathcal{A}^{\otimes m})$ to represent
the spectrum space, i.e., the space of eigenvalues. Because the number of
distinct eigenvalues of $\mathcal{A}^{\otimes m}$, denoted as
$|\mbox{sp}(\mathcal{A}^{\otimes m})|$, is bounded by the number of different
types of sequences of $N(M-1)^{N-1}\stackrel{{\scriptstyle\text{\tiny
def}}}{{=}}e_{\mathcal{A}}$ symbols of length $m$ from methods of types [5],
then we have

(25) $\displaystyle|\mbox{sp}(\mathcal{A}^{\otimes
m})|\leq{m+e_{\mathcal{A}}-1\choose
e_{\mathcal{A}}-1}\leq\frac{(m+e_{\mathcal{A}}-1)^{e_{\mathcal{A}}-1}}{(e_{\mathcal{A}}-1)!}=O(\mbox{poly}(m)),$

where $O(\mbox{poly}(m))$ represents a function that grows with $m$
polynomially. When $m=1$, the number of $|\mbox{sp}(\mathcal{A}^{\otimes})|$
is upper bounded by $e_{\mathcal{A}}$, which is the number of eigenvalues of
$\mathcal{A}$, see Theorem 1.1 in [21]. $\hfill\Box$

For any probability measure $\mu$ be a probability measure on a measurable
space $(X,\Sigma)$ and consider a sequence of nonnegative tensors
$\\{\mathcal{A}_{x}\\}_{x\in X}$, we have following triangle inequality:

(26)
$\displaystyle\left\|\int\mathcal{A}_{x}\mu(dx)\right\|_{p}\leq\int\left\|\mathcal{A}_{x}\right\|_{p}\mu(dx),$

due to the convexity of $p$-norm for $p\geq 1$. Quasi-norms with $p\in(0,1)$
are no longer convex. However, we demonstrate in Lemma 3.2 that these quasi-
norms still satisfy an asymptotic convexity property for Kronecker products of
tensors in the sense of allowing an extra term associated with the number of
tensors involving the Kronecker product.

######  Lemma 3.2.

Let $p\in(0,1)$, $\mu$ be a probability measure on a measurable space
$(X,\Sigma)$, and consider a sequence of nonnegative tensors
$\\{\mathcal{A}_{x}\\}_{x\in X}$ with
$\mathcal{A}_{x}\in\mathbb{C}^{I_{1}\times\cdots\times I_{M}}$ having
Canonical Polyadic (CP) decomposition, i.e., each $\mathcal{A}_{x}$ can be
expressed as
$\mathcal{A}_{x}=\sum\limits_{k_{x}}\lambda_{k_{x}}\mathbf{a}_{1,k_{x}}\otimes\mathbf{a}_{2,k_{x}}\otimes\cdots\otimes\mathbf{a}_{M,k_{x}}$,
where $\lambda_{k_{x}}\geq 0$, and $\mathbf{a}_{i,k_{x}}\in\mathbb{C}^{I_{i}}$
for $i\in[M]$. Then we have

(27) $\displaystyle\frac{1}{m}\log\left\|\int_{X}\mathcal{A}_{x}^{\otimes
m}\mu(dx)\right\|_{p}\leq\frac{1}{m}\log\left(\int_{X}\left\|\mathcal{A}_{x}^{\otimes
m}\right\|_{p}\mu(dx)\right)+\frac{\log\mbox{poly}(m)}{m}.$

Proof: Let $\mathfrak{H}$ be the Hilbert space where the tensor
$\mathcal{A}_{x}$ acts on. For any $x\in X$, consider the CP decomposition
$\mathcal{A}_{x}=\sum\limits_{k_{x}}\lambda_{k_{x}}\mathbf{a}_{1,k_{x}}\otimes\mathbf{a}_{2,k_{x}}\otimes\cdots\otimes\mathbf{a}_{M,k_{x}}$.
By introducing an isometric space $\mathfrak{H}^{\prime}$ to $\mathfrak{H}$,
we define the vector
$\mathbf{v}_{i,k_{x}}\in\mathfrak{H}\otimes\mathfrak{H}^{\prime}$ by
$\mathbf{v}_{i,k_{x}}=\sum\limits_{k_{x}}\lambda^{\frac{1}{M}}_{k_{x}}\mathbf{a}_{i,k_{x}}\otimes\mathbf{a}_{i,k_{x}}$
for $i\in[M]$, i.e., the purification of $\mathcal{A}_{x}$ indicating that
$\mathrm{Tr}_{\mathfrak{A}^{\prime}}(\sum\limits_{k_{x}}\lambda_{k_{x}}\mathbf{v}_{1,k_{x}}\otimes\mathbf{v}_{2,k_{x}}\otimes\cdots\otimes\mathbf{v}_{M,k_{x}})=\mathcal{A}_{x}$
[11]. Note that the projectors
$(\sum\limits_{k_{x}}\lambda_{k_{x}}\mathbf{v}_{1,k_{x}}\otimes\mathbf{v}_{2,k_{x}}\otimes\cdots\otimes\mathbf{v}_{M,k_{x}})^{\otimes
m}$ lie in the symmetric subspace of
$(\mathfrak{H}\otimes\mathfrak{H}^{\prime})^{\otimes m}$ whose dimension grows
with $\mbox{poly}(m)$ from Lemma 3.1. Then, we have

(28) $\displaystyle\int_{X}\mathcal{A}_{x}^{\otimes
m}\mu(dx)=\int_{X}\mathrm{Tr}_{\mathfrak{H}^{\prime\otimes
m}}\left(\sum\limits_{k_{x}}\lambda_{k_{x}}\mathbf{v}_{1,k_{x}}\otimes\mathbf{v}_{2,k_{x}}\otimes\cdots\otimes\mathbf{v}_{M,k_{x}}\right)^{\otimes
m}\mu(dx).$

From Caratheodory theorem (see Theorem 18 in [6]), there exists a discrete
probability measure $Pr(x)$, where $x\in X_{d}$ and $X_{d}\in X$ is the
discrete set with the cardinality as $\mbox{poly}(m)$ such that

$\displaystyle\int_{X}\mathcal{A}_{x}^{\otimes m}\mu(dx)=\sum\limits_{x\in
X_{d}}Pr(x)\mathcal{A}_{x}^{\otimes
m},\mbox{and}\int_{X}\left\|\mathcal{A}_{x}^{\otimes
m}\right\|_{p}\mu(dx)=\sum\limits_{x\in
X_{d}}Pr(x)\left\|\mathcal{A}_{x}^{\otimes m}\right\|_{p}.$

Therefore, we can get

(29) $\displaystyle\frac{1}{m}\log\left\|\int_{X}\mathcal{A}_{x}^{\otimes
m}\mu(dx)\right\|_{p}=\frac{1}{m}\log\left\|\sum\limits_{x\in
X_{d}}Pr(x)\mathcal{A}_{x}^{\otimes m}\right\|_{p}.$

When $p\in(0,1)$, the Schatten $p$-norm satisfies following triangle
inequality for tensors (see [10])

(30)
$\displaystyle\left\|\sum\limits_{i=1}^{n}\mathcal{A}_{i}\right\|_{p}^{p}\leq\sum\limits_{i=1}^{n}\left\|\mathcal{A}_{i}\right\|_{p}^{p},$

and from Eq. (29), we obtain

(31) $\displaystyle\frac{1}{m}\log\left\|\int_{X}\mathcal{A}_{x}^{\otimes
m}\mu(dx)\right\|_{p}$ $\displaystyle\leq$
$\displaystyle\frac{1}{m}\log\left(\sum\limits_{x\in
X_{d}}\left\|Pr(x)\mathcal{A}^{\otimes
m}_{x}\right\|_{p}^{p}\right)^{\frac{1}{p}}$ $\displaystyle=$
$\displaystyle\frac{1}{m}\log\left(|X_{d}|^{\frac{1}{p}}\left(\frac{1}{|X_{d}|}\sum\limits_{x\in
X_{d}}\left\|Pr(x)\mathcal{A}^{\otimes
m}_{x}\right\|_{p}^{p}\right)^{\frac{1}{p}}\right).$

Since the map $s\rightarrow s^{\frac{1}{p}}$ is convex for $p\in(0,1)$, we
have

$\displaystyle\frac{1}{m}\log\left\|\int_{X}\mathcal{A}_{x}^{\otimes
m}\mu(dx)\right\|_{p}$ $\displaystyle\leq$
$\displaystyle\frac{1}{m}\log\left(|X_{d}|^{\frac{1}{p}-1}\sum\limits_{x\in
X_{d}}\left\|Pr(x)\mathcal{A}^{\otimes m}_{x}\right\|_{p}\right)$
$\displaystyle=$ $\displaystyle\frac{1}{m}\log\left(\sum\limits_{x\in
X_{d}}Pr(x)\left\|\mathcal{A}^{\otimes
m}_{x}\right\|_{p}\right)+\frac{1-p}{mp}\log|X_{d}|$ $\displaystyle=$
$\displaystyle\frac{1}{m}\log\left(\int_{X}\left\|\mathcal{A}^{\otimes
m}_{x}\right\|_{p}\mu(dx)\right)+\frac{\log\mbox{poly}(m)}{m},$

where $|X_{d}|=\mbox{poly}(m)$ is applied at the last step. $\hfill\Box$

From Eq. (26) and from Lemma 3.2, we also have

(33) $\displaystyle\frac{1}{m}\log\left\|\int_{X}\mathcal{A}_{x}^{\otimes
m}\mu(dx)\right\|_{p}$ $\displaystyle\leq$ $\displaystyle\log\sup_{x\in
X}\left\|\mathcal{A}_{x}\right\|_{p}+\frac{\log\mbox{poly}(m)}{m}.$

for all $p>0$.

We need the following definition about a family of probability distribution to
represent a pinching map with integration.

######  Definition 3.3.

We define a family of probability distribution on $\mathbb{R}$, named as
$\mu_{\Delta}(x)$, which satisfies following properties:

  * •

$\tilde{\mu}_{\Delta}(0)=1$, where $\tilde{\mu}_{\Delta}$ is the Fourier
transform of the distribution function $\mu_{\Delta}$.

  * •

$\tilde{\mu}_{\Delta}(\omega)=0$ if and only if $|\omega|\geq\Delta$.

Following lemma will provide an integral representation of the pinching map.

######  Lemma 3.4.

[Integral Representation of Pinching Map]

Let $\mathcal{H},\mathcal{X}\in\mathbb{C}^{I_{1}\times\cdots\times I_{M}\times
I_{1}\times\cdots\times I_{M}}$ be Hermitian tensors with same dimensions and
$\mu_{\Delta_{\mathcal{H}}}$ is a probability measure with properties given in
Definition 3.3. The term $\Delta_{\mathcal{H}}$ is defined as
$\Delta_{\mathcal{H}}\stackrel{{\scriptstyle\text{\tiny
def}}}{{=}}\min\\{|\lambda_{j}-\lambda_{k}|:\lambda_{j}\neq\lambda_{k}\\}$
where $\lambda_{j},\lambda_{k}$ are two distinct eigenvalues in the spectral
decomposition of the tensor $\mathcal{H}$ given by Eq. (23), then we have
following integral representation for a pinching map

(34) $\displaystyle\mathfrak{P}_{\mathcal{H}}(\mathcal{X})$ $\displaystyle=$
$\displaystyle\int\limits_{-\infty}^{\infty}e^{\iota
s\mathcal{H}}\star_{M}\mathcal{X}\star_{M}e^{-\iota
s\mathcal{H}}\mu_{\Delta_{\mathcal{H}}}(s)ds,$

where $\iota$ is $\sqrt{-1}$.

Proof:

Because the spectral decomposition of tensor $\mathcal{H}$ is

(35) $\displaystyle\mathcal{H}$ $\displaystyle=$
$\displaystyle\sum\limits_{\lambda\in\mbox{sp}(\mathcal{H})}\lambda\mathcal{U}_{\lambda},$

where $\lambda\in\mbox{sp}(\mathcal{H})\in\mathbb{R}$ and
$\mathcal{U}_{\lambda}$ are mutually orthogonal tensors. For any
$s\in\mathbb{R}$, we then have

(36) $\displaystyle e^{\iota s\mathcal{H}}$ $\displaystyle=$
$\displaystyle\sum\limits_{\lambda\in\mbox{sp}(\mathcal{H})}e^{\iota
s\lambda}\mathcal{U}_{\lambda},$

and

(37) $\displaystyle e^{\iota
s\mathcal{H}}\star_{M}\mathcal{X}\star_{M}e^{-\iota s\mathcal{H}}$
$\displaystyle=$
$\displaystyle\sum\limits_{\lambda,\lambda^{\prime}\in\mbox{sp}(\mathcal{H})}e^{\iota
s(\lambda-\lambda^{\prime})}\mathcal{U}_{\lambda}\star_{M}\mathcal{X}\star_{M}\mathcal{U}_{\lambda^{\prime}}.$

If we integrate both sides of Eq. (35) with respect to measure
$\mu_{\Delta_{\mathcal{H}}}$, we obtain

$\displaystyle\int_{-\infty}^{\infty}e^{\iota
s\mathcal{H}}\star_{M}\mathcal{X}\star_{M}e^{-\iota
s\mathcal{H}}\mu_{\Delta_{\mathcal{H}}}(s)ds~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}$
$\displaystyle=\int_{-\infty}^{\infty}\left(\sum\limits_{\lambda,\lambda^{\prime}\in\mbox{sp}(\mathcal{H})}e^{\iota
s(\lambda-\lambda^{\prime})}\mathcal{U}_{\lambda}\star_{M}\mathcal{X}\star_{M}\mathcal{U}_{\lambda^{\prime}}\right)\mu_{\Delta_{\mathcal{H}}}(s)ds$
(38)
$\displaystyle=\sum\limits_{\lambda,\lambda^{\prime}\in\mbox{sp}(\mathcal{H})}\mathcal{U}_{\lambda}\star_{M}\mathcal{X}\star_{M}\mathcal{U}_{\lambda^{\prime}}\tilde{\mu}_{\Delta_{\mathcal{H}}}(\lambda-\lambda^{\prime}).~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}$

By applying the properties in Definition 3.3 and the definition of the
spectral gap $\Delta_{\mathcal{H}}$, we finally obtain

(39) $\displaystyle\int_{-\infty}^{\infty}e^{\iota
s\mathcal{H}}\star_{M}\mathcal{X}\star_{M}e^{-\iota
s\mathcal{H}}\mu_{\Delta_{\mathcal{H}}}(s)ds$ $\displaystyle=$
$\displaystyle\sum\limits_{\lambda\in\mbox{sp}(\mathcal{H})}\mathcal{U}_{\lambda}\star_{M}\mathcal{X}\star_{M}\mathcal{U}_{\lambda}=\mathfrak{P}_{\mathcal{H}}(\mathcal{X}),$

which asserts this Lemma. $\hfill\Box$

Following lemmas are introduced for those nice properties about pinching maps.

######  Lemma 3.5.

[commutativity of pinching map] Given a Hermitian tensor
$\mathcal{H}\in\mathbb{C}^{I_{1}\times\cdots\times I_{M}\times
I_{1}\times\cdots\times I_{M}}$ and any nonnegative tensors
$\mathcal{X}\in\mathbb{C}^{I_{1}\times\cdots\times I_{M}\times
I_{1}\times\cdots\times I_{M}}$, we have

(40)
$\displaystyle\mathfrak{P}_{\mathcal{H}}(\mathcal{X})\star_{M}\mathcal{H}=\mathcal{H}\star_{M}\mathfrak{P}_{\mathcal{H}}(\mathcal{X}).$

Proof:

Because we have

(41)
$\displaystyle\mathfrak{P}_{\mathcal{H}}(\mathcal{X})\star_{M}\mathcal{H}$
$\displaystyle=$ $\displaystyle\sum\limits_{\lambda,\lambda^{\prime}\in
sp(\mathcal{H})}\mathcal{U}_{\lambda}\star_{M}\mathcal{X}\star_{M}\mathcal{U}_{\lambda}\star_{M}\lambda^{\prime}\mathcal{U}_{\lambda^{\prime}}=\sum\limits_{\lambda\in
sp(\mathcal{H})}\lambda\mathcal{U}_{\lambda}\star_{M}\mathcal{X}\star_{M}\mathcal{U}_{\lambda}$
$\displaystyle=$ $\displaystyle\sum\limits_{\lambda,\lambda^{\prime}\in
sp(\mathcal{H})}\lambda^{\prime}\mathcal{U}_{\lambda^{\prime}}\star_{M}\mathcal{U}_{\lambda}\star_{M}\mathcal{X}\star_{M}\mathcal{U}_{\lambda}=\mathcal{H}\star_{M}\mathfrak{P}_{\mathcal{H}}(\mathcal{X}).$

$\hfill\Box$

######  Lemma 3.6.

[trace identity of pinching map] Given a Hermitian tensor
$\mathcal{H}\in\mathbb{C}^{I_{1}\times\cdots\times I_{M}\times
I_{1}\times\cdots\times I_{M}}$ and any nonnegative tensors
$\mathcal{X}\in\mathbb{C}^{I_{1}\times\cdots\times I_{M}\times
I_{1}\times\cdots\times I_{M}}$, we have

(42)
$\displaystyle\mathrm{Tr}\left(\mathfrak{P}_{\mathcal{H}}(\mathcal{X})\star_{M}\mathcal{H}\right)$
$\displaystyle=$
$\displaystyle\mathrm{Tr}\left(\mathcal{X}\star_{M}\mathcal{H}\right).$

Proof:

From linearity and cyclic properties of the trace, Lemma 3.4 and the fact that
the tensor $e^{-\iota s\mathcal{H}}$ commutes with the tensor $\mathcal{H}$
for all $s\in\mathbb{R}$, then we have

(43)
$\displaystyle\mathrm{Tr}\left(\mathfrak{P}_{\mathcal{H}}(\mathcal{X})\star_{M}\mathcal{H}\right)$
$\displaystyle=$
$\displaystyle\int_{-\infty}^{\infty}\mathrm{Tr}\left(e^{\iota
s\mathcal{H}}\star_{M}\mathcal{X}\star_{M}e^{-\iota
s\mathcal{H}}\star_{M}\mathcal{H}\right)\mu_{\Delta_{\mathcal{H}}}(s)ds$
$\displaystyle=$
$\displaystyle\int_{-\infty}^{\infty}\mathrm{Tr}\left(\mathcal{X}\star_{M}\mathcal{H}\right)\mu_{\Delta_{\mathcal{H}}}(s)ds=\mathrm{Tr}\left(\mathcal{X}\star_{M}\mathcal{H}\right).$

$\hfill\Box$

######  Lemma 3.7.

[Pinching Inequality] Let $\geq_{Lo}$ be Loewner order for two positive semi-
definite tensors, i.e., we say that tensors $\mathcal{A}\geq_{Lo}\mathcal{B}$
if $\mathcal{A}-\mathcal{B}$ is a positive semi-definite tensor. Given a
Hermitian tensor $\mathcal{H}\in\mathbb{C}^{I_{1}\times\cdots\times
I_{M}\times I_{1}\times\cdots\times I_{M}}$ and any nonnegative tensors
$\mathcal{X}\in\mathbb{C}^{I_{1}\times\cdots\times I_{M}\times
I_{1}\times\cdots\times I_{M}}$, we have

(44)
$\displaystyle\mathfrak{P}_{\mathcal{H}}(\mathcal{X})\geq_{Lo}\frac{\mathcal{X}}{|sp(\mathcal{H}|},$

where $|sp(\mathcal{H})|$ is the cardinality for the eigenvalues in the space
$sp(\mathcal{H})$.

Proof:

We first define the tensor $\mathcal{V}_{k}$ as following:

(45) $\displaystyle\mathcal{V}_{k}\stackrel{{\scriptstyle\text{\tiny
def}}}{{=}}\sum\limits_{j=1}^{|sp(\mathcal{H})|}\exp\left(\frac{\iota 2\pi
jk}{|sp(\mathcal{H})|}\right)\mathcal{U}_{\lambda_{j}}.$

Then, the pinching map $\mathfrak{P}_{\mathcal{H}}(\mathcal{X})$ can be
expressed as

(46)
$\displaystyle\mathfrak{P}_{\mathcal{H}}(\mathcal{X})=\sum\limits_{\lambda\in\mbox{sp}(\mathcal{H})}\mathcal{U}_{\lambda}\mathcal{X}\mathcal{U}_{\lambda}=_{1}\frac{1}{|sp(\mathcal{H})|}\sum\limits_{k=1}^{|sp(\mathcal{H})|}\mathcal{V}_{k}\mathcal{X}\mathcal{V}^{H}_{k}\geq_{Lo}\frac{\mathcal{X}}{|sp(\mathcal{H})|},$

where we use following fact in the equality $=_{1}$:

(47) $\displaystyle\sum\limits_{k=1}^{|sp(\mathcal{H})|}\exp\left(\frac{\iota
2k(j-j^{\prime})\pi}{|sp(\mathcal{H})|}\right)=|sp(\mathcal{H})|\mathbf{1}(j=j^{\prime}).$

For the inequality $\geq_{Lo}$, we use following relations

(48)
$\displaystyle\mathcal{V}_{k}\mathcal{X}\mathcal{V}^{H}_{k}\geq_{Lo}\mathcal{O},$

and

(49) $\displaystyle\mathcal{V}_{|sp(\mathcal{H}|}=\mathcal{I},$

where the zero tensor $\mathcal{O}$ and the identiy tensor $\mathcal{I}$ both
are with the same dimensions as $\mathcal{X}$ (or $\mathcal{H}$). $\hfill\Box$

######  Theorem 3.8 (Golden-Thompson inequality for tensors).

Given two Hermitian tensors
$\mathcal{H}_{1}\in\mathbb{C}^{I_{1}\times\cdots\times I_{M}\times
I_{1}\times\cdots\times I_{M}}$ and
$\mathcal{H}_{2}\in\mathbb{C}^{I_{1}\times\cdots\times I_{M}\times
I_{1}\times\cdots\times I_{M}}$, we have

(50)
$\displaystyle\mathrm{Tr}(\exp^{\mathcal{H}_{1}+\mathcal{H}_{2}})\leq\mathrm{Tr}(\exp^{\mathcal{H}_{1}}\star_{M}e^{\mathcal{H}_{2}}).$

Proof:

Let $\mathcal{A}_{1}=\exp(\mathcal{H}_{1})$ and
$\mathcal{A}_{2}=\exp(\mathcal{H}_{2})$, we have

(51)
$\displaystyle\log\mathrm{Tr}\left(\exp\left(\log\mathcal{A}_{1}+\log\mathcal{A}_{2}\right)\right)$
$\displaystyle=_{1}$
$\displaystyle\frac{1}{m}\log\mathrm{Tr}\left(\exp\left(\log\mathcal{A}^{\otimes
m}_{1}+\log\mathcal{A}^{\otimes m}_{2}\right)\right)$ $\displaystyle\leq_{2}$
$\displaystyle\frac{1}{m}\log\mathrm{Tr}\left(\exp\left(\log\mathfrak{P}_{\mathcal{A}^{\otimes
m}_{2}}(\mathcal{A}_{1}^{\otimes m})+\log\mathcal{A}^{\otimes
m}_{2}\right)\right)$ $\displaystyle+\frac{\log\mbox{poly}(m)}{m}$
$\displaystyle=_{3}$
$\displaystyle\frac{1}{m}\log\left(\mathrm{Tr}\left(\mathfrak{P}_{\mathcal{A}^{\otimes
m}_{2}}(\mathcal{A}_{1}^{\otimes m})\star_{M}\mathcal{A}^{\otimes
m}_{2}\right)\right)+\frac{\log\mbox{poly}(m)}{m}$ $\displaystyle=_{4}$
$\displaystyle\log\mathrm{Tr}\left(\mathcal{A}_{1}\star_{M}\mathcal{A}_{2}\right)+\frac{\log\mbox{poly}(m)}{m}.$

The equality $=_{1}$ comes from Lemmas 2.9 and 2.10. The inequality $\leq_{2}$
follows from pinching inequality (Lemma 3.7), the monotone of $\log$ and
$\mathrm{Tr}\exp\left(~{}\right)$ functions, and the number of eigenvalues of
$\mathcal{A}_{2}^{\otimes m}$ growing polynomially with $m$ (Lemma 3.1). The
equality $=_{3}$ utilizes the commutativity property for tensors
$\mathfrak{P}_{\mathcal{A}^{\otimes m}_{2}}(\mathcal{A}_{1}^{\otimes m})$ and
$\mathcal{A}^{\otimes m}_{2}$ based on Lemma 3.5. Finally, the equality
$=_{4}$ applies trace properties from Lemmas 2.9, 2.10, and Lemma 3.6. If
$m\rightarrow\infty$, the result of this theorem is established. $\hfill\Box$

######  Theorem 3.9 (Araki-Lieb-Thirring for tensors).

Given two positive semi-definite tensors
$\mathcal{A}_{1}\in\mathbb{C}^{I_{1}\times\cdots\times I_{M}\times
I_{1}\times\cdots\times I_{M}}$ and
$\mathcal{A}_{2}\in\mathbb{C}^{I_{1}\times\cdots\times I_{M}\times
I_{1}\times\cdots\times I_{M}}$, and $q>0$, then

(52)
$\displaystyle\mathrm{Tr}\left(\left(\mathcal{A}^{\frac{r}{2}}_{1}\mathcal{A}^{r}_{2}\mathcal{A}^{\frac{r}{2}}_{1}\right)^{\frac{q}{r}}\right)\leq\mathrm{Tr}\left(\left(\mathcal{A}^{\frac{1}{2}}_{1}\mathcal{A}_{2}\mathcal{A}^{\frac{1}{2}}_{1}\right)^{q}\right),\mbox{~{}~{}~{}~{}if
$r\in(0,1]$},$

with equality if and only if
$\mathcal{A}_{1}\star_{M}\mathcal{A}_{2}=\mathcal{A}_{2}\star_{M}\mathcal{A}_{1}$.
This inequality holds in the opposite direction for $r\geq 1$.

Proof:

Since we have

$\displaystyle\log\mathrm{Tr}\left(\left(\mathcal{A}^{\frac{r}{2}}_{1}\mathcal{A}^{r}_{2}\mathcal{A}^{\frac{r}{2}}_{1}\right)^{\frac{q}{r}}\right)=_{1}\frac{1}{m}\log\mathrm{Tr}\left(\left((\mathcal{A}^{\frac{r}{2}}_{1})^{\otimes
m}(\mathcal{A}^{r}_{2})^{\otimes m}(\mathcal{A}^{\frac{r}{2}}_{1})^{\otimes
m}\right)^{\frac{q}{r}}\right)$
$\displaystyle\leq_{2}\frac{1}{m}\log\mathrm{Tr}\left(\left((\mathcal{A}^{\frac{r}{2}}_{1})^{\otimes
m}\mathfrak{P}_{\mathcal{A}^{\otimes m}_{1}}((\mathcal{A}^{r}_{2})^{\otimes
m})(\mathcal{A}^{\frac{r}{2}}_{1})^{\otimes
m}\right)^{\frac{q}{r}}\right)+\frac{\log\mbox{poly}(m)}{m}$
$\displaystyle=_{3}\frac{1}{m}\log\mathrm{Tr}\left(\left(\mathfrak{P}_{\mathcal{A}^{\otimes
m}_{1}}\left((\mathcal{A}^{\frac{1}{2}}_{1})^{\otimes
m}(\mathcal{A}_{2})^{\otimes m}(\mathcal{A}^{\frac{1}{2}}_{1})^{\otimes
m}\right)\right)^{q}\right)+\frac{\log\mbox{poly}(m)}{m}$
$\displaystyle\leq_{4}\frac{1}{m}\log\mathrm{Tr}\left(\left((\mathcal{A}^{\frac{1}{2}}_{1})^{\otimes
m}(\mathcal{A}_{2})^{\otimes m}(\mathcal{A}^{\frac{1}{2}}_{1})^{\otimes
m}\right)^{q}\right)+\frac{\log\mbox{poly}(m)}{m}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}$
(53)
$\displaystyle=_{5}\log\mathrm{Tr}\left(\mathcal{A}^{\frac{1}{2}}_{1}\mathcal{A}_{2}\mathcal{A}^{\frac{1}{2}}_{1}\right)^{q}+\frac{\log\mbox{poly}(m)}{m}.~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}$

The equality $=_{1}$ comes from Lemmas 2.9 and 2.10. The inequality $\leq_{2}$
follows from pinching inequality (Lemma 3.7), the monotone of
$\mathcal{X}\rightarrow\mathrm{Tr}(\mathcal{X}^{\alpha})$ function for
$\alpha\geq 0$, and the number of eigenvalues of $\mathcal{A}_{1}^{\otimes m}$
growing polynomially with $m$ (Lemma 3.1). The equality $=_{3}$ utilizes the
commutativity property for tensors $\mathfrak{P}_{\mathcal{A}^{\otimes
m}_{1}}(\mathcal{A}_{2}^{\otimes m})$ and $\mathcal{A}^{\otimes m}_{1}$ based
on Lemma 3.5. The inequality $\leq_{4}$ utilizes Lemma 3.2, integral
representation of the pinching map (see Lemma 3.4) and the fact that $p$-norms
are unitary invariant for $p\geq 0$. Finally, the equality $=_{5}$ applies
trace properties from Lemmas 2.9 and 2.10. If $m\rightarrow\infty$, the result
of this theorem is established.

For case $r\geq 1$, if we perform following replacements
$\mathcal{A}^{r}_{1}\leftarrow\mathcal{A}_{1}$,
$\mathcal{A}^{r}_{2}\leftarrow\mathcal{A}_{2}$, $\frac{q}{r}\leftarrow q$, and
$\frac{1}{r}\leftarrow r$, the inequality in this theorem will be reversed.
$\hfill\Box$

###  3.2 Complex Interpolation Theory

In this section, we will mention those definitions and theorems about complex
interpolation theory which will be used to prove multivariate tensor trace
inequalities in Sec. 4. Complex interpolation theory enable us to control the
behaviors of the complex function defined on the strip
$S\stackrel{{\scriptstyle\text{\tiny
def}}}{{=}}\\{z\in\mathbb{C}:0\leq\Re(z)\leq 1\\}$ by its boundary values,
$\Re(z)=0$ and $\Re(z)=1$. We define a family of probability measure on
$\mathbb{R}$ as

(54) $\displaystyle\rho_{\theta}(s)\stackrel{{\scriptstyle\text{\tiny
def}}}{{=}}\frac{\sin(\pi\theta)}{2\theta(\cosh(\pi
s)+\cos(\pi\theta))}\mbox{~{}~{}~{}~{}~{}~{}~{}~{} for $\theta\in(0,1)$}.$

Moreover, we have following limiting behaviors for $\rho_{\theta}$:

(55) $\displaystyle\rho_{0}(s)\stackrel{{\scriptstyle\text{\tiny
def}}}{{=}}\lim_{\theta\searrow 0}\rho_{\theta}(s)$ $\displaystyle=$
$\displaystyle\frac{\pi}{2}(\cosh(\pi s)+1)^{-1},$

and

(56) $\displaystyle\rho_{1}(s)\stackrel{{\scriptstyle\text{\tiny
def}}}{{=}}\lim_{\theta\nearrow 1}\rho_{\theta}(s)$ $\displaystyle=$
$\displaystyle\delta(s),$

where $\delta$ is the Dirac $\delta$-distribution.

We will introduce Stein-Hirschman theorem [9, 23] about complex interpolation
theory.

######  Theorem 3.10.

Let $p_{0},p_{1}\in[1,\infty]$, $\theta\in(0,1)$, $\rho_{\theta}(s)$ defined
by Eq. (54), define $p_{\theta}$ by
$\frac{1}{p_{\theta}}=\frac{1-\theta}{p_{0}}+\frac{\theta}{p_{1}}$, and
$S\stackrel{{\scriptstyle\text{\tiny
def}}}{{=}}\\{z\in\mathbb{C}:0\leq\Re(z)\leq 1\\}$. Let $F$ be a map from $S$
to bounded linear operators on a separable Hilbert space that is holomorphic
in the interior of $S$ and continuous on the boundary. If
$z\rightarrow\left\|F(z)\right\|_{p\Re(z)}$ is uniformly bounded on $S$, we
have

(57)
$\displaystyle~{}~{}~{}\log\left\|F(\theta)\right\|_{p(\theta)}\leq\int_{-\infty}^{\infty}\left(\rho_{1-\theta}(s)\log\left\|F(\iota
s)\right\|^{1-\theta}_{p_{0}}+\rho_{\theta}(s)\log\left\|F(1+\iota
s)\right\|^{\theta}_{p_{1}}\right)ds$

##  4 Multivariate Tensor Trace Inequalities

In order to extend Theorems 3.8 and 3.9 involving two tensors to multiple
tensors, we require the following lemma about _Lie product formula_ for
tensors.

######  Lemma 4.1.

Let $m\in\mathbb{N}$ and $(\mathcal{L}_{k})_{k=1}^{m}$ be a finite sequence of
bounded tensors with dimensions
$\mathcal{L}_{k}\in\mathbb{C}^{I_{1}\times\cdots\times I_{M}\times
I_{1}\times\cdots\times I_{M}}$, then we have

(58)
$\displaystyle\lim_{n\rightarrow\infty}\left(\prod_{k=1}^{m}\exp(\frac{\mathcal{L}_{k}}{n})\right)^{n}$
$\displaystyle=$ $\displaystyle\exp\left(\sum_{k=1}^{m}\mathcal{L}_{k}\right)$

Proof:

We will prove the case for $m=2$, and the general value of $m$ can be obtained
by mathematical induction. Let $\mathcal{L}_{1},\mathcal{L}_{2}$ be bounded
tensors act on some Hilbert space. Define
$\mathcal{C}\stackrel{{\scriptstyle\text{\tiny
def}}}{{=}}\exp((\mathcal{L}_{1}+\mathcal{L}_{2})/n)$, and
$\mathcal{D}\stackrel{{\scriptstyle\text{\tiny
def}}}{{=}}\exp(\mathcal{L}_{1}/n)\star_{M}\exp(\mathcal{L}_{2}/n)$. Note we
have following estimates for the norm of tensors $\mathcal{C},\mathcal{D}$:

(59)
$\displaystyle\left\|\mathcal{C}\right\|,\left\|\mathcal{D}\right\|\leq\exp\left(\frac{\left\|\mathcal{L}_{1}\right\|+\left\|\mathcal{L}_{2}\right\|}{n}\right)=\left[\exp\left(\left\|\mathcal{L}_{1}\right\|+\left\|\mathcal{L}_{2}\right\|\right)\right]^{1/n}.$

From the Cauchy-Product formula, the tensor $\mathcal{D}$ can be expressed as:

$\displaystyle\mathcal{D}=\exp(\mathcal{L}_{1}/n)\star_{M}\exp(\mathcal{L}_{2}/n)=\sum_{i=0}^{\infty}\frac{(\mathcal{L}_{1}/n)^{i}}{i!}\star_{M}\sum_{j=0}^{\infty}\frac{(\mathcal{L}_{2}/n)^{j}}{j!}$
(60)
$\displaystyle=\sum_{l=0}^{\infty}n^{-l}\sum_{i=0}^{l}\frac{\mathcal{L}_{1}^{i}}{i!}\star_{M}\frac{\mathcal{L}_{2}^{l-i}}{(l-i)!},$

then we can bound the norm of $\mathcal{C}-\mathcal{D}$ as

(61) $\displaystyle\left\|\mathcal{C}-\mathcal{D}\right\|$ $\displaystyle=$
$\displaystyle\left\|\sum_{i=0}^{\infty}\frac{\left([\mathcal{L}_{1}+\mathcal{L}_{2}]/n\right)^{i}}{i!}-\sum_{l=0}^{\infty}n^{-l}\sum_{i=0}^{l}\frac{\mathcal{L}_{1}^{i}}{i!}\star_{M}\frac{\mathcal{L}_{2}^{l-i}}{(l-i)!}\right\|$
$\displaystyle=$
$\displaystyle\left\|\sum_{i=2}^{\infty}k^{-i}\frac{\left([\mathcal{L}_{1}+\mathcal{L}_{2}]\right)^{i}}{i!}-\sum_{m=l}^{\infty}n^{-l}\sum_{i=0}^{l}\frac{\mathcal{L}_{1}^{i}}{i!}\star_{M}\frac{\mathcal{L}_{2}^{l-i}}{(l-i)!}\right\|$
$\displaystyle\leq$
$\displaystyle\frac{1}{k^{2}}\left[\exp(\left\|\mathcal{L}_{1}\right\|+\left\|\mathcal{L}_{2}\right\|)+\sum_{l=2}^{\infty}n^{-l}\sum_{i=0}^{l}\frac{\left\|\mathcal{L}_{1}\right\|^{i}}{i!}\cdot\frac{\left\|\mathcal{L}_{2}\right\|^{l-i}}{(l-i)!}\right]$
$\displaystyle=$
$\displaystyle\frac{1}{n^{2}}\left[\exp\left(\left\|\mathcal{L}_{1}\right\|+\left\|\mathcal{L}_{2}\right\|\right)+\sum_{l=2}^{\infty}n^{-l}\frac{(\left\|\mathcal{L}_{1}\right\|+\left\|\mathcal{L}_{2}\right\|)^{l}}{l!}\right]$
$\displaystyle\leq$
$\displaystyle\frac{2\exp\left(\left\|\mathcal{L}_{1}\right\|+\left\|\mathcal{L}_{2}\right\|\right)}{n^{2}}.$

For the difference between the higher power of $\mathcal{C}$ and
$\mathcal{D}$, we can bound them as

(62) $\displaystyle\left\|\mathcal{C}^{n}-\mathcal{D}^{n}\right\|$
$\displaystyle=$
$\displaystyle\left\|\sum_{l=0}^{n-1}\mathcal{C}^{m}(\mathcal{C}-\mathcal{D})\mathcal{C}^{n-l-1}\right\|$
$\displaystyle\leq_{1}$
$\displaystyle\exp(\left\|\mathcal{L}_{1}\right\|+\left\|\mathcal{L}_{2}\right\|)\cdot
n\cdot\left\|\mathcal{L}_{1}-\mathcal{L}_{2}\right\|,$

where the inequality $\leq_{1}$ uses the following fact

(63)
$\displaystyle\left\|\mathcal{C}\right\|^{l}\left\|\mathcal{D}\right\|^{n-l-1}\leq\exp\left(\left\|\mathcal{L}_{1}\right\|+\left\|\mathcal{L}_{2}\right\|\right)^{\frac{n-1}{n}}\leq\exp\left(\left\|\mathcal{L}_{1}\right\|+\left\|\mathcal{L}_{2}\right\|\right),$

based on Eq. (59). By combining with Eq. (61), we have the following bound

(64) $\displaystyle\left\|\mathcal{C}^{n}-\mathcal{D}^{n}\right\|$
$\displaystyle\leq$
$\displaystyle\frac{2\exp\left(2\left\|\mathcal{L}_{1}\right\|+2\left\|\mathcal{L}_{2}\right\|\right)}{n}.$

Then this lemma is proved when $n$ goes to infity. $\hfill\Box$

###  4.1 Multivariate Araki-Lieb-Thirring Inequality

In this section, we will provide a theorem for multivariate Araki-Lieb-
Thirring (ALT) inequality for tensors.

######  Theorem 4.2.

Let $p\geq 1$, $\theta\in(0,1]$, probability distribution $\rho_{\theta}$
defined by (54), $n\in\mathbb{N}$, and consider a finite sequence
$(\mathcal{A})_{k=1}^{n}$ of positive semi-definite tensors. Then, we have

(65)
$\displaystyle\log\left\|\left|\prod_{k=1}^{n}\mathcal{A}_{k}^{\theta}\right|^{\frac{1}{\theta}}\right\|_{p}\leq\int_{-\infty}^{\infty}\log\left\|\prod_{k=1}^{n}\mathcal{A}_{k}^{1+\iota
s}\right\|_{p}\rho_{\theta}(s)ds.$

Proof:

For $\theta=1$, the both sides of Eq. (65) are equal to
$\log\left\|\left|\prod_{k=1}^{n}\mathcal{A}_{k}\right|\right\|_{p}$. We will
prove the cases for $\theta\in(0,1)$. We prove the result for strictly
positive definite tensors and note that the generalization to positive semi-
definite tensors follows by continuity. We define the function
$F(z)\stackrel{{\scriptstyle\text{\tiny
def}}}{{=}}\prod_{k=1}^{n}\mathcal{A}_{k}^{z}=\prod_{k=1}^{n}\exp(z\log\mathcal{A}_{k})$
which satisfies the conditions of Theorem 3.10. By selecting $p_{0}=\infty$,
$p_{1}=p$, and $p_{\theta}=\frac{p}{\theta}$ in Theorem 3.10, one can obtain

(66) $\displaystyle\log\left\|F(1+\iota
s)\right\|_{p_{1}}^{\theta}=\theta\log\left\|\prod_{k=1}^{n}\mathcal{A}_{k}^{1+\iota
s}\right\|_{p},$

and

(67) $\displaystyle\log\left\|F(\iota
s)\right\|_{p_{0}}^{1-\theta}=(1-\theta)\log\left\|\prod_{k=1}^{n}\mathcal{A}_{k}^{\iota
s}\right\|_{\infty}=0,$

since tensors $\mathcal{A}_{k}^{\iota s}$ are unitary. We also have

(68)
$\displaystyle\log\left\|F(\theta)\right\|_{p_{\theta}}=\log\left\|\prod_{k=1}^{n}\mathcal{A}_{k}^{\theta}\right\|_{\frac{p}{\theta}}=\theta\log\left\|\left|\prod_{k=1}^{n}\mathcal{A}_{k}^{\theta}\right|^{\frac{1}{\theta}}\right\|_{p},$

and this theorem is proved by putting Eqs. (66), (67), (68) into Eq. (57).
$\hfill\Box$

###  4.2 Multivariate Golden-Thompson Inequality

In this section, we will provide a theorem for multivariate Golden-Thompson
(GT) inequality for tensors.

######  Theorem 4.3.

Let $p\geq 1$, probability distribution $\rho_{0}$ defined by (55),
$n\in\mathbb{N}$, and consider a finite sequence $(\mathcal{H})_{k=1}^{n}$ of
Hermitian tensors. Then, we have

(69)
$\displaystyle\log\left\|\exp\left(\sum_{k=1}^{n}\mathcal{H}_{k}\right)\right\|_{p}\leq\int_{-\infty}^{\infty}\log\left\|\prod_{k=1}^{n}\exp\left((1+\iota
s)\mathcal{H}_{k}\right)\right\|_{p}\rho_{0}(s)ds.$

Proof: From Theorem 4.1 and Lie product formula given by Lemma 4.1, this
theorem is proved by taking $\theta\rightarrow 0$ in Eq. (65). $\hfill\Box$

The multivariate Golden-Thompson inequality provided by Theorem 4.3 is only
true for Hermitian tensors. The following theorem generalizes Theorem 4.3 to
general tensors.

######  Theorem 4.4.

Let $p\geq 1$, probability distribution $\rho_{0}$ defined by (55),
$n\in\mathbb{N}$, and consider a finite sequence $(\mathcal{A})_{k=1}^{n}$ of
tensors. Then, we have

(70)
$\displaystyle\log\left\|\exp\left(\sum_{k=1}^{n}\mathcal{A}_{k}\right)\right\|_{p}\leq\int_{-\infty}^{\infty}\log\left\|\prod_{k=1}^{n}\exp\left((1+\iota
s)\Re(\mathcal{A}_{k})\right)\right\|_{p}\rho_{0}(s)ds,$

where $\Re(\mathcal{A}_{k})$ is the real part of the tensor $\mathcal{A}_{k}$
defined as $\Re(\mathcal{A}_{k})\stackrel{{\scriptstyle\text{\tiny
def}}}{{=}}\frac{1}{2}(\mathcal{A}_{k}+\mathcal{A}^{H}_{k})$.

Proof: We also define the imaginary part of the tensor $\mathcal{A}_{k}$ as
$\Im(\mathcal{A}_{k})\stackrel{{\scriptstyle\text{\tiny
def}}}{{=}}\frac{1}{2\iota}(\mathcal{A}_{k}-\mathcal{A}^{H}_{k})$ and note
that the both $\Re(\mathcal{A}_{k})$ and $\Im(\mathcal{A}_{k})$ are Hermitian
tensors. We define the function $F(z)\stackrel{{\scriptstyle\text{\tiny
def}}}{{=}}\prod_{k=1}^{n}\exp(z\Re(\mathcal{A}_{k})+\iota\theta\Im(\mathcal{A}_{k})$
which satisfies the conditions of Theorem 3.10. By selecting $p_{0}=\infty$,
$p_{1}=p$, and $p_{\theta}=\frac{p}{\theta}$ in Theorem 3.10, one can obtain

(71)
$\displaystyle\theta\left\|\left|\exp\left(\theta\sum_{k=1}^{n}\mathcal{A}_{k}\right)\right|^{\frac{1}{\theta}}\right\|_{p}$
$\displaystyle=$ $\displaystyle\log\left\|F(\theta)\right\|_{p_{\theta}}$
$\displaystyle\leq$ $\displaystyle\int_{-\infty}^{\infty}\log\left\|F(1+\iota
s)\right\|^{\theta}_{p}\rho_{\theta}(s)ds$ $\displaystyle=$
$\displaystyle\theta\int_{-\infty}^{\infty}\log\left\|\prod_{k=1}^{n}\exp((1+\iota
s)\Re(\mathcal{A}_{k})+\iota\theta\Im(\mathcal{A}_{k}))\right\|_{p}\rho_{\theta}(s)ds$

where we used that $\log\left\|F(\iota s)\right\|_{\infty}=0$ since $F(\iota
s)$ is unitary in the inequality step. By dividing $\theta$ at both sides of
Eq. (71) and taking $\theta\rightarrow 0$, the theorem is proved by applying
Lie product formula given by Lemma 4.1 again. $\hfill\Box$

###  4.3 Multivariate Logarithmic Trace Inequality for Tensors

In this section, we will apply Theorem 4.3 to prove multivariate logarithmic
trace inequality. We have to define relative entropy between two tensors
first.

######  Definition 4.5.

Given two positive definite tensors
$\mathcal{A}\in\mathbb{C}^{I_{1}\times\cdots\times I_{M}\times
I_{1}\times\cdots\times I_{M}}$ and tensor
$\mathcal{B}\in\mathbb{C}^{I_{1}\times\cdots\times I_{M}\times
I_{1}\times\cdots\times I_{M}}$, where the tensor $\mathcal{A}$ has the trace
equal to one. The relative entropy between tensors $\mathcal{A}$ and
$\mathcal{B}$ is defined as

(72) $\displaystyle
D(\mathcal{A}\parallel\mathcal{B})\stackrel{{\scriptstyle\text{\tiny
def}}}{{=}}\mathrm{Tr}\mathcal{A}\star_{M}(\log\mathcal{A}-\log\mathcal{B}).$

Based on this relative entropy definition, we have the following lemma about
variational expression of relative entropy.

######  Lemma 4.6.

Given two positive definite tensors
$\mathcal{A}\in\mathbb{C}^{I_{1}\times\cdots\times I_{M}\times
I_{1}\times\cdots\times I_{M}}$ and tensor
$\mathcal{B}\in\mathbb{C}^{I_{1}\times\cdots\times I_{M}\times
I_{1}\times\cdots\times I_{M}}$, where the tensor $\mathcal{A}$ has the trace
equal to one. Then, we have

(73) $\displaystyle D(\mathcal{A}\parallel\mathcal{B})$ $\displaystyle=$
$\displaystyle\sup_{\mathcal{X}}\left(\mathrm{Tr}\left(\mathcal{A}\star_{M}\log\mathcal{X}\right)-\log\mathrm{Tr}(\exp(\log\mathcal{B}+\log\mathcal{X}))\right),$

and

(74) $\displaystyle D(\mathcal{A}\parallel\mathcal{B})$ $\displaystyle=$
$\displaystyle\sup_{\mathcal{X}}\left(\mathrm{Tr}\left(\mathcal{A}\star_{M}\log\mathcal{X}\right)+1-\mathrm{Tr}(\exp(\log\mathcal{B}+\log\mathcal{X}))\right),$

where $\mathcal{X}\in\mathbb{C}^{I_{1}\times\cdots\times I_{M}\times
I_{1}\times\cdots\times I_{M}}$ is a positive definite tensor.

Proof:

For any Hermitian $\mathcal{H}$ tensor with dimensions
$\mathcal{H}\in\mathbb{C}^{I_{1}\times\cdots\times I_{M}\times
I_{1}\times\cdots\times I_{M}}$, we first show that

(75)
$\displaystyle\log\mathrm{Tr}(e^{\mathcal{H}+\log\mathcal{B}})=\sup_{\mathcal{A}}\left(\mathrm{Tr}(\mathcal{A}\mathcal{H})-D(\mathcal{A}\parallel\mathcal{B})\right).$

We define a function with tensor argument as
$g(\mathcal{A})=\mathrm{Tr}(\mathcal{A}\mathcal{H})-D(\mathcal{A}\parallel\mathcal{B})$,
then let $\mathcal{A}=\sum_{\lambda\in
sp(\mathcal{A})}\lambda\mathcal{U}_{\lambda}$ denote the spectrum
decomposition of $\mathcal{A}$. Because the trace of $\mathcal{A}$ is one, we
have

(76) $\displaystyle g\left(\sum_{\lambda\in
sp(\mathcal{A})}\lambda\mathcal{U}_{\lambda}\right)$ $\displaystyle=$
$\displaystyle\sum_{\lambda\in
sp(\mathcal{A})}\left(\lambda\mathrm{Tr}\mathcal{U}_{\lambda}\mathcal{H}+\lambda\mathrm{Tr}\mathcal{U}_{\lambda}\log\mathcal{B}-\lambda\log\lambda\right).$

By taking derivative with respect to $\lambda$ for Eq. (76), we have

(77)
$\displaystyle\left.\frac{\partial}{\partial\lambda}g\left(\sum_{\lambda\in
sp(\mathcal{A})}\lambda\mathcal{U}_{\lambda}\right)\right|_{\lambda=0}=\infty,$

this shows that the minimizer for Eq. (75) is a strictly positive tensors
$\tilde{\mathcal{A}}$ with $\mathrm{Tr}\tilde{\mathcal{A}}=1$. For any
Hermitian tensor $\mathcal{Y}$ with $\mathrm{Tr}\tilde{\mathcal{Y}}=0$, we
have

(78) $\displaystyle
0=\left.\frac{dg(\tilde{\mathcal{A}}+t\mathcal{Y})}{dt}\right|_{t=0}=\mathrm{Tr}\left[\mathcal{Y}(\mathcal{H}+\log\mathcal{B}-\log\tilde{\mathcal{A}})\right].$

This indicates that $\mathcal{H}+\log\mathcal{B}-\log\tilde{\mathcal{A}}$ is
proportional to the identity tensor. Then, we will have

(79)
$\displaystyle\tilde{\mathcal{A}}=\frac{e^{\mathcal{H}+\log\mathcal{B}}}{\mathrm{Tr}e^{\mathcal{H}+\log\mathcal{B}}}~{}~{}\mbox{and}~{}~{}g(\tilde{\mathcal{A}})=\log\mathrm{Tr}e^{\mathcal{H}+\log\mathcal{B}},$

which proves Eq. (75).

We first prove Eq. (73) based on Eq. (75). Because Eq. (75) implies that the
functional
$\mathcal{H}\rightarrow\log\mathrm{Tr}e^{\mathcal{H}+\log\mathcal{X}}$ is
convex, then let $\tilde{\mathcal{H}}=\log\mathcal{A}-\log\mathcal{B}$, we can
have a concave function

(80) $\displaystyle f(\mathcal{H})\stackrel{{\scriptstyle\text{\tiny
def}}}{{=}}\mathrm{Tr}\mathcal{A}\mathcal{H}-\log\mathrm{Tr}e^{\mathcal{H}+\log\mathcal{B}}.$

For any Hermitian tensor $\mathcal{Y}$, we have

(81)
$\displaystyle\left.\frac{df(\tilde{\mathcal{H}}+t\mathcal{Y})}{dt}\right|_{t=0}=0,$

because $\mathrm{Tr}\mathcal{A}=1$ and
$\frac{d\mathrm{Tr}e^{\log\mathcal{A}+t\mathcal{Y}}}{dt}|_{t=0}=\mathrm{Tr}\mathcal{A}\mathcal{Y}$.
Therefore, the tensor $\tilde{\mathcal{H}}$ is the maximizer of function $f$
and

(82) $\displaystyle
f(\tilde{\mathcal{H}})=\mathrm{Tr}\mathcal{A}(\log\mathcal{A}-\log\mathcal{B})=D(\mathcal{A}\parallel\mathcal{B}).$

Since for any any Hermitian tensor $\mathcal{H}$ can be expressed as
$\mathcal{H}=\log\mathcal{X}$ for some positive semi-definite tensor, we
proved Eq. (73).

Now, we are ready to prove Eq. (74). From $\log x\leq x-1$ for $x\geq 0$, we
have
$\log\mathrm{Tr}e^{\log\mathcal{B}+\log\mathcal{X}}\leq\mathrm{Tr}e^{\log\mathcal{B}+\log\mathcal{X}}-1$.
Hence, we have

(83)
$\displaystyle\sup_{\mathcal{X}}(\mathrm{Tr}\mathcal{A}\log\mathcal{X}-\log\mathrm{Tr}e^{\log\mathcal{B}+\log\mathcal{X}})\geq\sup_{\mathcal{X}}(\mathrm{Tr}\mathcal{A}\log\mathcal{X}+1-\mathrm{Tr}e^{\log\mathcal{B}+\log\mathcal{X}}).$

Because
$\mathrm{Tr}\mathcal{A}\log\mathcal{X}-\log\mathrm{Tr}e^{\log\mathcal{B}+\log\mathcal{X}}$
is invariant under the scaling transform from $\mathcal{X}$ to
$\gamma\mathcal{X}$ for $\gamma\in\mathbb{R}_{+}$, we can assume that
$\mathrm{Tr}e^{\log\mathcal{B}+\log\mathcal{X}}=1$. Then, we have

(84)
$\displaystyle\sup_{\mathcal{X}}(\mathrm{Tr}\mathcal{A}\log\mathcal{X}-\log\mathrm{Tr}e^{\log\mathcal{B}+\log\mathcal{X}})$
$\displaystyle=$
$\displaystyle\sup_{\mathcal{X}}\left(\mathrm{Tr}\mathcal{A}\log\mathcal{X}-\log\mathrm{Tr}e^{\log\mathcal{B}+\log\mathcal{X}}:\right.$
$\displaystyle\left.\mathrm{Tr}e^{\log\mathcal{B}+\log\mathcal{X}}=1\right)$
$\displaystyle\leq$
$\displaystyle\sup_{\mathcal{X}}(\mathrm{Tr}\mathcal{A}\log\mathcal{X}-1+\mathrm{Tr}e^{\log\mathcal{B}+\log\mathcal{X}}).$

From both Eqs (83) and (84), we prove Eq. (74). $\hfill\Box$

We are ready to present multivariate logarithmic trace inequality for tensors
by the following theorem.

######  Theorem 4.7.

Let $0<q\leq 1$, probability distribution $\rho_{0}$ defined by (55),
$n\in\mathbb{N}$, and consider a finite sequence $(\mathcal{A})_{k=1}^{n}$ of
positive semi-definite tensors. Then, we have

$\displaystyle\sum_{k=1}^{n}\mathrm{Tr}\mathcal{A}_{1}\log\mathcal{A}_{k}\geq~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}$
(85)
$\displaystyle~{}~{}~{}\frac{1}{q}\int_{-\infty}^{\infty}\left(\mathrm{Tr}\mathcal{A}_{1}\log\mathcal{A}^{\frac{q(1+\iota
s)}{2}}_{n}\cdots\mathcal{A}^{\frac{q(1+\iota
s)}{2}}_{3}\mathcal{A}_{2}^{\frac{q}{2}}\mathcal{A}_{1}^{q}\mathcal{A}_{2}^{\frac{q}{2}}\mathcal{A}_{3}^{\frac{q(1-\iota
s)}{2}}\cdots\mathcal{A}_{n}^{\frac{q(1-\iota s)}{2}}\right)\rho_{0}(s)ds,$

which the equality will be valid when $q\rightarrow 0$.

Proof: Because the inequality given by Eq. (4.7) is invariant under
multiplication of the tensors
$\mathcal{A}_{1},\mathcal{A}_{2},\cdots,\mathcal{A}_{n}$ with positive numbers
$a_{1},a_{2},\cdots,a_{n}$, we can add constraints on the norms of tensors
without loss of generality. We assume that the $\mathrm{Tr}\mathcal{A}_{1}=1$.

From the relative entropy in Definition 4.5, we have

(86)
$\displaystyle\sum_{k=1}^{n}\mathrm{Tr}\mathcal{A}_{1}\log\mathcal{A}_{k}$
$\displaystyle=$ $\displaystyle
D(\mathcal{A}_{1}\parallel\exp(\sum_{k=2}^{n}\log\mathcal{A}^{-1}_{k}))$
$\displaystyle=$
$\displaystyle\sup_{\mathcal{X}}\left(\mathrm{Tr}\mathcal{A}_{1}\log\mathcal{X}+1-\mathrm{Tr}\exp\left(\log\mathcal{X}-\sum_{k=2}^{n}\log\mathcal{A}_{k}\right)\right),$

where we apply Lemma 4.6. From Theorem 4.4 and set
$\mathcal{H}_{k}=\log\mathcal{A}^{q}_{k}$, we have

$\displaystyle\mathrm{Tr}\exp\left(\sum_{k=1}^{n}\log\mathcal{A}_{k}\right)\leq~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}$
(87)
$\displaystyle\int_{-\infty}^{\infty}\mathrm{Tr}\left(\mathcal{A}^{\frac{q(1+\iota
s)}{2}}_{n}\cdots\mathcal{A}^{\frac{q(1+\iota
s)}{2}}_{3}\mathcal{A}_{2}^{\frac{q(1+\iota
s)}{2}}\mathcal{A}_{1}^{q}\mathcal{A}_{2}^{\frac{q(1-\iota
s)}{2}}\mathcal{A}_{3}^{\frac{q(1-\iota
s)}{2}}\cdots\mathcal{A}_{n}^{\frac{q(1-\iota
s)}{2}}\right)^{\frac{1}{q}}\rho_{0}(s)ds$

using the concavity of the logarithm and Jesen’s inequality. Applying Eq.
(4.3) to Eq. (86), we get

$\displaystyle\sum_{k=1}^{n}\mathrm{Tr}\mathcal{A}_{1}\log\mathcal{A}_{k}\geq\sup_{\mathcal{X}}\left(\int_{-\infty}^{\infty}\left(\mathrm{Tr}\mathcal{A}_{1}\log\mathcal{X}\right)\rho_{0}(s)ds+1\right.~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}$
(88)
$\displaystyle\left.-\mathrm{Tr}\left(\mathcal{A}^{\frac{-q}{2}}_{2}\mathcal{A}^{\frac{-q(1+\iota
s)}{2}}_{3}\cdots\mathcal{A}^{\frac{-q(1+\iota
s)}{2}}_{n}\mathcal{X}^{q}\mathcal{A}_{n}^{\frac{-q(1-\iota
s)}{2}}\cdots\mathcal{A}_{3}^{\frac{-q(1-\iota
s)}{2}}\mathcal{A}_{2}^{\frac{-q}{2}}\right)^{\frac{1}{q}}\right).$

If we set the tensor $\mathcal{X}$ as

(89) $\displaystyle\mathcal{X}\stackrel{{\scriptstyle\text{\tiny
def}}}{{=}}\left(\mathcal{A}^{\frac{q(1+\iota
s)}{2}}_{n}\cdots\mathcal{A}^{\frac{q(1+\iota
s)}{2}}_{3}\mathcal{A}_{2}^{\frac{q}{2}}\mathcal{A}_{1}^{q}\mathcal{A}_{2}^{\frac{q}{2}}\mathcal{A}_{3}^{\frac{q(1-\iota
s)}{2}}\cdots\mathcal{A}_{n}^{\frac{q(1-\iota s)}{2}}\right)^{\frac{1}{q}},$

the tensor $\mathcal{X}$ becomes a positive semi-definite tensor. Substituting
Eq. (89) into Eq. (4.3), this theorem is proved for $0<q\leq 1$.

For $q\rightarrow 0$, we wish to prove the equality at Eq. (4.7). Because
$\log\mathcal{X}\geq_{Lo}\mathcal{I}-\mathcal{X}^{-1}$ for any positive tensor
$\mathcal{X}$, we have

$\displaystyle\mathrm{Tr}\mathcal{A}_{1}\log\mathcal{A}^{\frac{q(1+\iota
s)}{2}}_{n}\cdots\mathcal{A}^{\frac{q(1+\iota
s)}{2}}_{3}\mathcal{A}_{2}^{\frac{q}{2}}\mathcal{A}_{1}^{q}\mathcal{A}_{2}^{\frac{q}{2}}\mathcal{A}_{3}^{\frac{q(1-\iota
s)}{2}}\cdots\mathcal{A}_{n}^{\frac{q(1-\iota
s)}{2}}\geq~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}$
$\displaystyle\mathrm{Tr}\mathcal{A}_{1}\left(\mathcal{I}-\mathcal{A}^{\frac{-q(1-\iota
s)}{2}}_{n}\cdots\mathcal{A}^{\frac{-q(1-\iota
s)}{2}}_{3}\mathcal{A}_{2}^{\frac{-q}{2}}\mathcal{A}_{1}^{-q}\mathcal{A}_{2}^{\frac{-q}{2}}\mathcal{A}_{3}^{\frac{-q(1+\iota
s)}{2}}\cdots\mathcal{A}_{n}^{\frac{-q(1+\iota s)}{2}}\right)$ (90)
$\displaystyle\stackrel{{\scriptstyle\text{\tiny
def}}}{{=}}h_{q}(s),~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}$

and we can assume that $h_{q}(s)\geq 0$ since we can scale each tensor
$\mathcal{A}_{k}$ by a positive number for $k\in[n]$. By Fatou’s lemma, we
have

(91) $\displaystyle\liminf_{q\rightarrow
0}\int_{-\infty}^{\infty}\frac{h_{q}(s)}{q}\rho_{0}(s)ds\geq\int_{-\infty}^{\infty}\liminf_{q\rightarrow
0}\frac{h_{q}(s)}{q}\rho_{0}(s)ds.$

We also have $h_{0}(s)=0$ and

(92) $\displaystyle\liminf_{q\rightarrow
0}\frac{h_{q}(s)}{q}=\sum_{k=1}^{n}\mathrm{Tr}\mathcal{A}_{1}\log\mathcal{A}_{k}.$

By Eqs (4.3) and (91), we have the equality at Eq (4.7) as $q\rightarrow 0$.
$\hfill\Box$

##  5 Applications: Random Tensors

This section will apply multivariate Golden-Thompson inequality from Theorem
4.3 to form the tail bound for independent sum of random tensors.

Consider a random Hermitian tensor
$\mathcal{X}\in\mathbb{C}^{I_{1}\times\cdots\times I_{M}\times
I_{1}\times\cdots\times I_{M}}$, i.e., each entry in this tensor is an
independent random variable with
$x_{i_{1},\cdots,i_{M},j_{1},\cdots,j_{M}}=\overline{x_{j_{1},\cdots,j_{M},i_{1},\cdots,i_{M}}}$.
We assume that the random tensor $\mathcal{X}$ has moments of all order $n$.
We can construct tensor extensions of the moment generating function (MGF),
and the cumulant generating function (CGF):

(93) $\displaystyle\mathrm{M}(t)\stackrel{{\scriptstyle\text{\tiny
def}}}{{=}}\mathbb{E}e^{t\mathcal{X}},\mbox{~{}~{}and~{}~{}}\mathrm{C}(t)\stackrel{{\scriptstyle\text{\tiny
def}}}{{=}}\log\mathbb{E}e^{t\mathcal{X}},$

where $t\in\mathbb{R}$. The tensor MGF and CGF can be expressed as power
series expansions:

(94)
$\displaystyle\mathrm{M}(t)=\mathcal{I}+\sum_{n=1}^{\infty}\frac{t^{n}\mathbb{E}(\mathcal{X}^{n})}{n!},\mbox{~{}~{}and~{}~{}}\mathrm{C}(t)=\sum_{n=1}^{\infty}\frac{t^{n}\Phi_{n}}{n!},$

where the coefficients $\mathbb{E}(\mathcal{X}^{n})$ are called _tensor
moments_ , and $\Phi_{n}$ are named as _tensor cumulants_. The tensor
cumpulant $\Phi_{n}$ has a formal expression as a noncommutative polynomial in
the tensor moments up to order $n$. For example, the first cumulant is the
mean and the second cumulant is the variance:

(95)
$\displaystyle\Phi_{1}=\mathbb{E}\mathcal{X},\mbox{~{}~{}and~{}~{}}\Phi_{2}=\mathbb{E}(\mathcal{X}^{2})-(\mathbb{E}\mathcal{X})^{2}.$

###  5.1 Laplace Transform Method for Tensors

We will apply Laplace transform bound to bound the maximum eigenvalue of a
random Hermitian tensor by following lemma. This Lemma help us to control tail
probabilities for the maximum eigenvalue of a random tensor by producing a
bound for the trace of the tensor MGF defined in Eq. (93).

######  Lemma 5.1.

Let $\mathcal{Y}\in\mathbb{C}^{I_{1}\times\cdots\times I_{M}\times
I_{1}\times\cdots\times I_{M}}$ be a random Hermitian tensor and assume that
$|I_{i}|=N$ for $1\leq i\leq M$. For $\zeta\in\mathbb{R}$, we have

(96)
$\displaystyle\mathbb{P}(\lambda_{\max}(\mathcal{Y})\geq\zeta)\leq(2M-1)^{N}\inf_{t>0}\left(e^{-\zeta
t}\mathbb{E}\mathrm{Tr}e^{t\mathcal{Y}}\right)$

Proof:

Given a fix value $t$, we have

(97)
$\displaystyle\mathbb{P}(\lambda_{\max}(\mathcal{Y})\geq\zeta)=\mathbb{P}(\lambda_{\max}(t\mathcal{Y})\geq
t\zeta)=\mathbb{P}(e^{\lambda_{\max}(t\mathcal{Y})}\geq e^{t\zeta})\leq
e^{-t\zeta}\mathbb{E}e^{\lambda_{\max}(t\mathcal{Y})}.$

The first equality uses the homogeneity of the maximum eigenvalue map, the
second equality comes from the monotonicity of the scalar exponential
function, and the last relation is Markov’s inequality. Because we have

(98) $\displaystyle
e^{\lambda_{\max}(t\mathcal{Y})}=\lambda_{\max}(e^{t\mathcal{Y}})\leq(2M-1)^{N}\mathrm{Tr}e^{t\mathcal{Y}},$

where the first equality used the spectral mapping theorem, and the inequality
holds because the exponential of an Hermitian tensor is positive definite and
the maximum eigenvalue of a positive definite tensor is dominated by the trace
[20]. From Eqs (97) and (98), this lemma is established. $\hfill\Box$

###  5.2 Tail Bounds for Independent Sum

This section contains abstract tail bounds for the sum of independent random
tensors. This general inequality can serve as the progenitor of other random
tensors majorization inequality.

######  Theorem 5.2.

Consider $n$ independent random Hermitian tensors
$\mathcal{X}_{k}\in\mathbb{C}^{I_{1}\times\cdots\times I_{M}\times
I_{1}\times\cdots\times I_{M}}$ for $k\in[n]$, for all $\zeta\in\mathbb{R}$,
we have

$\displaystyle\mathbb{P}(\lambda_{\max}\left(\sum\limits_{k=1}^{n}\mathcal{X}_{k}\right)\geq\zeta)\leq(2M-1)^{N}\inf_{t>0}\Bigg{\\{}e^{-t\zeta}\int_{-\infty}^{\infty}\mathrm{Tr}\left[\mathbb{E}(e^{t\mathcal{X}_{1}})\star_{M}\right.~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}$
(99) $\displaystyle\left.\left(\prod_{k=2}^{n-1}\mathbb{E}(e^{\frac{(1+\iota
s)t\mathcal{X}_{k}}{2}})\right)\star_{M}\mathbb{E}(e^{t\mathcal{X}_{n}})\star_{M}\left(\prod_{k=n-1}^{2}\mathbb{E}(e^{\frac{(1-\iota
s)t\mathcal{X}_{k}}{2}})\right)\right]\rho_{0}(s)ds\Bigg{\\}}$

Proof:

By settgin $\mathcal{H}_{k}=t\mathcal{X}_{k}$, $p=2$ in Theorem 4.3, we will
have

(100)
$\displaystyle\mathbf{Tr}e^{\left(\sum\limits_{k=1}^{n}t\mathcal{X}_{k}\right)}$
$\displaystyle\leq$
$\displaystyle\int_{-\infty}^{\infty}\mathrm{Tr}\left[e^{t\mathcal{X}_{1}}\star_{M}\right.$
$\displaystyle\left.\left(\prod_{k=2}^{n-1}e^{\frac{(1+\iota
s)t\mathcal{X}_{k}}{2}}\right)\star_{M}e^{t\mathcal{X}_{n}}\star_{M}\left(\prod_{k=n-1}^{2}e^{\frac{(1-\iota
s)t\mathcal{X}_{k}}{2}}\right)\right]\rho_{0}(s)ds.$

By taking the expectation of both sides and applying the indepedence property
for all random tensors $\mathcal{X}_{k}$, we obtain

(101)
$\displaystyle\mathbb{E}\mathbf{Tr}e^{\left(\sum\limits_{k=1}^{n}t\mathcal{X}_{k}\right)}$
$\displaystyle\leq$
$\displaystyle\int_{-\infty}^{\infty}\mathrm{Tr}\left[\mathbb{E}(e^{t\mathcal{X}_{1}})\star_{M}\left(\prod_{k=2}^{n-1}\mathbb{E}(e^{\frac{(1+\iota
s)t\mathcal{X}_{k}}{2}})\right)\star_{M}\right.$
$\displaystyle\left.\mathbb{E}(e^{t\mathcal{X}_{n}})\star_{M}\left(\prod_{k=n-1}^{2}\mathbb{E}(e^{\frac{(1-\iota
s)t\mathcal{X}_{k}}{2}})\right)\right]\rho_{0}(s)ds.$

By combing Eq. (101) with Lemma 5.1, the theorem is proved. $\hfill\Box$

##  6 Conclusions

In this work, we extend Araki–Lieb–Thirring (ALT) inequality,
Golden–Thompson(GT) inequality and logarithmic trace inequality to arbitrary
many tensors. Our proofs utilize complex interpolation theory and asymptotic
spectral pinching, providing a powerful mechanism to deal with multivariate
trace inequalities for tensors. We then apply tensor Golden–Thompson
inequality to provide the tail bound for the independent sum of tensors and
this bound will play a crucial role in high-dimensional probability and
statistical data analysis.

## Acknowledgments

The helpful comments of the referees are gratefully acknowledged.

## References

  * [1] R. Ahlswede and A. Winter, Strong converse for identification via quantum channels, IEEE Transactions on Information Theory, 48 (2002), pp. 569–579.
  * [2] T. Ando and F. Hiai, Log majorization and complementary golden-thompson type inequalities, Linear Algebra and its Applications, 197 (1994), pp. 113–131.
  * [3] H. Araki, On an inequality of lieb and thirring, LMaPh, 19 (1990), pp. 167–170.
  * [4] E. Carlen, Trace inequalities and quantum entropy: an introductory course, Entropy and the quantum, 529 (2010), pp. 73–140.
  * [5] I. Csiszár, The method of types [information theory], IEEE Transactions on Information Theory, 44 (1998), pp. 2505–2523.
  * [6] H. G. Eggleston, Convexity, no. 47, CUP Archive, 1958.
  * [7] P. J. Forrester and C. J. Thompson, The golden-thompson inequality: Historical aspects and random matrix applications, Journal of Mathematical Physics, 55 (2014), p. 023503.
  * [8] F. Hiai and D. Petz, The golden-thompson trace inequality is complemented, Linear algebra and its applications, 181 (1993), pp. 153–185.
  * [9] I. I. Hirschman, A convexity theorem for certain groups of transformations, Journal d’Analyse Mathématique, 2 (1952), pp. 209–218.
  * [10] F. Kittaneh, Norm inequalities for certain operator sums, Journal of Functional Analysis, 143 (1997), pp. 337–348.
  * [11] M. Kleinmann, H. Kampermann, T. Meyer, and D. Bruß, Physical purification of quantum states, Physical Review A, 73 (2006), p. 062309.
  * [12] H. Kosaki, An inequality of araki-lieb-thirring (von neumann algebra case), Proceedings of the American Mathematical Society, 114 (1992), pp. 477–481.
  * [13] A. Lenard and P. Halmos, Generalization of the Golden-Thompson inequality, Indiana University Mathematics Journal, (1971), pp. 457–467.
  * [14] M. Liang and B. Zheng, Further results on moore–penrose inverses of tensors with application to tensor nearness problems, Computers & Mathematics with Applications, 77 (2019), pp. 1282–1293.
  * [15] E. H. Lieb and M. B. Ruskai, A fundamental property of quantum-mechanical entropy, Physical Review Letters, 30 (1973), p. 434.
  * [16] E. H. Lieb and M. B. Ruskai, Proof of the strong subadditivity of quantum-mechanical entropy, Les rencontres physiciens-mathématiciens de Strasbourg-RCP25, 19 (1973), pp. 36–55.
  * [17] E. H. Lieb and W. E. Thirring, Inequalities for the moments of the eigenvalues of the schrodinger hamiltonian and their relation to sobolev inequalities, in The Stability of Matter: From Atoms to Stars, Springer, 1991, pp. 135–169.
  * [18] H. Neudecker, A note on kronecker matrix products and matrix equation systems, SIAM Journal on Applied Mathematics, 17 (1969), pp. 603–606.
  * [19] G. Ni, Hermitian tensor and quantum mixed state, arXiv preprint arXiv:1902.02640, (2019).
  * [20] L. Qi, Eigenvalues of a real supersymmetric tensor, Journal of Symbolic Computation, 40 (2005), pp. 1302–1324.
  * [21] L. Qi, H. Chen, and Y. Chen, Tensor eigenvalues and their applications, vol. 39, Springer, 2018.
  * [22] I. Segal, Notes towards the construction of nonlinear relativistic quantum fields. iii: Properties of the c*-dynamics for a certain class of interactions, Bulletin of the American Mathematical Society, 75 (1969), pp. 1390–1395.
  * [23] E. M. Stein, Interpolation of linear operators, Transactions of the American Mathematical Society, 83 (1956), pp. 482–492.
  * [24] C. J. Thompson, Inequality with applications in statistical mechanics, Journal of Mathematical Physics, 6 (1965), pp. 1812–1813.
  * [25] J. A. Tropp, User-friendly tail bounds for sums of random matrices, Foundations of computational mathematics, 12 (2012), pp. 389–434.
  * [26] B.-Y. Wang and F. Zhang, Trace and eigenvalue inequalities for ordinary and hadamard products of positive semidefinite hermitian matrices, SIAM journal on matrix analysis and applications, 16 (1995), pp. 1173–1183.
  * [27] A. Wehrl, General properties of entropy, Reviews of Modern Physics, 50 (1978), p. 221.