# The entropic doubling constant
and robustness of Gaussian codebooks for additive-noise channels

Lampros Gavalakis L.G. has received funding from the European Union’s Horizon
2020 research and innovation program under the Marie Sklodowska-Curie grant
agreement No 101034255 and by the Bézout Labex, funded by ANR, reference
ANR-10-LABX-58. Univ Gustave Eiffel, Univ Paris Est Creteil, CNRS, LAMA
UMR8050 F-77447 Marne-la-Vallée, France Ioannis Kontoyiannis  Statistical
Laboratory, Univ of Cambridge, Wilberforce Road, Cambridge CB3 0WB, U.K.
Mokshay Madiman  Department of Mathematical Sciences, Univ of Delaware, 501
Ewing Hall, Newark, DE 19716, U.S.A.

###### Abstract

Entropy comparison inequalities are obtained for the differential entropy
$h(X+Y)$ of the sum of two independent random vectors $X,Y$, when one is
replaced by a Gaussian. For identically distributed random vectors $X,Y$,
these are closely related to bounds on the entropic doubling constant, which
quantifies the entropy increase when adding an independent copy of a random
vector to itself. Consequences of both large and small doubling are explored.
For the former, lower bounds are deduced on the entropy increase when adding
an independent Gaussian, while for the latter, a qualitative stability result
for the entropy power inequality is obtained. In the more general case of non-
identically distributed random vectors $X,Y$, a Gaussian comparison inequality
with interesting implications for channel coding is established: For additive-
noise channels with a power constraint, Gaussian codebooks come within a
$\frac{{\sf snr}}{3{\sf snr}+2}$ factor of capacity. In the low-SNR regime
this improves the half-a-bit additive bound of Zamir and Erez (2004).
Analogous results are obtained for additive-noise multiple access channels,
and for linear, additive-noise MIMO channels.

Keywords —  Entropy inequalities; entropic doubling; entropy power; Gaussian
codebook; maximum entropy; capacity; multiple access channel; MIMO channel

##  1 Introduction

The (differential) entropy of an $\mathbb{R}^{d}$-valued random vector $X$
with density $f$ is defined as,

$h(X)=-\int_{\mathbb{R}^{d}}f(x)\log f(x)\,dx,$

whenever the integral exists. When $X$ is supported on a strictly lower-
dimensional subset of $\mathbb{R}^{d}$ or, more generally, when the law of $X$
does not have a density with respect to the $d$-dimensional Lebesgue measure,
we set $h(X)=-\infty$. The entropy power of $X$ is defined by
$N(X)=e^{\frac{2}{d}h(X)}$ and takes values in $[0,\infty]$.

###  1.1 Small and large entropic doubling

The (entropic) doubling constant of a random vector $X$ in $\mathbb{R}^{d}$ is
defined as,

$\sigma[X]:=\frac{N(X+{X}^{\prime})}{2N(X)}=\frac{1}{2}\exp\Big{\\{}\frac{2}{d}[h(X+X^{\prime})-h(X)]\Big{\\}},$

where $X,{X}^{\prime}$ are independent and identically distributed (i.i.d.).

Observe that $\sigma[X]$ is a simple monotonic function of the entropy
increase $h(X+X^{\prime})-h(X)$. By the entropy power inequality (EPI),
$\sigma[X]\geq 1$ for any random vector $X$, with equality if and only if $X$
is Gaussian. See [39] for further discussion of the doubling constant.

From the definition, we see that
$h(X+{X}^{\prime})-h(X)=\frac{d}{2}\log(2\sigma[X])$. If $Z$ is an independent
Gaussian with the same covariance matrix as $X$, the maximum entropy property
of the Gaussian trivially implies that,

$h(Z)\geq
h\Big{(}\frac{X+{X}^{\prime}}{\sqrt{2}}\Big{)}=h(X+{X}^{\prime})-\frac{d}{2}\log
2.$

Therefore, $h(Z)-h(X)\geq\frac{d}{2}\log\sigma[X]$, and since $h(X+Z)\geq
h(Z)$, we have:

$h(X+Z)-h(X)\geq\frac{d}{2}\log\sigma[X].$

This says that adding a Gaussian with the same covariance matrix increases the
entropy of $X$ by an amount that can be bounded below in terms of the doubling
constant. Our first result gives a quantitatively sharper version of that
statement.

######  Theorem 1 (Large doubling).

Suppose $X$ is a random vector in $\mathbb{R}^{d}$, and $Z$ is an independent
Gaussian random vector with the same covariance matrix as $X$. Then,

$h(X+Z)-h(X)\geq\frac{d}{2}\log\big{[}\max\big{\\{}\varphi\,\sigma[X],1+\sigma[X]\big{\\}}\big{]},$

where $\varphi=\frac{1+\sqrt{5}}{2}$ is the golden ratio.

We will prove the two parts of the theorem separately, in Section 3. The first
bound is simply a rewriting of Theorem 4, and the second bound is a special
case of Theorem 5, corresponding to $K_{Z}=K_{X}$ there. Moreover, the result
of Theorem 5, and by extension Theorem 1, is sharp in the sense that equality
holds if $X$ is Gaussian.

Theorem 1 is about the consequences of large doubling. In the reverse
direction, next, we explore the consequences of small doubling. First, we note
that stability of the lower bound $\sigma[X]\geq 1$ implied by the EPI may
fail for “strong” distance measures like the quadratic Wasserstein distance or
relative entropy. That is, it may be possible to have $\sigma[X]$ arbitrarily
close to 1, while the distance of its distribution from the Gaussian remains
bounded away from zero; see the discussion in [10].

But, as we show in Theorem 2 stated next, if we impose a mild moment
assumption, a more qualitative version of stability in the sense of weak
convergence does in fact hold. Also, under much stronger assumptions on the
densities considered, a different stability result for the quadratic
Wasserstein distance is established in [10].

Our stability result in Theorem 2 below follows from Theorem 6 in Section 3,
which is established by combining an earlier result of Carlen and Soffer [9]
with the stability of Cramér’s theorem due to Lévy [33].

For a random vector $X$ with law $\nu$, denote,

$\psi_{X}(R):=\mathbb{E}\big{(}\|X\|^{2};\|X\|>R\big{)}=\int_{\|x\|>R}\|x\|^{2}d\nu(x),\qquad
R\geq 0,$

where $\|\cdot\|$ denotes the usual Euclidean norm,
$\|x\|=[x_{1}^{2}+x_{2}^{2}+\cdots+x_{d}^{2}]^{1/2}$, $x\in\mathbb{R}^{d}$. We
may think of $\psi_{X}$ as the “variance profile” of $X$. If $X$ has density
$f$, we also write $\psi_{f}$ for $\psi_{X}$. By elementary reasoning, the
function $\psi_{X}:[0,\infty)\rightarrow[0,\infty]$ is non-increasing.
Moreover, if $X$ has a density $f$ and finite variance, then
$\psi_{X}(R)=\psi_{f}(R)=\int_{\|x\|>R}\|x\|^{2}f(x)dx$ is continuous in $R$,
with $\psi_{X}(0)=\mathbb{E}(\|X\|^{2})$ and
$\lim_{R\rightarrow\infty}\psi_{X}(R)=0$. Let $\mathcal{F}_{d}$ be the class
of probability density functions on $\mathbb{R}^{d}$, namely, of all functions
$f:\mathbb{R}^{d}\rightarrow[0,\infty)$ such that
$\int_{\mathbb{R}^{d}}f(x)dx=1$. For a given function
$\psi:[0,\infty)\rightarrow[0,\infty)$ with
$\lim_{R\rightarrow\infty}\psi(R)=0$, we define the class of densities ${\cal
C}_{\psi}\subset{\cal F}_{d}$ as:

$\mathcal{C}_{\psi}:=\big{\\{}f\in\mathcal{F}_{d}:\psi_{f}\leq\psi\big{\\}}.$
(1)

Write $\gamma_{\mu,\Sigma}$ for the Gaussian measure on $\mathbb{R}^{d}$ with
mean $\mu$ and covariance $\Sigma$, and $d_{\rm LP}$ for the Lévy-Prokhorov
metric on the space of probability measures (see Section 2 for the precise
definition).

######  Theorem 2 (Small doubling).

Suppose $X,{X}^{\prime}$ are i.i.d. random vectors with law $\nu$ on
$\mathbb{R}^{d}$, which have density in $\mathcal{C}_{\psi}$ for some
$\psi:[0,\infty)\rightarrow[0,\infty)$ with
$\lim_{R\rightarrow\infty}\psi(R)=0$ and invertible covariance matrix
$\Sigma_{X}$. Then, for each $\epsilon>0$, there is a
$\delta=\delta(\epsilon,\psi,\Sigma_{X},d)>0$ such that
$h(X+{X}^{\prime})>-\infty$, and,

$\sigma[X]<1+\delta,$

implies that $d_{\rm LP}({\nu},\gamma_{\mu,\Sigma})<\epsilon$ for some for
some $\mu,\Sigma$.

###  1.2 The Gaussian as worst-case noise

Suppose $Y,Z$ are independent random variables, where $Z$ is a Gaussian with
the same variance as $Y$. The maximum entropy property of the Gaussian states
that $h(Z)\geq h(Y)$. A natural but naive guess at a generalization of this
property could be that, for any random variable $X$ independent of $Y,Z$, we
always have:

$h(X+Z)\geq h(X+Y).$ (2)

As it turns out, this is not always true. This fact has been part of the
information-theoretic folklore for quite some time. An explicit counterexample
was provided by Eskenazis, Nayar and Tkocz (ENT) [19], and another
counterexample was found by Madiman, Nayar, and Tkocz [42], where $X$ and $Y$
are in fact log-concave, symmetric and with unit variance.

But there are also positive results, along several different directions.
First, one can restrict the class of distributions considered for $X$ and $Y$.
Indeed, ENT [19] show that (2) does actually hold for Gaussian scale mixtures,
that is, for independent random variables $X$ and $Y$ that can be expressed as
$\sqrt{W}Z^{\prime}$, where $Z^{\prime}\sim N(0,1)$ and $W$ is almost surely
positive and independent of $Z^{\prime}$.

Second, one might ask for approximate versions of (2). Zamir and Erez [60],
motivated by the question of robustness of Gaussian inputs in additive-noise
channels, showed that we always have:

$h(X+Z)\geq h(X+Y)-\frac{1}{2}\log{2}.$ (3)

Hence, even though (2) is not true in general, $h(X+Z)$ can not be much
smaller than $h(X+Y)$.

Third, we might restrict $Y$ to have the same distribution as $X$. The
question of whether (2) holds under this condition was raised in ENT [19], and
it remains open. But in this case we can establish that an improvement of
bound (3) holds. Specifically, in Theorem 4 we prove that, whenever $X,Y$ are
i.i.d.,

$h(X+Z)\geq h(X+Y)-\frac{1}{2}\log\Big{(}\frac{2}{\varphi}\Big{)},$

where $\varphi$ again denotes the golden ratio. This improves the
$\frac{1}{2}\log 2\approx 0.347$ error term in (3) to
$\frac{1}{2}\log(2/\varphi)\approx 0.106$.

###  1.3 Robustness of Gaussian channel codes

A first connection between entropic doubling and Gaussian channel inputs is
revealed by re-interpreting our Theorem 5. As noted in Corollary 1 in Section
4.1, it can be rewritten as the following channel capacity bound: If $Y=X+Z$
is an additive noise channel, with input $X$ constrained to average power $P$,
and with noise $Z$ with power $N$, then,

$C(Z;P)\geq
I(X^{*};X^{*}+Z)\geq\frac{1}{2}\log\Big{(}1+\frac{P}{N}\sigma[Z]\Big{)},$

where $C(Z;P)$ is the capacity of the channel, $X^{*}\sim N(0,P)$ is a
Gaussian input, and $\sigma[Z]$ is the doubling constant of the noise $Z$.
Since $\sigma[Z]\geq 1$ with equality if and only if $Z$ is Gaussian, this
provides a pleasing proof of the worst-case noise property of the Gaussian
distribution [11, 32].

The study of the robustness of Gaussian codebooks for additive noise channels
with a power constraint has a rich history, part of which is reviewed in
Sections 4.1 and 4.2. Notably, Zamir and Erez [60] showed that Gaussian inputs
always come within 1/2 a bit of channel capacity:

$I(X^{*};X^{*}+Z)\geq C(Z;P)-\frac{1}{2}\log 2.$

Since this bound is trivial when $C(Z;P)\leq\frac{1}{2}\log 2$, they also
raised the question of whether a “multiplicative” version of the above
“additive” bound could be obtained, that would also provide useful information
in the “low SNR” regime, that is, when the capacity is small.

Using an argument similar to that employed in the proof of Theorem 5, in
Theorem 8 we offer a partial answer to this question, by showing that, indeed,
we always have

$I(X^{*};X^{*}+Z)\geq\frac{{\sf snr}}{3{\sf snr}+2}C(Z;P),$

where ${\sf snr}=\frac{P}{N}$.

Finally, in Sections 4.4 and 4.5 we describe how the same technique leads to
analogous results on the performance of time-shared Gaussian inputs for
additive-noise multiple access channels, and on the performance of
uncorrelated Gaussian inputs for linear, additive-noise multiple-input
multiple-output (MIMO) channels, respectively.

###  1.4 Ideas and techniques

The technical tools employed in our derivations are a combination of
elementary techniques, probabi-lity-theoretic arguments, and applications of
recently developed entropy inequalities whose implications for communication
have perhaps not been fully tapped yet. The first such inequality – the
submodularity of the entropy of sums [30] – has seen numerous applications in
convex geometry, e.g. [7, 22], and in additive combinatorics, e.g. [40, 38].
The second – the fractional superadditivity of entropy power [1, 37] – has
found applications in information theory, e.g. [28, 16, 24, 53].

In the context of the analogies between information theory and convex geometry
outlined in [13, 41], we note that there exist volume analogues of the
submodularity of entropy stated in Lemma 1, see [22, 21], and of the
fractional superadditivity of entropy power stated in Lemma 2, see [8, 20].
But these correspondences are limited; for instance, the statements for
volumes sometimes require different constants. These limitations, and
especially the unavailability of an additive functional such as the variance
in the geometric setting, make it difficult to write down analogues of our
results there. Nonetheless, it is a somewhat curious coincidence that the
golden ratio appears both in our Theorem 4 and in [22].

##  2 Preliminaries

We begin by collecting some preliminary definitions and results that will be
used later on, and we also generalize a one-dimensional classical Gaussian
stability result due to Lévy to the case of random vectors in
$\mathbb{R}^{d}$.

Let $X,Y$ be random vectors taking values in $\mathbb{R}^{d}$, with
probability density functions $f$ and $u$ respectively. The relative entropy
is, as usual, defined by,

$D(X\|Y)=D(f\|u):=\int_{\mathbb{R}^{d}}f(x)\log\frac{f(x)}{u(x)}dx,$

and is always nonnegative though possibly $+\infty$. Furthermore, for a random
vector $X$ with density $f$ and finite second moment we define,

$D(X)=D(f):=\inf_{u}D(f\|u)=D(f\|g),$

with the infimum being taken over all Gaussian densities $u$ on
$\mathbb{R}^{d}$, and achieved by the Gaussian density $g$ with the same mean
and covariance matrix as $X$. The quantity $D(X)$ may be thought of as the
relative entropy from Gaussianity. It is easy to see that
$D(f)=D(f\|g)=h(g)-h(f)$, which implies the Gaussian is the unique maximizer
of entropy when the mean and covariance matrix are fixed.

Logarithms that appear in any expression involving entropies or relative
entropies may be taken with respect to an arbitrary base, with the
understanding that all entropies in that expression also use the same base.
For concreteness, we assume that the base is $e$ everywhere.

Standing assumption.  Throughout this paper, in any statement involving
entropy, entropy power, or a doubling constant, we implicitly assume that
those exist for the random variables to which they correspond.

We need two lemmas describing fundamental properties of the entropy under
convolution. The first expresses the submodularity of the entropy of sums.

######  Lemma 1.

[30] If $X_{1},X_{2},X_{3}$ are independent random vectors in
$\mathbb{R}^{d}$, then:

$h(X_{1}+X_{2}+X_{3})+h(X_{1})\leq h(X_{1}+X_{2})+h(X_{1}+X_{3}).$

A version of this inequality for random variables taking values in a discrete
group can be traced back to [29]; it extends in fact to all locally compact
groups [39].

The second lemma we need is the fractional superadditivity of the entropy
power of convolutions. We use $[n]$ to denote $\\{1,2,\ldots,n\\}$.

######  Lemma 2.

[1, 37] $(i)$ If $X_{1},X_{2},\ldots,X_{n}$ are independent random vectors in
$\mathbb{R}^{d}$, then for any $1\leq k\leq n$:

$N(X_{1}+X_{2}+\cdots+X_{n})\geq\frac{1}{\binom{n-1}{k-1}}\sum_{S\subset[n]:|S|=k}N\Big{(}\sum_{j\in
S}X_{j}\Big{)}.$ (4)

$(ii)$ In particular, for $k=n-1$:

$N(X_{1}+X_{2}+\cdots+X_{n})\geq\frac{1}{n-1}\sum_{i\in[n]}N\Big{(}\sum_{j\neq
i}X_{j}\Big{)}.$ (5)

The inequality (5) was proved in [1]; simpler proofs of it were later given in
[36, 56, 51]. The more general inequality (4) was obtained in [37].

We will also need a classical fact about the well-known theorem of Cramér
[12], which states that if the sum of two independent random vectors is
Gaussian, then the random vectors themselves are also Gaussian. Let $d_{\rm
LP}(P,Q)$ denote the Lévy-Prokhorov metric between two probability measures
$\mu,\nu$ on $\mathbb{R}^{d}$,

$d_{\rm
LP}(\mu,\nu):=\inf{\Big{\\{}\epsilon>0:\nu(A)\leq\mu(A^{\epsilon})+\epsilon\text{
and }\mu(A)\leq\nu(A^{\epsilon})+\epsilon,\,\text{for all Borel sets
}A\subset\mathbb{R}^{d}\Big{\\}}},$

where $A^{\epsilon}$ denotes the “$\epsilon$-blow-up” of $A$,

$A^{\epsilon}:=\Big{\\{}x\in\mathbb{R}^{d}:\inf_{y\in
A}\|x-y\|<\epsilon\Big{\\}}.$

When $\mu,\nu$ are the laws of random vectors, $X,Y$, respectively, we find it
convenient to write $d_{\rm LP}(X,Y)=d_{\rm LP}(\mu,\nu)$. We will only make
use of the fact that the Lévy-Prokhorov distance metrizes weak convergence in
$\mathbb{R}^{d}$.

Lévy [33] proved weak stability of Cramér’s theorem in dimension $1$. We now
extend Lévy’s proof to $\mathbb{R}^{d}$.

######  Theorem 3 (Multidimensional stability of Cramér’s theorem).

Let $Z$ be a Gaussian random vector in $\mathbb{R}^{d}$ with invertible
covariance matrix $\Sigma$. For any $\epsilon>0$ there is a
$\delta=\delta(\Sigma,\epsilon)>0$ such that, if the independent random
vectors $X,Y$ have $d_{\rm LP}(X+Y,Z)<\delta,$ then we also have $d_{\rm
LP}(X,U_{1})<\epsilon$ and $d_{\rm LP}(Y,U_{2})<\epsilon$ for some Gaussian
random vectors $U_{1},U_{2}$.

###### Proof.

Let $\\{(X_{n},Y_{n})\;;\;n\geq 1\\}$ be a sequence of pairs of random vectors
with each $X_{n}$ independent of $Y_{n}$, and let $Z_{n}=X_{n}+Y_{n}$, $n\geq
1.$ It suffices to show that if $\\{Z_{n}\\}$ converges weakly to $Z$ as
$n\to\infty$, then there is a sequence of constants $\\{b_{n}\\}$ such that
$\\{X_{n}+b_{n}\\}$ converges weakly to another Gaussian vector as
$n\to\infty$.

We assume without loss of generality that the median of each $X_{n}$ is zero.
Suppose that $\\{X_{n}\\}$ does not converge to a Gaussian. We will derive a
contradiction.

Let $\eta\in(0,1/2)$ and choose $\ell>0$ such that,

${\mathbb{P}}\big{(}Z\in[-\ell,\ell]^{d}\big{)}=\frac{1}{{\mathrm{det}(2\pi\Sigma)}^{1/2}}\int_{[-\ell,\ell]^{d}}e^{-\frac{1}{2}x^{T}\Sigma^{-1}x}dx>1-\eta.$

By weak convergence of the $Z_{n}$, this implies that the maximum
concentration of $Z_{n}$ in a cube of side $2\ell$ exceeds $1-\eta$
eventually. Therefore, the same must be true for the maximum concentration
cube of side $2\ell$ of $X_{n}$, since, eventually,

$\displaystyle 1-\eta$
$\displaystyle<\sup_{x\in\mathbb{R}^{d}}\mathbb{P}\Big{(}Z_{n}\in
x+[-\ell,\ell]^{d}\Big{)}$
$\displaystyle=\sup_{x\in\mathbb{R}^{d}}\mathbb{E}\left[\mathbb{P}\Big{(}X_{n}\in
x-Y_{n}+[-\ell,\ell]^{d}\Big{|}Y_{n}\Big{)}\right]$
$\displaystyle\leq\sup_{z\in\mathbb{R}^{d}}\mathbb{P}\Big{(}X_{n}\in
z+[-\ell,\ell]^{d}\Big{)}.$

Since $\eta<1/2$, there must be a $z$ such that this bound is satisfied and
$z+[-\ell,\ell]^{d}$ includes the origin, otherwise the median of some
coordinate would be nonzero. Therefore, for any $\epsilon>0$,

${\mathbb{P}}\big{(}\|X_{n}\|>2\ell\sqrt{d}\big{)}<\epsilon,$

eventually. The laws of $\\{X_{n}\\}$ thus form a compact set with respect to
weak topology.

Since $\\{X_{n}\\}$ do not tend to a Gaussian and $d_{\rm LP}(\cdot,\cdot)$
metrizes weak convergence, for any Gaussian $U_{1}$ there exists some
subsequence $\\{n_{k}\\}$ along which $d_{\rm LP}(X_{n_{k}},U_{1})>c>0$ for
every $k\geq 1$. By compactness, we may extract a further subsequence, which
converges weakly to some random variable, say $V$, which is not Gaussian.
Since we have convergence of $X_{n}$ and $Z_{n}$ along this subsequence and
$X_{n}$ is independent of $Y_{n}$, $Y_{n}$ must also converge along the same
subsequence to some random vector $W$. But we have thus obtained two
independent random vectors $V,W,$ whose sum, $Z,$ is Gaussian without $V$
being Gaussian. This contradicts the multivariate version of Cramér’s theorem
[12]. ∎

Let us remark here that the recent work [44] also considers a stability
theorem of another characterization of the normal distribution, namely
Bernstein’s theorem, which says that if the sum and difference of two
independent random variables are independent, then the random variables are
Gaussian. That stability result is also generalized to higher dimensions in
[44] and it is used to characterize capacity regions for certain additive
Gaussian noise channels.

Finally, we need a qualitative stability result for the EPI that was obtained
by Carlen and Soffer [9] under a finite Fisher information assumption. In
order to state it in a form more suited to our purposes, we need some
notation.

For a random vector $X$ with an absolutely continuous density function $f$,
the Fisher information of $X$ is defined as:

$I(X)=I(f):=\int_{\mathbb{R}^{d}}\bigg{[}\frac{\|\nabla
f(x)\|}{f(x)}\bigg{]}^{2}f(x)dx.$

Note that the derivative is well defined almost everywhere as a consequence of
the absolute continuity of $f$; if this condition is not satisfied, we set
$I(X)=\infty$. We can now define another useful class of densities: For a
given $I_{0}>0$ and a given function $\psi:[0,\infty)\rightarrow[0,\infty)$
with $\lim_{R\rightarrow\infty}\psi(R)=0$, let:

$\mathcal{C}_{I_{0},\psi}:=\big{\\{}f\in\mathcal{F}_{d}:I(f)\leq
I_{0},\psi_{f}\leq\psi\big{\\}}.$ (6)

Observe that $\mathcal{C}_{I_{0},\psi}$ is a subset of ${\mathcal{C}}_{\psi}$
and a strict subset of the densities with variance at most $\psi(0)$. The
following is a slight rephrasing of [9, Theorem 1.2].

######  Proposition 1.

[9] For any $I_{0}>0$, $\epsilon>0$, $a\in(0,1/2]$, and any
$\psi:[0,\infty)\rightarrow[0,\infty)$ with
$\lim_{R\rightarrow\infty}\psi(R)=0$, there is a
$\delta=\delta(\epsilon,I_{0},\psi,a)>0$ such that, if:

  1. 1.

$X_{1}$ and $X_{2}$ are independent, with densities in
$\mathcal{C}_{I_{0},\psi}$;

  2. 2.

both $X_{1}$ and $X_{2}$ have zero mean and identity covariance;

  3. 3.

$\lambda\in[a,1-a]$ for some $a\in(0,\frac{1}{2}]$;

  4. 4.

and $h(\sqrt{\lambda}X_{1}+\sqrt{1-\lambda}X_{2})-\big{[}\lambda
h(X_{1})+(1-\lambda)h(X_{2})\big{]}<\delta,$

then $D(X_{1})<\epsilon$ and $D(X_{2})<\epsilon$.

##  3 Entropy comparison inequalities

In this section we prove the entropy bounds discussed in Sections 1.1 and 1.2
of the Introduction.

###  3.1 Adding Gaussian noise

We first examine the $d$-dimensional version of one of the main problems
described in the Introduction, namely, comparing the entropy $h(X+X^{\prime})$
of the sum of two i.i.d. random vectors with the entropy $h(X+Z)$ obtained
when one of the random vectors is replaced by a Gaussian with the same
covariance. The question of the relationship between $h(X+X^{\prime})$ and
$h(X+Z)$ is of course a natural one, perhaps first explicitly discussed by ENT
[19]. While the validity of the inequality $h(X+Z)\geq(X+X^{\prime})$
suggested in [19] remains open, our first main result below establishes an
approximate version with an explicit error term.

######  Theorem 4 (Approximate generalized Gaussian maximum entropy).

Let $X,X^{\prime}$ be i.i.d. random vectors in $\mathbb{R}^{d}$, and $Z$ be a
Gaussian random vector with the same covariance matrix as $X$. Then,

$h(X+Z)\geq
h(X+{X}^{\prime})-\frac{d}{2}\log{\Big{(}\frac{2}{\varphi}\Big{)}}.$

where $\varphi=\frac{1+\sqrt{5}}{2}$ is the golden ratio.

###### Proof.

The main idea of the proof is to bound $h(X+Z+{X}^{\prime})$ from above using
Lemma 1 and from below using Lemma 2 $(ii)$. Thus, we obtain,

$\displaystyle 2h(X+Z)-h(Z)$ $\displaystyle\geq h(X+Z+{X}^{\prime})$
$\displaystyle\geq
h(X+{X}^{\prime})+\frac{d}{2}\log{\Bigl{[}1+2e^{\frac{2}{d}(h(X+Z)-h(X+{X}^{\prime}))}}\Bigr{]}-\frac{d}{2}\log{2},$

or,

$\displaystyle h(X+Z)$ $\displaystyle\;\;\geq
h(X+{X}^{\prime})+\frac{d}{4}\log{\Bigl{[}1+2e^{\frac{2}{d}(h(X+Z)-h(X+{X}^{\prime}))}}\Bigr{]}$
$\displaystyle\;\;\;\;\;\;-\frac{1}{2}\bigl{[}h(X+{X}^{\prime})-h(Z)\bigr{]}-\frac{d}{4}\log{2}$
$\displaystyle\;\;\geq
h(X+{X}^{\prime})+\frac{d}{4}\log{\Bigl{[}1+2e^{\frac{2}{d}(h(X+Z)-h(X+{X}^{\prime}))}}\Bigr{]}-\frac{d}{2}\log{2}.$
(7)

Consider the increasing function,

$g(a):=a-\frac{d}{4}\log{\bigl{[}1+2e^{\frac{2}{d}a}\bigr{]}},\quad
a\in\mathbb{R}.$

Using the fact that its inverse is,

$g^{-1}(y)=\frac{d}{2}\log{\Bigl{[}e^{\frac{4}{d}y}+e^{\frac{2}{d}y}\sqrt{e^{\frac{4}{d}y}+1}\Bigr{]}},$

and minimizing the quadratic in the argument of the logarithm, from (7) we
obtain,

$\displaystyle h(X+Z)-h(X+{X}^{\prime})$ $\displaystyle\geq
g^{-1}\Big{(}-\frac{d}{2}\log{2}\Big{)}$
$\displaystyle=-\frac{d}{2}\log{\Big{(}\frac{2}{\varphi}\Big{)}},$

completing the proof. ∎

###  3.2 Large doubling

Here we obtain another quantitative consequence of large doubling for the
entropy. Theorem 4 gives one such result – indeed, it can be rewritten as:

$h(X+Z)-h(X)\geq\frac{d}{2}\log{\big{(}\varphi\sigma[X]\big{)}}.$ (8)

The following theorem gives an alternative inequality of the form (8), with
the additional advantage that it is sharp.

######  Theorem 5 (Large doubling).

Suppose $X$ is a random vector in $\mathbb{R}^{d}$ with covariance matrix
$K_{X}$, and $Z$ is an independent Gaussian random vector with covariance
matrix $K_{Z}$. Then,

$h(X+Z)-h(X)\geq\frac{d}{2}\log{\big{(}1+a\sigma[X]\big{)}},$ (9)

where $a^{d}=\frac{\det(K_{Z})}{\det(K_{X})}$. Moreover, equality holds in (9)
if $X$ is Gaussian with $K_{X}=aK_{Z}$ for some $a>0$.

###### Proof.

By the multivariate central limit theorem and the upper semi-continuity of
entropy with respect to weak convergence,

$h(X+Z)\geq\limsup_{n\to\infty}{h\left(X_{0}+(K_{Z}^{1/2}K_{X}^{-1/2})\frac{1}{\sqrt{n}}\sum_{i=1}^{n}X_{i}\right)},$
(10)

where $\\{X_{n}\;;\;n\geq 0\\}$ are i.i.d. copies of $X$. For fixed $n\geq 1$,
the scaling property of entropy power implies that,

$\displaystyle N\Bigg{(}$ $\displaystyle
X_{0}+(K_{Z}^{1/2}K_{X}^{-1/2})\frac{1}{\sqrt{n}}\sum_{i=1}^{n}X_{i}\Bigg{)}$
$\displaystyle=\frac{1}{n}N\Bigg{(}\sqrt{n}X_{0}+\sum_{i\in[n]}K_{Z}^{1/2}K_{X}^{-1/2}X_{i}\Bigg{)}$
$\displaystyle\geq\frac{1}{n^{2}}\bigg{[}\sum_{i\in[n]}N\big{(}\sqrt{n}X_{0}+K_{Z}^{1/2}K_{X}^{-1/2}X_{i}\big{)}+\sum_{i\in[n]:i<j}N\big{(}K_{Z}^{1/2}K_{X}^{-1/2}(X_{i}+X_{j})\big{)}\bigg{]}$
(11)
$\displaystyle=\frac{1}{n^{2}}\bigg{[}nN\big{(}\sqrt{n}X_{0}+K_{Z}^{1/2}K_{X}^{-1/2}X_{1}\big{)}+\binom{n}{2}N\big{(}K_{Z}^{1/2}K_{X}^{-1/2}(X_{1}+X_{2})\big{)}\bigg{]}$
$\displaystyle\geq\frac{1}{n}N(\sqrt{n}X_{0})+\Big{(}\frac{n-1}{2n}\Big{)}N(K_{Z}^{1/2}K_{X}^{-1/2}(X_{1}+X_{2}))$
(12) $\displaystyle=N(X_{0})+\Big{(}\frac{n-1}{2n}\Big{)}aN(X_{1}+X_{2}),$
(13)

where in (11) we used Lemma 2 $(i)$ with $k=2$, and (12) follows from the fact
that entropy does not decrease on convolution. Combining (13) and (10) yields
the claimed bound (9).

The sufficient condition for equality follows by direct computation. ∎

Since we have equality in (9) if $X$ is Gaussian, and because of the use of
the EPI in the proof, it might be tempting to conjecture that the
corresponding ‘only if’ statement is also true, that is, that equality is
achieved only if $X$ is Gaussian with $K_{X}$ proportional to $K_{Z}$.
However, this is not the case, as the following example illustrates: Let $X$
be a random variable such that $h(X)=-\infty$, but
$h(X+{X}^{\prime})>-\infty$; see, e.g., [3] for an explicit such construction.
Then $X$ is necessarily not Gaussian, while both sides of (9) are $+\infty$.
We do not know whether an only if statement may be proved under additional
assumptions on $X$.

###  3.3 Small doubling

As observed in the Introduction, $D(X)=h(Z)-h(X)\geq\frac{d}{2}\log\sigma[X]$.
Therefore, $\sigma[X]$ being bounded away from 1 implies that $X$ is bounded
away from being Gaussian in the sense of relative entropy. It is a natural to
ask if the reverse holds – does $\sigma[X]$ close to 1 imply that $X$ is close
to Gaussian? Courtade, Fathi, and Pananjady [10] answer this stability
question by presenting a counterexample: They construct a sequence
$\\{X_{n}\\}$ of random variables with $\sigma[X_{n}]\to 1$ as $n\to\infty$,
but whose quadratic Wasserstein distance from the Gaussian is bounded away
from 0.

Nevertheless, stability can hold for restricted classes of distributions. The
qualitative stability result of [9] under a bounded Fisher information
condition was the first such result. A sharp, quantitative stability result
was obtained in [30] under the finite Poincaré constant assumption. And more
recently, quantitative stability results have been obtained under regularity
assumptions such as log-concavity [10, 18].

Our main observation in this section is that, although stability of the lower
bound on doubling may fail for strong distances, one can obtain a simpler
qualitative characterisation of random variables with small doubling in terms
of weak convergence, by combining observations from [9] with the stability of
Cramér’s theorem due to Lévy. Recall the definitions of the classes of
densities $\mathcal{C}_{\psi}$ and $\mathcal{C}_{I_{0},\psi}$ in (1) and (6),
respectively.

######  Theorem 6 (Small doubling implies near-Gaussianity).

Let $X_{1},X_{2}$ be independent random vectors in $\mathbb{R}^{d}$ with laws
$\nu_{1},\nu_{2}$, respectively, with densities in $\mathcal{C}_{\psi}$ for
some $\psi:[0,\infty)\rightarrow[0,\infty)$ with
$\lim_{R\rightarrow\infty}\psi(R)=0$. Suppose $X_{1},X_{2}$ have zero mean,
the same invertible covariance matrix $\Sigma$, and $h(X_{1}+X_{2})>-\infty$.
Then for each $\epsilon>0$, there is a
$\delta=\delta(\epsilon,\psi,\Sigma,d)>0$ such that,

$h\bigg{(}\frac{X_{1}+X_{2}}{\sqrt{2}}\bigg{)}-\frac{h(X_{1})+h(X_{2})}{2}<\delta,$

implies that $d_{\rm LP}({\nu_{1}},\gamma_{\mu_{1},\Sigma_{1}})<\epsilon$ and
$d_{\rm LP}({\nu_{2}},\gamma_{\mu_{2},\Sigma_{2}})<\epsilon$ for some for some
$\mu_{1},\Sigma_{1},\mu_{2},\Sigma_{2}$.

###### Proof.

Fix $\epsilon>0$. First, assume that $\Sigma=\mathrm{I}_{d}$.

For $i=1,2$ let $\tilde{X}_{i}=\frac{X_{i}+Z_{i}}{\sqrt{2}},$ where
$Z_{1},Z_{2}$ are independent standard Gaussian random vectors. We have the
elementary estimate [9, Eq. (1.38)],

$\psi_{\tilde{X}_{i}}(R)\leq
2\psi_{X_{i}}\bigl{(}R/2\bigr{)}+2\psi_{Z_{i}}\bigl{(}R/2\bigr{)}.$ (14)

Furthermore,

$I(\tilde{X}_{i})\leq 2I(Z_{i})=2d,$ (15)

and by [9, Eq. (1.30)] we also have,

$h(\tilde{X}_{1}+\tilde{X}_{2})-\frac{h(\tilde{X}_{1})+h(\tilde{X}_{2})}{2}\leq
h(X_{1}+X_{2})-\frac{h(X_{1})+h(X_{2})}{2}.$ (16)

Let $\epsilon_{1}>0$ to be chosen later. In view of (14)–(16), by Proposition
1 with $\lambda=1/2$, there is a $\delta(\epsilon_{1},\psi,d)>0$ such that,

$h(X_{1}+X_{2})-\frac{h(X_{1})+h(X_{2})}{2}<\frac{1}{2}\log{2}+\delta\quad{\rm
implies}\quad D(X_{i}+Z_{i})=D(\tilde{X}_{i})<\epsilon_{1},$ (17)

for $i=1,2.$ By Pinsker’s inequality,
$\|\nu_{i}*\gamma_{0,\mathrm{I}_{d}}-\gamma_{0,2\mathrm{I}_{d}}\|_{\rm
TV}\leq\sqrt{\frac{\epsilon_{1}}{2}}$, where ‘$*$’ denotes the convolution of
probability measures. Thus,

$d_{\rm
LP}(\nu_{1}*\gamma_{0,\mathrm{I}_{d}},\gamma_{0,2\mathrm{I}_{d}})\leq\|\nu_{1}*\gamma_{0,\mathrm{I}_{d}}-\gamma_{0,2\mathrm{I}_{d}}\|_{\rm
TV}\leq\sqrt{\frac{\epsilon_{1}}{2}},$

and,

$d_{\rm
LP}(\nu_{2}*\gamma_{0,\mathrm{I}_{d}},\gamma_{0,2\mathrm{I}_{d}})\leq\|\nu_{2}*\gamma_{0,\mathrm{I}_{d}}-\gamma_{0,2\mathrm{I}_{d}}\|_{\rm
TV}\leq\sqrt{\frac{\epsilon_{1}}{2}}.$

By Theorem 3,

$d_{\rm LP}(\nu_{1},\gamma_{\mu_{1},\Sigma_{1}})<\epsilon\quad{\rm and}\quad
d_{\rm LP}(\nu_{2},\gamma_{\mu_{2},\Sigma_{2}})<\epsilon,$

for some $\mu_{1},\Sigma_{1},\mu_{2},\Sigma_{2}$ and an appropriate choice of
$\epsilon_{1}$. Since $\epsilon>0$ was arbitrary, this gives the claimed
result.

Finally, to see that the assumption $\Sigma=\mathrm{I}_{d}$ incurs no loss of
generality, note that otherwise the assumption of the theorem holds for the
random variables $\Sigma^{-\frac{1}{2}}X_{i}$ and (17) is scale-invariant. ∎

Remarks.

  1. 1.

For identically distributed random vectors, the above result provides a
characterization of the distribution of random vectors with small doubling.
For random variables with the same variance (but not necessarily independent),
it may also be seen as a characterization of random variables with entropic
Ruzsa distance [30, 54] close to $\frac{1}{2}\log{2}.$

  2. 2.

The problem of stability in Cramér’s theorem is well studied for $d=1$. Under
the assumption that the random variables have median zero, stability results
for the Kolmogorov distance are known [47] with no moment assumptions.
Furthermore, assuming $\mathrm{Var}{(X_{1})}=\mathrm{Var}{(X_{2})}$ sharp
rates were obtained recently [4, 5, 6], improving earlier results of Sapogov
[47]. The book of Linnik and Ostrovskiĭ [34] contains further discussion and
references.

##  4 Channel coding bounds

###  4.1 Robustness of Gaussian inputs

The channel coding theorem for memoryless channels states that a channel’s
capacity, defined as the fastest possible reliable communication rate, is
equal to the maximum of the mutual information $I(X;Y)=h(X)-h(X|Y)$ between
the channel input $X$ and the channel output $Y$, where the maximum is over
all appropriate input distributions. Moreover, the capacity-achieving
distribution determines the distribution of optimal random codebooks and it
informs the construction of good codes [49, 23, 46]. The capacity-achieving
distribution of the the Additive White Gaussian Noise (AWGN) channel with an
average power constraint is itself Gaussian [50], but in general the
determination of capacity-achieving distributions is a fraught task; see,
e.g., [55, 25, 15], for discussions of the associated difficulties.

Gaussian codebooks occupy a special place in the study of additive-noise
channels. For example, i.i.d. Gaussian inputs and i.i.d. Gaussian noise are
minimax optimal choices in the channel coding game [11]. Moreover, for any
noise distribution with power $N$, Gaussian codebooks with power $P$ can
achieve all rates up to the Gaussian channel capacity,
$\frac{1}{2}\log(1+\frac{P}{N})$ [32, 48].

The robustness of Gaussian codebooks was further examined by Zamir and Erez
[60], who showed that, for any memoryless, additive-noise channel with an
average power constraint, Gaussian inputs incur a rate loss no greater that
half a bit compared to capacity. Specifically, consider a memoryless additive-
noise channel with input $X\in\mathbb{R}$, noise $Z\in\mathbb{R}$, and output
$Y=X+Z$. Let $N=\mathrm{Var}(Z)>0$ denote the noise power and $P>0$ the input
power constraint, that is, suppose that all potential input codewords
$(x_{1},x_{2},\ldots,x_{n})\in\mathbb{R}^{n}$ are constrained as,

$\frac{1}{n}\sum_{i=1}^{n}x_{i}^{2}\leq P.$

Let $C(Z;P)$ be the capacity of this channel:

$C(Z;P)=\sup_{X:{\mathbb{E}}(X^{2})\leq P}I(X;X+Z).$

First we note that Theorem 5 established earlier, immediately implies the
following simple bound on the performance of Gaussian inputs for an arbitrary
power-constrained additive-noise channel.

######  Corollary 1 (Quantitative minimax optimality of Gaussian noise).

Let $X^{*}\sim N(0,P)$. The capacity $C(Z;P)$ of the channel with additive
noise $Z$ with power $N=\mathrm{Var}(Z)$ and average power constraint $P$,
satisfies,

$C(Z;P)\geq I(X^{*};X^{*}+Z)\geq\frac{1}{2}\log\big{[}1+{\sf
snr}\,\sigma[Z]\big{]},$ (18)

where ${\sf snr}:=\frac{P}{N}$ denotes the signal-to-noise-ratio and
$\sigma[Z]$ is the doubling constant of the noise $Z$.

Since $\sigma[Z]\geq 1$ with equality if and only if $Z$ is Gaussian, from
(18) we immediately recover the worst-case property of Gaussian noise; see
also [26, 14] for further developments.

Zamir and Erez’s complementary result on the robustness of Gaussian inputs
reads as follows:

######  Theorem 7.

[60] Let $X^{*}\sim N(0,P)$. The capacity $C(Z;P)$ of the channel with
additive noise $Z$ and average power constraint $P$, satisfies:

$I(X^{*};X^{*}+Z)\geq C(Z;P)-\frac{1}{2}\log 2.$

As this additive bound is clearly only nontrivial if the capacity
$C(Z;P)>\frac{1}{2}\log 2$, Zamir and Erez [60] raised the question of whether
there exist a constant $\alpha>0$ and an input $X^{\prime}$, possibly non-
Gaussian, for which a multiplicative bound of the form,

$I(X^{\prime};X^{\prime}+Z)>\alpha C(Z;P),$ (19)

might hold, for any noise distribution for $Z$. We offer a partial answer to
this question in Theorem 8. Generalizing the proof method of our earlier
Theorem 5, we show that a Gaussian input $X^{\prime}=X^{*}$ indeed satisfies
(19), but with a constant $\alpha$ which depends on the signal-to-noise ratio
(SNR). Theorem 8, proved Section 4.3, gives meaningful information on the rate
achieved by Gaussian inputs at arbitrarily low SNR.

######  Theorem 8 (Robustness of Gaussian inputs).

Let $X^{*}\sim N(0,P)$. The capacity $C(Z;P)$ of the channel with additive
noise $X$ and ${\sf snr}=\frac{P}{N}$, where $P$ is the input average power
constraint and $N=\mathrm{Var}(Z)$ is the noise power, satisfies:

$I(X^{*};X^{*}+Z)\geq\frac{{\sf snr}}{3{\sf snr}+2}C(Z;P).$

An analogous result for additive-noise multiple access channels is established
in Theorem 9 in Section 4.4, and for linear additive-noise MIMO channels in
Theorem 10 in Section 4.5. Although all three of these results are stated for
one-dimensional channels, their extension to $\mathbb{R}^{d}$ for any finite
dimension $d\geq 1$ is entirely straightforward.

###  4.2 Bibliographical comments

Before proceeding to prove Theorems 8–10, we make some more bibliographical
remarks.

First, we note that further results on robust channel input distributions in a
similar spirit to [60] are given, e.g., in [59, 52, 35, 45], and rate-
distortion analogues to some of these bounds are derived, e.g., in [61, 31].

In a different line of work, structured random codes have been shown to be
both practical and effective; the survey [57] traces the story that began with
[27]. Moreover, although these “sparse superposition” codes are built with
Gaussian entries and were originally developed for the AWGN channel, they have
subsequently been shown to be universal [2]. The aforementioned robustness
results may be seen as approximate analogues of this for unstructured random
codes.

In yet another direction, the restricted capacity of an additive-noise channel
with noise $Z$ and input power constraint $P$ is defined by,

$C^{LC}(Z;P):=\sup_{X}I(X;X+Z),$

where the supremum is taken over all symmetric and log-concave random
variables $X$ such that $\mathbb{E}(X^{2})\leq P$. Then [43, Corollary 3]
shows that, if the noise $Z$ is symmetric and log-concave, we have:

$0\leq C(Z;P)-C^{LC}(Z;P)\leq\frac{1}{2}\log\Big{(}\frac{\pi e}{6}\Big{)}.$

In other words, if the noise is symmetric and log-concave, then the rate loss
with respect to the “true” channel capacity incurred by restricting the input
to also be symmetric and log-concave, is at most
$\frac{1}{2}\log_{2}\big{(}\frac{\pi e}{6}\big{)}\approx 0.254$ bits.

###  4.3 Proof of Theorem 8

Let $X$ be any input with variance $P_{X}\leq P$. Let $Z$ be the noise with
variance $N$. Assume without loss of generality that $X$ and $Z$ have zero
mean and that $h(Z)>-\infty$.

By the central limit theorem, $X^{*}$ can be expressed as the weak limit of,

$\frac{\sum_{i=1}^{n}{(X_{i}}+{Z_{i}})}{\sqrt{2n\frac{P_{X}+N}{P}}},$

where $\\{X_{i}\\}$ and $\\{Z_{i}\\}$ are i.i.d. copies of $X$ and $Z$,
respectively. By the upper-semicontinuity of entropy,

$h(X^{*}+Z)\geq\limsup_{n\to\infty}{h\left(\frac{\sum_{i=1}^{n}{(X_{i}}+{Z_{i}})}{\sqrt{2n\frac{P_{X}+N}{P}}}+Z\right)}.$
(20)

By the scaling property of differential entropy, we have, for any $n\geq 1$,

$\displaystyle
h\left(\frac{X_{1}+\cdots+X_{n}+Z_{1}+\cdots+Z_{n}}{\sqrt{2n\frac{P_{X}+N}{P}}}+Z\right)$
$\displaystyle=h\left(X_{1}+\ldots+X_{n}+Z_{1}+\ldots+Z_{n}+\sqrt{2n\frac{P_{X}+N}{P}}Z\right)-\frac{1}{2}\log{\Bigl{(}{2n\frac{P_{X}+N}{P}}\Bigr{)}}$
(21)

Now we apply [36, Theorem 3], with $\mathcal{C}$ being the following
collection of subsets of $\\{1,\ldots,2n+1\\}$ of cardinality $2$:
$\big{\\{}\\{1,2\\},\ldots,\\{1,n+1\\}\big{\\}}\cup\big{\\{}\\{i,j\\}:2\leq
i\leq n,n+1\leq j\leq 2n+1\big{\\}}$. This gives:

$\displaystyle
e^{2h\Bigl{(}X_{1}+\cdots+X_{n}+Z_{1}+\cdots+Z_{n}+\sqrt{2n\frac{P_{X}+N}{P}}Z\Bigr{)}}$
$\displaystyle\geq\frac{1}{n}\Biggl{[}ne^{2h\Bigl{(}\sqrt{2n\frac{P_{X}+N}{P}}Z+Z_{1}\Bigr{)}}+{n^{2}}e^{2h(X_{1}+Z_{1})}\Biggr{]}$
$\displaystyle\geq
e^{2h\Bigl{(}\sqrt{2n\frac{P_{X}+N}{P}}Z\Bigr{)}}+{n}e^{2h(X_{1}+Z_{1})},$

where the last inequality follows the fact that entropy does not decrease on
the addition of an independent random variable. Thus, by (21), and scaling,

$\displaystyle
h\Bigl{(}\frac{X_{1}+\cdots+X_{n}+Z_{1}+\ldots+Z_{n}}{\sqrt{2n\frac{P_{X}+N}{P}}}+Z\Bigr{)}$
$\displaystyle\qquad\geq\frac{1}{2}\log{\Biggl{(}e^{2h\Bigl{(}\sqrt{2n\frac{P_{X}+N}{P}}Z\Bigr{)}}+{n}e^{2h(X_{1}+Z_{1})}\Biggr{)}}-\frac{1}{2}\log{\Bigl{(}{2n\frac{P_{X}+N}{P}}\Bigr{)}}$
$\displaystyle\qquad=\frac{1}{2}\log{\Biggl{(}e^{2h(Z)}+\frac{P}{2(P_{X}+N)}e^{2h(X+Z)}\Biggr{)}}$
$\displaystyle\qquad\geq\frac{1}{2}\log{\Biggl{(}e^{2h(Z)}+\frac{P}{2(P+N)}e^{2h(X+Z)}\Biggr{)}},$

where in the last inequality we used that $P_{X}\leq P$. Therefore, since
$I(X;X+Z)=h(X+Z)-h(Z)$, we obtain:

$\displaystyle
h\Bigl{(}\frac{X_{1}+\cdots+X_{n}+Z_{1}+\ldots+Z_{n}}{\sqrt{2n\frac{P_{X}+N}{P}}}+Z\Bigr{)}$
$\displaystyle\qquad=I(X;X+Z)+\frac{1}{2}\log{\Biggl{(}e^{-2I(X;X+Z)}+\frac{P}{2(P+N)}\Biggr{)}}+h(Z).$
(22)

We have the elementary bound

$\frac{1}{2}\log{\Bigl{(}e^{-2x}+\beta\Bigr{)}}\geq-\frac{1}{1+\beta}x,\quad
x\geq 0,\beta\geq 0.$

Thus, by (22),

$\displaystyle
h\left(\frac{X_{1}+\cdots+X_{n}+Z_{1}+\cdots+Z_{n}}{\sqrt{2n\frac{P_{X}+N}{P}}}+Z\right)$
$\displaystyle\geq\Bigl{(}1-\frac{2}{2+\frac{P}{P+N}}\Bigr{)}I(X;X+Z)+h(Z)$
$\displaystyle=\frac{{\sf snr}}{3{\sf snr}+2}I(X;X+Z)+h(Z),$

where the right-hand side is independent of $n$. Therefore, by (20),

$h(X^{*}+Z)\geq\frac{{\sf snr}}{3{\sf snr}+2}I(X;X+Z)+h(Z),$

or,

$I(X^{*};X^{*}+Z)\geq\frac{{\sf snr}}{3{\sf snr}+2}I(X;X+Z).$ (23)

Finally, taking the supremum over all $X$ with $\mathbb{E}(X^{2})\leq P$,
gives the result. ∎

###  4.4 Multiple access channels

Consider a memoryless, additive-noise, multiple access channel (MAC) with two
senders $X_{1},X_{2}$, power constraints $P_{1},P_{2}$, and output
$Y=X_{1}+X_{2}+Z$, where $Z$ is arbitrary real-valued additive noise of power
$N=\mathrm{Var}(Z)$. The capacity region $\mathcal{C}(Z;P_{1},P_{2})$ of this
channel is given [58] by the closure of the set of all pairs $(R_{1},R_{2})$
of nonnegative rates $R_{1},R_{2}$ satisfying,

$\displaystyle R_{1}$ $\displaystyle\leq
I(X_{1};Y|X_{2},V)=\sum_{v}{\mathbb{P}}(V=v)I(X_{1};Y|X_{2},V=v),$ (24)
$\displaystyle R_{2}$ $\displaystyle\leq
I(X_{2};Y|X_{1},V)=\sum_{v}{\mathbb{P}}(V=v)I(X_{2};Y|X_{1},V=v),$ (25)
$\displaystyle R_{1}+R_{2}$ $\displaystyle\leq
I(X_{1},X_{2};Y|V)=\sum_{v}{\mathbb{P}}(V=v)I(X_{1},X_{2};Y|V=v),$ (26)

for some triplet $(V,X_{1},X_{2})$ of random variables with
$\mathbb{E}(X^{2}_{i})\leq P_{i}$ for $i=1,2$, with $X_{1}$ and $X_{2}$ being
conditionally independent given $V$, with $V$ taking at most $2$ values, and
with $(V,X_{1},X_{2})$ being independent of the noise $Z$.

In [45], the sum-capacity of the MIMO-MAC channel is studied, and it is noted
there that the half-a-bit bound of Theorem 7 extends to the capacity region of
the MAC. Here will apply our Theorem 8, more specifically the bound
established in (23), to obtain an improvement of this bound for the MAC
capacity region in the low-SNR regime.

######  Theorem 9 (MAC: Robustness of Gaussian inputs).

Let $\mathcal{C}(Z;P_{1},P_{2})$ denote the capacity region of the two-user,
additive-noise MAC with noise $Z$ having power $N$, and power constraints
$P_{1},P_{2}$. Consider the rate region $\mathcal{C^{*}}(Z;P_{1},P_{2})$
defined as the closure of the set of all pairs $(R_{1},R_{2})$ of nonnegative
rates that can be achieved by time-sharing Gaussian inputs, namely, rates
$(R_{1},R_{2})$ that satisfy,

$\displaystyle R_{1}$ $\displaystyle\leq I(X^{*}_{1};Y|X^{*}_{2},V),$ (27)
$\displaystyle R_{2}$ $\displaystyle\leq I(X^{*}_{2};Y|X^{*}_{1},V),$ (28)
$\displaystyle R_{1}+R_{2}$ $\displaystyle\leq I(X^{*}_{1},X^{*}_{2};Y|V),$
(29)

for some triplet $(V,X^{*}_{1},X^{*}_{2})$ of random variables with
$\mathbb{E}(X^{*2}_{i})\leq P_{i}$ for $i=1,2$, with $X^{*}_{1}$ and
$X^{*}_{2}$ being conditionally independent Gaussians given $V$, with $V$
taking at most $2$ values, and with $(V,X^{*}_{1},X^{*}_{2})$ being
independent of the noise $Z$.

Then $\mathcal{C}^{*}(Z;P_{1},P_{2})$ contains the region defined as
$\mathcal{C}(Z;P_{1},P_{2}),$ but with (24), (25) and (26) replaced with,

$\displaystyle R_{1}$
$\displaystyle\leq\sum_{v}\mathbb{P}(V=v)\Big{(}\frac{{\sf snr}_{1,v}}{3{\sf
snr}_{1,v}+2}\Big{)}I(X_{1};Y|X_{2},V=v),$ $\displaystyle R_{2}$
$\displaystyle\leq\sum_{v}\mathbb{P}(V=v)\Big{(}\frac{{\sf snr}_{2,v}}{3{\sf
snr}_{2,v}+2}\Big{)}I(X_{2};Y|X_{1},V=v),$ $\displaystyle R_{1}+R_{2}$
$\displaystyle\leq\sum_{v}\mathbb{P}(V=v)\Big{(}\frac{{\sf snr}_{v}}{3{\sf
snr}_{v}+2}\Big{)}I(X_{1},X_{2};Y|V=v),$

where ${\sf snr}_{i,v}=\mathrm{Var}(X_{i}|V=v)/N$, $i=1,2$, and ${\sf
snr}_{v}=\mathrm{Var}(X_{1}+X_{2}|V=v)/N$.

###### Proof.

It suffices to show that, for any triplet $(V,X_{1},X_{2})$ with
$\mathbb{E}(X^{2}_{i})\leq P_{i}$ for $i=1,2$, $X_{1}$ and $X_{2}$ being
conditionally independent given $V$, and with $(V,X_{1},X_{2})$ being
independent of $Z$, we can find a triplet $(V^{*},X_{1}^{*},X_{2}^{*})$ with
$\mathbb{E}(X^{*2}_{i})\leq P_{i}$ for $i=1,2$, $X_{1}^{*}$ and $X_{2}^{*}$
being conditionally independent Gaussians given $V^{*}$, and with
$(V^{*},X_{1}^{*},X_{2}^{*})$ being independent of $Z$, such that,

$\displaystyle I(X^{*}_{1};Y|X^{*}_{2},V)$
$\displaystyle\geq\sum_{v}\mathbb{P}(V=v)\frac{\mathrm{Var}{(X_{1}|V=v)}}{3\mathrm{Var}{(X_{1}|V=v)}+2N}I(X_{1};Y|X_{2},V=v)$
(30) $\displaystyle I(X^{*}_{2};Y|X^{*}_{1},V)$
$\displaystyle\geq\sum_{v}\mathbb{P}(V=v)\frac{\mathrm{Var}{(X_{2}|V=v)}}{3\mathrm{Var}{(X_{2}|V=v)}+2N}I(X_{2};Y|X_{1},V=v)$
(31) $\displaystyle I(X^{*}_{1},X^{*}_{2};Y|V)$
$\displaystyle\geq\sum_{v}\mathbb{P}(V=v)\frac{\mathrm{Var}{(X_{1}+X_{2}|V=v)}}{3\mathrm{Var}{(X_{1}+X_{2}|V=v)}+2N}I(X_{1},X_{2};Y|V=v).$
(32)

But this follows by taking $V^{*}$ an independent copy of $V$ and
$X_{1}^{*},X_{2}^{*}$ conditionally independent Gaussians given $V^{*}$,
having the same conditional variances as $X_{1}|V$ and $X_{2}|V$. Moreover, we
can take $(V,X_{1}^{*},X_{2}^{*})$ independent of $Z$. Then, clearly,
$\mathbb{E}(X^{*2}_{i})=\mathbb{E}(X^{2}_{i})\leq P_{i}.$ Expanding the mutual
informations on the left-hand side of (30) as,

$I(X^{*}_{1};Y|X^{*}_{2},V)=\sum_{v}\mathbb{P}(V=v)I(X^{*}_{1};Y|X^{*}_{2},V=v),$

and applying (23), gives (30); similarly, expanding the left-hand sides of
(31) and (32) and applying (23) gives (31)–(32), and the result follows. ∎

The conditioning random variable $V$ in the characterisation of the capacity
region ${\mathcal{C}}(Z;P_{1},P_{2})$ in (24)–(26) is necessary because of
convexity: Convex combinations of achievable rate pairs are achievable by
time-sharing [11, 17]. Similarly, rate pairs in the region ${\cal
C}^{*}(Z;P_{1},P_{2})$ defined by (27)–(29) are achievable by time-sharing
Gaussian codebooks. Clearly the inner bound to the capacity region given by
Theorem 9 is a strict improvement of the bound implied by one-receiver, one-
transmitter per-user case of the half-a-bit MIMO-MAC extension [45, Theorem 3]
in the low-SNR regime. This indicates that, even in the low-SNR regime, time-
sharing of Gaussian codebooks can always achieve a “fractional” subset of the
capacity region.

###  4.5 MIMO channels

Consider the linear, additive-noise MIMO channel, with $d_{t}$ transmitters
and $d_{r}$ receivers, input $X\in\mathbb{R}^{d_{t}}$, sum power constraint,

$\mathrm{tr}\mathbb{E}XX^{T}=\mathbb{E}\|X\|^{2}\leq P,$

and output,

$Y={\rm H}X+Z\in\mathbb{R}^{d_{r}},$

where ${\rm H}\in\mathbb{R}^{d_{r}\times d_{t}}$ is the channel matrix and
$Z\in\mathbb{R}^{d_{r}}$ is arbitrary additive noise with covariance matrix
$N$, independent of the input. The capacity of this channel, see e.g. [17,
45], is given by:

$C({\rm H},Z;P)=\sup_{X:\mathbb{E}\|X\|^{2}\leq P}I(X;{\rm H}X+N).$ (33)

The additive rate loss (in bits) of using uncorrelated Gaussian inputs was
shown in [45, Theorem 1] to be no more than:

$\begin{cases}\frac{d_{r}}{2}\log_{2}{\bigl{(}1+\frac{d_{t}}{d_{r}}\bigr{)}},&\text{if
}d_{r}\leq d_{t},\\\ \frac{d_{t}}{2},&\text{if }d_{r}\geq d_{t}.\end{cases}$

In this section we provide an alternative bound on the rate loss which also
remains valid at low SNR. The main tool we will need is the following
straightforward multidimensional extension of inequality (23) established in
the proof of Theorem 8.

######  Corollary 2.

Let $X,X^{*}$ and $Z$ be independent random vectors in $\mathbb{R}^{d}$ with
covariance matrices $K_{X},K_{X^{*}}$ and $N$, respectively. If $X^{*}$ is
Gaussian, then:

$I(X^{*};X^{*}+Z)\geq\frac{\det{(K_{X^{*}})}^{\frac{1}{d}}}{2\det{(K_{X}+N)}^{\frac{1}{d}}+\det{(K_{X^{*}})}^{\frac{1}{d}}}\times
I(X;X+Z).$ (34)

Using inequality (34) we obtain a multiplicative upper bound on the rate loss
(with respect to capacity), induced by using uncorrelated $N(0,P/d_{r})$
inputs. The resulting multiplicative factor depends on the number of
receivers, the channel matrix, and the SNR. The main point of Theorem 10 is
that it improves on the additive bound of [45] in the low-SNR regime:

######  Theorem 10.

Let $C({\rm H},Z;P)$ denote the capacity of the linear MIMO channel with
channel matrix $H\in\mathbb{R}^{d_{r}\times d_{t}}$, sum power constraint $P$,
and additive noise $Z$ having covariance matrix $N$. Let
$X^{*}\sim\mathcal{N}(0,\frac{P}{d_{r}}\mathrm{I}_{d_{r}})$ and denote ${\sf
snr}_{\text{\rm M}}=\frac{P}{\mathrm{tr}{N}}$. Then:

$I(X^{*};{\rm H}X^{*}+Z)\geq\frac{{\sf snr}_{\text{\rm M}}\det{({\rm H}{\rm
H}^{T})^{\frac{1}{d_{r}}}}}{{\sf snr}_{\text{\rm M}}\bigl{(}2\mathrm{tr}({\rm
H}{\rm H}^{T})+\det{({\rm H}{\rm H}^{T})^{\frac{1}{d_{r}}}}\bigr{)}+2}\times
C({\rm H},Z;P).$

###### Proof.

Let $X$ be any input with covariance $K_{X}$ satisfying (33), and take $X^{*}$
to be a Gaussian vector with covariance matrix
$\frac{P}{d_{r}}\mathrm{I}_{d_{r}}$. Then ${\rm H}X^{*}$ is a Gaussian with
covariance $\frac{P}{d_{r}}{\rm H}{\rm H}^{T}$, and applying (34) we get,

$\displaystyle I(X^{*};{\rm H}X^{*}+Z)$ $\displaystyle=I({\rm H}X^{*};{\rm
H}X^{*}+Z)$ $\displaystyle\geq\frac{\frac{P}{d_{r}}\det{({\rm H}{\rm
H}^{T})^{\frac{1}{d_{r}}}}}{2\det{({\rm H}K_{X}{\rm
H}^{T}+N)}^{\frac{1}{d_{r}}}+\frac{P}{d_{r}}\det{({\rm H}{\rm
H}^{T})^{\frac{1}{d_{r}}}}}\times I({\rm H}X;{\rm H}X+Z).$

Using that for square, symmetric, positive semidefinite matrices
$A,B\in\mathbb{R}^{d_{r}\times d_{r}}$, we have
$\det{(A)}^{\frac{1}{d_{r}}}\leq\frac{1}{d_{r}}\mathrm{tr}{A}$ and
$\mathrm{tr}(AB)\leq\mathrm{tr}(A)\mathrm{tr}(B)$, yields,

$\displaystyle I(X^{*};{\rm H}X^{*}+Z)$
$\displaystyle\geq\frac{\frac{P}{d_{r}}\det{({\rm H}{\rm
H}^{T})^{\frac{1}{d_{r}}}}}{\frac{2}{d_{r}}\mathrm{tr}({\rm H}K_{X}{\rm
H}^{T}+N)+\frac{P}{d_{r}}\det{({\rm H}{\rm H}^{T})^{\frac{1}{d_{r}}}}}\times
I({\rm H}X;{\rm H}X+Z)$ $\displaystyle=\frac{{P}\det{({\rm H}{\rm
H}^{T})^{\frac{1}{d_{r}}}}}{{2}\mathrm{tr}(K_{X}{\rm H}^{T}{\rm
H})+{2}\mathrm{tr}(N)+{P}\det{({\rm H}{\rm H}^{T})^{\frac{1}{d_{r}}}}}\times
I({\rm H}X;{\rm H}X+Z)$ $\displaystyle\geq\frac{P\det{({\rm H}{\rm
H}^{T})^{\frac{1}{d_{r}}}}}{2\mathrm{tr}(K_{X})\mathrm{tr}({\rm H}^{T}{\rm
H})+2\mathrm{tr}(N)+P\det{({\rm H}{\rm H}^{T})^{\frac{1}{d_{r}}}}}\times
I({\rm H}X;{\rm H}X+Z).$

Finally, since $X$ satisfies the sum power constraint (33),

$I(X^{*};{\rm H}X^{*}+Z)\geq\frac{P\det{({\rm H}{\rm
H}^{T})^{\frac{1}{d_{r}}}}}{P\bigl{(}2\mathrm{tr}({\rm H}^{T}{\rm
H})+\det{({\rm H}{\rm
H}^{T})^{\frac{1}{d_{r}}}}\bigr{)}+2\mathrm{tr}(N)}\times I(X;{\rm H}X+Z),$

and the result follows upon taking the supremum over all inputs $X$ satisfying
the constraint. ∎

## References

  * [1] S. Artstein, K. Ball, F. Barthe, and A. Naor.  Solution of Shannon’s problem on the monotonicity of entropy.  J. Amer. Math. Soc., 17(4):975–982, April 2004.
  * [2] J. Barbier, M. Dia, and N. Macris.  Universal sparse superposition codes with spatial coupling and GAMP decoding.  IEEE Trans. Inform. Theory, 65(9):5618–5642, September 2019.
  * [3] S.G. Bobkov and G.P. Chistyakov.  Entropy power inequality for the Rényi entropy.  IEEE Trans. Inform. Theory, 61(2):708–714, February 2015.
  * [4] S.G. Bobkov, G.P. Chistyakov, and F. Götze.  Stability problems in Cramer-type characterization in case of i.i.d. summands.  Theory Probab. Appl., 57(4):568–588, 2013.
  * [5] S.G. Bobkov, G.P. Chistyakov, and F. Götze.  Regularized distributions and entropic stability of Cramer’s characterization of the normal law.  Stochastic Process. Appl., 126(12):3865–3887, December 2016.
  * [6] S.G Bobkov, G.P. Chistyakov, and F. Götze.  Stability of Cramer’s characterization of normal laws in information distances.  In High dimensional probability VII, volume 71 of Progr. Probab., pages 3–35. Springer, 2016.
  * [7] S.G. Bobkov and M. Madiman.  On the problem of reversibility of the entropy power inequality.  In Limit theorems in probability, statistics and number theory, volume 42 of Springer Proc. Math. Stat., pages 61–74. Springer, Heidelberg, 2013.
  * [8] S.G. Bobkov, M. Madiman, and L. Wang.  Fractional generalizations of Young and Brunn-Minkowski inequalities.  In C. Houdré, M. Ledoux, E. Milman, and M. Milman, editors, Concentration, Functional Inequalities and Isoperimetry, volume 545 of Contemp. Math., pages 35–53. Amer. Math. Soc., 2011.
  * [9] E.A. Carlen and A. Soffer.  Entropy production by block variable summation and central limit theorems.  Comm. Math. Phys., 140(2):339–371, 1991.
  * [10] T.A. Courtade, M. Fathi, and A. Pananjady.  Quantitative stability of the entropy power inequality.  IEEE Trans. Inform. Theory, 64(8):5691–5703, August 2018.
  * [11] T.M. Cover and J.A. Thomas.  Elements of information theory.  John Wiley & Sons, New York, NY, second edition, 2012.
  * [12] H. Cramér.  Über eine Eigenschaft der normalen Verteilungsfunktion.  Math. Z., 41(1):405–414, 1936.
  * [13] A. Dembo, T.M. Cover, and J.A. Thomas.  Information-theoretic inequalities.  IEEE Trans. Inform. Theory, 37(6):1501–1518, November 1991.
  * [14] S.N. Diggavi and T.M. Cover.  The worst additive noise under a covariance constraint.  IEEE Trans. Inform. Theory, 47(7):3072–3081, November 2001.
  * [15] A. Dytso, S. Yagli, H.V. Poor, and S. Shamai.  The capacity achieving distribution for the amplitude constrained additive Gaussian channel: An upper bound on the number of mass points.  IEEE Trans. Inform. Theory, 66(4):2006–2022, April 2020.
  * [16] E. Ekrem and S. Ulukus.  On the secrecy of multiple access wiretap channel.  In Proceedings of the 46th Annual Allerton Conference on Communication, Control, and Computing, pages 1014–1021, Monticello, IL, September 2008.
  * [17] A. El Gamal and Y-H. Kim.  Network information theory.  Cambridge University Press, Cambridge, U.K., 2011.
  * [18] R. Eldan and D. Mikulincer.  Stability of the Shannon-Stam inequality via the Föllmer process.  Probab. Theory Related Fields, 177(3-4):891–922, 2020.
  * [19] A. Eskenazis, P. Nayar, and T. Tkocz.  Gaussian mixtures: Entropy and geometric inequalities.  Ann. Probab., 46(5):2908–2945, September 2018.
  * [20] M. Fradelizi, M. Madiman, A. Marsiglietti, and A. Zvavitch.  The convexification effect of Minkowski summation.  EMS Surv. Math. Sci., 5(1/2):1–64, 2019.
  * [21] M. Fradelizi, M. Madiman, M. Meyer, and A. Zvavitch.  On the volume of the Minkowski sum of zonoids.  J. Funct. Anal., 286(3), February 2024.
  * [22] M. Fradelizi, M. Madiman, and A. Zvavitch.  Sumset estimates in convex geometry.  arXiv e-prints, 2204.01565 [math.MG], June 2022.
  * [23] T.S. Han and S. Verdú.  Approximation theory of output statistics.  IEEE Trans. Inform. Theory, 39(3):752–772, May 1993.
  * [24] X. He and A. Yener.  The Gaussian many-to-one interference channel with confidential messages.  IEEE Trans. Inform. Theory, 57(5):2730–2745, May 2011.
  * [25] J. Huang, C. Pandit, S.P. Meyn, M. Médard, and V. Veeravalli.  Entropy, inference, and channel coding.  In Wireless communications, volume 143 of IMA Vol. Math. Appl., pages 99–124. Springer, New York, NY, 2007.
  * [26] S. Ihara.  On the capacity of channels with additive non-Gaussian noise.  Inf. Contr., 37(1):34–39, April 1978.
  * [27] A. Joseph and A.R. Barron.  Least squares superposition codes of moderate dictionary size are reliable at rates up to capacity.  IEEE Trans. Inform. Theory, 58(5):2541–2557, May 2012.
  * [28] A.M. Kagan, T. Yu, A.R. Barron, and M. Madiman.  Contribution to the theory of Pitman estimators.  Zap. Nauch. Sem. P.O.M.I., 408:245–267, 2012.  Special issue in honour of I.A. Ibragimov’s 80th birthday.
  * [29] V.A. Kaimanovich and A.M. Vershik.  Random walks on discrete groups: Boundary and entropy.  Ann. Probab., 11(3):457–490, August 1983.
  * [30] I. Kontoyiannis and M. Madiman.  Sumset and inverse sumset inequalities for differential entropy and mutual information.  IEEE Trans. Inform. Theory, 60(8):4503–4514, August 2014.
  * [31] I. Kontoyiannis and R. Zamir.  Mismatched codebooks and the role of entropy coding in lossy data compression.  IEEE Trans. Inform. Theory, 52(5):1922–1938, May 2006.
  * [32] A. Lapidoth.  Nearest neighbor decoding for additive non-Gaussian noise channels.  IEEE Trans. Inform. Theory, 42(5):1520–1529, September 1996.
  * [33] P. Lévy.  Théorie de l’addition des variables aléatoires.  Gauthier-Villars, Paris, France, 1937.
  * [34] Ju.V. Linnik and Ĭ.V. Ostrovskiĭ.  Decomposition of random variables and vectors, volume 48 of Translations of Mathematical Monographs.  American Mathematical Society, Providence, RI, 1977.  Translated from the Russian.
  * [35] D.J. Love.  Duplex distortion models for limited feedback MIMO communication.  IEEE Trans. Signal Process., 54(2):766–774, February 2006.
  * [36] M. Madiman and A.R Barron.  Generalized entropy power inequalities and monotonicity properties of information.  IEEE Trans. Inform. Theory, 53(7):2317–2329, July 2007.
  * [37] M. Madiman and F. Ghassemi.  Combinatorial entropy power inequalities: A preliminary study of the Stam region.  IEEE Trans. Inform. Theory, 65(3):1375–1386, March 2019.
  * [38] M. Madiman and I. Kontoyiannis.  The entropies of the sum and the difference of two IID random variables are not too different.  In 2010 IEEE International Symposium on Information Theory (ISIT), pages 1369–1372, Austin, TX, June 2010.
  * [39] M. Madiman and I. Kontoyiannis.  Entropy bounds on abelian groups and the Ruzsa divergence.  IEEE Trans. Inform. Theory, 64(1):77–92, January 2016.
  * [40] M. Madiman, A.W. Marcus, and P. Tetali.  Entropy and set cardinality inequalities for partition-determined functions.  Random Struct. Algorithms, 40(4):399–424, 2012.
  * [41] M. Madiman, J. Melbourne, and P. Xu.  Forward and reverse entropy power inequalities in convex geometry.  In E. Carlen, M. Madiman, and E. M. Werner, editors, Convexity and Concentration, volume 161 of IMA Volumes in Mathematics and its Applications, pages 427–485. Springer, 2017.
  * [42] M. Madiman, P. Nayar, and T. Tkocz.  Two remarks on generalized entropy power inequalities.  In B. Klartag and E. Milman, editors, Geometric aspects of functional analysis: Israel Seminar (GAFA) 2017-2019, volume 2266 of Lecture Notes in Mathematics, pages 169–185. Springer, 2020.
  * [43] M. Madiman, P. Nayar, and T. Tkocz.  Sharp moment-entropy inequalities and capacity bounds for log-concave distributions.  IEEE Trans. Inform. Theory, 67(1):81–94, January 2021.
  * [44] M.M. Mahvari and G. Kramer.  Stability of Bernstein’s theorem and soft doubling for vector Gaussian channels.  IEEE Trans. Inform. Theory, 69(10), October 2023.
  * [45] T. Philosof and R. Zamir.  The cost of uncorrelation and noncooperation in MIMO channels.  IEEE Trans. Inform. Theory, 53(11):3904–3920, November 2007.
  * [46] Y. Polyanskiy and S. Verdú.  Empirical distribution of good channel codes with nonvanishing error probability.  IEEE Trans. Inform. Theory, 60(1):5–21, January 2014.
  * [47] N.A. Sapogov.  The problem of stability for a theorem of Cramér’s.  Vestnik Leningrad. Univ., 10(11):61–64, 1955.
  * [48] J. Scarlett, V.Y.F. Tan, and G. Durisi.  The dispersion of nearest-neighbor decoding for additive non-Gaussian channels.  IEEE Trans. Inform. Theory, 63(1):81–92, January 2017.
  * [49] S. Shamai and S. Verdú.  The empirical distribution of good codes.  IEEE Trans. Inform. Theory, 43(3):836–846, May 1997.
  * [50] C.E. Shannon.  A mathematical theory of communication.  Bell System Tech. J., 27(3):379–423, 623–656, 1948.
  * [51] D. Shlyakhtenko.  A free analogue of Shannon’s problem on monotonicity of entropy.  Adv. Math., 208(2):824–833, January 2007.
  * [52] N. Shulman and M. Feder.  The uniform distribution as a universal prior.  IEEE Trans. Inform. Theory, 50(6):1356–1362, June 2004.
  * [53] X. Tang, R. Liu, P. Spasojević, and H.V. Poor.  Interference assisted secret communication.  IEEE Trans. Inform. Theory, 57(5):3153–3167, May 2011.
  * [54] T. Tao.  Sumset and inverse sumset theory for Shannon entropy.  Comb., Probab. Comput., 19(4):603–639, July 2010.
  * [55] A. Tchamkerten.  On the discreteness of capacity-achieving distributions.  IEEE Trans. Inform. Theory, 50(11):2773–2778, November 2004.
  * [56] A.M. Tulino and S. Verdú.  Monotonic decrease of the non-Gaussianness of the sum of independent random variables: A simple proof.  IEEE Trans. Inform. Theory, 52(9):4295–4297, September 2006.
  * [57] R. Venkataramanan, S. Tatikonda, and A.R. Barron.  Sparse regression codes.  Foundations and Trends in Communications and Information Theory, 15(1-2):1–195, June 2019.
  * [58] S. Verdú.  On channel capacity per unit cost.  IEEE Trans. Inform. Theory, 36(5):1019–1030, September 1990.
  * [59] R. Zamir.  The half a bit loss of robust source/channel codebooks.  In 2002 IEEE Workshop on Information Theory (ITW), pages 123–126, Bangalore, India, October 2002.
  * [60] R. Zamir and U. Erez.  A Gaussian input is not too bad.  IEEE Trans. Inform. Theory, 50(6):1362–1367, June 2004.
  * [61] R. Zamir and M. Feder.  On universal quantization by randomized uniform/lattice quantizers.  IEEE Trans. Inform. Theory, 38(2):428–436, March 1992.


# Robust quantisation of circular photogalvanic effect in multiplicative
topological semimetals

Adipta Pal  Max Planck Institute for Chemical Physics of Solids, Nöthnitzer
Strasse 40, 01187 Dresden, Germany  Max Planck Institute for the Physics of
Complex Systems, Nöthnitzer Strasse 38, 01187 Dresden, Germany  Dániel Varjas
Max Planck Institute for Chemical Physics of Solids, Nöthnitzer Strasse 40,
01187 Dresden, Germany  Max Planck Institute for the Physics of Complex
Systems, Nöthnitzer Strasse 38, 01187 Dresden, Germany  IFW Dresden and
Würzburg-Dresden Cluster of Excellence ct.qmat, Helmholtzstr. 20, 01069
Dresden, Germany  Ashley M. Cook  Max Planck Institute for Chemical Physics of
Solids, Nöthnitzer Strasse 40, 01187 Dresden, Germany  Max Planck Institute
for the Physics of Complex Systems, Nöthnitzer Strasse 38, 01187 Dresden,
Germany

###### Abstract

Nonlinear response signatures are increasingly recognized as useful probes of
condensed matter systems, in particular for characterisation of topologically
non-trivial states. The circular photogalvanic effect (CPGE) is particularly
useful in study of topological semimetals, as the CPGE tensor quantises for
well-isolated topological degeneracies in strictly linearly-dispersing band
structures. Here, we study multiplicative Weyl semimetal band-structures, and
find that the multiplicative structure robustly protects the quantization of
the CPGE even in the case of non-linear dispersion. Computing phase diagrams
as a function of Weyl node tilting, we find a variety of quantised values for
the CPGE tensor, revealing that the CPGE is also a useful tool in detecting
and characterising parent topology of multiplicative topological states.

Discovery of the quantum Hall effect Klitzing  _et al._ (1980); Tsui  _et al._
(1982) (QHE) has, over the past four decades, expanded into a large branch of
condensed matter physics and revealed the central role of topology in this
field Laughlin (1983a, b); Kallin and Halperin (1984); Zhang  _et al._ (1989);
Halperin  _et al._ (1993); Halperin (1982, 1984); Wen (1991); Jain (1989);
Bernevig  _et al._ (2002); Nayak  _et al._ (2008); Sarma  _et al._ (2015);
Freedman  _et al._ (2002); Kitaev (2001). Experimental signatures of the QHE
are undeniably topological: a two-dimensional electron gas subjected to an
out-of-plane applied magnetic field realizes Hall conductivity quantised to
rational numbers $\nu$ in units of $\frac{e^{2}}{h}$, to one part in a
billion. Many symmetry-protected topological phases (SPTs) descending from the
integer QHE lack such distinctive signatures Ryu  _et al._ (2010); Schnyder
_et al._ (2008); Qi  _et al._ (2008); Kitaev  _et al._ (2009); Altland and
Zirnbauer (1997); Chiu  _et al._ (2016); Hsieh  _et al._ (2008); Chang  _et
al._ (2013); Else  _et al._ (2021), however, and are instead characterized by
subtler response signatures or evidence of bulk-boundary correspondence. It is
therefore essential to seek out quantised response signatures and better
understand the origins of this distinction between the foundational QHE and
the large body of symmetry-protected topological phases now studied.

One of the more recent examples of quantised response in topological phases of
matter is the circular photogalvanic effect (CPGE), a second-order response to
circularly polarized light in topological semimetals De Juan  _et al._ (2017).
For Weyl semimetals Yan and Felser (2017); Jia  _et al._ (2016); Armitage  _et
al._ (2018); Burkov and Balents (2011); Halász and Balents (2012); Haldane
(2014); Mathai and Thiang (2017); Hosur (2012); Potter  _et al._ (2014);
Nielsen and Ninomiya (1983); Zyuzin and Burkov (2012); Son and Spivak (2013);
Parameswaran  _et al._ (2014); Huang  _et al._ (2015); Zhang  _et al._ (2016)
(WSMs), the CPGE response is quantised in the limit of small frequency
compared to the energy splitting to other bands at the Weyl-node De Juan  _et
al._ (2017), as additional bands yield unquantised corrections. The CPGE
response was also studied for multi-Weyl and multifold fermions Flicker  _et
al._ (2018); Le  _et al._ (2020); Le and Sun (2021); Ni  _et al._ (2021), but
quantisation relies on the linearity of the Hamiltonian near the degeneracy
point.

To probe the limits of quantised response in topological phases of matter, in
this work we therefore consider response signatures of the recently-introduced
multiplicative topological phases of matter Cook and Moore (2022),
distinguished by Bloch Hamiltonians with symmetry-protected tensor product
structure. We investigate the CPGE in multiplicative Weyl semimetals (MWSMs)
specifically as the canonical multiplicative topological semimetals Pal  _et
al._ (2023). Considering a variety of toy models for MWSMs, we find rich phase
diagrams displaying a wide range of quantised values of the CPGE coefficient.
Notably, we find quantisation of the CPGE that persists in the presence of
more than two bands, and also in the presence of non-linear corrections. We
therefore demonstrate that multiplicative topological phases are an invaluable
tool in pursuit of quantised topological response signatures.

Figure 1: Schematic band structures showing the parallel multiplicative Weyl
semimetal construction. Top: even-odd tilting, bottom: odd-odd tilting. All
the Weyl nodes are located at $\mathbf{k}=\left(0,0,\pm\pi/2\right)$ with the
energies shifted in a $\mathbf{k}$-even or odd fashion. The colors indicate
the the different band indices.

II. Results–

II.1. Multiplicative Weyl semimetal and CPGE response contributions– The
central quantity we focus on in this manuscript is the CPGE tensor $\beta$,
which characterizes the injection current response to applied circularly
polarized light De Juan  _et al._ (2017) defined through
$dj_{i}/dt=\beta_{ij}(\omega)(\mathbf{E}(\omega)\times\mathbf{E}(-\omega))^{j}$.
The trace of the CPGE tensor is given in terms of the band structure by

$\displaystyle\operatorname{Tr}\beta(\omega)=$ (1) $\displaystyle\frac{\pi
e^{3}}{\hbar
V}\epsilon_{ijk}\sum_{\mathbf{k},n,m}f_{\mathbf{k},n,m}\Delta_{\mathbf{k},n,m}^{i}r^{j}_{\mathbf{k},n,m}r^{k}_{\mathbf{k},m,n}\delta(\hbar\omega-
E_{\mathbf{k},n,m}),$

where $E_{\mathbf{k},n,m}=E_{\mathbf{k},n}-E_{\mathbf{k},m}$ with
$E_{\mathbf{k},m}$ the band energies,
$f_{\mathbf{k},n,m}=f(E_{\mathbf{k},n})-f(E_{\mathbf{k},m})$ with $f(E)$ the
Fermi function,
$\Delta_{\mathbf{k},n,m}^{i}=(1/\hbar)\partial_{i}E_{\mathbf{k},n,m}$, and
$r^{i}_{\mathbf{k},n,m}=i\bra{n,\mathbf{k}}\partial_{i}\ket{m,\mathbf{k}}$ the
inter-band Berry connection. The derivatives $\partial_{i}$ are with respect
to the momentum component $k_{i}$. After a change of variables, and using the
fact that the delta function is only nonzero on closed surfaces $S_{nm}$ where
$\hbar\omega=E_{\mathbf{k},n,m}$ is satisfied, we rewrite this in terms of a
sum of surface integrals:

$\operatorname{Tr}\beta(\omega)=\frac{\pi
e^{3}}{\hbar}\epsilon_{ijk}\sum_{n,m}\int_{S_{nm}}d\mathbf{k}^{2}\hat{n}_{\mathbf{k},n,m}^{i}f_{\mathbf{k},n,m}r^{j}_{\mathbf{k},n,m}r^{k}_{\mathbf{k},m,n},$
(2)

where $\hat{n}_{\mathbf{k},n,m}^{i}$ is the surface normal vector. In case of
a two-band system, using the identity ${\bf\it\Omega}_{n}=i\sum_{m\neq
n}\mathbf{r}_{nm}\times\mathbf{r}_{mn}$ expressing the Berry-curvature in
terms of the inter-band Berry connection, and assuming that at zero
temperature $f_{1,2}=1$ for the entire surface $S_{12}$, we obtain the
celebrated result $\operatorname{Tr}\beta(\omega)=\frac{i\pi e^{3}}{h^{2}}C$,
showing that CPGE is quantised proportional to the Weyl-node charge De Juan
_et al._ (2017). This result remains approximately valid for twofold
degenerate Weyl-nodes even when higher bands are included, provided that the
gap to the other bands is large compared to $\omega$. It is tempting to
conclude that the quantization is robust for multifold Fermions as well, as
long as full momentum shells of allowed transitions from all occupied bands to
all unoccupied bands are available Flicker  _et al._ (2018); Le  _et al._
(2020); Le and Sun (2021); Ni  _et al._ (2021), allowing us to rewrite (1) in
terms of Berry curvatures. This, however, is only valid for
$\mathbf{k}$-linear Hamiltonians, where the eigenstates do not depend on
$|\mathbf{k}|$, as noted in Ref. Flicker  _et al._ , 2018, resulting in
corrections to the quantised values if higher-order terms are included in the
Hamiltonian, even when no higher bands are present.

We study several variants of the MWSMs. Previously-studied examples are
characterised by Bloch Hamiltonians, which are symmetry-protected tensor-
products of two “parent” Bloch-Hamiltonians $H_{1}(\mathbf{k})$ and
$H_{2}(\mathbf{k})$ resulting in the child
$H_{c}(\mathbf{k})=H_{1}(\mathbf{k})\otimes H_{2}(\mathbf{k})$. We choose each
parent to be a two-band Weyl-semimetal, review the general results regarding
the CPGE, and in the next section we present phase-diagrams with various Weyl-
node configurations. In a MWSM with $2\times 2$ bands (two from each WSM
parent), we replace the band-indices $n$ with a pair of binary indices
$s_{1},s_{2}$ each taking $\pm$ values, referring to the pair of bands in the
parents. The energies and the eigenstates factorize as
$E_{\mathbf{k},s_{1}s_{2}}=E^{(1)}_{\mathbf{k},s_{1}}E^{(2)}_{\mathbf{k},s_{2}}$,
and
$\ket{\mathbf{k},s_{1}s_{2}}=\ket{\mathbf{k},s_{1}}_{1}\otimes\ket{\mathbf{k},s_{2}}_{2}$.
As a result, the inter-band Berry connection factorizes as:

$\mathbf{r}_{s_{1}s_{2},s^{\prime}_{1}s^{\prime}_{2}}=\delta_{s_{1}s^{\prime}_{1}}\mathbf{r}^{(2)}_{s_{2}s^{\prime}_{2}}+\delta_{s_{2}s^{\prime}_{2}}\mathbf{r}^{(1)}_{s_{1}s^{\prime}_{1}},$
(3)

where we suppressed the $\mathbf{k}$-dependence. Substituting this into (1),
and using the fact that, with $\omega\neq 0$, terms with $n=m$ do not
contribute, we find that the only remaining terms are

$r^{(1)j}_{+-}r^{(1)k}_{-+},\;r^{(1)j}_{-+}r^{(1)k}_{+-},\;r^{(2)j}_{+-}r^{(2)k}_{-+},\;r^{(2)j}_{-+}r^{(2)k}_{+-},$
(4)

which can be viewed as a type of selection rule for the allowed transitions,
only allowing four of the six possible transitions between the four bands, see
Fig. 2 d). As all of these contributions only involve the Berry connection of
a single two-band parent, they are individually quantised if the transitions
are allowed for entire closed momentum shells. These contributions may cancel
each other, however. For example, if
$f_{\mathbf{k},+-,--}=f_{\mathbf{k},-+,++}=1$ (for an entire shell of
$\mathbf{k}$), then the contributions from $r^{(1)j}_{+-}r^{(1)k}_{-+}$ and
$r^{(1)j}_{-+}r^{(1)k}_{+-}$ cancel, and similarly for parent 2. This is the
case when the two parents are both simple Weyl-nodes at the same $\mathbf{k}$
and at zero energy.

II.2. Models and symmetry considerations– Our starting point is the simple
two-band WSM Hamiltonian

$\mathcal{H}^{0}_{i,z}(\mathbf{k})=(M_{i}(\mathbf{k})\sigma^{z}+\sin
k_{x}\sigma^{x}+\sin k_{y}\sigma^{y})$ (5)

where $M_{i}(\mathbf{k})=2+\gamma_{i}-\sum_{j=x,y,z}\cos k_{j}$, we reserve
the index $i=1,2$ to index the parents later. This Hamiltonian describes a WSM
with Weyl-nodes separated along the $z$ axis. The separation is controlled by
$\gamma_{i}$, for most of the following we choose $\gamma_{i}=0$ so the Weyl-
nodes are located at $\mathbf{k}=\left(0,0,\pm\pi/2\right)$. We analogously
define $\mathcal{H}^{0}_{i,y}$ with Weyl-nodes separated in the $y$ direction,
the only difference from $\mathcal{H}^{0}_{i,z}$ is that the role of $k_{y}$
and $k_{z}$ is interchanged. Following Ref. Pal  _et al._ , 2023 (up to a
change of basis), we define the MWSM parallel and perpendicular Bloch
Hamiltonians:

$\begin{split}\mathcal{H}^{0}_{\parallel}(\mathbf{k})&=\mathcal{H}^{0}_{1,z}(\mathbf{k})\otimes\mathcal{H}^{0}_{2,z}(\mathbf{k}),\end{split}$
(6a)
$\begin{split}\mathcal{H}^{0}_{\perp}(\mathbf{k})&=\mathcal{H}^{0}_{1,y}(\mathbf{k})\otimes\mathcal{H}^{0}_{2,z}(\mathbf{k}),\end{split}$
(6b)

with further details in the Appendix, section B.

The two systems defined above have both time reversal $(\mathcal{T}^{2}=+1)$
and inversion symmetry. A net non-zero quantised CPGE is only possible when
inversion symmetry is broken, so that the contributions from Berry curvature
around both the Weyl nodes at the same energy do not cancel De Juan  _et al._
(2017). In order to break these symmetries, we introduce a _tilting_ term,
shifting the band energies:

$\begin{split}\mathcal{H}_{i,j}&=t_{0i}f_{i}(k_{j})\sigma^{0}+\mathcal{H}^{0}_{i,j}(\mathbf{k})\end{split}$
(7)

where we choose $f_{i}$ as either an even (specifically constant $1$) or an
odd (sine) function of $k_{j}$, and define the MWSM Hamiltonians the same way
as Eqns. (6) with the $0$ index dropped. See Fig. 2 a, b) for an example
parallel MWSM band structure.

In both the parallel and the perpendicular MWSM case, if $f_{1}$ is odd and
$f_{2}$ is even (or vice-versa), both inversion and time reversal symmetries
are broken. If $f_{1}$ and $f_{2}$ are both odd functions, inversion symmetry
is broken but time reversal symmetry is still preserved, and as a result, the
time-reversed Weyl nodes contribute the same to the CPGE, and only even
quantised values are allowed. If both parents have even tilting, inversion
symmetry is maintained and the CPGE vanishes.

II.4. Phase diagram– We calculate the CPGE from the lattice model by
numerically evaluating eqn. 1, see Appendix B for details. The resulting phase
diagram is shown in Fig. 3 b). As a function of the tilting parameters
$t_{01}$ and $t_{02}$, we observe large regions with the CPGE trace quantised
to high accuracy, with intervening areas of non-quantised values. This is
expected when the shift in the node energies becomes comparable to the band-
width, and the allowed transitions no longer occur on closed momentum-shells
as required for quantization.

Figure 2: Parallel multiplicative Weyl semimetal band structures along the
$k_{z}$ axis. Two parents $a)$ with even and odd energy shifts of the Weyl
nodes (orange and green) produce the child band structure $b)$. Panels $c)$
and $d)$ show the linearized band structure around the node at $k_{z}=\pi/2$
(red box).Panel $d)$ shows the child band structure (solid lines where
blue/red corresponds to bands with $s_{1}s_{2}=\pm 1$), the band structure
shifted by $\omega$ (dotted), the Fermi level (dashed black) and the Fermi
level shifted by $\omega$ (dash-dot black). The allowed transitions can be
read off the diagram as intersections of solid and dotted lines of different
color between the two black lines, marked by arrows.The parameters
$t_{01}=0.5$, $t_{02}=1$, $\omega=1$ and $E_{F}=0.8$ are used in all panels.

As a comparison, we also calculate a semi-analytical phase diagram using a
linearized continuum model. As argued in Sec. II.1., and confirmed by the
numerics on the lattice model, quadratic terms do not influence the
quantization, justifying the simplification. First we consider a single
isolated Weyl point in both parents, located at the same momentum:

$H(\mathbf{k})=(\mathbf{q}\cdot\boldsymbol{\sigma}+t_{01}\mathbbm{1})\otimes(\mathbf{q}\cdot\boldsymbol{\sigma}+t_{02}\mathbbm{1}),$
(8)

with $\mathbf{q}=\mathbf{k}-\mathbf{k}_{0}$ the distance from the Weyl point
momentum, see Fig. 2 c, d). Here we set the velocity and charge of both Weyl
points to $1$, with an overall energy shift determined by $t_{0i}$. The
spectrum is

$E(q)_{s_{1},s_{2}}=(s_{1}q+t_{01})(s_{2}q+t_{02}),$ (9)

where we use $q=|\mathbf{q}|$ as a result of the spherical symmetry of the
problem, and $s_{i}=\pm 1$ specify the branch of the dispersion. As we saw
before, only transitions where one of the band indices change contribute to
the CPGE. For specificity, let us assume that $s_{2}$ changes (the other case
can be treated by interchanging $1\leftrightarrow 2$). The condition for a
transition is:

$E(q)_{s_{1},s_{2}}-E(q)_{s_{1},-s_{2}}=\omega,$ (10)

which can be solved for the momentum $q_{0}$ of the allowed transitions:

$q_{0}=-s_{1}\frac{t_{01}}{2}\pm\sqrt{t_{01}^{2}-2s_{1}s_{2}\omega}.$ (11)

For this to correspond to an allowed transition from an occupied to an empty
state, we also require

$E(q_{0})_{s_{1},s_{2}}<E_{F}<E(q_{0})_{s_{1},-s_{2}},$ (12)

taking the Fermi function as a step function, see Fig. 2 d) for an
illustrative example. The value of $\beta$ changes where one of these
inequalities becomes an equality and a transition becomes allowed or
forbidden. These result in quadratic equations in $t_{01}$ and $t_{02}$,
giving phase boundaries along conic sections. The value of $\beta$ is given by
summing over all of the allowed transitions (with $q_{0}>0$ to avoid double-
counting), multiplying with the sign $s_{1}s_{2}$ and the sign of the velocity
difference between the two bands at $k_{0}$. The resulting phase diagram,
taking into account the total contribution of the two Weyl nodes treated the
above way, is shown in Fig. 3 a) and c). The two approaches result in similar
phase diagrams in both the even-odd and the odd-odd tilting case, with some
differences near the phase boundaries, especially for large parameters, as
expected.

Figure 3: Phase diagram of the parallel MWSM as a function of the tilting
parameters of the parents. Continuum approximation (a) and numerical
evaluation from the lattice model (b) in the even-odd case. Continuum
approximation (c) and lattice model (d) in the odd-odd case. All phase
diagrams are with parameters $\omega=1$ and $E_{F}=0.8$.

We also consider a perpendicular MWSM, where the role of $k_{y}$ and $k_{z}$
is reversed in the first parent relative to the second parent. The analytical
calculation of the phase diagram in this case is analogous to the parallel
case. The fact that only one parent has a Weyl-node in the vicinity of any
$\mathbf{k}$-point allows for simplifications. In the even-odd case, at linear
order the dispersion around the Weyl-node of the second parent located at
$\mathbf{k}=\left(0,0,\pm\pi/2\right)$ is

$E(q)_{s_{1},s_{2}}=(s_{1}+t_{01})(s_{2}q\pm t_{02}),$ (13)

where we assumed that $\gamma_{1}=\gamma_{2}=0$. The linearized dispersion
around the Weyl-nodes of the first parent located at
$\mathbf{k}=\left(0,\pm\pi/2,0\right)$ are

$E(q)_{s_{1},s_{2}}=(s_{1}q+t_{01})s_{2},$ (14)

with no energy shift proportional to $t_{02}$ at the node locations because of
the odd tilting in the second parent. We neglect the anisotropic
$q_{z}$-linear energy shift in the second parent, because our semi-analytical
treatment assumes spherical symmetry, this in essence means that we average
over rotations around the Weyl-node. The contributions to the CPGE are
opposite for the two nodes, resulting from the opposite charge of the nodes.
We use the same method as for the parallel case to find the allowed
transitions and their contributions for each node, the resulting total CPGE is
shown in Figs. 4. The phase diagram in the perpendicular odd-odd case is very
simple, as the dispersion around all four nodes has the form of (14), hence
the dependence of the CPGE on the tiltings completely decouples for the two
parents in the low frequency (continuum) limit.

Figure 4: Phase diagram of the perpendicular MWSM as a function of the tilting
parameters of the parents. Continuum approximation (a) and numerical
evaluation from the lattice model (b) in the even-odd case. Continuum
approximation (c) and lattice model (d) in the odd-odd case. All phase
diagrams are with parameters $\omega=1$ and $E_{F}=0.8$.

III. Discussion and Conclusion–In this work, we characterise the circular
photogalvanic effect in multiplicative topological semimetals, or topological
semimetals characterised by Bloch Hamiltonians with symmetry-protected tensor
product structure. Given past work on quantisation of the CPGE for systems
with Weyl nodes and multifold fermions, we explore the potential of
multiplicative Weyl semimetals in realising quantised CPGE.

For both parallel and perpendicular MWSMs with two parent Weyl semimetals as
introduced in past work, we find rich phase diagrams as a function of the
tilting parameters for the parents. We obtain these results both numerically
from lattice tight-binding models, and analytically using a continuum
approximation, and find the same phase diagrams up to expected corrections.
The multiplicative structure yields quantisation in systems with more than two
bands even in the presence of non-linear momentum dependence over large
regions of phase space, revealing an approach to more robust quantisation than
realized previously. This demonstrates the potential of multiplicative
topological states in realizing quantised observables and exotic optical
signatures promising for applications. Furthermore, our work demonstrates that
CPGE is a versatile probe in characterising complex topology, distinguishing
topological degeneracies of multiplicative topological semimetals from
ostensibly similar topological degeneracies according to the degree of the
degeneracy and net topological charge, such as Dirac nodes.

###### Acknowledgements.

We thank J. Winter for helpful discussions. This research was supported in
part by the National Science Foundation under Grants No.NSF PHY-1748958 and
PHY-2309135, and undertaken in part at Aspen Center for Physics, which is
supported by National Science Foundation grant PHY-2210452. D.V. was supported
by the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) under
Germany’s Excellence Strategy through the Würzburg-Dresden Cluster of
Excellence on Complexity and Topology in Quantum Matter – ct.qmat (EXC 2147,
project-id 57002544).

## References

  * Klitzing  _et al._ (1980) K. v. Klitzing, G. Dorda, and M. Pepper, Phys. Rev. Lett. 45, 494 (1980).
  * Tsui  _et al._ (1982) D. C. Tsui, H. L. Stormer, and A. C. Gossard, Phys. Rev. Lett. 48, 1559 (1982).
  * Laughlin (1983a) R. B. Laughlin, Phys. Rev. Lett. 50, 1395 (1983a).
  * Laughlin (1983b) R. B. Laughlin, Phys. Rev. Lett. 50, 1395 (1983b).
  * Kallin and Halperin (1984) C. Kallin and B. Halperin, Physical Review B 30, 5655 (1984).
  * Zhang  _et al._ (1989) S. C. Zhang, T. H. Hansson, and S. Kivelson, Phys. Rev. Lett. 62, 82 (1989).
  * Halperin  _et al._ (1993) B. I. Halperin, P. A. Lee, and N. Read, Physical Review B 47, 7312 (1993).
  * Halperin (1982) B. I. Halperin, Physical Review B 25, 2185 (1982).
  * Halperin (1984) B. I. Halperin, Physical Review Letters 52, 1583 (1984).
  * Wen (1991) X.-G. Wen, Physical Review B 43, 11025 (1991).
  * Jain (1989) J. K. Jain, Phys. Rev. Lett. 63, 199 (1989).
  * Bernevig  _et al._ (2002) B. A. Bernevig, C.-H. Chern, J.-P. Hu, N. Toumbas, and S.-C. Zhang, Annals of Physics 300, 185 (2002).
  * Nayak  _et al._ (2008) C. Nayak, S. H. Simon, A. Stern, M. Freedman, and S. Das Sarma, Rev. Mod. Phys. 80, 1083 (2008).
  * Sarma  _et al._ (2015) S. D. Sarma, M. Freedman, and C. Nayak, npj Quantum Information 1, 15001 (2015).
  * Freedman  _et al._ (2002) M. H. Freedman, M. J. Larsen, and Z. Wang, Communications in Mathematical Physics 228, 177 (2002).
  * Kitaev (2001) A. Y. Kitaev, Physics-Uspekhi 44, 131 (2001).
  * Ryu  _et al._ (2010) S. Ryu, A. P. Schnyder, A. Furusaki, and A. W. W. Ludwig, New Journal of Physics 12, 065010 (2010).
  * Schnyder  _et al._ (2008) A. P. Schnyder, S. Ryu, A. Furusaki, and A. W. W. Ludwig, Phys. Rev. B 78, 195125 (2008).
  * Qi  _et al._ (2008) X.-L. Qi, T. L. Hughes, and S.-C. Zhang, Phys. Rev. B 78, 195424 (2008).
  * Kitaev  _et al._ (2009) A. Kitaev, V. Lebedev, and M. Feigel’man, in  _AIP Conference Proceedings_ (AIP, 2009).
  * Altland and Zirnbauer (1997) A. Altland and M. R. Zirnbauer, Phys. Rev. B 55, 1142 (1997).
  * Chiu  _et al._ (2016) C.-K. Chiu, J. C. Y. Teo, A. P. Schnyder, and S. Ryu, Rev. Mod. Phys. 88, 035005 (2016).
  * Hsieh  _et al._ (2008) D. Hsieh, D. Qian, L. Wray, Y. Xia, Y. S. Hor, R. J. Cava, and M. Z. Hasan, Nature 452, 970 (2008).
  * Chang  _et al._ (2013) C.-Z. Chang, J. Zhang, X. Feng, J. Shen, Z. Zhang, M. Guo, K. Li, Y. Ou, P. Wei, L.-L. Wang, Z.-Q. Ji, Y. Feng, S. Ji, X. Chen, J. Jia, X. Dai, Z. Fang, S.-C. Zhang, K. He, Y. Wang, L. Lu, X.-C. Ma, and Q.-K. Xue, Science 340, 167 (2013), https://www.science.org/doi/pdf/10.1126/science.1234414 .
  * Else  _et al._ (2021) D. V. Else, R. Thorngren, and T. Senthil, Phys. Rev. X 11, 021005 (2021).
  * De Juan  _et al._ (2017) F. De Juan, A. G. Grushin, T. Morimoto, and J. E. Moore, Nature communications 8, 15995 (2017).
  * Yan and Felser (2017) B. Yan and C. Felser, Annual Review of Condensed Matter Physics 8, 337 (2017), https://doi.org/10.1146/annurev-conmatphys-031016-025458 .
  * Jia  _et al._ (2016) S. Jia, S.-Y. Xu, and M. Z. Hasan, Nature Materials 15, 1140 (2016).
  * Armitage  _et al._ (2018) N. P. Armitage, E. J. Mele, and A. Vishwanath, Rev. Mod. Phys. 90, 015001 (2018).
  * Burkov and Balents (2011) A. A. Burkov and L. Balents, Phys. Rev. Lett. 107, 127205 (2011).
  * Halász and Balents (2012) G. B. Halász and L. Balents, Physical Review B 85, 035103 (2012).
  * Haldane (2014) F. Haldane, arXiv preprint arXiv:1401.0529 (2014).
  * Mathai and Thiang (2017) V. Mathai and G. C. Thiang, Journal of Physics A: Mathematical and Theoretical 50, 11LT01 (2017).
  * Hosur (2012) P. Hosur, Physical Review B 86, 195102 (2012).
  * Potter  _et al._ (2014) A. C. Potter, I. Kimchi, and A. Vishwanath, Nature communications 5, 1 (2014).
  * Nielsen and Ninomiya (1983) H. Nielsen and M. Ninomiya, Physics Letters B 130, 389 (1983).
  * Zyuzin and Burkov (2012) A. Zyuzin and A. Burkov, Physical Review B 86, 115133 (2012).
  * Son and Spivak (2013) D. T. Son and B. Z. Spivak, Phys. Rev. B 88, 104412 (2013).
  * Parameswaran  _et al._ (2014) S. A. Parameswaran, T. Grover, D. A. Abanin, D. A. Pesin, and A. Vishwanath, Phys. Rev. X 4, 031035 (2014).
  * Huang  _et al._ (2015) X. Huang, L. Zhao, Y. Long, P. Wang, D. Chen, Z. Yang, H. Liang, M. Xue, H. Weng, Z. Fang, X. Dai, and G. Chen, Phys. Rev. X 5, 031023 (2015).
  * Zhang  _et al._ (2016) C.-L. Zhang, S.-Y. Xu, I. Belopolski, Z. Yuan, Z. Lin, B. Tong, G. Bian, N. Alidoust, C.-C. Lee, S.-M. Huang, T.-R. Chang, G. Chang, C.-H. Hsu, H.-T. Jeng, M. Neupane, D. S. Sanchez, H. Zheng, J. Wang, H. Lin, C. Zhang, H.-Z. Lu, S.-Q. Shen, T. Neupert, M. Zahid Hasan, and S. Jia, Nature Communications 7, 10735 (2016).
  * Flicker  _et al._ (2018) F. Flicker, F. de Juan, B. Bradlyn, T. Morimoto, M. G. Vergniory, and A. G. Grushin, Phys. Rev. B 98, 155145 (2018).
  * Le  _et al._ (2020) C. Le, Y. Zhang, C. Felser, and Y. Sun, Phys. Rev. B 102, 121111 (2020).
  * Le and Sun (2021) C. Le and Y. Sun, Journal of Physics: Condensed Matter 33, 503003 (2021).
  * Ni  _et al._ (2021) Z. Ni, K. Wang, Y. Zhang, O. Pozo, B. Xu, X. Han, K. Manna, J. Paglione, C. Felser, A. G. Grushin, _et al._ , Nature communications 12, 154 (2021).
  * Cook and Moore (2022) A. M. Cook and J. E. Moore, Communications Physics 5, 262 (2022), arXiv:2211.05317 [cond-mat.str-el] .
  * Pal  _et al._ (2023) A. Pal, J. H. Winter, and A. M. Cook, arXiv e-prints , arXiv:2301.02404 (2023), arXiv:2301.02404 [cond-mat.mes-hall] .

##  Appendix A Lattice Hamiltonians

For the MWSM parallel case,

$\begin{split}H_{\parallel}(\boldsymbol{k})=&\left[t_{01}\tau^{0}+\left(2+\gamma_{1}-\sum_{i}\cos
k_{i}\right)\tau^{z}+\sin k_{x}\tau^{x}+\sin k_{y}\tau^{y}\right]\\\
&\otimes\left[t_{02}\sin k_{z}\sigma^{0}-\left(2+\gamma_{2}-\sum_{i}\cos
k_{i}\right)\sigma^{z}-\sin k_{x}\sigma^{x}+\sin
k_{y}\sigma^{y}\right].\end{split}$ (15)

For the MWSM perpendicular case with inversion along $k_{z}$,

$\begin{split}H_{\perp}(\boldsymbol{k})=&\left[t_{01}\tau^{0}+\left(2+\gamma_{1}-\sum_{i}\cos
k_{i}\right)\tau^{z}+\sin k_{x}\tau^{x}+\sin k_{z}\tau^{y}\right]\\\
&\otimes\left[t_{02}\sin k_{z}\sigma^{0}-\left(2+\gamma_{2}-\sum_{i}\cos
k_{i}\right)\sigma^{z}-\sin k_{x}\sigma^{x}+\sin
k_{y}\sigma^{y}\right].\end{split}$ (16)

##  Appendix B Numerical evaluation of CPGE

We evaluate eqn. (1) using finite difference derivatives with $\delta
k_{d}=10^{-9}$, and gauge-fixed eigenstates that maximize the real part of the
overlap between eigenstates at neighboring $k$-points. We use a cubic grid
with spacing $\delta k_{i}=2\pi/N_{k}$ with $N_{k}=70$ for the integral. The
delta-function is replaced with a finite-width Gaussian of width $\delta
k_{g}=0.1$.