# New approach for normalization and photon-number distributions of photon-
added (-subtracted) squeezed thermal states

Li-Yun Hu1,2† and Zhi-Ming Zhang2‡ 1College of Physics & Communication
Electronics, Jiangxi Normal University, Nanchang 330022, China
2Key Laboratory of Photonic Information Technology of Guangdong Higher
Education Institutes,
SIPSE & LQIT, South China Normal University, Guangzhou 510006, China
${\dagger}$E-mail<EMAIL_ADDRESS>‡ E-mail<EMAIL_ADDRESS>
###### Abstract

Using the thermal field dynamics theory to convert the thermal state to a
“pure” state in doubled Fock space, it is found that the average value of
$e^{fa^{{\dagger}}a}$ under squeezed thermal state (STS) is just the
generating function of Legendre polynomials, a remarkable result. Based on
this point, the normalization and photon-number distributions of m-photon
added (or subtracted) STS are conviently obtained as the Legendre polynomials.
This new concise method can be expanded to the entangled case.

##  I Introduction

Nonclassicality of optical fields is helpful in understanding fundamentals of
quantum optics and have many applications in quantum information processing 1
. To generate and manipulate various nonclassical optical fields, subtracting
or adding photons from/to traditional quantum states or Gaussian states is
proposed 2 ; 2a ; 3 ; 4 ; 5 ; 6 ; 7 ; 8 . For example, the photon addition and
subtraction have been successfully demonstrated experimentally for probing
quantum commutation rules by Parigi et al. 6 . Recently, photon-added
(-subtracted) Gaussian states have received more attention from both
experimentalists and theoreticians 9 ; 10 ; 11 ; 11a ; 12 ; 13 ; 14 ; 15 ; 16
; 17 , since these states exhibit an abundant of nonclassical properties and
may give access to a complete engineering of quantum states and to fundamental
quantum phenomena.

Theoretically, the normalization factors of such quantum states are essential
for studying their nonclassical properties. Very recent, Fan and Jiang 18
present a new concise approach for normalizing m-photon-added (-subtracted)
squeezed vacuum state (pure state) by constructing generating function.
However, most systems are not isolated, but are immersed in a thermal
reservoir, thus it is often the case that we have no enough information to
specify completely the state of a system. In such a situation, the system only
can be described by mixed states, such as thermal states. In addition, the
squeezed thermal states (STSs) can be considered as the generalized Gaussian
states.

In this paper, we shall extend this case to the mixed state, i.e., by using
the thermal field dynamics (TFD) theory to convert the thermal state to a
“pure” state in doubled Fock space, we present a new concise method for
normalizing photon-added (-subtracted) squeezed thermal states (PASTSs,
PSSTSs) and for deriving their photon-number distributions (PNDs) which have
been a major topic of studies on quantum optics and quantum statistics. It is
found that the normalization factors and PNDs are related to the Legendre
polynomials in a compact form.

Our paper is arranged as follows. In section 2, based on Takahashi-Umezawa
TFD, we convert the thermal state $\rho_{th}$ to a pure state in doubled Fock
space by the partial trace,
$\rho_{th}=\mathtt{t\tilde{r}}\left[\left|0(\beta)\right\rangle\left\langle
0(\beta)\right|\right]$ (see Eq.(4) below). In section 3, we introduce the STS
$S_{1}\left|0(\beta)\right\rangle$ with $S_{1}$ being the single-mode
squeezing operator for the real mode. It is shown that the average value of
$e^{fa^{{\dagger}}a}$ under STS is just the generating function of Legendre
polynomials, a remarkable result. Based on this point, in sections 4 and 5,
the normalization factors and PNDs of m-photon added (or subtracted) STS are
obtained as the Legendre polynomials, respectively. The last section is
devoted to drawing a conclusion.

##  II Representation of thermal state in doubled Fock space

We begin with briefly reviewing the properties of thermal state. For a single
field mode with frequency $\omega$ in a thermal equilibrium state
corresponding to absolute temperature $T$, the density operator is

$\rho_{th}=\sum_{n=0}^{\infty}\frac{n_{c}^{n}}{\left(n_{c}+1\right)^{n+1}}\left|n\right\rangle\left\langle
n\right|,$ (1)

where $n_{c}=[\exp(\hbar\omega/(kT))-1]^{-1}$ being the average photon number
of the thermal state $\rho_{th}$ and $k$ being Bltzmann’s constant. Note
$\left|n\right\rangle=a^{\dagger n}/\sqrt{n!}\left|0\right\rangle$ and the
normally ordering form of vacuum projector $\left|0\right\rangle\left\langle
0\right|=\colon\exp(-a^{\dagger}a)\colon$(the symbol $\colon\colon$ denotes
the normal ordering), one can put Eq.(1) into the following form

$\rho_{th}=\colon\frac{1}{n_{c}+1}e^{-\frac{1}{n_{c}+1}a^{{\dagger}}a}\colon=\frac{1}{n_{c}+1}e^{a^{{\dagger}}a\ln\frac{n_{c}}{n_{c}+1}},$
(2)

where in the last step, the operator identity $\exp\left(\lambda
a^{{\dagger}}a\right)=\colon\exp\left[\left(e^{\lambda}-1\right)a^{{\dagger}}a\right]\colon$is
used.

Recalling the thermal field dynamics (TFD) introduced by Takahashi and Umezawa
19 ; 20 ; 21 , its elemental spirit is to convert the calculations of ensemble
averages for a mixed state $\rho$, $\left\langle
A\right\rangle=\mathtt{tr}\left(A\rho\right)/\mathtt{tr}\left(\rho\right),$ to
equivalent expectation values with a pure state $\left|0(\beta)\right\rangle$,
i.e.,

$\left\langle A\right\rangle=\left\langle
0(\beta)\right|A\left|0(\beta)\right\rangle,$ (3)

where $\beta=1/kT$, $k$ is the Boltzmann constant. Thus, for the density
operator $\rho_{th}$, by using the partial trace method 22 , i.e.,
$\rho_{th}=\widetilde{\mathtt{tr}}\left[\left|0(\beta)\right\rangle\left\langle
0(\beta)\right|\right]$ where $\widetilde{\mathtt{tr}}$ denotes the trace
operation over the environment freedom (denoted as operator
$\tilde{a}^{{\dagger}}$), one can obtain the explicit expression of
$\left|0(\beta)\right\rangle$ in doubled Fock space,

$\left|0(\beta)\right\rangle=\text{sech}\theta\exp\left[a^{\dagger}\tilde{a}^{\dagger}\tanh\theta\right]\left|0\tilde{0}\right\rangle=S\left(\theta\right)\left|0\tilde{0}\right\rangle,$
(4)

where $\left|0\tilde{0}\right\rangle$ is annihilated by $\tilde{a}$ and $a$,
$[\tilde{a},\tilde{a}^{{\dagger}}]=1$, and $S\left(\theta\right)$ is the
thermal operator,
$S\left(\theta\right)\equiv\exp\left[\theta\left(a^{\dagger}\tilde{a}^{\dagger}-a\tilde{a}\right)\right]\
$with a similar form to the a two-mode squeezing operator except for the tilde
mode, and $\theta$ is a parameter related to the temperature by
$\tanh\theta=\exp\left(-\frac{\hbar\omega}{2kT}\right)$.
$\left|0(\beta)\right\rangle$ is named thermal vacuum state.

Let $\mathtt{Tr}$ denote the trace operation over both the system freedom
(expressed by $\mathtt{tr}$) and the environment freedom by
$\widetilde{\mathtt{tr}}$, i.e.,
$\mathtt{Tr}=\mathtt{tr}\widetilde{\mathtt{tr}}$, then we have

$\displaystyle\mathtt{tr}\left(A\rho_{th}\right)$
$\displaystyle=\mathtt{Tr}\left[A\left|0(\beta)\right\rangle\left\langle
0(\beta)\right|\right]$
$\displaystyle=\mathtt{tr}\left[A\widetilde{\mathtt{tr}}\left|0(\beta)\right\rangle\left\langle
0(\beta)\right|\right],$ (5)

and the average photon number of the thermal state $\rho_{th}$ is

$n_{c}=\mathtt{Tr}\left[a^{\dagger}a\left|0(\beta)\right\rangle\left\langle
0(\beta)\right|\right]=\sinh^{2}\theta.$ (6)

Here we should emphasize that
$\widetilde{\mathtt{tr}}\left|0(\beta)\right\rangle\left\langle
0(\beta)\right|\neq\left\langle 0(\beta)\right|\left.0(\beta)\right\rangle,$
since $\left|0(\beta)\right\rangle$ involves both real mode $a$ and fictitious
mode $\tilde{a}$. From Eqs.(3) and (4) one can see that the worthwhile
convenience in Eq.(4) is at the expense of introducing a fictitious field (or
called a tilde-conjugate field) in the extended Hilbert space, i.e., the
original optical field state $\left|n\right\rangle$ in the Hilbert space
$\mathcal{H}$ is accompanied by a tilde state $\left|\tilde{n}\right\rangle$
in $\mathcal{\tilde{H}}$. A similar rule holds for operators: every Bose
annihilation operator $a$ acting on $\mathcal{H}$ has an image $\tilde{a}$
acting on $\mathcal{\tilde{H}}$. These operators in $\mathcal{H}$ are
commutative with those in $\mathcal{\tilde{H}}$.

##  III Suqeezed thermal vacuum state

To realize our purpose, we first introduce the squeezed thermal vacuum state,
defined as $S_{1}\left(r\right)\left|0(\beta)\right\rangle$, where
$S_{1}\left(r\right)=\exp[r(a^{2}-a^{\dagger 2})/2]$ is the single-mode
squeezing operator for the real mode with $r$ being squeezing parameter. Note
that Eq.(4) and the Baker-Hausdorf lemma

$S_{1}\left(r\right)a^{{\dagger}}S_{1}^{{\dagger}}\left(r\right)=a^{{\dagger}}\cosh
r+a\sinh r,$ (7)

then we get

$\displaystyle S_{1}\left(r\right)\left|0(\beta)\right\rangle$
$\displaystyle=S_{1}\left(r\right)\text{sech}\theta\exp\left[a^{\dagger}\tilde{a}^{\dagger}\tanh\theta\right]\left|0\tilde{0}\right\rangle$
$\displaystyle=\text{sech}\theta\text{sech}^{1/2}r\exp\left[\left(a^{{\dagger}}\cosh
r+a\sinh r\right)\tilde{a}^{\dagger}\tanh\theta\right]$
$\displaystyle\times\exp\left[-\frac{a^{\dagger 2}}{2}\tanh
r\right]\left|0\tilde{0}\right\rangle,$ (8)

where we have used
$S_{1}\left(\lambda\right)\left|0\right\rangle=$sech${}^{1/2}\lambda\exp[-a^{\dagger
2}/2\tanh\lambda]\left|0\right\rangle.$ Further, note $e^{\tau
a\tilde{a}^{\dagger}}a^{\dagger}e^{-\tau
a\tilde{a}^{\dagger}}=a^{\dagger}+\tau\tilde{a}^{\dagger},$ and for operators
$A,B$ satisfying the conditions $\left[A,[A,B]\right]=\left[B,[A,B]\right]=0,$
we have $e^{A+B}=e^{A}e^{B}e^{-[A,B]/2},$ thus Eq.(8) can be put into the
following form

$\displaystyle S_{1}\left(r\right)\left|0(\beta)\right\rangle$
$\displaystyle=\text{sech}\theta\text{sech}^{1/2}r\exp\left\\{\frac{\tanh\theta}{\cosh
r}a^{{\dagger}}\tilde{a}^{\dagger}\right.$ $\displaystyle+\left.\frac{\tanh
r}{2}\left(\tilde{a}^{\dagger 2}\tanh^{2}\theta-a^{\dagger
2}\right)\right\\}\left|0\tilde{0}\right\rangle.$ (9)

Next, we shall use Eq.(9) to derive the average of operator
$e^{fa^{\dagger}a}$ under the suqeezed thermal vacuum state
$S_{1}\left(r\right)\left|0(\beta)\right\rangle$, which is a bridge for our
whole calculations. Notice
$e^{f/2a^{\dagger}a}a^{\dagger}e^{-f/2a^{\dagger}a}=a^{\dagger}e^{f/2}$ and
$e^{-f/2a^{\dagger}a}ae^{f/2a^{\dagger}a}=ae^{f/2}$, so we have

$\displaystyle
e^{f/2a^{\dagger}a}S_{1}\left(r\right)\left|0(\beta)\right\rangle$
$\displaystyle=\text{sech}\theta\text{sech}^{1/2}r\exp\left\\{\frac{\tanh\theta}{\cosh
r}a^{{\dagger}}\tilde{a}^{\dagger}e^{f/2}\right.$
$\displaystyle+\left.\frac{\tanh r}{2}\left(\tilde{a}^{\dagger
2}\tanh^{2}\theta-a^{\dagger
2}e^{f}\right)\right\\}\left|0\tilde{0}\right\rangle,$ (10)

which leads to

$\displaystyle\left\langle e^{fa^{\dagger}a}\right\rangle$
$\displaystyle\equiv\left\langle
0(\beta)\right|S_{1}^{{\dagger}}\left(r\right)e^{fa^{\dagger}a}S_{1}\left(r\right)\left|0(\beta)\right\rangle$
$\displaystyle=\text{sech}^{2}\theta\text{sech}r\left\langle
0\tilde{0}\right|\exp\left\\{\frac{e^{f/2}\tanh\theta}{\cosh
r}a\tilde{a}\right.$ $\displaystyle+\left.\frac{\tanh
r}{2}\left(\tilde{a}^{2}\tanh^{2}\theta-a^{2}e^{f}\right)\right\\}$
$\displaystyle\times\exp\left\\{\frac{\tanh\theta}{\cosh
r}a^{{\dagger}}\tilde{a}^{\dagger}e^{f/2}\right.$
$\displaystyle+\left.\frac{\tanh r}{2}\left(\tilde{a}^{\dagger
2}\tanh^{2}\theta-a^{\dagger
2}e^{f}\right)\right\\}\left|0\tilde{0}\right\rangle$
$\displaystyle=\left[Ce^{2f}-2Be^{f}+A\right]^{-1/2},$ (11)

where we have set

$\displaystyle A$ $\displaystyle=\cosh^{4}\theta+\cosh 2\theta\sinh^{2}r$
$\displaystyle=n_{c}^{2}+\left(2n_{c}+1\right)\cosh^{2}r,$ $\displaystyle B$
$\displaystyle=\sinh^{2}\theta\cosh^{2}\allowbreak\theta=n_{c}\left(n_{c}+1\right),$
$\displaystyle C$ $\displaystyle=\cosh^{4}\theta-\cosh 2\theta\cosh^{2}r$
$\displaystyle=n_{c}^{2}-\left(2n_{c}+1\right)\sinh^{2}r,$ (12)

and we have used the completness relation of coherent state $\int
d^{2}zd^{2}\tilde{z}\left|z\tilde{z}\right\rangle\left\langle
z\tilde{z}\right|/\pi^{2}=1,$ here $\left|z\right\rangle$ and
$\left|\tilde{z}\right\rangle$ is the coherent state in real and fictitious
modes, respectively, and the formula 23

$\displaystyle\int\frac{d^{2}z}{\pi}\exp\left(\zeta\left|z\right|^{2}+\xi
z+\eta z^{\ast}+fz^{2}+gz^{\ast 2}\right)$
$\displaystyle=\frac{1}{\sqrt{\zeta^{2}-4fg}}\exp\left[\frac{-\zeta\xi\eta+\xi^{2}g+\eta^{2}f}{\zeta^{2}-4fg}\right],$
(13)

whose convergent condition is Re$\left(\zeta\pm f\pm g\right)<0,\
$Re($\zeta^{2}-4fg)/(\zeta\pm f\pm g)<0.$ Eq.(11) is very important for later
calculation of photon-number distribution (PND) and normalization of photon-
added (-subtracted) squeezed thermal states (PASTS, PSSTS).

It is interesting to notice that the standard generating function of Legendre
polynomials 25  is given by

$\frac{1}{\sqrt{1-2xt+t^{2}}}=\sum_{m=0}^{\infty}P_{m}\left(x\right)t^{m},$
(14)

thus comparing Eq.(11) with Eq.(14) we find

$\displaystyle\left\langle e^{fa^{\dagger}a}\right\rangle$
$\displaystyle=A^{-1/2}\left[\frac{C}{A}e^{2f}-2\frac{B}{A}e^{f}+1\right]^{-1/2}$
$\displaystyle=A^{-1/2}\sum_{m=0}^{\infty}P_{m}\left(B/\sqrt{AC}\right)\left(\sqrt{C/A}e^{f}\right)^{m},$
(15)

which indicates that the average value of $e^{fa^{{\dagger}}a}$ under squeezed
thermal state (STS) is just the generating function of Legendre polynomials, a
remarkable result. Next, we shall examine the normalizations and PNDs of PASTS
and PSSTS by using Eqs.(11) and (15).

##  IV Normalization and PND of PASTS

The $m$-photon-added scheme, denoted by the mapping $\rho\rightarrow
a^{{\dagger}m}\rho a^{m},$ was first proposed by Agarwal and Tara 2 . Here, we
introduce the PASTS. Theoretically, the PASTS can be obtained by repeatedly
operating the photon creation operator $a^{\dagger}$ on a STS, so its density
operator is given by

$\rho_{ad}=C_{a,m}^{-1}a^{{\dagger}m}S_{1}\rho_{th}S_{1}^{\dagger}a^{m},$ (16)

where $m$ is the added photon number (a non-negative integer), $C_{a,m}^{-1}$
is the normalization constant to be determined.

###  IV.1 Normalization of PASTS

To fully describe a quantum state, its normalization is usually necessary.
Next, we shall employ the fact (5) and (11), (15) to realize our aim.
According to the normalization condition tr$\rho_{ad}=1$ and the TFD, we have

$\displaystyle C_{a,m}$
$\displaystyle=\mathtt{tr}\left[a^{{\dagger}m}S_{1}\rho_{th}S_{1}^{\dagger}a^{m}\right]$
$\displaystyle=\mathtt{Tr}\left[a^{{\dagger}m}S_{1}\left|0(\beta)\right\rangle\left\langle
0(\beta)\right|S_{1}^{\dagger}a^{m}\right]$ $\displaystyle=\left\langle
0(\beta)\right|S_{1}^{\dagger}a^{m}a^{{\dagger}m}S_{1}\left|0(\beta)\right\rangle,$
(17)

which implies that the calculation of normaliation factor $C_{a,m}$ is
converted to a matrix element after introducing the thermal vacuum state
$\left|0(\beta)\right\rangle$.

Note the operator identity 24  $e^{\tau
a^{{\dagger}}a}=e^{-\tau}\vdots\exp[(1-e^{-\tau})a^{{\dagger}}a]\vdots$, we
see

$\sum_{m=0}^{\infty}\frac{\tau^{m}}{m!}a^{m}a^{{\dagger}m}=\vdots e^{\tau
a^{{\dagger}}a}\vdots=\left(\frac{1}{1-\tau}\right)^{a^{{\dagger}}a+1},$ (18)

where the symbol $\vdots$ $\vdots$ denotes antinormally ordering. Thus using
Eqs.(11), (17) and (18), ($e^{f}\rightarrow\frac{1}{1-\tau}$) we have

$\displaystyle\sum_{m=0}^{\infty}\frac{\tau^{m}}{m!}C_{a,m}$
$\displaystyle=\frac{1}{1-\tau}\left\langle
0(\beta)\right|S_{1}^{\dagger}e^{a^{{\dagger}}a\ln\frac{1}{1-\tau}}S_{1}\left|0(\beta)\right\rangle$
$\displaystyle=\left[A\tau^{2}-2D\tau+1\right]^{-1/2},$ (19)

where we have set

$\displaystyle D$ $\displaystyle=\cosh^{2}\theta\cosh 2r-\sinh^{2}r$
$\displaystyle=n_{c}\cosh 2r+\cosh^{2}r.$ (20)

Comparing Eq.(19) with Eq.(14), and taking
$\tau^{\prime}\rightarrow\sqrt{A}\tau$, we obtain

$\displaystyle\sum_{m=0}^{\infty}\tau^{\prime m}\frac{C_{a,m}}{m!A^{m/2}}$
$\displaystyle=\left[\tau^{\prime 2}-2D/\sqrt{A}\tau^{\prime}+1\right]^{-1/2}$
$\displaystyle=\sum_{m=0}^{\infty}P_{m}\left(D/\sqrt{A}\right)\tau^{\prime
m},$ (21)

thus the normalization constant of PASTS is given by

$C_{a,m}=m!A^{m/2}P_{m}\left(D/\sqrt{A}\right),$ (22)

which is identical with the result in Ref.26 . It is noted that, for the case
of no-photon-addition with $m=0$, $C_{a,0}=1$ as expected. Under the case of
$m$-photon-addition thermal state (no squeezing) with $D=\allowbreak n_{c}+1$,
$A=\allowbreak\left(n_{c}+1\right)^{2},$ and $P_{m}\left(1\right)=1$, then
$C_{a,m}=m!\left(n_{c}+1\right)^{m}.$ The same result as Eq.(32) found in
Ref.27 . In addition, when $r=0$ corresponding to photon-added thermal state,
Eq.(22) just reduces to $C_{a,m}=m!\cosh^{2m}\theta$ 27 .

###  IV.2 PND of PASTS

The photon-number distribution (PND) is a key characteristic of every optical
field. The PND, i.e., the probability of finding $n$ photons in a quantum
state described by the density operator $\rho$, is
$\mathcal{P}(n)=\mathtt{tr}\left[\left|n\right\rangle\left\langle
n\right|\rho\right]$. In a similar spirit of deriving Eq.(22), noting
$a^{m}\left|n\right\rangle=\sqrt{n!/(n-m)!}\left|n-m\right\rangle$ and
$\left|n\right\rangle=a^{\dagger n}/\sqrt{n!}\left|0\right\rangle,$ the PND of
the PASTS can be calculated as

$\displaystyle\mathcal{P}_{a}(n)$
$\displaystyle=C_{a,m}^{-1}\mathtt{tr}\left[\left|n\right\rangle\left\langle
n\right|a^{{\dagger}m}S_{1}\rho_{th}S_{1}^{\dagger}a^{m}\right]$
$\displaystyle=C_{a,m}^{-1}\mathtt{Tr}\left[\left|n\right\rangle\left\langle
n\right|a^{{\dagger}m}S_{1}\left|0(\beta)\right\rangle\left\langle
0(\beta)\right|S_{1}^{\dagger}a^{m}\right]$
$\displaystyle=\frac{n!C_{a,m}^{-1}}{l!}\left\langle
0(\beta)\right|S_{1}^{\dagger}\left|l\right\rangle\left\langle
l\right|S_{1}\left|0(\beta)\right\rangle$
$\displaystyle=\frac{n!C_{a,m}^{-1}}{\left(l!\right)^{2}}\left\langle
0(\beta)\right|S_{1}^{\dagger}a^{\dagger l}\left|0\right\rangle\left\langle
0\right|a^{l}S_{1}\left|0(\beta)\right\rangle$ (23)

which leads to

$\displaystyle\sum_{l=0}^{\infty}\tau^{l}\frac{l!}{n!}C_{a,m}\mathcal{P}_{a}(n)$
$\displaystyle=\sum_{l=0}^{\infty}\frac{\tau^{l}}{l!}\left\langle
0(\beta)\right|S_{1}^{\dagger}\colon\left(a^{\dagger}a\right)^{l}e^{-a^{\dagger}a}\colon
S_{1}\left|0(\beta)\right\rangle$ $\displaystyle=\left\langle
0(\beta)\right|S_{1}^{\dagger}e^{a^{\dagger}a\ln\tau}S_{1}\left|0(\beta)\right\rangle,$
(24)

where $l=n-m$ and the vacuum projector operator
$\left|0\right\rangle\left\langle 0\right|=\colon e^{-a^{\dagger}a}\colon$ and
the operator identity $e^{\lambda
a^{\dagger}a}=\colon\exp[(e^{\lambda}-1)a^{\dagger}a]\colon$ are used.

Using Eq.(11) again ($e^{f}\rightarrow\tau$) and comparing Eq.(24) with Eq.
(14) we see

$\displaystyle\sum_{l=0}^{\infty}\tau^{l}\frac{l!}{n!}C_{a,m}\mathcal{P}_{a}(n)$
$\displaystyle=A^{-1/2}\left[\frac{C}{A}\tau^{2}-2\frac{B}{A}\tau+1\right]^{-1/2}$
$\displaystyle=A^{-1/2}\sum_{l=0}^{\infty}P_{l}\left(B/\sqrt{AC}\right)\left(\sqrt{C/A}\tau\right)^{l},$
(25)

which leads to the PND of PASTS

$\mathcal{P}_{a}(n)=\frac{n!C_{a,m}^{-1}\left(C/A\right)^{\left(n-m\right)/2}}{\left(n-m\right)!\sqrt{A}}P_{n-m}\left(B/\sqrt{AC}\right),$
(26)

a Legendre polynomial with a condition $n\geqslant m$ which implies that the
photon-number ($n$) involved in PASTS is always no-less than the photon-number
($m$) operated on the STS, and there is no photon distribution when $n<m$). It
is obvious that when $m=0$ corresponding to the STS, then the PND of STS is
also a Legendre distribution 28 .

##  V Normalization and PND of PSSTS

Next, we turn our attention to discussing the photon-subtracted squeezed
thermal state (PSSTS), defined as

$\rho_{sb}=C_{s,m}^{-1}a^{m}S_{1}\rho_{th}S_{1}^{\dagger}a^{{\dagger}m},$ (27)

where $m$ is the subtracted photon number (a non-negative integer), and
$C_{s,m}$ is a normalized constant.

In a similar way to deriving Eq.(22), we have

$C_{s,m}=\left\langle
0(\beta)\right|S_{1}^{\dagger}a^{{\dagger}m}a^{m}S_{1}\left|0(\beta)\right\rangle,$
(28)

so employing $e^{\lambda
a^{\dagger}a}=\colon\exp[(e^{\lambda}-1)a^{\dagger}a]\colon$ and Eq.(11)
($e^{f}\rightarrow 1+\tau$) we see

$\displaystyle\sum_{m=0}^{\infty}\frac{\tau^{m}}{m!}C_{s,m}$
$\displaystyle=\left\langle
0(\beta)\right|S_{1}^{\dagger}e^{a^{{\dagger}}a\ln(1+\tau)}S_{1}\left|0(\beta)\right\rangle$
$\displaystyle=\left[C\tau^{2}-2E\tau+1\right]^{-1/2},$ (29)

where

$\displaystyle E$ $\displaystyle=\cosh 2r\cosh^{2}\theta-\cosh^{2}r$
$\displaystyle=\frac{1}{2}\left[(2n_{c}+1)\cosh 2r-1\right].$ (30)

Comparing Eq.(29) with Eq.(14) yields

$C_{s,m}=m!C^{m/2}P_{m}\left(E/\sqrt{C}\right),$ (31)

which is the normalization factor of PSSTS. When $r=0$ corresponding to
photon-subtracted thermal state, Eq.(31) just reduces to
$C_{s,m}=m!\sinh^{2m}\theta$ 27 .

Using the same procession as obtaining Eq.(26), the PND of PSSTS is given by

$\displaystyle\mathcal{P}_{s}(n)$ $\displaystyle=C_{s,m}^{-1}\left\langle
0(\beta)\right|S_{1}^{\dagger}a^{{\dagger}m}\left|n\right\rangle\left\langle
n\right|a^{m}S_{1}\left|0(\beta)\right\rangle$
$\displaystyle=\frac{1}{n!}C_{s,m}^{-1}\left\langle
0(\beta)\right|S_{1}^{\dagger}\colon
a^{{\dagger}m+n}a^{m+n}e^{-a^{{\dagger}}a}\colon
S_{1}\left|0(\beta)\right\rangle,$ (32)

so $\left(k=m+n\right)$

$\displaystyle\sum_{k=0}^{\infty}\frac{\tau^{k}}{k!}n!C_{s,m}\mathcal{P}_{s}(n)$
$\displaystyle=\left\langle
0(\beta)\right|S_{1}^{\dagger}e^{a^{{\dagger}}a\ln\tau}S_{1}\left|0(\beta)\right\rangle$
$\displaystyle=\text{R.H.S of (\ref{1.21}),}$ (33)

whcih leads to the PND of PSSTS

$\mathcal{P}_{s}(n)=\frac{\left(m+n\right)!}{n!C_{s,m}\sqrt{A}}\left(C/A\right)^{m+n/2}P_{m+n}\left(B/\sqrt{AC}\right),$
(34)

a Legendre polynomial, which is same as the result of Ref.28 .

##  VI Conclusion

In this paper, we present a new concise approach for normalizing m-photon-
added (-subtracted) STS and for deriving the PNDs, which improve the method
used in Refs. 26 ; 28 . That is to say, using the thermal field dynamics
theory, we convert the thermal state to a pure state in doubled Fock space in
which the calculations of ensemble averages under a mixed state $\rho$,
$\left\langle
A\right\rangle=\mathtt{tr}\left(A\rho\right)/\mathtt{tr}\left(\rho\right)\ $is
replaced by an equivalent expectation values with a pure state
$\left|0(\beta)\right\rangle$, i.e., $\left\langle A\right\rangle=\left\langle
0(\beta)\right|A\left|0(\beta)\right\rangle$. It is shown that the average
value of $e^{fa^{{\dagger}}a}$ under STS is just the generating function of
Legendre polynomials, a remarkable result. Based on this point, the
normalization and PNDs of m-photon added (or subtracted) STS are easily
obtained as the Legendre polynomials. The generating function of the Legendre
polynomials and the average value of $e^{fa^{{\dagger}}a}$ under STS are used
in the whole calculation.

ACKNOWLEDGEMENTS: Work supported by the National Natural Science Foundation of
China (Grant Nos. 11047133, 60978009), the Major Research Plan of the National
Natural Science Foundation of China (Grant No. 91121023), and the “973”
Project (Grant No. 2011CBA00200), as well as the Natural Science Foundation of
Jiangxi Province of China (No. 2010GQW0027).

## References

  * (1) D. Bouwmeester, A. Ekert and A. Zeilinger, The Physics of Quantum Information (Springer-Verlag, 2000).
  * (2) G. S. Agarwal and K. Tara, Phys. Rev. A 43, 492 (1991); Phys. Rev. A 46, 485 (1992).
  * (3) Z. X. Zhang and H. Y. Fan, Phys. Lett. A 165, 14 (1992).
  * (4) A. Zavatta, S. Viciani, and M. Bellini, Science, 306, 660 (2004).
  * (5) A. Zavatta, V. Parigi, and M. Bellini, Phys. Rev. A 75, 052106 (2007).
  * (6) A. Zavatta, S. Viciani, and M. Bellini, Phys. Rev. A 72, 023820 (2005).
  * (7) V. Parigi, A. Zavatta, M. S. Kim, and M. Bellini, Science, 317, 1890 (2007).
  * (8) A. Ourjoumtsev, R. Tualle-Brouri, J. Laurat, Ph. Grangier, Science 312, 83 (2006).
  * (9) A. Ourjoumtsev, A. Dantan, R. Tualle-Brouri, and Ph.Grangier, Phys. Rev. Lett. 98, 030502 (2007).
  * (10) P. Marek, H. Jeong and M. S. Kim, Phys. Rev. A 78, 063811 (2008).
  * (11) S. L. Zhang and P. van Loock, Phys. Rev. A 82, 06216 (2010).
  * (12) Li-yun Hu and Hong-yi Fan, J. Opt. Soc. Am. B, 25, 1955 (2008).
  * (13) Li-yun Hu and Hong-yi Fan, Chin. Phys. B 18, 4657 (2009).
  * (14) T. Opatrný, G. Kurizki and D-G. Welsch, Phys. Rev. A 61, 032302 (2000).
  * (15) S. Olivares and M. G. A. Paris, J. Opt. B: Quantum Semiclassical Opt. 7, S392 (2005).
  * (16) A. Kitagawa,M. Takeoka,M. Sasaki, and A. Chefles, Phys. Rev. A 73, 042310 (2006).
  * (17) Y. Yang and F. L. Li, J. Opt. Soc. Am. B, 26, 830 (2009).
  * (18) A. Biswas and G. S. Agarwal, Phys. Rev. A 75, 032104 (2007).
  * (19) S. Y. Lee and H. Nha, Phys. Rev. A 82, 053812 (2010).
  * (20) H. Y. Fan and N. Q. Jiang, Chin. Phys. Lett. 27, 044206 (2010).
  * (21) Y. Takahashi and H. Umezawa, Collective Phenomena 2, 55 (1975).
  * (22) Memorial Issue for H. Umezawa, Int. J. Mod. Phys. B 10, 1695 (1996) memorial issue and references therein.
  * (23) H. Umezawa, Advanced Field Theory—–Micro; Macro; and Thermal Physics (AIP, 1993).
  * (24) Li-yun Hu and Hong-yi Fan, Chin. Phys. Lett. 26, 090307 (2009).
  * (25) R. R. Puri, Mathematical Methods of Quantum Optics (Springer-Verlag, Berlin/Heidelberg/New York, 2001), p. 269 (A.29).
  * (26) George B. Arfken, Hans J. Weber, Mathematical Methods for Physicists, Elsevier Academic Press, p. 743, (2005).
  * (27) Hongyi Fan, Representation and Transformation Theory in Quantum Mechanics—–Progress of Dirac’s symbolic method (Shanghai Scientific and technical press, Shanghai, 1997).
  * (28) Li-yun Hu and Z. M. Zhang, [quant-ph] arXiv: 1110.6587.
  * (29) Li-yun Hu and Hong-yi Fan, Mod. Phys. Lett. A 24, 2263 (2009).
  * (30) Li-yun Hu, Xue-xiang Xu, Zi-sheng Wang, and Xue-fen Xu, Phys. Rev. A 82, 043842 (2010).


# DeepIC: Coding for Interference Channels via Deep Learning

Karl Chahine  Electrical and Computer Engineering
University of Texas at Austin
<EMAIL_ADDRESS>Nanyang Ye  Computer Science
Shanghai Jiao Tong University
<EMAIL_ADDRESS>Hyeji Kim  Electrical and Computer Engineering
University of Texas at Austin
<EMAIL_ADDRESS>
###### Abstract

The two-user interference channel is a model for multi one-to-one
communications, where two transmitters wish to communicate with their
corresponding receivers via a shared wireless medium. Two most common and
simple coding schemes are time division (TD) and treating interference as
noise (TIN). Interestingly, it is shown that there exists an asymptotic
scheme, called Han-Kobayashi scheme, that performs better than TD and TIN.
However, Han-Kobayashi scheme has impractically high complexity and is
designed for asymptotic settings, which leads to a gap between information
theory and practice.

In this paper, we focus on designing practical codes for interference
channels. As it is challenging to analytically design practical codes with
feasible complexity, we apply deep learning to learn codes for interference
channels. We demonstrate that DeepIC, a convolutional neural network-based
code with an iterative decoder, outperforms TD and TIN by a significant margin
for two-user additive white Gaussian noise channels with moderate amount of
interference.

###### Index Terms:

Interference channels, deep learning, autoencoder, convolutional neural
network, iterative decoding

##  I Introduction

The two-user interference channel is a fundamental building block of wireless
networks, where two pairs of transmitters and receivers share the
communication medium as depicted in Figure 1. We assume that the $i$-th sender
wishes to communicate a bit sequence $\mathbf{b}_{i}\in[0,1]^{K_{i}}$ to the
$i$-th receiver, for $i=1,2$. The $i$-th sender maps the message
$\mathbf{b}_{i}\in[0,1]^{K_{i}}$ to a length-$n$ sequence
$\mathbf{C}_{i}\in\mathbb{R}^{n}$ for $i=1,2$. The capacity region of the two-
user interference channel is defined as the set of achievable rate pairs
$(R_{1},R_{2}):=(K_{1}/n,K_{2}/n)$ such that the probability of error
$P((\mathbf{b}_{1},\mathbf{b}_{2})\neq(\hat{\mathbf{b}}_{1},\hat{\mathbf{b}}_{2}))\rightarrow
0$ as $(K_{1},K_{2},n)\to\infty$ for some encoders and decoders. The capacity
region is not known in general and there are inner and outer bounds that
coincide for several special cases [1, 2, 3], when the interference is very
weak or strong.

Moreover, practical coding schemes for finite $(K_{1},K_{2},n)$ are vastly
missing. Two commonly used schemes are time-division (TD) and treating-
interference-as-noise (TIN), both of which are an application of codes
designed for point-to-point channels. TD orthogonalizes the transmission of
two users and is commonly used in practice. TIN treats the interference as
noise and is known to be optimal for weak interference channels. These two are
extreme schemes, and a natural question is ‘can we design a code that is more
reliable than TD and TIN for channels with moderate interference?’ It is
challenging to analytically design practical coding schemes with feasible
complexity that minimize the probability of errors, however. Hence, in our
approach, we apply deep learning to _learn_ reliable codes for interference
channels.

Figure 1: Two-user Interference Channels

Over the past few years, deep learning based approach for designing channel
codes and decoders has demonstrated tremendous success for various
communication channels [4, 5, 6, 7, 8]. Several work also explored applying
deep learning to design modulation for interference channels [4, 9, 10]. [4]
demonstrates that feedforward networks can outperform uncoded QAM with time-
sharing for very short blocklengths (eg: $4$ bits). [9] proposes an adaptive
deep learning algorithm for multi-user symmetric interference channels and
show that their algorithm outperforms the conventional system using $PSK$ or
$QAM$. [10] shows that for interference channels with more than two users,
interference alignment schemes can be improved by deep learning.

Despite the differences, all of the existing work focuses on learning
modulation schemes with a very short blocklength. Hence, a very important
question remains unanswered. Can we apply deep learning to invent new coding
schemes for interference channels that have some memory and work well for
longer blocklengths at the order of 10s and 100s? There are several challenges
in learning a code with reasonably long blocklengths for interference channels
(and any channel in general). First, the number of messages increases
exponentially in blocklengths, which implies that we cannot train a code with
all possible messages (for blocklength 100, there are $2^{100}$ possible
messages). Naturally, strong generalization becomes necessary. Second,
learning a code is a non-convex problem, and there tends to be several local
optima, requiring a carefully devised optimization strategy. Last, for
interference channels, we have to learn a _pair_ of codes, which makes the
learning problem more challenging.

In this paper, we demonstrate DeepIC, a first neural network based code
designed for reasonably long blocklengths (eg: $K=100$) that outperforms the
state-of-the-art by a large margin. We show that carefully designed neural
architectures and training methodology allow us to address the challenges
associated with learning codes and achieve a high reliability of DeepIC. Our
main contributions are as follows.

  * •

We introduce DeepIC, which consists of two Convolutional Neural Network
(CNN)-based encoders and two CNN-based iterative decoders, designed for two-
user interference channels, and the training methodology for DeepIC.

  * •

We show that DeepIC achieves a significant reliability improvement for one
hundred coding bits ($K=100$) over the baselines such as TD and TIN, for AWGN
channels with moderate interference. DeepIC also outperforms the learning-
based methods that directly generalize the state-of-the-art schemes in [4].

  * •

Through the interpretation analysis, we empirically show that DeepIC has a
longer memory (up to around 10) than the direct application of well-known
neural architectures.

The rest of the paper is organized as follows. In Section II, we introduce the
problem statement and present a preview of the results. In Section III, we
provide the details of DeepIC including its neural architecture and training.
In Section IV, we present experiment results for evaluating the performance of
DeepIC as well as for interpreting DeepIC.

##  II Problem statement

We focus on symmetric AWGN interference channels, i.e.,
$Y_{1}=C_{1}+hC_{2}+Z_{1}$ and $Y_{2}=hC_{1}+C_{2}+Z_{2}$, where
$Z_{1},Z_{2}\sim\mathcal{N}(0,\sigma^{2})$ as depicted in Figure 2. For
concreteness, we focus on the rate-1/3 setting with $K=100$ and $n=300$ for
both users. We consider various SNRs, defined as SNR
$=-10\log_{10}(\sigma^{2})$, and various degree of interference
$h=0.3,0.6,0.8$. Our goal is to design reliable codes that minimize the bit
error rate (BER), defined as BER $=\frac{1}{K}\sum_{i=1}^{K}P(\hat{b}_{k}\neq
b_{k})$.

Figure 2: We focus on learning Rate-1/3 reliable codes for 2-user symmetric
AWGN interference channels.

###  II-A Preview of Results

A natural strategy to create a code for interference channels is to utilize
the well-known neural networks, such as feedforward networks (FF), CNNs, and
Recurrent Neural Networks (RNN). However, in our experiments, we observe that
this strategy by itself is not sufficient to achieve a noticeable improvement
over the the two baseline schemes, time-division (TD) and treating-
interference-as-noise (TIN), as shown in Figure 3. DeepIC, our CNN-based
neural codes with an iterative CNN decoding and well-thought-out training,
significantly outperforms all the baselines.

Figure 3: BER (averaged over two users) vs. SNR for h=0.8 and blocklength
$K=100$

##  III Architecture and Training of DeepIC

In this section, we present the _architecture_ and the _training methodology_
of DeepIC for two-user Gaussian interference channels, which are crucial in
achieving the improved reliability.

###  III-A Architecture

We construct an autoencoder network for the two-user interference channel by
concatenating two encoder-decoder pairs; we replace the encoders and decoders
in Figure 1 by neural networks.

Encoder architecture: Each encoder consists of three learnable blocks of 1-D
CNNs placed in parallel, followed by a power normalizing layer. As shown in
Figure 4, each message sequence b is encoded into a concatenation of three
sequences: $\bm{x_{1}}=g_{\theta_{1}}($b$)$, $\bm{x_{2}}=g_{\theta_{2}}($b$)$,
$\bm{x_{3}}=g_{\theta_{3}}(\textbf{b})$, where $g_{\theta_{l}}($b$)$ is the
function representing the block CNNl parametrized by the weights $\theta_{l}$,
$l\in\\{1,2,3\\}$. While two encoders share the same encoder architecture,
they do not share the weights.

Figure 4: DeepIC Encoder Architecture Figure 5: DeepIC Iterative Decoder
Architecture

The power normalizing block $d(.)$ is implemented by estimating the first and
second moments of each batch of size $J$. More specifically, for training
example $j$, $\bm{b}^{j}$ is encoded into $\bm{x}^{j}$, where $\bm{x}^{j}$ is
the concatenation of $\bm{x}_{1}^{j}$, $\bm{x}_{2}^{j}$ and $\bm{x}_{3}^{j}$.
Each power normalizer then performs the operation
$\bm{c}_{i}^{j}=d(\bm{x}_{i}^{j})$. This is equivalent to calculating
$\bm{c}_{i}^{j}=\frac{\bm{x}_{i}^{j}-\bm{\mu_{i}}}{\bm{\sigma_{i}}}$, where
$\bm{\mu_{i}}=\frac{1}{J}\sum_{j=1}^{J}\bm{x}_{i}^{j}$ and
$\bm{\sigma_{i}}=\sqrt{\frac{1}{J}\sum_{j=1}^{J}(\bm{x}_{i}^{j}-\bm{\mu_{i}})^{2}}$
are the batch mean and standard deviation, respectively.

Decoder architecture: Inspired by the dynamic programming decoder, we let the
decoder to update the belief in an iterative manner, as depicted in Figure 5.
Let $\bm{y}_{1}$, $\bm{y}_{2}$ and $\bm{y}_{3}$ be the noisy versions of
$\bm{c}_{1}$, $\bm{c}_{2}$ and $\bm{c}_{3}$ respectively. The decoder goes
through multiple iterations. Each iteration makes use of two sequential blocks
of 1-D CNN layers, $f_{\phi_{1}}$ and $f_{\phi_{2}}$, where $f_{\phi_{l}}$ is
the function representing the CNN block parametrized by the weights
$\phi_{l}$, $l\in\\{1,2\\}$. As shown in Figure 5, the first decoder
$f_{\phi_{1}}$ takes as input the noisy signals $\bm{y}_{1}$ and $\bm{y}_{2}$,
as well as a prior $\bm{p}$ of shape (K,F), where F is the information feature
size (We let $F=5$). The output of the first decoder is the posterior
$\bm{q}_{1}$ with same shape (K,F). The second decoder $f_{\phi_{2}}$ takes as
input the signals $\bm{y}_{1}$, $\bm{y}_{3}$ and the posterior $\bm{q}_{1}$ to
produce the second posterior $\bm{q}_{2}$. The first iteration takes 0 as a
prior. At the last iteration, the output posterior’s shape is (K,1), and is
passed through a sigmoid function to estimate to message bits:
$\hat{\bm{b}}=sigmoid(\bm{q}_{2})$.

###  III-B Training Methodology

The training algorithm used for DeepIC is shown in Algorithm 1. The main
differences from conventional training are as follows:

  * •

At every epoch, the users’ encoders and decoders are trained separately. The
number of iterations for the encoders and decoders are $T_{enc}$ and $T_{dec}$
respectively. This prevents the training from converging to a local optimum
[11].

  * •

Empirically, our results showed that using different training noise levels for
the encoders ($\sigma_{enc}$) and decoders ($\sigma_{dec}$) resulted in better
performance.

  * •

The used batch size was very large (500). This is important to average out the
channel noise effects.

  * •

Let $L_{1}$ and $L_{2}$ be the binary cross-entropy (BCE) losses of user 1 and
user 2 respectively. More specifically, let $\bm{b_{1}}$, $\bm{z_{1}}$ be the
message sequence and noise vector of user 1 and $\bm{b_{2}}$, $\bm{z_{2}}$ the
message sequence and noise vector of user 2. We compute
$L_{1}=BCE(\bm{b_{1}},f_{1}(g_{1}(\bm{b_{1}})+h*g_{2}(\bm{b_{2}})+\bm{z_{1}}))$
and
$L_{2}=BCE(\bm{b_{2}},f_{2}(g_{2}(\bm{b_{2}})+h*g_{1}(\bm{b_{1}})+\bm{z_{2}}))$,
where $g_{1}$, $f_{1}$ are the encoder and decoder of user 1, and $g_{2}$,
$f_{2}$ are the encoder and decoder of user 2. In our simulations, we found
that evaluating a weighted sum of the losses $L=\alpha*L_{1}+\beta*L{2}$
results in a fair and optimal training for the two users. Note that
$\alpha+\beta=1$, their initial value is 0.5, and they are continuously
updated throughout the training. The proposed method is similar as the one
described in [4], and is shown in Algorithm 1.

Inputs: Number of Epochs E, Batch Size J, Encoder Training Steps $T_{enc}$,
Decoder Training Steps $T_{dec}$, Encoder Training noise $\sigma_{enc}$,
Decoder Training noise $\sigma_{dec}$, $\alpha=\beta=0.5$ (initial values)

for  _$i\leq E$_ do

for  _$k\leq T_{enc}$_ do

Generate examples $\bm{b_{1}}$, noise $\bm{z_{1}}$ $\sim
N(0,\sigma_{enc}^{2})$

Generate examples $\bm{b_{2}}$, noise $\bm{z_{2}}$ $\sim
N(0,\sigma_{enc}^{2})$

Compute $L_{1}$, $L_{2}$ using $\bm{b_{1}}$, $\bm{b_{2}}$, $\bm{z_{1}}$,
$\bm{z_{2}}$

Compute $L=\alpha*L_{1}+\beta*L_{2}$

Train the encoders $g_{\theta}$ through back-propagation

Update $\alpha=\frac{L_{1}}{L{1}+L_{2}}$

Update $\beta=\frac{L_{2}}{L{1}+L_{2}}$

end for

for  _$k\leq T_{dec}$_ do

Generate examples $\bm{b_{1}}$, noise $\bm{z_{1}}$ $\sim
N(0,\sigma_{dec}^{2})$

Generate examples $\bm{b_{2}}$, noise $\bm{z_{2}}$ $\sim
N(0,\sigma_{dec}^{2})$

Compute $L_{1}$, $L_{2}$ using $\bm{b_{1}}$, $\bm{b_{2}}$, $\bm{z_{1}}$,
$\bm{z_{2}}$

Compute $L=\alpha*L_{1}+\beta*L_{2}$

Train the decoders $f_{\phi}$ through back-propagation

Update $\alpha=\frac{L_{1}}{L{1}+L_{2}}$

Update $\beta=\frac{L_{2}}{L{1}+L_{2}}$

end for

end for

Algorithm 1 DeepIC pseudo-code

Some critical hyperparameters used in our simulations are listed in Table 1.

Parameter | Assignment
---|---
h | $\\{0.3,0.6,0.8\\}$
Encoders | 2 layers 1-D CNN
Decoders | 5 layers 1-D CNN
Info Feature Size F | 5
Encoder Training Steps $T_{enc}$ | 100
Decoder Training Steps $T_{dec}$ | 500
Decoder Iterations | 6
Optimizer | Adam with initial learning rate 0.0001
Loss | Binary Cross-Entropy
Number of Epochs E | 100
TABLE I: Hyperparameters of DeepIC

##  IV Experiment Results

In this section, we present a set of experiment results and provide intuition
on DeepIC ’s performance. Throughout this section, BER refers to the average
BER of users 1 and 2.

###  IV-A Performance of DeepIC

To measure performance, we plot the BER of DeepIC as a function of SNR for
$h=0.8$ in Figure 6. As we mentioned in Section II (A), DeepIC shows a clearly
better performance when compared to TD and TIN, both of which were implemented
using classic Turbo codes. TIN uses a rate-1/3 Turbo code with BPSK, and TD
uses a rate-1/3 Turbo code with 4-PAM as in [4], [9]. In addition, we compared
DeepIC ’s performance with a CNN autoencoder (CNN-AE), a RNN autoencoder (RNN-
AE), FeedForward with one-hot encoding (One-hot FF), and a FeedForward
autoencoder (FF-AE). DeepIC ’s superior performance is coupled to its (a)
complex architecture, and (b) personalized training, inducing memory-dependent
codewords. This aspect is discussed in the next sub-section.

Figure 6: BER performance for h=0.8 and Block Length=100

###  IV-B Block Length Coding Gain

It is well known in coding theory that as the block length of codewords
increases, channel coding can achieve improved reliability [12]. To study if
DeepIC exhibits such behavior, we compare the performances of DeepIC and RNN-
AE when trained and tested with different training block lengths. In Figure 8,
we plot the BER vs. SNR with train/test blocklengths varying from 30, 60, to
100 for (a) RNN-AE and (b) DeepIC at $h=0.8$. Surprisingly, we observe that as
the block length increases, RNN-AE shows no improvement in BER, while DeepIC’s
BER decreases with a larger block length. We conclude that naively applying
general purpose neural networks such as RNN to channel interference problems
does not result in a better performance as we increase the block length. The
encoders and decoders of RNN-AE share the same architecture, shown in Figure
7.

Figure 7: RNN-AE Encoders and Decoders architecture

Figure 8: BER as a function of SNR evaluated at block lengths (BL): 30, 60,
100. (a) RNN-AE and (b) DeepIC. The reliability improves as the
training/testing block length increases for DeepIC but not for RNN-AE.

###  IV-C Degree of Interference

In Figure 6, we considered $h=0.8$, which is a moderate interference regime.
The strengths of the signal and the interference are similar, which makes
coding for interference channels challenging. In this section, we vary $h$; we
focus on smaller $h$’s to study if we can learn reliable codes for weak
interference regimes. In particular, we consider $h=0.3$ and $h=0.6$, and the
results are shown in Figure 9.

Figure 9: BER performance of Turbo Codes, RNN-AE and DeepIC for (a) h=0.3 (b)
h=0.6. As the level of interference $h$ increases, DeepIC starts to outperform
TIN.

While the superior performance of DeepIC is evident in the moderate
interference regime ($h=0.8$), as the value of h decreases, DeepIC’s
performance gets weaker when compared to classic Turbo codes with treating
interference as noise (TIN). In such cases, training the DeepIC to ignore the
interference and mimic Turbo codes would be optimal. We conjecture that DeepIC
is not learning to mimic Turbo codes as DeepIC does not include an
interleaver, which is the key component of Turbo codes. We will explore this
intriguing problem in our future work. We discuss the role of interleaver in
the following.

###  IV-D Interleaved Structure of DeepIC

As part of our experiments, we explore introducing an interleaver to both the
encoding and decoding processes of DeepIC, which allows us to mimic Turbo
autoencoders introduced in [13]. At the encoder side, the input to CNN3 of
Figure 4 would be shuffled by an interleaver $\pi$. Formally,
$\bm{x_{3}}=\pi(\bm{b})$. At the decoder side, similar to before, the first
decoder $f_{\phi_{1}}$ outputs the posterior $\bm{q_{1}}$. However,
$\bm{q_{1}}$ now goes through an interleaver $\pi$. The second decoder
$f_{\phi_{2}}$ takes as input $\bm{y_{1}}$, $\bm{y_{3}}$ and
$\pi(\bm{q_{1}})$, and outputs the second posterior $\bm{q_{2}}$, which passes
through a de-interleaver $\pi^{-1}$. This process is repeated through multiple
iterations. Figure 10 depicts the BER of DeepIC with and without an
interleaver, trained on $h=0.8$ and block length = 100.

Figure 10: BER performance of DeepIC with and without an interleaver:
Interleaver marginally hurts the performance.

Although the gap is not significant, introducing an interleaver does hurt the
training performance. This problem is an interesting research direction, and
perhaps the interleaved structure might yield better results when applied with
an even more complex training model. We will investigate this in our future
work.

###  IV-E Interpretation

A natural question is ‘what has DeepIC learned?’ Interpreting the behavior of
deep learning models is in general challenging, but we run several experiments
to better understand the behavior of DeepIC. In the following, we present the
perturbation analysis. We consider a special codeword $\bm{b^{*}}$, which
consists of all ’$0$’ bits except for the middle bit which is a ’$1$’. We feed
$\bm{b^{*}}$ as an input to DeepIC and RNN-AE, and plot the encoded output of
one of the two users, as shown in Figure 11.

Figure 11: Coded symbol vs. Bit position: Codeword representation of the
message sequence $b^{*}$ of one of the two users for (a) DeepIC and (b) RNN-
AE.

By looking at the plots, we can verify the increased memory-dependence of the
codeword when comparing DeepIC and RNN-AE, by noticing the duration of the
perturbances caused by encoding the message bit ’$1$’. Note that the three
peaks of Figure 11 (a) correspond to the three concatinated encoders
$g_{\theta_{1}}$, $g_{\theta_{2}}$ and $g_{\theta_{3}}$ presented in Figure 4,
and the single peak of Figure 11 (b) is due to the presence of just one
encoder in RNN-AE’s architecture.

Figure 12: Coded symbol vs. Bit position: Codeword representation of the
message sequence $b^{*}$ of one of the two users for (a) DeepIC zoomed on the
first peak (b) RNN-AE zoomed on the middle peak.

In Figure 12, we show the zoomed-in plots for Figure 11. As is shown in the
figure, the length of that perturbation is bigger for DeepIC than for RNN-AE.
We see that 10 positions nearby the center are perturbed for DeepIC while only
3 positions nearby the center are perturbed for RNN-AE. This better explains
the results described in Figure 8, and is one of the key reasons of the
dominating performance of DeepIC when compared to other schemes, as we saw in
Figure 6.

##  V Conclusion and discussion

In this paper, we show that properly designed and trained CNN codes with
iterative decoders, which we call DeepIC, outperform the state-of-the-art by a
significant margin on the challenging problem of communicating over two-user
AWGN interference channels. While it had been shown in the literature [4] that
neural network based codes, in particular, feedforward networks, can
outperform the time division for a very short block length, such as $K=4$
bits, this work is the first to show that such improvement is possible for
longer blocklengths, such as $K=100$ bits.

Learning codes for longer blocklengths. As blocklength increases,
outperforming the TD and TIN via deep learning becomes more challenging
because TD and TIN implement the Turbo codes, the reliability of which
improves as the blocklength increases. In order to outperform TD and TIN for
longer blocklengths, neural codes need to learn to utilize memory in the
encoding process. In other words, we want the $k$-th coded symbol to depend
not only on $b_{k}$, but also on other $b_{j}$’s for $j\neq k$. We empirically
show that DeepIC has a longer memory than other direct applications of
existing neural architecture and training methodology in the literature. We
also show that DeepIC outperforms all the baselines by a significant margin
(upto 2dB gain vs. up to 1dB gain in [4]).

Open problems. There are several interesting open problems that extend from
DeepIC. First is incorporating interleavers. While DeepIC has a longer memory
than prior work, we believe its memory is still not as large as Turbo codes. A
natural way to resolve this limitation is to incorporate interleavers;
harmoniously combining interleavers with DeepIC and devising appropriate
training strategy is a very interesting open problem. Second, we mainly
focused on a rate-1/3 setting. Learning codes for other rates are left as
future work. Last, applying DeepIC on more complicated interference channels,
including fading channels and channels with impediments, is another promising
direction of future work.

## References

  * [1] A. El Gamal and Y.-H. Kim, _Network Information Theory_. Cambridge University Press, 2011.
  * [2] R. H. Etkin, D. N. C. Tse, and H. Wang, “Gaussian interference channel capacity to within one bit,” _IEEE Transactions on Information Theory_ , vol. 54, no. 12, pp. 5534–5562, 2008.
  * [3] T. Han and K. Kobayashi, “A new achievable rate region for the interference channel,” _IEEE Transactions on Information Theory_ , vol. 27, no. 1, pp. 49–60, 1981.
  * [4] T. O’shea and J. Hoydis, “An introduction to deep learning for the physical layer,” _IEEE Transactions on Cognitive Communications and Networking_ , vol. 3, no. 4, pp. 563–575, 2017.
  * [5] E. Nachmani, E. Marciano, L. Lugosch, W. J. Gross, D. Burshtein, and Y. Be’ery, “Deep learning methods for improved decoding of linear codes,” _IEEE Journal of Selected Topics in Signal Processing_ , vol. 12, no. 1, pp. 119–131, 2018.
  * [6] A. Elkelesh, M. Ebada, S. Cammerer, and S. t. Brink, “Decoder-tailored polar code design using the genetic algorithm,” _IEEE Transactions on Communications_ , vol. 67, no. 7, pp. 4521–4534, July 2019.
  * [7] H. Kim, Y. Jiang, S. Kannan, S. Oh, and P. Viswanath, “Deepcode: Feedback codes via deep learning,” in _Advances in neural information processing systems_ , 2018, pp. 9436–9446.
  * [8] N. Shlezinger, Y. C. Eldar, N. Farsad, and A. J. Goldsmith, “Viterbinet: Symbol detection using a deep learning based viterbi algorithm,” in _2019 IEEE 20th International Workshop on Signal Processing Advances in Wireless Communications (SPAWC)_ , July 2019, pp. 1–5.
  * [9] D. Wu, M. Nekovee, and Y. Wang, “Deep learning-based autoencoder for m-user wireless interference channel physical layer design,” _IEEE Access_ , vol. 8, pp. 174 679–174 691, 2020.
  * [10] R. Mishra, K. Chahine, H. Kim, S. Jafar, and S. Vishwanath, “Distributed interference alignment for $k$-user interference channels via deep learning,” in _to appear in IEEE International Symposium on Information Theory (ISIT)_ , 2021.
  * [11] F. A. Aoudia and J. Hoydis, “End-to-end learning of communications systems without a channel model,” in _2018 52nd Asilomar Conference on Signals, Systems, and Computers_. IEEE, 2018, pp. 298–303.
  * [12] C. Berrou, A. Glavieux, and P. Thitimajshima, “Near shannon limit error-correcting coding and decoding: Turbo-codes. 1,” in _Proceedings of ICC’93-IEEE International Conference on Communications_ , vol. 2. IEEE, 1993, pp. 1064–1070.
  * [13] Y. Jiang, H. Kim, H. Asnani, S. Kannan, S. Oh, and P. Viswanath, “Turbo Autoencoder: Deep learning based channel codes for point-to-point communication channels,” _Advances in Neural Information Processing Systems (NeurIPS)_ , 2019.