# Ferromagnetic state with large magnetic moments
realized in epitaxially strained Sr3Ru2O7 films

Ren Oshima  Department of Physics, Tokyo Institute of Technology, Tokyo
152-8551, Japan  Tatsuto Hatanaka  Research Center for Advanced Science and
Technology, the University of Tokyo, Tokyo 153-8904, Japan  Shinichi Nishihaya
Department of Physics, Tokyo Institute of Technology, Tokyo 152-8551, Japan
Takuya Nomoto  Research Center for Advanced Science and Technology, the
University of Tokyo, Tokyo 153-8904, Japan  Markus Kriener  RIKEN Center for
Emergent Matter Science (CEMS), Wako 351-0198, Japan  Takahiro C. Fujita
Department of Applied Physics, the University of Tokyo, Tokyo 113-8656, Japan
Masashi Kawasaki  RIKEN Center for Emergent Matter Science (CEMS), Wako
351-0198, Japan  Department of Applied Physics, the University of Tokyo, Tokyo
113-8656, Japan  Ryotaro Arita  Research Center for Advanced Science and
Technology, the University of Tokyo, Tokyo 153-8904, Japan  RIKEN Center for
Emergent Matter Science (CEMS), Wako 351-0198, Japan  Masaki Uchida
<EMAIL_ADDRESS>Department of Physics, Tokyo Institute of
Technology, Tokyo 152-8551, Japan

###### Abstract

Technical advancement of oxide molecular beam epitaxy (MBE) has opened new
avenues for studying various quantum transport phenomena in correlated
transition-metal oxides, as exemplified by the exotic superconductivity of
Sr2RuO4 and quantum oscillations of SrRuO3. On the other hand, film research
of another Ruddlesden-Popper strontium ruthenate Sr3Ru2O7 which exhibits a
unique quantum phase related to metamagnetism in bulk systems did not progress
well. Here we report the fabrication of high-quality Sr3Ru2O7 thin films by
oxide MBE and the observation of a strain-induced ferromagnetic ground state.
The change in magnetic exchange coupling evaluated by first-principles
calculations indicates a systematic relation between the compression of the
$c$-axis length and induced ferromagnetism. Giant epitaxial strain in high-
quality films will be a key to a comprehensive understanding of the magnetism
in Ruddlesden-Popper strontium ruthenates Srn+1RunO3n+1, which sensitively
depends on the ratio of in-plane to out-of-plane Ru-Ru distances.

The family of Ruddlesden-Popper (RP) strontium ruthenates, denoted as
Srn+1RunO3n+1, has gathered significant interest due to its diverse quantum
phases emerging at low temperatures, including a superconducting phase in
Sr2RuO4 ($n$ = 1) SRO214_firstpaper , a nematic phase in Sr3Ru2O7 ($n$ = 2)
SRO327_nematic3 ; SRO327_nematic , and a ferromagnetic phase with magnetic
monopoles in SrRuO3 ($n$ = $\infty$) SRO113_Fang . The RP ruthenates consist
of SrO and RuO2 layers in a $n$+1 to $n$ ratio, and each repeated unit has
alternately stacked SrO and RuO2 with SrO termination along the $c$-axis. As
the electrons are itinerant in the RuO2 layer, the electronic state of the RP
ruthenates is quasi-two-dimensional for $n$ = 1 SRO214_dimensionality , and it
becomes more three-dimensional as $n$ increases SRO113_dimensionality ,
leading to a variety of quantum phases formed depending on $n$.

Experimentally, the sample quality of ruthenate samples is crucial for
studying these intrinsic properties of Srn+1RunO3n+1 in detail, as impurities
make the observation of quantum phases more challenging SRO214_disorder ;
SRO327_res_mag . While most of the research on RP ruthenates has been
performed with bulk single-crystals, recent technical advances in oxide
molecular beam epitaxy (MBE) have enabled quantum transport measurements of
high-quality RP ruthenate films for both end members $n$ = 1 (Sr2RuO4)
SRO214_MBEuchida1 ; SRO214_Nair ; SRO214_MBEuchida2 ; SRO214_MBEuchida3 ;
SRO214_MBETEM  and $n$ = $\infty$ (SrRuO3) SRO113_Nair ; SRO113_takiguchi ;
SRO113_wakabayashi ; SRO113_taiwan . This also enables to gain control over
the parameters unique to films such as strain, thickness, and heterointerface.
For example, an enhancement of the upper critical field has been universally
observed in Sr2RuO4 films possibly due to confinement SRO214_MBEuchida2  and
control of the easy-axis direction of the ferromagnetic ordering and its
ferromagnetic transition temperature have been studied in SrRuO3 films by
epitaxial strain SRO113_wakabayashi . On the other hand, Sr3Ru2O7 thin film
research has not yet reached such an advanced stage, strongly calling for
high-quality film samples.

Sr3Ru2O7 exhibits a paramagnetic ground state with strong magnetic
fluctuations SRO327_nematic_mag . Especially in high-purity bulk single-
crystal samples, it has been reported that the resistivity rapidly increases
at the metamagnetic transition SRO327_nematic_mag ; SRO327_nematic . Such
behavior of the magnetoresistivity has been understood to originate from an
electronic nematic phase, which spontaneously lowers the $C_{4}$ symmetry of
the Fermi surface to $C_{2}$ SRO327_nematic1 . While the formation of a spin-
density wave has been suggested as an origin of the complex nematic ordering
SRO327_neutron1 , the actual underlying mechanism remains still elusive,
demanding advanced experiments based on high-quality films. As in Sr2RuO4 and
SrRuO3, the quantum phase in Sr3Ru2O7 is potentially controllable by strain.
The magnetic and transport properties of RP systems could be sensitive to
changes in the lattice constants, and therefore, it is very promising to
engineer epitaxial strain in thin films.

While thin film growth of Sr3Ru2O7 has been reported by several methods
SRO327_PLDfilm ; SRO327_MBEfilm1 ; SRO327_MBEfilm2 , the main difficulty of
obtaining high-quality films stems from the thermodynamic stability of
Sr3Ru2O7. According to thermodynamics, Sr3Ru2O7 exists between Sr2RuO4 and
SrRuO3 only in very narrow ranges of temperature, pressure, and flux
SRO113_Nair ; SRO_diagram1 , so that intergrowth of other RP structures is
easily formed during the film growth, hindering the evaluation of the
intrinsic properties of Sr3Ru2O7. Here we report the successful growth of
high-quality Sr3Ru2O7 films on a SrTiO3 (001) substrate by a solid-source MBE
technique. Owing to an accurate adjustment of the Sr and Ru fluxes as well as
the supply of high-purity ozone, we have achieved high-purity Sr3Ru2O7 films
with a limited number of intergrowths of other RPs. Large in-plane epitaxial
strain results in a transition from a paramagnetic ground state to a
ferromagnetic ground state in Sr3Ru2O7 films.

Sr3Ru2O7 thin films were grown on (001) SrTiO3 step-terraced substrates using
a solid-source MBE system RuO2_MBEuchida . The SrTiO3 substrate was heated at
800 ∘C with a semiconductor-laser heating system, and mixed oxidant gas
consisting of 60% O3 and 40% O2 was supplied at a pressure of $4.5\times
10^{-7}$ Torr from a Meidensya ozone generator. 4N Sr flux was supplied from a
Knudsen cell and measured with a nude ionization gauge. 3N5 Ru flux was
simultaneously supplied using an electron beam evaporator and measured with a
quartz crystal microbalance, which is positioned to monitor and adjust the Ru
flux even during the growth process. Low-temperature transport measurements up
to 9 T were performed using a Cryomagnetics cryostat system equipped with a
superconducting magnet. The longitudinal resistivity was measured with the
conventional low-frequency lock-in technique. The measurement was performed by
a standard four-probe method with flowing an electric current along the in-
plane $a$-axis and applying a magnetic field along the out-of-plane $c$-axis.
The magnetization of the film was measured using a superconducting quantum
interference device magnetometer in a Quantum Design Magnetic Property
Measurement System (MPMS). Spin density functional theory (SDFT) was applied
to film and bulk crystal structures at 2 K, using a k-point mesh of
8$\times$8$\times$8 and a cutoff energy of 800 eV. Based on the SDFT results
obtained by the Vienna Ab initio Simulation Package (VASP) code VASP , Wannier
functions were constructed by the WANNIER90 code using an 8$\times$8$\times$8
k-point mesh, Sr $p$-orbitals, Ru $s$-/$p$-/$d$-orbitals and O $p$-orbitals
Wannier90 . The range for reproducing the band structure was set from –8 eV to
2 eV from the Fermi level. To estimate the magnitude of the exchange
interaction, the local force method was applied, and 12 k-mesh was used in the
calculations.

Figure 1 summarizes x-ray diffraction data taken on a typical Sr3Ru2O7 film.
As seen in the 2$\theta$-$\omega$ scan in Fig. 1(b), sharp peaks with Miller
indices of (002$l$) Sr3Ru2O7 ($l$: integer) are clearly observed up to high
2$\theta$ angles. This indicates that a $c$-axis oriented Sr3Ru2O7 film is
epitaxially grown on the SrTiO3 substrate. The film thickness is calculated to
be 41.7 nm from Kiessig fringes in the reflectivity measurement shown in Fig.
1(c).

The in-plane epitaxial relation between the Sr3Ru2O7 film and the SrTiO3
substrate is examined by performing x-ray reciprocal space mapping as shown in
Fig. 1(d). The Sr3Ru2O7 (1015) peak is slightly shifted from its original
position in bulk samples and its $q_{x}$ value becomes equal to the SrTiO3
(103) one. This demonstrates that the in-plane lattice of the Sr3Ru2O7 film
perfectly matches that of the substrate by a tensile strain of 0.27% along
both the $a$\- and the $b$-axes. If the strain is partially relaxed with
approaching the top surface, the film peak should be broadened with tail
structure toward the bulk peak position, but no such behavior is confirmed.
From the $q_{z}$ value it is also found that the out-of-plane lattice of the
film along the $c$-axis is compressively strained by 0.38%. The x-ray
reciprocal space mapping was performed after transport measurements at low
temperatures, also meaning that the film in-plane lattice is maintained to
match the substrate one even at low temperatures.

Figure 2 shows cross-sectional images of the Sr3Ru2O7 film taken by scanning
transmission electron microscopy. In this high-angle annular dark-field image,
Sr and Ru atoms are observed as bright objects. As can be seen more clearly in
the magnification in Fig. 2(b), unit layers, each of which is composed of
three SrO and two RuO2 layers, are periodically stacked to form the Sr3Ru2O7
lattice along the $c$-axis. The Sr3Ru2O7 lattice occupies most of the entire
observation area, demonstrating the dominant growth of Sr3Ru2O7 with high-
crystallinity. This is consistent with the observation of sharp x-ray
diffraction peaks in Fig. 1(b).

In this Sr3Ru2O7 film, the out-of-phase boundary, where SrO and RuO2 layers
are misaligned by a fraction of the unit layer dimension as often observed in
Sr2RuO4 films SRO214_PLD1 ; SRO214_PLD2 ; SRO214_PLDOPB , is not observed. By
contrast, two other types of stacking faults are confirmed. One is that a pair
of SrO and RuO2 layers are extra inserted or removed in the unit layer,
resulting in the formation of so called Ruddlesden-Popper faults. They are not
stacked consecutively but only included as a single unit layer of Sr4Ru3O10
($n=3$) or Sr2RuO4 ($n=1$). Please note that those are only hardly detected in
the x-ray diffraction scan shown in Fig. 1(b). The other is that SrO and RuO2
layers are exchanged with each other leading to a vertical insertion of a SrO-
SrO unit layer boundary, resulting in local formation of SrRuO3 (For details
see supplementary Fig. S1). Considering that Ruddlesden-Popper structures with
other $n$ values are easily mixed even in the case of bulk synthesis
SRO327_bulkgrowth , the successful suppression of such intergrowth compounds
indicates the accurate adjustment of fluxes during the growth, allowing us to
study the intrinsic properties in the tensile-strained Sr3Ru2O7 film.

Figure 3(a) shows the temperature dependence of the longitudinal resistivity
$\rho_{xx}$. The residual resistivity ratio (RRR), defined as the ratio
between the resistivity value at 300 K and its extrapolation to 0 K, is
calculated to be 24, which is the highest value among Sr3Ru2O7 films reported
so far. $\rho_{xx}$ under an out-of-plane field of $B_{\parallel c}$ = 9 T
exhibits small but finite negative magnetoresistance with approaching low
temperatures. This suggests that suppression of magnetic carrier scattering by
the applied magnetic field becomes dominant at low temperatures. This is also
in stark contrast to Sr3Ru2O7 bulk, which rather exhibits conventional
positive magnetoresistance derived from the Lorentz force on charge carriers
in the magnetic field. Such positive magnetoresistance is confirmed even in
case that RRR of Sr3Ru2O7 bulk is only half of that taken for the present
Sr3Ru2O7 film SRO327_res_RRR12 .

To see more details, magnetoresistance data measured upon sweeping
$B_{\parallel c}$ at various temperatures are shown in Fig. 3(b). Negative
magnetoresistance accompanied by a so-called butterfly-shaped hysteresis is
clearly confirmed at temperatures below 50 K. This indicates that a
ferromagnetic ground state is stabilized in the Sr3Ru2O7 film and
ferromagnetic domains are switched by the magnetic field with hysteresis,
different from the paramagnetic ground state realized in Sr3Ru2O7 bulk. In the
case of Sr3Ru2O7 bulk, a sharp peak also appears at metamagnetic transition
field of about 8 T, where the magnetic state changes from paramagnetic to
forced ferromagnetic one SRO327_nematic1_SdH ; SRO327_nematic_mag ;
SRO327_res_mag ; SRO327_nematic2 . Even in case that RRR of Sr3Ru2O7 bulk is
only half of the present Sr3Ru2O7 film one, a broad hump structure with
positive magnetoresistance is observed below 10 K SRO327_res_RRR12 , which
evolves into the sharp peak upon further decreasing the temperature
SRO327_nematic_mag . Therefore, the ferromagnetic behavior observed in the
present film is understood to be an intrinsic property unique to Sr3Ru2O7 thin
films under large epitaxial strain.

The intrinsic change of the magnetic ground state is more directly confirmed
by magnetization measurements. As seen in the temperature sweep in Fig. 3(c),
the magnetization $M$ steeply increases at about $T_{\text{C}}$ = 56 K for
both field cooling and zero-field cooling processes, and it becomes zero again
in the zero-field cooling measurement. This indicates that a ferromagnetic
transition, which is different from the one observed in SrRuO3 at about
$T_{\text{C}}$ = 150 K, is induced in the Sr3Ru2O7 film. We note that a small
hump in $M$ appearing around 150 K in Fig. 3(c) may be ascribed to local
intergrowth of SrRuO3, although its formation is suppressed as neither
detected by transport nor by x-ray diffraction measurements. As a possible
origin of the induced ferromagnetic state, we note that the in-plane $a$\- and
$b$-axes of the Sr3Ru2O7 film are more tensile-strained at 2 K, i.e., by 0.64%
or 0.80% depending on their alignment with respect to the respective axes
($a$-/$b$-axis or $c$-axis) of the SrTiO3 substrate with tetragonal
distortion. Actually, the observed transition temperature is very close to the
one reported for bulk Sr3Ru2O7 under uniaxial pressure up to 0.5 GPa along the
$c$-axis SRO327_uniaxialP1 , where the $a$\- and $b$-axes are estimated to be
elongated by 0.11% at maximum. While the in-plane tensile strain is much
larger in the present case, it has been also reported that in uniaxial
pressure experiment the transition temperature is almost independent of the
magnitude of the applied pressure once the ferromagnetic state is induced
SRO327_uniaxialP1 .

Figure 3(d) presents in-plane and out-of-plane magnetization curves measured
at 2 K. The tensile-strained Sr3Ru2O7 film exhibits giant magnetic saturation
moment $M_{\text{s}}$ of about 2 $\mu_{\text{B}}$/Ru with the easy
magnetization direction pointing along the $c$-axis. Taking into account the
$4d^{4}$ electronic configuration of the Ru4+ ion, $M_{\text{s}}$ = 2
$\mu_{\text{B}}$/Ru is reasonable assuming fully polarized unpaired spins in
the $t_{2\mathrm{g}}$ orbitals.

Figure 4(a) compares the magnetization curves of the film with those of bulk
samples under uniaxial pressure applied along the $c$-axis direction
SRO327_uniaxialP1 . Bulk samples under uniaxial pressure show a ferromagnetic
ground state with hysteresis, evidencing that ferromagnetism is induced by
pressure, while without pressure they show a paramagnetic ground state with a
metamagnetic transition SRO327_res_mag . The magnetization increases with
increasing the applied pressure.

At low temperatures, unstrained bulk Sr3Ru2O7 exhibits an in-plane rotation of
the RuO6 octahedra, as sketched in Fig. 4(b). The magnitude of such an in-
plane rotation is expected to decrease with the tensile strain. Given such
conditions and the known rotation angle of 7.18∘ reported for strain-free bulk
samples SRO327_neutron5 , a rotation angle of 0∘ corresponds to an in-plane
tensile strain of approximately 0.8%. Such a strain condition is close to the
situation in the tensile-strained film at 2 K, suggesting that the RuO6
octahedra in the film have almost no in-plane rotation at low temperatures as
shown in Fig. 4(b). On the other hand, the $c$-axis length is calculated to be
compressed 1.14% in the film at 2 K.

Based on the crystal structures of bulk samples and tensile-strained films at
2 K discussed above, we modeled the density of states (DOS) and the strain
effect on the magnetic exchange coupling by calculations Liechtenstein1 ;
Liechtenstein2 ; Liechtenstein3 . Using the spin density functional theory
method, we confirm that the largest contributions to the DOS originates from
the Ru d-orbitals near the Fermi level with spin splitting (For details see
supplementary Figs. S2 and S3). Figures 4(c) and 4(d) present the exchange
couplings $J$ for both cases calculated by the local force method. A
particularly important difference appears for the second nearest-neighbor (NN)
Ru atoms along the $c$-axis. This out-of-plane exchange coupling is positive
and the largest also in previous first-principles calculations on Sr3Ru2O7
SRO327_calc1 ; SRO327_calc2 . The exchange coupling of those is increased by
28% when their distance is reduced by 0.85%, which we propose as the main
factor enhancing the ferromagnetism in the tensile-strained film. Such an
enhancement of the ferromagnetic coupling accompanied by a shortened Ru-Ru
distance along the $c$-axis is also consistent with the trends seen in Fig.
4(a). In a recent report of SrRuO3-SrTiO3 heterostructure SROhetero , a
ferromagnetic transition is observed in a Ru-bilayer system, while it is not
observed in a Ru-monolayer one. This is also consistent with our argument that
the exchange interaction between the out-of-plane Ru-Ru atoms is important in
the magnetism of strontium ruthenates.

In summary, high-quality Sr3Ru2O7 films with RRR = 24 are successfully
epitaxially grown on a SrTiO3 substrate by solid-source MBE. A change of the
magnetic ground state to ferromagnetism with a saturation moment $M\sim
2\mu_{\mathrm{B}}$/Ru is induced by giant tensile strain. First-principles
calculations using the local force method highlight the importance of the
ferromagnetic interaction between the Ru atoms along the $c$-axis, which is
largely enhanced by the tensile strain. The realization of a ferromagnetic
ground state in tensile-strained Sr3Ru2O7 indicates that the Ru-Ru distance
along the $c$-axis is a key controlling parameter for magnetism in Sr3Ru2O7.
Our work also provides insight into the magnetism in the Ruddlesden-Popper
series of Srn+1RunO3n+1. Actually, SrRuO3, with its continuous stacking of the
RuO6 octahedra along the $c$-axis, exhibits ferromagnetic ordering, while
Sr2RuO4, without such a continuous stacking, does not. Hence, the large
epitaxial strain that is difficult to achieve in bulk samples will be the key
in comprehensively tuning the magnetism in the Ruddlesden-Popper series in
future works.

This work was supported by JSPS KAKENHI Grant Numbers JP22H04471, JP19H05825,
JP21H01804, JP22H04501, JP22K18967, JP22K20353, JP22H04958 from MEXT, Japan,
by JST FOREST Program Grant Number JPMJFR202N, and by The Asahi Glass
Foundation, Japan.

## References

  * (1) Y. Maeno, H. Hashimoto, K. Yoshida, S. Nishizaki, T. Fujita, J. G. Bednorz, and F. Lichtenberg, Superconductivity in a layered perovskite without copper, Nature 372, 532–534 (1994).
  * (2) S. A. Grigera, R. S. Perry, A. J. Schofield, M. Chiao, S. R. Julian, G. G. Lonzarich, S. I. Ikeda, Y. Maeno, A. J. Millis, and A. P. Mackenzie, Magnetic Field-Tuned Quantum Criticality in the Metallic Ruthenate Sr3Ru2O7, Science 294, 329–332 (2001).
  * (3) S. A. Grigera, P. Gegenwart, R. A. Borzi, F. Weickert, A. J. Schofield, R. S. Perry, T. Tayama, T. Sakakibara, Y. Maeno, A. G. Green, and A. P. Mackenzie, Disorder-Sensitive Phase Formation Linked to Metamagnetic Quantum Criticality, Science 306, 1154–1157 (2004).
  * (4) Z. Fang, N. Nagaosa, K. S. Takahashi, A. Asamitsu, R. Mathieu, T. Ogasawara, H. Yamada, M. Kawasaki, Y. Tokura, K. Terakura, The Anomalous Hall Effect and Magnetic Monopoles in Momentum Space, Science 302, 92–95 (2003).
  * (5) N. E. Hussey, A. P. Mackenzie, J. R. Cooper, Y. Maeno, S. Nishizaki, and T. Fujita, Normal-state magnetoresistance of Sr2RuO4, Phys. Rev. B 57, 5505 (1998).
  * (6) G. Cao, S. McCall, M. Shepard, J. E. Crow, and R. P. Guertin, Thermal, magnetic, and transport properties of single-crystal Sr1-xCaxRuO3 ($0\leq x\leq 1.0$), Phys. Rev. B 56, 321 (1997).
  * (7) A. P. Mackenzie, R.K.W. Haselwimmer, A. W. Tyler, G.G. Lonzarich, Y. Mori, S. Nishizaki, and Y. Maeno, Extremely Strong Dependence of Superconductivity on Disorder in Sr2RuO4, Phys. Rev. Lett. 80, 161 (1998).
  * (8) R. S. Perry, T. Tayama, K. Kitagawa, T. Sakakibara, K. Ishida, and Y. Maeno, Investigation into the Itinerant Metamagnetism of Sr3Ru2O7 for the Field Parallel to the Ruthenium Oxygen Planes, J. Phys. Soc. Jpn. 74, 1270–1274 (2005).
  * (9) M. Uchida, M. Ide, H. Watanabe, K. S. Takahashi, Y. Tokura, and M. Kawasaki, Molecular beam epitaxy growth of superconducting Sr2RuO4 films, APL Mater. 5, 106108 (2017).
  * (10) H. P. Nair, J. P. Ruf, N. J. Schreiber, L. Miao, M. L. Grandon, D. J. Baek, B. H. Goodge, J. P. C. Ruff, L. F. Kourkoutis, K. M. Shen, and D. G. Schlom, Demystifying the growth of superconducting Sr2RuO4 thin films, APL Mater. 6, 101108 (2018).
  * (11) M. Uchida, M. Ide, M. Kawamura, K. S. Takahashi, Y. Kozuka, Y. Tokura, and M. Kawasaki, Anomalous enhancement of upper critical field in Sr2RuO4 thin films, Phys. Rev. B 99, 161111(R) (2019).
  * (12) M. Uchida, I. Sakuraba, M. Kawamura, M. Ide, K. S. Takahashi, Y. Tokura, and M. Kawasaki, Characterization of Sr2RuO4 Josephson junctions made of epitaxial films, Phys. Rev. B 101, 035107 (2020).
  * (13) B. H. Goodge, H. P. Nair, D. J. Baek, N. J. Schreiber, L. Miao, J. P. Ruf, E. N. Waite, P. M. Carubia, K. M. Shen, D. G. Schlom, and L. F. Kourkoutis, Disentangling types of lattice disorder impacting superconductivity in Sr2RuO4 by quantitative local probes, APL Mater. 10, 041114 (2022).
  * (14) H. P. Nair, Y. Liu, J. P. Ruf, N. J. Schreiber, S. Shang, D. J. Baek, B. H. Goodge, L. F. Kourkoutis, Z. Liu, K. M. Shen, and D. G. Schlom, Synthesis science of SrRuO3 and CaRuO3 epitaxial films with high residual resistivity ratios, APL Mater. 6, 046101 (2018).
  * (15) K. Takiguchi, Y. K. Wakabayashi, H. Irie, Y. Krockenberger, T. Otsuka, H. Sawada, S. A. Nikolaev, H. Das, M. Tanaka, Y. Taniyasu and H. Yamamoto, Quantum transport evidence of Weyl fermions in an epitaxial ferromagnetic oxide, Nat. Commun. 11, 4969 (2020).
  * (16) Y. K. Wakabayashi, S. Kaneta-Takada, Y. Krockenberger, Y. Taniyasu, and H. Yamamoto, Wide-Range Epitaxial Strain Control of Electrical and Magnetic Properties in High-Quality SrRuO3 Films, ACS Appl. Electron. Mater. 3, 2712-2719 (2021).
  * (17) U. Kar, A. Kr. Singh, Y. Hsu, C. Lin, B. Das, C. Cheng, M. Berben, S. Yang, C. Lin, C. Hsu, S. Wiedmann, W. Lee and W. Lee, The thickness dependence of quantum oscillations in ferromagnetic Weyl metal SrRuO3, npj Quantum Mater. 8, 8 (2023).
  * (18) R. S. Perry, L. M. Galvin, S. A. Grigera, L. Capogna, A. J. Schofield, A. P. Mackenzie, M. Chiao, S. R. Julian, S. I. Ikeda, S. Nakatsuji, Y. Maeno, and C. Pfleiderer, Metamagnetism and Critical Fluctuations in High Quality Single Crystals of the Bilayer Ruthenate Sr3Ru2O7, Phys. Rev. Lett. 86, 2661 (2001).
  * (19) R. A. Borzi, S. A. Grigera, J. Farrell, R. S. Perry, S. J. S. Lister, S. L. Lee, D. A. Tennant, Y. Maeno, and A. P. Mackenzie, Formation of a Nematic Fluid at High Fields in Sr3Ru2O7, Science 315, 214–217 (2007).
  * (20) C. Lester, S. Ramos, R. S. Perry, T. P. Croft, R. I. Bewley, T. Guidi, P. Manuel, D. D. Khalyavin, E. M. Forgan, and S. M. Hayden, Field-tunable spin-density-wave phases in Sr3Ru2O7, Nat. Mater. 14, 373–378 (2015).
  * (21) W. Tian, J. H. Haeni, and D. G. Schlom, E. Hutchinson, B. L. Sheu, M. M. Rosario, P. Schiffer, Y. Liu, M. A. Zurbuchen, and X. Q. Pan, Epitaxial growth and magnetic properties of the first five members of the layered Srn+1RunO3n+1 oxide series, Appl. Phys. Lett. 90, 022507 (2007).
  * (22) P. B. Marshall, K. Ahadi, H. Kim, and S. Stemmer, Electron nematic fluid in a strained Sr3Ru2O7 film, Phys. Rev. B 97, 155160 (2018).
  * (23) T. Ohnishi, and K. Takada, Epitaxial Thin-Film Growth of SrRuO3, Sr3Ru2O7, and Sr2RuO4 from a SrRuO3 Target by Pulsed Laser Deposition, Appl. Phys. Express 4, 025501 (2011).
  * (24) K. T. Jacob, K. T. Lwin, and Y. Waseda, Strontium ruthenates: determination of Gibbs energies of formation using electrochemical cells, Mater. Sci. Eng. B 103, 152–161 (2003).
  * (25) M. Uchida, T. Nomoto, M. Musashi, R. Arita, and M. Kawasaki, Superconductivity in Uniquely Strained RuO2 Films, Phys. Rev. Lett. 125, 147001 (2020).
  * (26) G. Kresse, and J. Furthmüller, Efficient iterative schemes for ab initio total-energy calculations using a plane-wave basis set, Phys. Rev. B. 54, 11169 (1996).
  * (27) G. Pizzi, V. Vitale, R. Arita, S. Blùgel, F. Freimuth, G. Géranton, M. Gibertini, D. Gresch, C. Johnson, T. Koretsune, J. Ibañez-Azpiroz, H. Lee, J. Lihm, D. Marchand, A. Marrazzo, Y. Mokrousov, J. I. Mustafa, Y. Nohara, Y. Nomura, L. Paulatto, S. Poncé, T. Ponweiser, J. Qiao, F. Thöle, S. S. Tsirkin, M. Wierzbowska, N. Marzari, D. Vanderbilt, I. Souza, A. A. Mostofi, and J. R. Yates, Wannier90 as a community code: new features and applications, J. Phys.: Condens. Matter 32, 165902 (2020).
  * (28) Y. Krockenberger, M. Uchida, K. S. Takahashi, M. Nakamura, M. Kawasaki, and Y. Tokura, Growth of superconducting Sr2RuO4 thin films, Appl. Phys. Lett. 97, 082502 (2010).
  * (29) J. Kim, J. Mun, C. M. P. García, B. Kim, R. S. Perry, Y. Jo, H. Im, H. G. Lee, E. K. Ko, S. H. Chang, S. B. Chung, M. Kim, J. W. A. Robinson, S. Yonezawa, Y. Maeno, L. Wang, and T. W. Noh, Superconducting Sr2RuO4 Thin Films without Out-of-Phase Boundaries by Higher-Order Ruddlesden-Popper Intergrowth, Nano Lett. 21, 4185–4192 (2021).
  * (30) M. A. Zurbuchen, Y. Jia, S. Knapp, A. H. Carim, D. G. Schlom, L. Zou, and Y. Liu, Suppression of superconductivity by crystallographic defects in epitaxial Sr2RuO4 films, Appl. Phys. Lett. 78, 2351–2353 (2001).
  * (31) R. S. Perry, and Y. Maeno, Systematic approach to the growth of high-quality single crystals of Sr3Ru2O7, J. Cryst. Growth 271, 134–141 (2004).
  * (32) Y. Liu, R. Jin, Z. Q. Mao, K. D. Nelson, M. K. Haas, and R. J. Cava, Electrical transport properties of single-crystal Sr3Ru2O7: The possible existence of an antiferromagnetic instability at low temperatures, Phys. Rev. B 63, 174435 (2001).
  * (33) R. S. Perry, K. Kitagawa, S. A. Grigera, R. A. Borzi, A. P. Mackenzie, K. Ishida, and Y. Maeno, Multiple First-Order Metamagnetic Transitions and Quantum Oscillations in Ultrapure Sr3Ru2O7, Phys. Rev. Lett. 92, 166602 (2004).
  * (34) J. A. N. Bruin, R. A. Borzi, S. A. Grigera, A. W. Rost, R. S. Perry, and A. P. Mackenzie, Study of the electronic nematic phase of Sr3Ru2O7 with precise control of the applied magnetic field vector, Phys. Rev. B 87, 161106(R) (2013).
  * (35) S. Ikeda, N. Shirakawa, T. Yanagisawa, Y. Yoshida, S. Koikegami, S. Koike, M. Kosaka, and Y. Uwatoko, Uniaxial-Pressure Induced Ferromagnetism of Enhanced Paramagnetic Sr3Ru2O7, J. Phys. Soc. Jpn. 73, 1322–1325 (2004).
  * (36) Q. Huang, J. W. Lynn, R. W. Erwin, J. Jarupatrakorn, and R. J. Cava, Oxygen displacements and search for magnetic order in Sr3Ru2O7, Phys. Rev. B 58, 8515 (1998).
  * (37) A. I. Liechtenstein, M. I. Katsnelson, and V. A. Gubanov, Exchange interactions and spin-wave stiffness in ferromagnetic metals, J. Phys. F: Met. Phys. 14, L125 (1984).
  * (38) T. Nomoto, T. Koretsune, and R. Arita, Local force method for the ab initio tight-binding model: Effect of spin-dependent hopping on exchange interactions, Phys. Rev. B 102, 014444 (2020).
  * (39) T. Hatanaka, T. Nomoto, and R. Arita, Magnetic interactions in intercalated transition metal dichalcogenides: A study based on ab initio model construction, Phys. Rev. B 107, 184429 (2023).
  * (40) P. Rivero, V. Meunier, and W. Shelton, Half-metallic ferromagnetism in Sr3Ru2O7, Phys. Rev. B 95, 195106 (2017).
  * (41) A. Putatunda, G. Qin, W. Ren, and D. J. Singh, Competing magnetic orders in quantum critical Sr3Ru2O7, Phys. Rev. B 102, 014442 (2020).
  * (42) Z. Ali, Z. Wang, A. O’Hara, M. Saghayezhian, D. Shin, Y. Zhu, S. T. Pantelides, and J. Zhang, Origin of insulating and nonferromagnetic SrRuO3 monolayers, Phys. Rev. B 105, 054429 (2022).

Figure 1: (color online). Epitaxial growth of Sr3Ru2O7. (a) Crystal structure
of Sr3Ru2O7. (b) X-ray diffraction $2\theta$-$\omega$ scan for a Sr3Ru2O7 thin
film grown on (001) SrTiO3 substrate. Peaks marked with an asterisk, diamond,
and square are assigned to SrTiO3 substrate, Sr2RuO4 Ruddlesden-Popper faults,
and Al metal, respectively. (c) X-ray reflectivity with clear fringes
corresponding to a film thickness of 41.7 nm. (d) Reciprocal space mapping
around the (1015) Sr3Ru2O7 and (103) SrTiO3 peaks. The in-plane lattice
constant of the Sr3Ru2O7 film perfectly matches that of the SrTiO3 substrate.
(e) The in-plane lattice of bulk Sr3Ru2O7 and its tensile-strained state are
confirmed for the film at 300 K. Figure 2: (color online). Atomically resolved
stacking pattern. (a) Cross-sectional overview of the Sr3Ru2O7 film grown on
SrTiO3, taken by high-angle annular dark-field scanning transmission electron
microscopy. The stacking of the unit layers of Sr3Ru2O7, each composed of
three SrO and two RuO2 layers, is clearly resolved. Limited Ruddlesden-Popper
faults are confirmed as appearance of a single Sr4Ru3O10 or Sr2RuO4 unit
layer. (b) Magnified image of the Sr3Ru2O7 phase and (c) another one focusing
on the insertion of a single Sr2RuO4 unit layer. Figure 3: (color online).
Ferromagnetic ground state of strained Sr3Ru2O7. (a) Longitudinal resistivity
taken with applied out-of-plane magnetic fields $B_{\parallel c}$ = 0 T and 9
T as a function of temperature. (b) Magnetoresistance at various temperatures.
(c) Temperature dependence of magnetization measured for field cooling (FC)
and zero-field cooling (ZFC) processes with $B_{\parallel c}$ = 0.1 T,
exhibiting a clear ferromagnetic transition at $T_{\text{C}}=$ 56 K. The inset
depicts the $ab$-plane lattice of Sr3Ru2O7 bulk and film on SrTiO3 at 2 K.
Tensile strain from the SrTiO3 substrate is enhanced at low temperatures and
slightly depends on the SrTiO3 tetragonal domains. (d) Magnetization curves
for out of $B_{\parallel c}$ and in-plane fields $B_{\parallel a}$ at 2 K. A
giant magnetic saturation of about $2\mu_{\text{B}}$/Ru is confirmed for the
ferromagnetic state with the easy magnetization direction along the $c$-axis.
Figure 4: (color online). Theoretical calculation of the strain effect. (a)
Magnetization of in-plane tensile-strained Sr3Ru2O7 films and bulk Sr3Ru2O7
under uniaxial pressure $P_{||c}$ of 0.0, 0.4, and 0.5 GPa SRO327_res_mag ;
SRO327_uniaxialP1 . (b) Crystal structures of Sr3Ru2O7 bulk and film used in
the calculations. Pairs of nearest-neighbor (NN), second NN, and third NN Ru
atoms in Sr3Ru2O7 are also highlighted. The RuO6 octahedral rotation in the
film is suppressed by the epitaxial strain. Exchange coupling constants
calculated as a function of atomic distance for each pair of atoms in (c) bulk
and (d) film.


# Improved Channel Coding Performance Through Cost Variability

Adeel Mahmood and Aaron B. Wagner
School of Electrical and Computer Engineering, Cornell University

###### Abstract

Channel coding for discrete memoryless channels (DMCs) with mean and variance
cost constraints has been studied recently. In this work, we show that there
is coding performance improvement due to the cost variability, both with
feedback and without feedback. We demonstrate this improvement over the
almost-sure cost constraint that prohibits any cost variation above a fixed
threshold.

###### Index Terms:

Channel coding, feedback communications, second-order coding rate, stochastic
control.

##  I Introduction

Independent and identically distributed (i.i.d.) codewords are the most common
method for random codebook generation in information theory, both in source
coding and channel coding. It is notable that the almost-sure (a.s.) cost
constraint (also called the peak-power constraint), which is the most commonly
considered cost constraint ([1], [2]) in the context of discrete memoryless
channels (DMCs), prohibits the usage of i.i.d. codewords. In [3], we showed
that a feedback scheme, which alternates between i.i.d. codewords and
constant-composition codewords, can be used to improve the second-order coding
rate (SOCR) compared to the best non-feedback SOCR achievable under the
almost-sure cost constraint.

In this work, we strengthen the result in [3] by showing that the
aforementioned improvement also holds compared to the best feedback SOCR
achievable under the a.s. cost constraint. We do this by showing that feedback
does not improve the second-order performance for simple-dispersion DMCs with
the a.s. cost constraint (Theorem 2).

The result of Theorem 2 stands in contrast with [4, Theorem 3] where feedback
does improve the SOCR by allowing $O(1/n)$ variance in the average cost of the
codewords. This new cost constraint, referred to as the mean and variance cost
constraint in [4], was introduced as a relaxed version of the a.s. cost
constraint while still allowing the transmitted signal to have sufficient
concentration around the cost threshold to prevent non-ergodic use of power.
Unlike the expected cost constraint for example, the mean and variance cost
formulation:

  * •

allows for a strong converse [5, Theorem 77], [4],

  * •

allows for a finite second-order coding rate [4],

  * •

does not allow blasting power on errors in the feedback case [6].

Moreover, unlike the a.s. cost constraint, the mean and variance cost
formulation:

  * •

allows i.i.d. codewords,

  * •

allows feedback improvement to the second-order performance,

  * •

allows improved second-order performance in the non-feedback case as well
(Theorem 1).

The two main contributions of this paper pertain to the last two points above,
although the second-last point above follows jointly from [4, Theorem 3] and
Theorem 2.

##  II Preliminaries

Let $\mathcal{A}$ and $\mathcal{B}$ be finite input and output alphabets,
respectively, of the DMC $W$, where $W$ is a stochastic matrix from
$\mathcal{A}$ to $\mathcal{B}$. Let $\mathcal{P}_{n}(\mathcal{A})$ be the set
of $n$-types on $\mathcal{A}$. For a given $t\in\mathcal{P}_{n}(\mathcal{A})$,
$T^{n}_{\mathcal{A}}(t)$ denotes the type class, i.e., the set of sequences
$x^{n}\in\mathcal{A}^{n}$ with empirical distribution equal to $t$. For a
sequence $x^{n}$, we will use $t(x^{n})$ or $P_{x^{n}}$ to denote its type.
For a random variable $Z$, $||Z||_{\infty}$ denotes its essential supremum
(that is, the infimum of those numbers $z$ such that $\mathbb{P}(Z\leq z)=1$).
The cost function is denoted by $c(\cdot)$ where
$c:\mathcal{A}\to[0,c_{\max}]$ and $c_{\max}>0$ is a constant. Let
$\Gamma_{0}=\min_{a\in\mathcal{A}}c(a)$. Let $\Gamma^{*}$ denote the smallest
$\Gamma$ such that the capacity-cost function $C(\Gamma)$ is equal to the
unconstrained capacity. We assume $\Gamma^{*}>\Gamma_{0}$ and
$\Gamma\in(\Gamma_{0},\Gamma^{*})$ throughout the paper. For
$\Gamma\in(\Gamma_{0},\Gamma^{*})$, the capacity-cost function is defined as

$\displaystyle C(\Gamma)$
$\displaystyle=\max_{\begin{subarray}{c}P\in\mathcal{P}(\mathcal{A})\\\
c(P)\leq\Gamma\end{subarray}}I(P,W),$ (1)

where $c(P):=\sum_{a\in\mathcal{A}}P(a)c(a)$. The function $C(\Gamma)$ is
strictly increasing and differentiable [7, Problem 8.4] in the interval
$(\Gamma_{0},\Gamma^{*})$. For a given $x^{n}\in\mathcal{A}^{n}$, we define

$\displaystyle c(x^{n}):=\frac{1}{n}\sum_{i=1}^{n}c(x_{i}).$

Let $\Pi_{W,\Gamma}^{*}$ be the set of all capacity-cost-achieving
distributions, i.e., the set of maximizing distributions in
$(\ref{main_form})$. For any $P^{*}\in\Pi_{W,\Gamma}^{*}$, let $Q^{*}=P^{*}W$
be the marginal distribution on $\mathcal{B}$. Note that the output
distribution $Q^{*}$ is always unique, and without loss of generality, $Q^{*}$
can be assumed to satisfy $Q^{*}(b)>0$ for all $b\in\mathcal{B}$ [8,
Corollaries 1 and 2 to Theorem 4.5.1].

The following definitions will remain in effect throughout the paper:

$\displaystyle\nu_{a}$
$\displaystyle:=\text{Var}\left(\log\frac{W(Y|a)}{Q^{*}(Y)}\right),\quad\text{
where }Y\sim W(\cdot|a),$ $\displaystyle\nu_{\max}$
$\displaystyle:=\max_{a\in\mathcal{A}}\nu_{a},$ $\displaystyle i(a,b)$
$\displaystyle:=\log\frac{W(b|a)}{Q^{*}(b)}.$

###### Definition 1

A DMC $W$ with the cost threshold $\Gamma\in(\Gamma_{0},\Gamma^{*})$ is called
simple-dispersion if

$\displaystyle\min_{P^{*}\in\Pi_{W,\Gamma}^{*}}\sum_{a\in\mathcal{A}}P^{*}(a)\nu_{a}=\max_{P^{*}\in\Pi_{W,\Gamma}^{*}}\sum_{a\in\mathcal{A}}P^{*}(a)\nu_{a}.$

We will only focus on simple-dispersion channels for a fixed cost
$\Gamma\in(\Gamma_{0},\Gamma^{*})$ and thus define

$V(\Gamma):=\sum_{a\in\mathcal{A}}P^{*}(a)\nu_{a}$

for any $P^{*}\in\Pi_{W,\Gamma}^{*}$.

With a blocklength $n$ and a fixed rate $R>0$, let
$\mathcal{M}_{R}=\\{1,\ldots,\lceil\exp(nR)\rceil\\}$ denote the message set.
Let $M\in\mathcal{M}_{R}$ denote the random message drawn uniformly from the
message set.

###### Definition 2

An $(n,R)$ code for a DMC consists of an encoder $f$ which, for each message
$m\in\mathcal{M}_{R}$, chooses an input $X^{n}=f(m)\in\mathcal{A}^{n}$, and a
decoder $g$ which maps the output $Y^{n}$ to $\hat{m}\in\mathcal{M}_{R}$. The
code $(f,g)$ is random if $f$ or $g$ is random.

###### Definition 3

An $(n,R)$ code with ideal feedback for a DMC consists of an encoder $f$
which, at each time instant $k$ ($1\leq k\leq n$) and for each message
$m\in\mathcal{M}_{R}$, chooses an input
$x_{k}=f(m,x^{k-1},y^{k-1})\in\mathcal{A}$, and a decoder $g$ which maps the
output $y^{n}$ to $\hat{m}\in\mathcal{M}_{R}$. The code $(f,g)$ is random if
$f$ or $g$ is random.

###### Definition 4

An $(n,R,\Gamma)$ code for a DMC is an $(n,R)$ code such that
$c(X^{n})\leq\Gamma$ almost surely, where the message
$M\sim\text{Unif}(\mathcal{M}_{R})$ has a uniform distribution over the
message set $\mathcal{M}_{R}$.

###### Definition 5

An $(n,R,\Gamma)$ code with ideal feedback for a DMC is an $(n,R)$ code with
ideal feedback such that $c(X^{n})\leq\Gamma$ almost surely, where the message
$M\sim\text{Unif}(\mathcal{M}_{R})$ has a uniform distribution over the
message set $\mathcal{M}_{R}$.

###### Definition 6

An $(n,R,\Gamma,V)$ code for a DMC is an $(n,R)$ code such that
$\mathbb{E}\left[\sum_{i=1}^{n}c(X_{i})\right]\leq n\Gamma$ and
$\text{Var}\left(\sum_{i=1}^{n}c(X_{i})\right)\leq nV$, where the message
$M\sim\text{Unif}(\mathcal{M}_{R})$ has a uniform distribution over the
message set $\mathcal{M}_{R}$.

###### Definition 7

An $(n,R,\Gamma,V)$ code with ideal feedback for a DMC is an $(n,R)$ code with
ideal feedback such that $\mathbb{E}\left[\sum_{i=1}^{n}c(X_{i})\right]\leq
n\Gamma$ and $\text{Var}\left(\sum_{i=1}^{n}c(X_{i})\right)\leq nV$, where the
message $M\sim\text{Unif}(\mathcal{M}_{R})$ has a uniform distribution over
the message set $\mathcal{M}_{R}$.

Given $\epsilon\in(0,1)$, define

$\displaystyle
M^{*}_{\text{fb}}(n,\epsilon,\Gamma):=\max\\{\lceil\exp(nR)\rceil:\bar{P}_{\text{e,fb}}(n,R,\Gamma)\leq\epsilon\\},$

where $\bar{P}_{\text{e,fb}}(n,R,\Gamma)$ denotes the minimum average error
probability attainable by any random $(n,R,\Gamma)$ code with feedback.
Similarly, define

$\displaystyle
M^{*}(n,\epsilon,\Gamma):=\max\\{\lceil\exp(nR)\rceil:\bar{P}_{\text{e}}(n,R,\Gamma)\leq\epsilon\\},$

where $\bar{P}_{\text{e}}(n,R,\Gamma)$ denotes the minimum average error
probability attainable by any random $(n,R,\Gamma)$ code without feedback.
Define $M^{*}_{\text{fb}}(n,\epsilon,\Gamma,V)$ and
$M^{*}(n,\epsilon,\Gamma,V)$ similarly for codes with mean and variance cost
constraints.

##  III Main Results

We prove coding performance improvement in terms of the second-order coding
rate, although equivalent results in terms of error probability improvement
can also be shown as in [4, Theorem 1-3]. Let

  * •

$r_{\text{a.s.}}(\epsilon,\Gamma)$ denote the optimal SOCR with the a.s. cost
constraint without feedback,

  * •

$r_{\text{a.s.,fb}}(\epsilon,\Gamma)$ denote the optimal SOCR with the a.s.
cost constraint with feedback,

  * •

$r_{\text{m.v.}}(\epsilon,\Gamma,V)$ denote the optimal SOCR with the mean and
variance cost constraint without feedback and

  * •

$r_{\text{m.v.,fb}}(\epsilon,\Gamma,V)$ denote the optimal SOCR with the mean
and variance cost constraint with feedback

for channel codes operating with average error probability of at most
$\epsilon\in(0,1)$. In the non-feedback case, $C(\Gamma)$ is the optimal
first-order rate for DMCs with the a.s. cost constraint [7] as well as the
mean and variance cost formulation [4]. We thus define the second-order rates
with respect to the capacity-cost function $C(\Gamma)$ as follows:

$\displaystyle r_{\text{a.s.}}(\epsilon,\Gamma)$
$\displaystyle:=\liminf_{n\to\infty}\frac{\log
M^{*}(n,\epsilon,\Gamma)-nC(\Gamma)}{\sqrt{n}}$ $\displaystyle
r_{\text{m.v.}}(\epsilon,\Gamma,V)$
$\displaystyle:=\liminf_{n\to\infty}\frac{\log
M^{*}(n,\epsilon,\Gamma,V)-nC(\Gamma)}{\sqrt{n}}.$

For the feedback case, we simply take the convention to define the SOCR with
respect to $C(\Gamma)$ as follows:

$\displaystyle r_{\text{a.s.,fb}}(\epsilon,\Gamma)$
$\displaystyle:=\liminf_{n\to\infty}\frac{\log
M^{*}_{\text{fb}}(n,\epsilon,\Gamma)-nC(\Gamma)}{\sqrt{n}}$ $\displaystyle
r_{\text{m.v.,fb}}(\epsilon,\Gamma,V)$
$\displaystyle:=\liminf_{n\to\infty}\frac{\log
M^{*}_{\text{fb}}(n,\epsilon,\Gamma,V)-nC(\Gamma)}{\sqrt{n}}.$

For the a.s. cost constraint at least, this convention is justified because
from the result in Theorem 2, $C(\Gamma)$ is indeed the optimal first-order
rate for DMCs with the a.s. cost constraint in the feedback case as well.

###  III-A Performance improvement for non-feedback codes

From [1, Theorem 3], we have
$r_{\text{a.s.}}(\epsilon,\Gamma)=\sqrt{V(\Gamma)}\Phi^{-1}(\epsilon)$ for a
simple-dispersion111The result in [1, Theorem 3] is not restricted to simple-
dispersion DMCs. DMC $W$. On the other hand, [4, Theorem 1 and 2] proved that

$\displaystyle
r_{\text{m.v.}}(\epsilon,\Gamma,V)=\max\left\\{r\in\mathbb{R}:\mathcal{K}\left(\frac{r}{\sqrt{V(\Gamma)}},\frac{C^{\prime}(\Gamma)^{2}V}{V(\Gamma)}\right)\leq\epsilon\right\\}$

for a DMC $W$ with $|\Pi_{W,\Gamma}^{*}|=1$, where the function
$\mathcal{K}:\mathbb{R}\times(0,\infty)\to(0,1)$ is given by

$\displaystyle\mathcal{K}\left(r,V\right)$
$\displaystyle=\min_{\begin{subarray}{c}\Pi:\\\ \mathbb{E}[\Pi]=r\\\
\text{Var}(\Pi)\leq V\\\ |\text{supp}(\Pi)|\leq
3\end{subarray}}\mathbb{E}\left[\Phi(\Pi)\right].$

###### Theorem 1

Fix an arbitrary $\epsilon\in(0,1)$. Then for any
$\Gamma\in(\Gamma_{0},\Gamma^{*})$, $V>0$ and a DMC $W$ such that
$|\Pi_{W,\Gamma}^{*}|=1$ and $V(\Gamma)>0$, we have
$r_{\text{m.v.}}(\epsilon,\Gamma,V)>r_{\text{a.s.}}(\epsilon,\Gamma)$.

Proof: The proof is given in Section IV.

###  III-B Performance improvement for feedback codes

Random feedback codes can be constructed by controllers.

###### Definition 8

A controller is a function
$F:(\mathcal{A}\times\mathcal{B})^{*}\to\mathcal{P}(\mathcal{A})$.

A controller $F$ along with the channel $W$ specify a joint distribution over
$\mathcal{A}^{n}\times\mathcal{B}^{n}$ with probability assignments given by

$\displaystyle(F\circ
W)(x^{n},y^{n})=\prod_{k=1}^{n}F(x_{k}|x^{k-1},y^{k-1})W(y_{k}|x_{k}).$ (2)

###### Lemma 1

Consider a channel $W$ with cost constraint
$\Gamma\in(\Gamma_{0},\Gamma^{*})$. Then for every $n,\rho>0$ and
$\epsilon\in(0,1)$,

$\displaystyle\log M^{*}_{\text{fb}}(n,\epsilon,\Gamma)$
$\displaystyle\leq\log\rho-\log\left[\left(1-\epsilon-\sup_{F}\,\inf_{q\in\mathcal{P}(\mathcal{B}^{n})}(F\circ
W)\left(\frac{W(Y^{n}|X^{n})}{q(Y^{n})}>\rho\right)\right)^{+}\right],$ (3)

where the supremum in $(\ref{qandp})$ is over all controllers $F$ satisfying

$\displaystyle(F\circ W)\left(c(X^{n})\leq\Gamma\right)$
$\displaystyle=\sum_{x^{n}:c(x^{n})\leq\Gamma}\sum_{y^{n}\in\mathcal{B}^{n}}\prod_{k=1}^{n}F(x_{k}|x^{k-1},y^{k-1})W(y_{k}|x_{k})$
$\displaystyle\,=1.$ (4)

Proof: Lemma 1 is similar to [9, Theorem 27], [10, (42)], [11, Lemma 15], [4,
Lemma 2] and others. Its proof is omitted.

###### Theorem 2

Fix an arbitrary $\epsilon\in(0,1)$. Then for any
$\Gamma\in(\Gamma_{0},\Gamma^{*})$ and a simple-dispersion DMC $W$ such that
$V(\Gamma)>0$, we have
$r_{\text{a.s.,fb}}(\epsilon,\Gamma)=r_{\text{a.s.}}(\epsilon,\Gamma)$.

Proof: The proof is given in Section V.

From Theorems 1 and 2 alone, we have that
$r_{\text{m.v.,fb}}(\epsilon,\Gamma,V)>r_{\text{a.s.,fb}}(\epsilon,\Gamma,V)$.
More importantly, the mean and variance cost formulation admits feedback
mechanisms that improve the SOCR. In contrast to the almost-sure constraint
where $r_{\text{a.s.,fb}}(\epsilon,\Gamma)=r_{\text{a.s.}}(\epsilon,\Gamma)$,
we observe that
$r_{\text{m.v.,fb}}(\epsilon,\Gamma,V)>r_{\text{m.v.}}(\epsilon,\Gamma,V)$ for
simple-dispersion channels [4, Theorem 3]. Additionally, Theorem 2 extends the
previously known analogous result for DMCs without a cost constraint [11,
Theorem 3]. A similar result for parallel Gaussian channels with feedback with
the a.s. cost constraint was shown in [10]. To summarize,

$\displaystyle
r_{\text{m.v.,fb}}(\epsilon,\Gamma,V)>r_{\text{m.v.}}(\epsilon,\Gamma,V)>r_{\text{a.s.,fb}}(\epsilon,\Gamma)=r_{\text{a.s.}}(\epsilon,\Gamma).$

##  IV Proof of Theorem 1

Since $\mathcal{K}(r,V)$ is a continuous function [4, Lemma 3], it suffices to
show that for all $r\in\mathbb{R}$ and $V>0$,

$\displaystyle\min_{\begin{subarray}{c}\Pi:\\\ \mathbb{E}[\Pi]=r\\\
\text{Var}(\Pi)\leq V\\\ |\text{supp}(\Pi)|\leq
2\end{subarray}}\mathbb{E}\left[\Phi(\Pi)\right]<\Phi(r).$ (5)

The LHS of $(\ref{bv})$ can be written as

$\displaystyle\min_{\begin{subarray}{c}p,\pi:\\\ 0\leq p\leq 1\\\
\frac{p}{1-p}(\pi-r)^{2}\leq
V\end{subarray}}\left[p\Phi(\pi)+(1-p)\Phi\left(\frac{r-p\pi}{1-p}\right)\right],$

where we used the constraint $\mathbb{E}[\Pi]=r$ to eliminate one of the
decision variables.

Assume by contradiction that

$\displaystyle p\Phi(\pi)+(1-p)\Phi\left(\frac{r-p\pi}{1-p}\right)\geq\Phi(r)$
(6)

for all $\pi\geq r$, $p\in[0,1]$ and $\frac{p}{1-p}(\pi-r)^{2}\leq V$. The
assumption $\pi\geq r$ is without loss of generality since one of the two
point masses must be greater than or equal to $r$.

If $(\ref{eq})$ holds, then

$\displaystyle p\Phi(\pi)+(1-p)\Phi\left(\frac{r-p\pi}{1-p}\right)\geq\Phi(r)$

for all $\pi\geq r$, $p\in[0,1]$ and $\frac{p}{1-p}(\pi-r)^{2}=V$. Since
$\pi=r+\sqrt{\frac{V(1-p)}{p}}$ in this case, we must have

$\displaystyle
p\,\Phi\left(r+\sqrt{\frac{V(1-p)}{p}}\right)+(1-p)\Phi\left(\frac{r}{1-p}-\frac{p}{1-p}\left(r+\sqrt{\frac{V(1-p)}{p}}\right)\right)$
$\displaystyle\geq\Phi(r)$ $\displaystyle
p\,\Phi\left(r+\sqrt{\frac{V(1-p)}{p}}\right)+(1-p)\Phi\left(r-\frac{p}{1-p}\sqrt{\frac{V(1-p)}{p}}\right)$
$\displaystyle\geq\Phi(r)$ $\displaystyle
p\,\Phi\left(r+\sqrt{\frac{V(1-p)}{p}}\right)+(1-p)\Phi\left(r-\sqrt{\frac{Vp}{1-p}}\right)$
$\displaystyle\geq\Phi(r)$ (7)

for all $p\in[0,1]$.

Consider the function

$\displaystyle f(p)$
$\displaystyle=p\,\Phi\left(r+\sqrt{\frac{V(1-p)}{p}}\right)+(1-p)\Phi\left(r-\sqrt{\frac{Vp}{1-p}}\right)-\Phi(r)$

with domain $p\in[0,1]$. For any $p\in(0,1)$,

$\displaystyle\frac{f(p)-f(0)}{p}$
$\displaystyle=\Phi\left(r+\sqrt{\frac{V(1-p)}{p}}\right)+\frac{1-p}{p}\Phi\left(r-\sqrt{\frac{Vp}{1-p}}\right)-\frac{\Phi(r)}{p}$
$\displaystyle\leq\Phi\left(r+\sqrt{\frac{V(1-p)}{p}}\right)+\frac{1}{p}\Phi\left(r-\sqrt{\frac{Vp}{1-p}}\right)-\frac{\Phi(r)}{p}$
$\displaystyle\stackrel{{\scriptstyle(a)}}{{=}}\Phi\left(r+\sqrt{\frac{V(1-p)}{p}}\right)+\frac{1}{p}\left[\Phi(r)-\phi(\tilde{r})\sqrt{\frac{Vp}{1-p}}-\Phi(r)\right]$
$\displaystyle=\Phi\left(r+\sqrt{\frac{V(1-p)}{p}}\right)-\phi(\tilde{r})\sqrt{\frac{V}{p(1-p)}}.$
(8)

In equality $(a)$, we have $r-\sqrt{\frac{Vp}{1-p}}<\tilde{r}<r$ by mean value
theorem. It is easy to see that for sufficiently small $p>0$, the expression
in $(\ref{co})$ is negative. Since $f(0)=0$, we have $f(p)<0$ for some $p>0$,
which contradicts $(\ref{gp})$.

##  V Proof of Theorem 2

For any $t\in\mathcal{P}_{n}(\mathcal{A})$, define

$\displaystyle d_{W}(t):=\inf_{P\in\Pi_{W}^{*}}||t-P||_{1}.$

For any $0<\gamma\leq\frac{V(\Gamma)}{4|\mathcal{A}|\nu_{\max}}$, define

$\displaystyle\begin{split}\mathcal{P}_{n}^{\gamma}&=\left\\{x^{n}\in\mathcal{A}^{n}:c(x^{n})\leq\Gamma\text{
and }d_{W}(t(x^{n}))>\gamma\right\\}\\\
\mathcal{P}_{n}^{\gamma,c}&=\left\\{x^{n}\in\mathcal{A}^{n}:c(x^{n})\leq\Gamma\text{
and }d_{W}(t(x^{n}))\leq\gamma\right\\}.\end{split}$ (9)

###### Definition 9

For any distribution $P\in\mathcal{P}(\mathcal{A})$ and $S\subset\mathcal{A}$
such that $P(S)>0$, define the probability measure

$\displaystyle P|_{S}(x)=\begin{cases}\frac{P(x)}{P(S)}&x\in S\\\
0&\text{otherwise}.\end{cases}$

###### Definition 10

For any $k\geq 0$ and any $x^{k}\in\mathcal{A}^{k}$, let222For $k\geq n$,
$\mathcal{A}_{x^{k}}=\emptyset$ and for $k=0$, $(x^{k},x)=(x,)$.

$\displaystyle\mathcal{A}_{x^{k}}=\left\\{x\in\mathcal{A}:(x^{k},x)\text{ is a
prefix of some }x^{n}\in\mathcal{P}_{n}^{\gamma,c}\right\\}.$

Fix $(a_{0},a_{1},\ldots,a_{n-1})\in\mathcal{P}_{n}^{\gamma,c}$ arbitrarily
and let $\mathds{1}_{a_{i}}\in\mathcal{P}(\mathcal{A})$ denote a single point-
mass distribution at $a_{i}$. Then for any $0\leq k\leq n-1$,
$x^{k}\in\mathcal{A}^{k},y^{k}\in\mathcal{B}^{k}$ and a controller $F$
satisfying $(\ref{vc})$, we define the controller $F_{\gamma}$ as

$\displaystyle F_{\gamma}(x^{k},y^{k})$
$\displaystyle:=\begin{cases}F(x^{k},y^{k})|_{\mathcal{A}_{x^{k}}}&\text{ if
}F(\mathcal{A}_{x^{k}}|x^{k},y^{k})>0\\\ \text{Unif}(A_{x^{k}})&\text{ if
}F(\mathcal{A}_{x^{k}}|x^{k},y^{k})=0\text{ and }|\mathcal{A}_{x^{k}}|\neq
0\\\ \mathds{1}_{a_{k}}&\text{ otherwise}.\end{cases}$

###### Remark 1

Given any controller $F$ satisfying $(\ref{vc})$, Definition 10 constructs a
modified controller $F_{\gamma}$ which satisfies

$\displaystyle(F_{\gamma}\circ W)\left(\mathcal{P}_{n}^{\gamma,c}\right)$
$\displaystyle:=\sum_{x^{n}\in\mathcal{P}_{n}^{\gamma,c}}\sum_{y^{n}\in\mathcal{B}^{n}}\prod_{k=1}^{n}F_{\gamma}(x_{k}|x^{k-1},y^{k-1})W(y_{k}|x_{k})$
$\displaystyle=1.$

Intuitively, $F_{\gamma}$ amplifies the probability assignments of $F$ over
the set $\mathcal{P}_{n}^{\gamma,c}$ and nullifies the probability assignments
of $F$ over the set $\mathcal{P}_{n}^{\gamma}$ so that
$X^{n}\in\mathcal{P}_{n}^{\gamma,c}$ almost surely for
$(X^{n},Y^{n})\sim(F_{\gamma}\circ W)$.

###### Definition 11

For any type $t\in\mathcal{P}_{n}(\mathcal{A})$ such that
$T^{n}_{\mathcal{A}}(t)\subset\mathcal{P}_{n}^{\gamma}$, $k\geq 0$ and any
$x^{k}\in\mathcal{A}^{k}$, let

$\displaystyle\mathcal{A}_{x^{k}}^{t}=\left\\{x\in\mathcal{A}:(x^{k},x)\text{
is a prefix of some }x^{n}\in T^{n}_{\mathcal{A}}(t)\right\\}.$

Fix $(a_{0},a_{1},\ldots,a_{n-1})\in T^{n}_{\mathcal{A}}(t)$ arbitrarily and
let $\mathds{1}_{a_{i}}\in\mathcal{P}(\mathcal{A})$ denote a single point-mass
distribution at $a_{i}$. Then for any $0\leq k\leq n-1$,
$x^{k}\in\mathcal{A}^{k},y^{k}\in\mathcal{B}^{k}$ and a controller $F$
satisfying $(\ref{vc})$, we define the controller $F_{t}$ as

$\displaystyle F_{t}(x^{k},y^{k})$
$\displaystyle:=\begin{cases}F(x^{k},y^{k})|_{\mathcal{A}_{x^{k}}^{t}}&\text{
if }F(\mathcal{A}^{t}_{x^{k}}|x^{k},y^{k})>0\\\
\text{Unif}(A^{t}_{x^{k}})&\text{ if
}F(\mathcal{A}^{t}_{x^{k}}|x^{k},y^{k})=0\text{ and }|\mathcal{A}_{x^{k}}|\neq
0\\\ \mathds{1}_{a_{k}}&\text{ otherwise}.\end{cases}$

###### Remark 2

Given any controller $F$ satisfying $(\ref{vc})$, Definition 11 constructs a
modified controller $F_{t}$ which satisfies $(F_{t}\circ
W)\left(T^{n}_{\mathcal{A}}(t)\right)=1$ for
$T^{n}_{\mathcal{A}}(t)\subset\mathcal{P}_{n}^{\gamma}$.

Now let

$\displaystyle
q(y^{n})=\frac{1}{2}\prod_{i=1}^{n}Q^{*}(y_{i})+\frac{1}{2}\frac{1}{|\mathcal{P}_{n}(\mathcal{A})|}\sum_{t\in\mathcal{P}_{n}(\mathcal{A})}\prod_{i=1}^{n}q_{t}(y_{i}),$
(10)

where

$\displaystyle q_{t}(b):=\sum_{a\in\mathcal{A}}t(a)W(b|a).$

Let $P$ denote the distribution $F\circ W$. Let $P_{\gamma}$ denote the
distribution $F_{\gamma}\circ W$. Let $P_{t}$ denote the distribution
$F_{t}\circ W$. Note that all controllers $F,F_{\gamma}$ and $F_{t}$ satisfy
$(\ref{vc})$. We have

$\displaystyle P\left(\frac{W(Y^{n}|X^{n})}{q(Y^{n})}>\rho\right)$
$\displaystyle=P\left(\frac{W(Y^{n}|X^{n})}{q(Y^{n})}>\rho\cap
d_{W}(t(X^{n}))\leq\gamma\right)+P\left(\frac{W(Y^{n}|X^{n})}{q(Y^{n})}>\rho\cap
d_{W}(t(X^{n}))>\gamma\right)$
$\displaystyle=P\left(\frac{W(Y^{n}|X^{n})}{q(Y^{n})}>\rho\cap
d_{W}(t(X^{n}))\leq\gamma\right)+\sum_{t:T^{n}_{\mathcal{A}}(t)\subset\mathcal{P}_{n}^{\gamma}}P\left(\frac{W(Y^{n}|X^{n})}{q(Y^{n})}>\rho\cap
t(X^{n})=t\right)$
$\displaystyle\stackrel{{\scriptstyle(a)}}{{\leq}}P_{\gamma}\left(\frac{W(Y^{n}|X^{n})}{q(Y^{n})}>\rho\right)+\sum_{t:T^{n}_{\mathcal{A}}(t)\subset\mathcal{P}_{n}^{\gamma}}P_{t}\left(\frac{W(Y^{n}|X^{n})}{q(Y^{n})}>\rho\right).$
(11)

Inequality $(a)$ follows from the following argument. For any $(x^{n},y^{n})$
such that $x^{n}\in\mathcal{P}_{n}^{\gamma,c}$, note that for all $1\leq k\leq
n$, $\mathcal{A}_{x^{k-1}}\neq\emptyset$ and $x_{k}\in\mathcal{A}_{x^{k-1}}$
so that

$\displaystyle F(x_{k}|x^{k-1},y^{k-1})\leq
F(\mathcal{A}_{x^{k-1}}|x^{k-1},y^{k-1}).$ (12)

Then

$\displaystyle P_{\gamma}\left((x^{n},y^{n})\right)$
$\displaystyle=\prod_{k=1}^{n}F_{\gamma}(x_{k}|x^{k-1},y^{k-1})W(y_{k}|x_{k})$
$\displaystyle\stackrel{{\scriptstyle(a)}}{{\geq}}\prod_{k=1}^{n}\frac{F(x_{k}|x^{k-1},y^{k-1})}{F(\mathcal{A}_{x^{k-1}}|x^{k-1},y^{k-1})}W(y_{k}|x_{k})$
$\displaystyle\geq\prod_{k=1}^{n}F(x_{k}|x^{k-1},y^{k-1})W(y_{k}|x_{k})$
$\displaystyle=P((x^{n},y^{n})).$

With some abuse of notation, we assume in equality $(a)$ above that if
$F(\mathcal{A}_{x^{k-1}}|x^{k-1},y^{k-1})=0$, then

$\frac{F(x_{k}|x^{k-1},y^{k-1})}{F(\mathcal{A}_{x^{k-1}}|x^{k-1},y^{k-1})}=0$

which is justified by $(\ref{fw})$.

A similar derivation gives

$\displaystyle P_{t}((x^{n},y^{n}))\geq P(x^{n},y^{n})$

for all $(x^{n},y^{n})$ such that $c(x^{n})\leq\Gamma$ and $t(x^{n})=t$, where
$T^{n}_{\mathcal{A}}(t)\subset\mathcal{P}_{n}^{\gamma}$.

Let $\rho=\exp\left(nC(\Gamma)+\sqrt{n}r\right)$, where $r$ will be specified
later. Define

$\displaystyle\mathcal{G}_{i}$
$\displaystyle:=\sigma(X_{1},\ldots,X_{i+1},Y_{1},\ldots,Y_{i})$
$\displaystyle Z_{i}$
$\displaystyle:=i(X_{i},Y_{i})-\mathbb{E}\left[i(X_{i},Y_{i})|\mathcal{G}_{i-1}\right]$
$\displaystyle\mathcal{F}_{i}$
$\displaystyle:=\sigma(Z_{1},Z_{2},\ldots,Z_{i}).$

Two things are important to note here. First, by the Markov property
$(X^{i-1},Y^{i-1})-X_{i}-Y_{i}$, we have
$\mathbb{E}\left[i(X_{i},Y_{i})|\mathcal{G}_{i-1}\right]=\mathbb{E}\left[i(X_{i},Y_{i})|X_{i}\right]$
a.s. Second, $\mathcal{F}_{i}\subset\mathcal{G}_{i}$.

For the first term in $(\ref{q})$, we can upper bound it as follows:

$\displaystyle P_{\gamma}\left(\frac{W(Y^{n}|X^{n})}{q(Y^{n})}>\rho\right)$
$\displaystyle\leq
P_{\gamma}\left(\prod_{i=1}^{n}\frac{W(Y_{i}|X_{i})}{Q^{*}(Y_{i})}>\frac{\rho}{2}\right)$
$\displaystyle=P_{\gamma}\left(\sum_{i=1}^{n}\left[\log\left(\frac{W(Y_{i}|X_{i})}{Q^{*}(Y_{i})}\right)-C(\Gamma)\right]>\sqrt{n}r-\log(2)\right)$
$\displaystyle\stackrel{{\scriptstyle(a)}}{{\leq}}P_{\gamma}\left(\sum_{i=1}^{n}\left[i(X_{i},Y_{i})-\mathbb{E}\left[i(X_{i},Y_{i})|\mathcal{G}_{i-1}\right]\right]>\sqrt{n}r-\log(2)\right)$
$\displaystyle=P_{\gamma}\left(\sum_{i=1}^{n}Z_{i}>\sqrt{n}r-\log(2)\right).$
(13)

In inequality $(a)$, we used Lemma 2 and the fact that $c(X^{n})\leq\Gamma$
almost surely.

###### Lemma 2

For $\Gamma\in(\Gamma_{0},\Gamma^{*})$,

$\displaystyle\mathbb{E}\left[i(X,Y)|X\right]\leq
C(\Gamma)-C^{\prime}(\Gamma)\left(\Gamma-c(X)\right)$ (14)

almost surely, where $X$ has an arbitrary distribution and $Y$ is the output
of the channel $W$ when $X$ is the input.

Proof: See [4, Proposition 1] and its references.

We will now apply a martingale central limit theorem [12, Corollary to Theorem
2] to the expression in $(\ref{MCLT})$. We first verify that the hypotheses of
[12, Corollary to Theorem 2] are satisfied:

  1. 1.

First, we require that

$\max_{1\leq k\leq n}|Z_{k}|<\infty.$

Since $Q^{*}(b)>0$ for all $b\in\mathcal{B}$ by assumption and
$W(Y_{k}|X_{k})>0$ almost surely for each channel input and output pair
$(X_{k},Y_{k})$, we have

$\displaystyle|Z_{k}|$
$\displaystyle\leq\max_{a\in\mathcal{A},b\in\mathcal{B}:W(b|a)>0}2\,|i(a,b)|$
$\displaystyle:=2i_{\max}<\infty$

for all $1\leq k\leq n$.

  2. 2.

Second, we require that

$\displaystyle\mathbb{E}\left[Z_{k}|\sigma(Z_{1},\ldots,Z_{k-1})\right]=0$

almost surely for all $1\leq k\leq n$ [12, p. 672]. This is true because
$\mathbb{E}\left[Z_{k}|\mathcal{G}_{k-1}\right]=0$ implies

$\displaystyle\mathbb{E}\left[\mathbb{E}\left[Z_{k}|\mathcal{G}_{k-1}\right]|\sigma(Z_{1},\ldots,Z_{k-1})\right]$
$\displaystyle=0$
$\displaystyle\mathbb{E}\left[Z_{k}|\sigma(Z_{1},\ldots,Z_{k-1})\right]$
$\displaystyle=0.$

Under the above two conditions, it follows from [12, Corollary to Theorem 2]
that there exists a constant $\kappa>0$ depending only on $i_{\max}$ such that
for any $t$,

$\displaystyle
P_{\gamma}\left(\frac{1}{\sqrt{\sum_{i=1}^{n}\mathbb{E}\left[Z_{i}^{2}\right]}}\sum_{i=1}^{n}Z_{i}\leq
t\right)\geq\Phi\left(t\right)-\kappa\left[\frac{n\log
n}{\left(\sum_{i=1}^{n}\mathbb{E}_{\gamma}\left[Z_{i}^{2}\right]\right)^{\frac{3}{2}}}+\Bigg{|}\Bigg{|}\frac{\sum_{i=1}^{n}\mathbb{E}_{\gamma}[Z_{i}^{2}|\mathcal{F}_{i-1}]}{\sum_{i=1}^{n}\mathbb{E}_{\gamma}[Z_{i}^{2}]}-1\Bigg{|}\Bigg{|}_{\infty}^{1/2}\right].$
(15)

Using Lemma 3 in $(\ref{conv})$, we obtain

$\displaystyle
P_{\gamma}\left(\frac{1}{\sqrt{\sum_{i=1}^{n}\mathbb{E}\left[Z_{i}^{2}\right]}}\sum_{i=1}^{n}Z_{i}\leq
t\right)$ $\displaystyle\geq\Phi\left(t\right)-\kappa\left[\frac{n\log
n}{\left(nV(\Gamma)-n|\mathcal{A}|\gamma\nu_{\max}\right)^{\frac{3}{2}}}+\left(\frac{4|\mathcal{A}|\gamma\nu_{\max}}{V(\Gamma)}\right)^{1/2}\right]$
$\displaystyle\geq\Phi(t)-\beta_{\gamma},$ (16)

where the last inequality holds for sufficiently large $n$ for some constant
$\beta_{\gamma}>0$ which can be chosen such that $\beta_{\gamma}\to 0$ as
$\gamma\to 0$.

###### Lemma 3

We have

$\displaystyle
V(\Gamma)-2|\mathcal{A}|\gamma\nu_{\max}\leq\frac{1}{n}\sum_{i=1}^{n}\mathbb{E}_{\gamma}\left[Z_{i}^{2}\right]\leq
V(\Gamma)+2|\mathcal{A}|\gamma\nu_{\max}.$

Furthermore, for $\gamma\leq\frac{V(\Gamma)}{4|\mathcal{A}|\nu_{\max}}$,

$\displaystyle\Bigg{|}\Bigg{|}\frac{\sum_{i=1}^{n}\mathbb{E}_{\gamma}[Z_{i}^{2}|\mathcal{F}_{i-1}]}{\sum_{i=1}^{n}\mathbb{E}_{\gamma}[Z_{i}^{2}]}-1\Bigg{|}\Bigg{|}_{\infty}\leq\frac{8|\mathcal{A}|\gamma\nu_{\max}}{V(\Gamma)}$

almost surely according to the probability measure $P_{\gamma}$.

Proof: The proof of Lemma 3 is given in Appendix A.

Using the result in $(\ref{finres})$ and Lemma 3 in the expression in
$(\ref{MCLT})$, we obtain

$\displaystyle P_{\gamma}\left(\sum_{i=1}^{n}Z_{i}>\sqrt{n}r-\log(2)\right)$
$\displaystyle=P_{\gamma}\left(\frac{1}{\sqrt{\sum_{i=1}^{n}\mathbb{E}_{\gamma}\left[Z_{i}^{2}\right]}}\sum_{i=1}^{n}Z_{i}>\frac{\sqrt{n}r-\log(2)}{\sqrt{\sum_{i=1}^{n}\mathbb{E}_{\gamma}\left[Z_{i}^{2}\right]}}\right)$
$\displaystyle=1-P_{\gamma}\left(\frac{1}{\sqrt{\sum_{i=1}^{n}\mathbb{E}_{\gamma}\left[Z_{i}^{2}\right]}}\sum_{i=1}^{n}Z_{i}\leq\frac{\sqrt{n}r-\log(2)}{\sqrt{\sum_{i=1}^{n}\mathbb{E}_{\gamma}\left[Z_{i}^{2}\right]}}\right)$
$\displaystyle\leq\begin{cases}1-P_{\gamma}\left(\frac{1}{\sqrt{\sum_{i=1}^{n}\mathbb{E}_{\gamma}\left[Z_{i}^{2}\right]}}\sum_{i=1}^{n}Z_{i}\leq\frac{r-\frac{\log(2)}{\sqrt{n}}}{\sqrt{V(\Gamma)+|\mathcal{A}|\gamma\nu_{\max}}}\right)&\text{
if }r\geq\frac{\log 2}{\sqrt{n}}\\\
1-P_{\gamma}\left(\frac{1}{\sqrt{\sum_{i=1}^{n}\mathbb{E}_{\gamma}\left[Z_{i}^{2}\right]}}\sum_{i=1}^{n}Z_{i}\leq\frac{r-\frac{\log(2)}{\sqrt{n}}}{\sqrt{V(\Gamma)-|\mathcal{A}|\gamma\nu_{\max}}}\right)&\text{
if }r<\frac{\log 2}{\sqrt{n}}\end{cases}$
$\displaystyle\leq\begin{cases}1-\Phi\left(\frac{r-\frac{\log(2)}{\sqrt{n}}}{\sqrt{V(\Gamma)+|\mathcal{A}|\gamma\nu_{\max}}}\right)+\beta_{\gamma}&\text{
if }r\geq\frac{\log 2}{\sqrt{n}}\\\
1-\Phi\left(\frac{r-\frac{\log(2)}{\sqrt{n}}}{\sqrt{V(\Gamma)-|\mathcal{A}|\gamma\nu_{\max}}}\right)+\beta_{\gamma}&\text{
if }r<\frac{\log 2}{\sqrt{n}}.\end{cases}$ (17)

Let

$\displaystyle
r=\begin{cases}\sqrt{V(\Gamma)+|\mathcal{A}|\gamma\nu_{\max}}\,\Phi^{-1}\left(\epsilon+3\beta_{\gamma}\right)+\frac{\log
2}{\sqrt{n}}&\text{ if
}\epsilon\in\left[\frac{1}{2}-3\beta_{\gamma},1\right)\\\
\sqrt{V(\Gamma)-|\mathcal{A}|\gamma\nu_{\max}}\,\Phi^{-1}\left(\epsilon+3\beta_{\gamma}\right)+\frac{\log
2}{\sqrt{n}}&\text{ if
}\epsilon\in\left(0,\frac{1}{2}-3\beta_{\gamma}\right).\end{cases}$ (18)

Note that the upper bound in $(\ref{userhere})$ holds for any $r$. For a given
error probability $\epsilon\in(0,1)$, we choose $r$ according to
$(\ref{myrval})$. Then using $(\ref{myrval})$ in $(\ref{userhere})$, we obtain
for any given $\epsilon\in(0,1)$ that

$\displaystyle P_{\gamma}\left(\sum_{i=1}^{n}Z_{i}>\sqrt{n}r-\log(2)\right)$
$\displaystyle\leq 1-\epsilon-2\beta_{\gamma}.$ (19)

$(\ref{firstterm})$ provides an upper bound to the first term in $(\ref{q})$.

We now upper bound the second term in $(\ref{q})$.

Using again the choice of $q$ in $(\ref{choiceq})$, we have

$\displaystyle\sum_{t:T^{n}_{\mathcal{A}}(t)\subset\mathcal{P}_{n}^{\gamma}}P_{t}\left(\frac{W(Y^{n}|X^{n})}{q(Y^{n})}>\rho\right)$
$\displaystyle\leq\sum_{t:T^{n}_{\mathcal{A}}(t)\subset\mathcal{P}_{n}^{\gamma}}P_{t}\left(\frac{W(Y^{n}|X^{n})}{\prod_{i=1}^{n}q_{t}(y_{i})}>\frac{\rho}{2|\mathcal{P}_{n}(\mathcal{A})|}\right)$
$\displaystyle\leq\sum_{t:T^{n}_{\mathcal{A}}(t)\subset\mathcal{P}_{n}^{\gamma}}P_{t}\left(\sum_{i=1}^{n}\log\frac{W(Y_{i}|X_{i})}{q_{t}(Y_{i})}>nC(\Gamma)+\sqrt{n}r-\log
2(n+1)^{|\mathcal{A}|}\right)$
$\displaystyle\stackrel{{\scriptstyle(a)}}{{=}}\sum_{t:T^{n}_{\mathcal{A}}(t)\subset\mathcal{P}_{n}^{\gamma}}W\left(\sum_{i=1}^{n}\log\frac{W(Y_{i}|x_{t,i})}{q_{t}(Y_{i})}>nC(\Gamma)+\sqrt{n}r-\log
2(n+1)^{|\mathcal{A}|}\right),$ (20)

where in equality $(a)$, $(x_{t,1},\ldots,x_{t,n})$ is any arbitrary sequence
from the type class $T^{n}_{\mathcal{A}}(t)$. Equality $(a)$ holds because
under the probability measure $P_{t}$, $t(X^{n})=t$ a.s. (see Remark 2) and
the distribution of

$\sum_{i=1}^{n}\log\frac{W(Y_{i}|X_{i})}{q_{t}(Y_{i})}$

depends on $X^{n}$ only through its type. Continuing from $(\ref{typeetype})$,
we have

$\displaystyle\sum_{t:T^{n}_{\mathcal{A}}(t)\subset\mathcal{P}_{n}^{\gamma}}P_{t}\left(\frac{W(Y^{n}|X^{n})}{q(Y^{n})}>\rho\right)$
$\displaystyle\leq\sum_{t:T^{n}_{\mathcal{A}}(t)\subset\mathcal{P}_{n}^{\gamma}}W\left(\sum_{i=1}^{n}\left[\log\frac{W(Y_{i}|x_{t,i})}{q_{t}(Y_{i})}-\mathbb{E}_{W}\left[\log\frac{W(Y|x_{t,i})}{q_{t}(Y)}\right]\right]>\right.$
$\displaystyle\quad\quad\quad\quad\quad\quad\quad
n\left[C(\Gamma)-I(t,W)\right]+\sqrt{n}r-\log 2(n+1)^{|\mathcal{A}|}\Bigg{)}$
$\displaystyle\leq\sum_{t:T^{n}_{\mathcal{A}}(t)\subset\mathcal{P}_{n}^{\gamma}}W\left(\sum_{i=1}^{n}\left[\log\frac{W(Y_{i}|x_{t,i})}{q_{t}(Y_{i})}-\mathbb{E}_{W}\left[\log\frac{W(Y|x_{t,i})}{q_{t}(Y)}\right]\right]>n\frac{K}{2}\right)$
(21)

where the last inequality holds for sufficiently large $n$ because $r$, as
defined in $(\ref{myrval})$, is an $O(1)$ term, and from the construction of
the set $\mathcal{P}_{n}^{\gamma}$, we have

$\inf_{t:T^{n}_{\mathcal{A}}(t)\subset\mathcal{P}_{n}^{\gamma}}\,d_{W}(t)\geq\gamma>0$

which implies

$\inf_{t:T^{n}_{\mathcal{A}}(t)\subset\mathcal{P}_{n}^{\gamma}}\left[C(\Gamma)-I(t,W)\right]>K$

for some constant $K>0$.

Let
$i_{\max,t}:=\max_{a,b:q_{t}(b)W(b|a)>0}\big{|}\log\frac{W(b|a)}{q_{t}(b)}\big{|}$.
We now show that $i_{\max,t}\leq 2\log n$ for all $t$. Let
$W_{\min}:=\min_{a,b:W(b|a)>0}W(b|a)$ and
$q_{\min,t}:=\min_{b:q_{t}(b)>0}q_{t}(b)$. Then

$\displaystyle q_{\min,t}$
$\displaystyle=\min_{b:q_{t}(b)>0}\sum_{a\in\mathcal{A}}t(a)W(b|a)$
$\displaystyle\geq\min_{a,b:W(b|a)>0}W(b|a)\min_{a:t(a)>0}t(a)$
$\displaystyle=\frac{W_{\min}}{n}.$

Thus,

$\displaystyle i_{\max,t}$
$\displaystyle=\max_{a,b:q_{t}(b)W(b|a)>0}\big{|}\log\frac{W(b|a)}{q_{t}(b)}\big{|}$
$\displaystyle\leq\max_{a,b:q_{t}(b)W(b|a)>0}\big{|}\log
W(b|a)\big{|}+\max_{b:q_{t}(b)>0}\big{|}\log q_{t}(b)\big{|}$
$\displaystyle\leq\log\frac{n}{W_{\min}^{2}}$ $\displaystyle\leq 2\log n$

for all sufficiently large $n$. Hence, we can use Azuma’s inequality [13,
(33), p. 61] to upper bound $(\ref{recycle})$, giving us

$\displaystyle\sum_{t:T^{n}_{\mathcal{A}}(t)\subset\mathcal{P}_{n}^{\gamma}}P_{t}\left(\frac{W(Y^{n}|X^{n})}{q(Y^{n})}>\rho\right)$
$\displaystyle\leq(n+1)^{|\mathcal{A}|}\exp\left(-\frac{nK^{2}}{128\log^{2}n}\right)$
(22)

which goes to zero as $n\to\infty$.

Substituting the upper bounds $(\ref{firstterm})$ and $(\ref{pn4})$ in
$(\ref{q})$, we obtain

$\displaystyle(F\circ
W)\left(\frac{W(Y^{n}|X^{n})}{q(Y^{n})}>\exp\left(nC(\Gamma)+\sqrt{n}r\right)\right)$
$\displaystyle\leq 1-\epsilon-\beta_{\gamma}$

for sufficiently large $n$. Since the controller $F$ was arbitrary, we can
apply Lemma 1 to obtain

$\displaystyle\log M^{*}_{\text{fb}}(n,\epsilon,\Gamma)$ $\displaystyle\leq
nC(\Gamma)+\sqrt{n}r-\log\beta_{\gamma}$ $\displaystyle\frac{\log
M^{*}_{\text{fb}}(n,\epsilon,\Gamma)-nC(\Gamma)}{\sqrt{n}}$ $\displaystyle\leq
r-\frac{\log\beta_{\gamma}}{\sqrt{n}}$
$\displaystyle\limsup_{n\to\infty}\frac{\log
M^{*}_{\text{fb}}(n,\epsilon,\Gamma)-nC(\Gamma)}{\sqrt{n}}$ $\displaystyle\leq
r^{\prime},$

where $r^{\prime}$ is obtained from the expression of $r$ in $(\ref{myrval})$
after taking the limit as $n\to\infty$, i.e.,

$\displaystyle
r^{\prime}=\begin{cases}\sqrt{V(\Gamma)+|\mathcal{A}|\gamma\nu_{\max}}\,\Phi^{-1}\left(\epsilon+3\beta_{\gamma}\right)&\text{
if }\epsilon\in\left[\frac{1}{2}-3\beta_{\gamma},1\right)\\\
\sqrt{V(\Gamma)-|\mathcal{A}|\gamma\nu_{\max}}\,\Phi^{-1}\left(\epsilon+3\beta_{\gamma}\right)&\text{
if }\epsilon\in\left(0,\frac{1}{2}-3\beta_{\gamma}\right).\end{cases}$

Finally, since $\frac{V(\Gamma)}{4|\mathcal{A}|\nu_{\max}}>\gamma>0$ was
arbitrary, we can take $\gamma$ and $\beta_{\gamma}$ arbitrarily small, giving
us the converse result

$\displaystyle\limsup_{n\to\infty}\frac{\log
M^{*}_{\text{fb}}(n,\epsilon,\Gamma)-nC(\Gamma)}{\sqrt{n}}\leq\sqrt{V(\Gamma)}\Phi^{-1}(\epsilon).$

Since this matches the optimal non-feedback SOCR of simple dispersion DMCs
with a peak-power cost constraint, we have

$\displaystyle\lim_{n\to\infty}\frac{\log
M^{*}_{\text{fb}}(n,\epsilon,\Gamma)-nC(\Gamma)}{\sqrt{n}}=\sqrt{V(\Gamma)}\Phi^{-1}(\epsilon)$
(23)

for simple-dispersion DMCs with a peak-power cost constraint.

##  Appendix A Proof of Lemma 3

We have

$\displaystyle\sum_{i=1}^{n}\mathbb{E}_{\gamma}\left[Z_{i}^{2}|\mathcal{G}_{i-1}\right]$
$\displaystyle=\sum_{i=1}^{n}\mathbb{E}_{\gamma}\left[\left(i(X_{i},Y_{i})-\mathbb{E}\left[i(X_{i},Y_{i})|X_{i}\right]\right)^{2}|\mathcal{G}_{i-1}\right]$
$\displaystyle=\sum_{i=1}^{n}\mathbb{E}_{\gamma}\left[\left(i(X_{i},Y_{i})-\mathbb{E}\left[i(X_{i},Y_{i})|X_{i}\right]\right)^{2}|X_{i}\right]$
$\displaystyle=\sum_{i=1}^{n}\text{Var}\left(i(X_{i},Y_{i})|X_{i}\right)$
$\displaystyle=\sum_{i=1}^{n}\sum_{a\in\mathcal{A}}\mathds{1}\left(X_{i}=a\right)\nu_{a}$
$\displaystyle=n\sum_{a\in\mathcal{A}}P_{X^{n}}(a)\nu_{a}.$

From Remark 1, since $d_{W}(t(X^{n}))\leq\gamma$ a.s., there exists a
$\tilde{P}\in\Pi_{W}^{*}$ such that $||t(X^{n})-\tilde{P}||_{1}\leq 2\gamma$.
Hence,

$\displaystyle n\sum_{a\in\mathcal{A}}(\tilde{P}(a)-2\gamma)\nu_{a}$
$\displaystyle\leq\sum_{i=1}^{n}\mathbb{E}_{\gamma}\left[Z_{i}^{2}|\mathcal{G}_{i-1}\right]\leq
n\sum_{a\in\mathcal{A}}(\tilde{P}(a)+2\gamma)\nu_{a}$ $\displaystyle
nV(\Gamma)-2n|\mathcal{A}|\gamma\nu_{\max}$
$\displaystyle\leq\sum_{i=1}^{n}\mathbb{E}_{\gamma}\left[Z_{i}^{2}|\mathcal{G}_{i-1}\right]\leq
nV(\Gamma)+2n|\mathcal{A}|\gamma\nu_{\max}$

a.s., where the last step above invokes the fact that $W$ is simple-
dispersion. Furthermore, since $\mathcal{F}_{i-1}\subset\mathcal{G}_{i-1}$,

$\displaystyle nV(\Gamma)-2n|\mathcal{A}|\gamma\nu_{\max}$
$\displaystyle\leq\sum_{i=1}^{n}\mathbb{E}_{\gamma}\left[Z_{i}^{2}|\mathcal{F}_{i-1}\right]\leq
nV(\Gamma)+2n|\mathcal{A}|\gamma\nu_{\max}$

a.s. and

$\displaystyle nV(\Gamma)-2n|\mathcal{A}|\gamma\nu_{\max}$
$\displaystyle\leq\sum_{i=1}^{n}\mathbb{E}_{\gamma}\left[Z_{i}^{2}\right]\leq
nV(\Gamma)+2n|\mathcal{A}|\gamma\nu_{\max}.$

Finally, we have

$\displaystyle\Bigg{|}\frac{\sum_{i=1}^{n}\mathbb{E}_{\gamma}[Z_{i}^{2}|\mathcal{F}_{i-1}]}{\sum_{i=1}^{n}\mathbb{E}_{\gamma}[Z_{i}^{2}]}-1\Bigg{|}$
$\displaystyle\leq\Bigg{|}\frac{V(\Gamma)+2|\mathcal{A}|\gamma\nu_{\max}}{V(\Gamma)-2|\mathcal{A}|\gamma\nu_{\max}}-1\Bigg{|}$
$\displaystyle=\Bigg{|}\frac{4|\mathcal{A}|\gamma\nu_{\max}}{V(\Gamma)-2|\mathcal{A}|\gamma\nu_{\max}}\Bigg{|}$
$\displaystyle\leq\frac{8|\mathcal{A}|\gamma\nu_{\max}}{V(\Gamma)},$

assuming $\gamma\leq\frac{V(\Gamma)}{4|\mathcal{A}|\nu_{\max}}$.

## Acknowledgment

This research was supported by the US National Science Foundation under grant
CCF-1956192.

## References

  * [1] M. Hayashi, “Information spectrum approach to second-order coding rate in channel coding,” _IEEE Transactions on Information Theory_ , vol. 55, no. 11, pp. 4947–4966, 2009.
  * [2] V. Kostina and S. Verdú, “Channels with cost constraints: Strong converse and dispersion,” _IEEE Transactions on Information Theory_ , vol. 61, no. 5, pp. 2415–2429, 2015.
  * [3] A. Mahmood and A. B. Wagner, “Timid/bold coding for channels with cost constraints,” in _2023 IEEE International Symposium on Information Theory (ISIT)_ , 2023, pp. 1442–1447.
  * [4] ——, “Channel coding with mean and variance cost constraints,” 2024. [Online]. Available: arXiv:2401.16417
  * [5] Y. Polyanskiy, “Channel coding: Non-asymptotic fundamental limits,” Ph.D. dissertation, Dept. Elect. Eng., Princeton Univ., Princeton, NJ, USA, 2010.
  * [6] A. Sahai, S. Draper, and M. Gastpar, “Boosting reliability over AWGN networks with average power constraints and noiseless feedback,” in _Proceedings. International Symposium on Information Theory, 2005. ISIT 2005._ , 2005, pp. 402–406.
  * [7] I. Csiszár and J. Körner, _Information Theory: Coding Theorems for Discrete Memoryless Systems_ , 2nd ed. Cambridge University Press, 2011.
  * [8] R. G. Gallager, _Information Theory and Reliable Communication_. New York, NY, USA: Wiley, 1968.
  * [9] Y. Polyanskiy, H. V. Poor, and S. Verdu, “Channel coding rate in the finite blocklength regime,” _IEEE Transactions on Information Theory_ , vol. 56, no. 5, pp. 2307–2359, 2010.
  * [10] S. L. Fong and V. Y. F. Tan, “A tight upper bound on the second-order coding rate of the parallel Gaussian channel with feedback,” _IEEE Transactions on Information Theory_ , vol. 63, no. 10, pp. 6474–6486, 2017.
  * [11] A. B. Wagner, N. V. Shende, and Y. Altuğ, “A new method for employing feedback to improve coding performance,” _IEEE Transactions on Information Theory_ , vol. 66, no. 11, pp. 6660–6681, 2020.
  * [12] E. Bolthausen, “Exact convergence rates in some martingale central limit theorems,” _The Annals of Probability_ , vol. 10, no. 3, pp. 672–688, Aug. 1982.
  * [13] B. Bercu, B. Delyon, and E. Rio, _Concentration Inequalities for Sums and Martingales_. Cham, Switzerland: Springer, 2015.