#  Configuration entropy and confinement/deconfinement transiton in
holographic QCD

Nelson R. F. Braga, O. C. Junqueira
nrfbraga<EMAIL_ADDRESS>Universidade Federal do Rio de
Janeiro—Instituto de Física, Caixa Postal 68528, Rio de Janeiro, Brazil

###### Abstract

In the holographic AdS/QCD approach, the confinement/deconfinement transition
is associated with the Hawking-Page transition of a thermal anti-de Sitter
(AdS) space to an AdS black hole. In the case of the hard wall model, the
thermal transition takes place in the planar AdS thanks to the introduction of
an infrared cut-off in the geometry. The corresponding thermodynamic entropy
of the $SU(N)$ gauge theory jumps from proportional to $N^{0}$ in the confined
hadronic phase to proportional to $N^{2}$ in the plasma phase, corresponding
to the presence of the color degrees of freedom. The Hawking-Page transition
is understood by considering a semiclassical picture of a system consisting of
two different geometries that are asymptotically AdS. One is the AdS black
hole and the other the thermal AdS space. The relative stability between these
competing geometries varies with the temperature. So, the transition is
essentially a problem of stability. An interesting tool to study stability of
physical systems is the configuration entropy (CE), inspired in the Shannon
informational entropy. In this work we investigate the CE for the case of the
AdS/QCD hard wall model at finite temperature. We propose a regularized form
for the energy densities of the black hole (BH) and of the thermal AdS
geometries that makes it possible to calculate their CEs as a function of the
temperature. We find a relation between stability and the value of the CE for
the system of asymptotically AdS geometries. Remarkably, it is found that the
CE is proportional to $\log(T)$, where $T$ is the temperature. This result
makes it possible to write out a simple relation between the configuration and
the thermodynamic entropies.

##  I Introduction

Soon after the AdS/CFT correspondence was proposed Maldacena:1997re ;
Gubser:1998bc ; Witten:1998qj , it was pointed out by Witten Witten:1998zw
that one could associate the Hawking-Page (HP) transition Hawking:1982dh , in
the gravitational side of the correspondence, with the
confinement/deconfinement transition in the gauge theory side. The HP
transition emerges from a semiclassical analysis of a system consisting of
geometries that have constant negative curvature and are asymptotically anti-
de Sitter on the boundary. The amplitudes for the prevalence of each of the
two “competing” geometries: anti-de Sitter black hole (AdS-BH) and Thermal AdS
(ThAdS), depends on $\exp(-S)$, where $S$ is the Euclidean action integral.
For the case of spherical boundary/spherical horizon there is a finite
temperature $T_{c}$ where the action integrals of AdS-BH and ThAdS spaces
coincide. For larger temperatures the black hole action is smaller and thus
this geometry dominates or, in other words, is stable. For $T<T_{c}$ the
thermal AdS is the stable geometry.

For geometries with a planar boundary there is no HP transition because the
action integral of the black hole is smaller than the one of the ThAdS, except
at zero temperature, when the geometries coincide Witten:1998zw . On the other
hand, it is possible to find the HP transition in asymptotically AdS spaces
having a planar boundary, that are relevant for studying QCD-like gauge
theories, if one follows a phenomenological approach to gauge gravity duality.
In the so-called AdS/QCD bottom up framework, one gives up satisfying Einstein
equations and introduces a modification in the geometry, corresponding to
adding some energy parameter of the gauge theory. In particular, the hard wall
model Polchinski:2001tt ; BoschiFilho:2002ta ; BoschiFilho:2002vd , makes it
possible to calculate hadronic masses by assuming an approximate duality
between gravity in AdS space with a hard infrared cutoff in the geometry and a
non conformal gauge theory on the planar boundary. It was shown in
Herzog:2006ra ; BallonBayona:2007vp  that the hard wall model presents a HP
transition at some finite critical temperature $T_{c}$. Interestingly, the
thermodynamic entropy for the gauge theory is proportional to $N^{0}$ for
$T<T_{c}$ and to $N^{2}T^{3}$ for $T>T_{C}$, with $N$ the color group index,
showing an increase of degrees of freedom, associated with deconfinement of
color.

In summary, the AdS/QCD hard wall model represents the
confinement/deconfinement transition of a gauge theory in terms of a change in
the stability condition of a black hole geometry. On the other hand, in the
recent years, it was show that the configuration entropy (CE) Gleiser:2011di ;
Gleiser:2012tu ; Gleiser:2013mga  plays an important role in the analysis of
stability of many different physical systems, as for example in
Bernardini:2016qit ; Braga:2017fsb ; Braga:2018fyc ; Bernardini:2018uuy ;
Ferreira:2019inu ; Braga:2019jqg ; Correa:2015vka ; Braga:2016wzx ;
Karapetyan:2016fai ; Karapetyan:2017edu ; Karapetyan:2018oye ;
Karapetyan:2018yhm ; Lee:2018zmp ; Bazeia:2018uyg ; Ma:2018wtw ; Zhao:2019xle
; Ferreira:2019nkz ; Braga:2019jqg ; Bernardini:2019stn ; Karapetyan:2019ran ;
Braga:2020myi ; Karapetyan:2020yhs ; Ferreira:2020iry ; Alves:2020cmr ;
MarinhoRodrigues:2020ssq . Comparing the CE of the states of many different
physical systems, it was observed that, the smaller is the value of the CE,
the more stable is the state. The purpose of the present work is to understand
if the same behavior appears in the case of the transition from confining to
deconfining geometries in the AdS/QCD hard wall model. Following this idea, we
will develop an approach for calculating the configuration entropy for the
AdS-BH and ThAdS spaces and analyse the dependence of the CE on the
temperature and the corresponding stability of the associated states.

This work is organized as follows. In section (2) we review the hard wall
model at finite temperature and explain how does the Hawking-Page transition
takes place. In section (3) we develop an approach for expressing the energy
density of the black hole in a regularized way, that is essential for the
calculation of the CE in the black hole geometry. Section (4) is devoted to
the calculation of the CE and to the analysis of the results. A study of the
relation between the thermodynamic and the configuration entropies is
presented in section (5), where a simple relation between these quantities is
found. In section (6) we explore the possibility of a relation between the
results obtained in this letter and the so-called Gubser-Mitra conjecture. Our
final conclusions are in section (7) and some technical details are shown in
the appendices.

##  II Holographic model for confinement/deconfinement: Overview

###  II.1 Hard wall AdS/QCD model

One considers the five-dimensional Einstein-Hilbert action with a negative
cosmological constant and looks for geometries with boundary $R^{3}\times
S^{1}$ Herzog:2006ra ; BallonBayona:2007vp .There are two solutions. The first
one is thermal AdS (ThAdS), described in the Euclidean case by

$ds^{2}=\frac{L^{2}}{z^{2}}\left(dt^{2}+d\overrightarrow{x}^{2}+dz^{2}\right)\;,$
(1)

where $L$ is AdS space radius. The second solution is the AdS black hole ( BH-
AdS)

$ds^{2}=\frac{L^{2}}{z^{2}}\left(f(z)dt^{2}+d\overrightarrow{x}^{2}+\frac{dz^{2}}{f(z)}\right)\;,$
(2)

with $f(z)=1-z^{4}/z_{h}^{4}$, being $z_{h}$ the black hole horizon. In both
cases, one considers a compact time coordinate. For the black hole, the time
has a period $\beta$ and the temperature is $T=1/\beta=1/(\pi z_{h})$, to
avoid a conical singularity of the metric on the horizon Hawking:1982dh .

The on-shell action for both geometries is:

$I_{\text{on-shell}}=\frac{4}{L^{2}\kappa^{2}}\int d^{5}x\sqrt{g}\;.$ (3)

where $\kappa$ is the gravitational coupling. The hard wall model
Polchinski:2001tt ; BoschiFilho:2002ta ; BoschiFilho:2002vd  consists of
introducing a cut-off in the geometry in the form of a maximum value for the
coordinate $z$: $\,0\leq z\leq z_{0}$. The inverse of $z_{0}$ is interpreted
as an infrared energy cut-off in the gauge theory side.

The action integral of eq. (3) is singular at $z\to 0$ for both the ThAdS and
the BH-AdS spaces. Defining an action density $\mathcal{E}=I/V$, being $V$ the
trivial spacial volume $\int d^{3}x$ over the components $\overrightarrow{x}$,
the regularized action density $\mathcal{E}(\epsilon)$ for the ThAdS is
defined as

$\mathcal{E}_{AdS}(\epsilon)=\frac{4L^{3}}{\kappa^{2}}\int_{0}^{\beta^{\prime}}dt\int_{\epsilon}^{z_{0}}dz\,z^{-5}=\frac{L^{3}}{\kappa^{2}}\beta^{\prime}\left(\frac{1}{\epsilon^{4}}-\frac{1}{z_{0}^{4}}\right)\;,$
(4)

where $\epsilon$ is an ultraviolet regulator. Analogously, for the BH-AdS one
has

$\mathcal{E}_{BH}(\epsilon)=\frac{4L^{3}}{\kappa^{2}}\int_{0}^{\pi
z_{h}}dt\int_{\epsilon}^{\text{min}(z_{0},z_{h})}dz\,z^{-5}=\frac{L^{3}}{\kappa^{2}}\beta\left(\frac{1}{\epsilon^{4}}-\frac{1}{\bar{z}^{4}}\right)\;,$
(5)

with $\bar{z}\equiv\text{min}(z_{0},z_{h})$. Requiring that the two geometries
have the same asymptotic form at $z=\epsilon\to 0$, so that the time
periodicity in the limit $\epsilon\rightarrow 0$ must be the same, one finds
the condition $\beta^{\prime}=\pi z_{h}\sqrt{f(\epsilon)}$ Herzog:2006ra .
Using this expression for $\beta^{\prime}$, the difference between the actions
densities, defined as

$\bigtriangleup\mathcal{E}=\lim_{\varepsilon\rightarrow
0}{\left[\mathcal{E}_{BH}(\varepsilon)-\mathcal{E}_{AdS}(\varepsilon)\right]}\;,$
(6)

is independent of $\epsilon$

$\bigtriangleup\mathcal{E}=\begin{cases}\frac{L^{3}\pi
z_{h}}{\kappa^{2}}\frac{1}{2z_{h}^{4}}\;,&\mbox{if }z_{0}<z_{h}\;,\\\
\frac{L^{3}\pi
z_{h}}{\kappa^{2}}\left(\frac{1}{z_{0}^{4}}-\frac{1}{2z_{h}^{4}}\right)\;,&\mbox{if
}z_{h}<z_{0}\;.\end{cases}$ (7)

One notices that the actions are equal at the critical temperature:

$\displaystyle T_{c}=2^{1/4}/(\pi z_{0})\;.$ (8)

This corresponds to the Hawking-Page transition temperature. If $T<T_{c}$, the
thermal AdS state dominates as $\bigtriangleup\mathcal{E}>0$, and the black
hole is unstable. When $T>T_{c}$, $\,\,\bigtriangleup\mathcal{E}$ is negative,
and the black hole becomes the stable geometry. This is the holographic
description of confinement/deconfinement transition developed in Herzog:2006ra
. We will see now that it is possible to obtain finite actions in the
$\epsilon\to 0$ limit for each space using holographic renormalization.

###  II.2 Holographic renormalization and thermodynamics

In this procedure Balasubramanian:1999re ; Emparan:1999pm , the ultraviolet
divergences are removed by adding to the action a surface counterterm. In ref.
BallonBayona:2007vp , it was pointed out that besides the usual volumetric
action of eq. (3), in order to describe the planar AdS geometries considered
here, one needs also to include in the action the Gibbons-Hawking surface
term. This type of boundary term comes from the variational principle for a
gravity theory with a boundary Gibbons:1976ue . In the present case it reads

$\displaystyle I_{GH}=-\frac{1}{\kappa^{2}}\int_{\partial
M}d^{4}x\sqrt{h}\,K\;,$ (9)

where $K$ is the trace of the extrinsic curvature of the boundary. The details
of the calculation of the Gibbons-Hawking surface term can be found in
BallonBayona:2007vp . In short, at a boundary hypersurface,
$K=\frac{1}{\sqrt{g}}\partial_{a}(\sqrt{g}\,n^{a})\,,$ where $n^{a}$ is a
unitary vector normal to the boundary, see Arutyunov:1998ve . One gets the
following surface terms, divided by the trivial spacial volume factor:

$\displaystyle\mathcal{E}^{GH}_{AdS}$ $\displaystyle=$
$\displaystyle-\frac{4L^{3}}{\kappa^{2}}\frac{\beta^{\prime}}{\epsilon^{4}}\;,$
(10) $\displaystyle\mathcal{E}^{GH}_{BH}$ $\displaystyle=$
$\displaystyle-\frac{4L^{3}}{\kappa^{2}}\beta\left(\frac{1}{\epsilon^{4}}-\frac{1}{2z_{h}^{4}}\right)\;,$
(11)

Adding these surface terms to the corresponding volumetric ones, given in eqs.
(4) and (5) one finds it out that in order to cancel out the total
divergencies one should take the counterterm action, for both geometries, as

$I_{ct}=\frac{1}{\kappa^{2}}\int_{\partial M}d^{4}x\sqrt{h}\,\frac{3}{L}\;,$
(12)

where $h$ the determinant of the boundary induced metric $h_{\mu\nu}$.The
holographic renormalization procedure is concluded by defining total actions
for the geometries as $I_{total}=I+I_{GH}+I_{ct}$. The corresponding action
densities are given by

$\displaystyle\mathcal{E}^{total}_{AdS}$ $\displaystyle=$
$\displaystyle-\frac{L^{3}}{\kappa^{2}}\frac{\beta^{\prime}}{z_{0}^{4}}\;,$
(13) $\displaystyle\mathcal{E}^{total}_{BH}$ $\displaystyle=$
$\displaystyle-\frac{L^{3}}{\kappa^{2}}\beta\left(\frac{1}{\bar{z}^{4}}-\frac{1}{2z_{h}^{4}}\right)\;.$
(14)

So, using the holographic renormalization procedure, it is possible to define
finite actions for both ThAdS and BH-AdS, not only for their difference.

From the finite actions (13) and (14) one can determine the thermodynamic
entropy for each geometrical phase, and use it to characterize the
confinement/deconfinement phase transition. In the saddle point approximation,
using $\kappa^{2}=8\pi G_{5}$, and the relation between $G_{5}$ and the
fundamental string scale, see BallonBayona:2007vp  for details, the associated
thermodynamic entropies from the expression

$S=\beta\langle E\rangle+\log Z\approx\beta\langle E\rangle-I_{total}\;,$ (15)

where $\langle E\rangle=-\frac{\partial}{\partial\beta}\log
Z\approx\frac{\partial}{\partial\beta}I_{total}$ is the expectation value of
the energy, yields

$\displaystyle S_{AdS}$ $\displaystyle=$ $\displaystyle
0\;,\quad\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\text{if}\quad T<T_{c}\;,$
(16) $\displaystyle S_{BH}$ $\displaystyle=$
$\displaystyle\frac{N^{2}\pi^{2}}{2}T^{3}\;,\quad\,\text{if}\quad T>T_{c}\;.$
(17)

Where there is a jump from $N^{0}$ to $N^{2}$ dependence of the entropy,
representing the change from the confined phase to the deconfined one
BallonBayona:2007vp , when the color degrees of freedom are free. Such a
result is consistent with the free energy of the $\mathcal{N}=4$ super Yang-
Mills theory at the strong coupling limit Gubser:1996de ; Gubser:1998nz , and
shows that one can use the thermodynamic entropy to identify the
confinement/deconfinement phase transition.

##  III  Thermal AdS and BH AdS masses and regularized densities

In order to compute the configurational entropy (CE) from the finite actions
(13) and (14), one needs to calculate the associated energy (or mass)
densities. For both cases, the mass can be determined from the energy
expression Hawking:1982dh ; Witten:1998zw

$E=\frac{\partial I}{\partial\beta}=M\;,$ (18)

in natural units. The results obtained in the last section, in particular (7)
shows that the black hole is stable in the region $z_{h}\leq 2^{-1/4}z_{0}$.
Thus, one can take $\bar{z}=\text{min}(z_{h},z_{0})$ as $z_{h}$ in expression
(14).

Hence, applying (18) to the regularized actions constructed from the
holographic renormalzation, one finds the masses of the spaces, in the range
of temperatures where they are stable:

$\displaystyle M_{AdS}$ $\displaystyle=$
$\displaystyle-\frac{L^{3}}{\kappa^{2}}\frac{1}{z_{0}^{4}}\;,\,\,\,\,\,\,\,\,\quad\text{for}\quad
z_{h}>2^{-1/4}z_{0}\;,$ (19) $\displaystyle M_{BH}$ $\displaystyle=$
$\displaystyle+\frac{3L^{3}}{2\pi^{4}\kappa^{2}}\frac{1}{z_{h}^{4}}\;,\quad\text{for
}\quad z_{h}<2^{-1/4}z_{0}\;,.$ (20)

This means that $M_{BH}\sim 1/z_{h}^{4}$, while $M_{AdS}\sim 1/z_{0}^{4}$. The
negative sign of the ThAdS mass is interpreted as a consequence of the
subtraction of the region $z>z_{o}$ in the hard wall model. If one takes the
limit $z_{0}\to\infty$ the mass goes to zero, corresponding to the mass of the
empty AdS space.

The CE is defined as a function of the energy density that should, in
principle, be related to the total mass by

$\displaystyle\int_{0}^{z_{f}}dz\,\rho(z)=M\;,\quad\text{with}\quad
z_{f}\equiv\\{z_{h},z_{0}\\}\;,$ (21)

with the two possible values of the upper integration limit corresponding,
respectively, to the BH and AdS cases.

It is reasonable to assume that the mass density should not dependent on the
infrared cut-off $z_{0}$ since this parameter is not part of the original AdS
geometry. It is just an energy scale introduced in order to make an effective
description of a QCD like theory. So, in order to find a mass with a
dependence on $1/z_{f}^{4}$, searching for a power series solution, one would
find the simple form

$\rho(z)=\frac{C}{z^{5}}\;.$ (22)

with the constant $C$ taking different values for the AdS and BH cases. The
problem one faces using such a density is that one gets the mass but also a
singular term coming from the $z\to 0$ limit of the integral (21). In order to
avoid such singularities, one needs to find out a consistent regularization
process. An interesting approach is to define regularized densities $\rho(z)$,
related to the total mass by

$\displaystyle\lim_{\epsilon\to
0}\int_{\epsilon}^{z_{f}}dz\,\rho(z)\sim\frac{1}{z_{f}^{4}}\;,\quad\text{with}\quad
z_{f}\equiv\\{z_{h},z_{0}\\}\;.$ (23)

As we shall see, such a regularization can be implemented in a similar way for
both geometries. We propose the following expression for the regularized black
hole mass density,

$\rho_{BH}(z)=-\lim_{\epsilon\rightarrow
0}\frac{6L^{3}}{\pi^{4}\kappa^{2}}\frac{1}{z^{5}}\cos(\frac{2\pi\epsilon^{4}}{z^{4}})\;.$
(24)

One can easily verify that for small $\epsilon$,

$\displaystyle\int_{\epsilon}^{z_{h}}\rho_{BH}(z)\,dz$ $\displaystyle=$
$\displaystyle\frac{3L^{3}}{2\pi^{4}\kappa^{2}}\frac{1}{2\pi\epsilon^{4}}\sin(\frac{2\pi\epsilon^{4}}{z_{h}^{4}})$
(25) $\displaystyle=$ $\displaystyle M_{BH}+\mathcal{O}(\epsilon^{8})\;.$

So, taking the $\epsilon\to 0$ limit after the spatial integration, one
obtains the black hole mass.

The regularized density for the thermal AdS mass (19) can be obtained from the
same expression constructed in the regularization of the black hole. The only
difference is the constant factor, namely,

$\rho_{AdS}(z)=+\lim_{\epsilon\rightarrow
0}\frac{4L^{3}}{\kappa^{2}}\frac{1}{z^{5}}\cos(\frac{2\pi\epsilon^{4}}{z^{4}})\;.$
(26)

Using this density in eq. (23) one finds the ThAdS mass. An important issue,
that will be addressed in the next sections is that our results must be
independent of $\epsilon$. Or, in other words, the limit $\epsilon\to 0$ must
be well defined.

##  IV Configurational entropy and Hawking-Page transition

The configuration entropy (CE) Gleiser:2011di ; Gleiser:2012tu ;
Gleiser:2013mga  is inspired in the Shannon information entropy shannon , that
is a measure of information content. For a variable that can take $N_{d}$
discrete possible values, with probabilities given by $p_{i}$, it is defined
by

$S_{\text{info}}=-\sum_{i=1}^{N_{d}}p_{i}\ln p_{i}\;.$ (27)

The configurational entropy for a physical system is defined as a continuous
version of (27). It is defined in terms of the modal fraction, which is
constructed upon of the Fourier transform of the energy (mass) density,
$\widetilde{\rho}(k)$, that describes the corresponding physical states. For
the black hole and thermal AdS states in the hard wall model, the energy
density is a function of the Poincaré coordinate $z$, according to (24) and
(26), so that

$\widetilde{\rho}(k)=\frac{1}{2\pi}\int dz\,\rho(z)e^{ikz}\;.$ (28)

The modal fraction is defined as

$f(k)=\frac{|\widetilde{\rho}(k)|^{2}}{\mathcal{N}}\;,$ (29)

where the normalization constant is defined as:

$\displaystyle\mathcal{N}=\int dk\,\langle|\widetilde{\rho}(k)|^{2}\rangle\;,$
(30)

The corresponding configurational entropy for such localized energy densities
is then defined as the functional Gleiser:2012tu

$S_{C}[f]=-\int dk\,f(k)\ln f(k)\;.$ (31)

In the Appendix B we show that $-f(k)\ln f(k)$ is always positive. From the
Fourier transforms of $\rho_{BH}(z)$ and $\rho_{AdS}(z)$,

$\displaystyle\widetilde{\rho}_{BH}(k)$ $\displaystyle=$
$\displaystyle\frac{1}{2\pi}\lim_{\epsilon\to
0}\int_{\epsilon}^{z_{h}}dz\,\rho_{BH}(z)e^{ikz}\;,$ (32)
$\displaystyle\widetilde{\rho}_{AdS}(k)$ $\displaystyle=$
$\displaystyle\frac{1}{2\pi}\lim_{\epsilon\to
0}\int_{\epsilon}^{z_{0}}dz\,\rho_{AdS}(z)e^{ikz}\;,$ (33)

one can compute the modal fractions for the thermal and black hole AdS spaces
using (29), and then obtain the configurational entropies as functions of the
temperature, above and bellow $T_{c}$. From this, one can finally study the
confinement/deconfinement phase transition from the point of view of
stability. Replacing (24) and (26) into (32) and (33), respectively, one finds
out that $\widetilde{\rho}_{BH}(k)$ and $\widetilde{\rho}_{AdS}(k)$ do not
possess analytical solutions. So, the CE is calculated using numerical
methods.

The numerical computation of the Fourier transform of the energy density in
eqs. (32) and (32) is performed with a finite value of the UV regulator
$\epsilon$. The limit $\epsilon\to 0$ is obtained, in the numerical approach,
by identifying the order of magnitude of vaues of $\epsilon$ for wich taking
smaller values would not change the results. We found it out that from
$\epsilon/z_{o}\sim 10^{-13}$ to smaller values there is no change in any of
the results of this work. So, this value of $\epsilon$ was used in our
computations. In the Fourier space, the squared absolute value of the black
hole energy density, that defines the normalization factor and the modal
fraction, can be written as

$|\widetilde{\rho}_{BH}(k)|^{2}=\left[\lim_{\epsilon\to
0}\frac{1}{2\pi}\int_{\epsilon}^{z_{h}}dz\,\rho_{BH}(z)\cos(kz)\right]^{2}+\left[\lim_{\epsilon\to
0}\frac{1}{2\pi}\int_{\epsilon}^{z_{h}}dz\,\rho_{BH}(z)\sin(kz)\right]^{2}\;.$
(34)

For the the thermal AdS, $|\widetilde{\rho}_{AdS}(k)|^{2}$ is similar, only
replacing $\rho_{BH}(z)\rightarrow\rho_{AdS}(z)$ in the expression above.

With all these information we are now able to compute the modal fractions for
each space, and finally determine the corresponding configurational entropies
in the confined and deconfined phases. The transition of geometries occurs at
$\frac{\beta_{c}}{\pi z_{0}}\,=\frac{1}{T_{c}\pi z_{0}}=0.840896$. Above
$T_{c}$ the space is AdS BH and below $T_{c}$ it is the ThAdS. We take
$z_{0}=1$ and show in table of TABLE. 1 the values of the BH CE for different
values of $\beta/\pi$, which is represented by the points in FIG. 1 for
$\beta/\pi<\beta_{c}/\pi$.

$\beta/\pi z_{0}$ | Black hole CE | $\beta/\pi z_{0}$ | Black hole CE
---|---|---|---
$\,\,\,0.025\,\,\,$ | $17.5229$ | $\,\,\,0.4\,\,\,$ | $14.7761$
$\,\,\,0.038\,\,\,$ | $17.1049$ | $\,\,\,0.45\,\,\,$ | $14.6326$
$\,\,\,0.05\,\,\,$ | $16.8300$ | $\,\,\,0.5\,\,\,$ | $14.5268$
$\,\,\,0.075\,\,\,$ | $16.4244$ | $\,\,\,0.55\,\,\,$ | $14.4321$
$\,\,\,0.1\,\,\,$ | $16.1424$ | $\,\,\,0.6\,\,\,$ | $14.3455$
$\,\,\,0.15\,\,\,$ | $15.7313$ | $\,\,\,0.65\,\,\,$ | $14.2650$
$\,\,\,0.2\,\,\,$ | $15.4437$ | $\,\,\,0.7\,\,\,$ | $14.1940$
$\,\,\,0.25\,\,\,$ | $15.2204$ | $\,\,\,0.75\,\,\,$ | $14.1219$
$\,\,\,0.3\,\,\,$ | $15.0380$ | $\,\,\,0.8\,\,\,$ | $14.0573$
$\,\,\,0.35\,\,\,$ | $14.8840$ | $0.840896$ | $14.0074$
Table 1: Black hole configurational entropies at different temperatures.
Figure 1: CE versus $\beta/\pi z_{0}$.

For temperatures below $T_{c}$, see FIG. 1 for $\beta/\pi z_{0}>0.840896$, the
CE is constant, given by

$S_{C}^{AdS}=13.8344\;,$ (35)

indicating that the ThAdS does not change the stability condition up to the
critical temperature, where the BH is in the stable phase. Above $T_{c}$ the
CE depends on the temperature according to the description of FIG. 1. As we
increase the temperature, i.e., as we decrease $\beta$, the CE increases. From
the usual behavior of the CE, this would mean that black holes are more
unstable at higher temperatures. This is consistent with the fact that black
holes radiate and the radiation effect becomes stronger at higher
temperatures, leading to a loss of energy.

##  V Relation between the thermodynamic and configuration entropies

The fact that we calculated the configuration entropy for a system that is in
thermal equilibrium and therefore has a well defined thermodynamic entropy
opens up the possibility of answering an interesting question. We mean: is
there a simple relation between these two quantities? In this section we want
to investigate this possibility. We now write the temperature in units of
$1/z_{0}$ or, in other words, the dimensionless temperature that we use from
now on is equal to the dimensionful temperature multiplied by the energy
parameter $z_{0}$ of the hard wall model. In order to compare the black hole
thermodynamic entropy, which is proportional to $T^{3}$, with the CE one must
find out its temperature dependence. By plotting CE versus temperature for
$T>T_{c}$, one obtains the points of Figure 2-(A), corresponding to the CE
values displayed in the TABLE 1. This plot suggests a logarithmic behavior. In
order to see if this is true, we plot in Figure 2-(B) the black hole CE versus
$\log T$, which is given by a straight line. See TABLE 2 in Appendix A for the
values of $T$ and $\log(T)$ corresponding to each CE of TABLE 1, which were
used to plot the points of FIG 2-(A) and (B). From the Figure 2-(B) we
conclude that the BH configurational entropy is proportional to $\log T$,

$S_{C}^{BH}(T)=A_{0}\log T+B_{0}\;,$ (36)

being $A_{0}$ and $B_{0}$ constants, that can be numerically estimated as
$A_{0}=0.99\pm 0.02$ and $B_{0}=14.99\pm 0.02$.

Figure 2: (A) Black hole CE versus $T$; (B) Black hole CE versus $\log T$.

Now, using eq. (36), we can relate the configurational entropy to the
thermodynamic one. This can be done rewriting the constants $A_{0}$ and
$B_{0}$ in the form:

$A_{0}=3A\;,\quad\text{and}\quad
B_{0}=B+A\log\left(\frac{N^{2}\pi^{2}}{2}\right)\;,$ (37)

so that,

$S_{BH}=e^{-\frac{B}{A}}\exp\\{S_{C}/A\\}\;,$ (38)

Equation (38) shows the relation between the thermodynamic BH entropy $S_{BH}$
with the configurational one $S_{C}$, being the first given by the exponential
of the second up to constant factors. So, we found a simple relation between
these two quantities. For the confined phase the situation is trivial: the
thermodynamic entropy vanishes and the CE is constant.

##  VI Comparison with the Gubser-Mitra conjecture

It is interesting to check if the results for stability of the hard wall model
analyzed in the previous sections are consistent with the so-called Gubser-
Mitra (GM) conjecture Gubser:2000mm ; Gubser:2000ec , that relates dynamical
stability of a black hole with infinite horizon area to the thermodynamic
stability. The Hawking Page transition, considered in this letter, is obtained
from a semiclassical analysis of thermodynamic stability (see Monteiro:2010cq
for a review). It corresponds to a global phase transition between two
different geometries that are asymptotically AdS spacetimes with compact
Euclidean time coordinate. The result that emerges from the analysis of the
Hawking-Page transition for the hard wall model is that for $T>T_{c}$ the
black hole space is thermodynamically stable, while for $T<T_{c}$ the thermal
AdS space is thermodynamically stable.

Regarding dynamic stability, one should study the behavior of the black hole
and thermal AdS geometries under the effect of perturbations of the metric.
The hard wall model is a phenomenological framework where one places a hard
cut-off in the geometry in order to obtain results of hadronic physics
assuming gauge/gravity duality to hold. The hard wall geometry, with or
without a black hole, is not a solution of the classical equations of motion
(the Einstein equations). So, it would make no sense to investigate
perturbations in the equations of motion starting from such geometries.
Nevertheless, there is an interesting way of making contact with the GM
conjecture. As discussed in Gubser:2000mm , an intuitive explanation for the
dynamical instability of a black hole of infinite extent is that the entropy
of an array of finite size black holes is higher than the entropy of the
infinite size uniform black hole with the same mass. In other words, dynamical
instability would appear as a consequence of the possibility of a transition
to other geometry with larger entropy. We presented in equations (16) and (17)
the entropy densities for thermal AdS and for the black hole in the interval
of temperatures where they are the dominant phase. For the thermal AdS, eq.
(16) is valid at any temperature since the total action density of eq. (13)
does not change with $T$. But for the black hole geometry the action density
of eq. (14) depends on the relation between $z_{0}$ and $z_{h}$. The
expression shown in eq. (17) is valid for $z_{h}>z_{0}$ or, equivalently,
$T>1/(\pi z_{0})=T_{c}/2^{1/4}$. Calculating the entropy (per unity of
transverse area) for the black hole action for $T<1/(\pi z_{0})$, using eq.
(15), one would obtain the awkward result

$S_{BH}=-\frac{N^{2}\pi^{2}T^{3}}{2}\quad\text{if}\quad
T<\frac{T_{c}}{2^{1/4}}\;.$ (39)

Such an unphysical negative entropy shows up because the imposition of a hard
cut-off at a position $z=z_{0}<z_{h}$ eliminates from the space a region
containing the horizon. In this case one would have $S_{BH}<S_{AdS}$ for
$T<T_{c}/2^{1/4}$ and $S_{BH}>S_{AdS}$ for $T>T_{c}/2^{1/4}$. So, for
$T<T_{c}/2^{1/4}$ and for $T>T_{c}$ there is an agreement between
thermodynamic instability and the existence of other space with larger
entropy, that could be associated with dynamic instability. Only in the region
$T_{c}/2^{1/4}<T<T_{c}$ the thermodynamically stable geometry has a smaller
entropy. One should not take this analysis as a proof of a relation between
dynamical and thermodynamic stabilities for the hard wall model since this
space it is not a solution of Einstein equation and also the geometry is not
analytic at $z=z_{0}$. It is just an analyisis of self consistency of the
model. This discussion provides a motivation for performing a similar analysis
in some other AdS/QCD model that is consistent with Einstein equations and
does not present any non analyticity. We plan to do this in a future work.

##  VII Conclusions

In this work we calculate the configuration entropy for the two geometries
that represent, within the hard wall AdS/QCD model, the two phases of a QCD-
like gauge theory. One is the thermal AdS space that represents the confined
phase and is dominant for temperatures below $T_{c}$. The other is the AdS
black hole, which represents the deconfined phase that dominates for higher
temperatures. We found that for the ThAdS space, the CE is constant. This is
consistent with the fact that, below $T_{c}$ this space is stable. For the BH-
AdS geometry, we found that the CE increases with the temperature. This can be
seen from FIG. 1 where $\beta$ is the inverse of the temperature, so $\beta\to
0$ is equivalent to $T\to\infty$, or in FIG. 3 where we observe that the black
hole CE is proportional to $\log(T)$. Increasing CE means in general
increasing instability. The reason for this can be traced to the Hawking
radiation of the black hole, that increases with the temperature and causes
instability in the physical state. For discussions of Hawking radiation of AdS
black holes, see for example Hubeny:2009rc . So, we found it out that for the
hard wall description of QCD-like thermal media, the relation between increase
in the CE and increase in the instability holds. We also found a relation
between the black hole thermodynamic and configuration entropies, being the
first given by the exponential of the second, up to constant factors. For an
interesting study of deconfinement transition in holographic QCD using
entanglement entropy see Dudal:2018ztm .

##  Appendix A TABLE: Black hole CE, $T$, and $\log(T)$

We display in this table the values of the temperature $T$ and $\log(T)$, with
again $T$ in units of $1/z_{0}$, which were used to plot FIG 2-(A) and (B),
for the same black hole configuration entropies of TABLE 1.

$T$ | $\log(T)$ | Black hole CE | $T$ | $\log(T)$ | Black hole CE
---|---|---|---|---|---
$\,\,\,0.378537\,\,\,$ | $\,\,\,-0.971443\,\,\,$ | $14.0074$ | $\,\,\,0.909457\,\,\,$ | $\,\,\,-0.0949078\,\,\,$ | $14.8840$
$\,\,\,0.397887\,\,\,$ | $\,\,\,-0.921586\,\,\,$ | $14.0573$ | $\,\,\,1.06103\,\,\,$ | $\,\,\,0.0592429\,\,\,$ | $15.0380$
$\,\,\,0.424413\,\,\,$ | $\,\,\,-0.857048\,\,\,$ | $14.1219$ | $\,\,\,1.27324\,\,\,$ | $\,\,\,0.241564\,\,\,$ | $15.2204$
$\,\,\,0.454728\,\,\,$ | $\,\,\,-0.788055\,\,\,$ | $14.1940$ | $\,\,\,1.59155\,\,\,$ | $\,\,\,0.464708\,\,\,$ | $15.4437$
$\,\,\,0.489708\,\,\,$ | $\,\,\,-0.713947\,\,\,$ | $14.2650$ | $\,\,\,2.12207\,\,\,$ | $\,\,\,0.75239\,\,\,$ | $15.7313$
$\,\,\,0.530516\,\,\,$ | $\,\,\,-0.633904\,\,\,$ | $14.3455$ | $\,\,\,3.18310\,\,\,$ | $\,\,\,1.15786\,\,\,$ | $16.1424$
$\,\,\,0.578745\,\,\,$ | $\,\,\,-0.546893\,\,\,$ | $14.4321$ | $\,\,\,4.24413\,\,\,$ | $\,\,\,1.44554\,\,\,$ | $16.4244$
$\,\,\,0.636620\,\,\,$ | $\,\,\,-0.451583\,\,\,$ | $14.5268$ | $\,\,\,6.36620\,\,\,$ | $\,\,\,1.85100\,\,\,$ | $16.8300$
$\,\,\,0.707356\,\,\,$ | $\,\,\,-0.346222\,\,\,$ | $14.6326$ | $\,\,\,8.37658\,\,\,$ | $\,\,\,2.12544\,\,\,$ | $17.1049$
$\,\,\,0.795775\,\,\,$ | $\,\,\,-0.228439\,\,\,$ | $14.7761$ | $\,\,\,12.73240\,\,\,$ | $\,\,\,2.54415\,\,\,$ | $17.5229$
Table 2: BH CE, $T$ and $\log(T)$.

##  Appendix B  Positivity of the CE

In this appendix we will discuss the positivity of the configuration entropy,
defined by eq. (31). One can regard the integrand of this equation,

$\lambda(k)=-f(k)\ln f(k)\;,$ (40)

as representing the CE per unit of momentum. We show in Figure 3-(A) the form
of $\lambda(k)$ for the black hole case, for 3 different temperatures. One
notices that in the limit $k\rightarrow 0$, $\lambda(k)$ reaches a finite
value.

Figure 3: (A) Black hole CE per unit of $k$ versus $k$, for $T_{1}=1/(\pi
0.05)$ (blue), $T_{2}=1/(\pi 0.1)$ (orange), and $T_{3}=1/(\pi 0.2)$ (green).
(B) CE per unit of momentum for the thermal AdS space.

In the black hole case, see Figure 3-(A), one sees that this value increases
as the temperature increases (note that $T_{1}>T_{2}>T_{3}$). The curve, as
$k$ increases, oscillates until it passes through a local maximum, then
through a local minimum, where it begins to grow until reaching the global
maximum, to finally tend to zero as $k\rightarrow\infty$. For higher
temperatures, the global maximum is higher and is reached for lower values of
momentum. In the thermal AdS space, the behavior is similar, but $\lambda(k)$
does not depend on the temperature. The corresponding plot is shown in Figure
3-(B). Hence, since the configurational entropy, defined as the integration of
$\lambda(k)$ over all modes, it is always positive.

In order to explain the physical origin of the $\lambda(k)$ behavior displayed
above for different temperatures, we should take a look at the energy density
in the Fourier space, see eq. (32). In the FIG. 4 below, we observe the same
signature of the curve, whose pattern is transmitted to
$\lambda^{(2)}_{BH}(k)$ of FIG 3, which depends on
$|\widetilde{\rho}(k)|^{2}$, according to eq. (40) and the modal fraction
definition (29). The small values of $\lambda(k)$, if compared to the energy
density, is only attributed to the normalization factor, see eq. (30). At
$T_{2}$, for instance, $\mathcal{N}_{T_{2}}=5.99696\times 10^{57}$. As we
decrease the value of the UV regulator $\epsilon$, the small oscillations
along the $\lambda(k)$ curve tends to disappear. Such small oscillations must
then be interpreted as a consequence of the regularization of the BH energy
density employed in (24), these oscillations being harmless in the precision
in which we are working.

Figure 4: $|\widetilde{\rho}(k)|^{2}$ for a black hole at $T_{2}=1/(\pi
0.1)=3.1831$.

One can numerically estimate the maximum values of the $\lambda(k)$ function
and the value of $k\equiv k_{max}$ where it occurs, for the temperatures
analysed in FIG 3. The results are displayed in TABLE III.

Temperature | $\lambda(k_{max})$ | $k_{max}$
---|---|---
$T_{1}=6.3662\,$ | $\,2.9819\times 10^{-6}\,\,\,$ | $1,00\times 10^{6}/z_{0}$
$T_{2}=T_{1}/2$ | $\,5.6988\times 10^{-6}\,\,\,$ | $5,01\times 10^{5}/z_{0}$
$T_{3}=T_{1}/4$ | $\,\,\,1.0867\times 10^{-5}\,\,\,$ | $2,51\times 10^{5}/z_{0}$
Table 3: Maximum value of $\lambda_{BH}(k)$ at different temperatures $T$
(espressed in units of $1/z_{0}$).

From this table one notices that $k_{max}$ is approximately proportional to
$T$. This is consistent with the fact that the energy distribution should
depend only on two dimensionful parameters: $z_{0}$ and $T$. Performing the
calculations in terms of units of $1/z_{0}$ we remain with only $T$ as the
free energy parameter. That is why $k_{max}$ is proportional to $T$.

Acknowledgments: The authors are supported by CNPq - Conselho Nacional de
Desenvolvimento Científico e Tecnológico. This work received also support from
Coordenação de Aperfeiçoamento de Pessoal de Nível Superior - Brasil (CAPES) -
Finance Code 001.

## References

  * (1) J. M. Maldacena, Adv. Theor. Math. Phys. 2, 231 (1998) [Int. J. Theor. Phys. 38, 1113 (1999)]. [arXiv:hep-th/9711200].
  * (2) S. S. Gubser, I. R. Klebanov and A. M. Polyakov, Phys. Lett. B 428, 105 (1998). [arXiv:hep-th/9802109].
  * (3) E. Witten, Adv. Theor. Math. Phys. 2, 253 (1998). [arXiv:hep-th/9802150].
  * (4) E. Witten, Adv. Theor. Math. Phys. 2 (1998), 505-532 doi:10.4310/ATMP.1998.v2.n3.a3 [arXiv:hep-th/9803131 [hep-th]].
  * (5) S. W. Hawking and D. N. Page, Commun. Math. Phys. 87, 577 (1983).
  * (6) J. Polchinski and M. J. Strassler, Phys. Rev. Lett. 88, 031601 (2002) [arXiv:hep-th/0109174].
  * (7) H. Boschi-Filho and N. R. F. Braga, Eur. Phys. J. C 32, 529 (2004) [arXiv:hep-th/0209080].
  * (8) H. Boschi-Filho and N. R. F. Braga, JHEP 0305, 009 (2003) [arXiv:hep-th/0212207].
  * (9) C. P. Herzog, Phys. Rev. Lett. 98, 091601 (2007) doi:10.1103/PhysRevLett.98.091601 [arXiv:hep-th/0608151 [hep-th]].
  * (10) C. A. Ballon Bayona, H. Boschi-Filho, N. R. F. Braga and L. A. Pando Zayas, Phys. Rev. D 77, 046002 (2008) doi:10.1103/PhysRevD.77.046002 [arXiv:0705.1529 [hep-th]].
  * (11) M. Gleiser and N. Stamatopoulos, Phys. Lett. B 713, 304 (2012) [arXiv:1111.5597 [hep-th]].
  * (12) M. Gleiser and N. Stamatopoulos, Phys. Rev. D 86, 045004 (2012) [arXiv:1205.3061 [hep-th]].
  * (13) M. Gleiser and D. Sowinski, Phys. Lett. B 727, 272 (2013) [arXiv:1307.0530 [hep-th]].
  * (14) A. E. Bernardini, N. R. F. Braga and R. da Rocha, Phys. Lett. B 765, 81 (2017), [arXiv:1609.01258 [hep-th]].
  * (15) N. R. F. Braga and R. da Rocha, Phys. Lett. B 776, 78 (2018) [arXiv:1710.07383 [hep-th]].
  * (16) N. R. F. Braga, L. F. Ferreira and R. Da Rocha, Phys. Lett. B 787, 16 (2018) [arXiv:1808.10499 [hep-ph]].
  * (17) A. E. Bernardini and R. da Rocha, Phys. Rev. D 98 (2018) 126011 [arXiv:1809.10055 [hep-th]].
  * (18) L. F. Ferreira and R. Da Rocha, Phys. Rev. D 99, no. 8, 086001 (2019) [arXiv:1902.04534 [hep-th]].
  * (19) N. R. F. Braga, Phys. Lett. B 797, 134919 (2019) doi:10.1016/j.physletb.2019.134919 [arXiv:1907.05756 [hep-th]].
  * (20) R. A. C. Correa and R. da Rocha, Eur. Phys. J. C 75, no. 11, 522 (2015) doi:10.1140/epjc/s10052-015-3735-8 [arXiv:1502.02283 [hep-th]].
  * (21) N. R. F. Braga and R. da Rocha, Phys. Lett. B 767, 386 (2017) [arXiv:1612.03289 [hep-th]].
  * (22) G. Karapetyan, EPL 117, no. 1, 18001 (2017) [arXiv:1612.09564 [hep-ph]].
  * (23) G. Karapetyan, EPL 118 (2017) 38001 [arXiv:1705.10617 [hep-ph]].
  * (24) G. Karapetyan, Phys. Lett. B 781 (2018) 201 [arXiv:1802.09105 [nucl-th]].
  * (25) G. Karapetyan, Phys. Lett. B 786 (2018) 418 [arXiv:1807.04540 [nucl-th]].
  * (26) C. O. Lee, Phys. Lett. B 790 (2019) 197 [arXiv:1812.00343 [gr-qc]].
  * (27) D. Bazeia, D. C. Moreira and E. I. B. Rodrigues, J. Magn. Magn. Mater. 475 (2019) 734.
  * (28) C. W. Ma and Y. G. Ma, Prog. Part. Nucl. Phys. 99, 120 (2018) [arXiv:1801.02192 [nucl-th]].
  * (29) Q. Zhao, B. Z. Mi and Y. Li, Int. J. Mod. Phys. B 33, no. 12, 1950119 (2019).
  * (30) L. F. Ferreira and R. da Rocha, Eur. Phys. J. C 80 (2020) no.5, 375 doi:10.1140/epjc/s10052-020-7978-7 [arXiv:1907.11809 [hep-th]].
  * (31) A. E. Bernardini and R. da Rocha, Phys. Lett. B 796, 107 (2019) doi:10.1016/j.physletb.2019.07.028 [arXiv:1908.04095 [gr-qc]].
  * (32) G. Karapetyan, EPL 129, no. 1, 18002 (2020) doi:10.1209/0295-5075/129/18002 [arXiv:1912.10071 [hep-ph]].
  * (33) N. R. F. Braga and R. da Mata, Phys. Rev. D 101, no.10, 105016 (2020) doi:10.1103/PhysRevD.101.105016 [arXiv:2002.09413 [hep-th]].
  * (34) G. Karapetyan, [arXiv:2003.08994 [hep-ph]].
  * (35) L. F. Ferreira and R. da Rocha, Phys. Rev. D 101, no.10, 106002 (2020) doi:10.1103/PhysRevD.101.106002 [arXiv:2004.04551 [hep-th]].
  * (36) A. Alves, A. G. Dias and R. da Silva, doi:10.1016/j.nuclphysb.2020.115137 [arXiv:2004.08407 [hep-ph]].
  * (37) D. Marinho Rodrigues and R. da Rocha, [arXiv:2006.00332 [hep-th]].
  * (38) V. Balasubramanian and P. Kraus, Commun. Math. Phys. 208, 413 (1999) [arXiv:hep-th/9902121].
  * (39) R. Emparan, C. V. Johnson and R. C. Myers, Phys. Rev. D 60, 104001 (1999) [arXiv:hep-th/9903238].
  * (40) G. W. Gibbons and S. W. Hawking, Phys. Rev. D 15, 2752 (1977).
  * (41) G. E. Arutyunov and S. A. Frolov, Nucl. Phys. B 544, 576 (1999) [arXiv:hep-th/9806216].
  * (42) S. S. Gubser, I. R. Klebanov and A. W. Peet, Phys. Rev. D 54 (1996) 3915 [arXiv:hep-th/9602135].
  * (43) S. S. Gubser, I. R. Klebanov and A. A. Tseytlin, Nucl. Phys. B 534 (1998) 202 [arXiv:hep-th/9805156].
  * (44) C. E. Shannon, The Bell System Technical Journal, 27, 379 (1948).
  * (45) S. S. Gubser and I. Mitra, JHEP 08, 018 (2001) doi:10.1088/1126-6708/2001/08/018 [arXiv:hep-th/0011127 [hep-th]].
  * (46) S. S. Gubser and I. Mitra, Clay Math. Proc. 1, 221 (2002) [arXiv:hep-th/0009126 [hep-th]].
  * (47) R. Monteiro, doi:10.17863/CAM.16097 [arXiv:1006.5358 [hep-th]].
  * (48) V. E. Hubeny, D. Marolf and M. Rangamani, Class. Quant. Grav. 27, 095018 (2010) doi:10.1088/0264-9381/27/9/095018 [arXiv:0911.4144 [hep-th]]. Dudal:2018ztm
  * (49) D. Dudal and S. Mahapatra, JHEP 07, 120 (2018) doi:10.1007/JHEP07(2018)120 [arXiv:1805.02938 [hep-th]].


The results are plotted in Figure 12. We have tested three methods to compute
the $c_{k}$:

  * •

floating-point operations using 64 bits floats;

  * •

floating-point operations using 128 bits floats;

  * •

SymPy: make the whole computation using SymPy, a Python library of symbolic
computation (very slow).

In all three cases, instabilities appear before the sequence $(c_{k})_{k}$ has
fully converged to $0$. Moreover, the oscillations of $(c_{k})_{k}$ around $0$
have an increasing wavelength, which indicates that we may have to go far
beyond $k=500$ to get enough coefficients $(c_{k})_{k}$ to reconstruct the
wanted inverse Mellin transform.

In Figure 13, we have plotted two estimations of $f_{|Y|}$: the density
obtained directly by using $(c_{k})_{k\in[1,300]}$, and the density obtained
by using a sequence $(c_{k})_{k\in[1,20000]}$ where the values
$(c_{k})_{k\in[301,20000]}$ have been extrapolated from
$(c_{k})_{k\in[1,300]}$.212121The extrapolation has been performed by modeling
the graph of $(c_{k})_{k}$ as the product of a decreasing function and a
cosine with decreasing frequency.

The resulting estimations of the density $f_{|Y|}$ take negative values and
seem to be noised. So, more work is needed to obtain smooth and proper
densities, especially if we want them to meet Constraints 3 and 4 (density at
$0$ and decay rate at $\infty$).

Figure 12: Evolution of various numerical computations of $c_{k}$ as $k$
grows. Figure 13: Numerical inverse Mellin transform with two different
computations of $(c_{k})_{k}$: direct computation of $(c_{k})_{k\in[1,N]}$
with $N=300$; extrapolation of $(c_{k})_{k\in[301,20000]}$ from
$(c_{k})_{k\in[1,300]}$.

##  Appendix F Obtaining $f_{|Y|}$: experimental details

We optimize the vector of parameters $\Lambda$ with respect to the following
loss:

$\displaystyle\ell(\Lambda)$
$\displaystyle:=\|\hat{F}_{\Lambda}-F_{|G|}\|_{\infty}$
$\displaystyle\hat{F}_{\Lambda}(z)$
$\displaystyle:=\int_{0}^{\infty}F_{|W|}\left(\frac{z}{t}\right)g_{\Lambda}(t)\,\mathrm{d}t.$

##### Dataset.

We build the dataset $\mathcal{Z}$ of size $d$:

$\displaystyle\mathcal{Z}=\left\\{0,z_{\mathrm{max}}\frac{1}{d-1},z_{\mathrm{max}}\frac{2}{d-1},\cdots,z_{\mathrm{max}}\right\\}.$

In our setup, $d=200$ and $z_{\mathrm{max}}=5$.

##### Computing the loss.

For each $z$ in $\mathcal{Z}$, we compute numerically $\hat{F}_{\Lambda}(z)$.
Then, we are able to compute $\ell(\Lambda)$. We keep track of the
computational graph with PyTorch, in order to backpropagate the gradient and
train the parameters $\Lambda$ by gradient descent.

##### Initialization of the parameters.

We initialize $\Lambda=(\alpha,\gamma,\lambda_{1},\lambda_{2})$ in the
following way:

$\displaystyle\alpha=3,\quad\gamma=1,\quad\lambda_{1}=1,\quad\lambda_{2}=1.$

##### Optimizer.

We use the Adam optimizer (Kingma & Ba, 2015) with the parameters:

  * •

learning rate: $0.001$;

  * •

$\beta_{1}=0.9$;

  * •

$\beta_{2}=0.999$;

  * •

weight decay: $0$.

We train $\Lambda$ for $100$ epochs.

##### Learning rate scheduler.

We use a learning rate scheduler based on the reduction of the training loss.
If the training loss does not decrease at least by a factor $0.01$ for $20$
epochs, then the learning rate is multiplied by a factor $1/\sqrt[3]{10}$.
After a modification of the learning rate, we wait at least $20$ epochs before
any modification.

##### Scheduler for $\theta^{\prime}$.

We recall that the definition of $g_{\Lambda}$ involves $\theta^{\prime}$,
defined by: $\frac{1}{\theta}+\frac{1}{\theta^{\prime}}=\frac{1}{2}$. It is
not a parameter to train. Empirically, we found that the following schedule
improves the optimization process:

  * •

from epoch $0$ to epoch $49$, $\theta^{\prime}$ increases linearly from $2$ to
its theoretical value $(\frac{1}{2}-\frac{1}{\theta})^{-1}$;

  * •

at the beginning of epoch $50$, we reinitialize the optimizer and the learning
rate scheduler;

  * •

we finish the training normally, with
$\theta^{\prime}=(\frac{1}{2}-\frac{1}{\theta})^{-1}$.

##  Appendix G Experiments

###  G.1 Propagation of the correlations with MNIST

Within the setup of Section 2.2, we have plotted in Figure 14 the correlations
propagated in a multilayer perceptron with inputs sampled for MNIST ($10$
samples per class, that is, $100$ samples in total). The layers of the
perceptron have $n_{l}=10$ neurons each, and the activation function is
$\phi=\mathrm{ReLU}$.

We observe that, in this narrow NN case ($n_{l}=10$), some irregularities
appear as information propagates into the network: some classes seem to
(de)correlate in an inconsistent way with the others. Specifically, while most
of the classes tend to correlate exactly ($C^{*}\approx 1$), the third class
(corresponding to the digit “$2$”) tends to a correlation $C^{*}\approx
0.7\neq 1$. This result is contradictory to the EOC theory.

$\phi$ | input | $l=10$ | $l=30$ | $l=50$
---|---|---|---|---
$\mathrm{ReLU}$ (MNIST) |  |  |  |

Correlation

Figure 14: Propagation of correlations $c^{l}_{ab}$ in a multilayer perceptron
with activation function $\phi=\mathrm{ReLU}$ and inputs sampled from the
MNIST dataset. The neural network is initialized at the EOC. Each plot
displays a $10\times 10$ matrix $C_{pq}^{l}$ whose entries are the average
correlation between the pre-activations propagated by samples from classes
$p,q\in\\{0,\cdots,9\\}$, at the input and right after layers
$l\in\\{10,30,50\\}$. See also Figure 1 in Section 2.2 for results on
CIFAR-10.

###  G.2 Propagation of the correlations with $\phi=\phi_{\theta}$

Within the setup of Section 2.2, we have plotted in Figure 15 the correlations
propagated in a multilayer perceptron with $\phi=\phi_{\theta}$. The weights
have been sampled from $\mathcal{W}(\theta,1)$.

In this setup, the results are consistent, and are consistent with the results
for $\phi=\tanh$ (see Figure 1): the sequence $(C_{pq}^{l})_{l}$ converges to
$1$, which was not the case for $\phi=\mathrm{ReLU}$ and $n_{l}=10$.

| $\phi$ | input | $l=10$ | $l=30$ | $l=50$
---|---|---|---|---|---
$n_{l}=10$ neurons per layer |  $\phi_{2.05}$ |  |  |  |
|  $\phi_{7.00}$ |  |  |  |
$n_{l}=100$ neurons per layer |  $\phi_{2.05}$ |  |  |  |
|  $\phi_{7.00}$ |  |  |  |

Correlation

Figure 15: Propagation of correlations $c^{l}_{ab}$ in a multilayer perceptron
with activation function $\phi_{\theta}$ with $\theta\in\\{2.05,7.00\\}$ and
inputs sampled from the CIFAR-10 dataset. The weights are sampled from
$\mathcal{W}(\theta,1)$ and the biases are zero. Each plot displays a
$10\times 10$ matrix $C_{pq}^{l}$ whose entries are the average correlation
between the pre-activations propagated by samples from classes
$p,q\in\\{0,\cdots,9\\}$, at the input and right after layers
$l\in\\{10,30,50\\}$.

###  G.3 Limits of the KS test

In the KS test presented above, the null hypothesis $\mathbb{H}_{0}$ is
$\mathrm{P}^{\prime}=\mathcal{N}(0,1)$, which is exactly what we intend to
demonstrate when using our family
$\\{(\mathrm{P}_{\theta},\phi_{\theta}):\theta\in(2,\infty)\\}$, while the
alternative hypothesis $\mathbb{H}_{1}$ is
$\mathrm{P}^{\prime}\neq\mathcal{N}(0,1)$. With this test design, we face a
problem: it is impossible to claim that $\mathbb{H}_{0}$ holds. More
precisely, we have two possible outcomes: either $\mathbb{H}_{0}$ is rejected,
or it is not. Since $\mathbb{H}_{1}$ is the complementary of $\mathbb{H}_{0}$,
a reject of $\mathbb{H}_{0}$ means an accept of $\mathbb{H}_{1}$. But the
converse does not hold: if $\mathbb{H}_{0}$ is not rejected, then it is
impossible to conclude that $\mathbb{H}_{0}$ is true: the sample size $s$ may
simply be too small, or the KS test may be inadequate for our case.

So, to be able to conclude that the pre-activations are Gaussian, we should
have built an alternative test, where the null hypothesis
$\mathbb{H}_{0}^{\prime}$ is some _misfit_ between $\mathrm{P}^{\prime}$ and
$\mathcal{N}(0,1)$. Or, at least, we should have computed the power of the
current KS test.

However, our experimental results are sufficient to _compare_ the quality of
fit between $\mathrm{P}^{\prime}$ and $\mathcal{N}(0,1)$ in the tested setups.
It remains clear that, with
$\\{(\mathrm{P}_{\theta},\phi_{\theta}):\theta\in(2,\infty)\\}$, the Gaussian
pre-activations hypothesis is far more likely to hold than in setups involving
$\mathrm{tanh}$, $\mathrm{ReLU}$ and Gaussian weights.

###  G.4 Variance of the pre-activation when using $\phi=\phi_{\theta}$

As observed in Figure 6(a), the product $W\phi_{\theta}(X)$ is above the KS
threshold, corresponding to $s=10^{7}$ samples and a $p$-value of $0.05$. This
is not necessarily the case in Figure 6(b), where the samples are
standardized. So, we suspect that the variance of $W\phi_{\theta}(X)$ is not
exactly $1$.

Therefore, we have reported in Table 2 the empirical standard deviation of the
product $W\phi(X)$ (i.e., $Z^{\prime}$ with $n=1$), computed with $s=10^{7}$
samples, where $X\sim\mathcal{N}(0,1)$, $W\sim\mathcal{W}(\theta,1)$ if
$\phi=\phi_{\theta}$ and $W\sim\mathcal{N}(0,1)$ if $\phi=\tanh$ or
$\mathrm{ReLU}$. We observe that the standard deviation of $Z^{\prime}$, which
is expected to be $1$ with $\phi=\phi_{\theta}$, is actually a bit different.
This would largely explain the differences between Figure 6(a) and Figure
6(b).

Table 2: Empirical standard deviation of $Z^{\prime}$ with $n=1$. | $\phi_{\theta}$ | $\tanh$ | $\mathrm{ReLU}$
---|---|---|---
$\theta$ | $2.05$ | $2.5$ | $3$ | $4$ | $5$ | $7$ | $10$ |  |
$\bar{\sigma}$ | 1.003 | 0.994 | 0.989 | 0.984 | 0.981 | 0.979 | 0.978 | 0.628 | 0.707

###  G.5 Propagation of the pre-activations

In this subsection, we show how the choice of the input data points affect the
propagation of the pre-activations.

Let $\mathcal{D}^{l}$ be the distribution of the pre-activation $Z^{l}_{1}$
after layer $l$. In Figure 16, we have plotted the $\mathcal{L}^{\infty}$
distance between the CDFs of $\mathcal{D}^{l}$ and $\mathcal{N}(0,1)$ for
various input points, sampled from different classes to improve diversity
between them. We have chosen the following setup: CIFAR-10 inputs, multilayer
perceptron with $100$ layers and $100$ neurons per layer, Gaussian
initialization at the EOC when using the activation function $\phi=\tanh$ or
$\mathrm{ReLU}$, and symmetric Weibull initialization $\mathcal{W}(\theta,1)$
when using $\phi=\phi_{\theta}$.

This setup has been selected to illustrate clearly the variability of
$\mathcal{D}^{l}$ when using various data points. This variability can be
observed with $10$ or $1000$ neurons per layer, though it is less striking.

##### With input normalization over the whole dataset.

In this setup, we perform the usual normalization over the whole dataset:

$\displaystyle\hat{x}_{a;ij}$
$\displaystyle:=\frac{x_{a;ij}-\mu_{i}}{\sigma_{i}},$ $\displaystyle\mu_{i}$
$\displaystyle:=\frac{1}{Np_{i}}\sum_{\mathbf{x}\in\mathbb{D}}\sum_{j=1}^{p_{i}}x_{ij}$
$\displaystyle\sigma_{i}^{2}$
$\displaystyle:=\frac{1}{Np_{i}-1}\sum_{\mathbf{x}\in\mathbb{D}}\sum_{j=1}^{p_{i}}(x_{ij}-\mu_{i})^{2},$

where $\hat{\mathbf{x}}$ is the normalized data point, $x_{a;ij}$ is the
$j$-th component of the $i$-th channel of the input image $\mathbf{x}_{a}$,
$p_{i}$ is the size of the $i$-th channel, and $N$ is the size of the dataset
$\mathbb{D}$.

We observe that, when using the activation function $\phi=\tanh$, the shape of
the curves is the same for all data points. This is not the case for
$\mathrm{ReLU}$: for the input point “bird”, the sequence
$(\mathcal{D}^{l})_{l}$ drifts away from $\mathcal{N}(0,1)$ since the
beginning, while for the input “car”, $(\mathcal{D}^{l})_{l}$ first becomes
closer to $\mathcal{N}(0,1)$, then drifts away. It is also the case for
$\phi_{\theta}$ with $\theta=10$: for the input “deer”, the distance remains
high, while it starts low and increases very slowly (input “dog”), or even
decreases in the first place (input “truck”).

But, in general, when the curves have converged after $100$ layers, it seems
that the limit is the same whatever the starting data point, and depends only
on the choice of the activation function.

Figure 16: Setup with data points normalized over the whole dataset.
Propagation of the distribution $\mathcal{D}^{l}$ of the pre-activations
across the layers $l\in[1,100]$ when inputting various data points of
CIFAR-10. Each curve represents the $\mathcal{L}^{\infty}$ distance between
the CDF of $\mathcal{D}^{l}$ and the CDF of the Gaussian $\mathcal{N}(0,1)$.
The dashed green line is the threshold of rejection of the Kolmogorov–Smirnov
test with $p$-value $0.05$: if a distribution $\mathcal{D}^{l}$ is represented
by a point above this threshold, then the hypothesis
“$\mathcal{D}^{l}=\mathcal{N}(0,1)$” is rejected with $p$-value $0.05$.

##### With individual input normalization.

In this setup, we perform the individual normalization of the inputs. Without
loss of generality, we define it for inputs that are order-$1$ tensors as
follows:

$\displaystyle\hat{x}_{a;i}$ $\displaystyle:=\frac{x_{a;i}-\mu}{\sigma},$
$\displaystyle\mu$ $\displaystyle:=\frac{1}{p}\sum_{i=1}^{p}x_{i}$
$\displaystyle\sigma^{2}$
$\displaystyle:=\frac{1}{p-1}\sum_{i=1}^{p}(x_{i}-\mu)^{2},$

where $\hat{\mathbf{x}}$ is the normalized data point, $x_{a;i}$ is the $i$-th
component of $\mathbf{x}_{a}\in\mathbb{R}^{p}$.

According to Figure 17, the curves seem to be approximately identical for each
data point, contrary to the setup with normalization over the whole dataset.
And, naturally, the distance between $\mathcal{D}^{1}$ and $\mathcal{N}(0,1)$
is close to $0$ when using the activation functions $\phi_{\theta}$, which
confirms that our method to build Gaussian pre-activations is efficient at
least in the beginning.

Figure 17: Setup with data points normalized individually.
The rest of the setup is identical to the one used in Figure 16.

##### Conclusion.

In the specific case of CIFAR-10 images, it seems that the trajectory of the
sequence $(\mathcal{D}^{l})_{l}$ depends on the inputs, but only to the extent
that they have different norms. As it can be seen in Figure 17, images from
different classes which have been normalized individually lead to the same
trajectories of distribution of pre-activations.

###  G.6 Experimental details of the training procedure

##### Training, validation, and test sets.

For MNIST and CIFAR-10, we split randomly the initial training set into two
sets: the training set, which will be actually used to train the neural
network, and the validation set, which will be used to stop training when the
network begins to overfit.

The sizes of the different sets are as follows:

  * •

MNIST: 50000 training samples; 10000 validation samples; 10000 test samples;

  * •

CIFAR-10: 42000 training samples, 8000 validation samples; 10000 test samples.

The training sets are split into mini-batches with 200 samples each. No data
augmentation is performed.

##### Loss.

Given a classification task with $P$ classes, let
$\mathbf{z}^{L}\in\mathbb{R}^{P}$ be the pre-activation outputted by the last
layer of the neural network. First, we perform a $\mathrm{softmax}$ operation:

$\displaystyle
y_{p}:=\mathrm{softmax}(z^{L}_{p})=\frac{\exp(z^{L}_{p})}{\sum_{p^{\prime}=1}^{P}\exp(z^{L}_{p^{\prime}})}.$

where the $(y_{p})_{p}$ are the components of $\mathbf{y}\in\mathbb{R}^{P}$
and $(z^{L}_{p})_{p}$ are the components of $\mathbf{z}^{L}$. Then, we compute
the negative log-likelihood loss. For a target class $p\in\\{1,\cdots,P\\}$,
we pose:

$\displaystyle\ell(\mathbf{y},p):=-\log(y_{p}).$

##### Optimizer.

We use the Adam optimizer (Kingma & Ba, 2015) with the parameters:

  * •

learning rate: $0.001$;

  * •

$\beta_{1}=0.9$;

  * •

$\beta_{2}=0.999$;

  * •

weight decay: $0$.

##### Learning rate scheduler.

We use a learning rate scheduler based on the reduction of the training loss.
If the training loss does not decrease at least by a factor $0.01$ for $10$
epochs, then the learning rate is multiplied by a factor $1/\sqrt{10}$.

##### Early stopping.

We add an early stopping rule based on the reduction of the validation loss.
If the validation loss does not decrease at least by a factor of $0.001$ for
$30$ epochs, then we stop training.


# UTD-CRSS Systems for 2016 NIST Speaker Recognition Evaluation

###### Abstract

This document briefly describes the systems submitted by the Center for Robust
Speech Systems (CRSS) from The University of Texas at Dallas (UTD) to the 2016
National Institute of Standards and Technology (NIST) Speaker Recognition
Evaluation (SRE). We developed several UBM and DNN i-Vector based speaker
recognition systems with different data sets and feature representations.
Given that the emphasis of the NIST SRE 2016 is on language mismatch between
training and enrollment/test data, so-called domain mismatch, in our system
development we focused on: (1) using unlabeled in-domain data for centralizing
data to alleviate the domain mismatch problem, (2) finding the best data set
for training LDA/PLDA, (3) using newly proposed dimension reduction technique
incorporating unlabeled in-domain data before PLDA training, (4) unsupervised
speaker clustering of unlabeled data and using them alone or with previous
SREs for PLDA training, (5) score calibration using only unlabeled data and
combination of unlabeled and development (Dev) data as separate experiments.

##  1 INTRODUCTION

The main task for NIST’s speaker recognition evaluations is speaker detection,
i.e., to determine whether a specified target speaker is speaking during a
given segment of speech or not. Compared with previous SRE challenges, there
are some differences: (1) target speaker data will not be distributed in
advance like in SRE12, (2) fixed condition is introduced, (3) more duration
variability is introduced in the test data, (4) language mismatch between
training (mainly English) and enrollment/test (non-English) data. All these
new traits make this SRE very challenging, especially with limited labeled
data in the fixed condition [1].

This report introduces how CRSS systems address the problem. The whole report
is organized as follows: Sec.2 describes several baseline systems focusing on
the front-end level overview, including both data sets and feature
representations, Sec.3 introduces several core techniques that we used in
SRE16, including speaker clustering for unlabeled training data, discriminant
analysis via support vectors (SVDA) to reduce dimension and compensate domain
mismatch, unlabeled data PLDA, calibration and fusion strategies etc. Sec.4
and Sec.5 details the configuration of every CRSS sub-system and the formation
of CRSS final Eval submissions to NIST. Sec.6 shows the CRSS sub-system
performance on Dev data. We also briefly introduce the CPU/GPU hardware
systems that we used for NIST SRE 2016 in Sec.7.

##  2 CRSS baselines

We developed 4 baseline systems in this SRE, all of them are i-Vector based
systems but with different acoustic modeling, i.e., UBM or different DNN
models [2, 3]. For back-end, we mainly use LDA/SVDA to reduce the dimension of
i-Vectors and PLDA to calculate likelihood scores.

Table 1 summarized the number of speakers, speech segments used for training
UBM, total variability matrix (TV), LDA/SVDA and PLDA models as well as the
statistics for the Dev set provided by NIST for the system development
purposes.

Table 1:  Statistics of training data used for modeling DNN, UBM, TV,
LDA/SVDA/PLDA, and also Dev set enrollment and trials data.

Sub-system DNN UBM/TV LDA/SVDA/PLDA Enrollment Trials Spkrs Segments Spkrs
Segments Spkrs Segments Spkrs Segments Target nonTarget CRSS1 - - 38110 89860
2921 37037 80 120 4828 19312 CRSS2 4878 4878 5767 57517 2921 37037 CRSS3 - -
5756 57273 3794 36410 CRSS4 1239 1239 5756 57273 3794 36410

###  2.1 CRSS1: UBM i-Vector

This system is mainly modified version of Kaldi (sre10/v1). 60 dimensional
feature vectors for each frame is adopted here including 20 dimensional MFCC
features appended with $\Delta+\Delta\Delta$. Unvoiced parts of the utterances
are removed with energy based voice activity detection (VAD). For training
2048-mixture UBM and total variability (TV) matrix, SRE2004, 2005, 2006, 2008,
telephone data of SRE 2010, Switchboard II phase 2,3 and Switchboard Cellular
Part1 and Part2 (SWB) and Fisher English are used. Next, 600 dimensional
i-Vectors are extracted and their dimensions are reduced to 580 with LDA. For
training LDA/PLDA, only SRE 04-08 are used; in addition, speakers who have
less than 4 utterances is filtered out. Also, unsupervised speaker clustering
is performed (see Sec. 3.1 for the details of speaker clustering), 75 speaker
clusters for unlabeled minor data and 300 for unlabeled major data are
generated. These clustered in-domain data are used separately to train PLDA
and also for calibration. Before PLDA scoring, mean subtraction is also
applied. For SRE16 development (Dev) trials, the mean i-Vector is generated
using only unlabeled minor data, while for SRE16 evaluation (Eval), the mean
is from unlabeled major data.

###  2.2 CRSS2: SWB DNN i-Vector

We developed a DNN i-Vector system based on Kaldi (swbd/s5 & sre10/v2). The
DNN acoustic model is used to generate the soft alignments for i-Vector
extraction. The DNN architecture has 6 fully connected hidden layers with 1024
nodes for each layer. Cross-entropy objective function is employed to estimate
posterior probabilities of 3178 senones. The ASR corpus which we used for
training DNN acoustic model is Switchboard. 11-frame context of 39 dimensional
($\Delta+\Delta\Delta$ ) MFCC feature is projected into 40 dimensional using
fMLLR transform for each utterance, which relies on a GMM-HMM decoding
alignment.

The reason we apply fMLLR feature here is that, by speaker normalization, we
expect to acquire more accurate phonetic alignment in the following TV matrix
training, see more details in [3]. After i-Vector extraction, we apply similar
strategies for back-end such as LDA and PLDA, briefly described in the above
section (as CRSS1).

###  2.3 CRSS3: UBM i-Vector

An alternative UBM i-Vector system also adopted from Kaldi (sre10/v1). In this
system, feature vectors contain 20 MFCCs appended with ($\Delta+\Delta\Delta$)
coefficients. The window length and shift size are 25-ms and 10-ms,
respectively. In addition, we did cepstral mean normalization using 3-sec
sliding window. Next, non-speech frames are discarded using energy-based voice
activity detection. 2048-mixture full covariance UBM and total variability
matrix have been trained using data collected from SRE2004, 2005, 2006, 2008
and Switchboard II phase 2,3 and Switchboard Cellular Part1 and Part2. At the
back-end, after extracting i-Vectors, the global mean calculated from minor
and major unlabeled data is subtracted from all i-Vectors. Next, i-Vectors are
length-normalized and their dimension are reduced from 600 to 400 using
LDA/SVDA. In some developed systems based on CRSS3 configuration, we used both
LDA and SVDA; more specifically, first SVDA reduces the dimension from 600 to
500 and then LDA is used to reduce the dimension to 400. Again, i-Vectors are
length-normalized. Finally, trial-based mean subtraction is used (the
participant i-Vectors in a trial are averaged and the value is subtracted from
both i-Vectors) and scores are calculated using PLDA. The front end is trained
with SWB and SRE04-08; however, the back-end only uses SRE04-08 and unlabeled
training data. For back-end, mostly MSR [4] toolkit has been adopted.

###  2.4 CRSS4: Fisher English DNN i-Vector

The last baseline is a DNN i-Vector system using Kaldi (sre10/v2) that is
based on the multisplice time delay DNN (TDNN) [5]. TDNN is trained with only
a small portion of Fisher English data (1239 utterances). The feature vectors
contain 40 dimensional f-bank features. TDNN has six layers; the hidden layers
have an input dimension of 350 and an output dimension 3500. The softmax
output layer computes posteriors for 3859 triphone states. More details on the
TDNN structure and training procedure are provided in [5]. After TDNN
training, 20 MFCCs appended with ($\Delta+\Delta\Delta$) coefficients (overall
60 MFCCs) are employed for training TV matrix. Next, 600-dim i-Vectors are
extracted.

After i-Vector extraction, we apply similar strategies for back-end such as
LDA/SVDA and PLDA, briefly described in the above section (as CRSS3).

##  3 Core components in system development

In the fixed condition of SRE 2016, we have a huge amount of out-of-domain
data, i.e., previous SREs, SWB, Fisher English etc. Only a small in-domain
data is available (without speaker labels), which make existing techniques
very difficult to work with this so-called domain mismatch. In SRE 2016, NIST
provided unlabeled training data, which contains two subsets, i.e., unlabeled
minor and unlabeled major. The unlabeled minor data set has 200 utterances,
while the major set contains 2272 utterances. The minor set has two languages
for the purpose of system development, while the major set contains two
different languages corresponding to the final evaluation.

In order to address this problem, several techniques are proposed in this
evaluation.

###  3.1 Speaker clustering of unlabeled data

For compensating the domain mismatch, the use of unlabeled data becomes very
important. There are several stages where we can use the unlabeled data, such
as, LDA/PLDA training and calibration. First, it is very intuitive to do a
speaker clustering of the unlabeled data, and then generate an “estimated”
speaker label for each utterance, similar with the method that we used in 2015
NIST LRE i-Vector challenge[6]. With these labels, we incorporate the in-
domain information from unlabeled data to train LDA and PLDA. In fact, in the
experiment, this simple operation improved the LDA/PLDA baseline performance
for development set.

In practice, we train a gender identification using previous SRE data before
speaker clustering, and then apply a simple K-means algorithm over gender
dependent subsets, finally, we pool these two subsets together. In the
experiment, we found this can provide more accurate speaker clustering and
more benefits to the following LDA and PLDA training.

Table 2:  Description of the CRSS sub-systems. Speaker clustering means using
the unlabeled data to do clustering and using estimated labels for LDA/PLDA
training. The systems that have SVDA, for training SVDA they use LDA data in
addition to Minor and Major data. For sub-systems 3 and 4, for Dev set Minor
data are used for training PLDA; however, for Eval set Major data are used.

| Sub
---
system
i-Vector |  | LDA/SVDA data
---
| PLDA data
---
SVDA | LDA |  | Speaker
---
Clustering
Filtering
1 | CRSS 4 | SRE 04-08 | SRE 04-08 | ✕ | ✓ | ✕ | ✕
2 | CRSS 4 | SRE 04-08/ + Minor, Major | SRE 04-08 | ✓ | ✓ | ✕ | ✕
3 | CRSS 2 | SRE 04-08, Minor, Major | SRE 04-08, Minor or Major | ✕ | ✓ | ✓ | ✓
4 | CRSS 1 | SRE 04-08, Minor, Major | SRE 04-08, Minor or Major | ✕ | ✓ | ✓ | ✓
5 | CRSS 3 | SRE 04-08 | SRE 04-08 | ✕ | ✓ | ✕ | ✕
6 | CRSS 3 | SRE 04-08/+ Minor, Major | SRE 04-08 | ✓ | ✓ | ✕ | ✕
7 | CRSS 3 | SRE 04-08/+ Minor, Major | SRE 04-08 | ✓ | ✕ | ✕ | ✕

###  3.2 Discriminant analysis via support vectors (SVDA)

Discriminant analysis via support vectors (SVDA) is a variation of LDA that
only uses support vectors to calculate the between and within class covariance
matrices. In contrast to LDA, SVDA captures the boundary of classes, and
performs well for small sample size problem (i.e. when the dimensionality is
greater than sample size). The idea of using support vectors with discriminant
analysis has been previously introduced in [7] which made significant
improvement over LDA. In addition, the effectiveness of SVDA in i-Vector/PLDA
speaker recognition for NIST SRE2010 is studied in [8] previously for both
long and short duration test utterances.

More specifically, LDA defines speaker classes separation criterion in
direction $A$ as,

$\lambda=\dfrac{A^{T}S_{b}A}{A^{T}S_{w}A},$ (1)

where $S_{b}$ and $S_{w}$ are between class and within class covariance
matrices. In traditional LDA every sample of all classes participate in
calculating these covariance matrices; however, for SVDA only support vectors
are used. The between class covariance matrix in SVDA is defined as,

$S_{b}=\sum_{1\leq c_{1}\leq c_{2}\leq C}w_{c_{1}c_{2}}w_{c_{1}c_{2}}^{T}.$
(2)

Where $w_{c_{1}c_{2}}$ is the optimal direction to separate two classes
$c_{1}$ and $c_{2}$ by a linear SVM (for calculating $w_{c_{1}c_{2}}$ only
support vectors of the two classes $c_{1}$ and $c_{2}$ are participating). If
we define $\hat{X}=[\hat{x}_{1},\hat{x}_{2},...,\hat{x}_{\hat{N}}]$ to contain
all support vectors and $\hat{N}$ to be their total number; then, the within
class covariance matrix for SVDA will be formulated as,

$S_{w}=\sum_{c=1}^{C}\sum_{i\in\hat{I}_{c}}(\hat{x}_{i}-\hat{\mu}_{c})(\hat{x}_{i}-\hat{\mu}_{c})^{T},$
(3)

the index for support vectors in class $c$ and their mean are represented by
$\hat{I}_{c}$ and $\hat{\mu}_{c}$, respectively. Finally, similar to LDA, the
optimum transformation $\hat{A}$ will contain the $k$ eigenvectors
corresponding to the $k$ largest eigenvalues of $S^{-1}_{w}S_{b}$. More
details on the advantages and properties of SVDA are provided in [8].

Two strategies can be adopted here for training linear SVM in SVDA framework:
1) one-versus-one and 2) one-versus-rest. We used the second approach as it
can use minor and major unlabeled data optimally. More specifically, for
training the SVM classifier to separate one class against data from the rest
classes, minor and major unlabeled data are added to the rest class.
Therefore, the class labels are not needed here.

###  3.3 Unlabeled data PLDA

To fully explore the information from the in-domain unlabeled data, we did an
interesting experiment, which uses only the in-domain unlabeled data to train
PLDA (however, SRE04-08 are used for training LDA). To do that, we use the
“estimated” labels from speaker clustering. Surprisingly, PLDA with only 75
estimated speakers for 200 minor language i-Vectors achieved 20.5% EER on Dev
experiment (using i-Vectors for CRSS1 baseline), which is not so bad. However,
if we add more data (i.e., 75 + 300 estimated speakers from 2472 i-Vectors in
minor and major languages) for PLDA training, the performance degraded from
20.5% to 26.3%. This observation suggests that out-of-domain language data is
not helpful to train a discriminative classifier, because in the view of Dev
enrollment/test data, the major language data is also out-of-domain. We argue
that even for data-driven algorithm such as PLDA, choosing a proper data set
to train the classifier is still essential.

Motivated by this, we believe the use of unlabeled data in the SRE 2016
evaluation will be more beneficial. Compared with only 200 minor language
utterances, major language set has 2272 utterances. Although the speaker label
is not given, we say the estimated label is still useful, and could probably
perform better than 20.5% EER in the Eval set.

###  3.4 Calibration and fusion

The CRSS calibration and fusion system is mainly based on the BOSARIS toolkit
[9]. The PAV algorithm is used to create calibration transformation matrix. We
used two data sets for calibration. The first one is Dev data, we use all the
Dev trials information that NIST provided for system development to train the
calibration system. Again, because in this SRE, the Dev and evaluation set
have totally different languages, it’s not guaranteed that the calibration
will work well for the final Eval set. For this consideration, we created a
new trial list to calibrate evaluation scores, and we used unlabeled data with
estimated speaker labels. We believe the score distribution of unlabeled data
will be closer to that of evaluation.

After calibration, we fused our sub-systems for final submission. For system
fusion, we employed a simple linear fusion system using logistic regression.

##  4 CRSS sub-systems

We developed 7 sub-systems from 4 CRSS baselines that used SREs to train SVDA,
LDA and PLDA. Also, as described above, we developed 4 sub-systems with just
unlabeled data PLDA idea. The details of each system about data and techniques
they used are listed in Table 2 and Table 3. More specifically, sub-systems 8,
9, 10, 11 are respectively share the same configuration as sub-systems 3, 4,
6, 2; however, only unlabeled data are used to train PLDA.

Table 3:  Description of CRSS sub-systems using just unlabeled data to train PLDA. Sub-system | i-Vector | SVDA | LDA
---|---|---|---
8 | CRSS 1 | ✕ | ✓
9 | CRSS 2 | ✕ | ✓
10 | CRSS 3 | ✓ | ✓
11 | CRSS 4 | ✓ | ✓
Table 4: CRSS submission for NIST SRE2016. Submission | sub-systems | Calibration Data | Fusion
---|---|---|---
Primary | 1-7 | Dev+Unlabeled | LR
Contrastive1 | 1-7 | Dev | LR
Contrastive2 | 1-11 | Dev+Unlabeled | LR
Table 5:  Calculated scores for the single systems. With using different data
for calibration just the value of act-Cprimary will be changed. These scores
are calculated using NIST scoring software for the Dev set, the equalized and
unequalized scores are separated with – in the table.

Sub | EER/min-Cprimary | act-Cprimary
---|---|---
system |  | Calibrate With Dev | Calibrate With Unlabeled | Calibrate With Dev+Unlabeled
1 | 17.14/0.768 – 18.64/0.779 | 0.768 – 0.779 | 0.881 – 0.891 | 0.812 – 0.822
2 | 17.11/0.755 – 18.87/0.757 | 0.768 – 0.769 | 0.794 – 0.8 | 0.777 – 0.775
3 | 17.84/0.754 – 18.41/0.734 | 0.754 – 0.734 | 0.834 – 0.826 | 0.788 – 0.766
4 | 17.17/0.719 – 17.50/0.694 | 0.722 – 0.694 | 0.820 – 0.813 | 0.769 – 0.747
5 | 15.59/0.701 – 16.08/0.671 | 0.709 – 0.671 | 0.813 – 0.813 | 0.747 – 0.726
6 | 15.58/0.679 – 15.95/0.629 | 0.688 – 0.629 | 0.744 – 0.735 | 0.694 – 0.647
7 | 15.53/0.685 – 16.63/0.658 | 0.686 – 0.658 | 0.775 – 0.769 | 0.697 – 0.672

##  5 CRSS submissions

The final submissions of CRSS is the fusion of several sub-systems. In the
final submission, we tried different system combinations as well as different
calibration strategies. We submit a 1-7 sub-systems fusion with Dev+unlabeled
data for calibration as our primary submission. To test our hypothesis that
unlabeled PLDA idea will benefit for the Eval set, we submitted this as a
contrastive submission. All these combinations make our final submissions to
SRE 2016.

##  6 Performance of CRSS sub-systems on SRE 2016 Dev data

Tables 5, 6, and 7 shows the equal error rate (EER), minimum Cprimary(min-
Cprimary) and actual Cprimary (act-Cprimary) costs for single systems and
fusion systems using NIST scoring software. These results are evaluated on Dev
set.

Table 6: EER, min-Cprimary and act-Cprimary costs for single systems that only
use unlabeled data for training PLDA. NIST scoring software is used to
calculate the scores for Dev set. The equalized and unequalized scores are
separated with – in the table. For the calibration Dev + unlabeled data are
used.

Sub | EER/min-Cprimary | act-Cprimary
---|---|---
system |  |
8 | 29.48/0.901 – 26.84/0.9 | 0.917 – 0.914
9 | 29.72/0.898 – 26.15/0.908 | 0.92 – 0.933
10 | 26.48/0.943 – 24.96/0.954 | 0.956 – 0.96
11 | 26.76/0.957 – 26.45/0.968 | 0.986 – 0.989

Table 7:  Fusion scores calculated on Dev set with NIST scoring software. The
equalized and unequalized scores are separated with – in the table.

Submission | EER/minCprimary | act-Cprimary
---|---|---
Primary | 14.24/0.590 – 14.98/0.561 | 0.612 – 0.58
Contrastive1 | 13.81/0.585 – 14.66/0.554 | 0.589 – 0.56
Contrastive2 | 14.27/0.592 – 15.04/0.562 | 0.618 – 0.589

##  7 COMPUTATIONAL RESOURCES

###  7.1 CPU cluster

The speaker recognition system was implemented on our in-house high-
performance Dell computing cluster, running Rocks 6.0 (Mamba) Linux
distribution. The cluster comprises of eight 6C Intel Xeon 2.67 GHz CPU’s,
four 10C Intel Xeon 2.40 GHz CPU’s, and 18 quad-core Intel Xeon 2.33 GHz
CPU’s, yielding a total of 408 processors. The total amount of internal RAM on
the cluster exceeds 1 TB. All our data including audio files, features,
statistics, etc. are stored on a 30 TB Dell PowerVault MD1000 direct attached
storage.

###  7.2 GPU machines

For DNN training on SWB data, GeForce GTX TITAN Black graphic card is used,
6144 MB Ram. For DNN training on Fisher English, we used a 12 GB Tesla K40.

###  7.3 CPU execution time

We tested the system’s scoring process using one CPU of 2.67 GHz clock speed
and 32 GB RAM. We selected a 3 minute utterance (exact duration of 181.45
seconds) and calculated the time required to perform feature extraction (20
dimensional MFCC), voice activity detection (Kaldi SRE10/v1 default),
extraction of zero and first order statistics and the 600 dimensional
i-Vector. The time required for this chain of processes is for the selected
utterance is 37.58s. This is computed by averaging the elapsed time obtained
from three independent runs. Scoring an utterance using our PLDA model takes
less than 0.1 seconds on average. For training the models, it depends on how
many enrollment utterances are provided. Since the UBM and TV matrices are
trained off-line, speaker enrollment requires only to extract the
corresponding i-Vectors, thus the time required will be a multiple of the
number of enrollment utterances provided for a speaker.

##  8 Acknowledgement

We would like to thank Dr. Kong Aik Lee for providing the unlabeled trial
list, organizing I4U meetings, and other I4U group members sharing many ideas
and insights during I4U meetings. We would like to thank Qian Zhang, Abhinav
Misra, Dr. Finnian Kelly and other CRSS colleagues for their many insights and
helpful discussions in the development of the systems. We would like to thank
Dr. Gang Liu (CRSS graduate) from Alibaba Group for providing the fusion
system.

## References

  * [1] “NIST 2016 speaker recognition evaluation plan,” https://www.nist.gov/sites/default/files/documents/itl/iad/mig/SRE16_Eval_Plan_V1-0.pdf, 2016\.
  * [2] Y. Lei, N. Scheffer, L. Ferrer, and M. McLaren,  “A novel scheme for speaker recognition using a phonetically-aware deep neural network,”  in 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2014, pp. 1695–1699.
  * [3] S. O. Sadjadi, S. Ganapathy, and J. Pelecanos,  “The ibm 2016 speaker recognition system,”  arXiv preprint arXiv:1602.07291, 2016.
  * [4] S. O. Sadjadi, M. Slaney, and L. Heck,  “Msr identity toolbox v1. 0: A matlab toolbox for speaker-recognition research,”  Speech and Language Processing Technical Committee Newsletter, vol. 1, no. 4, 2013.
  * [5] D. Snyder, D. Garcia-Romero, and D. Povey,  “Time delay deep neural network-based universal background models for speaker recognition,”  in 2015 IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU). IEEE, 2015, pp. 92–97.
  * [6] C. Yu, C. Zhang, S. Ranjan, Q. Zhang, A. Misra, F. Kelly, and J. H.L. Hansen,  “Utd-crss system for the nist 2015 language recognition i-vector machine learning challenge,”  in 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2016, pp. 5835–5839.
  * [7] S. Gu, Y. Tan, and X. He,  “Discriminant analysis via support vectors,”  Neurocomputing, vol. 73, no. 10, pp. 1669–1675, 2010.
  * [8] F. Bahmaninezhad and J. H.L. Hanesn,  “i-vector/PLDA speaker recognition using support vectors with discriminant analysis,”  in (submitted to) IEEE ICASSP, 2017.
  * [9] N. Brümmer and E. de Villiers,  “The bosaris toolkit: Theory, algorithms and code for surviving the new dcf,”  arXiv preprint arXiv:1304.2865, 2013.