# Time-resolved quantum correlations in electronic noise

Jean-Olivier Simoneau  now at Nord Quantique, nordquantique.ca Sébastien
Jezouin  now at Alice $&$ Bob, alice-bob.com Christian Lupien  Bertrand Reulet
Département de physique, Institut quantique, Université de Sherbrooke,
Sherbrooke, Québec, Canada J1K 2R1

###### Abstract

The statistics of quantum transport in nanostructures can be tailored by a
time-dependent bias voltage $V(t)$. We demonstrate experimentally how
correlations of current fluctuations at two different times $t$ and $t+\tau$
depend on the shape of $V(t)$ via the phase accumulated by the electronic
wavefunctions between $t$ and $t+\tau$. For this we measure the current-
current correlation of the shot noise of an ac+dc biased tunnel junction using
a 10 GHz bandwidth, time-resolved detection. Our result allows to explore
correlations within a single excitation period. It demonstrates the
counterpart of the ac Josephson effect in superconducting junctions, to a
normal, non-superconducting mesoscopic device.

††preprint: APS/123-QED

Probing electron transport in coherent conductors beyond the average current
is essential to capture how quantum correlations affect the statistics of
electrical current. The effect of a time-dependent voltage $V(t)$ on the
current-current correlator has attracted a long lasting, both experimental and
theoretical interest. The current noise spectral density at frequency
$f=\omega/2\pi$, i.e. $S(\omega)=\expectationvalue{I(-\omega)I(\omega)}$ in
the presence of an ac excitation of various shape (sinusoidal[1], shifted
sinusoidal [2], bi-harmonic[3], Lorentzian[4]) at another frequency
$f_{0}=\Omega/2\pi$ has been observed in various quantum systems (tunnel
junction, diffusive wire, quantum Hall edge states, etc.) and interpreted in
terms of photo-assisted noise [1, 2], spectroscopy of the distribution
function [3, 4], as well as quantum tomography of electronic signals [5, 6].
The same ac excitation has been shown to induce correlations between currents
at different frequencies, captured by the noise susceptibilities
$\beta_{n}(\omega)=\expectationvalue{I(\omega)I(n\Omega-\omega)}$ with $n$
integer[7, 8], leading to the generation of squeezed vacuum in the microwave
domain[9]. All these experiments can be accounted for by a single, simple
formula which is enlightening in time-domain[10]. For a conductor with low
transmission like a tunnel junction, it reads:

$\expectationvalue{I(t)I(t+\tau)}=S_{\textrm{eq}}(\tau)\cos[\Phi(t,t+\tau)]$
(1)

with

$\Phi(t,t+\tau)=\frac{e}{\hbar}\int_{t}^{t+\tau}V(t^{\prime})dt^{\prime}$ (2)

the phase accumulated by electronic wave functions between $t$ and $t+\tau$
due to the applied voltage, and $S_{\textrm{eq}}(\tau)$ the current-current
correlator at equilibrium, independent of $t$ (for a general conductor, one
has to multiply the r.h.s. of Eq.(1) by the Fano factor $F$, and add the
voltage-independent contribution $(1-F)S_{\textrm{eq}}(\tau)$, to recover the
equilibrium noise when $V=0$). These formulas are direct consequences of the
voltage across the sample acting on the phase of the electronic
wavefunctions[11]. They bear strong similarities with Josephson equations for
the supercurrent $I_{s}(t)$ in a voltage-biased superconducting tunnel
junction: $I_{s}(t)=I_{c}\sin\Phi(t)$ with $\Phi(t)$ given by Eq.(2) where $e$
has to be replaced by $2e$, the charge of a Cooper pair, and the integral
spans from $-\infty$ to $t$. In a superconductor all the electrons have the
same phase and the action of the voltage on that phase is visible in the
average current. In a normal metal all electrons have different phases but the
voltage acts equally on all of them. As a consequence, the average current
follows Ohm’s law, but according to Eq.(1) the current-current correlator
reveals the direct coupling between the applied voltage and the phase of the
electronic wavefunctions (in the particular case of a ballistic system, a
single voltage pulse has been predicted to generate oscillations of the
current analoguous to the ac Josephson effect [12]). While causality implies
that the current-current correlator cannot depend on $V$ after $t+\tau$,
nothing prevents it to depend on the past of the voltage before $t$. Eq.(1)
claims that it does not: only the voltage between $t$ and $t+\tau$ matters. In
contrast, the supercurrent at any time in the Josephson junction depends on
the voltage at all times in the past.

Usual experiments, by averaging over time, have observed _consequences_ of
Eq.(1) integrated over $t$. In particular, the usual noise spectral density
$S(\omega)=\expectationvalue{I(\omega)I(-\omega)}$ corresponds to the Fourier
transform with respect to $\tau$ of $\expectationvalue{I(t)I(t+\tau)}$
averaged over the time $t$. From the frequency dependence of $S(\omega)$, the
$\tau$-dependence of the current-current correlator can be reconstructed,
leading to the observation of Pauli-Heisenberg oscillations for a time-
independent bias voltage[13]. But the dependence on $t$ cannot be recovered
from these experiments, and as far as we know Eq.(1) has never been
experimentally validated for a time-dependent voltage. Here we provide the
time-domain measurement of current-current correlators synchronously with an
ac sinusoidal excitation at frequency $f_{0}=4$GHz. This allows us to explore
correlations in current fluctuations inside a single excitation period, to
define phase-dependent noise and deduce noise susceptibilities and time /
frequency Wigner functions, all in agreement with predictions from Eq.(1).

_Phase-dependent noise._ In the following we show an experiment that aims at
demonstrating the validity of Eq.(1). In order to measure the average current-
current correlator, one must experimentally use a periodic excitation and take
the average over many repetitions of the same excitation. We have chosen to
work with a sine-wave excitation on which a dc voltage is superimposed, which
is enough for our goal: $V(t)=V_{\textrm{dc}}+V_{\textrm{ac}}\cos\Omega t$.
The current-current correlator $\expectationvalue{I(t)I(t+\tau)}$ is also
periodic in $t$, so the initial time $t$ plays the role of a phase
$\phi=\Omega t$. We define the phase-dependent noise
$S_{\phi}(\tau)=\expectationvalue{I(\phi/\Omega)I(\phi/\Omega+\tau)}$, which
can be expanded in terms of a Fourier series:

$S_{\phi}(\tau)=\sum_{n}\beta_{n}(\tau)e^{in\phi}$ (3)

The real functions $\textrm{Re}[\beta_{n}(\tau)]$ and
$\textrm{Im}[\beta_{n}(\tau)]$ represent the nth harmonic of the in-phase and
in-quadrature response of the noise to the ac excitation. We will also
consider the Fourier transform of the phase-dependent noise,
$S_{\phi}(\omega)$, which represents a phase-dependent spectrum. The noise
spectral density $S(\omega)$ usually considered in the literature corresponds
to the average of $S_{\phi}(\omega)$ over $\phi$. $S_{\phi}$ is
$2\pi$-periodic in $\phi$, and its Fourier coefficients are given by:
$\beta_{n}(\omega)=\expectationvalue{I(\omega)I(n\Omega-\omega)}$ i.e., they
are the noise susceptibilities, which measure the correlations between two
Fourier components of the current at frequencies separated by $n\Omega$ [7,
14]. And $\beta_{0}(\omega)=S(\omega)$.

Figure 1: Schematics of the experimental setup. The $\varphi$ symbol
represents a mechanical variable delay, the Att. symbol the attenuation along
the excitation line, which is distributed at different temperatures.

_Wigner distribution function._ While the definition of the phase-dependent
noise spectra presented above is quite natural and makes a clear link with the
noise susceptibilities, it suffers from being a complex quantity, unlike
$S(\omega)$, since in general $\expectationvalue{I(t)I(t+\tau)}$ is not an
even function of $\tau$. The Wigner distribution function $W(t,\omega)$ of the
time-dependent current $I(t)$, defined as the Fourier transform with respect
to $\tau$ of $\expectationvalue{I(t-\tau/2)I(t+\tau/2)}$ is also a real
quantity, closely related to $S_{\phi}(\omega)$. It is given by:

$W(t,\omega)=\sum_{n}M_{n}(\omega)e^{in\Omega t}$ (4)

with $M_{n}(\omega)=\beta_{n}(\omega+n\Omega/2)$. $W(t,\omega)$ is indeed real
since $\beta_{n}(\omega)=\beta_{n}(n\Omega-\omega)=\beta_{-n}(-\omega)^{*}$.
We discuss later its link with the electronic Wigner function, another
quantity that has been shown to be a key concept to perform tomography of
quantum signals[15, 5, 6, 16].

Figure 2: Voltage excess noise $\Delta S(\tau,V_{\textrm{dc}})$ as a function
of $eV_{\textrm{dc}}\tau/h=\Phi_{\textrm{dc}}/(2\pi)$ for various times $\tau$
in the absence of an ac excitation.

_Experiment._ The experiment, presented in Fig.1, is conceptually very simple:
the sample, a tunnel junction of dc resistance $40\Omega$, is ac+dc biased
through a microwave coupler and bias tee; the noise it generates is amplified,
digitized and recorded. From the time traces we calculate the correlators
$S_{\phi}(\tau)$. The ac excitation is at frequency $f_{0}=4$GHz and the
digitization takes place at a pace of 32GS/s, so we have access to 8 points
per cycle, i.e. 8 values of $\phi$ (or a time resolution of 31.25ps). For each
$\phi$ the current-current correlator is computed on the fly for 128 values of
$\tau$, i.e. from -2 to +2ns, which is much greater than the decay time of the
correlations [17]. It is then averaged over many repetitions. Since the data
averaging takes days, it is crucial to ensure that the digitization clock and
the excitation of the sample remain phase-locked over very long periods. To
achieve this we regularly measure the phase of the 4GHz excitation that is
reflected on the sample and digitized together with the sample noise. The
phase drift is calculated then corrected by a motorized mechanical phase
shifter on the excitation line. This ensures a phase variation of less than
one degree over weeks. A special care has to be taken in the calibration
process. As usual in noise measurements, the power gain and noise temperature
of the setup are obtained by applying a large dc voltage on the sample and
measuring noise vs. voltage[18]. In this limit the noise of the junction is
simply the classical shot noise $eV_{\textrm{dc}}/R$. But time domain
measurements are affected not only by the gain or attenuation of the setup,
but also by its phase response. Indeed, noise susceptibilities involve the
complex gain at different frequencies, and the frequency-dependent phase
response of the setup matters. To calibrate out that effect we again consider
the classical limit of shot noise. In this limit the noise of the junction
follows instantaneously the applied voltage, i.e.
$\expectationvalue{I(t)I(t+\tau)}=eV(t)\delta(\tau)/R$. Thus the noise
susceptibilities are real and independent of frequency:
$\beta_{n}(\omega)=e/R$. Measuring the noise susceptibilities in this regime
provides a calibration of the phase response of the experimental setup. The
averaged current-current correlator is deconvolved by the response function of
the setup to provide the calibrated experimental data shown in the following.

_Results: dc bias only._ In the absence of ac bias, $V_{\textrm{ac}}=0$, our
experiment allows for a wide bandwidth measurement of the noise spectral
density $S(\omega)$ vs. frequency in the presence of a dc voltage, as in [13].
From these data we can compute the so-called voltage excess noise $\Delta
S(\tau,V_{\textrm{dc}})=S(\tau,V_{\textrm{dc}})-S(\tau,V_{\textrm{dc}}=0)$,
with $S(\tau,V_{\textrm{dc}}=0)=S_{\textrm{eq}}(\tau)$. This quantity, once
put in time domain, has been shown theoretically a long time ago to oscillate
as[19]:

$\Delta
S(\tau,V_{\textrm{dc}})=-S_{\textrm{eq}}(\tau)\sin^{2}\Phi_{\textrm{dc}}$ (5)

which is nothing but a special case of Eq.(1) for a constant bias voltage.
$\Phi_{\textrm{dc}}$ is the phase generated by the dc voltage,
$\Phi_{\textrm{dc}}=eV_{\textrm{dc}}\tau/\hbar$. We provide the experimental
validation of this formula in Fig.2. These data have been obtained by sweeping
the dc voltage bias and measure the current-current correlations vs. $\tau$
for each voltage. They make it clear that the voltage enters via the flux
$\Phi_{\textrm{dc}}$. From these data, taking $\Phi_{\textrm{dc}}=\pi/2$
provides a measurement of the equilibrium noise in time domain. At zero
temperature, $S_{\textrm{eq}}(\tau)=-\hbar/\tau^{2}$. The divergence at short
time relates to the noise spectral density behaving as $G\hbar\omega$ at high
frequency. It corresponds to the zero-point motion of electrons.
$S_{\textrm{eq}}(\tau)$ is negative at short time as a result of the Pauli
exclusion principle.

Figure 3: Phase-dependent part of the time-domain noise $S_{\phi}(\tau)$ vs.
time $\tau$ for $V_{\textrm{dc}}=12.1\mu$V and $V_{\textrm{ac}}=3.63\mu$V.
Symbols linked by dashed lines are experimental data, solid lines are
theoretical prediction of Eq.(1) for $T=38$mK.

_Phase resolved noise._ To focus on the phase-dependence of the noise we
remove the phase-independent part $\beta_{0}=S$ to define, both in time- and
frequency domains, $\Delta_{\phi}S=S_{\phi}-S$, which is shown in Fig. 3 as a
function of $\tau$ for different phases $\phi$. Agreement between experiment
(symbols and dashed lines) and theoretical predictions of Eq.(1) (solid lines)
is very good.

In order to get a deeper insight into the observed behaviour of
$\Delta_{\phi}S(\tau)$, we calculate it for a small ac excitation and find:

$\Delta_{\phi}S(\tau)\simeq
S_{\textrm{eq}}(\tau)\Phi_{\textrm{ac}}(\phi,\tau)\sin\Phi_{\textrm{dc}}(\tau)$
(6)

It is a combination of an oscillation vs. the dc flux, an oscillation due to
the ac excitation, since
$\Phi_{\textrm{ac}}=z\sin(\Omega\tau/2)\cos(\phi+\Omega\tau/2)$, and a decay.
Here $z=eV_{\textrm{ac}}/(\hbar\Omega)$.

The theory in Fig. 3 is very close to Eq.(6) since $|\Phi_{\textrm{ac}}|\leq
z=0.24\ll 2\pi$. The curves show various symmetries: they are even functions
of $\tau$ for $\phi=0$ and $\pi$, and odd for $\phi=\pi/2$ and $3\pi/2$. This
comes from the symmetries in the excitation voltage:
$\cos(\phi+\Omega\tau)=\cos(\phi-\Omega\tau)$ for $\phi=0$ or $\pi$, and
$\cos(\phi+\Omega\tau)=-\cos(\phi-\Omega\tau)$ for $\phi=\pi/2$ or $3\pi/2$.
There is no particular symmetry for $\phi=\pi/4$. The curves have crossing
points. This comes from the total phase
$\Phi=\Phi_{\textrm{dc}}+\Phi_{\textrm{ac}}$ accumulated by the voltage being
the same for these curves, for example $\phi=0$ and $\phi=-\pi/4$ correspond
to the same phase for $\Omega\tau=\pi/4$. However in the first case the ac
voltage decays from its maximum value $V_{\textrm{ac}}$ to
$V_{\textrm{ac}}/\sqrt{2}$ whereas in the second it grows from
$V_{\textrm{ac}}/\sqrt{2}$ and keeps increasing to reach $V_{\textrm{ac}}$. In
both cases the past of the time-dependent voltage is totally different. Yet
$S_{\phi}$ is the same, in agreement with the fact that only the voltage
between $t$ and $t+\tau$ matters via the accumulated phase.

Figure 4: Measured real part, imaginary part and modulus of the time-domain
harmonic responses $\beta_{n}(\tau)$ as a function of the time $\tau$ for
$n=1$ and $V_{\textrm{dc}}=6.06\mu$V (left), and for $n=2$ and
$V_{\textrm{dc}}=0$ (right).

In Fig. 4 we show the real part, imaginary part and modulus of
$\beta_{n}(\tau)$, i.e. the noise susceptibility in time domain, for $n=1$ at
$V_{\textrm{dc}}\neq 0$ (left) and $n=2$ at $V_{\textrm{dc}}=0$ (right). At
equilibrium, $V(t)$ oscillates between $\pm V_{\textrm{ac}}$ at frequency
$\Omega$. One expects the noise to respond at twice that frequency, i.e.
$\beta_{1}=0$, thus we show the case $n=2$ at zero dc bias (we indeed observe
$\beta_{1}\simeq 0$). Both real and imaginary parts of $\beta_{n}(\tau)$ show
a fast oscillation modulated by a slow decay. The modulus $|\beta_{n}(\tau)|$
decays continuously at long times without oscillating. In order to understand
this behaviour we introduce the coefficients $M_{n}(\tau)$ of the Fourier
series decomposition of the Wigner function $W(t,\tau)$:

$W(t,\tau)=\sum_{n}M_{n}(\tau)e^{in\Omega t}$ (7)

$M_{n}(\tau)$ are the Fourier transform of $M_{n}(\omega)$, see Eq.(4). The
definitions of $S_{\phi}$ and $W$ imply:
$\beta_{n}(\tau)=M_{n}(\tau)\exp(in\Omega\tau/2)$. From Eq.(1), it is easy to
show that for an excitation that has a symmetry point in time, i.e.
$V(-t)=V(t)$, $M_{n}(\tau)$ is real. This implies that $\beta_{n}(\tau)$ is
purely real for $\tau=0$ and alternates from being purely real to purely
imaginary every $\tau=\pi/\Omega$, while $|M_{n}(\tau)|=|\beta_{n}(\tau)|$
acts as an envelope for the oscillation at frequency $n\Omega/2$ of
$\beta_{n}(\tau)$. This is precisely what is observed in Fig. 4 with
$\pi/\Omega=125$ps, i.e. half the period of $V(t)$. Oscillations occur indeed
twice as fast for $n=2$ (right) than for $n=1$ (left).

Figs. 3 and 4 report time-domain, direct measurements of the Wigner
distribution function of the shot noise generated by the tunnel junction. This
is the Wigner function of the detected _signal_. A few experiments have
achieved the measurement of the _electronic_ Wigner function $W_{e}(t,\omega)$
[5, 6] in quantum Hall edge channels, by measuring the shift in zero frequency
noise caused by the addition of a small ac sinusoidal signal at frequency
$n\Omega$, as suggested in [20]. The extra ac excitation causes a change in
$W(t,\tau)$, with a modulation in $\tau$ at frequency $n\Omega/2$, similar to
Eq.(6). From this the noise at zero frequency can be computed by averaging
over the time $t$ and integrating over $\tau$. While there is clearly a link
between $W_{e}$ and $W$, a theoretical effort is called for to make the
mathematically sound connection between them.

_Conclusion._ We have experimentally demonstrated how a time-dependent voltage
acts on the correlations in current fluctuations taken at different times, via
the phase accumulated by electrons. This is the corner stone for the
generation of quantum microwaves by electron quantum transport. While
squeezing and entanglement between electromagnetic fields at two different
frequencies has been demonstrated using ac-excited, non-superconducting
quantum conductors[9, 21, 22], our time-domain experiment may yield to the
detection of similar properties where the quadratures of the electromagnetic
field, given by sine and cosine at a given frequency, need to be extended to
time-domain [23] and the excitation voltage engineered accordingly, using
Eq.(1).

###### Acknowledgements.

We are very grateful to Gabriel Laliberté for his technical help and Eva
Dupont-Ferrier for lending us her microwave source. This work was supported by
the Canada Research Chair program, the NSERC, the Canada First Research
Excellence Fund, the FRQNT, and the Canada Foundation for Innovation.

## References

  * Schoelkopf  _et al._ [1998] R. J. Schoelkopf, A. A. Kozhevnikov, D. E. Prober, and M. J. Rooks, Observation of “photon-assisted” shot noise in a phase-coherent conductor, Phys. Rev. Lett. 80, 2437 (1998).
  * Gasse  _et al._ [2013a] G. Gasse, L. Spietz, C. Lupien, and B. Reulet, Observation of quantum oscillations in the photoassisted shot noise of a tunnel junction, Phys. Rev. B 88, 241402 (2013a).
  * Gabelli and Reulet [2013] J. Gabelli and B. Reulet, Shaping a time-dependent excitation to minimize the shot noise in a tunnel junction, Phys. Rev. B 87, 075403 (2013).
  * Dubois  _et al._ [2013] J. Dubois, T. Jullien, F. Portier, P. Roche, A. Cavanna, Y. Jin, W. Wegscheider, P. Roulleau, and D. Glattli, Minimal-excitation states for electron quantum optics using levitons, Nature 502, 659 (2013).
  * Jullien  _et al._ [2014] T. Jullien, P. Roulleau, B. Roche, A. Cavanna, Y. Jin, and D. Glattli, Quantum tomography of an electron, Nature 514, 603 (2014).
  * Bisognin  _et al._ [2019] R. Bisognin, A. Marguerite, B. Roussel, M. Kumar, C. Cabart, C. Chapdelaine, A. Mohammad-Djafari, J.-M. Berroir, E. Bocquillon, B. Plaçais, A. Cavanna, U. Gennser, Y. Jin, P. Degiovanni, and G. Fève, Quantum tomography of electrical currents, Nature Comm. 10, 3379 (2019).
  * Gabelli and Reulet [2008] J. Gabelli and B. Reulet, Dynamics of quantum noise in a tunnel junction under ac excitation, Phys. Rev. Lett. 100, 026601 (2008).
  * Farley  _et al._ [2023] C. Farley, E. Pinsolle, and B. Reulet, Noise dynamics in the quantum regime, in  _2023 International Conference on Noise and Fluctuations (ICNF)_ (2023) pp. 1–4.
  * Gasse  _et al._ [2013b] G. Gasse, C. Lupien, and B. Reulet, Observation of squeezing in the electron quantum shot noise of a tunnel junction, Phys. Rev. Lett. 111, 136601 (2013b).
  * Altshuler  _et al._ [1994] B. Altshuler, L. Levitov, and A. Yakovets, Nonequilibrium noise in a mesoscopic conductor : A microscopic analysis, JETP Letters 59(12), 857 (1994).
  * Tien and Gordon [1963] P. K. Tien and J. P. Gordon, Multiphoton process observed in the interaction of microwave fields with the tunneling between superconductor films, Phys. Rev. 129, 647 (1963).
  * Gaury  _et al._ [2015] B. Gaury, J. Weston, and X. Waintal, The a.c. josephson effect without superconductivity, Nat Commun 6, 6524 (2015).
  * Thibault  _et al._ [2015] K. Thibault, J. Gabelli, C. Lupien, and B. Reulet, Pauli-heisenberg oscillations in electron quantum transport, Phys. Rev. Lett. 114, 236604 (2015).
  * Gabelli and Reulet [2007] J. Gabelli and B. Reulet, The noise susceptibility of a coherent conductor, in  _Noise and Fluctuations in Circuits, Devices, and Materials_, Vol. 6600, International Society for Optics and Photonics (SPIE, 2007) p. 66000T.
  * Ferraro  _et al._ [2013] D. Ferraro, A. Feller, A. Ghibaudo, E. Thibierge, E. Bocquillon, G. Fève, C. Grenier, and P. Degiovanni, Wigner function approach to single electron coherence in quantum hall edge channels, Phys. Rev. B 88, 205303 (2013).
  * Roussel  _et al._ [2021] B. Roussel, C. Cabart, G. Fève, and P. Degiovanni, Processing quantum signals carried by electrical currents, PRX Quantum 2, 020314 (2021).
  * [17] https://github.com/JeanOlivier/aCorrs-OTF, .
  * Spietz  _et al._ [2003] L. Spietz, K. W. Lehnert, I. Siddiqi, and R. J. Schoelkopf, Primary electronic thermometry using the shot noise of a tunnel junction, Science 300, 1929 (2003).
  * Lesovik  _et al._ [2005] G. B. Lesovik, A. V. Lebedev, and G. Blatter, Wave function collapse in a mesoscopic device, Phys. Rev. B 71, 125313 (2005).
  * Grenier  _et al._ [2011] C. Grenier, R. Hervé, E. Bocquillon, F. D. Parmentier, B. Plaçais, J. M. Berroir, G. Fève, and P. Degiovanni, Single-electron quantum tomography in quantum hall edge channels, New Journal of Physics 13, 093007 (2011).
  * Forgues  _et al._ [2015] J.-C. Forgues, C. Lupien, and B. Reulet, Experimental violation of bell-like inequalities by electronic shot noise, Phys. Rev. Lett. 114, 130403 (2015).
  * Bartolomei  _et al._ [2023] H. Bartolomei, R. Bisognin, H. Kamata, J.-M. Berroir, E. Bocquillon, G. Ménard, B. Plaçais, A. Cavanna, U. Gennser, Y. Jin, P. Degiovanni, C. Mora, and G. Fève, Observation of edge magnetoplasmon squeezing in a quantum hall conductor, Phys. Rev. Lett. 130, 106201 (2023).
  * Virally and Reulet [2019] S. Virally and B. Reulet, Unidimensional time-domain quantum optics, Phys. Rev. A 100, 023833 (2019).


# A Refining Underlying Information Framework for Monaural Speech Enhancement

Rui Cao, Tianrui Wang, Meng Ge∗, Longbiao Wang∗, , and Jianwu Dang Rui Cao and
Tianrui Wang are with the Tianjin Key Laboratory of Cognitive Computing and
Application, College of Intelligence and Computing, Tianjin University,
Tianjin 300354, China (e-mail<EMAIL_ADDRESS>wangtianrui@tju.edu.cn).Meng Ge is with the Saw Swee Hock School of Public
Health, National University of Singapore, Singapore 117549 (e-mail:
gemeng@nus.edu.sg).Longbiao Wang is with the Tianjin Key Laboratory of
Cognitive Computing and Application, College of Intelligence and Computing,
Tianjin University, Tianjin 300354, China, and also with Huiyan Technology
(Tianjin) Company, Ltd., Tianjin 300384, China (e-mail:
longbiao_wang@tju.edu.cn).Jianwu Dang is with the Tianjin Key Laboratory of
Cognitive Computing and Application, College of Intelligence and Computing,
Tianjin University, Tianjin 300354, China, and also with the Pengcheng
Laboratory, Shenzhen 518000, China (e-mail: jdang@jaist.ac.jp). ∗
Corresponding author.

###### Abstract

Supervised speech enhancement has gained significantly from recent
advancements in neural networks, especially due to their ability to non-
linearly fit the diverse representations of target speech, such as waveform or
spectrum. However, these direct-fitting solutions continue to face challenges
with degraded speech and residual noise in hearing evaluations. By bridging
the speech enhancement and the Information Bottleneck principle in this
letter, we rethink a universal plug-and-play strategy and propose a Refining
Underlying Information framework called RUI to rise to the challenges both in
theory and practice. Specifically, we first transform the objective of speech
enhancement into an incremental convergence problem of mutual information
between comprehensive speech characteristics and individual speech
characteristics, e.g., spectral and acoustic characteristics. By doing so,
compared with the existing direct-fitting solutions, the underlying
information stems from the conditional entropy of acoustic characteristic
given spectral characteristics. Therefore, we design a dual-path multiple
refinement iterator based on the chain rule of entropy to refine this
underlying information for further approximating target speech. Experimental
results on DNS-Challenge dataset show that our solution consistently improves
0.3+ PESQ score over baselines, with only additional 1.18 M parameters. The
source code is available at https://github.com/caoruitju/RUI_SE.

###### Index Terms:

Monaural speech enhancement, information bottleneck, acoustic characteristics
of speech.

##  I Introduction

Monaural speech enhancement focuses on extracting the target speech from the
corresponding noisy recording, aiming to improve both the quality and
intelligibility of speech. Traditional speech enhancement methods usually rely
on mathematical formulas derived from assumptions about the statistical
characteristics of speech and noise, such as spectral subtraction [1], Wiener
filtering [2] and subspace-method [3]. Recent data-driven speech enhancement
methods have achieved significant performance gains by employing neural
networks to fit the non-linear mapping relationship between the input noisy
speech and the target speech, avoiding the statistical assumptions in
traditional methods. The existing data-driven methods can be categorized into
two streams, namely the time domain and the (time-frequency)-domain. Time-
domain methods [4, 5, 6] aims to model the distribution of waveform samples
via convolution-based speech encoding and decoding operations, while (time-
frequency)-domain methods [7, 8, 9, 10, 11, 12] work by separating noisy
complex spectrum into either magnitude-phase or real-imaginary components to
minimize the distance to the complex spectrum of target speech. However, these
data-driven direct-fitting methods continue to face difficulties in addressing
speech over-suppression and noise under-suppression [13, 14], which can
negatively impact hearing evaluations, especially in challenging acoustic
scenarios.

Recent related studies [15, 16, 17, 18] have shown that preserving acoustic
characteristics inherent in speech (e.g., articulatory attributes, acoustic
structural features, and auditory perceptual attributes[19, 20, 21, 22, 23,
24]) can ensure that enhanced speech remains consistent with human hearing
perception. These acoustic characteristics are closely linked to speech
production and reflect the fundamental characteristic of natural human speech.
Correcting these acoustic structures in speech processing makes the enhanced
speech sound more natural and familiar to human listeners. Motivated by this
fact, we rethink the speech enhancement task and prompt a research question:
Could we theoretically repair the incomplete intrinsic characteristics in
enhanced speech (e.g., articulatory attributes) to perfectly approximate the
hearing perception on target speech?

To answer this research question, we propose a universal hearing-repair speech
enhancement framework, called Refining Underlying Information (RUI). Our RUI
is inspired by the Information Bottleneck principle [25]. We find similarities
between speech enhancement task and Information Bottleneck principle, with
both primarily aiming to compress non-target information (i.e., noises) while
capturing the most relevant information (i.e., target speech) for the target
object. Motivated by this, we transform the objective of speech enhancement
into an incremental convergence process of mutual information among speech
characteristics shown in Fig. 1, including spectral characteristic
$\mathbb{P}$, acoustic characteristic $\mathbb{A}$, and comprehensive
characteristic $\mathbb{C}$. Thus, it can be further derived that the
optimization objective is essentially the sum of the entropy $H(\mathbb{P})$
and the conditional entropy $H(\mathbb{A|P})$. Existing direct-fitting methods
pay more attention to how to model $H(\mathbb{P})$, while our solution further
explore the underlying information originating from $H(\mathbb{A|P})$. This
theoretically answers how our proposed RUI can achieve the repair of
characteristics inherent in target speech.

Based on the above discussion, we employ a pre-enhancement module and an
underlying information extractor to explore $H(\mathbb{P})$ and
$H(\mathbb{A})$, respectively. Additionally, we iteratively expand the
conditional entropy according to the chain rule of entropy and design a
multiple refinement iterator through dual-path residual mechanism for modeling
$H(\mathbb{A|P})$. By doing so, the intrinsic characteristics in target speech
are asymptotically refined in the output-enhanced speech, gradually improving
the hearing perception in practice.

##  II Methodology

###  II-A Information Bottleneck in Speech Enhancement

The Information Bottleneck principle suggests that DNN should learn to extract
the most efficient informative representation in the input variable about the
output-label-variable and maximally compress irrelevant representation.
Overall, it exhibits significant consistency with speech enhancement, where
the goal is to extract the target speech from the noisy speech and suppress
noise as much as possible [26]. Derived from the Information Bottleneck
principle, the universal optimization process of DNN-based speech enhancement
can be formulated as the minimization of the following Lagrangian,

$\mathcal{L}=I({X;\hat{S}})-\beta I({\hat{S};S})$ (1)

where $X$, $\hat{S}$, and $S$ represent noisy speech, enhanced speech, and
clean speech, respectively. $I(\cdot;\cdot)$ denotes mutual information.
$\beta$ represents the positive Lagrange multiplier. Given the restrictions of
accurately modeling the infinite noise, minimizing the gap between $\hat{S}$
and $S$ is typically formulated as the optimization objective $\Gamma$ of
supervised speech enhancement [27]. However, as a bond between the suppression
of noise $I({X;\hat{S}})$ and the recovery of clean speech $I({\hat{S};S})$,
$\hat{S}$ is subjected to the unclear boundary between speech and noise during
the trade-off process, causing degraded speech and residual noise. We hold
that the occurrence of these issues is due to a lack of effective utilization
of acoustic characteristics of speech, as speech possesses distinct acoustic
characteristics that differentiate it from noise [16]. Explicitly modeling
them contributes to facilitating the formation of speech-specific feature
boundary. Therefore, we delve deeply into a universal way of incorporating the
acoustic characteristic (e.g., articulatory attributes) inherent in speech
signals. Essentially, we extend the recovery of clean speech to the
incremental convergence of the mutual information between the characteristic
information $\bm{c}$ of speech,

$\mathcal{F}(X;\mathbb{P})=\bm{c}^{\hat{S}},\mathcal{F}(X;\mathbb{C})=\bm{c}^{S}\vspace{-0.06cm}$
(2) $maximize\quad\Gamma=I({\bm{c}^{\hat{S}};\bm{c}^{S}})\vspace{-0.06cm}$ (3)

where $\mathcal{F}(\cdot;\mathbb{P})$ denotes the regression DNN $\mathcal{F}$
that performs the optimization process in Eq. (1) only in conjunction with
spectral characteristic $\mathbb{P}$. However, the existing spectrum
estimation approaches are inadequate to achieve the comprehensive
characteristic $\mathbb{C}$ of clean speech. This incompleteness of
characteristic results in the optimization objective converging to a supremum,

$sup\ \Gamma=I(\mathbb{P};\mathbb{C})\vspace{-0.06cm}$ (4)

The acoustic characteristic inherent in speech signals is represented as
$\mathbb{A}$. The specific interrelationships between each characteristic are
illustrated in Fig. 1. From the perspective of incremental convergence, the
upgraded upper bound $\Gamma_{u}$ after explicitly incorporating $\mathbb{A}$
can be formulated as,

$\begin{split}\Gamma_{u}&=I(\mathbb{P};\mathbb{C}|\mathbb{A})+I(\mathbb{P};\mathbb{A};\mathbb{C})+I(\mathbb{A};\mathbb{C}|\mathbb{P})\\\
&=I(\mathbb{P};\mathbb{C})+I(\mathbb{A};\mathbb{C}|\mathbb{P})\end{split}$ (5)

We formally define the conditional mutual information
$I(\mathbb{A};\mathbb{C}|\mathbb{P})$ as underlying information in this
letter. This is a crucial element for approximating the optimal information
theoretic limit of the optimization. The direct introduction of underlying
information in $\hat{S}$ also goes a step further to promoting Eq. (1), for
enhancing the essence of speech and eliminating noise impurity.

Figure 1: Venn diagram illustrating specific relationships between each
characteristic. The region overlapping with the red circle represents the
upgraded upper bound $\Gamma_{u}$. The green, blue, and red circles represent
characteristics $\mathbb{P}$, $\mathbb{A}$, and $\mathbb{C}$. Figure 2: The
proposed Refining Underlying Information Framework.

###  II-B Refining Underlying Information Framework

For the feedforward computation, the mutual information in Eq. (5) degenerates
into entropy. Decoupling the spectrum has been validated as an effective
strategy for obtaining sparse term [28, 29, 30, 31]. Motivated by this, to
thoroughly refine the underlying information, we iteratively expand the second
term according to the chain rule of entropy, which can be formulated as,

$\mathcal{F}_{u}(X)=H(\mathbb{P})+H(\mathbb{A|P})=H(\mathbb{P})+\sum^{N}_{i=1}H(\mathbb{A}_{i}|\mathbb{P},...,\mathbb{A}_{i-1})$
(6)

To indicate the direction of information in the framework, we use the concept
of information flow [32] as an intuitive alternative of Shannon entropy. In
this way, we design a refining underlying information framework (RUI) as shown
in Fig. 2, and the forward flow $\mathcal{F}_{u}(X)$ can be further
parameterized as follows,

$\mathcal{F}_{u}(X)=\bm{p}+\sum^{N}_{i=1}R_{i}(\bm{a},\bm{p}-\sum^{i-1}_{j=1}\bm{f}_{j})$
(7)

where $\bm{p}$, $\bm{a}$, and $\bm{f}_{i}$ represent the information flows
after passing through the pre-enhancement module (PEM), underlying information
extractor (UIE), and $i$-th refinement $R_{i}$ of multiple refinement iterator
(MRI). $\bm{p}-\sum^{i-1}_{j=1}\bm{f}_{j}$ imitates the condition term of
conditional entropy in Eq. (6).

The input of RUI is the noisy complex spectrum computed by Short-Time Fourier
Transform (STFT), denoted as $X\in\mathbb{R}^{T\times 2\cdot F}$, where $T$
and $2\cdot F$ denote the number of time frames and frequency bins (real and
imaginary part). Any complex-spectrum approach can serve as the PEM, and its
result $\bm{p}\in\mathbb{R}^{T\times 2\cdot F}$ is considered as a preliminary
enhanced output. To explicitly repair the incomplete articulatory attributes
in PEM, our previously proposed Harmonic
Attention111https://github.com/caoruitju/RUI_SE/blob/main/HA/HA.md [33] is
employed as the UIE to actively capture the comb-like harmonics based on the
correlation between the noisy spectrum and a comb-pitch conversation matrix,
resulting in the flow $\bm{a}\in\mathbb{R}^{T\times C\times F}$ with the
highlighted acoustic structural information. Finally, MRI performs structured
correction of flow $\bm{p}$ with the assistance of flow $\bm{a}$, through a
dual-path residual mechanism (S-path and A-path). Specifically, The S-path
filters out the refined information $\bm{f}_{j}\in\mathbb{R}^{T\times 2\cdot
F}$ from flow $\bm{p}$, formulated as $\bm{p}-\sum\bm{f}_{j}$, dynamically
regulating the underlying information absorbed from flow $\bm{a}$. This
gradually widening divergence between the two flows ensures the interaction of
complementary information. Benefiting from the channel information integration
capability of Harmonic Attention, we also employ it in each refinement $R$. In
this way, $\bm{p}-\sum\bm{f}_{j}$ and $\bm{a}$ are concatenated in channel
dimension, and fed into refinement $R_{i}$ to reconstruct the complete refined
information. Through the A-path, the information flow $\bm{p}$ is skip-
connected with each refined information $\bm{f}_{i}$ to form the final output
of RUI $\bm{p}+\sum\bm{f}_{i}$. The inclusion of skip connection in the A-path
ensures direct optimization of each flow.

###  II-C Auditory Constraint.

The integration of the articulatory attributes into the framework, along with
the incorporation of corresponding auditory perceptual attributes in the loss
function, can be viewed as mutually reinforcing elements. Therefore, we
introduce a combination of time-domain sample-level SI-SNR [34] and human
psychoacoustic perception-based PMSQE [35] as the loss function
$\mathcal{L}_{AC}=\mathcal{L}_{SI-SNR}+\mathcal{L}_{PMSQE}$ for the entire
framework. This ensures the gradient-based optimization encompasses not only
the numerical approximation but also the additional constraints of auditory
masking and threshold effects as another kind of underlying information.

##  III Experiments

###  III-A Experimental Setup

We conduct experiments on the 2020 DNS Challenge [36], which has 500 hours of
clean speech from 2150 speakers and over 180 hours of noise from 150 classes.
We generate 100 hours of noisy speech for the ablation and flexibility
experiments, and 300 hours for the final comparison, respectively. The signal-
to-noise ratio (SNR) ranges from -5 dB to 20 dB. The dataset is partitioned
into a training set and a validation set in a 4:1 ratio. For testing audio,
the SNR is between -5 dB and 30 dB, comprising a total of 10 hours of noisy
speech.

The 32 ms Hanning window with 25$\%$ overlap and 512-point STFT are used. The
channel number of each refinement is 14. Model training is conducted using
PyTorch with the Adam optimizer. The initial learning rate is set to 0.001,
and a 0.75 learning rate decay will be applied if the validation loss does not
decrease for 3 consecutive epochs. To provide a comprehensive assessment, we
employ the number of the parameters (Para.), perceptual evaluation of speech
quality (PESQ) [37], scale-invariant signal-to-distortion ratio (SI-SDR) [38],
and short-time objective intelligibility measure (STOI) [39]. For the last
three metrics, higher values indicate better performance.

###  III-B Ablation Studies

We utilize DPCRN [40] as the PEM for ablation studies. As shown in TABLE I,
with the number of refinements $i$ increasing from 1 to 4, a trend of
initially rising followed by a slight decrease is observed in the objective
evaluations. This can be attributed to the S-path of multiple refinement
iterator, where the remaining information obtained from flow $\bm{a}$ of Eq.
(7) tends to become saturated. Concretely, the effective correction for the
acoustic structure becomes limited. It is noteworthy that different PEMs lead
to diverse information flows $\bm{p}$ of Eq. (7), consequently causing
variations in the maximum value of $i$.

Furthermore, we increase the number of convolutional channels in PEM to be
comparable in parameters with RUI performing 4 times of refinements, denoted
as PEM (large). The experimental result validates that blindly increasing the
parameters of model is not advisable, the reasonable utilization of
articulatory attributes is helpful to achieve better performance with fewer
additional parameters.

We also compare the auditory constraint (AC) with the SI-SNR loss function.
The notable improvement underscores the complementarity of auditory and
articulatory attributes in acoustic modeling. They reinforce each other within
the established framework.

In addition, removing UIE results in an obvious drop in objective evaluations,
which directly confirms the necessity of the introduced underlying information
in our framework. Without the guidance of acoustic modeling based on
articulatory attributes, the MRI cannot effectively perform structured
correction on the output of PEM.

TABLE I: Ablation studies. | $i$ | Para. (M) | PESQ | SI-SDR (dB) | STOI ($\%$)
---|---|---|---|---|---
PEM (w/ $i$-ref) | 0 | 1.64 | 2.771 | 20.389 | 94.40
1 | 1.97 | 2.935 | 21.219 | 95.02
2 | 2.25 | 2.941 | 21.262 | 95.14
3 | 2.54 | 2.958 | 21.285 | 95.06
4 | 2.82 | 2.909 | 21.113 | 94.92
PEM (large) | 0 | 2.95 | 2.774 | 20.380 | 94.39
RUI | 3 | 2.54 | 3.072 | 21.520 | 95.36
\- AC | 3 | 2.54 | 2.958 | 21.285 | 95.06
\- UIE | 3 | 2.47 | 2.687 | 19.943 | 93.96

###  III-C Flexibility of RUI

Besides DPCRN, we additionally select two spectrum estimation strategies for
further exploration, including the baseline method NSNet [41] from the 2020
DNS challenge and CRN [10], a complex spectral mapping method. NSNet optimizes
only the magnitude, while CRN estimates both the real and imaginary parts to
enhance the magnitude and phase responses of noisy speech. As shown in TABLE
II, the flexibility of RUI enables it to better adapt to different PEMs.
Specifically, it reports that the extent of improvement $\Delta$ exhibits
variations. The retention of noisy phase in NSNet makes its supremum
$I(\mathbb{P};\mathbb{C})$ lower than CRN. After refinement under the same
condition, the lower supremum of NSNet, acting on the condition term
$\mathbb{P}$ of underlying information $I(\mathbb{A};\mathbb{C}|\mathbb{P})$,
leads to a higher extent of improvement $\Delta$. However, the obtained
underlying information necessitates a greater trade-off in ameliorating the
matter caused by retaining the noisy phase. In the case of CRN, the obtained
underlying information is utilized more sufficiently to finely recover the
acoustic structure of clean speech, culminating in a higher upgraded upper
bound $\Gamma_{u}$ of RUI.

TABLE II: Flexibility experiments. | Para. (M) | PESQ | SI-SDR (dB) | STOI ($\%$)
---|---|---|---|---
NSNet | 2.79 | 2.274 | 17.023 | 91.06
RUI [NSNet] | 3.97 | 2.829 | 20.460 | 94.56
$\Delta$ | 1.18 | 0.555 | 3.437 | 3.50
CRN | 1.70 | 2.732 | 20.295 | 94.31
RUI [CRN] | 2.88 | 3.034 | 21.304 | 95.23
$\Delta$ | 1.18 | 0.302 | 1.009 | 0.92

###  III-D Visual Analysis

To conduct a more in-depth analysis of information flow and the specific
implication of refinement, we visualize the output of each module and RUI, in
the final comparison, as shown in Fig. 3. Despite the noticeable improvement
in the output of PEM, there are still issues related to noise residual and
speech degradation (colorful boxes in Fig. 3(c)). Fig. 3(d) shows that RUI not
only removes the residual noise in the white box but also enhances the
harmonic structure of speech in the green box, actively correcting the
acoustic structural features of speech through articulatory attributes. The
output of each refinement not only presents sparse representations of the
spectrum, but also systematic harmonic structures, from the global level to
different frequency bands.

(a) Noisy

(b) Clean

(c) PEM

(d) RUI

(e) 1st-ref

(f) 2nd-ref

(g) 3rd-ref

(h) 4th-ref

Figure 3: Noisy and clean spectrograms are provided as reference. (c)-(h)
represent the output visualizations of each module and RUI, the $i$-th
refinement is denoted as $i$th-ref.

###  III-E Comparison on Public Test Dataset

Compared to the outstanding solutions on the public test set of DNS-Challenge
2020, as shown in TABLE III, our proposed RUI, using simple CRN as the
backbone of PEM achieves competitive performance with minimal model
parameters. The superiority of the proposed framework over the complex-
spectrum methods (e.g., DCCRN/DCCRN+) is predictable. For the FullSubNet+ with
8.67 M parameters, RUI achieves better performance with only 33$\%$ of its
parameters. For the HGCN, which only performs harmonic compensation, our
solution shows comprehensive improvement in objective evaluations. It is also
superior to the approach that combines DNN with a speech production model
(GARNNHS). Moreover, in terms of the SI-SDR, our RUI outperforms an advanced
method of hierarchical optimization in the complex spectrum (GAGNet), by a
significant margin.

TABLE III: System comparison on DNS-Challenge 2020 no reverb test set. “-” denotes no published result. Model | Para. (M) | $\text{PESQ}_{\text{WB}}$ | $\text{PESQ}_{\text{NB}}$ | SI-SDR (dB) | STOI ($\%$)
---|---|---|---|---|---
Noisy | - | 1.58 | 2.45 | 9.07 | 91.52
DCCRN [12] | 3.70 | - | 3.27 | - | -
DCCRN+ [42] | 3.30 | - | 3.33 | - | -
FullSubNet [43] | 2.97 | 2.78 | 3.31 | 17.29 | 96.11
FullSubNet+ [44] | 8.67 | 2.98 | 3.50 | 18.34 | 96.69
HGCN [16] | - | 2.88 | - | 18.14 | 96.50
CARNNHS [18] | - | 2.89 | 3.43 | 18.80 | 96.70
GaGNet [29] | 5.94 | 3.17 | 3.56 | 18.91 | 97.13
RUI (Ours) | 2.88 | 3.02 | 3.50 | 19.54 | 97.11

##  IV Conclusion

In this letter, we rethink the speech enhancement via Information Bottleneck
principle, theoretically and practically. We point out that the prevalent
noise suppression issues in existing methods stem from the incomplete
restoration of the characteristics inherent in speech, especially acoustic
characteristic. Theoretically, by defining the recovery of clean speech as
incremental convergence of mutual information, we further express the acoustic
characteristic of speech as conditional mutual information, (i.e., underlying
information). Such a perspective can facilitate understanding and provide
guidance for algorithmic design in speech enhancement. In practice, to ensure
that the underlying information can be fully refined, we propose a universal
framework called RUI with a dual-path residual mechanism, referring to the
chain rule of entropy. Experimental results demonstrate that our solution has
achieved highly competitive performance against other advanced methods, with a
minimal number of parameters.

## References

  * [1] K. Paliwal, K. Wójcicki, and B. Schwerin, “Single-channel speech enhancement using spectral subtraction in the short-time modulation domain,” _Speech communication_ , vol. 52, no. 5, pp. 450–475, 2010.
  * [2] J. Chen, J. Benesty, Y. Huang, and S. Doclo, “New insights into the noise reduction wiener filter,” _IEEE/ACM Trans. Audio, Speech, Lang. Process._ , vol. 14, no. 4, pp. 1218–1234, 2006.
  * [3] F. Asano, S. Hayamizu, T. Yamada, and S. Nakamura, “Speech enhancement based on the subspace method,” _IEEE transactions on speech and audio processing_ , vol. 8, no. 5, pp. 497–507, 2000.
  * [4] Y. Luo and N. Mesgarani, “Conv-tasnet: Surpassing ideal time–frequency magnitude masking for speech separation,” _IEEE/ACM Trans. Audio, Speech, Lang. Process._ , vol. 27, no. 8, pp. 1256–1266, 2019.
  * [5] H. R. Guimarães, H. Nagano, and D. W. Silva, “Monaural speech enhancement through deep wave-u-net,” _Expert Systems with Applications_ , vol. 158, p. 113582, 2020.
  * [6] Z. Kong, W. Ping, A. Dantrey, and B. Catanzaro, “Speech denoising in the waveform domain with self-attention,” in _Proc. IEEE Int. Conf. Acoust., Speech Signal Process._ , 2022, pp. 7867–7871.
  * [7] P. C. Loizou, “Speech enhancement based on perceptually motivated bayesian estimators of the magnitude spectrum,” _IEEE transactions on speech and audio processing_ , vol. 13, no. 5, pp. 857–869, 2005.
  * [8] Y. Wang, A. Narayanan, and D. Wang, “On training targets for supervised speech separation,” _IEEE/ACM Trans. Audio, Speech, Lang. Process._ , vol. 22, no. 12, pp. 1849–1858, 2014.
  * [9] K. Paliwal, K. Wójcicki, and B. Shannon, “The importance of phase in speech enhancement,” _speech communication_ , vol. 53, no. 4, pp. 465–494, 2011.
  * [10] K. Tan and D. Wang, “Complex spectral mapping with a convolutional recurrent network for monaural speech enhancement,” in _Proc. IEEE Int. Conf. Acoust., Speech Signal Process._ , 2019, pp. 6865–6869.
  * [11] ——, “Learning complex spectral mapping with gated convolutional recurrent networks for monaural speech enhancement,” _IEEE/ACM Trans. Audio, Speech, Lang. Process._ , vol. 28, pp. 380–390, 2019.
  * [12] Y. Hu, Y. Liu, S. Lv, M. Xing, S. Zhang, Y. Fu, J. Wu, B. Zhang, and L. Xie, “Dccrn: Deep complex convolution recurrent network for phase-aware speech enhancement,” in _Proc. Interspeech_ , 2020, pp. 2472–2476.
  * [13] W. Jiang, Z. Liu, K. Yu, and F. Wen, “Speech enhancement with neural homomorphic synthesis,” in _Proc. IEEE Int. Conf. Acoust., Speech Signal Process._ , 2022, pp. 376–380.
  * [14] Z. Kong, W. Ping, A. Dantrey, and B. Catanzaro, “Cleanunet 2: A hybrid speech denoising model on waveform and spectrogram,” in _Proc. Interspeech_ , 2023, pp. 790–794.
  * [15] D. Yin, C. Luo, Z. Xiong, and W. Zeng, “Phasen: A phase-and-harmonics-aware speech enhancement network,” in _Proceedings of the AAAI Conference on Artificial Intelligence_ , vol. 34, no. 05, 2020, pp. 9458–9465.
  * [16] T. Wang, W. Zhu, Y. Gao, J. Feng, and S. Zhang, “Hgcn: Harmonic gated compensation network for speech enhancement,” in _Proc. IEEE Int. Conf. Acoust., Speech Signal Process._ , 2022, pp. 371–375.
  * [17] T. Wang, W. Zhu, Y. Gao, Y. Chen, J. Feng, and S. Zhang, “Harmonic gated compensation network plus for icassp 2022 dns challenge,” in _Proc. IEEE Int. Conf. Acoust., Speech Signal Process._ , 2022, pp. 9286–9290.
  * [18] S. He, W. Rao, J. Liu, J. Chen, Y. Ju, X. Zhang, Y. Wang, and S. Shang, “Speech enhancement with intelligent neural homomorphic synthesis,” in _Proc. IEEE Int. Conf. Acoust., Speech Signal Process._ , 2023, pp. 1–5.
  * [19] C. H. Coker, “A model of articulatory dynamics and control,” _Proceedings of the IEEE_ , vol. 64, no. 4, pp. 452–460, 1976.
  * [20] C. P. Browman and L. Goldstein, “Articulatory phonology: An overview,” _Phonetica_ , vol. 49, no. 3-4, pp. 155–180, 1992.
  * [21] L. Welling and H. Ney, “Formant estimation for speech recognition,” _IEEE transactions on speech and audio processing_ , vol. 6, no. 1, pp. 36–48, 1998.
  * [22] B. C. Moore and B. R. Glasberg, “Suggested formulae for calculating auditory-filter bandwidths and excitation patterns,” _The journal of the acoustical society of America_ , vol. 74, no. 3, pp. 750–753, 1983.
  * [23] R. D. Kent and H. K. Vorperian, “Static measurements of vowel formant frequencies and bandwidths: A review,” _Journal of communication disorders_ , vol. 74, pp. 74–97, 2018.
  * [24] E. Plourde and B. Champagne, “Auditory-based spectral amplitude estimators for speech enhancement,” _IEEE/ACM Trans. Audio, Speech, Lang. Process._ , vol. 16, no. 8, pp. 1614–1623, 2008.
  * [25] N. Tishby and N. Zaslavsky, “Deep learning and the information bottleneck principle,” in _2015 ieee information theory workshop (itw)_. IEEE, 2015, pp. 1–5.
  * [26] N. Das, S. Chakraborty, J. Chaki, N. Padhy, and N. Dey, “Fundamentals, present and future perspectives of speech enhancement,” _International Journal of Speech Technology_ , vol. 24, pp. 883–901, 2021.
  * [27] N. Saleem and M. I. Khattak, “A review of supervised learning algorithms for single channel speech enhancement,” _International Journal of Speech Technology_ , vol. 22, no. 4, pp. 1051–1075, 2019.
  * [28] A. Li, W. Liu, X. Luo, C. Zheng, and X. Li, “Icassp 2021 deep noise suppression challenge: Decoupling magnitude and phase optimization with a two-stage deep network,” in _Proc. IEEE Int. Conf. Acoust., Speech Signal Process._ , 2021, pp. 6628–6632.
  * [29] A. Li, C. Zheng, L. Zhang, and X. Li, “Glance and gaze: A collaborative learning framework for single-channel speech enhancement,” _Applied Acoustics_ , vol. 187, p. 108499, 2022.
  * [30] A. Li, S. You, G. Yu, C. Zheng, and X. Li, “Taylor, can you hear me now? a taylor-unfolding framework for monaural speech enhancement,” in _Proceedings of the International Joint Conferences on Artificial Intelligence. (IJCAI)_ , 2022, pp. 4193–4200.
  * [31] A. Li, C. Zheng, G. Yu, J. Cai, and X. Li, “Filtering and refining: A collaborative-style framework for single-channel speech enhancement,” _IEEE/ACM Trans. Audio, Speech, Lang. Process._ , vol. 30, pp. 2156–2172, 2022.
  * [32] Z. Goldfeld, E. Van Den Berg, K. Greenewald, I. Melnyk, N. Nguyen, B. Kingsbury, and Y. Polyanskiy, “Estimating information flow in deep neural networks,” in _Proc. Int. Conf. Mach. Learn. (ICML)_ , Jun. 2019, pp. 2299–2308.
  * [33] T. Wang, W. Zhu, Y. Gao, S. Zhang, and J. Feng, “Harmonic attention for monaural speech enhancement,” _IEEE/ACM Trans. Audio, Speech, Lang. Process._ , vol. 31, pp. 2424–2436, 2023.
  * [34] Y. Isik, J. L. Roux, Z. Chen, S. Watanabe, and J. R. Hershey, “Single-channel multi-speaker separation using deep clustering,” in _Proc. Interspeech_ , 2016, pp. 545–549.
  * [35] J. M. Martin-Donas, A. M. Gomez, J. A. Gonzalez, and A. M. Peinado, “A deep learning loss function based on the perceptual evaluation of the speech quality,” _IEEE Signal processing letters_ , vol. 25, no. 11, pp. 1680–1684, 2018.
  * [36] C. K. Reddy, V. Gopal, R. Cutler, E. Beyrami, R. Cheng, H. Dubey, S. Matusevych, R. Aichner, A. Aazami, S. Braun, P. Rana, S. Srinivasan, and J. Gehrke, “The interspeech 2020 deep noise suppression challenge: Datasets, subjective testing framework, and challenge results,” in _Proc. Interspeech_ , 2020, pp. 2492–2496.
  * [37] A. W. Rix, J. G. Beerends, M. P. Hollier, and A. P. Hekstra, “Perceptual evaluation of speech quality (pesq)-a new method for speech quality assessment of telephone networks and codecs,” in _Proc. IEEE Int. Conf. Acoust., Speech Signal Process._ , vol. 2, 2001, pp. 749–752.
  * [38] J. Le Roux, S. Wisdom, H. Erdogan, and J. R. Hershey, “Sdr–half-baked or well done?” in _Proc. IEEE Int. Conf. Acoust., Speech Signal Process._ , 2019, pp. 626–630.
  * [39] J. Jensen and C. H. Taal, “An algorithm for predicting the intelligibility of speech masked by modulated noise maskers,” _IEEE/ACM Trans. Audio, Speech, Lang. Process._ , vol. 24, no. 11, pp. 2009–2022, 2016.
  * [40] X. Le, H. Chen, K. Chen, and J. Lu, “Dpcrn: Dual-path convolution recurrent network for single channel speech enhancement,” in _Proc. Interspeech_ , 2021, pp. 2811–2815.
  * [41] Y. Xia, S. Braun, C. K. Reddy, H. Dubey, R. Cutler, and I. Tashev, “Weighted speech distortion losses for neural-network-based real-time speech enhancement,” in _Proc. IEEE Int. Conf. Acoust., Speech Signal Process._ , 2020, pp. 871–875.
  * [42] S. Lv, Y. Hu, S. Zhang, and L. Xie, “Dccrn+: Channel-wise subband dccrn with snr estimation for speech enhancement,” in _Proc. Interspeech_ , 2021, pp. 2816–2820.
  * [43] X. Hao, X. Su, R. Horaud, and X. Li, “Fullsubnet: A full-band and sub-band fusion model for real-time single-channel speech enhancement,” in _Proc. IEEE Int. Conf. Acoust., Speech Signal Process._ , 2021, pp. 6633–6637.
  * [44] J. Chen, Z. Wang, D. Tuo, Z. Wu, S. Kang, and H. Meng, “Fullsubnet+: Channel attention fullsubnet with complex spectrograms for speech enhancement,” in _Proc. IEEE Int. Conf. Acoust., Speech Signal Process._ , 2022, pp. 7857–7861.