# Differentiable Artificial Reverberation

Sungho Lee, Hyeong-Seok Choi, and Kyogu Lee  Manuscript received October 21,
2021; revised March 27, 2022 and June 10, 2022; accepted July 15, 2022. This
work was supported by Institute of Information & communications Technology
Planning & Evaluation (IITP) grant funded by the Korea government (MSIT)
(No.2022-0-00641). The associate editor coordinating the review of this
manuscript and approving it for publication was Cecchi, Stefania
(Corresponding Author: Kyogu Lee). Sungho Lee is with the Music and Audio
Research Group, Graduate School of Convergence Science and Technology, Seoul
National University, Seoul, Republic of Korea (e-mail: sh-lee@snu.ac.kr).
Hyeong-Seok Choi was with the Music and Audio Research Group, Graduate School
of Convergence Science and Technology, Seoul National University, Seoul,
Republic of Korea. He is now with the Institute of New Media and
Communications, Seoul National University, Seoul, Republic of Korea (e-mail:
kekepa15@snu.ac.kr). Kyogu Lee is with the Music and Audio Research Group,
Graduate School of Convergence Science and Technology, Seoul National
University, Seoul, South Korea, and also with the Advanced Institutes of
Convergence Technology, Suwon, South Korea (e-mail: kglee@snu.ac.kr).

###### Abstract

Artificial reverberation (AR) models play a central role in various audio
applications. Therefore, estimating the AR model parameters (ARPs) of a
reference reverberation is a crucial task. Although a few recent deep-
learning-based approaches have shown promising performance, their non-end-to-
end training scheme prevents them from fully exploiting the potential of deep
neural networks. This motivates the introduction of differentiable artificial
reverberation (DAR) models, allowing loss gradients to be back-propagated end-
to-end. However, implementing the AR models with their difference equations
“as is” in the deep learning framework severely bottlenecks the training speed
when executed with a parallel processor like GPU due to their infinite impulse
response (IIR) components. We tackle this problem by replacing the IIR filters
with finite impulse response (FIR) approximations with the frequency-sampling
method. Using this technique, we implement three DAR models—differentiable
Filtered Velvet Noise (FVN), Advanced Filtered Velvet Noise (AFVN), and Delay
Network (DN). For each AR model, we train its ARP estimation networks for
analysis-synthesis (RIR-to-ARP) and blind estimation (reverberant-speech-to-
ARP) task in an end-to-end manner with its DAR model counterpart. Experiment
results show that the proposed method achieves consistent performance
improvement over the non-end-to-end approaches in both objective metrics and
subjective listening test results. Audio samples are available at https://sh-
lee97.github.io/DAR-samples/.

###### Index Terms:

Digital Signal Processing, Acoustics, Reverberation, Artificial Reverberation,
Deep Learning.

##  I Introduction

Reverberation is ubiquitous in a real acoustic environment. It provides the
listeners psychoacoustic cues for spatial characteristics. Therefore, adding
an appropriate reverberation to a dry audio is desirable for plausible
listening [1, 2, 3]. _Artificial reverberation_ (AR), efficient digital
filters that may be used to mimic real-world reverberation, have been
developed to achieve such auditory effects [4, 5, 6] and applied to room
acoustic enhancement [7], auditory scene generation [8, 9], post-production
[10], and many more.

Nevertheless, estimating the AR model parameters (ARPs) that match the
reference reverberation remains challenging. This is because the mapping from
the ARPs to the reverberation is highly nontrivial. We refer to this task as
_analysis-synthesis_ when the reference is a room impulse response (room IR,
RIR). When the reference is indirectly provided with a reverberant signal,
i.e., an RIR convolved with a dry signal, we refer to this task as _blind
estimation_. We limit the scope to reverberant speech signals in this paper,
yet the framework can easily be extended to other types of signals, such as
musical ones.

ARP estimation is a classical problem, and various attempts have been made
[11, 12, 13, 14]. However, they are only applicable to a specific AR model
(AR-model-dependent) and the analysis-synthesis task (task-dependent). This
inflexibility is problematic since every application has its own suitable AR
models and reference forms (e.g., RIR and reverberant speech).

(a) Training ARP estimation network with DAR models.

(b) Inference-time usage of the trained ARP estimation network.

Figure 1: The proposed ARP estimation framework. (a) With the DAR models, loss
gradients
$\partial\mathcal{L}/\partial\mathbf{P}_{1},\cdots,\partial\mathcal{L}/\partial\mathbf{P}_{n}$
can be back-propagated through the DAR models. Hence, the ARP estimation
network can be trained in an end-to-end manner. We can train the network to
perform the analysis-synthesis (RIR $h$ input), blind estimation (reverberant
speech $h*x$ input), or even both. The network has an AR-model-agnostic
encoder so that using a different DAR model only requires changing the tiny
projection layers $\text{Proj}_{1},\cdots,\text{Proj}_{n}$. (b) Each DAR model
generate an FIR approximation of its original AR model’s IIR. After the
training, estimated ARPs can be plugged in to the AR model which is highly
efficient and real-time controllable in CPU.

We can overcome such limitations with _differentiable artificial
reverberation_ (DAR) models. Each DAR model takes ARPs as input and generates
its corresponding AR model’s IR (see Figure 1a). After computing a loss
between the generated IR and the reference RIR, they allow the loss gradients
to be back-propagated through themselves. As a result, we can train a deep
neural network (DNN) to match the reference RIR in an end-to-end manner. By
leveraging the DNN’s flexibility and expressiveness, we can train the network
to perform any task with any AR model with minimal architecture change.

Since the AR models are exactly or close to linear time-invariant (LTI), we
can implement them differentiably “as is” with their difference equations[15,
16, 17, 18]. However, this approach has been impractical because the training
speed is limited by the recurrent sample generation of their IIR filter
components when executed on a parallel processor, such as a GPU. In this
context, we aim to modify the vanilla DAR models to sidestep the
aforementioned problem. Inspired by recent works [19, 20], we replace the IIR
filters with finite impulse response (FIR) approximations using the frequency-
sampling method [21]. This way, each DAR model becomes an FIR filter whose IR
can be generated without any recurrent step. Using this technique, we present
differentiable _Filtered Velvet Noise_ (FVN) [13], _Advanced Filtered Velvet
Noise_ (AFVN), and _Delay Network_ (DN) [22, 23]. Note that the AR-to-DAR
conversion can be applied systematically to any other AR model with IIR
components. While the DAR models are not identical to their originals, the
differences are little in most cases, allowing us to train the ARP estimation
networks reliably with them. Furthermore, even when differences exist, the
benefits of the end-to-end learning enabled by the DAR models outweigh them;
we show that the proposed approach brings consistent performance improvement
over non-end-to-end baselines where the difference is only in the training
scheme. After the training, we can revert to the original AR models for
efficient computation (see Figure 1b).

##  II Related Works

###  II-A Differentiable Digital Signal Processing

Digital signal processing (DSP) and deep learning are closely related. For
example, a temporal convolutional network [24] is a stack of dilated one-
dimensional convolutional layers with nonlinear activation layers and it has
been used for modeling dynamic range compression [25] and reverberation [26].
From the DSP viewpoint, it is FIR filter banks serially connected with
memoryless nonlinearities, i.e., an extension of the classical Wiener-
Hammerstein model [27].

If we look for more explicit DSP-deep learning relationships, so-called
“differentiable digital signal processing (DDSP)” approaches exist which aim
to import DSP components into deep learning frameworks as they provide strong
structural priors and interpretable representations. For example, a harmonic-
plus-noise model [28] was implemented in such a manner; then, a DNN was
trained to estimate its parameters to synthesize a monophonic signal and used
for various applications, including controllable synthesis and timbre transfer
[29].

In the case of LTI filters, it is well-known that both an FIR and IIR filter
are differentiable and can be directly imported into the deep learning
framework as a convolutional layer and recurrent layer, respectively [15, 16,
17, 18]. Since the IIR filter, e.g., parametric equalizer (PEQ), bottlenecks
the training speed, the frequency-sampling method [21] was utilized [19, 20].
Upon these works, we investigate the reliability of this technique, present a
general differentiable IIR filter based on state-variable filter (SVF) [30,
31], and use it to implement the DAR models.

###  II-B ARP Estimation

####  II-B1 Analysis-synthesis

Various methods have been proposed to systematically and automatically
estimate ARPs that mimic certain characteristics of a given RIR. Some works
first extract perceptually relevant features from the RIR, then estimate the
ARPs from them. For example, FDN parameters were derived from the RIR’s energy
decay relief (EDR) representation which encodes the frequency-dependent
reverberation decay [11]. Along with the FVN and AFVN, their parameter
estimation algorithm based on linear prediction were proposed together [13].
Another thread of work utilizes optimization. A genetic algorithm was proposed
to find the FDN parameters that achieve the desired “fitness” to the reference
RIR [12]. Since all these methods assume availability of the RIR, they cannot
be applied to the blind estimation task.

####  II-B2 Blind Estimation

End-users (e.g., audio engineers) often interact with the AR models provided
as audio plug-ins. As a result, several attempts have been made to implement
blind estimation algorithms for the plug-ins. For example, a preset
recommendation method based on the Gaussian mixture model was proposed [14].
More recently, DNNs were trained for the plug-in parameter estimation and
preset recommendation [10].

While these methods have shown promise, there is still room for improvement.
First, they generated the training data using the plug-ins to train their
models. However, this procedure may be time-consuming, and relying on data
generated with a single plug-in may increase the generalization error. Second,
their training objectives could be suboptimal. When the task is defined as
parameter regression [10], one must weigh each parameter’s perceptual
importance and apply it to the loss function design, which is a highly
challenging task. When the task is defined as classification [14, 10], it
could suffer from scalability problems. Finally, most plug-ins provide only a
few parameters, and each of them controls multiple internal ARPs
simultaneously. This might limit the performance due to the reduced degree of
freedom.

We tackle these problems with the DAR models as follows. First, we can use a
training objective that directly compares the estimated IR with ground-truth
RIR. Therefore, we can use any RIR for training and bypass the need to obtain
the ground-truth ARPs. Second, we replace the parameter-matching loss with the
end-to-end IR-matching loss and show that it improves the estimation
performance by a large margin in both objective metrics and subjective
listening test scores. Finally, we estimate the ARPs directly instead of
relying on the plug-in parameters.

####  II-B3 Two-Stage Approaches

Note that both analysis-synthesis and blind estimation tasks can be divided
into two subtasks. First, we can obtain reverberation parameters, e.g.,
reverberation time, from a given reference (reference-to-reverberation
parameters). We can directly compute the parameters from the RIR [32] for the
analysis-synthesis case. For the blind estimation, various methods have been
proposed [33, 34, 35, 36]. Then, we can tune the ARPs to match the desired
reverberation parameter values (reverberation parameters-to-ARPs) [37, 38,
39]. This two-stage approach can be a possible alternative to our single-stage
end-to-end estimation method. However, perceptually different reverberations
can have the same reverberation parameters, resulting in information loss.

###  II-C Modeling Reverberation in the Deep Learning Framework

An alternative to the proposed DAR approach could be using existing deep
learning modules to build another reverberation model. The most simplest
approach could be using a high-order FIR directly as an RIR model [29]. While
it is viable when the objective is to learn a single RIR, training a DNN to
generate various raw RIRs is highly challenging and inefficient.

Instead, recent approaches either use a DNN as a reverberator and control them
via conditioning methods [26, 40] or design a differentiable RIR generator and
use a DNN as a parameter estimator [41]. Our method is closely related to the
latter since the DAR model and the proposed ARP estimation network act in that
way. However, the key difference is that our DAR models are direct imports of
the AR models, which bring the following benefits. First, they are more
directly interpretable and controllable, allowing end-users to further tune
the parameters. Second, their structure can prevent unexpected artifacts.
Finally, they use sparse IIR and FIR filters, enabling efficient real-time
computation in CPU.

##  III Frequency-Sampling Method for Differentiable IIR Filters

For any real IIR filter $H$, its frequency response $H(e^{j\omega})$ is
continuous, $2\pi$-periodic, and conjugate symmetric. To obtain its FIR
approximation, we _frequency-sample_ $H(e^{j\omega})$ at angular frequencies
$\omega_{k}=2\pi k/N$ where $k=0,\cdots,\left\lfloor N/2\right\rfloor$ [21].
We denote the frequency-sampled filter with subscript $N$, i.e., $H_{N}$.

$H_{N}[k]=H(e^{j\omega_{k}}).$ (1)

Note that the frequency-sampling can also be defined on transfer functions,
e.g., $H(z)$ with $z=e^{j\omega_{k}}$.

Since each sampling is independent, $H_{N}$ can be generated simultaneously
with a parallel processor like GPU. Furthermore, the order of frequency-
sampling $(\cdot)_{N}$ and other basic arithmetic operations does not matter.
Therefore, we can frequency-sample an IIR filter by combining frequency-
sampled $m$-sample delays $(z^{-m})_{N}\in\mathbb{C}^{\left\lfloor
N/2\right\rfloor+1}$ as follows,

$H_{N}[k]=\left(\frac{\sum_{m=0}^{M}\beta_{m}z^{-m}}{\sum_{m=0}^{M}\alpha_{m}z^{-m}}\right)_{N}[k]=\frac{\sum_{m=0}^{M}\beta_{m}(z^{-m})_{N}[k]}{\sum_{m=0}^{M}\alpha_{m}(z^{-m})_{N}[k]}.$
(2)

Since $(z^{-m})_{N}[k]=e^{-j2\pi km/N}$, using the widely-used deep learning
libraries, we obtain $(z^{-m})_{N}$ as a tensor z_m by

angle $\displaystyle\texttt{= 2*pi*arange(N//2+1)/N},$ (3a) z_m
$\displaystyle\texttt{= e**(-1j*angle*m)}.$ (3b)

Then, we apply equation (2) to $\texttt{z\\_0},\cdots,\texttt{z\\_M}$ with
basic tensor operations and obtain every sample of $H_{N}$ in parallel. A
time-domain representation of $H_{N}$ is a length-$N$ FIR $h_{N}[n]$, inverse
discrete Fourier transform (inverse DFT) of $H_{N}[k]$, i.e.,

$h_{N}[n]=\frac{1}{N}\sum_{k=0}^{N-1}H_{N}[k]e^{j\omega_{k}n}(u[n]-u[n-N])$
(4)

where $u[n]$ is a unit step function.

Therefore, using the frequency-sampling method, we can replace any IIR filter
$H$ that causes the bottleneck with an FIR $h_{N}[n]$ whose convolution can be
performed efficiently via Fast Fourier Transform (FFT). In this paper, the
“differentiable IIR” filter refers to $H_{N}$, which is in fact an FIR filter.

###  III-A Reliability of the Frequency-Sampling Method

Every IIR filter’s frequency response has “infinite bandwidth” (in the time
domain) so that the frequency-sampling causes _time-aliasing_ [42].
Specifically, the frequency-sampled IR $h_{N}[n]$ is sum of length-$N$
segments of the original IR $h[n]$.

$h_{N}[n]=\sum_{m=0}^{\infty}h[mN+n](u[n]-u[n-N]).$ (5)

Since the energy of every stable IIR filter’s IR decays over time, more
frequency-sampling points $N$ give less time-aliasing, i.e., a closer
approximation (see Figure 2). Therefore, we can use the differentiable filter
$H_{N}$ in place of the original $H$ to match a reference response
$H_{\text{Ref}}$. We elaborate this with following analytical results.

####  III-A1 Time-aliasing Error

If $H$ has $M$ distinct poles where each of them $\nu_{i}\in\mathbb{C}$ has
multiplicity $r_{i}\in\mathbb{N}$, the time-aliasing error asymptotically
decreases as $N$ increases as follows,

$\|H-H_{N}\|_{2}=\sum_{i=1}^{M}O(N^{r_{i}-1}|\nu_{i}|^{N}).\\\ $ (6)

####  III-A2 Loss Error

The triangle inequality indicates that the loss error is bounded to the time-
aliasing error.

$\left|\|{H_{\text{Ref}}}-H\|_{2}^{2}-\|{H_{\text{Ref}}}-H_{N}\|_{2}^{2}\right|\leq\|H-H_{N}\|_{2}^{2}.$
(7)

####  III-A3 Loss Gradient Error

The loss gradient error has similar asymptotic behavior to the time-aliasing
error as follows,

$\left|\frac{\partial}{\partial
p}\|{H_{\text{Ref}}}-H\|_{2}^{2}-\frac{\partial}{\partial
p}\|{H_{\text{Ref}}}-H_{N}\|_{2}^{2}\right|\\\
=\sum_{i=1}^{M}O(N^{3r_{i}-1}|\nu_{i}|^{N})$ (8)

where $p$ is a parameter of $H$. See Appendix C for the proofs.

Figure 2: The frequency-sampling method with a various number of sampling
points $N$. Blue curves represent its magnitude response $|H(e^{j\omega})|$
(top row) and IR $h[n]$ (bottom row). Red curves are the magnitude response
and IR of the frequency-sampled filter $H_{N}$.

###  III-B Differentiable Dense IIR Filter with State-variable Filters

Since arbitrary IIR filter can be expressed as serially cascaded biquads
(second-order IIR filters), we can obtain a differentiable IIR filter
$\tilde{H}$ as a product of frequency-sampled biquads $(H_{i})_{N}$.

$\tilde{H}[k]=\prod_{i}(H_{i})_{N}[k]=\prod_{i}\left(\frac{\sum_{m=0}^{2}\beta_{i,m}z^{-m}}{\sum_{m=0}^{2}\alpha_{i,m}z^{-m}}\right)_{N}[k].$
(9)

Moreover, we use the _state variable filter_ (SVF) parameters $f_{i},R_{i}$,
$m^{\text{LP}}_{i}$, $m^{\text{BP}}_{i}$, and $m^{\text{HP}}_{i}$ to express
each biquad as follows,

$\displaystyle\beta_{i,0}$
$\displaystyle=f^{2}_{i}m^{\text{LP}}_{i}+f_{i}m^{\text{BP}}_{i}+m^{\text{HP}}_{i},$
(10a) $\displaystyle\beta_{i,1}$
$\displaystyle=2f^{2}_{i}m^{\text{LP}}_{i}-2m^{\text{HP}}_{i},$ (10b)
$\displaystyle\beta_{i,2}$
$\displaystyle=f^{2}_{i}m^{\text{LP}}_{i}-f_{i}m^{\text{BP}}_{i}+m^{\text{HP}}_{i},$
(10c) $\displaystyle\alpha_{i,0}$ $\displaystyle=f^{2}_{i}+2R_{i}f_{i}+1,$
(10d) $\displaystyle\alpha_{i,1}$ $\displaystyle=2f^{2}_{i}-2,$ (10e)
$\displaystyle\alpha_{i,2}$ $\displaystyle=f^{2}_{i}-2R_{i}f_{i}+1.$ (10f)

We prefer to use the SVF parameters rather than the biquad coefficients due to
the following reasons.

  * •

Interpretability. Each SVF has lowpass $H_{i}^{\text{LP}}$, bandpass
$H_{i}^{\text{BP}}$, and highpass filter $H_{i}^{\text{HP}}$. They share a
resonance $R_{i}$ and cutoff parameter $f_{i}=\tan(\pi\omega_{i}/\omega_{s})$
where $\omega_{i}$ is their cutoff frequency and $\omega_{s}$ is the sampling
rate. The filter outputs are multiplied with gains $m^{\text{LP}}_{i}$,
$m^{\text{BP}}_{i}$, and $m^{\text{HP}}_{i}$ and then summed. This
interpretability comes without any generality loss since an SVF can express
any biquad [31].

  * •

Simple Activation Functions. The SVF parameters have simpler stability
conditions, $R_{i}>0$ and $f_{i}>0$, than the biquad coefficients [18], which
leads to simpler activation function design. Refer to Section V-C for details.

  * •

Better Performance. We found that our ARP estimation networks performed better
when they estimate the SVF parameters rather than the biquad coefficients.
Refer to Appendix A-B for the comparison results.

From now on, we denote each SVF-parameterized biquad with a superscript
$H_{i}^{\text{SVF}}$ and call it simply “SVF”.

###  III-C Differentiable Parametric Equalizer

A low-shelving $H^{\text{LS}}$, peaking $H^{\text{Peak}}$, and high-shelving
filter $H^{\text{HS}}$ are widely used audio filters. Each of them can be
obtained using the SVF
$H^{\text{SVF}}(f,R,m^{\text{LP}},m^{\text{BP}},m^{\text{HP}})$ as follows
[30],

$\displaystyle H^{\text{LS}}(f,R,G)$
$\displaystyle=H^{\text{SVF}}(f,R,G,2R\sqrt{G},1),$ (11a) $\displaystyle
H^{\text{Peak}}(f,R,G)$ $\displaystyle=H^{\text{SVF}}(f,R,1,2RG,1),$ (11b)
$\displaystyle H^{\text{HS}}(f,R,G)$
$\displaystyle=H^{\text{SVF}}(f,R,1,2R\sqrt{G},G).$ (11c)

Here, $G$ is a new parameter that gives
$20\log_{10}(G)\,$\mathrm{d}\mathrm{B}$$ gain. A parametric equalizer (PEQ) is
serial composition of such filters. Following the recent works [19, 20], we
use one low-shelving, one high-shelving, and $K-2$ peaking filters.

$H^{\text{PEQ}}(z)=H^{\text{LS}}(z)H^{\text{HS}}(z)\prod_{i=1}^{K-2}H^{\text{Peak}}_{i}(z).$
(12)

Same as the IIR filter case, frequency-sampling the components and multiplying
them results in a differentiable PEQ $H^{\text{PEQ}}_{N}$.

##  IV Differentiable Artificial Reverberation

In this section, we derive differentiable FVN, AFVN, and DN. We first briefly
review the original AR models. Then, we obtain their DAR counterparts by
frequency-sampling the IIR components, replacing them with FIRs. Also, we
slightly modify the original models for efficient training, plausible
reverberation, and better overall estimation performance.

###  IV-A Differentiable Filtered Velvet Noise

####  IV-A1 Filtered Velvet Noise

One can divide an RIR $h[n]$ into $S$ segments (see Figure 3). As
reverberation can be regarded as mostly stochastic, we can model each
length-$L_{i}$ segment $h_{i}[n]$ with a source noise signal $s_{i}[n]$
“colored with” a filter $C_{i}$. When we use _velvet noise_ for each source,
this source-filter model becomes _Filtered Velvet Noise_ (FVN) [43, 44, 13].
The velvet noise $v_{i}[n]$ is sparse; in every length-$T_{i}$ interval, it
contains a single nonzero sample $\pm 1$ with random sign/position such that
its time-domain convolution is highly efficient. Moreover, an allpass filter
$U_{i}$ is introduced to smooth each source and make plausible sound. Finally,
an deterministic (bypass) FIR $\hat{h}_{0}[n]$ models the direct arrival and
early reflection. Then, an IR of FVN $\hat{h}[n]$ becomes

$\hat{h}[n]=\hat{h}_{0}[n]+\sum_{i=1}^{S}\underbrace{\overbrace{v_{i}[n-d_{i}]*u_{i}[n]}^{s_{i}[n-d_{i}]}*\;c_{i}[n]}_{\hat{h}_{i}[n-d_{i}]}$
(13)

where each $d_{i}$ is a delay for the segment alignment ($d_{1}=0$). In
practice, each allpass filter $U_{i}$ is composed with Schroeder allpass
filters (SAPs),
$U_{i,j}^{\text{SAP}}(z)=(1+\gamma_{i,j}z^{-\tau_{i,j}})/(\gamma_{i,j}+z^{-\tau_{i,j}})$,
where $\tau_{i,j}$ is a delay-line length and $\gamma_{i,j}$ is a feed-
forward/back gain) for efficiency [45]. A low-order dense IIR filter is used
as each coloration filter $C_{i}$.

Figure 3: Modified FVN, AVFN’s RIR approximation, and their differentiable
implementation strategy. FVN and AVFN divide a target RIR $h[n]$ into segments
$h_{i}[n]$ and approximate each segment with a velvet noise $v_{i}$ filtered
with an allpass filter $U_{i}$ and a coloration filter $C_{i}$. We obtain the
differentiable FVN and AVFN by converting the IIR filters $U_{i}$ and $C_{i}$
into FIR filters $\tilde{u}_{i}$ and $(c_{i})_{N}$. Each IIR filter is
emphasized with a double border box.

####  IV-A2 Differentiable Implementation

We modify each FVN component to obtain differentiable FVN.

  * •

_Velvet filters_. To batch the IR segments, we use the same segment lengths
$L_{1},\cdots,L_{S}$ and average pulse distance values $T_{1},\cdots,T_{S}$
for every reference RIR.

  * •

_Allpass filters_. We tune and fix the allpass filter parameters jointly with
the velvet filter parameters to avoid perceptual artifacts, e.g.,
discontinuous and rough sound, while being computationally efficient. Since we
fixed the filter, instead of the frequency-sampling method, we simply crop its
IR to obtain an FIR $\tilde{u}_{i}[n]$.

  * •

_Coloration filters_. We model $C_{i}$ with $K$ serial SVFs, i.e.,
$C_{i}(z)=\prod_{k=1}^{K}C_{i,k}^{\text{SVF}}(z)$. We frequency-sample each
filter $C_{i}(z)$ and convert it into a length-$N$ FIR $(c_{i})_{N}[n]$ so
that we can train our network to estimate its parameters.

Therefore, IR of the differentiable FVN $\tilde{h}[n]$ becomes

$\tilde{h}[n]=\hat{h}_{0}[n]+\text{overlap-
add}\\{\underbrace{\overbrace{v_{i}[n]*\tilde{u}_{i}[n]}^{\tilde{s}_{i}[n]}*(c_{i})_{N}[n]}_{\tilde{h}_{i}[n]}\\}.$
(14)

Since every component $v_{i}[n]$, $\tilde{u}_{i}[n]$, and $(c_{i})_{N}[n]$ is
an FIR, we can compute each segment
$\tilde{h}_{i}[n]=v_{i}[n]*\tilde{u}_{i}[n]*(c_{i})_{N}[n]$ efficiently by
convolving the FIRs in the frequency domain, i.e., zero-padding followed by
FFT, multiplication, then IFFT. Finally, we overlap-add all the segments and
add the deterministic FIR $\hat{h}_{0}[n]$ to obtain the full IR
$\tilde{h}[n]$. The overlap-add can be implemented with a transposed
convolution layer.

(a) Approximation error of differentiable FVN in EDR.

(b) Approximation error of differentiable AFVN in EDR.

Figure 4: EDR of IRs generated by differentiable FVN, AFVN, their original AR
counterparts, and EDR errors introduced during the AR-to-DAR conversion
process. Because the errors are small and mostly in the low-energy region,
they are imperceptibly small (note that the $z$ axis scale is different for
the EDR plots and the EDR error plots). We obtained the FVN/AFVN parameters
with the trained blind estimation networks discussed below.

While the modifications for the differentiable FVN introduce approximation
error, it is negligible in practice. We explain this with the EDR
$H^{\text{EDR}}$, a reverse cumulative sum of an IR’s short-time Fourier
transform (STFT) energy [11].

$H^{\text{EDR}}[k,n]=\sum_{m=n}^{\infty}|H^{\text{STFT}}[k,m]|^{2}.$ (15)

By comparing the FVN’s EDR and its differentiable counterpart, we can analyze
their differences in reverberation characteristics. Specifically, we obtain
the EDR error $E^{\text{EDR}}$ by calculating the difference between the two
log-magnitude EDRs.

$E^{\text{EDR}}=10\log_{10}\hat{H}^{\text{EDR}}-10\log_{10}\tilde{H}^{\text{EDR}}.$
(16)

Figure 4a shows the EDRs and the EDR error. It reveals that the error is small
(at most $3$\mathrm{d}\mathrm{B}$$) and mostly in the low-energy region (below
$-40$\mathrm{d}\mathrm{B}$$). Also, there is little EDR error at the first
time index, i.e., $|E^{\text{EDR}}[k,0]|\ll 1$, indicating that the overall
magnitude response remains the same during the conversion process. In
conclusion, the error is hardly noticeable, and the differentiable FVN can
replace the original FVN for the training (we use a loss function related to
the EDR; see Section V-D).

(a) Modified FVN in the time domain.

(b) Modified AVFN in the time domain.

(c) Each velvet filter $V_{i}$.  (d) Each coloration filter $C_{i}$.

Figure 5:  Modifications to the original FVN and AFVN. (a) Modified FVN. To
retain a sharp transient, we insert each SAP before its corresponding velvet
filter. For each velvet filter $V_{i}$, the output denoted with a down arrow
is a convolution of the input and the velvet noise $v_{i}[n]$. The other
output denoted with a right arrow is a $L_{i}$-sample-delayed input. Refer to
[13] for more details. (b) The same modification is performed to the AVFN
model. (c) For both FVN and AVFN, each velvet filter $V_{i}$ is divided into
smaller filters $V_{i,1},\cdots,V_{i,M}$ assigned with gains
$g_{i,1},\cdots,g_{i,M}$. This modification is equivalent to dividing each
velvet segment $v_{i}[n]$ into smaller segments $v_{i,1}[n],\cdots,v_{i,M}[n]$
and multiplying them with $g_{i,1},\cdots,g_{i,M}$. (d) The $i^{\text{th}}$
coloration filter $C_{i}$ in the time domain, which is serially connected SVFs
$C_{i,1}^{\text{SVF}},\cdots,C_{i,K}^{\text{SVF}}$.

####  IV-A3 Modifications to the Original Model

FVN was initially designed for the analysis-synthesis of late reverberation;
it used the direct arrival and early reflections cropped from the RIR and
modeled only the late part. However, we also aim to solve the blind estimation
problem where we must estimate the entire RIR. To this end, we modify the
original model as follows.

  * •

_Finer segment gain._ We divide each velvet segment $v_{i}[n]$ into $M$
smaller segments $v_{i,1}[n],\cdots,v_{i,M}[n]$ multiplied with gains
$g_{i,1},\cdots,g_{i,M}$. Figure 5c shows the resulting velvet filter $V_{i}$
in the time domain.

  * •

_Cumulative allpass._ The original model smoothes every velvet noise with the
same allpass filter. This is undesirable in our context since it could smear
the direct arrival and early reflections. Instead, we gradually cascade the
SAPs to obtain each allpass filter
$U_{i}(z)=\prod_{j=1}^{i}U_{j}^{\text{SAP}}(z)$ so the earlier segments are
less smeared than the later ones. As shown in Figure 5a, this modification can
be implemented in the time domain by inserting each SAP $U_{j}^{\text{SAP}}$
before the corresponding velvet filter $V_{j}$.

  * •

_Shorter deterministic FIR._ With the above changes, the stochastic segments
can fit the early RIR to some extent. Therefore, we shorten the early
deterministic FIR $\hat{h}_{0}[n]$.

Refer to Appendix A-A for the evaluation of the modifications.

####  IV-A4 Model Configuration Details

Exact configurations of our FVN used for the experiments are as follows.

We generated $2.5$ seconds of IR ($48$\mathrm{k}\mathrm{H}\mathrm{z}$$
sampling rate, total $L=120$\mathrm{k}$$ samples) with FVN. We used $S=20$
non-uniform velvet segments whose lengths $L_{i}$ are $L/40,L/20$, and $L/10$
for $10,5$, and $5$ segments, respectively, and their average pulse distances
were set to $\mathbf{T}=[10$, $20$, $35$, $50$, $65$, $90$, $120$, $135$,
$180$, $220$, $270$, $320$, $370$, $420$, $480$, $540$, $610$, $680$, $750$,
$820]$. The SAPs $U_{j}^{\text{SAP}}$ were forced to have gains
$\gamma_{j}=0.75+0.01j$ and delay lengths $\boldsymbol{\tau}=[23$, $48$, $79$,
$109$, $113$, $127$, $163$, $191$, $229$, $251$, $293$, $337$, $397$, $421$,
$449$, $509$, $541$, $601$, $641$, $691]$. Each velvet segment had $M=4$ sub-
segment gains and filtered with $K=8$ SVFs. We frequency-sampled the
coloration filters with $N=4$\mathrm{k}$$ points. The length of the
deterministic FIR $h_{0}[n]$ was set to $50$. We set the gain
$\mathbf{g}\in\mathbb{R}^{S\times M}$, SVF parameters
$\mathbf{f},\mathbf{R},\mathbf{m^{\text{LP}}},\mathbf{m^{\text{BP}}},$
$\mathbf{m^{\text{HP}}}\in\mathbb{R}^{S\times K},$ and the bypass FIR
$\mathbf{h}_{0}\in\mathbb{R}^{Z}$ to estimation targets (total $930$ ARPs).
The time-domain FVN $\hat{H}$ requires $2166$ floating point operations per
sample (FLOPs).

###  IV-B Differentiable Advanced Filtered Velvet Noise

Since frequency-dependent decay of reverberation is gradual, one can model
each coloration filter $C_{i}$ as an initial coloration filter $C_{1}$
cascaded with delta-coloration filters $C_{\Delta 2},\cdots,C_{\Delta i}$.

$C_{i}(z)=C_{1}(z)\prod_{j=2}^{i}C_{\Delta j}(z).$ (17)

FVN with this modification is called _Advanced Filtered Velvet Noise_ (AFVN)
[13]. The delta filters’ orders $\Delta K_{2},\cdots,\Delta K_{S}$ are set
lower than the initial filter’s $K_{1}$ for the efficiency. See Figure 5b for
its time-domain implementation. We used $K_{1}=8$ and $K_{\Delta}=K_{\Delta
2}=\cdots=K_{\Delta S}=2$ in the experiments. This results in a total of
$\bar{K}=K_{1}+(S-1)K_{\Delta}=46$ SVFs. The other settings are the same as
the FVN. This AFVN has $360$ ARPs to estimate and its time-domain model
$\hat{H}$ requires $1511$ FLOPs. While order of the coloration filters are
higher than the FVN’s counterparts, the introduced frequency-sampling error
remains approximately the same in practice (see Figure 4b).

###  IV-C Differentiable Delay Network

####  IV-C1 Delay Network

By interconnecting multiple delay lines in a recursive manner, one can obtain
a _Delay Network_ (DN) [22, 23] structure. A difference equation of the DN can
be written in a general form as follows (we omit the bracket notation for the
filtering),

$\displaystyle y[n]$
$\displaystyle=\mathbf{C}^{T}\bar{\mathbf{y}}[n]+\hat{H}_{0}x[n],$ (18a)
$\displaystyle\bar{\mathbf{y}}[n+\mathbf{d}]$
$\displaystyle=\mathbf{A}\bar{\mathbf{y}}[n]+\mathbf{B}x[n].$ (18b)

That is, an input $x[n]$ is distributed and filtered (or simply scaled) with
$\mathbf{B}$, then go through $\mathbf{d}$-sample parallel delay lines which
are recursively interconnected to themselves through a mixing filter matrix
$\mathbf{A}$. The delay line outputs $\bar{\mathbf{y}}[n]$ are filtered and
summed with $\mathbf{C}$. We add a bypass signal filtered with $\hat{H}_{0}$,
resulting in an output $y[n]$. Transfer function of the DN is

$\hat{H}(z)=\mathbf{C}(z)^{T}(\mathbf{D}^{-1}(z)-\mathbf{A}(z))^{-1}\mathbf{B}(z)+\hat{H}_{0}(z).$
(19)

Here, $\mathbf{D}(z)=\text{diag}(z^{-\mathbf{d}})$ is an $M\times M$ transfer
function matrix for the delay lines, i.e., $\mathbf{D}_{ii}(z)=z^{-d_{i}}$.
$\mathbf{B}(z)$, $\mathbf{C}(z)$, and $\mathbf{A}(z)$ are input, output, and
feedback transfer function matrices of shape $M\times 1$, $1\times M$, and
$M\times M$, respectively.

####  IV-C2 Differentiable Implementation

The DN components are recursively interconnected such that one cannot divide
its IR into independent segments and generate them in parallel like we did
with the FVN and AFVN. Instead, we frequency-sample its entire transfer
function to obtain differentiable DN, which is equivalent to a composition of
individually frequency-sampled transfer function matrices.

$\tilde{h}[n]=\text{IFFT}\left\\{\mathbf{C}_{N}^{T}(\mathbf{D}_{N}^{-1}-\mathbf{A}_{N})^{-1}\mathbf{B}_{N}+(\hat{H}_{0})_{N}\right\\}.$
(20)

Here, $\mathbf{D}_{N}=\text{diag}((z^{-\mathbf{d}})_{N})\in\mathbb{C}^{M\times
M\times\left\lfloor N/2+1\right\rfloor}$ is a frequency-sampled version of the
delay line transfer function matrix $\mathbf{D}$, i.e.,
$(\mathbf{D}_{N})_{ii}=(z^{-d_{i}})_{N}\in\mathbb{C}^{\left\lfloor
N/2+1\right\rfloor}$. Likewise, we can frequency-sample the transfer function
matrices $\mathbf{B}$, $\mathbf{C}$, and $\mathbf{A}$ to obtain their
approximations $\mathbf{B}_{N}$, $\mathbf{C}_{N}$, and $\mathbf{A}_{N}$,
respectively. Each frequency-sampled transfer function matrix is a batch of
$\left\lfloor N/2+1\right\rfloor$ matrices, the matrix multiplications and
inversions of equation (20) can be performed in parallel.

Figure 6: Our DN in the time domain. We restrict the general filter matrices
to accelerate the training and obtain an efficient and controllable model. We
simplify the pre-filters $\mathbf{B}$ and the post-filters $\mathbf{C}$ to
have only one coloration filter $C_{1}$ and use the same $C_{\Delta}$ for all
feedback loops. A time-varying mixing matrix $\mathbf{Q}_{n}$ and allpass
filters $\mathbf{U}$ are introduced to use DN with small $M$ (i.e., fast
training) with minimal quality loss.

####  IV-C3 Restrictions to the General Model

The presented DN structure is fully general and able to express many variants,
including the _Feedback Delay Network_ (FDN) [46]. Therefore, we can also
derive their differentiable versions following the proposed method. However,
when following the general practice that uses many delay lines (e.g., $M=16$)
for a high-quality reverberation, its differentiable model’s frequency-sampled
transfer function matrices consume too much memory, and their multiplications
and inversions bottleneck the training speed. To tackle this, we reduce $M$
and modify the components as follows to make DN plausible with low $M$ (see
Figure 6).

  * •

_Pre and Post Filter Matrix._ Following most previous works [47, 48, 11], the
pre filter matrix $\mathbf{B}$ is set to a (constant) gain vector
$\mathbf{b}$. The post filter matrix $\mathbf{C}$ is a combination of a gain
vector $\mathbf{c}$ and a common filter $C_{1}$, i.e.,
$\mathbf{C}=C_{1}\mathbf{c}$. The common filter $C_{1}$ is composed of serial
$K_{C_{1}}$ SVFs.

  * •

_Feedback Filter Matrix._ While most DN model combines channel-wise parallel
delta-coloration (absorption) filters $\mathbf{C}_{\Delta}$ and an inter-
channel mixing matrix $\mathbf{Q}$ to compose the feedback filter matrix
$\mathbf{A}=\mathbf{Q}\mathbf{C}_{\Delta}$. Additionally, we insert channel-
wise allpass filters $\mathbf{U}$ and introduce _time-variance_ by modulating
the mixing matrix $\mathbf{Q}$. This results in the time-varying feedback
filter matrix $\mathbf{A}_{n}=\mathbf{Q}_{n}\mathbf{U}\mathbf{C}_{\Delta}$.

  * •

_Allpass Filters._ To achieve faster echo density build-up without adding more
delay lines, we insert serial $K_{\mathbf{U}}$ SAPs in each feedback path.
Unlike the FVN and AFVN cases, we estimate the SAP gains
$\boldsymbol{\gamma}\in\mathbb{R}^{M\times K_{\mathbf{U}}}$ with the
estimation network and frequency-sample the allpass filter matrix $\mathbf{U}$
for the differentiable model.

  * •

_Time-varying Mixing Matrix._ We further reduce the audible “ringing”
artifacts by modulating the stationary poles with the time-varying mixing
matrix, which is set to $\mathbf{Q}_{n}=\mathbf{Q}_{0}\mathbf{R}^{n}$ where
$\mathbf{Q}_{0}$ is a Householder matrix and $\mathbf{R}$ is a tiny rotational
matrix constructed from a random matrix [48, 49]. Since the frequency-sampling
method is only applicable to LTI filters, we fix the mixing matrix to
$\mathbf{Q}_{0}$ when obtaining the differentiable model.

  * •

_Absorption Filters._ Unlike the conventional FDN [46], we use the same
absorption filter $C_{\Delta}$ for every channel. Since the mixing matrix
$\mathbf{Q}_{n}$ is always unitary, our DN becomes stable when $C_{\Delta}$
has magnitude response less than $1$ [48]. We achieve this by using a PEQ with
additional constraints (see Section V-C) as $C_{\Delta}$.

  * •

_Bypass FIR._ We use a length-$Z$ FIR $h_{0}[n]$ for the bypass.

Figure 7: EDR of differentiable DN, its original DN with two versions (LTI and
time-varying), and EDR errors between those three. Same as the FVN and AFVN,
the frequency-sampling introduce little error, as shown in the upper right
plot. However, rotating the mixing matrix introduces a considerable amount of
EDR error since it modulates the pole positions. The error is mostly in the
low-frequency region, where the poles are sparsely located.

We summarize the modifications. The difference equation of our time-varying DN
$\hat{H}_{n}^{\text{TV}}$ is

$\displaystyle y[n]$
$\displaystyle=C_{1}\mathbf{c}^{T}\bar{\mathbf{y}}[n]+H_{0}x[n],$ (21a)
$\displaystyle\bar{\mathbf{y}}[n+\mathbf{d}]$
$\displaystyle=\mathbf{Q}_{n}\mathbf{U}\mathbf{C}_{\Delta}\bar{\mathbf{y}}[n]+\mathbf{b}x[n].$
(21b)

The transfer function of its LTI approximation $\hat{H}(z)$ is given as
equation (19) with the restrictions $\mathbf{B}(z)=\mathbf{b}$,
$\mathbf{C}(z)=C_{1}(z)\mathbf{c}$, and
$\mathbf{A}(z)=\mathbf{Q}_{0}\,\text{diag}(\mathbf{U}_{N}(z)\odot\mathbf{C}_{\Delta}(z))$.
Hence, the IFFT of

$(C_{1})_{N}\mathbf{c}^{T}(\mathbf{D}_{N}^{-1}-\mathbf{Q}_{0}\,\text{diag}(\mathbf{U}_{N}\odot(\mathbf{C}_{\Delta})_{N}))^{-1}\mathbf{b}$
(22)

summed with the bypass FIR $\hat{h}_{0}[n]$ results in our differentiable DN’s
IR $\tilde{h}[n]$.

Figure 7 shows the EDR errors between the three different DN modes, i.e.,
differentiable DN, original LTI, and the time-varying one. The only difference
between the first two is the frequency-sampling; unless the DN decays very
slowly, there is little time-aliasing and EDR error. However, due to the pole
modulation, the EDR error increases significantly when the time-varying mixing
matrix is applied. Despite this, we find that other perceptually important
factors, such as reverberation time, remain largely unchanged (see Section
VI-A).

####  IV-C4 Model Configuration Details

We used $M=6$ delay lines with delay lengths $\mathbf{d}=[233$, $311$, $421$,
$461$, $587$, $613]$ in samples. We used $K_{\mathbf{U}}=4$ SAPs for each
delay line. We fixed the SAP delay lengths to $\boldsymbol{\tau}=[[131,$
$151,$ $337,$ $353],$ $[103,$ $173,$ $331,$ $373],$ $[89,$ $181,$ $307,$
$401],$ $[79,$ $197,$ $281,$ $419],$ $[61,$ $211,$ $257,$ $431],$ $[47,$
$229,$ $251,$ $443]]$. Note that the SAPs introduce additional delays so that
the effective delay lengths are $[1205$, $1291$, $1399$, $1437$, $1547$,
$1583]$ or about $[25.1$, $26.9$, $29.1$, $29.9$, $32.2$, $33.0]$ in
milliseconds. Our rotational matrix satisfies
$\mathbf{R}^{30$\mathrm{k}$}=\mathbf{I}$ so that the mixing matrix
$\mathbf{Q}_{n}$ has a period of $0.625$ second. Both post $C_{1}$ and
absorption filter $C_{\Delta}$ have $K_{C_{1}}=K_{c_{\Delta}}=8$ components.
We used $N=120$\mathrm{k}$$ frequency-sampling points. The order of the bypass
FIR is $Z=100$. The post filter parameters $\mathbf{f}_{C_{1}}$,
$\mathbf{R}_{C_{1}}$, $\mathbf{m}^{\text{LP}}_{C_{1}}$,
$\mathbf{m}^{\text{BP}}_{C_{1}}$,
$\mathbf{m}^{\text{HP}}_{C_{1}}\in\mathbb{R}^{1\times K_{C_{1}}}$, feedback
filter parameters $\mathbf{f}_{C_{\Delta}}$, $\mathbf{R}_{C_{\Delta}}$,
$\mathbf{G}_{C_{\Delta}}\in\mathbb{R}^{K_{C_{\Delta}}}$, pre and post gain
vectors $\mathbf{b}$, $\mathbf{c}\in\mathbb{R}^{M\times 1}$, SAP gain vector
$\boldsymbol{\gamma}\in\mathbb{R}^{M\times K_{\mathbf{U}}}$, and the bypass
FIR $\mathbf{h}_{0}\in\mathbb{R}^{Z}$ are the estimation targets (total $200$
ARPs). The resulting time-domain model $\hat{H}$ and the time-varying model
$\hat{H}^{\text{TV}}_{n}$ consume $889$ and $1285$ FLOPs, respectively.

##  V Artificial Reverberation Parameter Estimation with a Deep Neural Network

The details of our ARP estimation network are as follows. As shown in Figure
8, it transforms single channel audio input (either RIR or reverberant speech)
into a shared latent $\mathbf{z}$ with the model/task-agnostic encoder. Then,
each ARP-groupwise layer projects the latent $\mathbf{z}$ into an ARP tensor
$\mathbf{P}_{i}$.

###  V-A AR-Model/Task-Agnostic Encoder

We first transform the reference RIR or reverberant speech into a log-
frequency log-magnitude spectrogram. Then, the spectrogram goes through five
two-dimensional convolutional layers, each followed by a rectified linear unit
(ReLU) activation. We apply gated recurrent unit (GRU) layers [50] along with
the frequency axis regarding the channels as features and the time axis
regarding the channels and frequencies as features, widening the receptive
field. Finally, we apply two linear layers followed by layer normalization
[51] and ReLU, resulting in the two-dimensional shared latent $\mathbf{z}$.
The detailed network configurations are shown in Figure 8, and the encoder has
about $7.3$\mathrm{M}$$ parameters.

###  V-B ARP-Groupwise Projection Layers

Now we aim to transform the two-dimensional shared latent $\mathbf{z}$ into
each ARP tensor $\mathbf{P}_{i}$ which is at most two-dimensional. To achieve
this, we use two axis-by-axis linear layers. We first transpose and apply a
linear layer to the latent to match the first axis shape. Then, another
transposition and a linear layer match the other axis. This approach
factorizes a single large layer into two smaller ones, reducing the number of
parameters.

###  V-C Nonlinear Activation Functions and Bias Initialization

Each ARP group has different stability conditions and desired distribution. To
satisfy these, we attach a nonlinear activation to each projection layer and
initialize the last linear layer’s bias as follows. Here, $x$ denotes a pre-
activation element.

####  V-C1 Nonlinear Activation Functions

  * •

For the resonance $\mathbf{R}$ of the SVF, we use a scaled softplus
$\zeta(x)=\log(1+e^{x})/\log(2)$ to center the initial distribution at $1$ and
satisfy the stability condition $\mathbf{R}>\mathbf{0}$.

  * •

For cutoff $\mathbf{f}$ we also have the same stability condition
$\mathbf{f}>\mathbf{0}$. However, instead of the softplus, we use
$\tan(\pi\sigma(x)/2)$ where $\sigma(x)=1/(1+e^{-x})$ is a logistic sigmoid.
With this activation, $\sigma(x)=0$ and $1$ represent cutoff frequency of
$0$\mathrm{H}\mathrm{z}$$ and half of the sampling rate, respectively.

  * •

The delta-coloration PEQ $C_{\Delta}$ inside the DN must satisfy the stability
condition $|C_{\Delta}(e^{j\omega})|<1$. To achieve this, we use
$10^{-\zeta(x)}$ for the component gain $\mathbf{G}$. Also, the shelving
filters’ resonances should satisfy $\mathbf{R}>\sqrt{2}/2$ to prevent their
magnitude responses to “spike” over $1$. Therefore, we add $\sqrt{2}/2$ after
the softplus activation.

  * •

For the feed-forward/back gains $\boldsymbol{\gamma}$ of channel-wise allpass
filters $\mathbf{U}$ inside the DN, we use sigmoid activation $\sigma(x)$.

####  V-C2 Bias Initialization

  * •

We initialize bias of the $\mathbf{f}$ projection layer to
$\sigma^{-1}(2\omega_{k}/\omega_{s})$ where each $\omega_{k}$ is desired
initial cutoff frequency of $k^{\text{th}}$ SVF. We set
$\omega_{k}=\omega_{\text{min}}(\omega_{\text{max}}/\omega_{\text{min}})^{(k-1)/(K-1)}$
such that the frequencies are equally spaced in the logarithmic scale from
$\omega_{\text{min}}=40$\mathrm{H}\mathrm{z}$$ to
$\omega_{\text{max}}=12$\mathrm{k}\mathrm{H}\mathrm{z}$$. To the SVFs inside
the AFVN’s initial and delta-coloration filters, we perform a random
permutation to the frequency index $k$ and obtain a new index $\bar{k}$. This
prevents SVFs for the latter segments having higher initial cutoff
frequencies.

  * •

Decoder biases for the SVF mixing coefficients
$\mathbf{m}^{\text{LP}},\mathbf{m}^{\text{BP}}$, and $\mathbf{m}^{\text{HP}}$
are initialized to $1,2$, and $1$, respectively, so that each SVF’s initial
magnitude response slightly deviates from $1$. This prevents the coloration
filters’ responses and the estimation network loss gradients to vanish or
explode.

  * •

For the PEQ gains $\mathbf{G}$ of the DN’s delta-coloration filters, we set
each of their biases to $-10$ such that the initially generated IRs have long
enough reverberation time.

Figure 8: Architecture of the ARP estimation network. For each two-dimensional
convolutional (Conv2D) layer, $c$, $k$, and $s$ denote number of output
channel, kernel size, and stride, respectively. For each gated reccurent unit
(GRU) layer, $n$ and $h$ denote number of layers and hidden features. Shape
and size of the intermediate output tensors are provided in the parentheses,
where $F$, $T$, and $C$ denote frequency, time, and channel axis,
respectively. $Z1$, and $Z2$ denote each ARP tensor’s first and second axis.
We omitted the batch axis.

###  V-D Loss Function

####  V-D1 Match Loss

We utilize a multi-scale spectral loss [29] and define a match loss as
follows,

$\mathcal{L}_{\text{\sc
Match}}(h,\tilde{h})=\sum_{i}\left\||H^{\text{STFT}}_{i}|-|\tilde{H}^{\text{STFT}}_{i}|\right\|_{1}.$
(23)

$i$ denotes STFT with different FFT sizes. We use the FFT sizes of $256$,
$512$, $1024$, $2048$, and $4096$, hop sizes of $25\%$ of the respective FFT
sizes, and Hann windows. Each spectrogram’s frequency axis is log-scaled like
the encoder’s spectrogram.

####  V-D2 Regularization

We additionally apply a regularization to reduce time-aliasing of the
estimation. As shown in equation (6) and (8), pole radii of each SVF affects
the reliability of the frequency-sampled one (hence DAR model) and its
parameter estimator. Since each SVF’s IR $(c^{\text{SVF}}_{i,k})_{N}[n]$ can
be computed, we encourage reducing its pole radii by penalizing the IR’s
decreasing speed. Each decreasing speed $\gamma_{i,k}$ is calculated by the
ratio of the average amplitude of first and last $n_{0}$ samples of
$(c^{\text{SVF}}_{i,k})_{N}[n]$. We fix $n_{0}$ to $N/8$. Then, each
$\gamma_{i,k}$ is weighted with a softmax function along the SVF-axis to
penalize higher $\gamma_{i,k}$ more. Sum of the weighted decreasing speed
values results in the regularization loss $\mathcal{L}_{\text{\sc Reg}}$ as
follows,

$\displaystyle\gamma_{i,k}$
$\displaystyle=\frac{\sum_{n=N-n_{0}}^{N}|(c^{\text{SVF}}_{i,k})_{N}[n]|}{\sum_{n=0}^{n_{0}}|(c^{\text{SVF}}_{i,k})_{N}[n]|},$
(24a) $\displaystyle\mathcal{L}_{\text{\sc Reg}}$
$\displaystyle=\sum_{i=1}^{S}\frac{\sum_{k=1}^{K}\gamma_{i,k}e^{\gamma_{i,k}}}{\sum_{j=1}^{K}e^{\gamma_{i,j}}}.$
(24b)

We omit the regularization loss for the DN networks since with DN the time-
aliasing is inevitable to match the reference with reverberation time longer
than the number of frequency-sampling points $N$. Therefore, our full loss
function $\mathcal{L}$ is

$\mathcal{L}(h,\tilde{h})=\mathcal{L}_{\text{\sc
Match}}(h,\tilde{h})+\beta\mathcal{L}_{\text{\sc
Reg}}(\mathbf{c}^{\text{SVF}}_{N})$ (25)

where $\beta=1$ for the FVN and AFVN and $\beta=0$ for the DN.

##  VI Experimental Setup

###  VI-A Data

####  VI-A1 Room Impulse Response

We collected $1835$ real-world RIR measurements from various datasets
including OpenAIR [52] and ACE Challenge [53]. The amount of the RIRs were
insufficient for the training purpose, so we used them only for the validation
and test ($836$ and $999$ RIRs, respectively), ensuring that each set consists
of RIRs from different rooms.

For the training, we synthesized $200$\mathrm{k}$$ RIRs with shoebox room
simulations using the image-source method [54, 55]. To reflect various
acoustic environments, we randomized simulation parameters for every RIR
synthesis, which include room size, each wall’s frequency-dependent absorption
coefficient, and source/microphone positions. We tuned the randomization
scheme to match the training dataset to the validation set in terms of
reverberation parameter statistics.

In addition, we pre-processed each RIR as following.

  * •

We removed the pre-onset part of the RIR. We detected the onset by extracting
a local energy envelope of the RIR then finding its maximum point [56].

  * •

Following the recent RIR augmentation method [57], we multiplied random gain
sampled from $\mathcal{U}(-12$\mathrm{d}\mathrm{B}$,3$\mathrm{d}\mathrm{B}$)$
to the first $5$\mathrm{m}\mathrm{s}$$ of each train RIR. This slightly
reduces the average direct-to-reverberant ratio (DRR) of the training set and
matches the validation set. At the evaluation/test, we omitted this procedure.

  * •

Finally, we normalized the RIR to have the energy of $1$.

####  VI-A2 Reverberant Speech

We used VCTK [58] for dry speech. We split it into the train ($61808$),
validation ($21608$), and test ($4912$) set so that each set is composed of
the speech recordings from different speakers. We sampled an RIR and a dry
speech sample from their respective datasets and convolved them to generate
reverberant speech. We random-cropped $2.5$-second segment from it for the
input.

###  VI-B Training

We trained each network with every proposed DAR model for the three tasks:
analysis-synthesis, blind estimation, and both tasks. For the both-performing
ones, we fed an RIR or reverberant speech in a $50$-$50\%$ probability.

We set the initial learning rate to $10^{-4}$ for FVN and AFVN and $10^{-5}$
for DN. We used Adam optimizer [59]. For DN, gradients were clipped to $\pm
10$ for stable training. After $250$\mathrm{k}$$ steps, we performed the
learning rate decay. We multiplied the learning rate by $10^{-0.2}$ and
$10^{-0.1}$ every $50$\mathrm{k}$$ step for the analysis-synthesis networks
and the others, respectively. We finished the training if the validation loss
did not improve after $50$\mathrm{k}$$ steps, which took no more than
$500$\mathrm{k}$$ steps for the analysis-synthesis and $1$\mathrm{M}$$ steps
for the others.

###  VI-C Evaluation Metrics

We evaluated our networks with the match loss $\mathcal{L}_{\text{\sc Match}}$
and EDR distance, defined as an average of the absolute EDR error.
Furthermore, we evaluated reverberation parameter differences, i.e.,
reverberation time $\text{T}_{\text{30}}$, DRR, and clarity
$\text{C}_{\text{50}}$ difference, denoted as $\Delta\text{T}_{\text{30}}$,
$\Delta\text{DRR}$, and $\Delta\text{C}_{\text{50}}$, respectively [32, 60].
We calculated both full-band differences and average of differences measured
at octave bands of center frequencies of $125$, $250$, $500$, $1$\mathrm{k}$$,
$2$\mathrm{k}$$, $4$\mathrm{k}$$ and $8$\mathrm{k}\mathrm{H}\mathrm{z}$$.
Then, we obtained their respective median values.

We can interpret the reverberation parameter differences by comparing them
with their respective just noticeable differences [9]. Reported just
noticeable differences of $\text{T}_{\text{30}}$ from previous works vary from
$5\%$ [61] up to about $25\%$ [62]. For $\text{C}_{\text{50}}$,
$1$\mathrm{d}\mathrm{B}$$ [63] and $1.1$\mathrm{d}\mathrm{B}$$ [64] were
reported. For DRR it differs for various range, e.g.,
$6$\mathrm{d}\mathrm{B}$$ at $-10$\mathrm{d}\mathrm{B}$$,
$2$\mathrm{d}\mathrm{B}$$ at $0$\mathrm{d}\mathrm{B}$$,
$3$\mathrm{d}\mathrm{B}$$ at $10$\mathrm{d}\mathrm{B}$$, and
$8$\mathrm{d}\mathrm{B}$$ at $20$\mathrm{d}\mathrm{B}$$ [65].

###  VI-D Subjective Listening Test

Following the previous researches [10, 41], we conducted a modified MUltiple
Stimuli with Hidden Reference and Anchor (MUSHRA) test [66]. We asked subjects
to score similarities of reverberation between a given reference reverberant
speech $h*x$ and followings ($h$ and $x$ denote a test RIR and dry speech
signal, respectively).

  * •

A hidden reference $h*x$ (exactly the same audio).

  * •

A lower anchor $h^{\prime}*x$ obtained by averaging the test set RIRs [41].

  * •

Another anchor $h^{\prime\prime}*x$ obtained by random-sampling an RIR from
the test set [10].

  * •

Estimations to evaluate $\hat{H}_{1}(x),\cdots,\hat{H}_{n}(x)$. We used the AR
models $\hat{H}_{1},\cdots,\hat{H}_{n}$ to obtain the reverberant speech.

The similarity score ranged from $0$ to $100$, where we provided text
descriptions for $5$ equally-divided ranges ($0$-$20$: totally different,
$20$-$40$: considerably different, $40$-$60$: slightly different, $60$-$80$:
noticeable, but not much, and $80$-$100$: imperceptible). While we asked the
participants to score both analysis-synthesis and blind estimation results, we
divided them into two separate pages. As a result, a total of $10$ stimulus
($3$ proposed models, $4$ baselines which we will explain in Section VI-E, $2$
anchors, and $1$ hidden reference) were scored for each task. We conducted the
test using the webMUSHRA API [67]. A total of $24$ sets were scored for each
task, where the first $3$ sets were used for training. A total of $12$ people
participated where $7$ people were audio engineers and the others were
instrument players. We discarded $1$ subject’s results since he scored the
hidden references lower than $90$ for more than $15\%$ of the entire sets.

(a) ARP estimation framework without the DAR models.

(b) RIR estimation with CNN decoder model.

Figure 9: Baseline methods for the experiments.

###  VI-E Baselines

We compared our framework with following baselines. Refer to Appendix B for
further details.

####  VI-E1 Parameter-matching Networks

Assuming that the DAR models are not available, similar to the previous non-
end-to-end approach [10], we generated the training data (reference-and-ARPs
pair) with the DAR models and trained the same networks with a ARP-matching
loss (see Figure 9a). Note that we used the DAR models simply for the
convenience of on-the-fly data generation in GPU, but the same procedure can
be performed with the AR models.

####  VI-E2 CNN Decoder

We trained the same estimation network but instead of the DAR model we
attached a decoder composed of one-dimensional transposed convolutional layers
and TCNs. With this decoder, the entire network resembles an autoencoder (see
Figure 9b). From now, we denote this model as CNN. We trained two CNNs, one
for each of the two tasks.

Note that we did not compare the our methods with possible two-stage solutions
[32, 33, 34, 35, 36, 37, 38, 39] (see Section II-B3) since each DAR model’s
architecture became different from its original.

##  VII Evaluation Results

###  VII-A Comparison with the Baseline Models

####  VII-A1 Advantage of the End-to-end Learning

Figure 10 compares the proposed end-to-end approach with the baselines on
objective metrics and subjective scores. By a large margin, the end-to-end
model outperformed the non-end-to-end ARP-matching baselines. The baselines
struggled to match the reverberation decay, reporting noticeably large
reverberation time differences $\Delta T_{30}\geq 20\%$. A two-sided t-test
for each AR/DAR model and task showed that the performance difference was
statistically significant (all $p<10^{-5}$). Subjective listening test results
also agreed with the objective metrics; median scores of the end-to-end
approaches were from $70$ to $80$, showing close matches to the reference,
while the parameter matching models scored less than $60$. Also, refer to
Figure 13 that visualizes each estimation result with an EDR and EDR error
plots. This shows that the end-to-end learning enabled by the DAR models gives
a huge performance gain relative to the non-end-to-end parameter-matching
scheme.

####  VII-A2 Advantage of the DSP Prior

Since the CNN baselines have more trainable parameters and complex
architecture, they reach lower losses than the DAR-equipped networks. However,
they produce audible artifacts. While being more restrictive, the AR models
avoid such artifacts and achieve higher subjective listening scores than the
CNN baselines. Again, these subjective score differences were statistically
significant (all $p<10^{-5}$).

Figure 10: ARP estimation results on various tasks with the AR/DAR models.
Here, AS and BE denote analysis-synthesis and blind estimation, respectively.
The Full and Freq denote the full-band reverberation parameter difference and
average of octave-band reverberation parameter differences, respectively.
Along with the proposed approaches and baselines, we additionally report the
lower anchor’s evaluation results and the (hidden) reference’s MUSHRA score
result. The both-performing networks’ performance are denoted as small dots in
the figure.

Figure 11: Reliability of the DAR models. Diff, LTI, and LTV denote the
differentiable, original LTI, and the time-varying model, respectively. The
hatched results are the same values of Figure 10 (analysis-synthesis).

Figure 12:  Magnitude responses of the coloration filters of the AR models. We
used the proposed analysis-synthesis networks to obtain the filter parameters
for this plot. Each AR model has a different approach to model the frequency-
dependent characteristic of given reverberation.

###  VII-B Reliability of the DAR Models

We demonstrated in previous sections that each DAR model is close to the
original AR model, making them reliable to use as an alternative for the
training. Here, we empirically validate this argument again. Figure 11
summarizes the objective metrics calculated with the DAR and AR models. We
used the trained analysis-synthesis network for this evaluation. First,
metrics calculated between the AR model IR and the DAR model IR were orders of
magnitude smaller than the metrics calculated with the ground truth. DN was
the only exception, showing more deviation than the other AR models due to the
time variance. Nevertheless, the reverberation parameter differences between
the DN IR and differentiable model IR were less than the just noticeable
difference values (for example, $\Delta T_{30}<3\%$). Second, as expected with
equation (7), each metric difference was smaller than the corresponding AR-to-
DAR conversion error. As a result, overall evaluation results were very
similar regardless of the used model. In short, the DAR models are reliable
for training in practice.

###  VII-C Performance Difference Between Target Tasks

Without surprise, the analysis-synthesis networks performed better in most
metrics than the blind estimation networks since the reference reverberation
is directly given, i.e., the former ones have fewer burdens than the latter.
Likewise, the both-performing networks (shown as dots in Figure 10) show
slightly degraded results than their single-task counterparts. Yet, the
reverberation parameter difference only slightly increased (less than the just
noticeable differences), which hint at a possibility of a universal reference-
form-agnostic ARP estimation network.

Figure 13: EDR plots of the reference RIRs and their estimations with the
trained networks. For each reference, a total of twelve networks’ estimations
and their errors are visualized where each network is trained for one of the
three AR models (FVN, AFVN, and DN), one of the two tasks (analysis-synthesis
and blind estimation), and one of the two training approaches (end-to-end
learning with the proposed DAR models and the baseline non-end-to-end
parameter-matching learning). The number provided above each plot is the EDR
distance of the estimation.

###  VII-D AR Model Efficiency and Estimation Performance

Each AR model approximates reference reverberation differently, and
consequently, its computational efficiency and its estimation networks’
performance varies.

####  VII-D1 Filtered Velvet Noise

FVN filters each source segment independently (see Figure 12). Such
flexibility enables its parameter estimators to perform better than the other
AR models’ counterparts experimented in this paper. However, at the same time,
it also has the largest number of ARPs ($930$), which makes it cumbersome to
control manually. Also, it could be computationally expensive to use multiple
FVN instances in real-time ($2546$ FLOPs for each).

####  VII-D2 Advanced Filtered Velvet Noise

Instead, AVFN optimizes each coloration filter by decomposing it into the
initial coloration and delta-coloration filters, making the model more
controllable and efficient ($360$ ARPs and $1511$ FLOPs) than the FVN. Such
simplification costs estimation performance since the underlying assumption of
the optimization that coloration of real-world RIRs changes gradually does not
necessarily hold. In addition, the AFVN networks could be more challenging to
optimize than the FVN’s since each segment’s coloration depends on the
previous ones’. Indeed, we empirically observed that their training losses
decreased slower than the FVN’s.

####  VII-D3 Delay Network

In DN, we simplified the filter structure even further by using a single
absorption filter for each feedback loop. Similar to the AVFN case, DN trade-
offs its estimators’ performance with its controllability and efficiency
($200$ ARPs and $1285$ FLOPs). Again, the estimation performance degradation
comes from the DN’s restricted expressibility (forced exponential decay) and
training difficulty.

##  VIII Conclusion

We proposed differentiable artificial reverberation (DAR) models that can be
integrated with deep neural networks (DNNs). Among numerous pre-existing
artificial reverberation (AR) models, we selectively implemented Filtered
Velvet Noise (FVN), Advanced Filtered Velvet Noise (AFVN), and Delay Network
(DN) differentiably. Nonetheless, the proposed method, which replaces the
infinite impulse response (IIR) components with finite impulse response (FIR)
approximations via frequency-sampling, is applicable to any other AR model.
Then, using the DAR models, we trained the proposed artificial reverberation
parameter (ARP) estimation networks end-to-end. The evaluation results showed
that our networks captured the target reverberation accurately in both
analysis-synthesis and blind estimation tasks. We showed, in particular, that
the end-to-end training significantly improves the estimation performance at
the tolerable cost of the approximation errors caused by the frequency-
sampling. Additionally, we demonstrated that the structural priors of the AR
models avoid perceptual artifacts that a DNN as a room impulse response (RIR)
generator might produce. Thus, our framework successfully combined powerful
and fully general deep learning techniques with well-established domain
knowledge of reverberation.

Here, we outline the remaining challenges and future work. First, we used the
shoe-box simulation to obtain a large amount of training data, which is
slightly different from the real-world RIRs. Interestingly, energy decay
relief (EDR) errors of the end-to-end models showed similar patterns
regardless of the equipped AR/DAR model (see Figure 13). This suggests that
the data characteristics discrepancy might be the cause of the performance
degradation. Therefore, collecting more real-world RIRs and using powerful
augmentation methods could replace the simulation-based data and improve the
performance. Second, we only used the RIRs and the reverberant speech signals
as references. However, other possible applications with different reference
types exist (for example, automatic mixing of musical signals). Finally, we
investigated how each AR model can be compared in terms of its expressive
power and difficulty of parameter optimization under the deep learning
environment. The evaluation results revealed a trade-off relationship; when a
more compact AR model is used, the estimation performance was upper-bounded by
its limited expressive power. Also, the proposed modifications to the original
AR models improved estimation performance and training speed. All of these
indicate that other than the presented AR models and estimation networks,
there may be more “deep-learning-friendly” models and “AR-model-friendly” DNNs
that yield better performance-efficiency trade-offs, and seeking those is left
as future work.

##  Appendix A Ablations and Comparisons of the AR/DAR Models

###  A-A AR/DAR Model Ablations

TABLE I: Ablation on the AR/DAR Model configurations. Model | $\mathcal{L}_{\text{Match}}$ | $\Delta\text{T}_{30}\,(\%)$ | $\Delta\text{DRR}\,($\mathrm{d}\mathrm{B}$)$ | $\Delta\text{C}_{50}\,($\mathrm{d}\mathrm{B}$)$
---|---|---|---|---
$(\times 10^{-1})$ | Full | Freq | Full | Freq | Full | Freq
FVN | $\mathbf{1.236}$ | $\mathbf{1.89}$ | $\mathbf{4.20}$ | $\mathbf{0.94}$ | $\mathbf{3.39}$ | $0.60$ | $\mathbf{1.95}$
$T_{i}=10$ | $1.257$ | $2.74$ | $6.14$ | $1.56$ | $4.21$ | $\mathbf{0.48}$ | $2.25$
$\Delta l_{i}=L/S$ | $1.255$ | $2.56$ | $5.43$ | $1.48$ | $3.68$ | $0.89$ | $2.29$
$-h_{0}[n]$ | $1.250$ | $2.23$ | $10.71$ | $3.17$ | $11.94$ | $0.81$ | $9.88$
$M=1$ | $1.261$ | $2.53$ | $4.42$ | $1.51$ | $3.65$ | $0.82$ | $2.21$
AFVN | $\mathbf{1.277}$ | $\mathbf{4.77}$ | $\mathbf{9.96}$ | $1.59$ | $\mathbf{3.64}$ | $\mathbf{0.61}$ | $\mathbf{2.06}$
$-C_{1}$ | $1.446$ | $6.75$ | $12.60$ | $\mathbf{1.46}$ | $8.43$ | $1.76$ | $3.64$
$-\Delta C_{j}$ | $1.324$ | $7.12$ | $32.16$ | $2.99$ | $5.01$ | $0.92$ | $2.62$
$-C_{1},\Delta C_{j}$ | $1.695$ | $8.91$ | $28.21$ | $2.15$ | $6.62$ | $2.64$ | $5.89$
DN | $1.334$ | $8.47$ | $11.59$ | $1.71$ | $3.83$ | $1.08$ | $2.30$
$-C_{1}$ | $1.446$ | $10.47$ | $20.75$ | $3.69$ | $4.55$ | $2.31$ | $3.03$
$-\mathbf{C}_{\Delta}$ | $1.351$ | $7.83$ | $31.22$ | $2.22$ | $4.71$ | $0.94$ | $2.38$
$-C_{1},\mathbf{C}_{\Delta}$ | $1.729$ | $11.62$ | $31.20$ | $4.00$ | $6.68$ | $2.03$ | $4.54$
$-\mathbf{Q}_{n}$ | $1.307$ | $5.93$ | $12.48$ | $1.81$ | $3.93$ | $1.08$ | $2.39$
$-\mathbf{U}$ | $1.535$ | $21.10$ | $\mathbf{10.76}$ | $1.03$ | $3.44$ | $0.88$ | $2.28$
$-\mathbf{U},\mathbf{Q}_{n}$ | $\mathbf{1.251}$ | $\mathbf{7.43}$ | $10.79$ | $\mathbf{1.00}$ | $\mathbf{3.39}$ | $\mathbf{0.70}$ | $\mathbf{2.09}$
TABLE II: Differentiable FVN with various coloration filters and parameterization approaches. Filter | $\mathcal{L}_{\text{Match}}$ | $\Delta\text{T}_{30}\,(\%)$ | $\Delta\text{DRR}\,($\mathrm{d}\mathrm{B}$)$ | $\Delta\text{C}_{50}\,($\mathrm{d}\mathrm{B}$)$
---|---|---|---|---
$(\times 10^{-1})$ | Full | Freq | Full | Freq | Full | Freq
SSVF | $1.236$ | $\mathbf{1.89}$ | $\mathbf{4.20}$ | $\mathbf{0.94}$ | $\mathbf{3.39}$ | $0.60$ | $1.95$
PSVF | $\mathbf{1.235}$ | $1.95$ | $4.52$ | $0.96$ | $3.64$ | $0.53$ | $\mathbf{1.86}$
PEQ | $1.254$ | $2.50$ | $5.47$ | $0.97$ | $3.61$ | $0.62$ | $1.89$
SBIQ | $1.322$ | $3.83$ | $9.44$ | $1.55$ | $4.61$ | $0.59$ | $2.38$
PBIQ | $1.337$ | $4.88$ | $11.77$ | $1.57$ | $4.01$ | $0.62$ | $2.19$
FIR | $1.370$ | $2.72$ | $10.10$ | $1.80$ | $3.73$ | $\mathbf{0.51}$ | $2.16$
LFIR | $1.384$ | $3.54$ | $11.84$ | $1.64$ | $4.31$ | $0.62$ | $2.65$

We conducted ablation studies to verify that the modifications to the original
AR models improve the estimation performance. We evaluated the modified models
with the analysis-synthesis task since their performance differences remained
the same regardless of the task in our initial experiments.

Table I summarizes the results. FVN, AFVN, and DN denote the proposed models.
$T_{i}=10$, $\Delta l_{i}=L/S$, $-h_{0}[n]$, and $M=1$ denote the proposed FVN
model without non-uniform pulse distance and allpass filters, non-uniform
segmentation, a deterministic FIR, and finer segment gains. Their results show
that the proposed modifications contribute to better performance. We omitted
the same ablations for the AFVN. Instead, we discarded the initial filter
$C_{1}$, delta filter $\Delta C_{j}$, or both. We also evaluated the DN
without the initial filter $C_{1}$ or delta filter $\mathbf{C}_{\Delta}$. The
results reveal the importance of modelling the frequency-dependent nature of
reverberation.

For DN, $-\mathbf{Q}_{n}$, $-\mathbf{U}$, and $-\mathbf{U},\mathbf{Q}_{n}$
denote DN without the time-varying mixing matrix $\mathbf{Q}_{n}$, the allpass
filters $\mathbf{U}$, and both, respectively. While
$-\mathbf{U},\mathbf{Q}_{n}$ reports best evaluation metrics, its low echo
density produced unrealistic sound. Using both $\mathbf{U}$ and
$\mathbf{Q}_{n}$ improved plausibility of reverberation with an acceptable
performance degradation. Refer to the provided audio samples.

###  A-B Comparisons on Different Coloration Filters

We compared different IIR/FIR filters and parameterization approaches. We
evaluated them with the analysis-synthesis task using FVN. Table II summarizes
the results. SSVF, PSVF, PEQ, SBIQ, PBIQ, FIR, and LFIR denote serial SVF,
parallel SVF, PEQ, serial biquad, parallel biquad, FIR, and linear-phase FIR
from [29]. The order of every filter was set to $40$. The activation functions
and bias initialization for each filter’s decoder are as follows. PSVF has the
same setting as SSVF. For SBIQ and PBIQ decoders, we used activations from
[20], and their biases are initialized to match that of SSVF and PSVF. For
PEQ, unlike DN, we used $10^{x}$ for the activation of the gain $\mathbf{G}$.
FIR/LFIR have no activation and custom bias initialization.

In spite of having identical expressive power, SSVF/PSVF performed better than
SBIQ/PBIQ by large margins. PEQ performed slightly worse than SSVF due to its
restricted degree of freedom. FIR/LFIR also performs worse than SSVF/PSVF and
PEQ because they cannot change their frequency responses radically as other
IIR filters could.

TABLE III: Effects of frequency-sampling resolution and regularization on the match loss $\mathcal{L}_{\text{Match}}\,(\times 10^{-1})$. Model | Frequency-Sampling Resolution
---|---
$\times 1$ (Full) | $\times 0.5$ | $\times 0.25$ | $\times 0.125$
DAR | AR | DAR | AR | DAR | AR | DAR | AR
$\text{FVN}_{\beta=1}$ | $\mathbf{{1.236}}$ | $\mathbf{{1.236}}$ | ${1.259}$ | $1.259$ | ${1.252}$ | $1.249$ | ${1.574}$ | $1.570$
$\text{FVN}_{\beta=0}$ | $\mathbf{{{1.231}}}$ | ${1.237}$ | ${1.283}$ | $1.282$ | ${1.264}$ | $1.263$ | ${1.280}$ | $1.773$
DN | $\mathbf{{1.277}}$ | ${1.307}$ | $-$ | $1.328$ | $-$ | $1.361$ | $-$ | $1.393$

###  A-C Effects of the Frequency-Sampling Resolution

Table III demonstrates the effects of the frequency-sampling resolution and
the regularization loss $\mathcal{L}_{\text{\sc Reg}}$. From the table,
$\times x$ denotes the relative sampling resolution compared to the default
configuration, e.g., $\times 0.5$ denotes $N=2$\mathrm{k}$$ for the FVN and
$N=6$\mathrm{k}$$ for the DN. For each resolution and regularization setup, we
report match loss with the DAR and AR models. Results of the differentiable
DNs with lower resolutions are omitted since they are calculated with shorter
IRs.

As expected, higher sampling resolution led to smaller loss difference. Also,
introducing the regularization term ($\beta=1$) reduced the difference
further. As a result, the final differentiable FVN model (with full resolution
and regularization) showed little loss difference. Moreover, higher resolution
led to better performance, which might be because increased resolution helped
each network to find better ARPs.

##  Appendix B Details on the Baseline Methods

###  B-A ARP Match Models

####  B-A1 Training

We trained the ARP match baseline models as follows. First, we randomized ARPs
$\mathbf{P}_{1},\cdots,\mathbf{P}_{n}$ and generated an IR with using the DAR
model. Then, each baseline network estimated ARPs
$\hat{\mathbf{P}}_{1},\cdots,\hat{\mathbf{P}}_{n}$ from the given IR. We
evaluated the estimation and trained the baseline using an ARP-match loss
$\mathcal{L}_{\text{ARP}}$ defined as follows,

$\mathcal{L}_{\text{ARP}}(\mathbf{P}_{i},\hat{\mathbf{P}}_{i})=\sum_{i}\alpha_{i}\left\|f_{i}(\mathbf{P}_{i})-f_{i}(\hat{\mathbf{P}}_{i})\right\|_{1}.$
(26)

Here, the index $i$ denotes a different ARP tensor, e.g.,
$\mathbf{P}_{0}=\mathbf{g}$ and $\mathbf{P}_{1}=\mathbf{h}_{0}$.
$f_{i}(\cdot)$ and $\alpha_{i}$ are an elementwise function and a constant,
respectively. We set $\alpha_{i}=10$ for the all DAR models’ bypass FIRs and
the DN’s absorption filter parameters. We used $f_{i}(x)=\log_{10}(x)$ for the
FVN and AFVN segment gains. We chose $f_{i}(x)=x$ and $\alpha_{i}=1$
otherwise.

####  B-A2 Network Architecture

We trained an almost identical network to the proposed network for each
model/task. The only difference is that we added an activation
$10^{-\zeta(x)/20}$ to the $\mathbf{g}$ decoders of FVN and AFVN to improve
their performance.

####  B-A3 Data Generation

We tuned each DAR model’s ARP randomization scheme to match the synthesized
IRs’ reverberation parameter statistics to the validation set. For the FVN
baselines, we first sampled a reverberation time
$\text{T}_{\text{30}}\sim\mathcal{U}_{\log}[50$\mathrm{m}\mathrm{s}$,8$\mathrm{s}$)$
then generated the segment gains $\mathbf{g}$ that match the
$\text{T}_{\text{30}}$. Here, $\mathcal{U}_{\log}[\cdot,\cdot)$ denotes a
uniform distribution in log scale. In this procedure, we also compensated the
average pulse distance by dividing each gain $g_{i}$ by $\sqrt{T_{i}}$. For
the initial coloration filter $C_{1}$, we first generated PEQ parameters as
$\omega_{i}\sim\mathcal{U}_{\log}[40$\mathrm{H}\mathrm{z}$,16$\mathrm{k}\mathrm{H}\mathrm{z}$)$,
$R_{i}\sim\mathcal{U}_{\log}[0.2,5)$, and
$G_{i}\sim\mathcal{U}[-18$\mathrm{d}\mathrm{B}$,18$\mathrm{d}\mathrm{B}$)$ (we
sorted the cutoff frequencies after the sampling) and derived their SVF
parameters. After that, we perturbed each parameter’s value slightly. This ARP
generation method was motivated by the observation that the differentiable
SVFs act like a relaxed PEQ (see Figure 12). We gradually changed the
parameters of $C_{1}$ to obtain $C_{2},\cdots,C_{K}$, modeling the frequency-
dependent decay. Finally, we set $h_{0}[n]$ as an uniform noise with a gain
sampled from $\mathcal{U}[-24$\mathrm{d}\mathrm{B}$,0$\mathrm{d}\mathrm{B}$)$.
We followed similar procedures for the AFVN baselines. One difference is that
the first delta filter $C_{\Delta 2}$ is randomized and the delta filters
afterwords are slight deviation of the first one. For the DN baselines, we
sampled each absorption PEQ’s gain with
$G_{i}\sim\mathcal{U}[-2.8$\mathrm{d}\mathrm{B}$,0$\mathrm{d}\mathrm{B}$)$.

###  B-B DNN Decoder Models

Regarding the first and second axis of the shared latent $\mathbf{z}$ as a
time and channel axis, respectively, our decoder upsamples $\mathbf{z}$ with
seven one-dimensional transposed convolution layers. Configurations of these
layers are as follows. Channels: $128$, $128$, $64$, $64$, $64$, $64$, then
$64$. Kernel sizes: $5$, $5$, $5$, $5$, $5$, $5$, then $3$. Strides: $3$, $3$,
$3$, $3$, $3$, $3$, then $2$. Furthermore, after each upsampling, we inserted
a small temporal convolutional network with configurations as follows.
Channels: $128$, $128$, $64$, $64$, $64$, $64$, and $64$. The number of
layers: $3$, $3$, $2$, $2$, $2$, $2$, and $2$. Kernel sizes: all $7$. Finally,
we added $1\times 1$ convolution as the last layer to mix all channels. We
used the ReLU activation between the layers. This decoder has about
$2.5$\mathrm{M}$$ parameters.

##  Appendix C Details on the Frequency-sampling Method

###  C-A Proof of Equation 6

With equation (5), the time-aliasing error can be written as

$\|H-H_{N}\|_{2}^{2}=\sum_{n=0}^{N-1}\left|\sum_{m=1}^{\infty}h[mN+n]\right|^{2}+\sum_{n=N}^{\infty}\left|h[n]\right|^{2}.$
(27)

With the triangle inequality and $\|X\|_{2}^{2}\leq\|X\|_{1}^{2}$, we can
upper-bound $\|H-H_{N}\|_{2}^{2}$ with $2S_{N}^{2}$ where
$S_{N}=\sum_{n=N}^{\infty}\left|h[n]\right|$ is an absolute sum of the tail of
the original IR $h[n]$ $(n\geq N)$. Next, we expand $h[n]$ with partial
fraction expansion [68, 69]. For $H(z)$ with $M$ distinct poles where each of
them $\nu_{i}\in\mathbb{C}$ has multiplicity $r_{i}\in\mathbb{N}$, we can
express $h[n]$ as a sum of an FIR $h^{\text{FIR}}[n]$ and IIRs
$h^{\text{IIR}}_{i,k}[n]$ where following holds,

$H(z)=H^{\text{FIR}}(z)+\sum_{i=1}^{M}\sum_{k=1}^{r_{i}}\underbrace{\frac{\zeta_{i,k}}{(1-\nu_{i}z^{-1})^{k}}}_{H^{\text{IIR}}_{i,k}(z)}.\quad\left(\zeta_{i,k}\in\mathbb{C}\right)$
(28)

Then, applying the triangle inequality to the summation gives an upper bound
$U_{N}$, an absolute sum of the IIRs’ tails.

$S_{N}\leq\sum_{i=1}^{M}\sum_{k=1}^{r_{i}}\sum_{n=N}^{\infty}\left|h^{\text{IIR}}_{i,k}[n]\right|=U_{N}.$
(29)

From above equation, each IIR, first sum, and second sum show
$O(n^{k-1}|\nu_{i}|^{n})$, $O(N^{k-1}|\nu_{i}|^{N})$, and
$O(N^{r_{i}-1}|\nu_{i}|^{N})$ asymptotic behavior, respectively [69]. We
conclude the proof:

$\|H-H_{N}\|_{2}\leq\sqrt{2}S_{N}\leq\sqrt{2}U_{N}=\sum_{i=1}^{M}O(N^{r_{i}-1}\left|\nu_{i}\right|^{N}).$
(30)

###  C-B Proof of Equation 8

We denote $H(e^{j\omega})$ and $\partial X/\partial p$ with $H$ and
$X^{\prime}$. $X_{N}^{\prime}$ denotes $(X_{N})^{\prime}=(X^{\prime})_{N}$. We
upper-bound the gradient difference $\Delta\mathcal{G}$ using integral
inequalities as

$\Delta\mathcal{G}\leq\underbrace{\int_{0}^{2\pi}|H_{\text{Ref}}(H^{\prime}-H_{N}^{\prime})|\frac{d\omega}{\pi}}_{\leq
2\|H_{\text{Ref}}\|_{2}\|H-H_{N}^{\prime}\|_{2}}+\underbrace{\int_{0}^{2\pi}|HH^{\prime}-H_{N}H_{N}^{\prime}|\frac{d\omega}{\pi}}_{\leq
2\|HH^{\prime}-H_{N}H^{\prime}_{N}\|_{2}}$ (31)

where the under-braced ones are the Cauchy-Schwartz inequalities. Considering
the first integral’s upper bound, $\|H_{\text{Ref}}\|_{2}$ is a finite
constant. In $\|H^{\prime}-H_{N}^{\prime}\|_{2}$, $H^{\prime}$ is another
stable LTI filter if $p$ is well-defined. If $\nu_{i}$ is a function of $p$,
its multiplicity $r_{i}$ is doubled in $H^{\prime}$. In the worst case, $p$
controls every pole of $H$ and
$\|H^{\prime}-H_{N}^{\prime}\|_{2}=\sum_{i=1}^{M}O(N^{2r_{i}-1}|\nu_{i}|^{N})$
holds. Similarly, the upper bound of the second integral consists an $l_{2}$
distance between $HH^{\prime}$ and $H_{N}H_{N}^{\prime}$. Since $HH^{\prime}$
has $2r_{i}$ or $3r_{i}$ multiplicity for each pole $\nu_{i}$,
$\|HH^{\prime}-H_{N}H^{\prime}_{N}\|_{2}=\sum_{i=1}^{M}O(N^{3r_{i}-1}|\nu_{i}|^{N})$
holds at the worst case, which concludes the proof.

## References

  * [1] J. W. Bayless, “Innovations in studio design and construction in the capitol tower recording studios,” _Journal of the Audio Engineering Society_ , vol. 5, no. 2, pp. 71–76, April 1957.
  * [2] B. A. Blesser, “An interdisciplinary synthesis of reverberation viewpoints,” _Journal of the Audio Engineering Society_ , vol. 49, no. 10, pp. 867–903, October 2001.
  * [3] J. Traer and J. H. McDermott, “Statistics of natural reverberation enable perceptual separation of sound and space,” _Proceedings of the National Academy of Sciences_ , vol. 113, no. 48, pp. E7856–E7865, 2016.
  * [4] V. Välimäki, J. D. Parker, L. Savioja, J. O. Smith, and J. S. Abel, “Fifty years of artificial reverberation,” _IEEE Transactions on Audio, Speech, and Language Processing_ , vol. 20, no. 5, pp. 1421–1448, 2012\.
  * [5] V. Välimäki, J. Parker, L. Savioja, J. O. Smith, and J. Abel, “More than 50 years of artificial reverberation,” _Journal of the Audio Engineering Society_ , January 2016.
  * [6] J. Dattorro, “Effect design, part 1: reverberator and other filters,” _Journal of the Audio Engineering Society_ , vol. 45, no. 9, pp. 660–684, 1997.
  * [7] W. G. Gardner, “A real‐time multichannel room simulator,” _Journal of the Acoustical Society of America_ , vol. 92, pp. 2395–2395, 1992.
  * [8] E. De Sena, H. Hacιhabiboğlu, Z. Cvetković, and J. O. Smith, “Efficient synthesis of room acoustics via scattering delay networks,” _IEEE/ACM Transactions on Audio, Speech, and Language Processing_ , vol. 23, no. 9, pp. 1478–1492, 2015.
  * [9] N. Agus, H. Anderson, J.-M. Chen, S. Lui, and D. Herremans, “Minimally simple binaural room modeling using a single feedback delay network,” _Journal of the Audio Engineering Society_ , vol. 66, no. 10, pp. 791–807, October 2018\.
  * [10] A. Sarroff and R. Michaels, “Blind arbitrary reverb matching,” in _Proceedings of the 23rd International Conference on Digital Audio Effects_ , 2020.
  * [11] J.-M. Jot, “An analysis/synthesis approach to real-time artificial reverberation,” in _[Proceedings] ICASSP-92: 1992 IEEE International Conference on Acoustics, Speech, and Signal Processing_ , vol. 2, 1992, pp. 221–224.
  * [12] J. Coggin and W. Pirkle, “Automatic design of feedback delay network reverb parameters for impulse response matching,” _Journal of the Audio Engineering Society_ , September 2016.
  * [13] V. Välimäki, B. Holm-Rasmussen, B. Alary, and H.-M. Lehtonen, “Late reverberation synthesis using filtered velvet noise,” _Applied Sciences_ , vol. 7, no. 5, 2017.
  * [14] N. Peters, J. Choi, and H. Lei, “Matching artificial reverb settings to unknown room recordings: a recommendation system for reverb plugins,” _Journal of the Audio Engineering Society_ , October 2012.
  * [15] F. Gao and W. Snelgrove, “An adaptive backpropagation cascade IIR filter,” _IEEE Transactions on Circuits and Systems II: Analog and Digital Signal Processing_ , vol. 39, no. 9, pp. 606–610, 1992.
  * [16] A. D. Back and A. C. Tsoi, “FIR and IIR synapses, a new neural network architecture for time series modeling,” _Neural Computation_ , vol. 3, no. 3, pp. 375–385, 1991.
  * [17] P. Campolucci, A. Uncini, and F. Piazza, “Fast adaptive IIR-MLP neural networks for signal processing applications,” in _1996 IEEE International Conference on Acoustics, Speech, and Signal Processing Conference Proceedings_ , vol. 6, 1996, pp. 3529–3532 vol. 6.
  * [18] J. D. P. Boris Kuznetsov and F. Esqueda, “Differentiable IIR filters for machine learning application,” in _Proceedings of the 23rd International Conference on Digital Audio Effects_ , 2020.
  * [19] S. Nercessian, “Neural parametric equalizer matching using differentiable biquads,” in _Proceedings of the 23rd International Conference on Digital Audio Effects_ , 2020.
  * [20] S. Nercessian, A. Sarroff, and K. J. Werner, “Lightweight and interpretable neural modeling of an audio distortion effect using hyperconditioned differentiable biquads,” _arXiv preprint arXiv:2103.08709_ , 2021.
  * [21] L. Rabiner, B. Gold, and C. McGonegal, “An approach to the approximation problem for nonrecursive digital filters,” _IEEE Transactions on Audio and Electroacoustics_ , vol. 18, no. 2, pp. 83–106, 1970.
  * [22] J. Stautner and M. Puckette, “Designing multichannel reverberators,” _Computer Music Journal_ , vol. 3, pp. 569–582, 1989.
  * [23] M. Gerzon, “Synthetic reverberation, part I,” _Studio Sound_ , vol. 13, pp. 632–635, 1971.
  * [24] S. Bai, J. Z. Kolter, and V. Koltun, “An empirical evaluation of generic convolutional and recurrent networks for sequence modeling,” 2018.
  * [25] C. J. Steinmetz and J. D. Reiss, “Efficient neural networks for real-time analog audio effect modeling,” _arXiv preprint arXiv:2102.06200_ , 2021.
  * [26] C. J. Steinmetz, J. Pons, S. Pascual, and J. Serrà, “Automatic multitrack mixing with a differentiable mixing console of neural audio effects,” in _ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)_ , 2021, pp. 71–75.
  * [27] S. Billings and S. Fakhouri, “Identification of systems containing linear dynamic and static nonlinear elements,” _Automatica_ , vol. 18, no. 1, pp. 15–26, 1982.
  * [28] X. Serra and J. Smith, “Spectral modeling synthesis: A sound analysis/synthesis based on a deterministic plus stochastic decomposition,” _Computer Music Journal_ , vol. 14, pp. 12–24, 1990.
  * [29] J. Engel, L. H. Hantrakul, C. Gu, and A. Roberts, “DDSP: differentiable digital signal processing,” in _International Conference on Learning Representations_ , 2020.
  * [30] V. Zavalishin, _The Art of VA Filter Design_. Native Instruments, 2020.
  * [31] A. Wishnick, “Time-varying filters for musical applications,” in _Proceedings of the 17th International Conference on Digital Audio Effects_ , 2014.
  * [32] “Acoustics — Measurement of room acoustic parameters — Part 2: Reverberation time in ordinary rooms,” International Organization for Standardization, Geneva, CH, Standard, Mar. 2008.
  * [33] R. Ratnam, D. L. Jones, B. C. Wheeler, W. D. O’Brien, C. R. Lansing, and A. S. Feng, “Blind estimation of reverberation time,” _The Journal of the Acoustical Society of America_ , vol. 114, no. 5, pp. 2877–2892, 2003.
  * [34] J. Y. C. Wen, E. A. P. Habets, and P. A. Naylor, “Blind estimation of reverberation time based on the distribution of signal decay rates,” in _2008 IEEE International Conference on Acoustics, Speech and Signal Processing_ , 2008, pp. 329–332.
  * [35] H. Gamper and I. J. Tashev, “Blind reverberation time estimation using a convolutional neural network,” in _2018 16th International Workshop on Acoustic Signal Enhancement (IWAENC)_ , 2018, pp. 136–140.
  * [36] S. Li, R. Schlieper, and J. Peissig, “A hybrid method for blind estimation of frequency dependent reverberation time using speech signals,” in _ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)_ , 2019, pp. 211–215.
  * [37] S. Schlecht and E. Habets, “Accurate reverberation time control in feedback delay networks,” in _Proceedings of the 20th International Conference on Digital Audio Effects_ , September 2017.
  * [38] K. Prawda, V. Välimäki, and S. Schlecht, “Improved reverberation time control for feedback delay networks,” in _Proceedings of the 22nd International Conference on Digital Audio Effects_ , September 2019.
  * [39] E. T. Chourdakis and J. D. Reiss, “A machine-learning approach to application of intelligent artificial reverberation,” _Journal of the Audio Engineering Society_ , vol. 65, no. 1/2, pp. 56–65, January 2017.
  * [40] A. Richard, D. Markovic, I. D. Gebru, S. Krenn, G. A. Butler, F. Torre, and Y. Sheikh, “Neural synthesis of binaural speech,” in _International Conference on Learning Representations_ , 2021.
  * [41] C. J. Steinmetz, V. K. Ithapu, and P. Calamia, “Filtered noise shaping for time domain room impulse response estimation from reverberant speech,” _arXiv preprint arXiv:2107.07503_ , 2021.
  * [42] J. O. Smith, _Mathematics of the Discrete Fourier Transform (DFT)_. http://www.w3k.org/books/http://www.w3k.org/books/: W3K Publishing, 2007.
  * [43] V. Välimäki, H.-M. Lehtonen, and M. Takanen, “A perceptual study on velvet noise and its variants at different pulse densities,” _IEEE Transactions on Audio, Speech, and Language Processing_ , vol. 21, no. 7, pp. 1481–1488, 2013.
  * [44] H. Järveläinen and M. Karjalainen, “Reverberation modeling using velvet noise,” _journal of the audio engineering society_ , March 2007.
  * [45] M. R. Schroeder and B. F. Logan, “‘colorless’ artificial reverberation,” _Journal of the Audio Engineering Society_ , vol. 9, no. 3, pp. 192–197, July 1961.
  * [46] J.-M. Jot and A. Chaigne, “Digital delay networks for designing artificial reverberators,” _Journal of the Audio Engineering Society_ , February 1991\.
  * [47] D. Rocchesso and J. O. Smith, “Circulant and elliptic feedback delay networks for artificial reverberation,” _IEEE Transactions on Speech and Audio Processing_ , vol. 5, no. 1, pp. 51–63, 1997.
  * [48] S. J. Schlecht and E. A. P. Habets, “Time-varying feedback matrices in feedback delay networks and their application in artificial reverberation,” _The Journal of the Acoustical Society of America_ , vol. 138, no. 3, pp. 1389–1398, 2015.
  * [49] S. Schlecht, “Fdntb: The feedback delay network toolbox,” in _Proceedings of the 23rd International Conference on Digital Audio Effects_ , 2020.
  * [50] J. Chung, C. Gulcehre, K. Cho, and Y. Bengio, “Empirical evaluation of gated recurrent neural networks on sequence modeling,” _arXiv preprint arXiv:1412.3555_ , 2014.
  * [51] J. L. Ba, J. R. Kiros, and G. E. Hinton, “Layer normalization,” _arXiv preprint arXiv:1607.06450_ , 2016.
  * [52] S. Shelley and D. Murphy, “Openair: An interactive auralization web resource and database,” _129th Audio Engineering Society Convention 2010_ , vol. 2, pp. 1270–1278, 01 2010.
  * [53] J. Eaton, N. D. Gaubitch, A. H. Moore, and P. A. Naylor, “Estimation of room acoustic parameters: The ace challenge,” _IEEE/ACM Transactions on Audio, Speech, and Language Processing_ , vol. 24, no. 10, pp. 1681–1693, 2016\.
  * [54] P. Svensson and U. R. Kristiansen, “Computational modelling and simulation of acoutic spaces,” _Journal of the Audio Engineering Society_ , June 2002.
  * [55] R. Scheibler, E. Bezzam, and I. Dokmanic, “Pyroomacoustics: A python package for audio room simulations and array processing algorithms,” _arXiv preprint arXiv:1710.04196_ , 2017.
  * [56] G. Defrance, L. Daudet, and J.-D. Polack, “Finding the onset of a room impulse response: straightforward?” _The Journal of the Acoustical Society of America_ , vol. 124, no. 4, pp. EL248–EL254, 2008.
  * [57] N. J. Bryan, “Impulse response data augmentation and deep neural networks for blind room acoustic parameter estimation,” in _2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)_ , 2020, pp. 1–5.
  * [58] J. Yamagishi, C. Veaux, K. MacDonald _et al._ , “Cstr vctk corpus: English multi-speaker corpus for cstr voice cloning toolkit (version 0.92),” 2019.
  * [59] D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” _arXiv preprint arXiv:1412.6980_ , 2017.
  * [60] C. Hak, R. Wenmaekers, and L. Luxemburg, “Measuring room impulse responses: Impact of the decay range on derived room acoustic parameters,” _Acta Acustica united with Acustica_ , vol. 98, 11 2012.
  * [61] Z. Meng, F. Zhao, and M. He, “The just noticeable difference of noise length and reverberation perception,” in _2006 International Symposium on Communications and Information Technologies_ , 2006, pp. 418–421.
  * [62] M. G. Blevins, A. Buck, Z. E. Peng, and L. Wang, “Quantifying the just noticeable difference of reverberation time with band-limited noise centered around 1000 hz using a transformed up-down adaptive method,” June 2013.
  * [63] J. Bradley, R. Reich, and S. Norcross, “A just noticeable difference in c50 for speech,” _Applied Acoustics_ , vol. 58, no. 2, pp. 99 – 108, 1999.
  * [64] F. Martellotta, “The just noticeable difference of center time and clarity index in large reverberant spaces,” _The Journal of the Acoustical Society of America_ , vol. 128, pp. 654–63, 08 2010.
  * [65] E. Larsen, N. Iyer, C. Lansing, and A. Feng, “On the minimum audible difference in direct-to-reverberant energy ratio,” _The Journal of the Acoustical Society of America_ , vol. 124, pp. 450–61, 07 2008.
  * [66] “Method for the subjective assessment of intermediate quality level of audio systems,” International Telecommunication Union Radio Communication Assembly, Geneva, CH, Standard, Mar. 2014.
  * [67] M. Schoeffler, S. Bartoschek, F.-R. Stöter, M. Roess, S. Westphal, B. Edler, and J. Herre, “webMUSHRA — a comprehensive framework for web-based listening tests. journal of open research software,” _Journal of Open Research Software_ , vol. 6, p. 8, 2018.
  * [68] J. Liski, B. Bank, J. Smith, and V. Välimäki, “Converting series biquad filters into delayed parallel form: Application to graphic equalizers,” _IEEE Transactions on Signal Processing_ , vol. PP, pp. 1–1, 05 2019.
  * [69] J. O. Smith, _Introduction to Digital Filters with Audio Applications_. W3K Publishing, 2007.