Reversibility, Irreversibility, Friction and

nonequilibrium ensembles in N-S equations

Giovanni Gallavotti111Posta-e<EMAIL_ADDRESS>
Web: https://ipparco.roma1.infn.it

Università “La Sapienza” e INFN, Roma

Abstract: Viscosity, as a physical property of fluids, reflects an average
effect over a chaotic microscopic motion described by Hamiltonian equations.
It is proposed, as an example, that stationary states of an incompressible
fluid subject to a constant force, can be described via several ensembles, in
strict analogy with equilibrium Statistcal Mechanics.

##  I Regular NS evolutions

Question: can the phenomenological notion of friction be represented in
alternative ways?

Related Question: is it possible to set up a theory of statistical ensembles,
and their equivalence, extending to stationary non-equilibria the ideas behind
the canonical and microcanonical ensembles, [1, 2].

A guide could be provided by the existence of a fundamental symmetry like
“time reversal” which cannot be “spontaneouly broken”.222In the subatomic
world time reversal is not a symmetry but another more fundamental symmetry,
CPT, could replace it in the following discussion

Therefore even the stationary states of dissipative systems ought to be
describable via time reversible equations. Clearly the question is not an easy
one: hence it will be better to specialize here on a paradigmatic example,
namely the Navier-Stokes (NS) fluid in a $2\pi$-periodic box, in $2$ or $3$
dimensions (2D or 3D), viscosity $\nu$.

The $d$D equation, $d=2,3$, is:

$\displaystyle{\dot{{\bf u}}({{\bf x}})_{a}=}$ | $\displaystyle{{}-(({{\bf u}}\cdot{\mbox{\boldmath$\partial$}}){{\bf u}})_{a}-{\mbox{\boldmath$\partial$}}_{a}p}$
---|---
| $\displaystyle{{}+\nu\Delta{{\bf u}}_{a}+{\bf
F}_{a}=0,\quad{\mbox{\boldmath$\partial$}}\cdot{{\bf u}}=0}$
(1.1)

where ${{\bf u}}({{\bf x}})$ is the velocity field that in 2D can be
represented via Fourier’s series as:

${{\bf u}}({{\bf x}})_{a}=\sum_{{{\bf k}}\neq\bf 0}iu_{{{\bf k}}}\frac{{{\bf
k}}^{\perp}_{a}}{||{{\bf k}}||}e^{-i{{\bf k}}\cdot{{\bf x}}}$ (1.2)

and $\bf F$ is likewise represented via its Fourier’s series.

In terms of the complex scalars $u_{{\bf k}}={\overline{{\bf u}}}_{-{{\bf
k}}}$ the 2D NS equation is:

$\displaystyle{\dot{u}_{{\bf k}}=}$ | $\displaystyle{{}-\sum_{{{\bf k}}_{1}+{{\bf k}}_{2}={{\bf k}}}\frac{({{\bf k}}_{1}^{\perp}\cdot{{\bf k}}_{2})({{\bf k}}_{2}\cdot{{\bf k}})}{||{{\bf k}}_{1}||\,||{{\bf k}}_{2}||\,||{{\bf k}}||}u_{{{\bf k}}_{1}}u_{{{\bf k}}_{2}}}$
---|---
| $\displaystyle{{}-\nu{{\bf k}}^{2}u_{{\bf k}}+F_{{\bf k}}}$
(1.3)

Although the 2D-NS admit general smooth solution $t\to S_{u}{{\bf u}}$
starting from smooth initial data ${{\bf u}}$, it is convenient (aiming to
discuss also the 3D-NS) to imagine them as truncated at $|{{\bf
k}}|=\max_{i}|k_{i}|\leq N$. The ultraviolet (UV) cut-off $N$ will be
temporarily fixed. The 2D-NS become $M_{N}=(2N+1)^{2}-1$ dimensional ODE’s, on
phase space ${\mathcal{M}}_{N}$.

In the 3D case the 3D-NS equations with UV cut-off $N$ can be likewise written
on a $M_{N}=2((2N+1)^{3}-1)$ dimensional phase space ${\mathcal{M}}_{N}$.

The time reversal transformation $I{{\bf u}}=-{{\bf u}}$ does not imply
$IS_{t}{{\bf u}}=S_{-t}I{{\bf u}}$ if $\nu>0$: hence these are irreversible
equations.

Let ${{\bf u}}$ be an initial state: then $t\to S_{t}{{\bf u}}{\buildrel
def\over{=}}{{\bf u}}(t)$ evolves and it is easly seen that $||S_{t}{{\bf
u}}||$ will verify an a priori estimate on $\|{{\bf
u}}(t)\|^{2}_{2}=\sum_{{\bf k}}|u_{{\bf k}}(t)|^{2}$:

$\|{{\bf u}}(t)\|^{2}_{2}=\leq\|{{\bf u}}(0)\|^{2}_{2}+\frac{\|\bf
F\|^{2}_{2}}{\nu^{2}}<\Big{(}\frac{\|\bf F\|^{2}_{2}}{\nu^{2}}\Big{)}_{+}$
(1.4)

where the last inequality abridges stating that if $\|{{\bf u}}(0)\|^{2}_{2}$
is larger than $\frac{\|\bf F\|^{2}_{2}}{\nu^{2}}$ then it will decrease to
become eventually, smaller than any target $>\frac{\|\bf
F\|^{2}_{2}}{\nu^{2}}$ (in a time depending on ${{\bf u}}(0)$ and on the
prefixed target).

Definition: A smooth dynamical system $S_{t}$ on a manifold ${\mathcal{M}}$
will be called regular if in ${\mathcal{M}}$ there are a finte number of open
sets ${\mathcal{A}}_{i},i=1,2,\ldots,n$ which:
(1) are $S_{t}$-invariant,
(2) their union is ${\mathcal{M}}$ up to a set of zero volume, (3) the
evolution of almost all data ${{\bf u}}\in{\mathcal{A}}_{i}$ assigns an
average value to the observables $O$ (i.e. to functions on ${\mathcal{M}}$)
$\mu_{i}(O)$ where $\mu_{i}$ is an invariant distribution.
The $\mu_{i}$ will be called “physical distributions” and if $n>1$ the system
will be said to admit $n$ phases.

The distributions are called “physical dustribution” because they allow to
compute the time averages of functions $O({{\bf u}})$ on phase space: hence
determine the statistical properties of almost all data.

If the motion is chaotic then generates a “stationary state” on
${\mathcal{M}}_{N}$, i.e. a stationary probability distribution which, aside
exceptions on the initial data collected in a $0$-volume set in
${\mathcal{M}}_{N}$, will be supposed unique, for simplicity, and denoted
$\mu_{\nu}(d{{\bf u}})$.

Stationary probability distributions generalize to the case of much more
general systems, the regular ones, what in many classes of Hamiltonian systems
are the equilibrium distributions studied in equilibrium statistical mechanics
(SM).

Here we assume that the regularized NS equations, denoted $INS^{N}$, on
${\mathcal{M}}_{N}$ parameterized by the viscosity $\nu$ are regular in the
above sense. And it is natural to define the collection
${\mathcal{E}}^{N}_{viscosity}$ of all physical distributions
$\mu^{N}_{\nu}(d{{\bf u}})$ on ${\mathcal{M}}_{N}$. In the cases in which
there are several physical distributions corresponding to the same viscosity a
further label $\beta=1,\ldots,n_{\nu}^{N}$ will be attached as
$\mu^{N}_{\nu,j}$ to distinguish them.

Furthermore we fix once and for all the forcing $\Phi$, with $||\bf
F||_{2}=1$, and with $|F_{{\bf k}}|=0$, if $|{{\bf k}}|>k_{max}$ with
$k_{m}ax<\infty$: physically this is read that $\bf F$ is assumed to be a
large scale forcing.

##  II Enstrophy ensemble

Consider the new equation, [3], with UV-cut-off $N$:

$\displaystyle{\dot{u}_{{\bf k}}=}$ | $\displaystyle{{}-\sum_{{{\bf k}}_{1}+{{\bf k}}_{2}={{\bf k}}}\frac{({{\bf k}}_{1}^{\perp}\cdot{{\bf k}}_{2})({{\bf k}}_{2}\cdot{{\bf k}})}{||{{\bf k}}_{1}||\,||{{\bf k}}_{2}||\,||{{\bf k}}||}u_{{{\bf k}}_{1}}u_{{{\bf k}}_{2}}}$
---|---
| $\displaystyle{{}-\alpha({{\bf u}}){{\bf k}}^{2}u_{{\bf k}}+F_{{\bf
k}},\qquad|{{\bf k}}|,|{{\bf k}}_{1}|,|{{\bf k}}_{2}|\leq N}$
(2.1)

differing from the Eq.(1.3) because the multiplier $\alpha({{\bf u}})$
replaces the viscosity $\nu$. The non linear term in Eq.(2.1) will be denoted
${\bf n}({{\bf u}},{{\bf u}})_{{\bf k}}$.

The multiplier $\alpha({{\bf u}})$ will be so defined that the solutions of
the Eq.(2.1) will admidt ${\mathcal{D}}({{\bf u}}){\buildrel
def\over{=}}\sum_{{\bf k}}{{\bf k}}^{2}|u_{{\bf k}}|^{2}$, usually called
enstrophy, as an exact constant of motion. From Eq.(2.1) this means that

$\alpha({{\bf u}})=\frac{\sum_{{\bf k}}{{\bf k}}^{2}\Big{(}{\bf n}({{\bf
u}},{{\bf u}})_{{\bf k}}+F_{{\bf k}}\Big{)}{\overline{u}}_{{\bf
k}}}{\sum_{{\bf k}}{{\bf k}}^{4}|u_{{\bf k}}|^{2}}$ (2.2)

(in 2D the term with $\bf n({{\bf u}},{{\bf u}})$ vanishes identically).

The new equation is reversible: $IS_{t}{{\bf u}}=S_{-t}I{{\bf u}}$ (as
$\alpha({{\bf u}})$ is odd) and will be called $RNS^{N}$. So $\alpha$ can be
called a “reversible friction”.333In the 3D case the equations are very
similar (for instance the $u_{{\bf k}}$ are replaced bu vectors orthogonal to
${{\bf k}}$, ..): the main and important difference is that the expression of
$\alpha({{\bf u}})$ receives a contribution from the quadratic transport term
which, although present also in 2D, cancels from $\alpha$ essentially because
in 2D when $\nu=0$ the ${\mathcal{D}}({{\bf u}})$ is conserved.

The non Newtonian forces $\nu\Delta{{\bf u}}$, $\alpha({{\bf u}})\Delta{{\bf
u}}$ can be imagined to play the role of a thermostat:[4] the forcing performs
work on the fluid but the fluid density remains the same. This means that the
temperature must change so that the equation of state linking pressure density
and temperature remains fulfilled. Heat has to be removed or inserted and both
forces can be viewed as external forces allowing to achieve respect of the
equation of state. Then it is natural to think that the two equations should
be equivalent.

This leads to an equivalence conjecture, [5], which we formulate still making
use of our knowledge of Statistical Mechanics where examples of equivalences
are well known and can guide us.

The evolution with $RNS^{N}$ wil be assumed regular in the sense of the above
definition; it generates a family of stationary by the distributions on phase
space: $\mu^{N}_{D}$ parameterized by the constant value $D$ of the enstrophy
${\mathcal{D}}({{\bf u}})$. Again for a given $D$ there may be $m^{N}_{D}>1$
physical states which will be distinguished by extra label
$j=1,\ldots,n_{D}^{N}$. The collection of such distributions will form the
“enstrophy ensemble”, ${\mathcal{E}}^{N}_{enstrophy}$.

To formulate the connection between the two ensembles
${\mathcal{E}}^{N}_{viscosity},{\mathcal{E}}^{N}_{enstrophy}$ it is necessary
do define the local observables $O({{\bf u}})$: these are functions on phase
space whose value depends on the harmonics ${{\bf k}}$ contained in a finite
region $\Delta$ independent on $N$,444Hence depend on finitely many Fourier’s
harmonics of ${{\bf u}}$, i.e. are “large scale” observables”, but unlike the
forcing, which only has the harmonics $|{{\bf k}}|<k_{max}$ with $k_{max}$
fixed, no limit is set on the size scale.

The local observables here are analogous to the SM observables on phase space
depending only on the positions and velocities of particles located in finite
region $\Delta$ independent on the size $V$ of the container. So once more
arises a similarity between the SM of a system enclosed in a volume $V$ and of
a NS fluid with UV cut-off $N$: one can say that locality in SM is in position
space while in NS it is in momentum space.

With this in mind the following conjecture has been proposed to relate
physical distributions $\mu^{N}_{\nu,\beta},\rho^{N}_{D,\beta^{\prime}}$ in
${\mathcal{E}}^{N}_{viscosity},{\mathcal{E}}^{N}_{enstrophy}$ where
$\beta=1,\ldots,n^{N}_{\nu};\beta^{\prime}=1,\ldots,m^{N}_{D}$ are labels
distinguishing the physical distributions, if more then one: note however that
cases in which $n^{N}_{\nu},m^{N}_{D}>1$ are expected to be rare. Then:[5]

Conjecture: Let $\mu^{N}_{\nu,\beta}\in{\mathcal{E}}^{N}_{viscosity}$ and
$\rho^{N}_{D,\beta^{\prime}}\in{\mathcal{E}}^{N}_{enstrophy}$ be physical
distributions with the same enstrophy

$\mu^{N}_{\nu,\beta}({\mathcal{D}})=D$ (2.3)

Then if $N$ is large enough it is $n^{N}_{\nu}=m^{N}_{D}$ 555In many cases no
“intermittency” is expected, i.e. $n^{N}_{\nu}=m^{N}_{D}=1$. and for each
$\beta$ there is $\beta^{\prime}$

$\lim_{N\to\infty}\nu^{N}_{\nu,\beta}(O)=\lim_{N\to\infty}\rho^{N}_{D,\beta^{\prime}}(O)$
(2.4)

for all local observables.

So the averages of large scale observables will show the same statistical
properties, as $N\to\infty$ in $INS^{N}$ and $RNS^{N}$ evolutions, under the
correspondence condition of equal enstrophy, Eq.(2.3).

The $\alpha({{\bf u}})$ in the $RNS^{N}$ evolution will fluctuate strongly in
turbulence regime and it will “self-average” to a constant $\nu$ thus
“homogenizing” the equation and turning it into the $INS^{N}$ with friction
$\nu$.

The conjecture however does not mention a condition like for $\nu$ small
enough: it should hold also at high viscosity where often $INS^{N}$ exhibits
periodic attractors. The reason it is proposed also in such cases is that in
all cases the NS equations should be regarded as yielding macroscopic
descriptions of microscopic, certainly chaotic, evolutions derived via scaling
limits without modifyig the microscopic equations, [6].

It is natural to think that there should be no condition for strong chaos. The
microscopic motion is always strongly chaotic and the chaoticity condition
should be always fulfilled even when motion appears laminar.

The analogy with SM becomes even more clear: the $N\to\infty$ limit
corresponds to the thermodynamic limit $V\to\infty$.

As a final remark other ensembles can be imagined: first is the “energy
ensemble” formed by the physical distributions for the equation Eq.(2.1) with
$\alpha$ so defined that the energu $\|{{\bf u}}\|^{2}_{2}$ is an exact
constant (i.e. $\alpha$ is given, in 2D and also in 3D by Eq.(2.2) replacing
${{\bf k}}^{2}$ with $1$ and ${{\bf k}}^{4}$ with ${{\bf k}}^{2}$) and a
similar analysis and conjecture be can be set up: for results obtained with
this approach see [7].

At this point it is convenient to pause and show a few results from
simulations which begin to test the equivalence proposal.

##  III Some 2D simulations

Here are collected results the earlier (¡2018) simulations that might interest
readers: skipping this part does not preclude following the final comments in
Sec.IV.

There is a first obvious test suggested by a rigorous consequence of the
conjecture based on the remark that the work per unit time of the forcing is a
local observable, being $W={\bf F}\cdot{{\bf u}}\equiv\sum_{|{{\bf
k}}|<k_{m}ax}F_{{\bf k}}{\overline{u}}_{{\bf k}}$. Multiplying both sides of
$RNS^{N}$ or $INS^{N}$ by ${\overline{u}}_{{{\bf k}}}$ one finds the energy
conservation identity:

$\displaystyle{\alpha({{\bf u}})D=}$ | $\displaystyle{{}{\bf F}\cdot{{\bf u}},\qquad RNS^{N}}$
---|---
$\displaystyle{\nu{\mathcal{D}}({{\bf u}})=}$ | $\displaystyle{{}{\bf F}\cdot{{\bf u}},\qquad INS^{N}}$
(3.1)

The conjecture implies that $\rho^{N}_{D}(\alpha)D$ has to be equal in the
$\lim_{N\to\infty}$ to the average $W$ which in the $INS^{N}$ by the second
line of Eq.(3.1) is $\nu\mu^{N}_{\nu}({\mathcal{D}})$. Hence in absence of
intermittency the equivalence condition $\mu^{N}_{\nu}({\mathcal{D}})=D$
yields

$\lim_{N\to\infty}\rho^{N}_{D}(\alpha)=\nu$ (3.2)

Hence a test is: i) fix $\nu,N$ and run the $INS^{N}$ evolution from a random
initial ${{\bf u}}$ until the average enstrophy value $D$ of the enstrophy
${\mathcal{D}}({{\bf u}}(t))$ is numerically reached. The run the $RNS^{N}$
with initial ${{\bf u}}$ adjusted to have enstrophy $D$: the result should be
that if $N$ is large enough the running average of $\alpha$,i.e.
$\frac{1}{t}\int_{0}^{t}\frac{\alpha({{\bf
u}}(t^{\prime}))}{\nu}dt^{\prime}\,\vtop{\halign{#\cr\rightarrowfill\crcr\kern-1.0pt\nointerlineskip\cr\hskip
3.0pt${\scriptstyle t\to\infty}$\hskip 3.0pt\crcr}}\,1$. An example is:

Fig.1: The running average of the reversible friction $\frac{\alpha({{\bf
u}}(t))}{\nu}\equiv\frac{1}{\nu}\frac{\sum_{{{\bf k}}}{{\bf k}}^{2}2f_{-{{\bf
k}}}u_{{{\bf k}}}}{\sum_{{\bf k}}{{\bf k}}^{4}|u_{{\bf k}}|^{2}}$, superposed
to the conjectured value $1$ and to the fluctuating values $\frac{\alpha({{\bf
u}}(t))}{\nu}$: Evolution is by $RNS^{N}$, R=2048, 224 modes ($N=7$),
Lyap.-max $\simeq 2$, $x$-axis unit $2^{19}$, forcing only on modes ${{\bf
k}}=\pm(2,-1)$. Data are obtained via a sequence of integration steps of size
$h=2^{-19}$ registered every $4h$. The plot gives $4000$ successive registered
results but only every $10$ of them to avoid too dense a plot.

Fig.1-detail: The running average of the reversible friction
$\frac{\alpha({{\bf u}}(t))}{\nu}$, superposed to the conjectured limit value
$1$. Same data as the Fig.1 but for the shorter time interval $[0,300]$ to
show the initial transient.

The Fig.2 shows the preliminary evaluation of the average enstrophy: it shows
that in the case considered the average avaleu of the enstrophy is reached
quite rapidly in spite of the strong fluctuations.

Fig.2: Running average of $\frac{W({{\bf u}}(t))}{\nu}$ (dark green) in
$INS^{N}$;
the average $\frac{1}{\nu}D$ of $\frac{1}{\nu}{\mathcal{D}}({{\bf u}})$ in
$INS^{N}$ (straight red line)
running average of $\frac{1}{\nu}{\mathcal{D}}({{\bf u}}(t))$ in $INS^{N}$
’converging’ to $\frac{1}{\nu}D$ (very close to $\frac{1}{\nu}D$)
large fluctuations are those of $\frac{1}{\nu}{\mathcal{D}}({{\bf u}}(t))$.
Data are the same in the previous figures.

It is natural to study the observable $\alpha({{\bf u}})$ as it evolves under
the $INS^{N}$ equation. It is not covered by the conjecture, that only implies
that its average should be, under the equivalence condition, close to $1$ in
the $RNS^{N}$ evolution. An unexpected result is that the running average of
$\frac{1}{\nu}\alpha({{\bf u}}(t))$ also has running average very close to $1$
as indicated by the following figure:

Fig.7: Same as Fig.1 but for $INS^{N}$. The running average of the reversible
friction $\frac{\alpha({{\bf u}}(t))}{\nu}$ in a $N=7$ regularized $INS^{N}$
evolution forced on the mode ${{\bf k}}=\pm(2,-1)$: the running average is in
the large fluctuations curve.

This is one more example of a non local observable with equal averages in
corresponding physical distributions for the two evolutions considered.

The equality to $\nu$ under the equivalence condition between the average
value of $\alpha({{\bf u}})=\frac{\sum_{{\bf k}}{{\bf k}}^{2}F_{{\bf
k}}{\overline{u}}_{{\bf k}}}{\sum_{{\bf k}}{{\bf k}}^{4}|u_{{\bf k}}|^{2}}$
considered as an observable for both $RNS^{N}$ and $INS^{N}$ with $\nu$ is
perhaps surprising. It is a theorem (consequence of the conjecture) in the
$RNS^{N}$ evolution but it might not be even expected in the case of $INS^{N}$
because $\alpha({{\bf u}})$ is not a local observable.

Therefore it is tempting to test possible equality of the averages of other
nonlocal quantities, as such equalities are well known to hold in the
thermodynamic limit for several non local observables.

One such observable is the spectrum of the symmetric part of the Jacobian
$J({{\bf u}})$, the $M_{N}\times M_{N}$ matrix $J_{{{\bf k}},{{\bf
k}}^{\prime}}=\frac{{\partial}\cdot u_{{\bf k}}}{{\partial}u_{{{\bf
k}}^{\prime}}}$, formally $\frac{{\partial}\cdot{{\bf u}}}{{\partial}{{\bf
u}}}$. The spectrum can be called the “local Lyapunov spectrum”.

Fig.3: The spectrum of the symmetrized Jacobian called the “local Lyapunov
exponents” for $N=3$ ($48$ modes) truncation for $INS^{N}$ and $RNS^{N}$ under
the equivalence condition: they are superposed but undistinguishable on the
scale of the picture which plots $(k,\lambda_{k})$, same data as in the
previous pictures (but truncated at $N=3$).

The above coincidence to some extent is due to the wide ordinate scale used.
It is expected the two spectra are subject to computatinal errors which should
be more visible near the $k$ to which correspond $\lambda_{k}$’s close to $0$.
This is clarified in

Fig.4: Relative difference betweeen (local) Lyapunov exponents in the previous
Fig.3 ($\nu=2048^{-1}$, $N=3$ (i.e. $48$ modes)).

Certainly a cut-off at $N=3$ is much too small to be of any significance. In
fact agreement between the $INS^{N}$ and $RNS^{N}$ is expected also at fixed
$N$ and small $\nu$, [8], as a consequence of appearance of turbulence: which
generates apparently random fluctuations on $\alpha$ with a corresponding
homogenization phenomenon: and physically diferent phenomenon.

The following figure tests the equivalence in the case of a higher UV cut-off:

Fig.5: Local Lyapunov spectra in a $15\times 15$ ($960$ modes) truncation for
$INS^{N}$ and $RNS^{N}$ (keys ending respectively in 0 or 1), with the
$\lambda_{k}$ interpolated versus $k$ by lines, $\nu=\frac{1}{2048}$, forcing
onthe modes${{\bf k}}=(2,-1)$. Each of the $\lambda_{k}({{\bf u}}(t))$ is
evalueted every $2^{19}$ integration steps and the graf reports the average of
of each $\lambda_{k}({{\bf u}}(t))$ over $2200$ successive evaluations.

The latter graph shows the spectra for both $INS^{N}$ and $RNS^{N}$ cases:
again they are superposed. As in the previous case the relative difference can
be studied more clesely:

Fig.6: Relative difference betweeen (local) Lyapunov exponents in the previous
Fig.5

##  IV Further results and problems

(1) The question of exhibiting examples of “regular systems” in the sense of
Sec.II has been essentially answered in the proposal by Ruelle that I intepret
as saying that “generically” all systems exhibiting chaotic evolution should
be “regular”.

In more mathematically oriented works the idea emerges from the theory of
Anosov flows: they play the role, in chaotic dynamics, of the harmonic
oscillators in ordered dynamicsl they are the paradigm of Chaos. This idea
rests on fundamental works of Sinai (on Anosov sys.), and Ruelle, Bowen (on
Axioms A systems). A strict, general, heuristic, interpretation of original
ideas on turbulence phenomena, [9, 10, 11, 12, 13], led to the, [14]: Chaotic
hypothesis: A chaotic evolution takes place on a smooth surface $A$,
“attracting surface”, contained in phase space, and on $A$ the maps $S$ (or
the flow $S_{t}$) is an Anosov map (or flow).

So a regular system is a system with $n\geq 1$ attractors $A_{i}$ whose basins
of attraction are open sets ${\mathcal{A}}_{i}$ whose union of the entire
phase space up to a set of zero volume.

(2) The cahotic hypothesis is dismissed (by many) with arguments like (1999)
More recently Gallavotti and Cohen have emphasized the “nice” properties of
Anosov systems. Rather than finding realistic Anosov examples they have
instead promoted their “Chaotic Hypothesis”: if a system behaved “like” a
[wildly unphysical but well-understood] time reversible Anosov system there
would be simple and appealing consequences, of exactly the kind mentioned
above. Whether or not speculations concerning such hypothetical Anosov systems
are an aid or a hindrance to understanding seems to be an aesthetic
question.[15]

While giving up any evaluation of the statement I stress that Statistical
Mechanics, after Clausius, Boltzmann and Maxwell was a simple and appealing
consequence of the “[wildly unphysical but well-understood]” periodicity of
motions of atoms in a gas.

(3) The same tests mentioned here (dated up to 2018) have been made in 2D NS
with up to $920$ modes and in 3D NS even up to $\sim 5\cdot 10^{6}$ modes.[4]

(4) The 3D tests have suggested that the notion of local observable should be
made more strict defining $O$ a local observable if $O({{\bf u}})$ depends
finitely many harmonics ${{\bf k}}$, as above, further verifying $|{{\bf
k}}|<cK_{\nu}$ where $K_{\nu}=(\frac{\eta}{\nu^{3}})^{\frac{1}{4}})$ is
Kolmogorov’s scale and $c$ is a constant and $\eta=\nu\langle W\rangle$ is the
average work per unit time: it is believed widely the in the limit $\nu\to 0$
$\eta$ should remain finite and positive.[16]

Although the 3D tests seem to suggest $c$ is of order $O(1)$ in my view the
possibility that $c=\infty$ should be studied with further simulations.

(5) one of the consequences of the chaotic hypothesis is that if the system is
time reversible then a general result is the “fluctuation theorem” (FT). The
problem is that there is strong evidence that in the NS the asymptotic motion
is attracted on a set of dimension lower than that of phase space: this is
shown by the fact that as soon as $N$ is large enough and $\nu$ small enough
the attractor has dimension less that that of phase space: this is shown by
the $>0$ Lyapunov exponents seem to be less than half the number of negative
ones.[5]

In 2D this seems to be the case with $\nu=1/2048$ and $N=7$ and $15$. There
are a few examples in which even though the attractor $A$ has dimension lower
than that of phase space the motion on $A$ admits a symmetry $I^{\prime}:A\to
A$ which has the same properties as time reversal (i.e.
$I^{\prime}S_{t}=S_{-t}I^{\prime}$): but is is unclear that the NS admit such
a symmetry. Positive results in testing validity of the FT have been found,
[17], in 2D in the $RNS^{N}$ with $N=48,\nu=1/2048$, way too small for being
really interesting, while already for the case $N=224,\nu=1/2048$ it is
unclear if FT holds.

Appendix: A path through the theme

1) A first equivalence example: [2]
2) Path to the conjecture: [18, 19, 5, 16]
3) 3D enstrophy ensemble: [16, 20]
4) 3D energy ensemble: [7]
5) Shell model: [21]
7) Stat-Mech: [22, 23, 13, 12, 24]
8) Turbulence physics: [25, 26, 27, 28, 29]

Acknowledgements: This is a redacted and updated version of a talk at the
DinAmici meeting on 21/Dec/2018 at the Accademia dei Lincei, Roma.

## References

  * [1] J.C. Maxwell.  On the dynamical theory of gases.  In: The Scientific Papers of J.C. Maxwell, Cambridge University Press, Ed. W.D. Niven, Vol.2, pages 26–78, 1866.
  * [2] Z.S. She and E. Jackson.  Constrained Euler system for Navier-Stokes turbulence.  Physical Review Letters, 70:1255–1258, 1993.
  * [3] G. Gallavotti.  Equivalence of dynamical ensembles and Navier Stokes equations.  Physics Letters A, 223:91–95, 1996.
  * [4] G. Gallavotti.  Viscosity, Reversibillity, Chaotic Hypothesis, Fluctuation Theorem and Lyapunov Pairing.  Journal of Statistical Physics, 185:21:1–19, 2021.
  * [5] G. Gallavotti.  Nonequilibrium and Fluctuation Relation.  Journal of Statistical Physics, 180:1–55, 2020.
  * [6] J. L. Lebowitz.  From Time-symmetric Microscopic Dynamics to Time-asymmetric Macroscopic Behavior: An Overview.  in Boltzmann’s Legacy, Ed. G.Gallavotti, W.Reiter, Y.Yngvason, Birkhauser, pages 63–87, 2008.
  * [7] V. Shukla, B. Dubrulle, S. Nazarenko, G. Krstulovic, and S. Thalabard.  Phase transition in time-reversible Navier-Stokes equations.  arxiv, 1811:11503, 2018.
  * [8] G. Gallavotti.  Fluctuation patterns and conditional reversibility in nonequilibrium systems.  Annales de l’ Institut H. Poincaré, 70:429–443, 1999 and chao-dyn/9703007.
  * [9] Ya. G. Sinai.  Markov partitions and $C$-diffeomorphisms.  Functional Analysis and its Applications, 2(1):64–89, 1968.
  * [10] R. Bowen and D. Ruelle.  The ergodic theory of axiom A flows.  Inventiones Mathematicae, 29:181–205, 1975.
  * [11] R. Bowen.  Equilibrium states and the ergodic theory of Anosov diffeormorphisms, volume 470 of Lecture Notes in Mathematics.  Springer-Verlag, Berlin-Heidelberg, 1975.
  * [12] D. Ruelle.  Chaotic motions and strange attractors.  Accademia Nazionale dei Lincei, Cambridge University Press, Cambridge, 1989.
  * [13] D. Ruelle.  What are the measures describing turbulence.  Progress in Theoretical Physics Supplement, 64:339–345, 1978.
  * [14] G. Gallavotti and D. Cohen.  Dynamical ensembles in stationary states.  Journal of Statistical Physics, 80:931–970, 1995.
  * [15] W. Hoover and C. Griswold.  Time reversibility Computer simulation, and Chaos.  Advances in Non Linear Dynamics, vol. 13, 2d edition. World Scientific, Singapore, 1999.
  * [16] G. Margazoglu, L. Biferale, M. Cencini, G. Gallavotti, and V. Lucarini.  Non-equilibrium Ensembles for the 3D Navier-Stokes Equations.  Physical Review E, 105:065110, 2022.
  * [17] G. Gallavotti.  Ensembles, Turbulence and Fluctuation Theorem.  European Physics Journal, E, 43:37, 2020.
  * [18] G. Gallavotti.  Dynamical ensembles equivalence in fluid mechanics.  Physica D, 105:163–184, 1997.
  * [19] G. Gallavotti.  Reversible viscosity and Navier–Stokes fluids.  Springer Proceedings in Mathematics & Statistics, 282:569–580, 2019.
  * [20] A. Jaccod and S. Chibbaro.  Constrained Reversible system for Navier-Stokes Turbulence.  Physical Review Letters, 127:194501, 2021.
  * [21] L. Biferale, M. Cencini, M. DePietro, G. Gallavotti, and V. Lucarini.  Equivalence of non-equilibrium ensembles in turbulence models.  Physical Review E, 98:012201, 2018.
  * [22] D. Ruelle.  Statistical Mechanics, Rigorus Results.  World Scientific, 3d edition, London, 1999.
  * [23] D. Ruelle.  Dynamical systems with turbulent behavior, volume 80 of Lecture Notes in Physics.  Springer, 1977.
  * [24] D. Ruelle.  Hydrodynamic turbulence as a problem in nonequilibrium statistical mechanics.  Proceedings of the National Academy of Science, 109:20344–20346, 2012.
  * [25] U. Frisch.  Turbulence.  Cambridge University Press, 1995.
  * [26] R. Benzi and U. Frisch.  Turbulence.  Scholarpedia, 5(3):3439, 2010.
  * [27] W. George.  Lectures in Turbulence for the 21st Century.  Lecture_Notes/Turbulence_Lille/TB_16January2013.pdf. https://www.turbulence-online.com/Publications/, Chalmers University of Technology, Gothenburg, Sweden, 2013.
  * [28] T. Buckmaster and V. Vicol.  Nonuniqueness of weak solutions to the Navier-Stokes equation.  Annals of Mathematics, 189:101–144, 2019.
  * [29] C. Fefferman.  Existence & smoothness of the Navier–Stokes equation.  The millennium prize problems. Clay Mathematics Institute, Cambridge, MA, 2000.


# Learning Disentangled Audio Representations through Controlled Synthesis

Yusuf Brima, Ulf Krumnack, Simone Pika & Gunther Heidemann
Institute of Cognitive Science
Universität Osnabrück
Wachsbleiche 27, D-49090 Osnabrück, Germany
<EMAIL_ADDRESS>

###### Abstract

This paper tackles the scarcity of benchmarking data in disentangled auditory
representation learning. We introduce SynTone, a synthetic dataset with
explicit ground truth explanatory factors for evaluating disentanglement
techniques. Benchmarking state-of-the-art methods on SynTone highlights its
utility for method evaluation. Our results underscore strengths and
limitations in audio disentanglement, motivating future research.

##  1 Introduction

Disentangled representation learning shows promise for creating useful data
encodings. However, it remains underexplored for auditory data, despite
potential benefits (Locatello et al., 2019; Higgins et al., 2017; Kim & Mnih,
2018). Disentangled representations refer to learned encodings (codes for
short) where explanatory factors are encoded in independent, compact, and
informative dimensions. These factorized representations offer an improved
generalization, explainability, reduced sample complexity, and avoidance of
shortcut learning (Carbonneau et al., 2020).

Progress in auditory disentanglement is hampered by the lack of suitable
benchmarking datasets for evaluation (Hsu et al., 2017; Mo et al., 2019).
Auditory signals present unique challenges due to their high dimensionality,
temporal dynamics, and rich hierarchical structure. To help address this
evaluation gap, we introduce SynTone: a basic dataset of synthetic tones
curated for quantitatively assessing disentangled representation learning
techniques in the acoustic domain. We used this new dataset to conduct a
benchmarking study comparing state-of-the-art (SOTA) disentanglement
frameworks. Performance was measured using supervised disentanglement metrics.
Our goals were to (i) demonstrate SynTone’s utility for method
evaluation/comparison, and (ii) highlight the strengths and limitations of
audio disentanglement algorithms to help guide future research.

##  2 Methodology

###  2.1 SynTone Dataset

Following ideas from disentanglement in the vision domain (Matthey et al.,
2017; Burgess & Kim, 2018), we develop SynTone - a simple synthetic audio
dataset with explicit ground truth factors to facilitate disentanglement
analysis. SynTone comprises $32,000$ 1-second audio samples (16kHz sampling
rate) synthesized by systematically varying three generative parameters:
timbre $\mathbb{T}$ (sine, triangle, square, sawtooth waveforms), amplitude
$\mathbb{A}$ (20 discrete levels from 0 to 1), and frequency $\mathbb{F}$ (400
discrete steps from $440$ to $8000$Hz). The dataset structure is formally:
$\mathbb{T}\times\mathbb{A}\times\mathbb{F}$, with each sample having a unique
$(T\in\mathbb{T},A\in\mathbb{A},F\in\mathbb{F})$ tuple defining its generative
factors. Having full knowledge of the data generative process allows
quantitative assessment of models’ effectiveness at learning the ground truth
factors from the audio samples. This dataset aims to provide a benchmark for
the analysis of disentanglement techniques on a perceptual signal domain where
the factorized generative sources are specified.

###  2.2 Modeling

We employ four VAE-based models to disentangle SynTone’s ground truth factors:
vanilla VAE (Kingma & Welling, 2013), $\beta$-VAE (Higgins et al., 2017),
Factor-VAE (Kim & Mnih, 2018), and $\beta$-TCVAE (Chen et al., 2018). The use
of VAEs offers interpretable latent spaces, and these model variants introduce
objectives specifically targeting greater independence among latent factors.
SynTone’s rich ground truth provides a basis for quantifying the success of
recovering explicitly defined variations. Our focus lies on testing the
explanatory disentanglement capabilities afforded by VAE techniques, in
contrast to assessing the sample quality of less interpretable GANs (Chen et
al., 2016).

To evaluate the disentanglement achieved by these different approaches, we
apply various metrics that quantify each model’s ability to recover ground
truth factors, emphasizing different desired properties of disentanglement.
These properties include modularity, where variation in one factor does not
affect other factors (i.e., there is no causal effect between them);
compactness, where a factor should be explained by a minimal subset of the
latent space, ideally a single dimension; and explicitness, where learned
codes should be semantically meaningful. Specifically, we use the Mutual
Information Gap (MIG) (Chen et al., 2018) and Attribute Predictability Score
(SAP) (Kumar et al., 2017) to measure compactness. We also use Joint Entropy
Minus Mutual Information Gap (JEMMIG) (Do & Tran, 2019) and Disentanglement,
Completeness, and Informativeness MIG (DCIMIG) (Sepliarskaia et al., 2019) to
assess holisticness, as they capture modularity, compactness, and explicitness
properties in a single score. Moreover, the code modularity is measured with
the Modularity score (Ridgeway & Mozer, 2018). For a more in-depth analysis of
disentanglement metrics, refer to Carbonneau et al. (2020).

##  3 Experiments

We trained four model architectures on SynTone: vanilla VAE, $\beta$-VAE,
Factor-VAE, and $\beta$-TC-VAE. To facilitate training, the audio data is
transformed into a time-frequency domain, resulting in a 2D input for our
networks (refer to appendix A.1). All variational autoencoders used the same
convolutional architecture (detailed in appendix A.2), with variations only in
the objective function. Extensive exploration of hyperparameter settings for
the different models was conducted, and only the best-performing configuration
was adopted for the final experiment. Table 1 provides a summary of these
metrics for each model.

| Compactness | Holistic | Modularity
---|---|---|---
Model | MIG | SAP | DCIMIG | JEMMIG | Mod. Score
Vanilla VAE | 0.280 $\pm$ 0.0 | 0.045 $\pm$ 0.000 | 0.005 $\pm$ 0.000 | 0.300 $\pm$ 0.0 | 0.651 $\pm$ 0.031
$\beta$-VAE | 0.184 $\pm$ 0.0 | 0.035 $\pm$ 0.000 | 0.004 $\pm$ 0.000 | 0.201 $\pm$ 0.0 | 0.750 $\pm$ 0.022
Factor-VAE | 0.118 $\pm$ 0.0 | 0.035 $\pm$ 0.000 | 0.003 $\pm$ 0.000 | 0.126 $\pm$ 0.0 | 0.762 $\pm$ 0.014
$\beta$-TCVAE | 0.166 $\pm$ 0.0 | 0.023 $\pm$ 0.000 | 0.005 $\pm$ 0.000 | 0.183 $\pm$ 0.0 | 0.747 $\pm$ 0.016
Table 1: Disentanglement metrics with standard deviation for different models
over 10 evaluation runs each. Values are presented as mean $\pm$ standard
deviation.

In our analysis, the vanilla VAE’s superior performance in compactness metrics
(MIG, SAP) and comparable DCIMIG to $\beta$-TCVAE was unexpected, especially
as compactness is of lesser practical importance. Its simpler structure may
better isolate factors in the SynTone dataset. Factor-VAE led in JEMMIG and
Modularity, as expected from its training approach. The underperformance of
$\beta$-VAE and $\beta$-TCVAE in certain metrics suggests their loss functions
might not be ideally suited for SynTone, emphasizing the need for dataset-
specific approaches in audio-disentangled representation learning. For deeper
analysis, see the qualitative results in the appendix A.3.

##  4 Conclusion

Our work addresses the unexplored realm of disentangled representation
learning in audio by introducing SynTone, a dataset with explicit ground truth
factors. We evaluated state-of-the-art methods, revealing the challenges in
achieving comprehensive disentanglement in audio representations. While the
vanilla VAE demonstrates robust compactness, the Factor-VAE excels in
modularity. However, limitations persist, emphasizing the need for further
research in audio disentanglement. Our dataset proves valuable for method
evaluation, though potential extensions to more diverse datasets and real-
world scenarios warrant exploration. Our findings provide insights for future
efforts in enhancing disentangled audio representation learning.

#### Acknowledgements

This work was funds of the research training group (RTG) in “Computational
Cognition” (GRK2340) provided by the Deutsche Forschungsgemeinschaft (DFG),
Germany, and an EU-Consolidator grant (772000, TurnTaking).

#### URM Statement

The authors acknowledge that at least one key author of this work meets the
URM criteria of the ICLR 2024 Tiny Papers Track.

## References

  * Burgess & Kim (2018) Chris Burgess and Hyunjik Kim.  3d shapes dataset.  https://github.com/deepmind/3dshapes-dataset/, 2018.
  * Carbonneau et al. (2020) Marc-André Carbonneau, Julian Zaidi, Jonathan Boilard, and Ghyslain Gagnon.  Measuring disentanglement: A review of metrics.  _arXiv preprint arXiv:2012.09276_ , 2020.
  * Chen et al. (2018) Ricky TQ Chen, Xuechen Li, Roger B Grosse, and David K Duvenaud.  Isolating sources of disentanglement in variational autoencoders.  In _NeurIPS_ , pp. 2615–2625, 2018.
  * Chen et al. (2016) Xi Chen, Yan Duan, Rein Houthooft, John Schulman, Ilya Sutskever, and Pieter Abbeel.  Infogan: Interpretable representation learning by information maximizing generative adversarial nets.  _Advances in neural information processing systems_ , 29, 2016.
  * Do & Tran (2019) Kien Do and Truyen Tran.  Theory and evaluation metrics for learning disentangled representations.  _arXiv preprint arXiv:1908.09961_ , 2019.
  * Higgins et al. (2017) Irina Higgins, Loic Matthey, Arka Pal, Christopher Burgess, Xavier Glorot, Matthew Botvinick, Shakir Mohamed, and Alexander Lerchner.  beta-vae: Learning basic visual concepts with a constrained variational framework.  _International Conference on Learning Representations (ICLR)_ , 2017.
  * Hsu et al. (2017) Wei-Ning Hsu, Bernd Bolte, Yao-Hung Tsai, Kushal Lakhotia, Ruslan Salakhutdinov, and Abdelrahman Mohamed.  Unsupervised learning for sequential speech separation in noisy environments.  _IEEE/ACM transactions on audio, speech, and language processing_ , 25(5):1098–1112, 2017.
  * Kim & Mnih (2018) Hyunjik Kim and Andriy Mnih.  Disentangling by factorising.  In _International Conference on Machine Learning_ , pp. 2649–2658. PMLR, 2018.
  * Kingma & Welling (2013) Diederik P Kingma and Max Welling.  Auto-encoding variational bayes.  _arXiv preprint arXiv:1312.6114_ , 2013.
  * Kumar et al. (2017) Abhishek Kumar, Prasanna Sattigeri, and Avinash Balakrishnan.  Variational inference of disentangled latent concepts from unlabeled observations.  _arXiv preprint arXiv:1711.00848_ , 2017.
  * Locatello et al. (2019) Francesco Locatello, Gabriele Abbati, Tom Rainforth, Stefan Bauer, Bernhard Schölkopf, and Olivier Bachem.  Challenging common assumptions in the unsupervised learning of disentangled representations.  _International conference on machine learning_ , pp. 4114–4124, 2019.
  * Matthey et al. (2017) Loic Matthey, Irina Higgins, Demis Hassabis, and Alexander Lerchner.  dsprites: Disentanglement testing sprites dataset.  https://github.com/deepmind/dsprites-dataset/, 2017.
  * Mo et al. (2019) Kaizhi Mo, Jimmy Ba, Xuchen Zheng, Zhenyu Liu, Wei Xie, and Takuya Yoshioka.  Semantic speech separation using an unsupervised spatial feature.  In _ICASSP_ , pp. 7205–7209, 2019.
  * Ridgeway & Mozer (2018) Karl Ridgeway and Michael C Mozer.  Learning deep disentangled embeddings with the f-statistic loss.  _Advances in neural information processing systems_ , 31, 2018.
  * Sepliarskaia et al. (2019) Anna Sepliarskaia, Julia Kiseleva, and Maarten de Rijke.  How to not measure disentanglement.  _arXiv preprint arXiv:1910.05587_ , 2019.

##  Appendix A Appendix

###  A.1 Model Input Preprocessing

The input representation to each model is a mel-spectrogram transformation
with a window size of $2024$ and a hop size of $512$, aligning with speech
processing best practices. Mel-spectrograms efficiently encapsulate audio
details into perceptually relevant frequency bands. The encoder outputs
parameterize a multivariate Gaussian distribution over latent variables,
guided by a $\beta$ hyperparameter during training for incentivizing
disentangled dimensions. Supervised by $\mathbb{T}$, $\mathbb{A}$, and
$\mathbb{F}$ generative factors, designated latent dimensions are manipulated
post-convergence to control amplitude, timbre, and frequency in generated
samples. Performance assessment includes quantitative and qualitative
evaluations of disentanglement metrics, reconstructions, and sample quality
respectively.

###  A.2 Model Architecture

Our VAE model applies four convolutional layers to extract hierarchical
spatial features from input mel-spectrograms, followed by two fully connected
layers to parameterize a Gaussian latent code capturing explanatory factors of
variation. The decoder mirrors this pathway using transpose convolutions to
recover the original input dimensions from sampled codes. All layers use leaky
ReLU activations (slope=0.01) for faster convergence, with no batch norm for
clearer disentanglement analysis. We strategically upsample in the decoder to
balance reconstruction quality without overparameterization. This
convolutional VAE structure provides an interpretable latent space to analyze
captured generative factors while modeling complex mel-spectrogram
dependencies. By parameterizing the distributions, we additionally obtain
useful uncertainty information.

Building upon this, the full implementation details, including the
architecture, training procedures, and hyperparameters used in our VAE models,
are meticulously documented. This comprehensive resource is designed to assist
in replicating or building upon our work. The provided link offers access to
our codebase VAE Models Full Implementation.

###  A.3 Reconstruction and Sampling

We visually assess models’ performance through reconstructions of held-out
mel-spectrograms and novel samples synthesized by decoding random codes.
Reconstructions indicate how effectively the data compression is captured.
Sampling assesses generative modeling abilities when guided solely by the
learned latent distributions. Comparing the similarity of reconstructions and
coherence of samples provides intuitive insight into encoding quality between
models, complementing metric-based evaluation.

Figure 1: Time-domain representation (left), Fourier transforms (middle), and
Time-Frequency (Mel-spectrogram) representation (right) for original and
reconstructed samples using the VAE model. This model shows a lower
reconstruction quality as indicated in all plots due to the addition of noise
frequencies around 7000Hz as well as lower harmonic partials as portrayed in
the mel-spectrogram. Figure 2: Visualization of generated audio samples using
the VAE model. The left column depicts the time-domain representation, the
middle column illustrates the Fourier transforms, and the right column
showcases the Time-Frequency (Mel-spectrogram) representation. This plot shows
that the model can isolate is single major frequency band close to 3000Hz as
illustrated by the Fourier domain periodogram. Figure 3: Time-domain
representation (left), Fourier transforms (middle), and Time-Frequency (Mel-
spectrogram) representation (right) for original and reconstructed samples
using the $\beta$-VAE model. This model achieved a suitable reconstruction
quality compared to the vanilla VAE as shown in both the Fourier space where
the frequency factor of 4248.42Hz is correctly isolated. Figure 4:
Visualization of generated audio samples using the $\beta$-VAE model. The left
column depicts the time-domain representation, the middle column illustrates
the Fourier transforms, and the right column showcases the Time-Frequency
(Mel-spectrogram) representation. This figure, however, shows minor harmonic
partials close to 1000Hz while the major frequency component is also
predominant in both the periodogram and mel-spectrogram plots. Figure 5: Time-
domain representation (left), Fourier transform (middle), and Time-Frequency
(Mel-spectrogram) representation (right) for original and reconstructed
samples using the FactorVAE model. This model achieved the best reconstruction
quality for an assessment of audio representation in both the Fourier space
where the frequency factor of 4248.42Hz is correctly isolated. Figure 6:
Visualization of generated audio samples using the Factor-VAE model. The left
column depicts the time-domain representation, the middle column illustrates
the Fourier transforms, and the right column showcases the Time-Frequency
(Mel-spectrogram) representation. The model can decode a complex latent code
that is smoothly represented in both the Fourier plot and mel-spectrograms
with minimal noise levels. Figure 7: Time-domain representation (left),
Fourier transforms (middle), and Time-Frequency (Mel-spectrogram)
representation (right) for original and reconstructed samples using the
$\beta$-TCVAE model. Like the vanilla VAE, this model also suffers from low-
frequency noise in encoding and decoding the audio signal. Figure 8:
Visualization of generated audio samples using the $\beta$-TCVAE model. The
left column depicts the time-domain representation, the middle column
illustrates the Fourier transforms, and the right column showcases the Time-
Frequency (Mel-spectrogram) representation. Like the Factor-VAE decoder, this
model is also able to decode complex waveforms however, it has more
predominate noise levels as indicated in the plots where minor frequency
contributions are visible. Figure 9: Latent space interpolation for the
vanilla VAE model, showcasing variations in a single latent code while others
are fixed. Each row represents 10 discrete samples per latent code. Figure 10:
Latent space interpolation for the $\beta$-VAE model, showcasing variations in
a single latent code while others are fixed. Each row represents 10 discrete
samples per latent code. Figure 11: Latent space interpolation for the Factor-
VAE model, showcasing variations in a single latent code while others are
fixed. Each row represents 10 discrete samples per latent code. Figure 12:
Latent space interpolation for the $\beta$-TCVAE model, showcasing variations
in a single latent code while others are fixed. Each row represents 10
discrete samples per latent code.