# Generative Flow Networks in Covariant Loop Quantum Gravity

Joseph Bunao  Pietropaolo Frisoni  Department of Physics and Astronomy,
University of Western Ontario, London, ON N6A 5B7, Canada  Athanasios Kogios
Perimeter Institute, 31 Caroline Street North, Waterloo, ON, N2L 2Y5, CAN
Department of Physics and Astronomy, University of Waterloo, 200 University
Avenue West, Waterloo, ON, N2L 3G1, Canada  Jared Wogan  Department of Physics
and Astronomy, University of Western Ontario, London, ON N6A 5B7, Canada

###### Abstract

Spin foams arose as the covariant (path integral) formulation of quantum
gravity depicting transition amplitudes between different quantum geometry
states. As such, they provide a scheme to study the no boundary proposal,
specifically the nothing to something transition and compute relevant
observables using high performance computing (HPC). Following recent advances,
where stochastic algorithms (Markov Chain Monte Carlo-MCMC) were used, we
employ ”Generative Flow Networks”, a newly developed machine learning
algorithm to compute the expectation value of the dihedral angle for a
4-simplex and compare the results with previous works.

##  I Introduction

A universe described by a Big Bang, assumes the existence of an initial
singularity (that means a manifold with a boundary), an initial state from
which the expansion begins and evolves resulting (eventually) in the large
scale structure we observe today. Considering this model in a quantum
mechanical setting, we can view the whole process as a transition amplitude
between two states: the initial singularity and the universe in a later
moment. But quantum theory studies the dynamical evolution of a system given
some initial conditions (defined on the boundary), which in our case, for the
initial singularity, should be put by hand, as the model suggests that
everything (including matter and spacetime itself) were created exactly at the
moment of Big Bang.

Instead of dealing with a fine tuning issue, a rather more agnostic approach
is to consider a no boundary initial state PhysRevD.28.2960  (for a pedagogic
review see LEHNERS20231 ) where matter and spacetime were not created
(nothing). A schematic way to understand this would be to think of a $2$-dim
sphere, which encompasses everything but still has no boundary. Of course, a
sphere has positive curvature and would correspond to a Euclidean spacetime.
Generalizing this idea to a $4$-dim spacetime, we can regard a $S^{4}$ as the
initial state in our transition amplitude without the need to define initial
conditions for our equations and inspect the transition amplitude to a state
describing our current universe ie homogeneous and isotropic (something).
Fortunately, there is a framework to express transition amplitudes using a
path integral formulation of quantum gravity, namely spin foams.

In our case, the tunnelling process from nothing to something corresponds to
what is called a $4$-simplex, with additional constraints to account for the
cosmological context we are working on111A general $4$-simplex as a spin foam
amplitude describes the transition between two general quantum geometries,
from nothing to a 4D one.Therefore, we require the graph to have homogeneous
boundary data (homogeneity) put differently all spins in the boundary of the
graph will have the same value $j$ which will be fixed for each simulation.
Additionally, the graph should be regular that is to say that being in any
node should not change the ”view” (isotropy). Our aim will be to compute an
observable on it, the dihedral angle. The latter (contrary to the usual
definition) will denote the angle between two triangles in the same
tetrahedron thus providing a notion of local geometry.

Despite the multiple analytical techniques developed to describe our physical
environment, it is, still, a cumbersome procedure to actually perform
computations and derive results. In the context of quantum gravity and,
especially, spinfoams we are interested in solving complicated path integrals,
which is, analytically, an almost intractable matter. As a consequence, high
performance computing (HPC) techniques Francesco_draft_new_code  along with
stochastic methods (mainly Markov Chain Monte Carlo/MCMC)
Spinfoam_Lefschetz_thimble ; Steinhaus:2024qov  were proposed as a viable
alternative. In Frisoni:2022urv , a simple spinfoam model describing the
simplest triangulation of a 3 sphere emerging from a 4-simplex was studied
using MCMC. Specifically, only the homogeneous sector, where all boundary
links have the same spin value, was considered, therefore the only degrees of
freedom (dof) come from the boundary intertwiners (nodes). Then, expectation
values of some operators and their correlations were derived. This process
included the numerical computation of the spinfoam amplitude in terms of the
aforementioned dof and the use of a MCMC sampler.

As explained, there is an ever increasing implementation of stochastic
algorithms as a way to solve our equations in practice. Although they present
a lot of advantages, MCMC methods are not free from drawbacks. To be more
specific, they tend to be problematic in cases where the high probability
peaks are well separated from each other, particularly in high dimensional
distributions. Since the MCMC moves through the distribution in stochastic
random-walk-like small perturbations finding new isolated modes becomes less
probable as the algorithm focuses on the peaks it has already found. In other
words, given multiple high peaks and a random initialization of the algorithm,
its effectiveness on finding all peaks depends on how well the random initial
points cover the distribution. Its ability to find and focus to a peak is,
also, its drawback.

To tackle this issue, some machine learning algorithms arose. To be more
specific, Generative Flow Networks (GFlowNets) were developed in the last
years as a possible alternative to traditional MCMC methods in order to try to
gauge an unknown, complex probability distribution. Kanwar_2020 ; Albergo_2019
; malkin2022trajectory ; NEURIPS2021_e614f646 ; bengio2022gflownet ;
Bengio_tutorial ; lahlou2023torchgfn . In spite of being computationally more
expensive to train than an MCMC, the cost is amortized as the network needs to
be trained only once. Then, sampling a constructive sequence is faster than
the corresponding MCMC one, as the latter needs to be rerun for every sampling
occasion. In addition, a GFlownet is more efficient in finding the isolated
peaks, as it utilizes all of the pairs of points-probabilities visited during
the training procedure and, also, generalizes and infers the locations of
other peaks in areas which it did not visit yet.

The paper starts with a brief introduction on the Generative Flow Networks’
(GFlowNets) mathematical foundations and the different loss functions
considered in our calculations, section II. In section III, we shortly present
the theory behind spinfoams and how to formulate them in the context of
stochastic methods and as a GFlowNet hypergrid. Finally, we compare the
effectiveness of the two methods and discuss the respective results.

##  II GFlowNet background

Generative Flow Networks (GFlowNets) are algorithms that generate composite
samples according to a target unnormalized distribution via a series of
assembling steps. These algorithms comprise (1) an agent that can
appropriately map intermediate object states to a probability distribution of
constructive actions and (2) a training objective whose global minimum
enforces consistent probability flow throughout the building steps and fits
the target distribution at the terminal states. Their novelty lies on the fact
that instead of optimizing a problem to a single-maximizing sequence, they
generate a wide variety of high-return solutions based on a reward function
$R(x)$.

###  II.1 Mathematical Preliminaries

Consider a Directed Acyclic Graph (DAG), $G=(\mathcal{S},\mathcal{A})$ where
the nodes $s\in\mathcal{S}$ are all the possible states in the system being
considered and the edges $a=s_{i}\rightarrow
s_{i+1}\in\mathcal{A}\subset\mathcal{S}\times\mathcal{S}$ are the constructive
actions transforming $s_{i}$ to $s_{i+1}$. We now define $s_{0}$ as the unique
state without incoming edges (no parent states) and
$\mathcal{S}_{T}\subseteq\mathcal{S}$ as the set of states without outgoing
edges (no children states). Consequently, we name $s_{0}$ the initial state
and $s_{T}\in\mathcal{S}_{T}$ a terminal state. Additionally, we call a
sequence of states connecting $s_{0}$ and $s_{T}$ as a complete trajectory
which we can explicitly write as $\tau=s_{0}\rightarrow s_{1}\rightarrow
s_{2}\rightarrow...\rightarrow s_{T}$. Each state transition in $\tau$
corresponds to an action in $\mathcal{A}$ with the last action transitioning a
state into $s_{T}$ being called the terminate action.

Given a trajectory $\tau$, we can define the trajectory flow $F:\mathcal{T}$,
which corresponds to the percentage of flow along this trajectory. For a state
$s$, we can define the state flow: $F(s)=\sum_{s\in\tau}F(\tau)$.

Additionally, the forward probability (remember that our graph is directed) of
visiting state $s^{\prime}$ starting from $s$ is: $P_{F}(s^{\prime}|s)$, while
the reverse is: $P_{B}(s|s^{\prime})$.

Given a target reward distribution
$R:\mathcal{S}_{T}\rightarrow\mathbb{R}^{+}$, the goal of a GFlowNet agent is
then to sample terminal states $s_{T}$ with probability $P(s_{T})=R(s_{T})/Z$
where $Z$ is some proportionality constant given as
$Z=\sum_{\tau\in\mathcal{T}}F(s)$. Since a GFlowNet agent generates terminal
states through the construction of complete trajectories, the terminal state
probability $P(s_{T})$ arises from the sum over all probabilities of
trajectories ending in $s_{T}$. Assuming a Markovian process, the contribution
of a particular trajectory $\tau$ factorizes over the forward transition
probabilities $P_{F}(s_{i+1}|s_{i})$ over all $s\in\tau$. In other words, to
sample terminal states with the desired probability, the transition
probabilities must satisfy

$R(s_{T})/Z=\sum_{\partial\tau=s_{T}}\prod_{s_{i},s_{i+1}\in\tau}P_{F}(s_{i+1}|s_{i}),\forall
s_{T}\in\mathcal{S}_{T}$ (1)

The GFlowNet agent then estimates $P_{F}(s_{i+1}|s_{i};\theta)$ (more
precisely, its logarithm) for every edge $s_{i}\rightarrow s_{i+1}$ in $G$,
where $\theta$ represents a set of learnable parameters. We call $P_{F}$ the
agent’s forward policy since it governs the actions of the agent for every
encountered state.

###  II.2 Losses

Neural networks are trained through optimizing an appropriately chosen loss
function. For GFlowNets there is, already, a variety of them in the literature
and some of them were used in this paper as different training objectives
might lead to different results.

####  II.2.1 Log-Partition Variance Loss

Introduced in zhang2023robust , the log-partition variance loss (ZVar),
initially, samples a smaller graph and a terminal state/reward function
stemming from the given dataset, along with a number of trajectories ending in
this terminal state, called a mini batch. On this limited situation, it then
proceeds to compute the difference between the $\log Z$ of a trajectory in
this subgraph and the mean $\log Z$ from the mini batch:

$\mathcal{L}_{V}(s)=\left(\log Z(s;\theta)-\mathbb{E}_{s}[\log
Z(s;\theta)]\right)^{2}$ (2)

####  II.2.2 Flow Matching Loss

The flow matching loss (FM) NEURIPS2021_e614f646  relates the ingoing with the
outgoing flow for every noninitial and nonterminal state $s$. In other words,
the sum of the ingoing flows to $s$ stemming from all parent nodes $s"$ should
be equal to the total outgoing flow to all children nodes $s^{\prime}$ ie:

$\mathcal{L}_{FM}(s)=\left(\log\frac{\sum_{(s"\to
s)\in\mathcal{A}}F_{\theta}(s",s)}{\sum_{s\to
s^{\prime}\in\mathcal{A}}F_{\theta}(s,s^{\prime})}\right)^{2}$ (3)

####  II.2.3 Trajectory Balance

The trajectory balance loss (TB) malkin2022trajectory  is concerned with the
conservation of the flow along a full trajectory instead of focusing on a
state. To be more exact. suppose a trajectory $\tau$: the percentage of the
flow moving from the initial state $s_{0}$ to a terminal one $s_{n}$
normalized to the initial number of ”particles” $Z$ should equal the
corresponding backward flow along the trajectory normalized with the number of
output particles ie the forward flow along the given trajectory should match
the backward flow:

$\mathcal{L}_{TB}(\tau)=\left(\log\frac{Z_{\theta}\prod_{t=1}^{n}P_{F}(s_{t}|_{t-1};\theta)}{R(x)\prod_{t=1}^{n}P_{B}(s_{t-1}|s_{t};\theta)}\right)^{2}$
(4)

####  II.2.4 Detailed Balance Loss

The detailed balance loss (DB) bengio2022gflownet  ensures that for every
nonterminal state $s$ the flow moving from $s$ to $s^{\prime}$ should be equal
to the respective backward flow ie moving from $s^{\prime}$ to $s$.

$\mathcal{L}_{DB}(s,s^{\prime})=\left(\log\frac{F_{\theta}(s)P_{F}(s^{\prime}|s;\theta)}{F_{\theta}(s^{\prime})P_{B}(s|s^{\prime};\theta)}\right)^{2}$
(5)

####  II.2.5 SubTrajectory Loss

A generalization of the detailed balance subsubsection II.2.4 and trajectory
balance subsubsection II.2.3, the subtrajectory balance loss (SubTB)
madan2023learning  was introduced where instead of comparing the incoming and
outgoing flows moving to a state, only along its immediate neighbors (parents
and children), this loss function takes into consideration a larger portion of
the graph.

$\mathcal{L}_{SubTB}(\tau)=\left(\log\frac{F(s_{m};\theta)\prod_{i=m}^{n-1}P_{F}(s_{i+1}|s_{i};\theta)}{F(s_{n};\theta)\prod_{i=m}^{n-1}P_{B}(s_{i}|s_{i+1};\theta)}\right)^{2}$
(6)

As this loss function is more general than the others, one can define
different weighings ie how to treat the subtrajectories. Although not all of
them led to results, for the sake of completeness we will present all of them.

  * •

”DB”: Same as Detailed Balance loss. The loss function is computed for every
one-step transition and contributes the same irrespectively of the length of
the subtrajectory.

  * •

”ModifiedDB”: Same as the previous but the result for every one-step
transition is weighed inversely proportional to the length of the trajectory.
Smaller trajectories matter more.

  * •

”TB”: Equivalent to Trajectory loss.

  * •

”equal_within”: A main trajectory is chosen and the loss function is computed
for every subtrajectory within and weighed equally. Then, each main trajectory
is weighed equally.

  * •

”equal”: All subtrajectories of all main trajectories are weighed equally.

  * •

”geometric_within”: In the main trajectory, every subtrajecotry is weighed
proportionally to: $\lambda^{l_{sub\tau}}$, where $\lambda$ is a user decided
parameter and $l_{sub\tau}$ is the length of the subtrajectory. Then, all
trajectories are weighed equally.

  * •

”geometric”: Each subtrajectory within every main trajectory is weighed
proportionally to: $\lambda^{l_{sub\tau}}$ along all the possible
subtrajectories.

##  III Application to spinfoams

###  III.1 Boundary state and expectation values

The LQG Hilbert space associated with a graph $\Gamma$ with $L$ links and $N$
nodes is usually written as:

$\mathcal{H}_{\Gamma}=L_{2}\left[SU(2)^{L}/SU(2)^{N}\right]\ .$ (7)

From now on, we omit the $\Gamma$ subscript not to weigh down the notation.
The spin network states $|j_{l},i_{n}\rangle$ constitute a basis in (7), where
$n=1\dots N$, $l=1\dots L$222isolated indices range over multiple values. The
set $j_{l}$ consists of half-integer spins, while $i_{n}$ is an intertwiner
set. Each intertwiner $i$ is a basis element of the $SU(2)$-invariant subspace
of the tensor product of four representations at the corresponding node, since
in 4D every vertex should be $4$-valent meaning that $4$ edges should be
connected to it. In this paper, we consider homogeneous boundary data,
imposing all the boundary links’ spins to be equal so that $j_{l}=j$.
Therefore, only one common spin is attached to all the links in this subspace.
We write a boundary spin network state as:

$|j,i_{n}\rangle=|j,i_{1}\rangle\otimes\dots\otimes|j,i_{N}\rangle\ ,$ (8)

where $N$ is a fixed number. We consider the state $|\psi\rangle$ introduced
in Gozzini_primordial . In the Hilbert space (7) with homogeneous boundary
data, it is defined as:

$|\psi\rangle=\sum_{i_{n}}W\left(j,i_{n}\right)|j,i_{n}\rangle\ ,$ (9)

where $W\left(j,i_{n}\right)$ is the LQG spinfoam amplitude in the spin
network basis. In spinfoam cosmology, amplitudes with a regular boundary graph
and homogeneous data can be interpreted as cosmological states Bianchi:2010zs
. The amplitude truncates the spinfoam vertex expansion from nothing into a
3-dimensional geometry, compatible with the 4-dimensional Lorentzian bulk
Vidotto:2011qa . This provides a spinfoam Lorentzian version of the Hartle-
Hawking wave function of the universe Hartle:1983ai . We refer to the spinfoam
cosmology literature for the physical interpretation of such states
Vidotto:2011qa ; Vidotto:2010kw ; Roken:2010vp ; Bianchi:2011ym ;
Hellmann:2011jn ; Kisielowski:2011vu ; Livine:2011up ; Kisielowski:2012yv ;
Rennert:2013pfa . The same state has been considered in Markov_chain_paper  to
introduce (MCMC) methods in spinfoam theories’ low quantum numbers regime. In
such context, it was used to investigate the cellular decomposition from one
4-simplex by splitting each boundary tetrahedra into four tetrahedra
(performing a graph refinement), computing entanglement entropy, expectation
values, and correlations of geometrical operators. This paper considers the
boundary state (9) with one of the spinfoam models studied in
Markov_chain_paper , namely the 4-simplex, described in section IV.

We focus on the dihedral angle operator. It describes the external dihedral
angle between faces $a$ and $b$ on the tetrahedron dual to node $k$ giving a
notion of local geometry. Its expression in the basis states (9) turns out to
be:

$\langle
j,i_{n}|\cos(\theta)_{k}|j,i_{n}\rangle=\frac{i_{k}(i_{k}+1)-2j(j+1)}{2j(j+1)}\
.$ (10)

The dihedral angle operator (10) has an immediate geometrical interpretation.
Furthermore, it is the simplest (diagonal) operator to compute in the spin
network basis (8). The expectation value of the same operator on the state (9)
has been studied in Gozzini_primordial ; Markov_chain_paper . Therefore, it is
an optimal candidate for testing the Generative Flow Networks algorithms
applied to spinfoam theories. The expectation value on the state (9) of a
local operator $O$ over node $k$ is defined as:

$\langle
O_{k}\rangle\equiv\frac{\langle\psi|O_{k}|\psi\rangle}{\langle\psi|\psi\rangle}\
.$ (11)

For the dihedral angle operator (10), the expectation value (11) can be
computed as:

$\langle\cos(\theta)_{k}\rangle=\frac{1}{Z}\sum_{i_{n}}W\left(j,i_{n}\right)^{2}\langle
j,i_{n}|\cos(\theta)_{k}|j,i_{n}\rangle\ ,$ (12)

where the normalization factor is:

$Z\equiv\langle\psi|\psi\rangle=\sum_{i_{n}}W\left(j,i_{n}\right)^{2}\ .$ (13)

###  III.2 Markov Chain Monte Carlo

In this paper, we employ the Metropolis-Hastings Markov Chain Monte Carlo
algorithm MH_original_paper . A summary of the MH algorithm in its general
form is reported in appendix A. It was originally applied to spinfoams in
Markov_chain_paper  to study the low quantum numbers regime and in
Spinfoam_Lefschetz_thimble  to investigate the large limit, combining it with
the analytical estimates due to the stationary phase approximation. In the
present context, we compare the effectiveness of MCMC used in
Markov_chain_paper  to compute the expectation value of quantum observables
with the Generative Flow Network algorithm described in section II.

To compute (12) exploiting MCMC, we first define the normalized target
distribution over the state space (8) using the spinfoam amplitude:

$f_{j}\left(i_{n}\right)=\frac{W^{2}\left(j,i_{n}\right)}{\sum\limits_{i_{n}}W^{2}\left(j,i_{n}\right)}\
.$ (14)

Each diagram has a different spinfoam amplitude, which can be numerically
computed. We build a Markov chain with length $N_{mc}$ constituted by
intertwiner states $[i_{n}]_{1},[i_{n}]_{2},\dots,[i_{n}]_{N_{mc}}$ using the
MH algorithm. Ideally, each intertwiner state is sampled (indirectly)
according to the multivariate distribution (14). Each state along the chain is
composed of $2j+1$ intertwiners. We sample each intertwiner from a one-
dimensional truncated normal distribution rounded to integers (20). The center
of the proposal distribution is the value of the corresponding intertwiner in
the previous state, truncated between $0$ and $2j$.

The complete algorithm implementation to store Markov chains constituted by
intertwiner states, using the target distribution (14), is shown in the
flowchart 1. The parameters of the MCMC algorithm, the definition of the
proposal distribution, and the expression of the truncated coefficients
related to the proposal distributions are reported in appendix A.

Algorithm 1 MCMC - Random walk

1:for $j=1\dots j_{max}$ do

2: Choose $N_{mc}$, the batch size $B$, and the standard deviation $\sigma$ as
in Table 1

3: Sample a random intertwiners configuration $[i_{n}]_{1}$ and compute
$W\left(j,[i_{n}]_{1}\right)$

4: Set initial multiplicity to $1$

5: for $s=1\dots N_{mc}$ do

6: if s % B = 0 then

7: Store $[i_{n}]_{s}$, $W\left(j,[i_{n}]_{s}\right)$, and the corresponding
multiplicity

8: Dump to disk the states and multiplicities for this batch

9: Set the multiplicity to $1$

10: continue

11: end if

12: Generate a new state $[i_{n}]$ from $[i_{n}]_{s}$

13: if $[i_{n}]=[i_{n}]_{s}$ then

14: Increase the multiplicity by $1$

15: continue

16: else

17: Compute $W\left(j,[i_{n}]\right)$

18: Compute
$p=\textrm{min}\Bigl{\\{}1,\frac{W^{2}(j,[i_{n}])}{W^{2}(j,[i_{n}]_{s})}\frac{C_{0,2j,\sigma}([i_{n}]_{s})}{C_{0,2j,\sigma}([i_{n}])}\Bigl{\\}}$

19: Generate a uniform random number $r$ between $0$ and $1$

20: if $r<p$ then

21: Store $[i_{n}]_{s}$, $W\left(j,[i_{n}]_{s}\right)$, and the corresponding
multiplicity

22: Set $[i_{n}]_{s}\rightarrow[i_{n}]$,
$W\left(j,[i_{n}]_{s}\right)\rightarrow W\left(j,[i_{n}]\right)$

23: Set the multiplicity to $1$

24: else

25: Increase the multiplicity by $1$

26: end if

27: end if

28: end for

29:end for

After running algorithm 1, we use the intertwiner draws along the chain to
evaluate the expectation value (12) by applying the Monte Carlo summation:

$\langle\cos(\theta)_{k}\rangle_{mc}=\frac{1}{N_{MC}}\sum\limits_{s=1}^{N_{mc}}\langle
j,[i_{k}]_{s}|\cos(\theta)_{k}|j,[i_{k}]_{s}\rangle\approx\langle\cos(\theta)_{k}\rangle\
.$ (15)

With respect to (12), we replaced a sum over $(2j+1)^{N}$ intertwiners with a
sum over $N_{mc}$ intertwiner states, where $N$ is the number of boundary
nodes of the corresponding spinfoam diagram. This simplifies the calculation
for many nodes, which would be impractical otherwise.

Additionally, there is a slight difference with respect to the older
implementation, as the code now is run in batches. Specifically, resembling
the GFlowNet batches as explained in section V, a number of random initial
points is used (batch size) and the algorithm is executed parallely for each
one of them. The distribution is scanned as the step grows, mapping an ever
increasing volume in the configuration space.

##  IV The 4 simplex

In this Section, we describe the spinfoam diagrams considered in this paper:
the 4-simplex. The 4-simplex (or “5-cell”) triangulation has been deeply
studied and discussed in LQG literature. It has five boundary tetrahedra glued
on ten triangular faces. The dual triangulation is the usual vertex spinfoam
amplitude, with five boundary nodes and ten links.

In Figure 1, we report a schematic representation of the 2-complex spinfoam
diagrams considered in this paper.

Figure 1: Schematic representation of the $4$-simplex spinfoam model
considered in this paper. Each azure circle represents a vertex, each black
line an edge, and each black dot a boundary node.

We write the EPRL vertex amplitude according to the factorization first
introduced in Speziale2016 . Since then, it has been used in many papers
focused on spinfoam numerical computations with the Lorentzian EPRL model
Dona2019 ; Dona2018 ; Francesco_draft_new_code ; Frisoni_2023 ;
Self_energy_paper ; Radiative_corrections_paper ; VertexRenormalizationPaper ;
dona2022spinfoams ; frisoni2023numerical . We refer to Review_numerical_LQG
for an exhaustive review and description of all the steps required to switch
from the holonomy representation of the EPRL vertex amplitude
book:Rovelli_Vidotto_CLQG  to the one used in the numerical calculations. In
the present context, we write the vertex amplitude as:

$\displaystyle W_{v}^{\gamma}\left(j,\,i_{a};\,\Delta l\right)$
$\displaystyle=\sum\limits_{l_{eq}=j}^{j+\Delta
l}\sum\limits_{k_{e}}\left(\prod_{e}(2k_{e}+1)B_{\gamma}(j,l_{eq};i_{e},k_{e})\right)\\{15j\\}\left(j,l_{eq};k_{e},i_{1}\right)\
,$ (16)

where $a=1\dots 5$ and $e,q=2\dots 5$, $e\neq q$. The integer truncation
parameter $\Delta l$ cuts off the infinite sum over the auxiliary spins. These
appear in the model due to the non-compactness of the $SL(2,\mathrm{C})$
group. The exact value of the amplitude is recovered only in the (ideal) limit
$\Delta l\longrightarrow\infty$, although increasing such parameter
considerably boosts the required computational time. The definitions of the
booster function $B_{\gamma}$ and the $SU(2)$ invariant $\\{15j\\}$ Wigner
symbols are reported in appendix B. The EPRL vertex amplitude (16) can be
efficiently computed with the sl2cfoam-next numerical library
Francesco_draft_new_code . In this paper, we set the Barbero-Immirzi parameter
$\gamma=1$ and $\Delta l=20$.

We report the vertex wiring diagram in Figure 2. It highlights the links’
combinatorics. The label for each intertwiner corresponds to those used in
equation (16). The boundary intertwiners with a blue box are those with one
$SL(2,\mathrm{C})$ integral removed to regularize the amplitude (one for each
vertex). We don’t explicitly label the spins not to clutter the picture.

Figure 2: Wiring of the 2-complex of the vertex diagram.

##  V Spinfoam Probabilities as a (GFlowNet) Hypergrid

The expectation value of the dihedral angle as expressed in (10) defines a
distribution over the (discrete) spin values $j$ and the intertwiners $i_{k}$.
As we are working with homogeneous boundary data all links on the boundary
assume the same (fixed) value of $j$, thus the only remaining variables are
$i_{k}$. In this case, the number of the boundary intertwiners (which are now
the only dof) determine the dimensionality of the hypergrid, which for a
$4$-simplex is $5$. Each one of them can take values in the interval
$[0,2j+1]$, so we can represent the probability distribution as a discrete
$5$-d dypergird, where each intertwiner is mapped to a coordinate:

$\left(i_{1},i_{2},i_{3},i_{4},i_{5}\right),\quad
i_{1},i_{2},\dots,i_{5}\in[0,2j+1]$ (17)

and the corresponding probability amplitude is given by (10).

Using the GFlowNet language this discrete space defines a hypergrid, where,
every point in the configuration space can be mapped to a GFlowNet state.
Using this context, an agent, starting in the beginning of the grid ie for all
spins equal to $0$, is set to explore it using incremental steps along each
possible direction (moving only in one of them per step). Notice how we do not
allow it to move in the backward direction, lowering any of the spin values.
Despite not allowed to get beyond the borders of the grid (values of $j$ more
than the one defined by the user), the agent can stop its scanning randomly at
any point, put it in GFlowNet terms every state can be a terminal state.
Subsequently, the algorithm’s learning process involves getting info for the
whole grid and figuring out which parts are the most relevant for our
calculations.

Figure 3: L1 error and observable estimation for SubTB, exploration: rate
0.01, weighing: geometric within, $\lambda$: 0.9.

##  VI Results

As discussed in section II there are multiple loss functions, we could employ
in order to train our GFN. Specifically, we used: DB, TB, ZVar, FM and SubTB.
For the last, we included the following weighings: DB, ModifiedDB, equal,
equal within and geometric within (also, for different lambdas) . There are
two additional ones, namely, TB and geometric, which did not work in our case.

Apart from the multiple loss functions and weighings, we considered different
sets of of exploration rates, thus, the number of results quickly rises. We
should note that the computation time between the MCMC and the relevant GFNs
differs significantly Table 2.

The GFN was composed of 2 hidden layers each coming with 256 units. It was
trained in 16 batches for $1e5$ iterations each (the same numbers was used for
the corresponding MCMC calculations).

For the analysis, we used three kinds of plots. To be more specific, on the
one hand we calculated the sum of all the absolute differences in between the
true value and the predicted value for both the MCMC and the GFN for the whole
grid. This gives us an understanding of how much each simulation is
approximating the entire grid (in a brute way). In the same figure, we
included a plot presenting how close each simulation is to the observable we
want to compute.

On the other, we wanted to get a more quantitative understanding of how each
simulation works for each point in the grid. Since we are working in a $5D$
setting doing so is a bit tricky. Therefore, we plotted the ”euclidean
distance” of each point from the origin together with the predicted value.

Figure 4: Euclidean distance plot for SubTB, exploration: rate 0.01, weighing:
geometric within, $\lambda$: 0.9.

In order to keep the document short we only included one plot, which gives
results comparable to the ones obtained from MCMC, to justify our later
discussion. The rest can be found in the accompanying repository Repository .

##  VII Discussion

It is evident that for some of the possible parameters’ combinations, the
value of the observable computed is comparable (or in some cases better) to
the MCMC ones, although we should emphasize here that the corresponding
training times are orders of magnitude longer Table 2. Nevertheless, both the
l1 error and the ”euclidean distance” one suggest that the GFN is under-
performing, (despite predicting the correct location of the peaks in the
latter). This contradiction can be resolved if we look closer on the
philosophy of the the GFN and how they were implemented.

That is to say, GFN is used to predict the most relevant points in the grid.
It moves though the latter and while doing so, learns where the most important
values are and focuses on the corresponding points. As a matter of fact, that
explains why in the Figure 3 it reaches the theoretical value faster (in terms
of iterations) than the MCMC, despite the l1 error being higher. In other
words, the GFN revolves around the points that contribute more to the
observable we want to compute, whereas the MCMC learns more about the grid as
whole (lowering the total errors).

This fact can be supported from Figure 4. MCMC approximates better the grid in
its general form, while the GFN assumes narrower peaks ie concentrates on the
most dominant values.

Another argument in favor of the GFN is that it is developed as a tool to be
used in cases where the peaks of the distribution are far away from each other
for example higher dimensional spaces, where MCMC fails because of convergence
issues (only finds some of the peaks). The problem we studied is a fairly
trivial one (after all it can be solved analytically!), thus, we should expect
the MCMC to perform better. Additionally as seen in Figure 4, the peaks are
not far away from each other. Nonetheless, we should focus more on the success
of the algorithm finding the correct value for the observable rather than the
times and resources needed, as our aim is to use it in more complicated cases
where MCMC fails or would need significantly more (possibly infinite)
resources.

Furthermore, a limitation of our implementation lies on the fact that while
the MCMC starts from a different point in the grid each time. The GFN always
starts from 0 only moving forward, requiring longer routes (more resources) to
investigate the grid. There are, though, multiple environments defined in the
literature such as the EBM zhang2022generative  in which only the states in
the border can be terminal state, however the agent can move freely in the
grid. A future direction would be to follow and, possibly, mix these different
GFlowNet constructions so that the algorithm decides on itself its next steps
(without any restrictions on the orientation) freely finding possibly better
(and, also, faster) the high peak areas. In addition, the exploration could
start from the middle (shorter routes) or a random point every time.

Additionally, another interesting direction would be to use a hybrid
algorithm. MCMC is way faster when the peak is located, whereas GFN is better
at finding where the peaks are in complex situations. Combining both of them
could help make the computations faster and more accurate.

Finally, concerning resources, we have used a limited number of batches as
well as hidden dimensions and layers for our simulations restricting the
learning capabilities of our neural network. Therefore, it would be
interesting to check how the results depend on these parameters.

The previous ideas were all based on using the GFlowNets in the problem in
discussion ie calculating expectation values for operators in the context of
spinfoam diagrams. Another possibility would be to apply the algorithm to
building new graphs. As mentioned earlier, spinfoams are essentially graphs
describing quantum geometry in a path integral format. It is, therefore useful
to know which states in the phase space contribute more and a way to access
these in a stochastic way. Since GFlowNets were initially used for the
formation of drug molecules NEURIPS2021_e614f646  an analogous constructive
procedure could be implemented, so that the agent predicts which state will be
more probable during the sampling process.

###### Acknowledgements.

The authors acknowledge the Anishinaabek, Haudenosaunee, Lūnaapéewak,
Attawandaron, and neutral peoples, on whose traditional lands Western
University and the Perimeter Institute are located. The research of PF and JW
at Western University is supported by the by the Natural Science and
Engineering Council of Canada (NSERC) through the Francesca Vidotto’s
Discovery Grant “Loop Quantum Gravity: from Computation to Phenomenology” and
by the ID# 62312 grant from the John Templeton Foundation, as part of the
project “The Quantum Information Structure of Spacetime” (QISS). AK is
supported by an NSERC grant awarded to Lee Smolin. Research at Perimeter
Institute is supported in part by the Government of Canada through the
Department of Innovation, Science and Economic Development Canada and by the
Province of Ontario through the Ministry of Colleges and Universities.

##  Appendix A Metropolis-Hastings algorithm

The Metropolis-Hastings algorithm MH_original_paper  is a Markov Chain Monte
Carlo algorithm that generates samples from a high-dimensional target
probability distribution. We briefly review the algorithm by considering a
multidimensional discrete variable $x$ on a state space $\chi$. We define
$W^{2}_{\chi}(x)$ as an unnormalized target distribution on $\chi$. We refer
to the normalized target distribution as follows:

$f_{\chi}(x)\equiv\frac{W^{2}_{\chi}\left(x\right)}{\sum\limits_{x\in\chi}W^{2}_{\chi}\left(x\right)}\
.$ (18)

If we interpret $\chi$ as a (discrete) Hilbert space, we can compute the
expectation value of a quantum observable $O$ on $\chi$ using the normalized
target distribution (18) as follows:

$O=\frac{\sum\limits_{x\in\chi}W^{2}_{\chi}\left(x\right)o\left(x\right)}{\sum\limits_{x\in\chi}W^{2}_{\chi}\left(x\right)}\
,$ (19)

where $o\left(x\right)$ represents the observable’s matrix element in a
quantum basis of the Hilbert space $\chi$. The Metropolis-Hastings algorithm
allows constructing an ergodic Markov chain on $\chi$ with finite length
$N_{mc}$ such that each state is (indirectly) sampled from the normalized
target distribution (18). The condition for this is that $W_{\chi}(x)$ can be
computed up to a multiplying constant. A positive proposal distribution
$g_{\chi}$ on the space $\chi$ is required to transit from each state to the
next one. We denote such Markov chain as: $[x]_{1},[x]_{2},\
\dots,[x]_{N_{mc}}$ where $[x]_{s}$ indicates the $s$-th state along the
chain.

In this paper, we employ the random walk Metropolis-Hastings. The sampler
locally explores the neighborhood of the current state $[x]_{s}$ of the Markov
chain, proposing a candidate state $[x]^{\prime}$ sampling from the proposal
distribution $g_{\chi}$. Therefore, the algorithm suggests a candidate state
depending on the chain’s current state $[x]_{s}$. The distribution $g_{\chi}$
is multivariate if the space $\chi$ is multi-dimensional (each component of
the state $x$ is sampled from a one-dimensional distribution independent of
the others). We consider a multivariate truncated normal distribution rounded
to integers as a proposal distribution in the Metropolis-Hastings algorithm.
Each component of $x$ is sampled from a one-dimensional truncated normal
distribution rounded to integers:

$g_{n_{1},n_{2},\sigma}\left(n\right)=\frac{\Phi_{\sigma}(n+0.5)-\Phi_{\sigma}(n-0.5)}{\sum\limits_{k=n_{1}}^{n_{2}}\left[\Phi_{\sigma}(k+0.5)-\Phi_{\sigma}(k-0.5)\right]}\
,$ (20)

where $\Phi_{\sigma}(x)$ is the cumulative distribution function of a one-
dimensional normal distribution with zero mean and standard deviation
$\sigma$. Its expression is:

$\Phi_{\sigma}(x)=\frac{1}{\sigma\sqrt{2\pi}}\int_{-\infty}^{x}e^{-\frac{t^{2}}{2\sigma^{2}}}dt\
.$ (21)

For convenience, we define the truncated coefficients:

$C_{n_{1},n_{2},\sigma}\left([x]\right)=\prod\limits_{i=1}^{N}\left\\{\sum\limits_{n=n_{1}-x_{i}}^{n_{2}-x_{i}}\left[\Phi_{\sigma}(n+0.5)-\Phi_{\sigma}(n-0.5)\right]\right\\}\
,$ (22)

where we defined with $x_{i}$ the $i$-th component of the draw $[x]$. The
Markov Chain is built by evaluating the ratio between the target distribution
(18) times the truncated coefficients (22) at the proposal state and the same
quantity at the current state. Then, we accept the proposal state with a
probability equal to this state. Otherwise, we stay at the current point. The
initial steps of the algorithm are usually removed as burn-in iterations
during the thermalization phase, although in this paper, we set the burn-in
parameter to zero.

After storing a Markov chain with the desired length $N_{mc}$, we can compute
(19) using a standard Monte Carlo approach on the multi-dimensional sum over
$x$. We consider all the states of the chain as a statistical sample in the
Monte Carlo evaluation. We obtain a reasonably good estimate of the original
quantity if the number of samples is large enough:

$O_{{mc}}=\frac{1}{N_{mc}}\sum\limits_{s=1}^{N_{mc}}o\left([x]_{s}\right)\approx
O\hskip 8.53581pt\textrm{for $N_{mc}\gg 1$}\ .$ (23)

The Monte Carlo estimate (23) is precisely equal to (19) only in the (ideal)
limit of an infinite number of samples. Moreover, the convergence is faster
than the standard version of Monte Carlo VertexRenormalizationPaper  (in which
the draws are sampled randomly) because each draw is generated from the
distribution (18).

There is a non-zero correlation between $[x]_{s}$ and $[x]_{s+d}$ where $d\geq
1$. This is because each proposed state depends on the previous one (as the
process is Markovian). For each quantity (23) we can compute the
autocorrelation function with lag $d$:

$R_{O}\left(d\right)=\frac{\sum\limits_{s=d+1}^{N_{mc}}\left(o([x]_{s})-O_{N_{mc}}\right)\left(o([x]_{s-d})-O_{N_{mc}}\right)}{\sum\limits_{s=1}^{N_{mc}}\left(o([x]_{s})-O_{N_{mc}}\right)^{2}}\
.$ (24)

Since the Markov Chain converges to a stationary distribution, the
autocorrelation (24) should decrease as the lag $d$ increases.

Finally, we report in Table 1 the parameters used in the MCMC algorithm 1 for
the calculations considered in this paper.

MH - parameters (4-simplex)
---
$j$ |  $N_{mc}$ |  B |  $\sigma$
0.5 |  $10^{7}$ |  $10^{6}$ |  $0.8$
1.0 |  $10^{7}$ |  $10^{6}$ |  $0.8$
1.5 |  $10^{7}$ |  $10^{6}$ |  $0.8$
2.0 |  $10^{7}$ |  $10^{6}$ |  $0.8$
2.5 |  $10^{7}$ |  $10^{6}$ |  $0.8$
3.0 |  $10^{7}$ |  $10^{6}$ |  $0.8$
3.5 |  $10^{7}$ |  $10^{6}$ |  $0.8$
4.0 |  $10^{7}$ |  $10^{6}$ |  $0.8$
4.5 |  $10^{7}$ |  $10^{6}$ |  $0.8$
5.0 |  $10^{7}$ |  $10^{6}$ |  $0.8$
5.5 |  $10^{7}$ |  $10^{6}$ |  $0.8$
6.0 |  $10^{7}$ |  $10^{6}$ |  $0.8$

MH - parameters (star)
---
$j$ |  $N_{mc}$ |  B |  $\sigma$
0.5 |  $10^{7}$ |  $10^{6}$ |  $0.8$
1.0 |  $10^{7}$ |  $10^{6}$ |  $0.8$
1.5 |  $10^{7}$ |  $10^{6}$ |  $0.8$
2.0 |  $10^{7}$ |  $10^{6}$ |  $0.8$
2.5 |  $10^{7}$ |  $10^{6}$ |  $0.8$
3.0 |  $10^{7}$ |  $10^{6}$ |  $0.8$
3.5 |  $10^{7}$ |  $10^{6}$ |  $0.8$
4.0 |  $10^{7}$ |  $10^{6}$ |  $0.8$
4.5 |  $10^{7}$ |  $10^{6}$ |  $0.8$
5.0 |  $10^{7}$ |  $10^{6}$ |  $0.8$
5.5 |  $10^{7}$ |  $10^{6}$ |  $0.8$
6.0 |  $10^{7}$ |  $10^{6}$ |  $0.8$

Table 1: Parameters used in the MCMC algorithm 1. From left to right: $j$ is
the boundary spin of spinfoam amplitude, $N_{mc}$ is the total number of
iterations over the chain, “B” is the number of states in each batch, and
$\sigma$ is the standard deviation of the proposal distribution.

##  Appendix B Wigner symbols and booster function

This Appendix reports the expression of some invariant Wigner symbols and the
booster function. These appear in the vertex amplitude expression (16).

###  B.1 Wigner symbols

First, we write the expression of the $(4jm)$ Wigner symbols. These can be
defined as the contraction of two $(3jm)$ Wigner symbols labeled by one
additional spin:

$\left(\begin{array}[]{cccc}j_{1}&j_{2}&j_{3}&j_{4}\\\
m_{1}&m_{2}&m_{3}&m_{4}\end{array}\right)^{(k)}=\sum_{m}(-1)^{k-m}\left(\begin{array}[]{ccc}j_{1}&j_{2}&k\\\
m_{1}&m_{2}&m\end{array}\right)\left(\begin{array}[]{ccc}k&j_{3}&j_{4}\\\
-m&m_{3}&m_{4}\end{array}\right)\ .$ (25)

Second, we report the canonical definition of the $\\{6j\\}$ symbol in terms
of $(3jm)$ symbols, which is:

$\displaystyle\left\\{\begin{array}[]{ccc}j_{1}&j_{2}&j_{3}\\\
j_{4}&j_{5}&j_{6}\end{array}\right\\}$ $\displaystyle=\sum\limits_{m_{1}\dots
m_{6}}(-1)^{\sum\limits_{k=1}^{6}(j_{k}-m_{k})}\left(\begin{array}[]{ccc}j_{1}&j_{2}&j_{3}\\\
m_{1}&m_{2}&-m_{3}\end{array}\right)\left(\begin{array}[]{ccc}j_{1}&j_{5}&j_{6}\\\
-m_{1}&m_{5}&m_{6}\end{array}\right)\times$ (32) $\displaystyle\hskip
17.07164pt\times\left(\begin{array}[]{ccc}j_{4}&j_{5}&j_{3}\\\
m_{4}&-m_{5}&m_{3}\end{array}\right)\left(\begin{array}[]{ccc}j_{4}&j_{2}&j_{6}\\\
-m_{4}&-m_{2}&-m_{6}\end{array}\right)\ .$ (37)

The $\\{6j\\}$ Wigner symbol (32) can be computed and stored efficiently using
high-performance computing libraries such as
johanssonFastAccurateEvaluation2015 . Finally, write the $\\{15j\\}$ symbol of
the first kind according to the conventions of GraphMethods . It can be
written using (32) as follows:

$\begin{split}\left\\{\begin{array}[]{ccccc}j_{1}&j_{2}&j_{3}&j_{4}&j_{5}\\\
l_{1}&l_{2}&l_{3}&l_{4}&l_{5}\\\
p_{1}&p_{2}&p_{3}&p_{4}&p_{5}\end{array}\right\\}&=(-1)^{\sum_{k=1}^{5}j_{k}+l_{k}+p_{k}}\sum_{x}(2x+1)\left\\{\begin{array}[]{ccc}j_{1}&p_{1}&x\\\
p_{2}&j_{2}&l_{1}\end{array}\right\\}\left\\{\begin{array}[]{ccc}j_{2}&p_{2}&x\\\
p_{3}&j_{3}&l_{2}\end{array}\right\\}\\\ &\hskip
17.07164pt\times\left\\{\begin{array}[]{ccc}j_{3}&p_{3}&x\\\
p_{4}&j_{4}&l_{3}\end{array}\right\\}\left\\{\begin{array}[]{ccc}j_{4}&p_{4}&x\\\
p_{5}&j_{5}&l_{4}\end{array}\right\\}\left\\{\begin{array}[]{ccc}j_{5}&p_{5}&x\\\
j_{1}&p_{1}&l_{5}\end{array}\right\\}\ .\end{split}$ (38)

We refer to GraphMethods ; book:varshalovic  for the analytical definitions
and graphical representations of the generalized $\\{nj\\}$ invariant symbol
appearing in the $SU(2)$ recoupling theory.

###  B.2 Booster function

The booster function encodes the contribution due to the boosts in the
Lorentzian EPRL model. It was first introduced in Speziale2016  and is defined
as an integral over the boost rapidity parameter $r$ in the $\gamma$-simple
$SL(2,\mathrm{C})$ unitary representation:

$\begin{split}B_{\gamma}&\left(j_{1},j_{2},j_{3},j_{4},l_{1},l_{2},l_{3},l_{4};i,k\right)=\\\
&\sum_{m_{f}}\left(\begin{array}[]{cccc}l_{1}&l_{2}&l_{3}&l_{4}\\\
m_{1}&m_{2}&m_{3}&m_{4}\end{array}\right)^{(k)}\left(\int_{0}^{\infty}dr\frac{1}{4\pi}\sinh^{2}r\,\bigotimes_{f=1}^{4}d^{\gamma
j_{f},j_{f}}_{l_{f}j_{f}m_{f}}(r)\right)\left(\begin{array}[]{cccc}j_{1}&j_{2}&j_{3}&j_{4}\\\
m_{1}&m_{2}&m_{3}&m_{4}\end{array}\right)^{(i)}\ ,\end{split}$ (39)

where the $(4jm)$ Wigner symbol has been defined in (25). The expression of
the reduced matrix elements $d_{jlm}^{\gamma j,j}(r)$ has been originally
written in Ruhl:1970fk , and it reads:

$\begin{split}d^{(\gamma
j,j)}_{jlp}(r)=&(-1)^{\frac{j-l}{2}}\frac{\Gamma\left(j+i\gamma
j+1\right)}{\left|\Gamma\left(j+i\gamma
j+1\right)\right|}\frac{\Gamma\left(l-i\gamma
j+1\right)}{\left|\Gamma\left(l-i\gamma
j+1\right)\right|}\frac{\sqrt{2j+1}\sqrt{2l+1}}{(j+l+1)!}\\\
&\times\left[(2j)!(l+j)!(l-j)!\frac{(l+p)!(l-p)!}{(j+p)!(j-p)!}\right]^{1/2}e^{-(j-i\gamma
j+p+1)r}\\\
&\times\sum_{s}\frac{(-1)^{s}\,e^{-2sr}}{s!(l-j-s)!}\,{}_{2}F_{1}[l+1-i\gamma
j,j+p+1+s,j+l+2,1-e^{-2r}]\ ,\end{split}$ (40)

where ${}_{2}F_{1}$ is the Gauss hypergeometric function.

##  Appendix C GFN parameters and Times

In the following table, we present the times needed for the training of the
algorithm for the different sets of the parameters. The simulations were run
in 40 cores (each core supports 2 ”hyper-threads”) and 200 GB of memory in
Perimeter Institute’s computer cluster. For comparison, in the same nodes the
MCMC needed (for the same parameters) is $33.67s$.

Parametrization | Exploration Rate | Weighing | $lambda$ | Time (in s)
---|---|---|---|---
DB | 0.01 |  |  | 24233.27
DB | 0.10 |  |  | 25473.84
DB | 0.90 |  |  | 19307.97
FM | 0.01 |  |  | 28059.27
FM | 0.10 |  |  | 28797.55
FM | 0.90 |  |  | 18619.80
ZVar | 0.01 |  |  | 22933.50
ZVar | 0.10 |  |  | 23698.39
ZVar | 0.90 |  |  | 17302.39
SubTB | 0.01 | DB |  | 27629.19
SubTB | 0.01 | DB |  | 34006.91
SubTB | 0.10 | DB |  | 28957.59
SubTB | 0.10 | DB |  | 35847.32
SubTB | 0.90 | DB |  | 21059.68
SubTB | 0.01 | ModifiedDB |  | 28199.58
SubTB | 0.01 | ModifiedDB |  | 28557.65
SubTB | 0.10 | ModifiedDB |  | 30001.29
SubTB | 0.10 | ModifiedDB |  | 30329.59
SubTB | 0.90 | ModifiedDB |  | 20081.74
SubTB | 0.90 | ModifiedDB |  | 20226.71
SubTB | 0.01 | equal |  | 26841.31
SubTB | 0.01 | equal |  | 26309.74
SubTB | 0.10 | equal |  | 28822.90
SubTB | 0.10 | equal |  | 27456.59
SubTB | 0.90 | equal |  | 21543.87
SubTB | 0.90 | equal |  | 21971.17
SubTB | 0.01 | equal_within |  | 25821.10
SubTB | 0.10 | equal_within |  | 26651.84
SubTB | 0.90 | equal_within |  | 20849.78
SubTB | 0.01 | geometric_within | 0.1 | 27996.97
SubTB | 0.01 | geometric_within | 0.9 | 25057.91
SubTB | 0.01 | geometric_within | 3 | 25803.05
SubTB | 0.10 | geometric_within | 0.1 | 29644.58
SubTB | 0.10 | geometric_within | 0.9 | 26601.28
SubTB | 0.10 | geometric_within | 3 | 26717.54
SubTB | 0.90 | geometric_within | 0.1 | 20455.08
SubTB | 0.90 | geometric_within | 0.9 | 20818.05
SubTB | 0.90 | geometric_within | 3 | 21188.65
Table 2: Times for different parameters.

## References

  * (1) J. B. Hartle and S. W. Hawking, “Wave function of the Universe,” Phys. Rev. D 28 (Dec, 1983) 2960–2975. https://link.aps.org/doi/10.1103/PhysRevD.28.2960.
  * (2) J.-L. Lehners, “Review of the no-boundary wave function,” Physics Reports 1022 (2023) 1–82. https://www.sciencedirect.com/science/article/pii/S0370157323001904. Review of the No-Boundary Wave Function.
  * (3) F. Gozzini, “A high-performance code for EPRL spin foam amplitudes,” Classical and Quantum Gravity 38 no. 22, (Oct, 2021) 225010. http://dx.doi.org/10.1088/1361-6382/ac2b0b.
  * (4) M. Han, Z. Huang, H. Liu, D. Qu, and Y. Wan, “Spinfoam on a Lefschetz thimble: Markov chain Monte Carlo computation of a Lorentzian spinfoam propagator,” Physical Review D 103 no. 8, (Apr, 2021) . http://dx.doi.org/10.1103/PhysRevD.103.084026.
  * (5) S. Steinhaus, “A Monte Carlo algorithm for spin foam intertwiners,” arXiv:2403.04836 [gr-qc].
  * (6) P. Frisoni, F. Gozzini, and F. Vidotto, “Markov chain Monte Carlo methods for graph refinement in spinfoam cosmology,” Class. Quant. Grav. 40 no. 10, (2023) 105001, arXiv:2207.02881 [gr-qc].
  * (7) G. Kanwar, M. S. Albergo, D. Boyda, K. Cranmer, D. C. Hackett, S. Racanière, D. J. Rezende, and P. E. Shanahan, “Equivariant Flow-Based Sampling for Lattice Gauge Theory,” Physical Review Letters 125 no. 12, (Sep, 2020) . https://doi.org/10.1103%2Fphysrevlett.125.121601.
  * (8) M. Albergo, G. Kanwar, and P. Shanahan, “Flow-based generative models for Markov chain Monte Carlo in lattice field theory,” Physical Review D 100 no. 3, (Aug, 2019) . https://doi.org/10.1103%2Fphysrevd.100.034515.
  * (9) N. Malkin, M. Jain, E. Bengio, C. Sun, and Y. Bengio, “Trajectory balance: Improved credit assignment in GFlowNets,” 2022.
  * (10) E. Bengio, M. Jain, M. Korablyov, D. Precup, and Y. Bengio, “Flow Network based Generative Models for Non-Iterative Diverse Candidate Generation,” in Advances in Neural Information Processing Systems, M. Ranzato, A. Beygelzimer, Y. Dauphin, P. Liang, and J. W. Vaughan, eds., vol. 34, pp. 27381–27394.  Curran Associates, Inc., 2021.  https://proceedings.neurips.cc/paper_files/paper/2021/file/e614f646836aaed9f89ce58e837e2310-Paper.pdf.
  * (11) Y. Bengio, S. Lahlou, T. Deleu, E. J. Hu, M. Tiwari, and E. Bengio, “GFlowNet Foundations,” 2022.
  * (12) Y. Bengio, “The GFlowNet Tutorial,” 2022.  https://yoshuabengio.org/2022/03/05/generative-flow-networks/.
  * (13) S. Lahlou, J. D. Viviano, V. Schmidt, and Y. Bengio, “torchgfn: A PyTorch GFlowNet library,” 2023.
  * (14) D. W. Zhang, C. Rainone, M. Peschl, and R. Bondesan, “Robust Scheduling with GFlowNets,” 2023.
  * (15) K. Madan, J. Rector-Brooks, M. Korablyov, E. Bengio, M. Jain, A. C. Nica, T. Bosc, Y. Bengio, and N. Malkin, “Learning GFlowNets from partial episodes for improved convergence and stability,” 2023.  https://openreview.net/forum?id=UYS38ssi1M.
  * (16) F. Gozzini and F. Vidotto, “Primordial Fluctuations From Quantum Gravity,” Frontiers in Astronomy and Space Sciences 7 (Feb, 2021) . http://dx.doi.org/10.3389/fspas.2020.629466.
  * (17) E. Bianchi, C. Rovelli, and F. Vidotto, “Towards Spinfoam Cosmology,” Phys. Rev. D82 (2010) 84035, arXiv:1003.3483.
  * (18) F. Vidotto, “Many-nodes/many-links spinfoam: the homogeneous and isotropic case,” Class. Quant Grav. 28 no. 245005, (2011) , arXiv:1107.2633.
  * (19) J. B. Hartle and S. W. Hawking, “Wave Function of the Universe,” Phys. Rev. D28 (1983) 2960–2975.
  * (20) F. Vidotto, “Spinfoam Cosmology: quantum cosmology from the full theory,” J. Phys.: Conf. Ser. 314 no. 012049, (2010) , arXiv:1011.4705.
  * (21) C. Roken, “The first-order $\backslash$hbar correction in covariant, holomorphic spinfoam cosmology,” arXiv:1011.3335 [gr-qc].
  * (22) E. Bianchi, T. Krajewski, C. Rovelli, and F. Vidotto, “Cosmological constant in spinfoam cosmology,” Phys. Rev. D83 (2011) 104015, arXiv:1101.4049 [gr-qc].
  * (23) F. Hellmann, “On the Expansions in Spin Foam Cosmology,” Phys. Rev. D 84 (2011) 103516.
  * (24) M. Kisielowski, J. Lewandowski, and J. Puchta, “Feynman diagrammatic approach to spin foams,” arXiv:1107.5185 [gr-qc].
  * (25) E. R. Livine and M. Martin-Benito, “Classical Setting and Effective Dynamics for Spinfoam Cosmology,” arXiv:1111.2867 [gr-qc].
  * (26) M. Kisielowski, J. Lewandowski, and J. Puchta, “One vertex spin-foams with the Dipole Cosmology boundary,” arXiv:1203.1530 [gr-qc].
  * (27) J. Rennert and D. Sloan, “Towards Anisotropic Spinfoam Cosmology,” arXiv:1304.6688 [gr-qc].
  * (28) P. Frisoni, F. Vidotto, and F. Gozzini, “Markov Chain Monte Carlo methods for graph refinement in Spinfoam Cosmology,” Classical and Quantum Gravity (Mar, 2023) . https://doi.org/10.1088%2F1361-6382%2Facc5d6.
  * (29) W. K. Hastings, “Monte Carlo Sampling Methods Using Markov Chains and Their Applications,” Biometrika 57 no. 1, (1970) 97–109. http://www.jstor.org/stable/2334940.
  * (30) S. Speziale, “Boosting Wigner’s nj-symbols,” Journal of Mathematical Physics 58 no. 3, (Sep, 2017) , arXiv:1609.01632. http://arxiv.org/abs/1609.01632http://dx.doi.org/10.1063/1.4977752.
  * (31) P. Donà, M. Fanizza, G. Sarno, and S. Speziale, “Numerical study of the Lorentzian Engle-Pereira-Rovelli-Livine spin foam amplitude,” Physical Review D 100 no. 10, (Mar, 2019) , arXiv:1903.12624.
  * (32) P. Donà and G. Sarno, “Numerical methods for EPRL spin foam transition amplitudes and Lorentzian recoupling theory,” General Relativity and Gravitation 50 no. 10, (2018) .
  * (33) P. Frisoni, “Studying the EPRL spinfoam self-energy,” in The Sixteenth Marcel Grossmann Meeting.  WORLD SCIENTIFIC, Jan, 2023.  https://doi.org/10.1142%2F9789811269776_0336.
  * (34) P. Frisoni, F. Gozzini, and F. Vidotto, “Numerical analysis of the self-energy in covariant loop quantum gravity,” Physical Review D 105 no. 10, (May, 2022) . https://doi.org/10.1103%2Fphysrevd.105.106018.
  * (35) P. Donà , P. Frisoni, and E. Wilson-Ewing, “Radiative corrections to the Lorentzian Engle-Pereira-Rovelli-Livine spin foam propagator,” Physical Review D 106 no. 6, (Sep, 2022) . https://doi.org/10.1103%2Fphysrevd.106.066022.
  * (36) P. Dona and P. Frisoni, “Summing bulk quantum numbers with Monte Carlo in spin foam theories,” 2023.
  * (37) P. Dona, M. Han, and H. Liu, “Spinfoams and high performance computing,” 2022\.
  * (38) P. Frisoni, “A numerical approach to the Black-to-White hole transition,” 2023\.
  * (39) P. Donà and P. Frisoni, “How-to Compute EPRL Spin Foam Amplitudes,” Universe 8 no. 4, (Mar, 2022) 208. https://doi.org/10.3390%2Funiverse8040208.
  * (40) C. Rovelli and F. Vidotto, Covariant Loop Quantum Gravity.  Cambridge University Press, Cambridge, 2015.
  * (41) J. Bunao, P. Frisoni, J. Wogan, and A. Kogios, “GFN in Spin Foams,” 2024.  https://github.com/JosephRRB/torchgfn_spinfoams.
  * (42) D. Zhang, N. Malkin, Z. Liu, A. Volokhova, A. Courville, and Y. Bengio, “Generative Flow Networks for Discrete Probabilistic Modeling,” 2022.
  * (43) H. T. Johansson and C. Forssén, “Fast and Accurate Evaluation of Wigner 3j, 6j, and 9j Symbols Using Prime Factorisation and Multi-Word Integer Arithmetic,” SIAM Journal on Scientific Computing 38 no. 1, (Apr., 2015) A376–A384.
  * (44) A. P. Yutsin, I. B. Levinson, and V. V. Vanagas, Mathematical Apparatus of the Theory of Angular Momentum.  Israel Program for Scientific Translation, Jerusalem, Israel, 1962.
  * (45) V. D. Aleksandroviic, A. N. Moskalev, and K. V. Kel’manoviic, Quantum theory of angular momentum: irreducible tensors, spherical harmonics, vector coupling coefficients, 3nj symbols.  World scientific, 1988.
  * (46) W. Ruhl, The Lorentz group and harmonic analysis.  W.A. Benjamin, Inc, New York, 1970.


# A Speech Representation Anonymization Framework via Selective Noise
Perturbation

###### Abstract

Privacy and security are major concerns when communicating speech signals to
cloud services such as automatic speech recognition (ASR) and speech emotion
recognition (SER). Existing solutions for speech anonymization mainly focus on
voice conversion or voice modification to convert a raw utterance into another
one with similar content but different, or no, identity-related information.
However, an alternative approach to share speech data under the form of
privacy-preserving representation has been largely under-explored. In this
paper, we propose a speech anonymization framework that achieves privacy via
noise perturbation to a selected subset of the high-utility representations
extracted using a pre-trained speech encoder. The subset is chosen with a
Transformer-based privacy-risk saliency estimator. We validate our framework
on four tasks, namely, Automatic Speaker Verification (ASV), ASR, SER and
Intent Classification (IC) for privacy and utility assessment. Experimental
results show that our approach is able to achieve a competitive, or even
better, utility compared to the speech anonymization baselines from the
VoicePrivacy2022 Challenges, providing the same level of privacy. Moreover,
the easily-controlled amount of perturbation allows our framework to have a
flexible range of privacy-utility trade-offs without re-training any
component.

Index Terms—  privacy, speech representations, transformer

##  1 Introduction

Collecting and sharing speech data from local devices are essential for cloud-
based speech processing models. From the service providers’ perspective,
gaining access to more data enables them to train more robust models and
provide better services. From the users’ perspective, sharing personal data
allows experiencing more personalized services such as healthcare [1] or
automatic speech recognition (ASR) [2]. However, privacy and security are
major concerns when communicating personal speech data, as raw utterances
often contain sensitive identifiable information such as ethnicity, gender, or
even health condition. One promising line of solution focuses on client-side
privacy [3], providing users full control to protect their privacy by
sanitizing raw utterances locally before uploading the data for cloud-based
processing without the need for a trusted server.

Anonymization, which aims at removing identifiable information from speech
signals while retaining other information, is a commonly used approach. The
majority of speech anonymization methods focus on voice anonymization that
either modifies audio inputs via signal processing techniques [4, 5] or
utilizes speech synthesis to perturb the x-vector [6] of the inputs [7, 3, 8].
On the other hand, existing studies on privacy-preserving representation
learning for speech generally leverage adversarial training to remove
identity-related information from representations, but are limited to a single
target application such as ASR [9] or emotion recognition [10]. To the best of
our knowledge, there has been no prior work on privacy-preserving
representation learning for multiple applications (as in this work).

In this study, we propose a speech anonymization framework to sanitize the
representations extracted from pre-trained speech encoders. At the core of our
approach is a Privacy-risk Saliency Estimator (PSE) that learns to predict the
importance of individual representation positions from a speaker
identification system. Based on the estimations from PSE, the top $k\%$
positions with the highest estimated privacy-risk would be perturbed by adding
easily-controlled Laplacian noise. We validate our method on four tasks,
namely, Automatic Speaker Verification (ASV) for privacy evaluation, Automatic
Speaker Verification (ASR), Emotion Recognition (ER), and Intent
Classification (IC) for utility performance. Our work provides preliminary
evidence that using speech data in the form of extracted representations
results in systems achieving competitive performance compared to the commonly-
used voice conversion/modification approaches. Moreover, our approach enables
a flexible privacy-utility trade-off.

##  2 Proposed Method

Figure 1 shows a high-level overview of the proposed method. We first train a
Speaker Identification model on a closed set of speakers with dataset $D$,
which is used to construct a Saliency Map Dataset. We then train a Privacy-
risk Saliency Estimator (PSE) to predict the privacy-salient positions given
the extracted representations. The PSE module allows our framework to be
extensible to unseen speakers outside $D$. Finally, we add controlled noise to
selected positions within the input representations to produce sanitized
output representations.

Fig. 1: Overview of our framework. Solid lines show the workflow of the
training process for the Privacy-risk Saliency Estimator. Dashed lines show
the workflow during the inference process to produce a sanitized
representation for a raw utterance.

Privacy-risk Saliency Map Dataset. The idea of saliency maps was originally
proposed in computer vision to identify the most salient regions in images for
trained classification models’ decisions. Gradient-based methods construct the
saliency map for a given image as the absolute value of the gradient of the
input pixels with respect to the loss for the true class of the image, which
is accessible via backpropagation. In this work, we use SmoothGrad [11] to
compute saliency maps. It is important to note that we can only compute
saliency maps if ground-truth labels (speaker identities in this case) are
available. We do not have this information during the inference process, which
motivates the development of a model that can estimate the saliency maps
directly from the input representations.

For a given 2D representation $x\in\mathbb{R}^{t\times d}$ of an utterance, we
hypothesize that some positions in $x$ contain more sensitive information than
the others (i.e., these positions can leak more identifiable information than
the others). Note that these positions can change given different speech
signals with different representations. We use SmoothGrad to find these
positions. Specifically, we first train a Speaker Identification (SID) model
$M$ on a dataset $D=\\{(x_{i},y_{i})\\}$, with $x_{i}$ being the extracted
features from a pre-trained model (e.g., wav2vec2) and $y_{i}$ being the
identity of $x_{i}$. For each sample $(x_{i},y_{i})$ in $D$, we compute the
privacy-risk saliency map $s_{i}$ with SmoothGrad, which is the average of
$\lvert\frac{\delta L_{y_{i}}}{\delta x_{i}+\epsilon}\rvert$, where
$L_{y_{i}}$ is the loss of $M$ on the true class $y_{i}$ of $x_{i}$ with
different versions of noise $\epsilon$. As a result, we construct a privacy-
risk saliency map dataset $D^{\prime}=\\{(x_{i},s_{i})\\}$ containing the same
number of samples as $D$ with $x_{i}$ and $s_{i}$ of the same sizes. This
dataset is then used to train our PSE module.
Privacy-risk Saliency Estimator. The purpose of the Privacy-risk Saliency
Estimator (PSE) is to take in the extracted feature representations $x_{i}$
for an utterance $u$ and output an estimate for the privacy-risk saliency map
$s^{\prime}_{i}$ such that $s^{\prime}_{i}\approx s_{i}$. A good PSE should be
able to (a) model the dynamics of the saliency maps given different speech
representations; and (b) generalize to unseen speakers outside $D$. We use a
Transformer encoder [12] as our architecture for the PSE. As a high-level
overview, the Transformer encoder is a stack of Transformer layers, each of
which consists of a self-attention module followed by a feed-forward neural
network. For a given $x_{i}\in\mathbb{R}^{t\times d}$, we expect the self-
attention modules to capture information along the temporal dimension while
the feed-forward neural networks capture important signals from the feature
dimension, which enables the architecture to make precise position-level
predictions. The model’s parameters are optimized with the $L1$ Loss between
the outputs of PSE and the ground-truth saliency maps extracted with a trained
Speaker Identification model, as explained above.
Noise Perturbation. We assume that not all positions within the extracted
representations from pre-trained encoders carry the same amount of speaker-
related information. By injecting noise to the more sensitive positions, we
hope to reduce the sensitive information carried within these dimensions to
achieve privacy. To improve the utility of the output representations, we keep
the less sensitive, i.e., insensitive positions, intact. In this study, we
define the group of sensitive positions as positions with the top $k\%$
highest privacy-risk values estimated by PSE. We later investigate the effect
of $k$ on the privacy and utility of the sanitized outputs. Motivated by
Differential Privacy and the Laplace mechanism [13], we set a fixed input-
independent bound of $[-1,1]$ and add Laplacian noises
$\stackrel{{\scriptstyle iid}}{{\sim}}Lap(\frac{2}{\epsilon})$ independently
to the selected positions.

##  3 Experimental Setups

In this study, we focus on sanitizing the extracted representations from
wav2vec2 [14]. However, the proposed framework should be applicable to other
pre-trained speech encoders. Our implementations and training procedures
follow the SUPERB benchmark [15] to enhance reproducibility. To show that the
sanitized representations is applicable to multiple applications, we pick
three downstream tasks with emphasis on different aspects of speech, namely,
ASR for content, ER for paralinguistics, and IC for semantics. Our code is
publicly available111https://github.com/mtran14/dp_w2v2.git.

###  3.1 Tasks, Datasets & Models

VoxCeleb1 (SID & ASV). We use the VoxCeleb1 dataset [16] for training the SID
and ASV models. It contains approximately $150,000$ utterances from $1,211$
speakers collected from around $21,819$ videos on YouTube. For a fair privacy
evaluation, we first split the dataset into two partitions. The first subset
contains approximately $50,000$ utterances from $400$ speakers, and is used to
train the SID model and construct the Privacy-risk Saliency Map dataset. The
second subset, which contains around $100,000$ utterances from the remaining
$811$ speakers, is used to train the ASV model for privacy evaluation. The
output representations from wav2vec2 is fed to the SID model consisting of a
mean-pooling layer followed by a fully-connected layer to make the speaker
identity predictions. The SID model is trained using a cross-entropy loss. For
ASV, we train the x-vector model [6] with an AMSoftmax loss [17]. Following
prior work [18, 3, 5], we report the equal error rate (EER) as our privacy
metric (a higher EER implies a better privacy-preserving representation).

IEMOCAP (ER). We use the IEMOCAP dataset [19] for training the ER model. The
original dataset contains approximately $12$ hours of data from $10$ actors
performing improvised or scripted scenarios that are designed to invoke
emotions. Following the SUPERB benchmark’s setting, we only keep four emotion
classes (neutral, happy, sad, angry) from the IEMOCAP dataset to get a more
balanced dataset. The ER model adds a mean-pooling layer followed by a fully-
connected layer to the encoder for emotion recognition. We report the
classification accuracy as the utility metric.

Fluent Speech Commands (IC). We use the Fluent Speech Commands dataset [20] to
train our IC model. The dataset contains around $30,000$ utterances from $97$
speakers with $31$ unique intents. Similar to ER, the IC model contains a
mean-pooling layer followed by a fully-connected layer. We report the
classification accuracy as the utility metric.

LibriSpeech (ASR). We use the commonly used LibriSpeech audiobook dataset [21]
to train our ASR model. Specifically, we use LibriSpeech’s train-
clean-100/dev-clean/test-clean subsets to train, validate and test our model.
The training set contains more than $100$ hours of transcribed speech from
$251$ speakers while both the validation and test set contains more than $5$
hours of transcribed speech from $40$ speakers. Our ASR model is a two-layer
1024-unit Bidirectional LSTM, trained with the CTC loss [22] on characters. We
do not use any language model to improve the model’s performance during
inference. The Word Error Rate (WER) is used as the utility metric.

###  3.2 Baselines

We use the baselines provided in the VoicePrivacy2022 Challenge [23], in
addition to a signal processing-based methods from [5] as our baseline.
Baseline 1. We use the first baseline provided in the VoicePrivacy2022
Challenge, which attempts to modify the x-vector for input speech signals. The
method first extracts the x-vector [6], the fundamental frequency (F0), and
bottleneck features from the input speech signal. The method then modifies the
extracted x-vector with an external pool of x-vectors before using a speech
synthesizer to produce an anonymized speech from the extracted features. We
use a variant of the first baseline with features extracted from a finetuned
wav2vec2.0 model [14] and a HifiGAN-based speech synthesizer (more details can
be found from Section 6.4.3 in [23]).
Baseline 2. We use the second baseline provided in the VoicePrivacy2022
Challenge, which is a signal processing approach using McAdams transformation
[4]. The McAdam Coefficient is uniformly sampled from a fixed range to get a
randomized version of the anonymization method in [24].
Baseline 3. We use the R+MS method proposed by Kai et al. [5]. It is a
combination of Resampling and Modulation Spectrum Smoothing [25], and is the
method suggested by Kai et al. to achieve strong privacy-utility trade-offs.
Baseline 4. We use the R+MS+M+CH+CL method proposed by Kai et al. [5]. The
method consists of Resampling, Modulation Spectrum Smoothing, McAdams
Transformation [4], clipping, and Chorus. It is the most secured approach from
their provided toolkit [5].

###  3.3 Details.

Implementation. Our PSE is a standard Transformer encoder, which consists of
$6$ layers with a dropout rate of $0.1$. Each transformer encoder layer
contains a self-attention module with $12$ heads. The sizes of the feed-
forward layers in each Transformer block are $3072$. The model is optimized
with the $L1$ loss using the Adam optimizer with a learning rate of $1e^{-4}$
and a batch size of $32$ for $60$ epochs. Before training, we split the
Privacy-risk Saliency Map dataset into a training and validation sets of
approximately $45,000$ and $5,000$ samples, respectively. We perform model
selection based on the lowest loss on the validation set to avoid over-
fitting. For downstream tasks, we follow the training configurations provided
in [15].

Fig. 2: Proposed framework utility performance. Dashed lines represent the
“random perturbed positions” baselines. Fig. 3: Proposed framework privacy
performance. Dashed lines represent the “random perturbed positions”
baselines.

Evaluation. For each task, we train a single model on the extracted features
of wav2vec2 on the original utterances from the corresponding dataset, and use
the trained model for assessing the privacy-utility performances of both our
framework and the baselines. We report the performance of these models in the
Original row in Table 1. Assuming an Ignorant Attack model might underestimate
the risk of speaker-re-identification, however, we are unable to train
separate ASV models given the large number of $(k,\epsilon)$ combinations (25
in total) and computation constraints. Training a single ASV (or ASR) model
takes around 2 days on a Tesla V100 GPU. Moreover, although the model is
unaware of the added noises to the representation, it is also not aware that
the input speech are synthesized, making the comparisons fairer. We believe
that the EER evaluated by the Ignorant Attack model can still inform crucial
information about the relative privacy rankings of different methods. For
example, as shown in Table 1, Baseline 1 is more privacy-preserving than
Baseline 2, which is consistent with the findings in [23]. It is also
important to note that the Lazy-informed and Semi-informed attack models
provided in the VoicePrivacy2022 Challenge are not applicable to our work as
we are evaluating ASV at a feature-level, i.e., we do not produce speech
outputs.

##  4 Results and Discussion

| ASV | IC | ER | ASR
---|---|---|---|---
Original | 6.30 | 92.67 | 64.81 | 6.54
Baseline 1 | 31.63 | 86.97 | 37.73 | 10.70
Baseline 2 | 27.18 | 57.84 | 43.89 | 23.65
Baseline 3 | 21.60 | 82.78 | 48.49 | 8.52
Baseline 4 | 45.40 | 20.01 | 31.18 | 77.12
Ours ($k=20\%,\epsilon=4$) | 28.66 | 74.19 | 52.20 | 10.38
Ours ($k=20\%,\epsilon=1$) | 47.85 | 51.23 | 44.18 | 39.60
Table 1: Performance comparisons between the proposed method and the
baselines.

Table 1 shows the privacy-utility evaluation for the baseline models. Figure 2
and 3 show the evaluation results for the utility and privacy metrics,
respectively. To highlight our PSE’s ability to correctly identify positions
with sensitive information, we add another baseline that randomly chooses
$k\%$ of the positions in the extracted wav2vec2 representations for
perturbation with similar settings to our approach.

We can observe that the proposed approach enables a very flexible range of
privacy-utility trade-offs, with EER ranging from $0.13$ to $0.50$ while the
utility metrics ranging from random guessing performance to approximately the
original performance. On the higher EER range ($>40\%$), the approach
($k=20\%,\epsilon=1$) shows better utility performance (51.23%, 44.18%, and
39.60% in IC, ER, and ASR respectively) compared to Baseline 4. Compared to
Baseline 1 and Baseline 2 in the lower EER range (around 30%), our approach
achieves superior ER performance (52.20% vs. 43.89%), comparable ASR WER
performance (10.38% vs. 10.7%), and lower IC performance (74.19% vs. 86.97%)
with $k=20\%,\epsilon=4$. In general, feature-level anonymization can be a
competitive alternative to voice anonymization when voice outputs are not
required, e.g., supervised machine learning applications.

From Figure 3, we can see that our method achieves superior (higher) EER
curves compared to the random perturbation baselines in all $\epsilon$ and
perturbation ratio variations. It is also important to note that the gap
between our method and the random perturbation baselines gets smaller as the
proportion of perturbed positions $k$ gets larger, with the most significant
gap observed at $k=20\%$. These confirm our assumption that certain positions
within the extracted representations carry more sensitive information, and
show that our PSE is able to identify high privacy-risk positions for
perturbations. However, from Figure 2, we can see that the dashed curves
generally have better utility performances compared to the solid curves, with
the exception of Speech Recognition with a reversed effect. This implies that
the selected positions from PSE might potentially be also high-utility
positions for downstream tasks.

##  5 Conclusion

In this study, we propose a framework to perturb the extracted representations
from pre-trained speech models, with the goal of generating privacy-preserving
representation for data sharing. The framework consists of three stages:
generating a privacy-risk saliency map dataset for representations extracted
from a selected pre-trained model, training a Privacy-risk Saliency Estimator
(PSE), and perturbing the extracted representations based on the selected
positions from PSE. We validate our framework on a wide range of downstream
tasks and find that sharing speech data in the form of extracted
representations is competitive to the existing approaches that share speech
via voice conversion or voice modification. We hope the results motivate
further exploration in the direction of privacy-preserving pre-trained
representation learning.

## References

  * [1] Yiqiang Chen et al.,  “Fedhealth: A federated transfer learning framework for wearable healthcare,”  IEEE Intelligent Systems, vol. 35, no. 4, pp. 83–93, 2020.
  * [2] Ian McGraw et al.,  “Personalized speech recognition on mobile devices,”  in 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2016, pp. 5955–5959.
  * [3] Peter Wu et al.,  “Understanding the tradeoffs in client-side privacy for speech recognition,”  arXiv preprint arXiv:2101.08919, 2021.
  * [4] Stephen Edward McAdams,  Spectral fusion, spectral parsing and the formation of auditory images,  Stanford university, 1984.
  * [5] Hiroto Kai et al.,  “Lightweight voice anonymization based on data-driven optimization of cascaded voice modification modules,”  in 2021 IEEE Spoken Language Technology Workshop (SLT). IEEE, 2021, pp. 560–566.
  * [6] David Snyder et al.,  “X-vectors: Robust dnn embeddings for speaker recognition,”  in 2018 IEEE international conference on acoustics, speech and signal processing (ICASSP). IEEE, 2018, pp. 5329–5333.
  * [7] Brij Mohan Lal Srivastava et al.,  “Design choices for x-vector based speaker anonymization,”  in INTERSPEECH 2020, 2020.
  * [8] Gauri P Prajapati et al.,  “Voice privacy using cyclegan and time-scale modification,”  Computer Speech & Language, vol. 74, pp. 101353, 2022.
  * [9] Brij Mohan Lal Srivastava et al.,  “Privacy-preserving adversarial representation learning in asr: Reality or illusion?,”  in INTERSPEECH, 2019.
  * [10] Mimansa Jaiswal et al.,  “Privacy enhanced multimodal neural representations for emotion recognition,”  in Proceedings of the AAAI Conference on Artificial Intelligence, 2020, vol. 34, pp. 7985–7993.
  * [11] Daniel Smilkov et al.,  “Smoothgrad: removing noise by adding noise,”  arXiv preprint arXiv:1706.03825, 2017.
  * [12] Ashish Vaswani et al.,  “Attention is all you need,”  Advances in neural information processing systems, vol. 30, 2017\.
  * [13] Cynthia Dwork et al.,  “The algorithmic foundations of differential privacy.,”  Found. Trends Theor. Comput. Sci., vol. 9, no. 3-4, pp. 211–407, 2014.
  * [14] Alexei Baevski et al.,  “wav2vec 2.0: A framework for self-supervised learning of speech representations,”  Advances in Neural Information Processing Systems, vol. 33, pp. 12449–12460, 2020.
  * [15] Shu-wen Yang et al.,  “Superb: Speech processing universal performance benchmark,”  arXiv preprint arXiv:2105.01051, 2021.
  * [16] Arsha Nagrani et al.,  “Voxceleb: A large-scale speaker identification dataset,”  Proc. Interspeech 2017, pp. 2616–2620, 2017.
  * [17] Feng Wang et al.,  “Additive margin softmax for face verification,”  IEEE Signal Processing Letters, vol. 25, no. 7, pp. 926–930, 2018\.
  * [18] N. Tomashenko et al.,  “Introducing the VoicePrivacy Initiative,”  in Proc. Interspeech 2020, 2020, pp. 1693–1697.
  * [19] Carlos Busso et al.,  “Iemocap: Interactive emotional dyadic motion capture database,”  Language resources and evaluation, vol. 42, no. 4, pp. 335–359, 2008.
  * [20] Loren Lugosch et al.,  “Speech model pre-training for end-to-end spoken language understanding,”  Proc. Interspeech 2019, pp. 814–818, 2019.
  * [21] Vassil Panayotov et al.,  “Librispeech: an asr corpus based on public domain audio books,”  in 2015 IEEE international conference on acoustics, speech and signal processing (ICASSP). IEEE, 2015, pp. 5206–5210.
  * [22] Alex Graves et al.,  “Connectionist temporal classification: labelling unsegmented sequence data with recurrent neural networks,”  in Proceedings of the 23rd international conference on Machine learning, 2006, pp. 369–376.
  * [23] Natalia Tomashenko et al.,  “The voiceprivacy 2022 challenge evaluation plan,”  arXiv preprint arXiv:2203.12468, 2022.
  * [24] Jose Patino et al.,  “Speaker anonymisation using the mcadams coefficient,”  in Interspeech 2021. ISCA, 2021, pp. 1099–1103.
  * [25] Shinnosuke Takamichi et al.,  “The naist text-to-speech system for the blizzard challenge 2015,”  in Proc. Blizzard Challenge workshop. Berlin, Germany, 2015, vol. 2.