$\mathcal{GC}_{g}^{\dagger}(\mathbf{\Gamma})$. The key is the following
correspondence between the variables:

  1. 1)

For $\phi$\- and $\varphi$-functions,
$\phi_{kl}(W_{0}^{-1}UW_{0})=\varphi_{kl}(U)$ (see Remark 7.3);

  2. 2)

For $g_{ij}$ and $h_{ji}$ from the initial extended clusters of
$\mathcal{GC}_{g}^{\dagger}(\mathbf{\Gamma})$ and
$\mathcal{GC}_{h}^{\dagger}(\mathbf{\Gamma}^{\mathrm{op}})$,
$g_{ij}(U)=(-1)^{\varepsilon_{ji}}h_{ji}(U^{T})$ (see Proposition 7.5).

Both correspondences preserve the Poisson brackets (see Remark 7.3 and Lemma
3.14).

######  Proposition 7.10.

Let $\mathbf{\Gamma}:=(\Gamma_{1},\Gamma_{2},\gamma)$ be a BD triple of type
$A_{n-1}$ and $G\in\\{\operatorname{SL}_{n},\operatorname{GL}_{n}\\}$. Then
the following statements hold:

  1. 1)

The generalized cluster structure
$\mathcal{GC}_{g}^{\dagger}(\mathbf{\Gamma},G)$ is regular;

  2. 2)

All the variables in the initial extended cluster of
$\mathcal{GC}_{g}^{\dagger}(\mathbf{\Gamma},G)$ are irreducible (as elements
of $\mathcal{O}(G)$) and (for cluster variables) coprime with their mutations;

  3. 3)

The generalized cluster structure
$\mathcal{GC}_{g}^{\dagger}(\mathbf{\Gamma},G)$ is complete.

###### Proof.

Let $\Psi_{0}$ be the initial extended cluster of
$\mathcal{GC}_{g}^{\dagger}(\mathbf{\Gamma},G)$. In the case of
$\mathbf{\Gamma}=\mathbf{\Gamma}_{\text{std}}$, the proof can be executed in
the same way as in [20] for
$\mathcal{GC}_{h}^{\dagger}(\mathbf{\Gamma}_{\text{std}},G)$. There, the proof
was based on various normal forms of a generic element
$U\in\operatorname{GL}_{n}$. For instance, in the case of the initial extended
cluster of $\mathcal{GC}_{h}^{\dagger}(\mathbf{\Gamma}_{\text{std}},G)$, one
decomposes $U$ as $U=U_{\oplus}U_{-}$ and as
$U=N_{-}B_{+}C\,\left.N_{-}\right.^{-1}$ for some
$N_{-},U_{-}\in\mathcal{N}_{-}$, $U_{\oplus},B_{+}\in\mathcal{B}_{+}$ and
$C:=e_{1n}+\sum_{i=1}^{n-1}e_{i+1,i}$. Then $\varphi(U)=\varphi(B_{+})$ and
$h(U)=h(U_{\oplus})$ for $\varphi$\- and $h$-variables. In the case of
$\mathcal{GC}_{g}^{\dagger}(\mathbf{\Gamma}_{\text{std}},G)$, one decomposes
$U$ as $U=U_{+}U_{\ominus}$ and $U=N_{+}B_{-}C^{T}\,\left.N_{+}\right.^{-1}$,
where $N_{+},U_{+}\in\mathcal{N}_{+}$ and
$U_{\ominus},B_{-}\in\mathcal{B}_{-}$. Then, to express $u_{ij}$ in terms of
$\mathcal{L}_{\mathbb{C}}(\Psi_{0})$, it is enough to express the entries of
$B_{-}$ and $U_{\ominus}$ in terms of $\mathcal{L}_{\mathbb{C}}(\Psi_{0})$.
This process is described in [20]. For $|\Gamma_{1}|=1$, one checks the
conditions of Proposition 2.28 for the marked variable, which in turn can be
transferred from the case of $\mathcal{GC}_{h}^{\dagger}(\mathbf{\Gamma},G)$
via the transposition map. For $|\Gamma_{1}|>1$, the proof proceeds along the
same lines as in the case of $\mathcal{GC}_{h}^{\dagger}(\mathbf{\Gamma})$,
and it is based on the inductive argument on the size of $|\Gamma_{1}|$ and
Proposition 2.30, which in turn relies on the existence of birational quasi-
isomorphisms $\mathcal{G}^{\mathrm{op}}$ with distinct sets of marked
variables. ∎

Let us recall that the number $k_{\mathbf{\Gamma}}$ is given by
$k_{\mathbf{\Gamma}}=|\Pi\setminus\Gamma_{1}|$.

######  Proposition 7.11.

Let $\mathbf{\Gamma}$ be a BD triple of type $A_{n-1}$. The following
statements hold:

  1. 1)

The action of $\mathcal{H}_{\mathbf{\Gamma}}$ upon $\operatorname{SL}_{n}$
induces a global toric action upon
$\mathcal{GC}_{g}^{\dagger}(\mathbf{\Gamma},\operatorname{SL}_{n})$ of rank
$k_{\mathbf{\Gamma}}$;

  2. 2)

The action of $\mathcal{H}_{\mathbf{\Gamma}}\times\mathbb{C}^{*}$ upon
$\operatorname{GL}_{n}$ induces a global toric action upon
$\mathcal{GC}_{g}^{\dagger}(\mathbf{\Gamma},\operatorname{GL}_{n})$ of rank
$k_{\mathbf{\Gamma}}+1$.

###### Proof.

The fact that the action by scalars $\mathbb{C}^{*}$ induces a global toric
action upon
$\mathcal{GC}_{g}^{\dagger}(\mathbf{\Gamma},\operatorname{GL}_{n})$ can be
proved in the same way as in the case of
$\mathcal{GC}_{h}^{\dagger}(\mathbf{\Gamma},\operatorname{GL}_{n})$ (see Lemma
4.21). Due to Proposition 4.16 and the commutative diagram (3.55),
$\mathcal{Q}^{\mathrm{op}}:(G,\pi_{\text{std}}^{\dagger})\dashrightarrow(G,\pi_{\mathbf{\Gamma}}^{\dagger})$
is equivariant with respect to the action of $\mathcal{H}_{\mathbf{\Gamma}}$;
as a consequence, the $g$-functions are semi-invariant with respect to the
action of $\mathcal{H}_{\mathbf{\Gamma}}$. Furthermore, due to the semi-
invariance of the $\varphi$-functions and Remark 7.3, the $\phi$-functions are
also semi-invariant with respect to the action; hence,
$\mathcal{H}_{\mathbf{\Gamma}}$ induces a local toric action upon the initial
extended cluster $\Psi_{0}$ of $\mathcal{GC}_{g}^{\dagger}(\mathbf{\Gamma})$.
Due to Proposition 7.5, Remark 7.3 and Proposition 5.11, all the $y$-variables
in $\Psi_{0}$ are invariant with respect to $\mathcal{H}_{\mathbf{\Gamma}}$
except $y(\phi_{kl})$, $k+l=n$, which require a separate check. In turn, this
reduces to showing that $\pi_{0}([\nabla_{U}\log y(\phi_{kl}),U])=0$ for
$\mathbf{\Gamma}=\mathbf{\Gamma}_{\text{std}}$. Set
$\Delta(i,j):=\sum_{k=i}^{j}e_{kk}$. It is not difficult to see that

$\displaystyle\pi_{0}([\nabla_{U}\log\phi_{kl},U])=\Delta(1,l)-\Delta(k+1,n)+(n-k-l)\Delta(1,1);$
(7.28) $\displaystyle\pi_{0}([\nabla_{U}\log
g_{ij},U])=\Delta(j,n-i+j)-\Delta(i,n).$ (7.29)

Now, the neighborhoods of $\phi_{kl}$ for $k+l=n$ are given by Figure 15b,
Figure 16b and Figure 17b, and it is a matter of a direct computation that
$\pi_{0}([\nabla_{U}\log y(\phi_{kl}),U])=0$, $k+l=n$. The Casimirs
$\hat{p}_{ir}$ from the statement of Proposition 2.19 are the same as in
$\mathcal{GC}_{h}^{\dagger}(\mathbf{\Gamma})$, hence they are invariant with
respect to $\mathcal{H}_{\mathbf{\Gamma}}$. It follows from Proposition 2.19
that the local toric action of $\mathcal{H}_{\mathbf{\Gamma}}$ upon
$\mathcal{GC}_{g}^{\dagger}(\mathbf{\Gamma})$ is global. To verify that the
action is of rank $k_{\mathbf{\Gamma}}$, one applies the same argument as in
Proposition 5.11 based on Proposition 7.10. ∎

######  Lemma 7.12.

Let $g_{ij}$ be a $g$-function and $\phi_{kl}$ be a $\phi$-function from the
initial extended cluster of $\mathcal{GC}_{g}^{\dagger}(\mathbf{\Gamma})$.
Then

$\\{g_{ij},\phi_{kl}\\}_{\mathbf{\Gamma}}^{\dagger}=\langle
R_{0}\pi_{0}[\nabla_{U}g_{ij},U],[\nabla_{U}\phi_{kl},U]\rangle-\langle\pi_{0}(U\nabla_{U}g_{ij}),[\nabla_{U}\phi_{kl},U]\rangle.$
(7.30)

Moreover, the $g$-functions are log-canonical with the $\phi$-functions.

###### Proof.

Set
$\mathring{\nu}^{*}(g_{ij}):=U\nabla_{U}g_{ij}-\mathring{\gamma}^{*}(\nabla_{U}g_{ij}\cdot
U)$. Due to the invariance property (7.7),
$\mathring{\nu}^{*}(g_{ij})\in\mathfrak{b}_{+}$; therefore,

$-\frac{\mathring{\gamma}^{*}}{1-\mathring{\gamma}^{*}}\pi_{<}[\nabla_{U}g_{ij},U]=\pi_{<}(U\nabla_{U}g_{ij}).$
(7.31)

Together with $[\nabla_{U}\phi_{kl},U]\in\mathfrak{b}_{+}$, expanding the
bracket $\\{g_{ij},\phi_{kl}\\}_{\mathbf{\Gamma}}^{\dagger}$ yields formula
(7.30). Since the map
$\mathcal{Q}^{\mathrm{op}}:(\operatorname{GL}_{n},\pi_{\text{std}}^{\dagger})\dashrightarrow(\operatorname{GL}_{n},\pi_{\mathbf{\Gamma}}^{\dagger})$
is a Poisson isomorphism (see Proposition 3.15) and a birational quasi-
isomorphism (see Proposition 7.8), it is enough to verify the log-canonicity
for $\mathbf{\Gamma}=\mathbf{\Gamma}_{\text{std}}$, and that readily follows
from combining the formulas (7.28), (7.29) and (7.30) along with
$\pi_{0}(U\nabla_{U}\log g_{ij})=\Delta(i,n)$. ∎

######  Proposition 7.13.

Let $\mathbf{\Gamma}$ be a BD triple of type $A_{n-1}$. For any choice of
$r_{0}$ relative $\mathbf{\Gamma}$, the generalized cluster structure
$\mathcal{GC}_{g}^{\dagger}(\mathbf{\Gamma})$ is compatible with
$\pi_{\mathbf{\Gamma}}^{\dagger}$.

###### Proof.

In the initial extended cluster $\Psi_{0}$ of
$\mathcal{GC}_{g}^{\dagger}(\mathbf{\Gamma})$, the log-canonicity between the
$\phi$-functions is a consequence of Remark 7.3, the log-canonicity between
the $g$-functions is a consequence of Proposition 7.5 and Lemma 3.14, and the
log-canonicity between the $g$\- and the $\phi$-functions is a consequence of
Lemma 7.12. Based on Proposition 2.16, to verify the compatibility of
$\mathcal{GC}_{g}^{\dagger}(\mathbf{\Gamma})$ with
$\pi_{\mathbf{\Gamma}}^{\dagger}$, for any cluster variable
$\psi_{1}\in\Psi_{0}$ and any variable $\psi_{2}\in\Psi_{0}$, one verifies
that

$\\{\log y(\psi_{1}),\psi_{2}\\}=\begin{cases}1\ &\text{if}\
\psi_{1}=\psi_{2}\\\ 0\ &\text{otherwise.}\end{cases}$ (7.32)

Due to Remark 7.3, if $\psi_{1},\psi_{2}\in\\{\phi_{kl}\ |\ k+l<n\\}$, then
(7.32) holds, and due to Proposition 7.5 and Lemma 3.14, if
$\psi_{1},\psi_{2}\in\\{g_{ij}\ |\ 2\leq j\leq i\leq n\\}$, (7.32) holds as
well (note that this covers the case of the marked variables of
$\mathcal{Q}^{\mathrm{op}}$ as well). By Proposition 3.15, it remains to
verify (7.32) in the case $\mathbf{\Gamma}=\mathbf{\Gamma}_{\text{std}}$ and
1) when $\psi_{1}$ is a $\phi$-function and $\psi_{2}$ is a $g$-function; 2)
when $\psi_{1}$ is a $g$-function and $\psi_{2}$ is a $\phi$-function; 3) when
$\psi_{1}=\phi_{kl}$ for $k+l=n$ and $\psi_{2}\in\Psi_{0}$ is any variable.
This is a direct computation based on formulas (7.28), (7.29) and (7.30)
together with an analysis of the neighborhood of $\psi_{1}$ provided in
Appendix B. ∎

##  Appendix A Initial quiver for
$\mathcal{GC}_{h}^{\dagger}(\mathbf{\Gamma})$

In this appendix, we describe the initial quiver $Q_{h}(\mathbf{\Gamma})$ for
$\mathcal{GC}^{\dagger}_{h}(\mathbf{\Gamma})$. We start with describing the
quiver $Q_{h}(\mathbf{\Gamma}_{\text{std}})$ in Section A.1, which is
exemplified in Figure 7 for $n=5$. In Section A.2, we explain how to obtain
the quiver $Q_{h}(\mathbf{\Gamma})$ from $Q_{h}(\mathbf{\Gamma}_{\text{std}})$
for a nontrivial BD triple $\mathbf{\Gamma}$. The procedure of obtaining
$Q_{h}(\mathbf{\Gamma})$ from $Q_{h}(\mathbf{\Gamma}_{\text{std}})$ consists
in adding extra arrows to $Q_{h}(\mathbf{\Gamma}_{\text{std}})$ based on
$\mathbf{\Gamma}$, as well as in unfreezing some of the vertices.
Alternatively, in Section A.3 we provide explicit neighborhoods of the
vertices of $Q_{h}(\mathbf{\Gamma})$ that are different from the corresponding
neighborhoods in $Q_{h}(\mathbf{\Gamma}_{\text{std}})$. For explicit examples
of $Q_{h}(\mathbf{\Gamma})$ for nontrivial BD triples, see Appendix D and the
supplementary note [33]. Throughout the section, we assume that $n\geq 3$.

###  A.1 The quiver for the trivial BD triple

Below one can find pictures of the neighborhoods of all vertices of the
initial quiver of
$\mathcal{GC}_{h}^{\dagger}(\mathbf{\Gamma}_{\text{std}},\operatorname{GL}_{n})$.
A few remarks beforehand:

  * •

The circled vertices are mutable (in the sense of ordinary exchange relations
(2.23)), the square vertices are frozen, the rounded square vertices may or
may not be mutable depending on the indices, and the hexagon vertex is a
mutable vertex with a generalized mutation relation (see formula (1.20));

  * •

Since $c_{1},\ldots,c_{n-1}$ are isolated variables, they are not shown on the
resulting quiver;

  * •

For $k=2$ and $n>3$, the vertices $\varphi_{1k}$ and $\varphi_{k-1,2}$
coincide; hence, the pictures provided below suggest that there are two edges
pointing from $\varphi_{21}$ to $\varphi_{12}$ (however, there is only one
arrow in $n=3$);

  * •

When the indices of the $h$-variables are out of range, we use convention
(5.1).;

  * •

The initial quiver for
$\mathcal{GC}_{h}^{\dagger}(\mathbf{\Gamma}_{\text{std}},\operatorname{SL}_{n})$
is the same as for
$\mathcal{GC}_{h}^{\dagger}(\mathbf{\Gamma}_{\text{std}},\operatorname{GL}_{n})$
except the vertex with $h_{11}$ has to be removed.

Figure 1: The neighborhood of $\varphi_{kl}$ for $k,l\neq 1$, $k+l<n$.

a Case $2\leq l\leq n-2$.

b Case $l=n-1$.

Figure 2: The neighborhood of $\varphi_{1l}$ for $2\leq l\leq n-1$.

a Case $2\leq k\leq n-2$.

b Case $k=n-1$.

Figure 3: The neighborhood of $\varphi_{k1}$ for $2\leq k\leq n-1$.

a Case $k=l=1$.

b Case $k+l=n$.

Figure 4: The neighborhood of $\varphi_{kl}$ for (a) $k=l=1$ and (b) $k+l=n$.

a Case $j<n$.

b Case $j=n$.

Figure 5: The neighborhood of $h_{ij}$ for $2\leq i\leq n-1$, $2\leq j\leq n$,
$i\leq j$.

a Case $i=1$.

b Case $2\leq i\leq n-1$.

c Case $i=n$.

Figure 6: The neighborhood of $h_{ii}$ for $1\leq i\leq n$.

Figure 7: The initial quiver for
$\mathcal{GC}_{h}^{\dagger}(\mathbf{\Gamma}_{\text{std}},\operatorname{GL}_{5})$.

###  A.2 The quiver for a nontrivial BD triple (algorithm)

Let $\mathbf{\Gamma}:=(\Gamma_{1},\Gamma_{2},\gamma)$ be a nontrivial BD
triple. In order to obtain the quiver $Q_{h}(\mathbf{\Gamma})$, one proceeds
as follows. First, draw the quiver $Q_{h}(\mathbf{\Gamma}_{\text{std}})$, as
described in Appendix A.1. Second, add new arrows as prescribed by the
following algorithm:

  1. 1)

If $j\in\Gamma_{2}$ is such that $j-1\notin\Gamma_{2}$ or
$\gamma|_{\Delta(i)\cap\Gamma_{1}}<0$, add new arrows as indicated in Figure
8a;

  2. 2)

If $j\in\Gamma_{2}$ is such that $j-1\in\Gamma_{2}$ and
$\gamma|_{\Delta(i)\cap\Gamma_{1}}>0$, add new arrows as indicated in Figure
8b;

  3. 3)

Repeat for all roots in $\Gamma_{1}$.

In the algorithm, $\Delta(i)$ denotes the $X$-run that contains
$i:=\gamma^{*}(j)$ for $j\in\Gamma_{2}$.

a Case $j-1\notin\Gamma_{2}$ or $\gamma|_{\Delta(i)\cap\Gamma_{1}}<0$

b Case $j-1\in\Gamma_{2}$ and $\gamma|_{\Delta(i)\cap\Gamma_{1}}>0$

Figure 8: Additional arrows for $j\in\Gamma_{2}$ and $i:=\gamma^{*}(j)$.

###  A.3 The quiver for a nontrivial BD triple (explicit)

As an alternative to the algorithm described in the previous subsection, we
provide explicit neigh- borhoods of the variables $h_{jj}$ ($2\leq j\leq n$),
$h_{in}$ ($2\leq i\leq n$), $\varphi_{n-1,1}$ in the case of a nontrivial BD
triple. All the other neighborhoods of $Q_{h}(\mathbf{\Gamma})$ are the same
as in $Q_{h}(\mathbf{\Gamma}_{\text{std}})$. Let us mention that if $\Delta$
is a nontrivial $X$-run such that $|\Delta|=2$, then both conditions
$\gamma|_{\Delta\cap\Gamma_{1}}>0$ and $\gamma|_{\Delta\cap\Gamma_{2}}<0$ are
considered to be satisfied; however, if $|\Delta|>2$, then the conditions are
mutually exclusive.

a Case $j-1\notin\Gamma_{2}$.

b Case $j-2,j-1\in\Gamma_{2}$ and $\gamma|_{\Delta(i)\cap\Gamma_{1}}>0$.

c Case $j-1\in\Gamma_{2}$, $j\notin\Gamma_{2}$ and ($j-2\notin\Gamma_{2}$ or
$\gamma|_{\Delta(i)\cap\Gamma_{1}}<0$).

d Case $j-1,j\in\Gamma_{2}$ and $\gamma|_{\Delta(i)\cap\Gamma_{1}}<0$.

Figure 9: The neighborhood of $h_{jj}$ for $2\leq j\leq n-1$; here,
$i:=\gamma^{*}(j-1)+1$ when $j-1\in\Gamma_{2}$.

a Case $n-1\notin\Gamma_{1}$, $n-1\notin\Gamma_{2}$

b Case $n-1\in\Gamma_{1}$, $n-1\notin\Gamma_{2}$.

c Case $n-1\notin\Gamma_{1}$, $n-1,n-2\in\Gamma_{2}$ and
$\gamma|_{\Delta(i)\cap\Gamma_{1}>0}$.

d Case $n-1\in\Gamma_{1}$, $n-1,n-2\in\Gamma_{2}$ and
$\gamma|_{\Delta(i)\cap\Gamma_{1}>0}$.

e Case $n-1\notin\Gamma_{1}$, $n-1\in\Gamma_{2}$ and ($n-2\notin\Gamma_{2}$ or
$\gamma|_{\Delta(i)\cap\Gamma_{1}}<0$).

f Case $n-1\in\Gamma_{1}$, $n-1\in\Gamma_{2}$ and ($n-2\notin\Gamma_{2}$ or
$\gamma|_{\Delta(i)\cap\Gamma_{1}}<0$).

Figure 10: The neighborhood of $h_{nn}$. In the figures, we set
$j:=\gamma(n-1)+1$ if $n-1\in\Gamma_{1}$ and $i:=\gamma^{*}(n-1)+1$ if
$n-1\in\Gamma_{2}$.

a Case $i-1,i\notin\Gamma_{1}$.

b Case $i-1\notin\in\Gamma_{1}$, $i\in\Gamma_{1}$.

c Case $i-1\in\Gamma_{1}$, $i\notin\Gamma_{1}$.

d Case $i-1,i\in\Gamma_{1}$.

Figure 11: The neighborhood of $h_{in}$ for $2\leq i\leq n-1$; here,
$j:=\gamma(i-1)+1$ if $i-1\in\Gamma_{1}$ and $j^{\prime}:=\gamma(i)+1$ if
$i\in\Gamma_{1}$.

a Case $1\notin\Gamma_{1}$.

b Case $1\in\Gamma_{1}$.

Figure 12: The neighborhood of $\varphi_{n-1,1}$. Here,
$j^{\prime}:=\gamma(1)+1$ if $1\in\Gamma_{1}$.

a Case $1\notin\Gamma_{2}$

b Case $1\in\Gamma_{2}$

Figure 13: The neighborhood of $h_{11}$.

##  Appendix B Initial quiver for
$\mathcal{GC}_{g}^{\dagger}(\mathbf{\Gamma})$

In this appendix, we describe the initial quiver $Q_{g}(\mathbf{\Gamma})$ for
$\mathcal{GC}^{\dagger}_{g}(\mathbf{\Gamma})$. As in the $h$-convention, we
start with describing the quiver $Q_{g}(\mathbf{\Gamma}_{\text{std}})$ in
Section B.1, which is exemplified in Figure 20 for $n=5$. In Section B.2, we
explain how to obtain the quiver $Q_{g}(\mathbf{\Gamma})$ from
$Q_{g}(\mathbf{\Gamma}_{\text{std}})$ for a nontrivial BD triple
$\mathbf{\Gamma}$. The procedure of obtaining $Q_{g}(\mathbf{\Gamma})$ from
$Q_{g}(\mathbf{\Gamma}_{\text{std}})$ consists in adding extra arrows to
$Q_{g}(\mathbf{\Gamma}_{\text{std}})$ based on $\mathbf{\Gamma}$, as well as
in unfreezing some of the vertices. Alternatively, in Section B.3 we provide
explicit neighborhoods of the vertices of $Q_{g}(\mathbf{\Gamma})$ that are
different from the corresponding neighborhoods in
$Q_{g}(\mathbf{\Gamma}_{\text{std}})$. Some explicit examples of
$Q_{g}(\mathbf{\Gamma})$ for nontrivial BD triples are given in the
supplementary note [33]. Throughout the section, we assume that $n\geq 3$.

Let us mention that the quiver $Q_{g}(\mathbf{\Gamma})$ can be obtained from
$Q_{h}(\mathbf{\Gamma}^{\mathrm{op}})$ via the following steps:

  * •

Replace each vertex $\varphi_{kl}$ with $\phi_{kl}$, $2\leq k+l\leq n$,
$k,l\geq 1$ and each $h_{ji}$ with $g_{ij}$, $2\leq j\leq i\leq n$;

  * •

For each $g_{ij}$, $2\leq j\leq i\leq n$, reverse the orientation of the
arrows in its neighborhood;

  * •

For the vertices $\phi_{kl}$ with $k+l=n$ and $k\geq 2$, add an arrow
$\phi_{kl}\rightarrow\phi_{k-1,l+1}$;

  * •

Remove the arrow $\phi_{1,n-1}\rightarrow g_{11}$.

###  B.1 The quiver for the trivial BD triple

Below one can find pictures of the neighborhoods of all vertices of the
initial quiver $Q_{g}(\mathbf{\Gamma}_{\text{std}})$ of
$\mathcal{GC}_{g}^{\dagger}(\mathbf{\Gamma}_{\text{std}})$. A few remarks
beforehand:

  * •

The circled vertices are mutable (in the sense of ordinary exchange relations
(2.23)), the square vertices are frozen, the rounded square vertices may or
may not be mutable depending on the indices, and the hexagon vertex is a
mutable vertex with a generalized mutation relation (see Formula (1.20));

  * •

Since $c_{1},\ldots,c_{n-1}$ are isolated variables, they are not shown on the
resulting quiver;

  * •

Unlike in the $h$-convention, there are always two arrows
$\phi_{21}\rightrightarrows\phi_{12}$ in the resulting quiver;

  * •

When the indices of the $g$-variables are out of range, we use the following
conventions:

$\displaystyle g_{i1}(U)$ $\displaystyle:=\phi_{i-1,n-i+1}(U),$ $\displaystyle
i\in[2,n];$ (B.1) $\displaystyle g_{n+1,j}(U)$ $\displaystyle:=1,$
$\displaystyle j\in[1,n].$ (B.2)

Figure 14: The neighborhood of $\varphi_{kl}$ for $k,l\neq 1$, $k+l<n$.

a Case $2\leq l\leq n-2$.

b Case $l=n-1$.

Figure 15: The neighborhood of $\varphi_{1l}$ for $2\leq l\leq n-1$.

a Case $2\leq k\leq n-2$.

b Case $k=n-1$.

Figure 16: The neighborhood of $\varphi_{k1}$ for $2\leq k\leq n-1$.

a Case $k=l=1$.

b Case $k+l=n$.

Figure 17: The neighborhood of $\varphi_{kl}$ for (a) $k=l=1$ and (b) $k+l=n$.

a Case $j<n$.

b Case $j=n$.

Figure 18: The neighborhood of $g_{ij}$ for $2\leq i\leq n-1$, $2\leq j\leq
n$, $i\geq j$.

a Case $i=1$.

b Case $2\leq i\leq n-1$.

c Case $i=n$.

Figure 19: The neighborhood of $g_{ii}$ for $1\leq i\leq n$. Figure 20: The
initial quiver for
$\mathcal{GC}_{g}^{\dagger}(\mathbf{\Gamma}_{\text{std}},\operatorname{GL}_{5})$.

###  B.2 The quiver for a nontrivial BD triple (algorithm)

Let $\mathbf{\Gamma}:=(\Gamma_{1},\Gamma_{2},\gamma)$ be a nontrivial BD
triple. In order to obtain the quiver $Q_{g}(\mathbf{\Gamma})$, one proceeds
as follows. First, draw the quiver $Q_{g}(\mathbf{\Gamma}_{\text{std}})$, as
described in Appendix B.1. Second, add new arrows as prescribed by the
following algorithm:

  1. 1)

If $i\in\Gamma_{1}$ is such that $i-1\notin\Gamma_{1}$ or
$\gamma|_{\Delta(i)\cap\Gamma_{1}}<0$, add new arrows as indicated in Figure
21a;

  2. 2)

If $i\in\Gamma_{1}$ is such that $i-1\in\Gamma_{1}$ and
$\gamma|_{\Delta(i)\cap\Gamma_{1}}>0$, add new arrows as indicated in Figure
21b;

  3. 3)

Repeat for all roots in $\Gamma_{1}$.

In the algorithm, ${\Delta}(i)$ denotes the $X$-run that contains $i$.

a Case $i-1\notin\Gamma_{1}$ or $\gamma|_{\Delta(i)\cap\Gamma_{1}}<0$.

b Case $i-1\in\Gamma_{1}$ and $\gamma|_{\Delta(i)\cap\Gamma_{1}}>0$.

Figure 21: Additional arrows for $i\in\Gamma_{1}$ and $j:=\gamma(i)$.

###  B.3 The quiver for a nontrivial BD triple (explicit)

As an alternative to the algorithm described in the previous subsection, we
provide explicit neigh- borhoods of the variables $g_{jj}$ ($2\leq j\leq n$),
$g_{in}$ ($2\leq i\leq n$), $\phi_{n-1,1}$ in the case of a nontrivial BD
triple. All the other neighborhoods of $Q_{g}(\mathbf{\Gamma})$ are the same
as in $Q_{g}(\mathbf{\Gamma}_{\text{std}})$. Let us mention that if $\Delta$
is a nontrivial $X$-run such that $|\Delta|=2$, then both conditions
$\gamma|_{\Delta\cap\Gamma_{1}}>0$ and $\gamma|_{\Delta\cap\Gamma_{2}}<0$ are
considered to be satisfied; however, if $|\Delta|>2$, then the conditions are
mutually exclusive.

a Case $i-1\notin\Gamma_{1}$.

b Case $i-2,i-1\in\Gamma_{1}$ and $\gamma|_{\Delta(i)\cap\Gamma_{1}}>0$.

c Case $i-1\in\Gamma_{1}$, $i\notin\Gamma_{1}$ and ($i-2\notin\Gamma_{1}$ or
$\gamma|_{\Delta(i)\cap\Gamma_{1}}<0$).

d Case $i-1,i\in\Gamma_{1}$ and $\gamma|_{\Delta(i)\cap\Gamma_{1}}<0$.

Figure 22: The neighborhood of $g_{ii}$ for $2\leq i\leq n-1$; here,
$j:=\gamma(j-1)+1$ when $i-1\in\Gamma_{1}$.

a Case $n-1\notin\Gamma_{2}$, $n-1\notin\Gamma_{1}$.

b Case $n-1\in\Gamma_{2}$, $n-1\notin\Gamma_{1}$.

c Case $n-1\notin\Gamma_{2}$, $n-1,n-2\in\Gamma_{1}$ and
$\gamma|_{\Delta(n-1)\cap\Gamma_{2}>0}$.

d Case $n-1\in\Gamma_{2}$, $n-1,n-2\in\Gamma_{1}$ and
$\gamma|_{\Delta(n-1)\cap\Gamma_{2}>0}$.

e Case $n-1\notin\Gamma_{2}$, $n-1\in\Gamma_{1}$ and ($n-2\notin\Gamma_{1}$ or
$\gamma|_{\Delta(n-1)\cap\Gamma_{2}}<0$).

f Case $n-1\in\Gamma_{2}$, $n-1\in\Gamma_{1}$ and ($n-2\notin\Gamma_{1}$ or
$\gamma|_{\Delta(n-1)\cap\Gamma_{2}}<0$).

Figure 23: The neighborhood of $g_{nn}$. In the figures, we set
$j:=\gamma(n-1)+1$ if $n-1\in\Gamma_{1}$ and $i:=\gamma^{*}(n-1)+1$ if
$n-1\in\Gamma_{2}$.

a Case $j-1,j\notin\Gamma_{2}$.

b Case $j-1\notin\in\Gamma_{2}$, $j\in\Gamma_{2}$.

c Case $j-1\in\Gamma_{2}$, $j\notin\Gamma_{2}$.

d Case $j-1,j\in\Gamma_{2}$.

Figure 24: The neighborhood of $g_{nj}$ for $2\leq j\leq n-1$; here,
$i:=\gamma^{*}(j-1)+1$ if $j-1\in\Gamma_{2}$ and $i^{\prime}:=\gamma^{*}(j)+1$
if $j\in\Gamma_{2}$.

a Case $1\notin\Gamma_{2}$.

b Case $1\in\Gamma_{2}$.

Figure 25: The neighborhood of $\varphi_{n-1,1}$. Here,
$i^{\prime}:=\gamma^{*}(1)+1$ if $1\in\Gamma_{2}$.

a Case $1\notin\Gamma_{1}$.

b Case $1\in\Gamma_{1}$.

Figure 26: The neighborhood of $g_{11}$.

##  Appendix C Determinantal identities

In this appendix, we list various determinantal identities that are used
throughout the text.

##### Gauss decomposition.

Let $U\in\operatorname{GL}_{n}$ be a generic element. Decompose $U$ as
$U=U_{\oplus}U_{-}$ where $U_{\oplus}$ is an upper triangular and $U_{-}$ is a
unipotent lower triangular matrix. For $1\leq j\leq i\leq n$, the entries of
$U_{\oplus}$, $U_{-}$ and their inverses are given by the well-known formulas
(see, e.g. [14]).

ji $\displaystyle=\frac{\det U^{[i,n]}_{\\{j\\}\cup[i+1,n]}}{\det
U^{[i+1,n]}_{[i+1,n]}},$ $\displaystyle[U_{-}]_{ij}$ $\displaystyle=\frac{\det
U^{\\{j\\}\cup[i+1,n]}_{[i,n]}}{\det U^{[i,n]}_{[i,n]}},$ (C.1)
$\displaystyle[\,\left.U_{\oplus}\right.^{-1}]_{ji}$
$\displaystyle=(-1)^{j+i}\frac{\det U^{[j+1,n]}_{[j,n]\setminus\\{i\\}}}{\det
U^{[j,n]}_{[j,n]}},$ $\displaystyle[\,\left.U_{-}\right.^{-1}]_{ij}$
$\displaystyle=(-1)^{i+j}\frac{\det U^{[j,n]\setminus\\{i\\}}_{[j+1,n]}}{\det
U^{[j+1,n]}_{[j+1,n]}}.$

The next two identities are needed in the derivation of formulas for minors of
$\tilde{\gamma}^{*}(U_{-})$. They are used in Section 4.5 for deriving a more
explicit formula for birational quasi-isomorphisms, as well as in Sections
6.1-6.2 for deriving explicit formulas for the $h$-variables.

######  Lemma C.1.

Let $U\in\operatorname{GL}_{n}$ be a generic element, $[a,b]\subseteq[1,n]$,
$J\subseteq[1,n]$ be nontrivial subintervals such that $\max J\leq b$ and
$|J|=|[a,b]|$. Then

$\det[U_{-}]_{[a,b]}^{J}=\frac{\det U^{J\cup[b+1,n]}_{[a,n]}}{\det
U^{[a,n]}_{[a,n]}}.$ (C.2)

###### Proof.

The identity follows from an application of the Cauchy-Binet formula to
$U_{\oplus}U_{-}$. ∎

######  Lemma C.2.

Let us set $\Delta:=[p+1,p+k]\subseteq[1,n]$, $I:=[a,b]\subseteq\Delta$ with
$b:=p+k$, $J\subseteq\Delta$ a subset such that $|J|=|[a,b]|$. Let
$\hat{I}:=[\hat{a},\hat{b}]$ and $\hat{J}$ be the subintervals of $[1,k]$
obtained as images of $I$ and $J$ under the unique increasing bijection
$\Delta\rightarrow[1,k]$. Set $w_{0}:\Delta\rightarrow\Delta$ and
$\hat{w}_{0}:[1,k]\rightarrow[1,k]$ to be the unique decreasing bijections,
and let $W_{0}:=\sum_{i=1}^{k}(-1)^{i-1}e_{k-i+1,i}$. For a generic
$U\in\operatorname{GL}_{n}$, the following identity holds:

$\det\left[W_{0}[(U_{-})^{-T}]_{\Delta}^{\Delta}W_{0}^{-1}\right]_{\hat{I}}^{\hat{J}}=\frac{\det
U^{w_{0}(\Delta\setminus J)\cup[b+1,n]}_{[w_{0}(a)+1,n]}}{\det
U^{[w_{0}(a)+1,n]}_{[w_{0}(a)+1,n]}}.$ (C.3)

###### Proof.

It follows from the Cauchy-Binet formula that

$\begin{split}\det[W_{0}&[(U_{-})^{-T}]_{\Delta}^{\Delta}W_{0}^{-1}]_{\hat{I}}^{\hat{J}}=(-1)^{\varepsilon}\det\left[[U^{T}_{\oplus}U^{-T}]_{\Delta}^{\Delta}\right]_{[1,\hat{w_{0}}(\hat{a})]}^{\hat{w_{0}}(\hat{J})}=(-1)^{\varepsilon}\det[U^{T}_{\oplus}U^{-T}]_{[p+1,w_{0}(a)]}^{w_{0}(J)}=\\\
=&(-1)^{\varepsilon}\det[U^{T}_{\oplus}U^{-T}]_{[1,w_{0}(a)]}^{[1,p]\cup
w_{0}(J)}=(-1)^{\varepsilon}\frac{\det U}{\det
U^{[w_{0}(a)+1,n]}_{[w_{0}(a)+1,n]}}\det[U^{-T}]_{[1,w_{0}(a)]}^{[1,p]\cup
w_{0}(J)}=\\\ =&(-1)^{\varepsilon+\varepsilon^{\prime}}\frac{\det
U^{w_{0}(\Delta\setminus J)\cup[b+1,n]}_{[w_{0}(a)+1,n]}}{\det
U^{[w_{0}(a)+1,n]}_{[w_{0}(a)+1,n]}}.\end{split}$

where we introduced

$\begin{array}[]{l}\varepsilon:=\sum_{j\in\hat{w}_{0}(\hat{J})}(j-1)+\sum_{j\in\hat{w}_{0}(\hat{I})}(j-1),\\\
\varepsilon^{\prime}:=\sum_{i=w_{0}(a)+1}^{n}i+\sum_{i\in
w_{0}(\Delta\setminus J)}i+\sum_{i=p+k+1}^{n}i.\end{array}$

A simple computation shows that $\varepsilon+\varepsilon^{\prime}\equiv 0\mod
2$. Thus the formula holds. ∎

##### Desnanot-Jacobi identity.

The following identity is used in Section 5.1.

######  Proposition C.3.

Let $A$ be an $m\times m$ matrix with entries in an arbitrary field. If $1\leq
i<j\leq m$ and $1\leq k<l\leq m$, then the following identity holds:

$\det A\det A^{\hat{i}\hat{j}}_{\hat{k}\hat{l}}+\det A_{\hat{l}}^{\hat{i}}\det
A_{\hat{k}}^{\hat{j}}=\det A^{\hat{i}}_{\hat{k}}\det A_{\hat{l}}^{\hat{j}},$

where the hatted upper (lower) indices indicate that the corresponding column
(row) is removed.

##### Derivatives of flag minors.

For $1\leq i\leq j\leq n$, let us set

$\hat{h}_{ij}(U):=\det U_{[i,n-j+i]}^{[j,n]},\ \ U\in\operatorname{GL}_{n}.$
(C.4)

In our conventions, $\nabla_{U}^{R}f=U\cdot\nabla_{U}f$ and
$\nabla_{U}^{L}f=\nabla_{U}f\cdot U$, $U\in\operatorname{GL}_{n}$, where
$\nabla_{U}f$ is given by formula (1.25). The following identities are used in
Sections 5.2-5.3:

######  Lemma C.4.

The following formula holds:

$\langle\pi_{>}\nabla_{U}^{R}\log\hat{h}_{ij},\nabla_{U}^{R}\log\hat{h}_{ks}\rangle=\max\\{\min\\{i-k,n-s+1\\}-\max\\{j-s,0\\},0\\}.$
(C.5)

###### Proof.

It is a matter of a direct computation that

$(\nabla^{R}_{U}\hat{h}_{ij})_{qr}=\begin{cases}(-1)^{r-i}\det
U^{[j,n]}_{\\{q\\}\cup([i,n-j+i]\setminus\\{r\\})}&\text{if}\ q<i\ \text{and}\
r\in[i,n-j+i];\\\ (-1)^{n-j+i-r}\det
U^{[j,n]}_{([i,n-j+i]\setminus\\{r\\})\cup\\{q\\}}&\text{if}\ q>n-j+i\
\text{and}\ r\in[i,n-j+i];\\\ \hat{h}_{ij}\ &\text{if}\ q=r\ \text{and}\
r\in[i,n-j+i];\\\ 0\ &\text{if}\ r\notin[i,n-j+i]\ \text{or}\
q\in[i,n-j+i]\setminus\\{r\\}.\end{cases}$

We see that

$\begin{split}\langle\pi_{>}\nabla_{U}^{R}\hat{h}_{ij},&\nabla^{R}_{U}\hat{h}_{ks}\rangle=\\\
=&\sum_{\begin{subarray}{c}(q,r):\,q\in[k,\min\\{i-1,n-s+k\\}]\\\
r\in[\max\\{i,n-s+k+1\\},n-j+i]\end{subarray}}(-1)^{n-s+k-q+r-i}\det
U^{[j,n]}_{\\{q\\}\cup([i,n-j+i]\setminus\\{r\\})}\det
U^{[s,n]}_{([k,n-s+k]\setminus\\{q\\})\cup\\{r\\}}.\end{split}$

Now, the above sum simplifies via a long Plücker relation and is proportional
to the product $\hat{h}_{ij}\hat{h}_{ks}$. A case-by-case analysis shows that
the proportionality constant is given by (C.5). ∎

######  Lemma C.5.

The following identities hold:

$\pi_{0}\left(\nabla_{U}^{R}\log\hat{h}_{ij}\right)=\sum_{k=i}^{n-j+i}e_{kk},\
\ \pi_{0}\left(\nabla_{U}^{L}\log\hat{h}_{ij}\right)=\sum_{k=j}^{n}e_{kk}.$
(C.6)

###### Proof.

The identities follow from the fact that, given
$T=\operatorname{diag}(t_{1},\ldots,t_{n})$,

$\hat{h}_{ij}(TU)=\hat{h}_{ij}(U)\prod_{k=i}^{n-j+i}t_{k},\ \
\hat{h}_{ij}(UT)=\hat{h}_{ij}(U)\prod_{k=j}^{n}t_{k}.\qed$

##  Appendix D Selected examples

In this appendix, we provide explicit examples of the initial extended seeds
of $\mathcal{GC}_{h}^{\dagger}(\mathbf{\Gamma},\operatorname{GL}_{n})$. More
examples (including the $g$-convention) can be found in the supplementary note
[33].

###  D.1 Cremmer-Gervais in $n=3$, $\gamma:1\mapsto 2$

The initial quiver is illustrated in Figure 27.

Figure 27: The initial quiver for
$\mathcal{GC}_{h}^{\dagger}(\mathbf{\Gamma},\operatorname{GL}_{3})$ with
$\gamma:1\mapsto 2$.

The variables in the initial extended cluster are given as follows:

$c_{1}(U)=\operatorname{tr}(U),\ \
c_{2}(U)=\frac{1}{2!}(\operatorname{tr}(U)^{2}-\operatorname{tr}(U^{2}));$
(D.1) $\varphi_{21}(U)=u_{13},\ \ \varphi_{12}(U)=\det U^{[2,3]}_{[1,2]},\ \
\varphi_{11}(U)=u_{23}\det U^{[2,3]}_{[1,2]}+u_{13}\det
U^{\\{1,3\\}}_{[1,2]};$ (D.2) $h_{23}(U)=-u_{23}u_{33}-u_{13}u_{32},\ \
h_{22}(U)=u_{33}\det U^{[2,3]}_{[2,3]}+u_{32}\det U^{[2,3]}_{\\{1,3\\}};$
(D.3) $h_{11}(U)=\det U,\ \ h_{33}(U)=u_{33}.$ (D.4)

###  D.2 Cremmer-Gervais in $n=4$, $\gamma:i\mapsto i-1$

The initial quiver is illustrated in Figure 28. In the initial extended
cluster, all cluster and frozen variables are given as in
$\mathcal{GC}_{h}^{\dagger}(\mathbf{\Gamma}_{\text{std}},\operatorname{GL}_{4})$
except for the variables $h_{34}$, $h_{33}$, $h_{44}$. These are given by:

$\displaystyle h_{34}(U)=-u_{34}\det U^{[2,4]}_{[2,4]}-u_{24}\det
U^{\\{1\\}\cup[3,4]}_{[2,4]};$ (D.5) $\displaystyle h_{33}(U)=\det
U^{[3,4]}_{[3,4]}\det U^{[2,4]}_{[2,4]}+\det U^{[3,4]}_{\\{2,4\\}}\det
U^{\\{1\\}\cup[3,4]}_{[2,4]}+\det U^{[3,4]}_{[2,3]}\det
U^{[1,2]\cup\\{4\\}}_{[2,4]};$ (D.6) $\begin{split}h_{44}(U)=&u_{44}\left(\det
U^{[3,4]}_{[3,4]}\det U^{[2,4]}_{[2,4]}+\det U^{[3,4]}_{\\{2,4\\}}\det
U^{\\{1\\}\cup[3,4]}_{[2,4]}+\det U^{[3,4]}_{[2,3]}\det
U^{[1,2]\cup\\{4\\}}_{[2,4]}\right)+\\\ +&u_{34}\left(\det
U^{\\{2,4\\}}_{[3,4]}\det U^{[2,4]}_{[2,4]}+\det U^{\\{2,4\\}}_{\\{2,4\\}}\det
U^{\\{1\\}\cup[3,4]}_{[2,4]}+\det U^{\\{2,4\\}}_{[2,3]}\det
U^{[1,2]\cup\\{4\\}}_{[2,4]}\right)+\\\ +&u_{24}\left(\det
U^{\\{1,4\\}}_{[3,4]}\det U^{[2,4]}_{[2,4]}+\det U^{\\{1,4\\}}_{\\{2,4\\}}\det
U^{\\{1\\}\cup[3,4]}_{[2,4]}+\det U^{\\{1,4\\}}_{[2,3]}\det
U^{[1,2]\cup\\{4\\}}_{[2,4]}\right).\end{split}$ (D.7)

Figure 28: The initial quiver for
$\mathcal{GC}_{h}^{\dagger}(\mathbf{\Gamma},\operatorname{GL}_{4})$,
$\gamma:i\mapsto i-1$, $i\in\\{2,3\\}$.

###  D.3 Negative orientation in $n=5$, $\gamma:1\mapsto 4$, $\gamma:2\mapsto
3$

The initial quiver is illustrated in Figure 29. In the initial extended
cluster, all cluster and frozen variables are given as in
$\mathcal{GC}_{h}^{\dagger}(\mathbf{\Gamma}_{\text{std}},\operatorname{GL}_{5})$
except for the variables $h_{25}$, $h_{24}$, $h_{23}$, $h_{35}$, $h_{34}$,
$h_{33}$. These variables are given by the following formulas:

$\displaystyle h_{25}(U)=-u_{25}u_{55}-u_{15}u_{54};$ (D.8) $\displaystyle
h_{24}(U)=\det U^{[4,5]}_{[2,3]}u_{55}+\det U^{[4,5]}_{\\{1,3\\}}u_{54}+\det
U^{[4,5]}_{[1,2]}u_{53};$ (D.9) $\displaystyle h_{23}(U)=-\det
U^{[3,5]}_{[2,4]}u_{55}-\det U^{[3,5]}_{\\{1\\}\cup[3,4]}u_{54}-\det
U^{[3,5]}_{[1,2]\cup\\{4\\}}u_{53};$ (D.10) $\displaystyle h_{22}(U)=\det
U^{[2,5]}_{[2,5]}u_{55}+\det U^{[2,5]}_{\\{1\\}\cup[3,5]}u_{54}+\det
U^{[2,5]}_{[1,2]\cup[4,5]}u_{53};$ (D.11) $\displaystyle h_{35}(U)=u_{35}\det
U^{[4,5]}_{[4,5]}+u_{25}\det U^{\\{3,5\\}}_{[4,5]}+u_{15}\det
U^{[3,4]}_{[4,5]};$ (D.12) $\displaystyle h_{34}(U)=\det U^{[4,5]}_{[3,4]}\det
U^{[4,5]}_{[4,5]}+\det U^{[4,5]}_{\\{2,4\\}}\det U^{\\{3,5\\}}_{[4,5]}+\det
U^{[4,5]}_{\\{1,4\\}}\det U^{[3,4]}_{[4,5]};$ (D.13) $\displaystyle
h_{33}(U)=\det U^{[3,5]}_{[3,5]}\det U^{[4,5]}_{[4,5]}+\det
U^{[3,5]}_{\\{2\\}\cup[4,5]}\det U^{\\{3,5\\}}_{[4,5]}+\det
U^{[3,5]}_{\\{1\\}\cup[4,5]}\det U^{[3,4]}_{[4,5]}.$ (D.14)

Figure 29: The initial quiver for
$\mathcal{GC}_{h}^{\dagger}(\mathbf{\Gamma},\operatorname{GL}_{5})$ with
$\gamma:1\mapsto 4$, $\gamma:2\mapsto 3$.

## References

  * [1] A. Belavin and V. Drinfeld, ’Solutions of the classical Yang-Baxter equation for simple Lie algbras’, _Funktsional. Anal. i Prilozhen_ 16(1982), 159–180. doi:10.1007/BF01081585
  * [2] A. Belavin and V. Drinfeld, _Triangle Equations and Simple Lie Algebras_ (Hardwood Academic, 1998).
  * [3] A. Berenstein, S. Fomin and A. Zelevinsky, ’Cluster algebras III: Upper bounds and double Bruhat cells’, _Duke Math. J._ 126(1) (2005), 1–52. doi:10.1215/S0012-7094-04-12611-9
  * [4] R. Brahami, ’Cluster $\mathcal{X}$-varieties for dual Poisson-Lie groups, I’, _Algebra i Analiz_ 22(2) (2010), 14–104. doi:10.1090/S1061-0022-2011-01138-0
  * [5] V. Chari and A. Pressley, _A Guide to Quantum Groups_ (Cambridge Univ. Press, 1995).
  * [6] L. Chekhov and M. Shapiro, ’Teichmüller spaces of Riemann surfaces with orbifold points of arbitrary order and cluster variables’ _Int. Math. Res. Not. IMRN_ 10 (2014), 2746–2772.
doi:10.1093/imrn/rnt016

  * [7] P. Delorme, ’Classification des triples de Manin pour les algebres de Lie reductives complexes: Avec un appendice de Guillaume Macey’, _J. Algebra_ 246(1) (2001), 97–174.
doi:10.1006/jabr.2001.8887

  * [8] P. Etingof and O. Schiffmann, _Lectures on Quantum Groups_ (International Press, 1998).
  * [9] P. Etingof, T. Schedler, O. Schiffmann, ’Explicit quantization of dynamical r-matrices for finite dimensional semisimple Lie algebras’, _J. Amer. Math. Soc._ 13(3) (2000), 595–609.
doi:10.1090/S0894-0347-00-00333-7

  * [10] V. Fock and A. Goncharov, ’Moduli spaces of local systems and higher Teichmüller theory’, _Publ. Math. Inst. Hautes Études Sci._ 103(2006), 1–211.
  * [11] C. Fraser, ’Quasi-Homomorphisms of Cluster Algebras’, _Adv. in Appl. Math._ 81(2016), 40–77.
doi:10.1016/j.aam.2016.06.005

  * [12] S. Fomin, L. Williams and A. Zelevinsky, ’Introduction to Cluster algebras. Chapter 6’, Preprint, 2020. arXiv:2008.09189.
  * [13] S. Fomin and A. Zelevinsky, ’Cluster algebras I: Foundations’, _J. Amer. Math. Soc._ 15(2) (2002), 497–529. doi:10.1090/S0894-0347-01-00385-X
  * [14] F. Gantmacher, _The theory of matrices, vol. 1_ (Amer. Math. Soc., Providence, RI, 1998)
  * [15] M. Gekhtman, M. Shapiro and A. Vainshtein, ’A unified approach to exotic cluster structures on simple Lie groups’, Preprint, 2023. arXiv:2308.16701.
  * [16] M. Gekhtman, M. Shapiro and A. Vainshtein, ’Cluster algebras and Poisson geometry’, _Math. Surveys Monogr._ 167(2010). doi:10.1090/surv/167
  * [17] M. Gekhtman, M. Shapiro and A. Vainshtein, ’Cluster algebras and Poisson geometry’, _Mosc. Math. J._ 3(3) (2003), 899-934. doi:10.17323/1609-4514-2003-3-3-899-934; arXiv:0208033.
  * [18] M. Gekhtman, M. Shapiro and A. Vainshtein, ’Cluster structures on simple complex Lie groups and Belavin-Drinfeld classification’, _Mosc. Math. J._ 12(2) (2010), 293–312.
  * [19] M. Gekhtman, M. Shapiro, and A Vainshtein, ’Exotic cluster structures on $\operatorname{SL}_{n}$: the Cremmer-Gervais case’, _Mem. Amer. Math. Soc._ 246(1165) (2017), 1–94. doi:10.1090/memo/1165
  * [20] M. Gekhtman, M. Shapiro and A. Vainshtein, ’Drinfeld double of $\operatorname{GL}_{n}$ and generalized cluster structures’, _Proc. Lond. Math. Soc._ 116(3) (2018), 429–484. doi:10.1112/plms.12086
  * [21] M. Gekhtman, M. Shapiro and A. Vainshtein, ’Generalized cluster structures related to the Drinfeld double of $\operatorname{GL}_{n}$’, _J. Lond. Math. Soc._ 105(3), (2022), 1601–1633.
doi:10.1112/jlms.12542

  * [22] M. Gekhtman, M. Shapiro and A. Vainshtein, ’Periodic staircase matrices and generalized cluster structures’, _Int. Math. Res. Not. IMRN_ 2022(6) (2022), 4181–4221.
doi:10.1093/imrn/rnaa148

  * [23] M. Gekhtman, M. Shapiro and A. Vainshtein, ’Plethora of cluster structures on $\operatorname{GL}_{n}$’, Preprint, 2019. arXiv:1902.02902.
  * [24] M. Gekhtman, M. Shapiro and A. Vainshtein, ’Poisson geometry of directed networks in a disk’, _Selecta Math. (N.S.)_ 15 (2009), 61–103. doi:10.1007/s00029-009-0523-z
  * [25] K. R. Goodearl, M. T. Yakimov, ’Cluster algebra structures on Poisson nilpotent algebras’, Preprint, 2018. doi:10.48550/arXiv.1801.01963
  * [26] I. Ip, ’Cluster realization of $U_{q}(\mathfrak{g})$ and factorizations of the universal $R$-matrix, _Selecta Math._ 24(5) (2018), 4461–4553. doi:10.1007/s00029-018-0432-0
  * [27] T. J. Hodges, ’On the Cremmer-Gervais quantizations of $\operatorname{SL}(n)$’, _Int. Math. Res. Not. IMRN_ 10 (1995), 465–481. doi:10.1155/S107379289500033X
  * [28] B. Leclerc, ’Cluster structures on strata of flag varieties’, _Adv. Math._ 300(10) (2016), 190–228.
doi:10.1016/j.aim.2016.03.018

  * [29] B. Nguyen, K. Trampel and M. Yakimov, ’Noncommutative discriminants via Poisson primes’, _Adv. Math._ 322(2017), 269–307. doi:10.1016/j.aim.2017.10.018
  * [30] A. Reyman and M. Semenov-Tian-Shansky, _Integrable systems._ (Institute of Computer Studies, Moscow, 2003).
  * [31] D. Voloshyn, ’Multiple generalized cluster structures on $D(\mathrm{GL}_{n})$’, _Forum of Mathematics, Sigma_ 11(46) (2023), 1–78. doi:10.1017/fms.2023.44
  * [32] D. Voloshyn, ’Starfish lemma via birational quasi-isomorphisms’, Preprint, 2023.
arXiv:2311.00404.

  * [33] D. Voloshyn and M. Gekhtman, ’Supplementary note for generalized cluster structures on $\operatorname{SL}_{n}^{\dagger}$’, GitHub repository, 2023. github.com/Grabovskii/GSV_Examples
  * [34] G. Schrader and A. Shapiro, ’A cluster realization of $U_{q}(\operatorname{sl}_{n})$ from quantum character varieties’, _Invent. Math_ 216(2019), 799–846. doi:10.1007/s00222-019-00857-6
  * [35] G. Schrader and A. Shapiro, ’$K$-theoretic Coulomb branches of quiver gauge theories and cluster varieties’, Preprint, 2019. arXiv:1910.03186.
  * [36] J. Scott, ’Grassmannians and cluster algebras’, _Proc. Lond. Math. Soc._ 92 (2006), 345–380.
doi:10.1112/S0024611505015571

  * [37] L. Shen and D. Weng, ’Cluster structures on double Bott-Samelson cells’, _Forum of Mathematics, Sigma_ 9(66) (2021), 1-89. doi:10.1017/fms.2021.59
  * [38] L. Shen, ’Duals of semisimple Poisson-Lie groups and cluster theory of moduli spaces of $G$-local systems’, _Int. Math. Res. Not. IMRN_ 2022(18), 14295–14318. doi:10.1093/imrn/rnab094


# Unsupervised Source Separation By Steering Pretrained Music Models

###### Abstract

We showcase an unsupervised method that repurposes deep models trained for
music generation and music tagging for audio source separation, without any
retraining. An audio generation model is conditioned on an input mixture,
producing a latent encoding of the audio used to generate audio. This
generated audio is fed to a pretrained music tagger that creates source
labels. The cross-entropy loss between the tag distribution for the generated
audio and a predefined distribution for an isolated source is used to guide
gradient ascent in the (unchanging) latent space of the generative model. This
system does not update the weights of the generative model or the tagger, and
only relies on moving through the generative model’s latent space to produce
separated sources. We use OpenAI’s Jukebox as the pretrained generative model,
and we couple it with four kinds of pretrained music taggers (two
architectures and two tagging datasets). Experimental results on two source
separation datasets, show this approach can produce separation estimates for a
wider variety of sources than any tested supervised or unsupervised system.
This work points to the vast and heretofore untapped potential of large
pretrained music models for audio-to-audio tasks like source separation.

Index Terms—  music source separation, generative music models, automatic
music tagging, gradient ascent

##  1 Introduction

The research area of Music Information Retrieval (MIR) is constrained by a
lack of labeled data sets, which limits our ability to train robust systems
and evaluate them well. Specifically, the task of musical source separation
has been hindered by a dearth of well-labeled data [1]. This leads to severe
shortcoming in terms of the range of instrument source classes that current
systems can separate. Many systems, in fact, only separate the four classes
(voice, bass, drums and “other”) in the widely-used MUSDB18 [2] dataset,
making them unsuitable for separating most musical instruments.

Simultaneously, the recent availability of large pretrained models has
revolutionized generative and discriminative tasks in the domains of computer
vision and natural language processing. The combination of VQGAN [3] and CLIP
[4] has captured the attention of many artists, who have been captivated by
the system’s ability to use natural language to create generative art.
Similarly, researchers have shown how to steer large pretrained language
models for downstream discriminative tasks either using transfer learning [5]
or so-called few-shot “prompt engineering” [6]. Recent work has taken this
ethos to the MIR domain, leveraging the representations learned by the large
training regime of an unsupervised generative music model for downstream MIR
tasks, like key detection and music tagging [7].

Fig. 1: Our system performs gradient ascent in the Jukebox VQ-VAE embedding
space such that when the audio is input into a music tagger it matches a
predefined set of tags. The weights of VQ-VAE and the Music Tagger are frozen.
With this setup we can perform unsupervised source separation.

In this work, we further this ethos by exploring how large, pretrained music
models can be used for musical source separation, leveraging the vast amounts
of unlabeled or weakly labeled data that these models see during training. We
combine the VQ-VAE from OpenAI’s Jukebox, a generative model of musical audio,
with a music tagger. We task Jukebox with producing audio that matches a
predefined set of tags that correspond with the musical source we wish to
separate. To do this, we perform gradient ascent in the embedding space of the
VQ-VAE and use the decoded audio as a mask on the input mixture. We
demonstrate experimentally that this setup is able to separate a wider variety
of sources than previous purpose-built separation systems consider, all
without updating the weights of Jukebox or the tagger. We provide additional
demos and runnable code on our demo site.111https://ethman.github.io/tagbox

##  2 Prior Work

Recently, many source separation researchers have focused on methods that
produce high-quality results on the datasets for which there is sufficient
ground truth source data. For instance, the website Papers with Code shows a
steady increase in the best performing separation systems on the MUSDB18 [2]
dataset over the past few years.222https://bit.ly/pwc_musdb18 Similarly, the
recent Music Demixing Challenge [8] invited people to compete to determine the
best performing system on a test set that had the same source definitions as
MUSDB18. As a result, the community has produced a large number of deep
learning-based supervised separation systems that are purpose-built to
separate sources as defined by MUSDB18. However, the source definitions in
MUSDB18 are limiting, [1] including isolated source data for only Vocals,
Bass, Drums, and a catchall “Other” source for all other source types.
Furthermore, MUSDB18 is relatively small, totalling 150 songs, which leads the
authors of many state-of-the-art systems [8, 9, 10] to collect additional data
or lean heavily on augmentation.

Prior to the deep learning era, unsupervised source separation was the norm.
One of the most popular algorithms was Non-negative Matrix Factorization (NMF)
[11]. While NMF is theoretically flexible enough to separate any source, it
often required hand-designed algorithms to determine how to cluster spectral
templates into coherent sources. Musical priors, such as repetition [12] or
harmonicity vs. percussiveness [13], have also been used to create
unsupervised separation algorithms, however such algorithms are limited to
separating sources that match the prior (e.g., a backing band) vs those that
do not (e.g., a singing voice), and have been surpassed by deep learning-based
methods.

Recent work in speech and environmental sound separation has explored
unsupervised deep learning. Mixture Invariant Training (MixIT) [14] is a
technique which creates mixtures of mixtures (MoMs) and tasks a network with
overseparating each MoM such that when sources are recombined, a mixture
reconstruction loss can be used, forgoing the need for isolated source data
altogether. While we are unaware of anyone using MixIT for music separation,
MixIT makes an implicit presumption that any two sources in a mix are
independent [14], an assumption that may not hold for music. Similarly, Neri
et. al [15] propose a technique for training a variational auto-encoder (VAE)
for unsupervised source separation, however in our work we do not train
networks at all, rather we use frozen, pretrained models for separation.

Previous works have explored using additional networks for separation instead
of directly optimizing a separation net on ground truth sources. For instance,
the work of Pishdadian et. al. [16] is most similar to ours; they explore
using a pretrained sound event detection (SED) system and the goal of the
separator network is to maximize estimated SED labels during training.
Similarly, Hung et. al [17] use a pretrained transcription network to train a
separator. Our work differs from Pishdadian et. al. and Huang et. al. in that
we do not not train any networks, instead we repurpose off-the-shelf networks
that have never been trained for source separation.

##  3 Background

###  3.1 OpenAI’s Jukebox

OpenAI recently released Jukebox [18], a generative audio model that creates
music. Jukebox is composed of two components: a hierarchical VQ-VAE [19] that
learns to turn raw waveforms into tokens and back, and a language model that
learns how to generate new tokens which can be passed through the decoder to
create musical audio. Both the VQ-VAE and the language model are unsupervised.
In this work, we are interested in the VQ-VAE, specifically.

Jukebox’s VQ-VAE is a three-level hierarchical VQ-VAE that generates discrete
tokens at different sample rates, compressing the 44.1kHz input audio to
tokens with sample rates of 5.51kHz, 1.37kHz, and 344Hz for each level,
respectively. Each level has a codebook size of 2048 with each code having 64
dimensions. All levels are trained to reconstruct the input waveform and are
optimized with a multi-scale spectral loss. The VQ-VAE also uses a codebook
loss to ensure that non-discretized latent vectors are close to their nearest
neighbor discretized token vectors and a commitment loss, which stabilizes the
encoder. The VQ-VAE is trained on 1.2 million songs scraped from the web. We
refer the reader to the Jukebox paper for further training details [18].
Because we are interested in producing the highest-quality separation results
possible, we only focus on the “Bottom” level, which compresses the input
audio to tokens at a sample rate of 5.51kHz.

Algorithm 1 Our Method

1: $e\leftarrow V_{encoder}(x)$ Encode the input mixture.

2: $X\leftarrow STFT(x)$

3: repeat

4: $j\leftarrow V_{decoder}(e)$ Decode the embedding.

5: $j\leftarrow STFT(j)$

6: $\bar{M}\leftarrow\frac{|J|}{\max(|J|,|X|)+\varepsilon}$ Build the mask.

7: $\bar{S}\leftarrow\bar{M}\odot X$ Mask the mixture.

8: $T_{\bar{S}}\leftarrow Tagger(ISTFT(\bar{S}))$ Get the probability over
tags.

9: $e\leftarrow\delta\nabla L(T_{\bar{S}},T_{target};e)$ Update the embedding.

10: until max steps

11: $s_{out}\leftarrow x-ISTFT(\bar{S})$

###  3.2 Automatic Music Tagging

Music tagging is the task of labeling musical audio clips with semantic labels
called “tags” [20, 21]. These tags are useful for music search and
recommendation systems, enabling automatic labelling of large music corpora.
The content that the tags represent can vary, sometimes indicating information
about a song’s genre, the song’s mood or theme, or whether particular
instruments are audible.

Music tagging systems are designed to predict a set of multi-hot, binary
labels (i.e., tags) based on the acoustic contents of an input signal. Many
recent works use convolutional neural networks at their core, varying the
convolutional filter size and input representation of the audio [22]. Common
datasets for music tagging are an order of magnitude larger than source
separation datasets: MagnaTagATune (MTAT) [20] contains 25,877 30-second
labeled audio clips ($\approx$21x more hours of audio than MUSDB18) and MTG-
Jamendo (MTG) [21] contains 55,701 labeled audio clips with a minimum song
length of 30 seconds ($\geq$ 46x more hours of audio than MUSDB18). We refer
the reader to Won et. al. for an overview of recent advances in music tagging
[22].

In this work, we use pretrained music taggers provided by Won et. al [22]. We
examine using two pretrained music tagging systems, with each having a
different input representation: FCN [23] with Mel spectrogram inputs, and
HarmonicCNN [24], which inputs a variant of a constant-Q transform that has
learnable filters. We also explore using taggers trained on different
datasets, namely MagnaTagATune (MTAT) [20] and MTG-Jamendo (MTG) [21].

##  4 Proposed System

Method | Unsupervised? |  Neural Network? |  MUSDB18 [2] |  Slakh2100 [1]
---|---|---|---|---
Vocals | Bass | Drums | Bass | Drums | Guitar | Piano | Strings
Open-Unmix [10] |  | ✓ | $14.972$ | $11.850$ | $11.598$ | $9.753$ | $14.395$ |  black!5 – |  black!5 – |  black!5 –
Demucs [9] |  | ✓ | $15.494$ | $13.147$ | $12.659$ | $10.759$ | $15.533$ |  black!5 – |  black!5 – |  black!5 –
Cerberus [25] |  | ✓ |  black!5 – | $8.301$ | $7.474$ | $10.843$ | $15.370$ | $10.181$ | $10.520$ | $12.4715$
HPSS [13] | ✓ |  |  black!5 – |  black!5 – | $-0.069$ |  black!5 – | $0.314$ |  black!5 – |  black!5 – |  black!5 –
REPET-SIM [12] | ✓ |  | $7.816$ |  black!5 – |  black!5 – |  black!5 – |  black!5 – |  black!5 – |  black!5 – |  black!5 –
TagBox (Ours) | ✓ | ✓ | $7.388$ | $7.128$ | $5.857$ | $6.934$ | $7.328$ | $9.304$ | $8.749$ | $10.523$
Table 1: Comparison of source separation systems in terms of mean SDR
improvement (dB) over the unprocessed mixture. Grey cells indicate that the
system is unable to separate that source type. TagBox is the only system that
is able to separate all of the sources we test.

At the heart of our proposed system are two components: a pretrained
generative music model (i.e., Jukebox) and a pretrained music tagging model.
Because our system combines music taggers and Jukebox, we call our system
TagBox. The core of the idea is simple, given an input audio clip, the
generative model iteratively alters that input audio such that, when the
altered audio is given to a tagger, the tagger’s output increasingly matches a
target set of tags that describe the desired set of sources. An illustration
of the proposed approach is shown in Figure 1. Algorithm 1 outlines the
approach in pseudocode. We now describe the steps in our method.

We first create a target tag distribution $T_{target}$ by setting the tags
that correspond to the desired instrument sources to $1$ (e.g., “guitar” or
“drums”) and all other tags to $0$. We then use $V_{encoder}$, the encoder
portion of an autoencoder (in this case, the one in Jukebox), to produce an
embedding $e$ from the input audio mixture $x$. This embedding is then decoded
into a waveform $j$ by the decoder $V_{decoder}$.

Rather than pass $j$ directly to the $Tagger$, we use it as a mask on the
input mixture. In this way, the embedding $e$ essentially determines what
information must be removed from the input mix to produce the desired source
as defined by the tags. For an input mixture waveform $x\in\mathbb{R}^{t}$ and
a Jukebox-decoded waveform $j\in\mathbb{R}^{t}$, both with length $t$ samples,
we convert both to a spectrogram representation, $X\in\mathbb{R}^{T\times F}$
and $J\in\mathbb{R}^{T\times F}$, with $T$ time frames and $F$ frequency bins.
We then compute a real-valued mask, $\bar{M}\in\mathbb{R}^{T\times F}$ as
follows:

$\bar{M}=\frac{|J|}{\max(|J|,|X|)+\varepsilon}$ (1)

where $\max()$ is an element-wise max function between each time-frequency bin
in a pair of spectrograms and a small epsilon, e.g $\varepsilon=1e-8$,
prevents division by zero. This mask $\bar{M}$ is multiplied by the mixture
spectrogram to get an estimate of the audio data that should be removed from
the mix like $\bar{S}=\bar{M}\odot X$, where $\odot$ indicates element-wise
multiplication. $\bar{S}$ is then converted to a waveform of the source
estimate $\bar{s}\in\mathbb{R}^{t}$ using an inverse STFT. This waveform,
$\bar{s}$, is then put into the music tagger to determine the estimated tags.
A binary cross-entropy loss $\nabla L(T_{\bar{S}},T_{target};e)$ is computed
between the estimate tags and the predetermined instrument tags. This loss is
used to perform gradient ascent step in the Jukebox embedding space, where
$\delta$ governs the step size. This approach is similar to adversarial
example generation [26], where the goal is also to optimize the input to
produce a desired label. Because the mask made by the Jukebox-decoded audio
determines what should be removed from the mix, the final estimate for a
target source, $\hat{s}$, is the difference between the input mixture waveform
$x$ and the final $\bar{s}$ produced by gradient ascent. The final source
estimate is therefore $s_{out}=x-\bar{s}$.

We note that neither the generative model nor the music tagger were trained
for source separation and that no additional training or alteration of the
weights of either model happens at any point. These models were, however,
trained on datasets with a wider range of audio than is typical for deep
models trained specifically for source separation.

Our system is able to produce separation results for a larger set of sources
than any previous deep learning system that we are aware of. This system is
limited only by the tags of the music tagging system, of which there are 12
distinct instrument tags in MTG-Jamendo (MTG). MagnaTagATune (MTAT) has 31
tags that could be interpreted as instrument tags, although the tags
conceptually overlap somewhat (e.g., MTAT contains distinct tags for “vocals”,
“voice”, “male vocals”, etc). Additionally, separating different source types
does not require any changes to the system setup other than altering a set of
predefined tags. Compare this to typical music separation networks like Open-
Unmix [10] which would require training a whole model for each new source or
Demucs [9] which would require altering the network architecture to add a new
source output.

Tagger Settings |  MUSDB18 [2] |  Slakh [1]
---|---|---
Dataset | Architecture | Vox | Bass | Drums | Bass | Drums | Guitar | Piano | Strings
MTAT | FCN | $7.943$ |  black!5 – | $5.687$ |  black!5 – | $7.317$ | $9.551$ | $8.596$ | $10.412$
HCNN | $6.641$ |  black!5 – | $4.955$ |  black!5 – | $6.506$ | $8.788$ | $7.322$ | $8.484$
MTG-Jamendo | FCN | $7.388$ | $7.128$ | $5.857$ | $6.934$ | $7.328$ | $9.304$ | $8.749$ | $10.523$
HCNN | $6.758$ | $6.744$ | $5.777$ | $6.690$ | $7.280$ | $8.332$ | $8.077$ | $8.980$
Table 2: Comparison of using different pretrained, frozen taggers for gradient
ascent with TagBox in terms of mean SDR improvement (dB) over the unprocessed
mixture. Note the MTAT taggers have no “bass” tag.

##  5 Experimental Validation

We conduct a series of experiments to validate our system, aimed at answering
two questions. The first and main experiment is intended to compare the
proposed system to existing systems, taking special care to try to understand
TagBox’s ability to separate many types of sources. The second experiment is
designed to determine how the choice of the pretrained, frozen Tagger model
affects separation quality.

In our main experiment, we compare our system to existing systems on two
established test sets for source separation, namely MUSDB18 [2] and Slakh2100
[1]. In this experiment we compare our proposed system against recent deep
learning-based supervised separation systems as well unsupervised separations
based on musical priors. We compare our system on a wide variety of source
types across both of these datasets.

The first dataset we examine is MUSDB18. MUSDB18 contains 150 mixtures and
corresponding sources from real live recording sessions, 100 of these are
reserved for training and the remaining 50 are used for testing. For this
experiment, we exclude MUSDB18’s “other” source because it could map to many
possible tags using TagBox. The supervised systems that we compare against,
namely Open-Unmix [10] and Demucs [9], are trained using the MUSDB18 training
set. Contrast this to the unsupervised systems we test, HPSS [13] and REPET-
SIM [12], which are run on the test set without any training. Our proposed
system falls into this second camp; it is also unsupervised and therefore does
not have a training phase, ignoring the MUSDB18 training set.

The main experiment also uses the Slakh2100 [1] dataset. Slakh2100 contains
2100 mixtures with corresponding sources that were synthesized using
professional-grade sample-based synthesis engines. We chose 50 songs from the
test set to evaluate on. We chose songs that have source data for following
five source types: bass, drums, guitar, piano, strings. We select mixes where
all 5 sources are active, and we say a source is active if it has 100 or more
note onsets throughout the entirety of the song, as determined by the
corresponding MIDI data. We create mixes by instantaneously mixing together
the sources and use these mixtures as input to the systems. With this setup we
compare against Cerberus [25], which was trained to separate these five
instruments, specifically.

For TagBox, we use a pretrained FCN [23] tagger trained on the MagnaTagATune
(MTAT) [20] dataset. We run gradient ascent with a learning rate of 5.0 using
the Adam optimizer for 10 steps (in the interest of brevity), and use a
spectrogram with 1024 FFT bins for the mask. Additionally, we use the
“foreground” from REPET-SIM as the vocals estimate, following prior work [12],
and use the “percussion” output from HPSS as the drums estimate. We omit the
other source outputs of these systems because they are ill-defined (e.g.,
HPSS’s “harmonic” could be many possible sources).

In the second experiment, we compare four different configurations of our
proposed system, varying the architecture and training data of the music
tagger. We look at the FCN [23] and HarmonicCNN [24] architectures, trained
either MagnaTagATune (MTAT) [20] or MTG-Jamendo [21]. We use the same learning
rate and number of steps as the previous experiment.

We evaluate the outcome of our experiments using the source-to-distortion
ratio improvement (SDRi) over the unprocessed mixture [27] using the museval
toolbox [28].

##  6 Results and Discussion

Table 1 shows the results of our main experiment. In terms of SDRi, our system
is better than or competitive with both of the hand-designed unsupervised
algorithms that we test against, HPSS and REPET-SIM. Additionally, while our
system does not show as good of performance as the purpose-built supervised
separation systems (i.e., Open-Unmix, Demucs, and Cerberus), it still shows a
considerable SDRi boost for all sources that we test. Importantly, our system
is able to boost performance over a wider array of source types than any other
system we compare against.

The results from our second experiment are shown in Table 2. Of the two
architectures we test, using FCN always produces better separation results.
Interestingly, the opposite trend was observed when the taggers were evaluated
for music tagging performance by Won et. al. [22]: HCNN was among the top
performing systems and FCN was towards the bottom of the pack.

In many cases, TagBox can leave much to be desired perceptually; in most cases
its separation performance is not up to the same level as the purpose-built
separation systems we compare against. However, when listening to the output,
there is no doubt that TagBox is able to separate the desired source, despite
apparent artifacts. We have informally noticed a few tricks for better
perceptual performance, like using multiple FFT sizes when making the masks (à
la a multi-scale spectral loss) and doing gradient ascent for 100 steps. These
perceptual tricks were however not reflected in the SDR evaluation numbers.
Furthermore, because producing each output example requires its own gradient
ascent, adding more steps increases the computation time linearly, which can
be a costly process when run on an entire dataset. However, this might be
tolerable for musicians needing a flexible source separation solution on a
single song.

There are also a few other variants of the TagBox setup that can lead to fun
and unexpected creative results. In the first case, we remove the masking step
and allow TagBox to create audio freely, without the constraint of having to
only remove information from the mix. With this setup, TagBox performs a kind
of style transfer, mapping certain features of the audio to the desired tag.
In one example, a mixture had a singer and we selected the “guitar” tag.
TagBox made the resultant audio sound like a guitar was performing the melody.
Additionally, another variant involves selecting non-instrument tags, like
genre tags, and optimizing those.

What we find most impressive is that neither Jukebox nor the music taggers
were trained for source separation. Furthermore, the weights of both networks
do not change during the gradient ascent process; only the location of the
audio in the Jukebox embedding space changes. The combination of Jukebox and
the taggers have seen up to 1.25 million songs and combined these systems are
able to leverage their shared priors about music and musical sources to
isolate individual musical sources in a mixture. We believe that these priors
could be leveraged in many ways to overcome the data scarcity problems endemic
to many MIR tasks, as has already been investigated to great effect by
Castellon et. al [7]. We are excited about future explorations in this area.

##  7 Conclusion

In this paper, we have proposed a method for unsupervised source separation by
combining pretrained models, called TagBox. We use pretrained music taggers to
do gradient ascent in the embedding space of OpenAI’s Jukebox with the goal of
maximizing a pre-defined tag corresponding to the source we want to separate.
The output of Jukebox is used as a mask on the input audio before being sent
to the tagger, which ensures that Jukebox does not generate new, unseen data
that is not present in the input mixture. Importantly, neither the tagger nor
Jukebox have been trained for source separation and the weights of both models
remain fixed during the gradient ascent process. We demonstrate results
showing that our system is able to separate a wider variety of source types
than many recent purpose-built, supervised separation systems. We are excited
by the promise that pretrained systems hold for the future of MIR and source
separation research.

##  8 Acknowledgements

The authors would like to thank Ian Simon, Sander Dieleman, Jesse Engel, and
Curtis Hawthorne for their fruitful conversations about this work.
Additionally, we would like to thank the creators of Jukebox for help with
their codebase: Prafulla Dhariwal, Heewoo Jun, Christine Payne, Jong Wook Kim,
Alec Radford, and Ilya Sutskever.

## References

  * [1] Ethan Manilow, Gordon Wichern, Prem Seetharaman, and Jonathan Le Roux,  “Cutting music source separation some slakh: A dataset to study the impact of training data quality and quantity,”  in 2019 IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA). IEEE, 2019, pp. 45–49.
  * [2] Zafar Rafii, Antoine Liutkus, Fabian-Robert Stöter, Stylianos Ioannis Mimilakis, and Rachel Bittner,  “The MUSDB18 corpus for music separation,” Dec. 2017.
  * [3] Patrick Esser, Robin Rombach, and Bjorn Ommer,  “Taming transformers for high-resolution image synthesis,”  in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021, pp. 12873–12883.
  * [4] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever,  “Learning transferable visual models from natural language supervision,” 2021.
  * [5] Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu,  “Exploring the limits of transfer learning with a unified text-to-text transformer,”  Journal of Machine Learning Research, vol. 21, pp. 1–67, 2020.
  * [6] Tom B Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al.,  “Language models are few-shot learners,”  arXiv preprint arXiv:2005.14165, 2020.
  * [7] Rodrigo Castellon, Chris Donahue, and Percy Liang,  “Codified audio language modeling learns useful representations for music information retrieval,”  arXiv preprint arXiv:2107.05677, 2021.
  * [8] Yuki Mitsufuji, Giorgio Fabbro, Stefan Uhlich, and Fabian-Robert Stöter,  “Music demixing challenge at ismir 2021,”  arXiv preprint arXiv:2108.13559, 2021.
  * [9] Alexandre Défossez, Nicolas Usunier, Léon Bottou, and Francis Bach,  “Music source separation in the waveform domain,”  arXiv preprint arXiv:1911.13254, 2019.
  * [10] Fabian-Robert Stöter, Stefan Uhlich, Antoine Liutkus, and Yuki Mitsufuji,  “Open-unmix-a reference implementation for music source separation,”  Journal of Open Source Software, vol. 4, no. 41, pp. 1667, 2019\.
  * [11] Paris Smaragdis,  “Blind separation of convolved mixtures in the frequency domain,”  Neurocomputing, vol. 22, no. 1-3, pp. 21–34, 1998.
  * [12] Zafar Rafii and Bryan Pardo,  “Music/voice separation using the similarity matrix,”  in In Proceedings of the 13th International Society for Music Information Retrieval Conference (ISMIR), 2012.
  * [13] Derry Fitzgerald,  “Harmonic/percussive separation using median filtering,”  in Proceedings of the International Conference on Digital Audio Effects (DAFx), 2010, vol. 13.
  * [14] Scott Wisdom, Efthymios Tzinis, Hakan Erdogan, Ron Weiss, Kevin Wilson, and John Hershey,  “Unsupervised sound separation using mixture invariant training,”  in Advances in Neural Information Processing Systems, H. Larochelle, M. Ranzato, R. Hadsell, M. F. Balcan, and H. Lin, Eds. 2020, vol. 33, pp. 3846–3857, Curran Associates, Inc.
  * [15] Julian Neri, Roland Badeau, and Philippe Depalle,  “Unsupervised blind source separation with variational auto-encoders,”  in 29th European Signal Processing Conference (EUSIPCO 2021), 2021\.
  * [16] Fatemeh Pishdadian, Gordon Wichern, and Jonathan Le Roux,  “Finding strength in weakness: Learning to separate sounds with weak supervision,”  IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 28, pp. 2386–2399, 2020.
  * [17] Yun-Ning Hung, Gordon Wichern, and Jonathan Le Roux,  “Transcription is all you need: Learning to separate musical mixtures with score as supervision,”  in ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2021, pp. 46–50.
  * [18] Prafulla Dhariwal, Heewoo Jun, Christine Payne, Jong Wook Kim, Alec Radford, and Ilya Sutskever,  “Jukebox: A generative model for music,” 2020.
  * [19] Ali Razavi, Aaron van den Oord, and Oriol Vinyals,  “Generating diverse high-fidelity images with vq-vae-2,”  in Advances in neural information processing systems, 2019, pp. 14866–14876.
  * [20] Edith Law, Kris West, Michael I Mandel, Mert Bay, and J Stephen Downie,  “Evaluation of algorithms using games: The case of music tagging.,”  in In Proceedings of the 10th International Society for Music Information Retrieval Conference (ISMIR), 2009, pp. 387–392.
  * [21] Dmitry Bogdanov, Minz Won, Philip Tovstogan, Alastair Porter, and Xavier Serra,  “The mtg-jamendo dataset for automatic music tagging,”  2019\.
  * [22] Minz Won, Andres Ferraro, Dmitry Bogdanov, and Xavier Serra,  “Evaluation of cnn-based automatic music tagging models,”  in Proc. of 17th Sound and Music Computing, 2020.
  * [23] Keunwoo Choi, George Fazekas, and Mark Sandler,  “Automatic tagging using deep convolutional neural networks,” 2016.
  * [24] Minz Won, Sanghyuk Chun, Oriol Nieto, and Xavier Serrc,  “Data-driven harmonic filters for audio representation learning,”  in ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2020, pp. 536–540.
  * [25] Ethan Manilow, Prem Seetharaman, and Bryan Pardo,  “Simultaneous separation and transcription of mixtures with multiple polyphonic and percussive instruments,”  in ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2020, pp. 771–775.
  * [26] Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy,  “Explaining and harnessing adversarial examples,”  arXiv preprint arXiv:1412.6572, 2014.
  * [27] Emmanuel Vincent, Rémi Gribonval, and Cédric Févotte,  “Performance measurement in blind audio source separation,”  IEEE transactions on audio, speech, and language processing, vol. 14, no. 4, pp. 1462–1469, 2006.
  * [28] Fabian-Robert Stöter, Antoine Liutkus, and Nobutaka Ito,  “The 2018 signal separation evaluation campaign,”  in Latent Variable Analysis and Signal Separation: 14th International Conference, LVA/ICA 2018, Surrey, UK, 2018, pp. 293–305.