# Axisymmetric dynamo action
produced by differential rotation,
with anisotropic electrical conductivity
and anisotropic magnetic permeability

Franck Plunian1<EMAIL_ADDRESS>Thierry Alboussière2
1Université Grenoble Alpes, Université Savoie Mont Blanc, CNRS, IRD, IFSTTAR,
ISTerre, 38000 Grenoble, France 2Univ. Lyon, Univ. Lyon 1, ENSL, CNRS, LGL-
TPE, F-69622, Villeurbanne, France

###### Abstract

The effect on dynamo action of an anisotropic electrical conductivity
conjugated to an anisotropic magnetic permeability is considered. Not only is
the dynamo fully axisymmetric, but it requires only a simple differential
rotation, which twice challenges the well-established dynamo theory. Stability
analysis is conducted entirely analytically, leading to an explicit expression
of the dynamo threshold. The results show a competition between the anisotropy
of electrical conductivity and that of magnetic permeability, the dynamo
effect becoming impossible if the two anisotropies are identical. For
isotropic electrical conductivity, Cowling’s neutral point argument does imply
the absence of an azimuthal component of current density, but does not prevent
the dynamo effect as long as the magnetic permeability is anisotropic.

###### keywords:

Magnetohydrodynamics, Dynamo effect, Anisotropy

##  1 Introduction

The dynamo effect is a magnetic instability produced by the displacement of an
electrically conducting medium, without the aid of a magnet or a remanent
magnetic field. A part of the kinetic energy of the moving medium is thus
transferred into magnetic energy. This process is the most likely candidate to
explain the ubiquity of magnetic fields observed in astrophysical objects
(Rincon, 2019). The increasing resolution of numerical simulations of dynamo
equations makes it possible to reproduce, ever better, the magnetic features
measured in natural objects (Schaeffer et al., 2017). Experiments have also
successfully demonstrated the possibility of dynamo action (Gailitis et al.,
2001; Stieglitz & Müller, 2001; Monchaux et al., 2007), although it remains
difficult to replicate in the laboratory processes occurring on a geophysical
or astrophysical scale (Alboussière et al., 2011; Tigrine et al., 2019). This
is why, since the very first dynamo experiments (Lowes & Wilkinson, 1963,
1968), the use of materials with the highest product of electrical
conductivity and magnetic permeability, has been favoured. In most cases it
led to the choice of solid iron alloys (high permeability) or liquid sodium
(high conductivity) for the moving parts. In the VKS experiment, in which
liquid sodium was driven by impellers, the high magnetic permeability of the
impellers was revealed to be crucial to achieve a dynamo effect (Miralles et
al., 2013; Kreuzahler et al., 2017; Nore et al., 2018). The choice of
materials for the static parts, like the walls of the container, has also been
proved to be crucial in relation to the electromagnetic boundary conditions
(Avalos-Zuñiga et al., 2003; Avalos-Zuñiga & Plunian, 2005), leading for
example to the use of copper walls (Monchaux et al., 2007).

The role in the dynamo effect of an anisotropic electrical conductivity has
been studied for different geometries: Cartesian (Ruderman & Ruzmaikin, 1984;
Alboussière et al., 2020), cylindrical (Plunian & Alboussière, 2020) and
toroidal (Lortz, 1989). Although at first glance it is difficult to imagine
such an anisotropic electrical conductivity in natural objects, it is far from
impossible. For example, in a plasma subjected to a magnetic field, it is
known that the electrical conductivity in the direction parallel to the
magnetic field is twice that in the direction perpendicular to the magnetic
field (Braginskii, 1965). In the case of the Earth, seismic observations
provide strong evidence that the elastic response of the solid inner core is
anisotropic. This is most likely due to the alignment of hexagonal close-
packed iron crystals, occurring during the solidification of the inner core
(Deuss, 2014). Incidentally it has been shown that, in hexagonal close-packed
iron, the thermal conductivity and the electrical conductivity, which are
directly related, are anisotropic (Ohta et al., 2018). Eventually, this
suggests that the electrical conductivity of the inner core is anisotropic.
Finally, in a turbulent electrically conducting fluid, the interaction between
the small scales of the velocity field and those of the magnetic field can
generate a large-scale magnetic field by dynamo action. Such a process can be
modeled using the so-called mean-field approach (Krause & Rädler, 1980),
possibly leading to a large-scale anisotropic electrical conductivity
(Brandenburg, 2018).

From a theoretical point of view, an interesting consequence of considering an
anisotropic conductivity is the possibility of obtaining an axisymmetric
dynamo effect (Plunian & Alboussière, 2020), allowing one to bypass the well-
known Cowling’s antidynamo theorem (Cowling, 1934; Kaiser & Tilgner, 2014).
Indeed, if it is anisotropic, then the electrical conductivity becomes a
tensor, instead of being a scalar, which defeats Cowling’s neutral point
argument. In addition, the use of an anisotropic conductivity leads to dynamo
action for a motion as simple as shear (Ruderman & Ruzmaikin, 1984;
Alboussière et al., 2020; Plunian & Alboussière, 2020), which is otherwise
impossible to achieve.

Here we investigate the role of an anisotropic electrical conductivity
conjugated to an anisotropic magnetic permeability. An anisotropic magnetic
permeability is not expected in natural objects. However, as explained above,
this may be of interest for dynamo experiments, in order to reduce the dynamo
threshold. In contrast to previous dynamo studies (Busse & Wicht, 1992; Kaiser
& Tilgner, 1999), here the electrical conductivity and magnetic permeability
do not depend on time or space coordinates. In addition, they are stationary
and axisymmetric.

##  2 Conductivity and permeability anisotropy

We consider a material such that the electrical conductivity and magnetic
permeability are denoted $\sigma^{\parallel}$ and $\mu^{\parallel}$ in a given
direction q, and $\sigma^{\perp}$ and $\mu^{\perp}$ in the directions
perpendicular to q.

Writing Ohm’s law, ${\bf J}=\sigma^{\parallel}{\bf E}$ in the direction of q
and ${\bf J}=\sigma^{\perp}{\bf E}$ in the directions perpendicular to q,
leads to the following conductivity tensor:

$[\sigma_{ij}]=\sigma^{\perp}\delta_{ij}+(\sigma^{\parallel}-\sigma^{\perp})q_{i}q_{j}.$
(1)

Inverting (1) leads to the resistivity tensor: (Ruderman & Ruzmaikin, 1984)

$[\sigma_{ij}]^{-1}=\frac{1}{\sigma^{\perp}}\delta_{ij}+(\frac{1}{\sigma^{\parallel}}-\frac{1}{\sigma^{\perp}})q_{i}q_{j}.$
(2)

Similarly, a magnetic permeability tensor can be defined as

$[\mu_{ij}]=\mu^{\perp}\delta_{ij}+(\mu^{\parallel}-\mu^{\perp})q_{i}q_{j},$
(3)

with the inverse tensor

$[\mu_{ij}]^{-1}=\frac{1}{\mu^{\perp}}\delta_{ij}+(\frac{1}{\mu^{\parallel}}-\frac{1}{\mu^{\perp}})q_{i}q_{j}.$
(4)

We choose q as a unit vector in the horizontal plane:

$\textit{{q}}=c\;{\bf e}_{r}+s\;{\bf e}_{\theta},$ (5)

where $({\bf e}_{r},{\bf e}_{\theta},{\bf e}_{z})$ is a cylindrical coordinate
system, with $c=\cos\alpha$ and $s=\sin\alpha$, $\alpha$ being a prescribed
angle.

In figure 1 the curved lines are perpendicular to q and describe logarithmic
spirals. They correspond to the directions along which $\sigma=\sigma^{\perp}$
and $\mu=\mu^{\perp}$. We consider the solid-body rotation ${\bf U}$ of a
cylinder of radius $R$ embedded in an infinite medium at rest. Both regions
are made of the same material, with therefore identical conductivity tensors
and identical permeability tensors.

Figure 1: Left: The inner-cylinder of radius $R$ rotates as a solid-body
within an outer cylinder at rest. The radius $R^{\prime}$ of the outer
cylinder is taken as infinite. Right: The curved lines are perpendicular to q
and describe logarithmic spirals. They correspond to the directions along
which $\sigma=\sigma^{\perp}$ and $\mu=\mu^{\perp}$.

##  3 Induction equation

In the magnetohydrodynamic approximation, the Maxwell equations and Ohm’s law
take the form

$\displaystyle{\bf H}=[\mu_{ij}]^{-1}{\bf B},$ (6) $\displaystyle{\bf
J}=\nabla\times{\bf H},$ (7) $\displaystyle\partial_{t}{\bf
B}=-\nabla\times{\bf E}$ (8) $\displaystyle\nabla\cdot{\bf B}=0,$ (9)
$\displaystyle{\bf J}=[\sigma_{ij}]({\bf E}+{\bf U}\times{\bf B}),$ (10)

where ${\bf H}$, ${\bf B}$, ${\bf J}$, ${\bf E}$ and ${\bf U}$ are the
magnetic field, the induction field, the current density, the electric field
and the velocity field. The induction equation then takes the form

$\partial_{t}{\bf B}=\nabla\times({\bf U}\times{\bf
B})-\nabla\times\left([\sigma_{ij}]^{-1}\nabla\times\left([\mu_{ij}]^{-1}{\bf
B}\right)\right).$ (11)

Renormalizing the distance, electrical conductivity, magnetic permeability and
time by respectively $R,\mu^{\perp},\sigma^{\perp}$ and
$\mu^{\perp}\sigma^{\perp}R^{2}$, the dimensionless form of the induction
equation is identical to (11), but with

$\displaystyle[\sigma_{ij}]^{-1}$ $\displaystyle=$
$\displaystyle\delta_{ij}+\sigma
q_{i}q_{j},\;\;\;\sigma=\frac{\sigma^{\perp}}{\sigma^{\parallel}}-1,$ (12)
$\displaystyle[\mu_{ij}]^{-1}$ $\displaystyle=$ $\displaystyle\delta_{ij}+\mu
q_{i}q_{j},\;\;\;\mu=\frac{\mu^{\perp}}{\mu^{\parallel}}-1$ (13)

and

${\bf U}=\left\\{\begin{split}r\Omega{\bf e}_{\theta}&,&r<1\\\
0&,&r>1\end{split}\;\;\;,\right.$ (14)

where $\Omega$ is the dimensionless angular velocity of the inner cylinder. We
note that $(\sigma,\mu)\in[-1,+\infty[^{2}$, with $\sigma=0$ and $\mu=0$
corresponding to respectively isotropic conductivity and isotropic
permeability.

Provided the velocity is stationary and $z$-independent, an axisymmetric
magnetic induction can be searched in the form

${\bf B}(r,z,t)=\tilde{B}{\bf e}_{\theta}+\nabla\times\left(\tilde{A}{\bf
e}_{\theta}\right),$ (15)

with $(\tilde{A},\tilde{B})=(A(r),B(r))\exp(\gamma t+\mathrm{i}kz)$, where
$\gamma$ is the instability growth rate and $k$ the vertical wavenumber of the
corresponding eigenmode, and where $A$ and $B$ depend only on the radial
coordinate $r$. Thus the magnetic induction takes the form

${\bf B}=\left(-\mathrm{i}kA,B,\frac{1}{r}\partial_{r}(rA)\right)\exp(\gamma
t+\mathrm{i}kz),$ (16)

with dynamo action corresponding to $\Re\\{\gamma\\}>0$.

From (14) and (16), it can be shown that $\nabla\times({\bf U}\times{\bf
B})=0$ in each region $r<1$ and $r>1$ (see appendix A). Replacing (12), (13)
and (16) into the induction equation (11) leads to

$\displaystyle\gamma A+(1+\sigma s^{2})D_{k}(A)+\mu c^{2}k^{2}A$
$\displaystyle=$ $\displaystyle\mathrm{i}csk(\sigma-\mu)B,$ (17)
$\displaystyle\gamma B+(1+\mu s^{2})D_{k}(B)+\sigma c^{2}k^{2}B$
$\displaystyle=$ $\displaystyle-\mathrm{i}csk(\sigma-\mu)D_{k}(A),$ (18)

where
$D_{\nu}(X)=\nu^{2}X-\partial_{r}\left(\frac{1}{r}\partial_{r}(rX)\right)$.
The derivation of (17) and (18) is given in appendix B. For $\sigma=\mu=0$,
corresponding to isotropy of both conductivity and permeability, (17) and (18)
are diffusion equations, leading to a free decaying solution (no dynamo
action). For an isotropic permeability, $\mu=0$, (17) and (18) are identical
to the equations derived in Plunian & Alboussière (2020).

##  4 Dynamo threshold

###  4.1 General form of the solutions

Looking for non-oscillating solutions, the dynamo threshold then corresponds
to $\gamma=0$. Thus, taking $\gamma=0$ in (17) and (18), it can be shown
(Appendix C) that

$\left(D_{k_{\mu}}\circ D_{k_{\sigma}}\right)(A)=\left(D_{k_{\sigma}}\circ
D_{k_{\mu}}\right)(B)=0,$ (19)

where

$k_{\sigma}=k\left(\frac{1+\sigma}{1+\sigma
s^{2}}\right)^{1/2},\;\;\;\;\;k_{\mu}=k\left(\frac{1+\mu}{1+\mu
s^{2}}\right)^{1/2}.$ (20)

We note that the two operators $D_{k_{\sigma}}$ and $D_{k_{\mu}}$ are
commutative. Therefore in (19) we can apply the two operators in the order we
want, $D_{k_{\mu}}\circ D_{k_{\sigma}}$ or $D_{k_{\sigma}}\circ D_{k_{\mu}}$,
to both $A$ and $B$. The set of functions $X(r)$, satisfying the fourth-order
differential equation $\left(D_{k_{\mu}}\circ D_{k_{\sigma}}\right)(X)=0$, is
a vector space of dimension 4. Now, we know that, whatever $\nu$, the
solutions of $D_{\nu}(X)=0$ are a linear combination of $I_{1}(\nu r)$ and
$K_{1}(\nu r)$, where $I_{1}$ and $K_{1}$ are modified Bessel functions of
first and second kind, of order 1. Therefore the solutions of (19) are a
linear combination of $I_{1}(k_{\sigma}r)$, $K_{1}(k_{\sigma}r)$,
$I_{1}(k_{\mu}r)$ and $K_{1}(k_{\mu}r)$.

Looking for $A$ in the form

$A=\alpha_{\sigma}I_{1}(k_{\sigma}r)+\beta_{\sigma}K_{1}(k_{\sigma}r)+\alpha_{\mu}I_{1}(k_{\mu}r)+\beta_{\mu}K_{1}(k_{\mu}r),$
(21)

and specifying that $A$ must be finite at $r=0$ and that
$\lim\limits_{r\to\infty}A=0$, leads to

$\displaystyle A=$
$\displaystyle\left\\{\begin{split}r<1,&\;\;\;\;\;\alpha_{\sigma}I_{1}(k_{\sigma}r)+\alpha_{\mu}I_{1}(k_{\mu}r)&\\\
r>1,&\;\;\;\;\;\beta_{\sigma}K_{1}(k_{\sigma}r)+\beta_{\mu}K_{1}(k_{\mu}r)&\end{split}\right.,$
(22)

where $\alpha_{\sigma}$, $\beta_{\sigma}$, $\alpha_{\mu}$ and $\beta_{\mu}$
are free parameters that will be constrained by additional boundary conditions
at $r=1$. Replacing (22) in (17) for $\gamma=0$ leads to the following
expression for $B$

$\displaystyle B=$
$\displaystyle\left\\{\begin{split}r<1,&\;\;\;\frac{\mathrm{i}ck}{s}\left(\alpha_{\sigma}I_{1}(k_{\sigma}r)+\frac{\mu
s^{2}}{1+\mu s^{2}}\alpha_{\mu}I_{1}(k_{\mu}r)\right)&\\\
r>1,&\;\;\;\frac{\mathrm{i}ck}{s}\left(\beta_{\sigma}K_{1}(k_{\sigma}r)+\frac{\mu
s^{2}}{1+\mu s^{2}}\beta_{\mu}K_{1}(k_{\mu}r)\right)&\end{split}\right.,$ (23)

the derivation of which being given in Appendix D.

###  4.2 Boundary conditions at $r=1$

From the Maxwell equations and Green-Ostrogradski and Stokes theorems, the
radial component of ${\bf B}$ and the tangential components of ${\bf
H}=[\mu_{ij}]^{-1}{\bf B}$ must be continuous at $r=1$. Taking the expression
of ${\bf B}$ and ${\bf H}$ given in (16) and (49), these continuity conditions
can be written in terms of $A$ and $B$ as

$\displaystyle A(r=1^{-})$ $\displaystyle=$ $\displaystyle A(r=1^{+}),$ (24)
$\displaystyle B(r=1^{-})$ $\displaystyle=$ $\displaystyle B(r=1^{+}),$ (25)
$\displaystyle\partial_{r}A(r=1^{-})$ $\displaystyle=$
$\displaystyle\partial_{r}A(r=1^{+}).$ (26)

Taking $A$ and $B$ given in (22) and (23) and replacing them in (24), (25) and
(26) leads to (Appendix E)

$\displaystyle\alpha_{\sigma}I_{1}(k_{\sigma})-\beta_{\sigma}K_{1}(k_{\sigma})$
$\displaystyle=$ $\displaystyle 0$ (27)
$\displaystyle\alpha_{\mu}I_{1}(k_{\mu})-\beta_{\mu}K_{1}(k_{\mu})$
$\displaystyle=$ $\displaystyle 0$ (28)
$\displaystyle\alpha_{\sigma}k_{\sigma}I_{0}(k_{\sigma})+\alpha_{\mu}k_{\mu}I_{0}(k_{\mu})+\beta_{\sigma}k_{\sigma}K_{0}(k_{\sigma})+\beta_{\mu}k_{\mu}K_{0}(k_{\mu})$
$\displaystyle=$ $\displaystyle 0,$ (29)

where $I_{0}$ and $K_{0}$ are modified Bessel functions of first and second
kind, of order 0. It is convenient to introduce the parameters
$\lambda_{\sigma}=\alpha_{\sigma}I_{1}(k_{\sigma})\mathrm{i}k/s$ and
$\lambda_{\mu}=\alpha_{\mu}I_{1}(k_{\mu})\mathrm{i}k/s$. Then, using (27) and
(28), we can rewrite $A$ and $B$ in the following form:

$\displaystyle\mathrm{i}kA=$
$\displaystyle\left\\{\begin{split}r<1,&\;\;\;\;\;s\left(\lambda_{\sigma}\frac{I_{1}(k_{\sigma}r)}{I_{1}(k_{\sigma})}+\lambda_{\mu}\frac{I_{1}(k_{\mu}r)}{I_{1}(k_{\mu})}\right)&\\\
r>1,&\;\;\;\;\;s\left(\lambda_{\sigma}\frac{K_{1}(k_{\sigma}r)}{K_{1}(k_{\sigma})}+\lambda_{\mu}\frac{K_{1}(k_{\mu}r)}{K_{1}(k_{\mu})}\right)&\end{split}\right.,$
(30)

$\displaystyle B=$
$\displaystyle\left\\{\begin{split}r<1,&\;\;\;c\left(\lambda_{\sigma}\frac{I_{1}(k_{\sigma}r)}{I_{1}(k_{\sigma})}+\frac{\mu
s^{2}}{1+\mu
s^{2}}\lambda_{\mu}\frac{I_{1}(k_{\mu}r)}{I_{1}(k_{\mu})}\right)&\\\
r>1,&\;\;\;c\left(\lambda_{\sigma}\frac{K_{1}(k_{\sigma}r)}{K_{1}(k_{\sigma})}+\frac{\mu
s^{2}}{1+\mu
s^{2}}\lambda_{\mu}\frac{K_{1}(k_{\mu}r)}{K_{1}(k_{\mu})}\right)&\end{split}\right..$
(31)

The continuity of $\partial_{r}A$ at $r=1$, given by (29), then leads to the
following identity between $\lambda_{\sigma}$ and $\lambda_{\mu}$

$\lambda_{\sigma}\Gamma(k_{\sigma})+\lambda_{\mu}\Gamma(k_{\mu})=0,$ (32)

with

$\Gamma(x)=x\left(\frac{I_{0}(x)}{I_{1}(x)}+\frac{K_{0}(x)}{K_{1}(x)}\right)\equiv\left(I_{1}(x)K_{1}(x)\right)^{-1},$
(33)

the last equality coming from the Wronskian relation

$I_{m}(x)K_{m+1}(x)+I_{m+1}(x)K_{m}(x)=1/x.$ (34)

In figure 2 the eigenmodes $\mathrm{i}kA$ and $B$ are plotted versus $r$ for
$\lambda_{\sigma}=\Gamma(k_{\mu})$ and $\lambda_{\mu}=-\Gamma(k_{\sigma})$
such that (32) is satisfied.

Figure 2: Eigenmodes $\mathrm{i}kA$ (dashed lines) and $B$ (solid lines)
versus $r$, for $\sigma=10^{6},k=1.1,\alpha=0.16\pi$,
$\lambda_{\sigma}=\Gamma(k_{\mu})$, $\lambda_{\mu}=-\Gamma(k_{\sigma})$ and
for $\mu\in\\{-0.99,0,0.5\\}$.

Finally, the tangential components ${\bf E}_{\theta}$ and ${\bf E}_{z}$ of the
electric field

${\bf E}=-{\bf U}\times{\bf B}+[\sigma_{ij}]^{-1}{\bf J}$ (35)

have to be continuous at $r=1$. The expression of the current density ${\bf
J}$, which is derived in Appendix F, is given by

$\displaystyle{\bf J}_{r}=$
$\displaystyle-\mathrm{i}ck\lambda_{\sigma}\left\\{\begin{split}r<1,&\;\;\;I_{1}(k_{\sigma}r)/I_{1}(k_{\sigma})\\\
r>1,&\;\;\;K_{1}(k_{\sigma}r)/K_{1}(k_{\sigma})\end{split}\right.,$ (36)
$\displaystyle{\bf J}_{\theta}=$ $\displaystyle-\frac{\sigma sc}{1+\sigma
s^{2}}{\bf J}_{r},$ (37) $\displaystyle{\bf J}_{z}=$ $\displaystyle
ck_{\sigma}\lambda_{\sigma}\left\\{\begin{split}r<1,&\;\;\;I_{0}(k_{\sigma}r)/I_{1}(k_{\sigma})\\\
r>1,&\;\;\;-K_{0}(k_{\sigma}r)/K_{1}(k_{\sigma})\end{split}\right.,$ (38)

where the coefficient $\exp(\mathrm{i}kz)$ has been dropped for convenience.

From (12), (14) and (37), we find that ${\bf E}_{\theta}=0$, which is in
agreement with axisymmetric solutions. Indeed, Maxwell equation (8) taken at
the threshold implies $\nabla\times{\bf E}=0$. Applying the Stokes theorem to
the integral of $\nabla\times{\bf E}$ on a disc of radius $r$, and assuming
axisymmetry, then leads to ${\bf E}_{\theta}(r)=0$.

The continuity of ${\bf E}_{z}$ implies the following identity:

$({\bf J}_{z}+r\Omega{\bf B}_{r})(r=1^{-})={\bf J}_{z}(r=1^{+}).$ (39)

Replacing (30) and (38) in (39), and using (32), leads to the dynamo threshold

$\Omega^{c}=\frac{c}{s}\left(I_{1}(k_{\sigma})K_{1}(k_{\sigma})-I_{1}(k_{\mu})K_{1}(k_{\mu})\right)^{-1}.$
(40)

Figure 3: Isovalues of $\Omega^{c}\in\\{\pm 5,\pm 10,\pm 20,\pm 40,\pm 100\\}$
in the ($k_{\sigma},k_{\mu}$) map, for $\alpha=0.16\pi$. The diagonal
$k_{\sigma}=k_{\mu}$ corresponds to $\Omega^{c}\rightarrow\pm\infty$. Figure
4: Curves of the dynamo threshold $\Omega^{c}$ versus $k$, for $\mu=0$,
$\alpha=0.16\pi$ and $\sigma\in\\{-0.99,-0.8,-0.6,-0.5,1,2,4,+\infty\\}$.
Figure 5: Curves of the dynamo threshold $\Omega^{c}$ versus $k$, for
$\sigma=10^{5}$, $\alpha=0.16\pi$ and $\mu\in\\{-0.99,-0.9,-0.5,0,1,2,5\\}$.

##  5 Analysis of the results

###  5.1 Dispersion relation

A striking consequence of (40) is that $\Omega^{c}(\sigma,\mu)$ is
antisymmetric, satisfying

$\Omega^{c}(\sigma,\mu)=-\Omega^{c}(\mu,\sigma).$ (41)

In addition for identical anisotropies of conductivity and permeability,
$\mu=\sigma$, the threshold is infinite, leading to the impossibility of an
axisymmetric dynamo,

$\lim\limits_{|\sigma-\mu|\rightarrow
0}|\Omega^{c}(\sigma,\mu)|\rightarrow+\infty.$ (42)

This is illustrated in figure 3, in which the curves of a few isovalues of
$\Omega^{c}$ are plotted versus $k_{\sigma}$ and $k_{\mu}$. In particular,
having both $\sigma\gg 1$ and $\mu\gg 1$ is detrimental for dynamo action, as
in this case, from (20), $k_{\sigma}\approx k_{\mu}\approx k/s$.

The antisymmetry property (41) of $\Omega^{c}(\sigma,\mu)$ can also be derived
directly from the set of equations (17-18) taken for $\gamma=0$, the boundary
conditions (24-26) and (39), without deriving explicitly the expressions of
$A$ and $B$. This is shown in Appendix G.

Alternatively, changing $\alpha$ to $-\alpha$ in (40) also changes
$\Omega^{c}$ to $-\Omega^{c}$. This can be also derived directly from (17-18),
taken for $\gamma=0$, the boundary conditions (24-26), and (39), by changing
$A$ to $-A$ (or $B$ to $-B$).

We check that for an isotropic permeability, $\mu=0$, $\Omega^{c}$ is the same
as that given in Plunian & Alboussière (2020). In figure 4 the curves of the
dynamo threshold $\Omega^{c}$ versus $k$ are plotted for $\mu=0$ (isotropic
magnetic permeability), $\alpha=0.16\pi$ and different values of $\sigma$. The
negative values of $\sigma$ correspond to an electrical conductivity that is
the highest in the direction parallel to q.

In figure 5 the curves of the dynamo threshold $\Omega^{c}$ versus $k$ are
plotted for $\sigma=10^{5}$, $\alpha=0.16\pi$ and different values of $\mu$.
For $\mu=0$, the minimum value of $|\Omega^{c}|$ is obtained for $k=1.1$ and
$\alpha=0.16\pi$, and is equal to $\min_{k,\alpha}|\Omega^{c}|=14.61$ (Plunian
& Alboussière, 2020). For positive values of $\mu$, $|\Omega^{c}|$ increases
with $\mu$, showing the detrimental effect of having both a high $\sigma$ and
a high $\mu$. For negative values of $\mu$, $|\Omega^{c}|$ decreases with
$|\mu|$, showing that the dynamo effect is favoured if the permeability is
higher in the direction parallel to q.

For $s=0$ (radial q) or $c=0$ (azimuthal q), the dynamo is impossible. This is
obvious for $s=0$ as the threshold given by (40) is infinite. For $c=0$, (31)
implies that $B=0$. In addition, as $s^{2}=1$, (20) implies that
$k_{\sigma}=k_{\mu}=k$. Then, from (30) and (32), we find that $A=0$.

###  5.2 Current density

Concerning the current density ${\bf J}$, given at the threshold by (36-38),
we note that it only depends on $\sigma$, and not on $\mu$. In other words,
taking an anisotropic magnetic permeability $\mu\neq 0$ does not change the
geometry of the current density with respect to the isotropic case $\mu=0$.

For an isotropic conductivity $\sigma=0$, we find that ${\bf J}_{\theta}=0$.
This corresponds to the neutral point argument of Cowling (1934), after which
a toroidal current density cannot be produced if axisymmetry is assumed.
However, and although such a neutral point argument is satisfied for
$\sigma=0$, this does not exclude the possibility of dynamo action for an
anisotropic magnetic permeability $\mu\neq 0$.

From (37), we note that the projection in the ($r$,$\theta$) plane of the
current density ${\bf J}$ describes spiralling trajectories. In the limit
$\sigma\rightarrow+\infty$, we find that ${\bf J}\cdot\textit{{q}}=0$.

###  5.3 Magnetic induction

From the expression of ${\bf B}$ given in (16), and applying (30), (31) and
(77), leads to the following expressions for the magnetic induction components

$\displaystyle{\bf B}_{r}=$
$\displaystyle-s\left\\{\begin{split}r<1,&\;\;\;\lambda_{\sigma}\frac{I_{1}(k_{\sigma}r)}{I_{1}(k_{\sigma})}+\lambda_{\mu}\frac{I_{1}(k_{\mu}r)}{I_{1}(k_{\mu})}\\\
r>1,&\;\;\;\lambda_{\sigma}\frac{K_{1}(k_{\sigma}r)}{K_{1}(k_{\sigma})}+\lambda_{\mu}\frac{K_{1}(k_{\mu}r)}{K_{1}(k_{\mu})}\end{split}\right.,$
(43) $\displaystyle{\bf B}_{\theta}=$ $\displaystyle
c\left\\{\begin{split}r<1,&\;\;\;\lambda_{\sigma}\frac{I_{1}(k_{\sigma}r)}{I_{1}(k_{\sigma})}+\lambda_{\mu}\left(\frac{\mu
s^{2}}{1+\mu s^{2}}\right)\frac{I_{1}(k_{\mu}r)}{I_{1}(k_{\mu})}\\\
r>1,&\;\;\;\lambda_{\sigma}\frac{K_{1}(k_{\sigma}r)}{K_{1}(k_{\sigma})}+\lambda_{\mu}\left(\frac{\mu
s^{2}}{1+\mu
s^{2}}\right)\frac{K_{1}(k_{\mu}r)}{K_{1}(k_{\mu})}\end{split}\right.,$ (44)
$\displaystyle{\bf B}_{z}=$
$\displaystyle-\frac{\mathrm{i}s}{k}\left\\{\begin{split}r<1,&\;\;\;\lambda_{\sigma}k_{\sigma}\frac{I_{0}(k_{\sigma}r)}{I_{1}(k_{\sigma})}+\lambda_{\mu}k_{\mu}\frac{I_{0}(k_{\mu}r)}{I_{1}(k_{\mu})}\\\
r>1,&\;\;\;-\lambda_{\sigma}k_{\sigma}\frac{K_{0}(k_{\sigma}r)}{K_{1}(k_{\sigma})}-\lambda_{\mu}k_{\mu}\frac{K_{0}(k_{\mu}r)}{K_{1}(k_{\mu})}\end{split}\right.,$
(45)

where, again, the coefficient $\exp(\mathrm{i}kz)$ has been dropped for
convenience. In contrast to ${\bf J}$, the induction field depends not only on
$\sigma$, but also on $\mu$, implying the following remarks.

In the case of identical anisotropic conductivity and permeability,
$\sigma=\mu$, as mentioned earlier the dynamo is impossible. From (20) and
(32) we have $k_{\sigma}=k_{\mu}$ and $\lambda_{\sigma}+\lambda_{\mu}=0$,
implying that ${\bf B}_{r}={\bf B}_{z}=0$. In that case the induction field
${\bf B}$ is then purely toroidal. This is in agreement with the antidynamo
theorem of Kaiser et al. (1994), after which an invisible dynamo, with a
purely toroidal magnetic field, is impossible.

In the limit $\mu\rightarrow\infty$, from (43) and (44) we have $c{\bf
B}_{r}=-s{\bf B}_{\theta}$, implying that ${\bf B}\cdot\textit{{q}}=0$. The
projection in the ($r$,$\theta$) plane of the induction field ${\bf B}$ thus
describes spiralling trajectories perpendicular to q.

In figure 6 the current lines of ${\bf B}$ and ${\bf J}$ are plotted in the
horizontal plane for different values of $(\sigma,\mu)$. In figures 6a, 6b and
6c, $\sigma=10^{5}$ and $\mu\in\\{-0.99;0;5\\}$. The current lines of ${\bf
J}$ are identical because, as previously seen, ${\bf J}$ does not depend on
$\mu$. From 6a to 6c, increasing $\mu$ has the effect of distorting the ${\bf
B}$ current lines in the outer cylinder, such that the current lines of ${\bf
B}$ reach the same curvature as the current lines of ${\bf J}$, which
eventually is detrimental to dynamo action. In figures 6d, 6e and 6f,
$\mu=10^{5}$ and $\sigma\in\\{-0.99;0;10\\}$. From 6d to 6f, increasing
$\sigma$ has the effect of distorting the ${\bf J}$ current lines in both
inner and outer cylinders, such that the current lines of ${\bf J}$ reach the
same curvature direction as the current lines of ${\bf B}$, which ultimately
is again detrimental to the dynamo action. We note that the ${\bf J}$ current
lines in figures 6a, 6b and 6c and the ${\bf B}$ current lines in figures 6d,
6e and 6f are identical. This is because $\sigma\gg 1$ in figures 6a, 6b and
6c, implying ${\bf J}\cdot\textit{{q}}\approx 0$, and $\mu\gg 1$ in figures
6d, 6e and 6f, implying ${\bf B}\cdot\textit{{q}}\approx 0$.

Figure 6: Current lines of ${\bf B}$ (solid lines) and ${\bf J}$ (dashed
lines) in the horizontal plane for $\alpha=0.16\pi$, $k=1.1$, and for (a)
$(\sigma,\mu)=(10^{5},-0.99)$, (b) $(\sigma,\mu)=(10^{5},0)$, (c)
$(\sigma,\mu)=(10^{5},5)$, (d) $(\sigma,\mu)=(-0.99,10^{5})$, (e)
$(\sigma,\mu)=(0,10^{5})$, (f) $(\sigma,\mu)=(10,10^{5})$.

##  6 Dynamo mechanism

The set of equations (17-18) can be rewritten in terms of ${\bf B}_{r}$ and
${\bf B}_{\theta}$ as

$\displaystyle\gamma{\bf B}_{r}$ $\displaystyle=$ $\displaystyle
cs(\sigma-\mu)k^{2}{\bf B}_{\theta}-(1+\sigma
s^{2})D_{\tilde{k}_{\sigma\mu}}({\bf B}_{r})$ (46) $\displaystyle\gamma{\bf
B}_{\theta}$ $\displaystyle=$ $\displaystyle cs(\sigma-\mu)D_{k}({\bf
B}_{r})-(1+\mu s^{2})D_{\tilde{k}_{\mu\sigma}}({\bf B}_{\theta}),$ (47)

with

$\tilde{k}_{\sigma\mu}=k\left(1+\frac{\mu c^{2}}{1+\sigma
s^{2}}\right)^{1/2},\;\;\;\;\;\tilde{k}_{\mu\sigma}=k\left(1+\frac{\sigma
c^{2}}{1+\mu s^{2}}\right)^{1/2}.$ (48)

On the right hand side of each equation (46) and (47), the first term is a
source term for the dynamo effect, while the second term is a decay term. In
(46), resp. (47), the term $cs(\sigma-\mu)k^{2}{\bf B}_{\theta}$, resp.
$cs(\sigma-\mu)D_{k}({\bf B}_{r})$, corresponds to the generation of ${\bf
B}_{r}$ from ${\bf B}_{\theta}$, resp. ${\bf B}_{\theta}$ from ${\bf B}_{r}$.
The differential rotation between the inner and outer cylinders also
participates in the generation of ${\bf B}_{\theta}$ from ${\bf B}_{r}$,
through the boundary condition (39). The latter is, however, not sufficient in
itself. Therefore, it is clear why increasing the value of $|\sigma-\mu|$
helps for the dynamo effect, and why the dynamo is impossible for
$\sigma=\mu$. Dynamo action thus occurs through differential rotation
conjugated to anisotropic diffusion.

From the point of view of basic Maxwell and Ohm equations, and in the case of
an isotropic conductivity ($\sigma=0$) and anisotropic magnetic permeability
($\mu\neq 0$), dynamo action can be understood in the following way. Suppose
there exists an axisymmetric magnetic induction disturbance with a non-zero
radial component ${\bf B}_{r}$ at some height $z$ along the shear zone
($r=1$). Ohm’s law (10) then drives two opposite currents in the axial
direction ${\bf e}_{z}$ within the rotor and stator. In a medium of isotropic
electrical conductivity, this current forms closed loops in the meridian
planes, as can be seen in Fig. 6(e). From Ampère’s law (7), this generates an
azimuthal magnetic field ${\bf H}_{\theta}$. Finally from (6), a radial
component of the induction vector ${\bf B}_{r}$ is generated from ${{\bf
H}_{\theta}}$ because of the anisotropic magnetic permeability. Depending on
the orientation of the anisotropic permeability tensor and the direction of
the solid rotation, the generated ${\bf B}_{r}$ can either reinforce (dynamo
action is possible) or oppose the initial seed of radial magnetic induction
(no dynamo).

##  7 Conclusions

For an anisotropic electrical conductivity
($\sigma^{\perp}\neq\sigma^{\parallel}$) conjugated to an anisotropic magnetic
permeability ($\mu^{\perp}\neq\mu^{\parallel}$), we could think that
maximizing the ratio
$(\sigma^{\perp}\mu^{\perp})/(\sigma^{\parallel}\mu^{\parallel})$ could help
for the dynamo action. This would correspond to minimizing magnetic
diffusivity in the perpendicular direction relative to that in the parallel
direction. This is not true for two reasons. First, contrary to the isotropic
case, defining an anisotropic magnetic diffusivity is meaningless, because the
electrical conductivity and magnetic permeability are now tensors. Second, it
has been shown that taking $\sigma^{\perp}/\sigma^{\parallel}\gg 1$ and
$\mu^{\perp}/\mu^{\parallel}\gg 1$ is in fact highly detrimental to the dynamo
effect, these two conditions having the effect of aligning the current lines
of respectively the current density ${\bf J}$ and magnetic induction ${\bf B}$
in the same direction $\textit{{q}}^{\perp}$. In contrast, having
$\sigma^{\perp}/\sigma^{\parallel}\gg 1$ and $\mu^{\perp}/\mu^{\parallel}=1$,
or $\sigma^{\perp}/\sigma^{\parallel}=1$ and $\mu^{\perp}/\mu^{\parallel}\gg
1$ leads to the same dynamo threshold $|\Omega^{c}|=14.61$, for $k=1.1$ and
$\alpha=0.16\pi$.

As an application let us consider an experimental demonstration of the dynamo
effect based on such conductivity and permeability spiral anisotropy, with
differential rotation between two cylinders, as sketched in figure 1. An
anisotropic conductivity, resp. permeability, can be manufactured by
alternating thin layers of two materials with different conductivities, resp.
permeabilities. Although the resulting medium is no longer axisymmetric, our
model is still a good approximation of such an experiment. To realize the
first case $\sigma^{\perp}/\sigma^{\parallel}\gg 1$ and
$\mu^{\perp}/\mu^{\parallel}=1$, we can alternate spiral layers of a high
electrical conductivity material, e.g. copper, and a material which is
electrically insulating, e.g. epoxy resin, both having a relative magnetic
permeability equal to unity. To realize the second case
$\sigma^{\perp}/\sigma^{\parallel}=1$ and $\mu^{\perp}/\mu^{\parallel}\gg 1$ ,
we can alternate spiral layers of a high magnetic permeability material, e.g.
$\mu$-metal (permalloy), and a material with a relative magnetic permeability
equal to unity, e.g. stainless steel, both having approximately the same
electrical conductivity. The current lines of ${\bf B}$ and ${\bf J}$ of these
two cases are illustrated in figure 6b and 6e. For the second case, a crucial
issue will be to guarantee a good electrical contact between both materials,
$\mu$-metal and stainless steel. Indeed, if this is not the case, this would
correspond to having $\sigma^{\perp}/\sigma^{\parallel}\gg 1$ and
$\mu^{\perp}/\mu^{\parallel}\gg 1$ which, again, would be highly detrimental
to the dynamo effect.

In the case of an isotropic electrical conductivity,
$\sigma^{\perp}/\sigma^{\parallel}=1$, and as illustrated in figure 6e, the
azimuthal current density is null, ${\bf J}_{\theta}=0$ , which is in
agreement with the neutral point argument of Cowling. As ${\bf
J}=\nabla\times{\bf H}$, this implies that the circulation of the poloidal
component of ${\bf H}$ on a closed current line is zero (Cowling, 1934).
However, as shown in (49), in the case of an anisotropic magnetic permeability
this does not imply that the poloidal component of ${\bf B}$ is zero.
Therefore, although the neutral point argument of Cowling still holds, it does
not imply the impossibility of a dynamo effect.

## Acknowledgements

We thank R. Deguen for fruitful discussions on Earth’s inner core anisotropy.

##  Appendix A Derivation of $\nabla\times({\bf U}\times{\bf B})=0$

Assuming axisymmetry ($\partial_{\theta}=0$), the curl of the cross product of
${\bf U}=r\Omega{\bf e}_{\theta}$ and ${\bf
B}=\left(B_{r},B_{\theta},B_{z}\right)$ is given by $\nabla\times({\bf
U}\times{\bf B})=\left(\partial_{z}(r\Omega B_{z})+\partial_{r}(r\Omega
B_{r})\right){\bf e}_{\theta}$. Assuming that $\Omega$ is constant in space
and using the solenoidality of ${\bf B}$, $\nabla\cdot{\bf B}=0$, leads to
$\nabla\times({\bf U}\times{\bf B})=0$.

##  Appendix B Derivation of (17) and (18)

The product of $[\mu_{ij}]^{-1}=\begin{pmatrix}1+\mu c^{2}&\mu cs&0\\\ \mu
cs&1+\mu s^{2}&0\\\ 0&0&1\end{pmatrix}$ given by (13), and the induction field
${\bf B}=\begin{pmatrix}-\mathrm{i}kA\\\ B\\\
\frac{1}{r}\partial_{r}(rA)\end{pmatrix}\exp(\gamma t+\mathrm{i}kz)$ given by
(16), leads to the magnetic field

${\bf H}=[\mu_{ij}]^{-1}{\bf B}=\begin{pmatrix}-\mathrm{i}k(1+\mu c^{2})A+\mu
csB\\\ -\mathrm{i}k\mu csA+(1+\mu s^{2})B\\\
\frac{1}{r}\partial_{r}(rA)\end{pmatrix},$ (49)

where, from now, the exponential term is dropped for convenience. Assuming
axisymmetry, the curl of ${\bf H}$ takes the form $\nabla\times{\bf
H}=\begin{pmatrix}-\mathrm{i}kH_{\theta}\\\
\mathrm{i}kH_{r}-\partial_{r}H_{z}\\\
\frac{1}{r}\partial_{r}(rH_{\theta})\end{pmatrix}$, leading to the current
density

${\bf J}=\nabla\times{\bf H}=\begin{pmatrix}-k^{2}\mu csA-\mathrm{i}k(1+\mu
s^{2})B\\\ D_{k}(A)+\mu c^{2}k^{2}A+\mathrm{i}\mu cskB\\\ -\mathrm{i}\mu
csk\frac{1}{r}\partial_{r}(rA)+(1+\mu
s^{2})\frac{1}{r}\partial_{r}(rB)\end{pmatrix},$ (50)

where
$D_{\nu}(X)=\nu^{2}X-\partial_{r}\left(\frac{1}{r}\partial_{r}(rX)\right)$.
The product of $[\sigma_{ij}]^{-1}=\begin{pmatrix}1+\sigma c^{2}&\sigma
cs&0\\\ \sigma cs&1+\sigma s^{2}&0\\\ 0&0&1\end{pmatrix}$ given by (12), and
${\bf J}$ given by (50), leads to

$[\sigma_{ij}]^{-1}{\bf J}=\begin{pmatrix}-k^{2}\mu csA+\sigma
csD_{k}(A)-\mathrm{i}k(1+\sigma c^{2}+\mu s^{2})B\\\ \mu c^{2}k^{2}A+(1+\sigma
s^{2})D_{k}(A)-\mathrm{i}csk(\sigma-\mu)B\\\ -\mathrm{i}\mu
csk\frac{1}{r}\partial_{r}(rA)+(1+\mu
s^{2})\frac{1}{r}\partial_{r}(rB)\end{pmatrix}.$ (51)

Taking the curl leads to $\nabla\times[\sigma_{ij}]^{-1}{\bf
J}=\begin{pmatrix}-\mathrm{i}kF\\\ G\\\
\frac{1}{r}\partial_{r}(rF)\end{pmatrix},$ with

$\displaystyle F$ $\displaystyle=$ $\displaystyle\mu c^{2}k^{2}A+(1+\sigma
s^{2})D_{k}(A)-\mathrm{i}kcs(\sigma-\mu)B,$ (52) $\displaystyle G$
$\displaystyle=$ $\displaystyle\mathrm{i}kcs(\sigma-\mu)D_{k}(A)+\sigma
k^{2}c^{2}B+(1+\mu s^{2})D_{k}(B).\;\;\;\;\;\;\;$ (53)

As $\nabla\times({\bf U}\times{\bf B})=0$, the induction equation (11) is
reduced to $\partial_{t}{\bf B}=-\nabla\times[\sigma_{ij}]^{-1}{\bf J}$,
leading to

$\displaystyle\gamma A$ $\displaystyle=$ $\displaystyle-F,$ (54)
$\displaystyle\gamma B$ $\displaystyle=$ $\displaystyle-G,$ (55)
$\displaystyle\frac{\gamma}{r}\partial_{r}(rA)$ $\displaystyle=$
$\displaystyle-\frac{1}{r}\partial_{r}(rF),$ (56)

and then to (17) and (18).

##  Appendix C Derivation of the fourth-order differential equation (19)
satisfied by $A$ and $B$ at the dynamo threshold

Replacing $\gamma=0$ in (17) and (18) leads to the following system

$\displaystyle(1+\sigma s^{2})D_{k}(A)+\mu c^{2}k^{2}A$ $\displaystyle=$
$\displaystyle\mathrm{i}csk(\sigma-\mu)B,$ (57) $\displaystyle(1+\mu
s^{2})D_{k}(B)+\sigma c^{2}k^{2}B$ $\displaystyle=$
$\displaystyle-\mathrm{i}csk(\sigma-\mu)D_{k}(A),\;\;\;\;\;$ (58)

where, again,
$D_{\nu}(X)=\nu^{2}X-\partial_{r}\left(\frac{1}{r}\partial_{r}(rX)\right)$.

It is straightforward to show that

$\displaystyle(1+\sigma s^{2})D_{k}(X)=(1+\sigma
s^{2})D_{k_{\sigma}}(X)-\sigma c^{2}k^{2}X,\;\;\;\;\;$ (59)
$\displaystyle(1+\mu s^{2})D_{k}(X)=(1+\mu s^{2})D_{k_{\mu}}(X)-\mu
c^{2}k^{2}X,\;\;\;\;\;$ (60)

where $k_{\sigma}$ and $k_{\mu}$ are defined in (20) and that we rewrite here
for convenience

$k_{\sigma}=k\left(\frac{1+\sigma}{1+\sigma
s^{2}}\right)^{1/2},\;\;\;\;\;k_{\mu}=k\left(\frac{1+\mu}{1+\mu
s^{2}}\right)^{1/2}.$

Using (59) and (60) in (57) and (58) then leads to

$\displaystyle(1+\sigma s^{2})D_{k_{\sigma}}(A)$ $\displaystyle=$
$\displaystyle ck(\sigma-\mu)(ckA+\mathrm{i}sB),$ (61) $\displaystyle(1+\mu
s^{2})D_{k_{\mu}}(B)$ $\displaystyle=$ $\displaystyle-
ck(\sigma-\mu)(ckB+\mathrm{i}sD_{k}(A)).\;\;\;\;\;$ (62)

Then to obtain (19) we need to demonstrate that
$D_{k_{\mu}}(ckA+\mathrm{i}sB)=0$ and
$D_{k_{\sigma}}(ckB+\mathrm{i}sD_{k}(A))=0$. For that, we rewrite (57) and
(58) as

$\displaystyle D_{k}(A)$ $\displaystyle=$
$\displaystyle\sigma\left(-s^{2}D_{k}(A)+\mathrm{i}cskB\right)-\mu\left(c^{2}k^{2}A+\mathrm{i}cskB\right),$
(63) $\displaystyle D_{k}(B)$ $\displaystyle=$
$\displaystyle-\sigma\left(c^{2}k^{2}B+\mathrm{i}cskD_{k}(A)\right)+\mu
D_{k}\left(-s^{2}B+\mathrm{i}cskA\right).$ (64)

Multiplying (63) by $ck$, (64) by $\mathrm{i}s$, and adding both quantities
leads to

$(1+\mu s^{2})D_{k}(ckA+\mathrm{i}sB)=-\mu c^{2}k^{2}(ckA+\mathrm{i}sB),$ (65)

which, from (60) with $X=ckA+\mathrm{i}sB$, is equivalent to

$D_{k_{\mu}}(ckA+\mathrm{i}sB)=0.$ (66)

Applying (66) to (61) then leads to

$\left(D_{k_{\mu}}\circ D_{k_{\sigma}}\right)(A)=0.$ (67)

Taking $D_{k}$ of (63) multiplied by $\mathrm{i}s$ on the one hand, and (64)
multiplied by $ck$ on the other hand, and adding both quantities leads to

$(1+\sigma s^{2})D_{k}(ckB+\mathrm{i}sD_{k}(A))=-\sigma
c^{2}k^{2}(ckB+\mathrm{i}sD_{k}(A)),$ (68)

which, from (59) with $X=ckB+\mathrm{i}sD_{k}(A)$, is equivalent to

$D_{k_{\sigma}}(ckB+\mathrm{i}sD_{k}(A))=0.$ (69)

Applying (69) to (62) leads to

$\left(D_{k_{\sigma}}\circ D_{k_{\mu}}\right)(B)=0,$ (70)

which, together with (67), corresponds to (19).

##  Appendix D Derivation of $B$, given in (23), at the dynamo threshold

Starting from (22), which we rewrite here as

$\displaystyle A=$
$\displaystyle\left\\{\begin{split}r<1,&\;\;\;\;\;\alpha_{\sigma}I_{1}(k_{\sigma}r)+\alpha_{\mu}I_{1}(k_{\mu}r)&\\\
r>1,&\;\;\;\;\;\beta_{\sigma}K_{1}(k_{\sigma}r)+\beta_{\mu}K_{1}(k_{\mu}r)&\end{split}\right.,$

we will derive $B$ from (17), which we write here for $\gamma=0$ as

$B=\left((1+\sigma s^{2})D_{k}(A)+\mu
c^{2}k^{2}A\right)/\left(\mathrm{i}csk(\sigma-\mu)\right).\\\ $ (71)

Using the relations (59) and (60), and knowing that, whatever $\nu$,
$D_{\nu}(I_{1}(k_{\nu}r))=D_{\nu}(K_{1}(k_{\nu}r))=0$ we find that

$\displaystyle D_{k}(A)=\left\\{\begin{split}r<1,&\;\;\;-\frac{\sigma
c^{2}k^{2}}{1+\sigma s^{2}}\alpha_{\sigma}I_{1}(k_{\sigma}r)-\frac{\mu
c^{2}k^{2}}{1+\mu s^{2}}\alpha_{\mu}I_{1}(k_{\mu}r)&\\\
r>1,&\;\;\;-\frac{\sigma c^{2}k^{2}}{1+\sigma
s^{2}}\beta_{\sigma}K_{1}(k_{\sigma}r)-\frac{\mu c^{2}k^{2}}{1+\mu
s^{2}}\beta_{\mu}K_{1}(k_{\mu}r)&\end{split}\right..$ (72)

Then, replacing in (71) the expressions of $A$ and $D_{k}(A)$ given by (22)
and (72) leads to the following expression for $B$, which is also given in
(23):

$\displaystyle B=$
$\displaystyle\left\\{\begin{split}r<1,&\;\;\;\frac{\mathrm{i}ck}{s}\left(\alpha_{\sigma}I_{1}(k_{\sigma}r)+\frac{\mu
s^{2}}{1+\mu s^{2}}\alpha_{\mu}I_{1}(k_{\mu}r)\right)&\\\
r>1,&\;\;\;\frac{\mathrm{i}ck}{s}\left(\beta_{\sigma}K_{1}(k_{\sigma}r)+\frac{\mu
s^{2}}{1+\mu s^{2}}\beta_{\mu}K_{1}(k_{\mu}r)\right)&\end{split}\right..$

##  Appendix E Derivation of the boundary conditions (27), (28) and (29) at
the dynamo threshold

The continuity of $A$ and $B$ at $r=1$, taken from their expressions (22) and
(23), takes the following form:

$\displaystyle\alpha_{\sigma}I_{1}(k_{\sigma})+\alpha_{\mu}I_{1}(k_{\mu})$
$\displaystyle=$
$\displaystyle\beta_{\sigma}K_{1}(k_{\sigma})+\beta_{\mu}K_{1}(k_{\mu})$ (73)
$\displaystyle\alpha_{\sigma}I_{1}(k_{\sigma})+\frac{\mu s^{2}}{1+\mu
s^{2}}\alpha_{\mu}I_{1}(k_{\mu})$ $\displaystyle=$
$\displaystyle\beta_{\sigma}K_{1}(k_{\sigma})+\frac{\mu s^{2}}{1+\mu
s^{2}}\beta_{\mu}K_{1}(k_{\mu})$ (74)

leading to (27) and (28), which we rewrite here as

$\displaystyle\alpha_{\sigma}I_{1}(k_{\sigma})-\beta_{\sigma}K_{1}(k_{\sigma})$
$\displaystyle=$ $\displaystyle 0$
$\displaystyle\alpha_{\mu}I_{1}(k_{\mu})-\beta_{\mu}K_{1}(k_{\mu})$
$\displaystyle=$ $\displaystyle 0.$

To write the continuity of $\partial_{r}A$ at $r=1$ we first need to calculate
the expression of $\partial_{r}A$ at any $r$. Using the following relations
satisfied whatever $\nu$:

$\displaystyle\partial_{r}\left(I_{1}(\nu r)\right)$ $\displaystyle=$
$\displaystyle\nu I_{0}(\nu r)-\frac{1}{r}I_{1}(\nu r),$ (75)
$\displaystyle\partial_{r}\left(K_{1}(\nu r)\right)$ $\displaystyle=$
$\displaystyle-\nu K_{0}(\nu r)-\frac{1}{r}K_{1}(\nu r),$ (76)

the expression of $\partial_{r}A$ is obtained by deriving (22):

$\displaystyle\partial_{r}A=$
$\displaystyle\left\\{\begin{split}r<1,&\;\;\;\;\;\alpha_{\sigma}\left(k_{\sigma}I_{0}(k_{\sigma}r)-\frac{1}{r}I_{1}(k_{\sigma}r)\right)+\alpha_{\mu}\left(k_{\mu}I_{0}(k_{\mu}r)-\frac{1}{r}I_{1}(k_{\mu}r)\right)&\\\
r>1,&\;\;\;\;\;\beta_{\sigma}\left(-k_{\sigma}K_{0}(k_{\sigma}r)-\frac{1}{r}K_{1}(k_{\sigma}r)\right)+\beta_{\mu}\left(-k_{\mu}K_{0}(k_{\mu}r)-\frac{1}{r}K_{1}(k_{\mu}r)\right)&\end{split}\right..\;\;\;\;\;$
(77)

Then, the continuity of $\partial_{r}A$ at $r=1$ leads to

$\displaystyle\alpha_{\sigma}\left(k_{\sigma}I_{0}(k_{\sigma})-I_{1}(k_{\sigma})\right)+\alpha_{\mu}\left(k_{\mu}I_{0}(k_{\mu})-I_{1}(k_{\mu})\right)=$
$\displaystyle\beta_{\sigma}\left(-k_{\sigma}K_{0}(k_{\sigma})-K_{1}(k_{\sigma})\right)+\beta_{\mu}\left(-k_{\mu}K_{0}(k_{\mu})-K_{1}(k_{\mu})\right).$
(78)

Then, taking advantage of (27) and (28), (78) can be simplified to

$\alpha_{\sigma}k_{\sigma}I_{0}(k_{\sigma})+\alpha_{\mu}k_{\mu}I_{0}(k_{\mu})+\beta_{\sigma}k_{\sigma}K_{0}(k_{\sigma})+\beta_{\mu}k_{\mu}K_{0}(k_{\mu})=0,$

which is (29).

##  Appendix F Derivation of the current density ${\bf J}$ at the dynamo
threshold

We rewrite the current density ${\bf J}$ which is given in (50) as

${\bf J}=\nabla\times{\bf H}=\begin{pmatrix}-\mathrm{i}k\phi\\\ D_{k}(A)+\mu
c^{2}k^{2}A+\mathrm{i}\mu cskB\\\
\frac{1}{r}\partial_{r}\left(r\phi\right)\end{pmatrix},$

with $\phi=-\mathrm{i}k\mu csA+(1+\mu s^{2})B$. At the dynamo threshold $A$
and $B$ can be replaced by their expressions (22) and (23), leading to

$\displaystyle\phi=$
$\displaystyle\left\\{\begin{split}r<1,&\;\;\;\frac{\mathrm{i}ck}{s}\alpha_{\sigma}I_{1}(k_{\sigma}r)&\\\
r>1,&\;\;\;\frac{\mathrm{i}ck}{s}\beta_{\sigma}K_{1}(k_{\sigma}r)&\end{split}\right..$
(79)

Using the relations (75) and (76) leads to

$\displaystyle\frac{1}{r}\partial_{r}\left(r\phi\right)=$
$\displaystyle\left\\{\begin{split}r<1,&\;\;\;\frac{\mathrm{i}ck}{s}\alpha_{\sigma}k_{\sigma}I_{0}(k_{\sigma}r)&\\\
r>1,&\;\;\;-\frac{\mathrm{i}ck}{s}\beta_{\sigma}k_{\sigma}K_{0}(k_{\sigma}r)&\end{split}\right..$
(80)

Using (60), we find that

$D_{k}(A)+\mu c^{2}k^{2}A+\mathrm{i}\mu
cskB=D_{k_{\mu}}(A)+\frac{\mathrm{i}\mu csk}{1+\mu s^{2}}\phi.$ (81)

Then from the expression of $A$ given at the threshold by (22), we have

$\displaystyle D_{k_{\mu}}(A)=$
$\displaystyle\left\\{\begin{split}r<1,&\;\;\;\alpha_{\sigma}D_{k_{\mu}}(I_{1}(k_{\sigma}r))&\\\
r>1,&\;\;\;\beta_{\sigma}D_{k_{\mu}}(K_{1}(k_{\sigma}r))&\end{split}\right..$
(82)

Combining (59) and (60) we have

$D_{k_{\mu}}(X)=D_{k_{\sigma}}(X)+c^{2}k^{2}\left(\frac{\mu}{1+\mu
s^{2}}-\frac{\sigma}{1+\sigma s^{2}}\right)X,\;\;\;\;\;$ (83)

implying that

$\displaystyle D_{k_{\mu}}(A)=$
$\displaystyle\left\\{\begin{split}r<1,&\;\;\;\alpha_{\sigma}c^{2}k^{2}\left(\frac{\mu}{1+\mu
s^{2}}-\frac{\sigma}{1+\sigma s^{2}}\right)I_{1}(k_{\sigma}r)&\\\
r>1,&\;\;\;\beta_{\sigma}c^{2}k^{2}\left(\frac{\mu}{1+\mu
s^{2}}-\frac{\sigma}{1+\sigma
s^{2}}\right)K_{1}(k_{\sigma}r)&\end{split}\right.,$ (84)

where, again, we used the property that, whatever $\nu$,
$D_{\nu}(I_{1}(k_{\nu}r))=D_{\nu}(K_{1}(k_{\nu}r))=0$. Therefore we find that

$\displaystyle D_{k_{\mu}}(A)+\frac{\mathrm{i}\mu csk}{1+\mu s^{2}}\phi=$
$\displaystyle\left\\{\begin{split}r<1,&\;\;\;-\frac{\sigma
c^{2}k^{2}}{1+\sigma s^{2}}\alpha_{\sigma}I_{1}(k_{\sigma}r)&\\\
r>1,&\;\;\;-\frac{\sigma c^{2}k^{2}}{1+\sigma
s^{2}}\beta_{\sigma}K_{1}(k_{\sigma}r)&\end{split}\right..$ (85)

Then the current density takes the following form

$\mbox{for}\;\;r<1,\;\;{\bf
J}=\begin{pmatrix}\frac{ck^{2}}{s}\alpha_{\sigma}I_{1}(k_{\sigma}r)\\\
-\frac{\sigma c^{2}k^{2}}{1+\sigma s^{2}}\alpha_{\sigma}I_{1}(k_{\sigma}r)\\\
\frac{\mathrm{i}ck}{s}\alpha_{\sigma}k_{\sigma}I_{0}(k_{\sigma}r)\end{pmatrix},$
(86) $\mbox{for}\;\;r>1,\;\;{\bf
J}=\begin{pmatrix}\frac{ck^{2}}{s}\beta_{\sigma}K_{1}(k_{\sigma}r)\\\
-\frac{\sigma c^{2}k^{2}}{1+\sigma s^{2}}\beta_{\sigma}K_{1}(k_{\sigma}r)\\\
-\frac{\mathrm{i}ck}{s}\beta_{\sigma}k_{\sigma}K_{0}(k_{\sigma}r)\end{pmatrix}.$
(87)

Then, substituting $\alpha_{\sigma}$ and $\beta_{\sigma}$ by their expressions
in terms of $\lambda_{\sigma}$,
$\alpha_{\sigma}=-\mathrm{i}s\lambda_{\sigma}/(kI_{1}(k_{\sigma}))$ and
$\beta_{\sigma}=-\mathrm{i}s\lambda_{\sigma}/(kK_{1}(k_{\sigma}))$, leads to
(36-38).

##  Appendix G Derivation of the antisymmetric relation (41)

Let us rewrite the set of equations (17-18) for $\gamma=0$, the boundary
conditions (24-26) and (39):

$\displaystyle(1+\sigma s^{2})D_{k}(A)+\mu c^{2}k^{2}A$ $\displaystyle=$
$\displaystyle\mathrm{i}csk(\sigma-\mu)B,$ (88) $\displaystyle(1+\mu
s^{2})D_{k}(B)+\sigma c^{2}k^{2}B$ $\displaystyle=$
$\displaystyle-\mathrm{i}csk(\sigma-\mu)D_{k}(A),$ (89)
$\displaystyle\left[A\right]^{r=1^{+}}_{r=1^{-}}=\left[B\right]^{r=1^{+}}_{r=1^{-}}=\left[\partial_{r}A\right]^{r=1^{+}}_{r=1^{-}}$
$\displaystyle=$ $\displaystyle 0,$ (90) $\displaystyle(1+\mu
s^{2})\left[\partial_{r}B\right]^{r=1^{+}}_{r=1^{-}}$ $\displaystyle=$
$\displaystyle-\mathrm{i}k\Omega A(r=1^{-}),$ (91)

where $\left[X\right]^{r=1^{+}}_{r=1^{-}}=X(r=1^{+})-X(r=1^{-})$, and (91)
being derived from (39) using (50). The system (88-91) is the complete system
of equations leading to the dynamo threshold (40).

Now, defining the new variables $A^{\prime}$ and $B^{\prime}$ as

$\displaystyle A^{\prime}$ $\displaystyle=$ $\displaystyle-\frac{1+\sigma
s^{2}}{1+\mu s^{2}}A,$ (92) $\displaystyle B^{\prime}$ $\displaystyle=$
$\displaystyle B-\mathrm{i}k\frac{c}{s}\left(\frac{1+\sigma s^{2}+\mu
s^{2}}{1+\sigma s^{2}}\right)A,$ (93)

and replacing them into (88-91) leads to

$\displaystyle(1+\mu s^{2})D_{k}(A^{\prime})+\sigma c^{2}k^{2}A^{\prime}$
$\displaystyle=$ $\displaystyle\mathrm{i}csk(\mu-\sigma)B^{\prime},$ (94)
$\displaystyle(1+\sigma s^{2})D_{k}(B^{\prime})+\mu c^{2}k^{2}B^{\prime}$
$\displaystyle=$ $\displaystyle-\mathrm{i}csk(\mu-\sigma)D_{k}(A^{\prime}),$
(95)
$\displaystyle\left[A^{\prime}\right]^{r=1^{+}}_{r=1^{-}}=\left[B^{\prime}\right]^{r=1^{+}}_{r=1^{-}}=\left[\partial_{r}A^{\prime}\right]^{r=1^{+}}_{r=1^{-}}$
$\displaystyle=$ $\displaystyle 0,$ (96) $\displaystyle(1+\sigma
s^{2})\left[\partial_{r}B^{\prime}\right]^{r=1^{+}}_{r=1^{-}}$
$\displaystyle=$ $\displaystyle\mathrm{i}k\Omega A^{\prime}(r=1^{-}).$ (97)

It shows that the new variables $A^{\prime}$ and $B^{\prime}$ obey to the same
equations as $A$ and $B$, in which $\sigma$ and $\mu$ have been changed to
$\mu$ and $\sigma$, and $\Omega$ to $-\Omega$.

## References

  * Alboussière et al. (2020) Alboussière, T., Drif, K. & Plunian, F. 2020 Dynamo action in sliding plates of anisotropic electrical conductivity. Physical Review E 101, 033107.
  * Alboussière et al. (2011) Alboussière, T., Cardin, P., Debray, F., La Rizza, P., Masson, J.-P., Plunian, F., Ribeiro, A. & Schmitt, D. 2011 Experimental evidence of Alfvén wave propagation in a gallium alloy. Physics of Fluids 23 (9), 096601.
  * Avalos-Zuñiga & Plunian (2005) Avalos-Zuñiga, R. & Plunian, F. 2005 Influence of inner and outer walls electromagnetic properties on the onset of a stationary dynamo. The European Physical Journal B - Condensed Matter and Complex Systems 47, 127–135.
  * Avalos-Zuñiga et al. (2003) Avalos-Zuñiga, R., Plunian, F. & Gailitis, A. 2003 Influence of electromagnetic boundary conditions onto the onset of dynamo action in laboratory experiments. Physical Review E 68, 066307\.
  * Braginskii (1965) Braginskii, S.I. 1965 Transport Processes in a Plasma. Reviews of Plasma Physics 1, 205.
  * Brandenburg (2018) Brandenburg, A. 2018 Advances in mean-field dynamo theory and applications to astrophysical turbulence. Journal of Plasma Physics 84 (4), 735840404.
  * Busse & Wicht (1992) Busse, F. H. & Wicht, J. 1992 A simple dynamo caused by conductivity variations. Geophysical & Astrophysical Fluid Dynamics 64 (1-4), 135–144.
  * Cowling (1934) Cowling, T. G. 1934 The magnetic field of sunspots. Mon. Not. R. Astr. Soc. 94, 39–48.
  * Deuss (2014) Deuss, A. 2014 Heterogeneity and anisotropy of earth’s inner core. Annual Review of Earth and Planetary Sciences 42 (1), 103–126.
  * Gailitis et al. (2001) Gailitis, A., Lielausis, O., Platacis, E., Dement’ev, S., Cifersons, A., Gerbeth, G., Gundrum, T., Stefani, F., Christen, M. & Will, G. 2001 Magnetic field saturation in the riga dynamo experiment. Physical Review Letters 86, 3024–3027.
  * Kaiser et al. (1994) Kaiser, R., Schmitt, B. J. & Busse, F. H. 1994 On the invisible dynamo. Geophysical & Astrophysical Fluid Dynamics 77 (1-4), 93–109.
  * Kaiser & Tilgner (1999) Kaiser, R. & Tilgner, A. 1999 On vainshtein’s dynamo conjecture. Proceedings of the Royal Society of London. Series A: Mathematical, Physical and Engineering Sciences 455 (1988), 3139–3162.
  * Kaiser & Tilgner (2014) Kaiser, R. & Tilgner, A. 2014 The axisymmetric antidynamo theorem revisited. SIAM Journal on Applied Mathematics 74 (2), 571–597.
  * Krause & Rädler (1980) Krause, F. & Rädler, K. H. 1980 Mean-field magnetohydrodynamics and dynamo theory.
  * Kreuzahler et al. (2017) Kreuzahler, S., Ponty, Y., Plihon, N., Homann, H. & Grauer, R. 2017 Dynamo enhancement and mode selection triggered by high magnetic permeability. Physical Review Letters 119, 234501.
  * Lortz (1989) Lortz, D. 1989 Axisymmetric dynamo solutions. Z. Naturforsch 44a, 1041–1045.
  * Lowes & Wilkinson (1963) Lowes, F. J. & Wilkinson, I. 1963 Geomagnetic dynamo: A laboratory model. Nature 198, 1158–1160.
  * Lowes & Wilkinson (1968) Lowes, F. J. & Wilkinson, I. 1968 Geomagnetic dynamo: An improved laboratory model. Nature 219, 717–718.
  * Miralles et al. (2013) Miralles, S., Bonnefoy, N., Bourgoin, M., Odier, P., Pinton, J.-F., Plihon, N., Verhille, G., Boisson, J., Daviaud, F. & Dubrulle, B. 2013 Dynamo threshold detection in the Von Kármán sodium experiment. Physical Review E 88, 013002.
  * Monchaux et al. (2007) Monchaux, R., Berhanu, M., Bourgoin, M., Moulin, M., Odier, Ph., Pinton, J.-F., Volk, R., Fauve, S., Mordant, N., Pétrélis, F., Chiffaudel, A., Daviaud, F., Dubrulle, B., Gasquet, C., Marié, L. & Ravelet, F. 2007 Generation of a magnetic field by dynamo action in a turbulent flow of liquid sodium. Physical Review Letters 98, 044502\.
  * Nore et al. (2018) Nore, C., Castanon Quiroz, D., Cappanera, L. & Guermond, J.-L. 2018 Numerical simulation of the von kármán sodium dynamo experiment. Journal of Fluid Mechanics 854, 164–195.
  * Ohta et al. (2018) Ohta, K., Nishihara, Y., Sato, Y., Hirose, K., Yagi, T., Kawaguchi, S. I., Hirao, N. & Ohishi, Y. 2018 An experimental examination of thermal conductivity anisotropy in hcp iron. Frontiers in Earth Science 6, 176.
  * Plunian & Alboussière (2020) Plunian, F. & Alboussière, T. 2020 Axisymmetric dynamo action is possible with anisotropic conductivity. Physical Review Research 2, 013321.
  * Rincon (2019) Rincon, F. 2019 Dynamo theories. Journal of Plasma Physics 85 (4), 205850401.
  * Ruderman & Ruzmaikin (1984) Ruderman, M. S. & Ruzmaikin, A. A. 1984 Magnetic field generation in an anisotropically conducting fluid. Geophysical & Astrophysical Fluid Dynamics 28 (1), 77–88.
  * Schaeffer et al. (2017) Schaeffer, N., Jault, D., Nataf, H.-C. & Fournier, A. 2017 Turbulent geodynamo simulations: a leap towards Earth’s core. Geophysical Journal International 211 (1), 1–29.
  * Stieglitz & Müller (2001) Stieglitz, R. & Müller, U. 2001 Experimental demonstration of a homogeneous two-scale dynamo. Physics of Fluids 13 (3), 561–564.
  * Tigrine et al. (2019) Tigrine, Z, Nataf, H-C, Schaeffer, N, Cardin, P & Plunian, F 2019 Torsional alfvén waves in a dipolar magnetic field: experiments and simulations. Geophysical Journal International 219, S83–S100.


:
1 2023 Symmetry and Geometry in Neural Representations

# AMES: A Differentiable Embedding Space Selection Framework for Latent Graph
Inference

Yuan Lu<EMAIL_ADDRESS>
University of Cambridge  Department of Engineering  Haitz Sáez de Ocáriz Borde
<EMAIL_ADDRESS>
University of Oxford  Oxford Robotics Institute  Pietro Liò<EMAIL_ADDRESS>
University of Cambridge  Department of Computer Science & Technology

###### Abstract

In real-world scenarios, although data entities may possess inherent
relationships, the specific graph illustrating their connections might not be
directly accessible. Latent graph inference addresses this issue by enabling
Graph Neural Networks (GNNs) to operate on point cloud data, dynamically
learning the necessary graph structure. These graphs are often derived from a
latent embedding space, which can be modeled using Euclidean, hyperbolic,
spherical, or product spaces. However, currently, there is no principled
differentiable method for determining the optimal embedding space. In this
work, we introduce the Attentional Multi-Embedding Selection (AMES) framework,
a differentiable method for selecting the best embedding space for latent
graph inference through backpropagation, considering a downstream task. Our
framework consistently achieves comparable or superior results compared to
previous methods for latent graph inference across five benchmark datasets.
Importantly, our approach eliminates the need for conducting multiple
experiments to identify the optimal embedding space. Furthermore, we explore
interpretability techniques that track the gradient contributions of different
latent graphs, shedding light on how our attention-based, fully differentiable
approach learns to choose the appropriate latent space. In line with previous
works, our experiments emphasize the advantages of hyperbolic spaces in
enhancing performance. More importantly, our interpretability framework
provides a general approach for quantitatively comparing embedding spaces
across different tasks based on their contributions, a dimension that has been
overlooked in previous literature on latent graph inference.

###### keywords:

Metric Learning, Latent Graph Inference, Neural Latent Geometry Search, Graph
Neural Networks.

††editors: Sophia Sanborn, Christian Shewmake, Simone Azeglio, Nina Miolane

##  1 Introduction

Graph Neural Networks (GNNs) efficiently leverage the geometric prior provided
by the edge connections of graph-structured data through the utilization of an
adjacency matrix. However, in many real-world applications, the graph
connectivity structure may be incomplete, noisy, or even completely unknown.
Utilizing latent graph inference enables the application of GNNs to point
cloud data without requiring an input adjacency matrix, and it facilitates on-
the-fly inference of the optimal graph structure for computation. Numerous
studies have employed the Differentiable Graph Module (DGM) (Kazi et al.,
2022) framework to infer latent graphs. This technique hinges on an embedding
space, which is employed to deduce distances between nodes within the point
cloud. Subsequently, connections are established based on these distances,
ultimately shaping a graph. However, the task of selecting the most suitable
embedding space is far from straightforward. Various approaches have explored
the use of Euclidean space, hyperbolic spaces, spherical spaces, and
stereographic projections of these spaces (Sáez de Ocáriz Borde et al.,
2023b), as well as product spaces (Sáez de Ocáriz Borde et al., 2023c).
Nevertheless, a principled approach for systematically selecting an optimal
embedding space is lacking, particularly in a differentiable fashion. The
latent geometry search procedure often relies on random search, which is
computationally inefficient, particularly for product manifolds due to the
multitude of possible manifold constructions.

In this study, we present an attention-based mechanism for latent graph
inference: the Attentional Multi Embedding Selection (AMES) framework. Our
approach effectively harnesses multiple latent candidate graphs derived from
various latent spaces simultaneously. By employing an attention mechanism, it
selects the most optimal latent representations while remaining
differentiable. Consequently, this results in an end-to-end framework for the
automated selection of latent embedding spaces in the context of latent graph
inference. Importantly, we can achieve the same level of performance as
existing latent graph inference methods with a single model, eliminating the
need for experimental searches to find the optimal latent geometry.
Furthermore, we put forward methods for interpretability that enable us to
monitor the learning process of the selection mechanism and provide insights
into its internal behavior.

##  2 Background

Geometric Deep Learning and Neural Latent Geometry Search. Geometric Deep
Learning (GDL) recognizes that certain real-world problems involve data that
cannot be adequately represented using simple grids or Euclidean vectors, and
aims to generalize traditional machine learning models to a more diverse set
of data types, such as point clouds, graphs, and manifolds, to name a few
(Bronstein et al., 2021). Within the realm of GDL many concepts from
differential geometry and topology have been imported into the machine
learning community to enhance the performance of learning algorithms (De
Bortoli et al., 2022; Hensel et al., 2021; Chamberlain et al., 2021; Huang et
al., 2022). This includes using manifolds such as the Poincaré ball model
(Mathieu et al., 2019), the hyperboloid (Chami et al., 2019), or the
hypersphere (Mettes et al., 2019) to encode latent representations of data
instead of the Euclidean hyperplane. These manifolds provide a means to
represent diverse geometries in a reasonably straightforward and
computationally manageable manner. This is possible due to their well-defined
mathematical expressions for notions like exponential maps and geodesic
distance functions. Despite the promising results obtained from these
approaches, a principled method for selecting the appropriate latent embedding
manifold construction is still lacking. This has led to the formulation of
Neural Latent Geometry Search (NLGS) (Sáez de Ocáriz Borde et al., 2023a),
which can be formalized as follows: given a search space $\mathfrak{G}$
denoting the set of all possible latent geometries, and the objective function
$L_{T,A}(g)$ which evaluates the performance of a given geometry $g$ on a
downstream task $T$ for a machine learning model architecture $A$, the
objective is to find an optimal latent geometry $g^{*}$:
$g^{*}=\textrm{argmin}_{g\in\mathfrak{G}}L_{T,A}(g)$, where in the domain of
latent graph inference, $\mathfrak{G}$ represents the set encompassing
conceivable geometric similarity measures employed for generating latent
graphs.

Latent Graph Inference. Utilizing latent graph inference allows for the
integration of GNNs with point cloud data, potentially enhancing performance
by harnessing the inductive bias of input graphs or acquiring an entirely new
graph from scratch. Some previous contributions have focused on improving
existing graph topology by pre-processing the existing adjacency matrices
(Alon and Yahav, 2021; Gasteiger et al., 2019; Deac et al., 2022). The
emphasis of this study lies in techniques that dynamically learn and adapt the
graph topology without necessitating a rewiring of the initial graph
structure. Numerous methods have been put forth to achieve this dynamic
learning of the underlying graph topology, in conjunction with the
optimization of GNN model parameters: from the original Dynamic Graph
Convolutional Neural Network (DGCNN) (Wang et al., 2019) which generates
graphs dynamically based on the latent feature activations of the network, to
the discrete Differentiable Graph Module (dDGM) (Cosmo et al., 2020; Kazi et
al., 2022) which performs latent graph inference and message passing in
parallel rather than sharing the same embedding space. A comprehensive
overview of comparable methods is available in Zhu et al. (2021), with
classical approaches encompassing LDS-GNN (Franceschi et al., 2019), IDGL
(Chen et al., 2020), and Pro-GNN (Jin et al., 2020).

The Differentiable Graph Module (DGM) is a general end-to-end approach for
learning latent graphs based on similarity measures between latent node
features. Similarity is computed using a distance function that depends on the
geometry of the embedding space. Then, based on the distance, the probability
of there existing an edge between two nodes is computed, assuming a higher
likelihood the closer latent representations of nodes are. Lastly, edge
sampling is performed using the Gumbel Top-k trick. Note that in this work, we
focus on the discrete form of the DGM module, the discrete Differentiable
Graph Module (dDGM), as dDGM produces sparse graphs which are computationally
more efficient and it was also originally recommended over other continuous
versions of DGM by Kazi et al. (2022). Sáez de Ocáriz Borde et al. (2023c)
generalized the dDGM module to incorporate non-Euclidean constant curvature
hyperbolic and spherical spaces as potential latent graph embedding spaces, as
well as Cartesian products of these. This led to a combinatorial space of
candidate manifolds to model the dDGM embedding. In this work, we will
introduce a differentiable framework to select between potential embeddings.

##  3 Method: Attentional Multi Embedding Selection for Latent Graph Inference

Framework Overview. As mentioned earlier, the primary limitation of existing
latent graph inference models is the requirement to predefine the embedding
space used to generate the latent graph. This constraint is restrictive
because we often lack prior knowledge of which manifold is most suitable for
the specific problem we are tackling. Consequently, we aim to introduce a
novel framework for differentiable embedding space selection, which we coin
Attentional Multi Embedding Selection (AMES). This framework will have the
capacity to autonomously learn and select an appropriate embedding space
during the training process, providing greater flexibility and adaptability.
To do so, we start from a set of candidate latent geometries, as illustrated
in Figure 1. These candidate latent geometries give rise to distinct latent
graphs. These are then input into GNN diffusion layers, which facilitate the
propagation of the original node features based on the neighborhoods defined
by the suggested latent graphs. Subsequently, we employ self-attention
(Bahdanau et al., 2014; Vaswani et al., 2017) to aggregate the new node
features, which are then utilized for downstream tasks. Finally, we apply
backpropagation based on downstream performance criteria to dynamically
determine the optimal embedding space. More precisely, our model learns to
assign varying degrees of importance to different candidate latent graphs.
Next, we will provide a more detailed explanation of how the feature
aggregation and backpropagation processes are executed.

Figure 1: AMES model depiction. AMES employs multiple dDGM modules running in
parallel, each functioning within its distinct embedding space. Consequently,
every dDGM generates a unique unweighted adjacency matrix representing its
inferred latent graph. Adjacency matrices
$\mathbf{A}_{E},\mathbf{A}_{H},\mathbf{A}_{S}$ correspond to the Euclidean
hyperplane, hyperboloid, and hypersphere models, respectively. These adjacency
matrices are then passed to their respective diffusion layers denoted as
$g_{\Phi_{\mathcal{M}_{i}}}$. In parallel, new node features are computed
based on the different latent graphs. Finally, attention is utilized to
combine the features
$\mathbf{X}_{\mathcal{M}_{i}}\in[\mathbf{X}_{E},\mathbf{X}_{H},\mathbf{X}_{S}]$
into a unified representation, $\mathbf{X}^{\prime}$. This is subsequently fed
into the downstream layers. To update the diffusion layers
$g_{\Phi_{\mathcal{M}_{i}}}$ for each dDGM, a shared gradient denoted as
$\widehat{\nabla_{\mathbf{W}}\mathcal{L}_{T}}$ is employed.

Combining Feature Representations from Candidate Latent Graphs using
Attention. The optimal embedding space for latent graph inference is unknown
and varies from task to task. Different embedding spaces come with distinct
metrics, resulting in varying latent graphs. Instead of choosing an embedding
space in advance, we aim to empower our model to select the best embedding
space in a differentiable manner. To achieve this, we will not work with a
single dDGM module for latent graph inference but rather employ multiple of
them simultaneously. Each dDGM will utilize a different embedding space,
suggesting a distinct candidate latent graph. Consequently, this approach will
generate different downstream features as information from the upstream layer
diffuses through each unique latent graph. Let us denote by
$\mathbf{X}_{\mathcal{M}_{i}}\in\mathbb{R}^{N\times d}$ the node feature
representations obtained based on different embedding space manifolds
$\mathcal{M}_{i}$, where $N$ is the number of nodes in the graph and $d$ the
dimensionality of the node features. Next, we define $\mathcal{W}$ as the
collection of potential embedding space manifolds for each dDGM with
cardinality $M$. Additionally, consider $\mathbf{x}_{k\mathcal{M}_{i}}$ to
represent the $k^{th}$ row (or node) of the matrix
$\mathbf{X}_{\mathcal{M}_{i}}$. We calculate attention in the following
manner:

$\alpha_{\mathcal{M}_{i}\mathcal{M}_{j}}^{k}=\frac{exp\left(\frac{\big{(}\mathbf{W}_{Q}\mathbf{x}_{k\mathcal{M}_{i}}\big{)}^{T}\mathbf{W}_{K}\mathbf{x}_{k\mathcal{M}_{j}}}{\sqrt{d}}\right)}{\sum_{\mathcal{M}_{j^{\prime}}\in\mathcal{W}}exp\left(\frac{\big{(}\mathbf{W}_{Q}\mathbf{x}_{k\mathcal{M}_{i}}\big{)}^{T}\mathbf{W}_{K}\mathbf{x}_{k\mathcal{M}_{j^{\prime}}}}{\sqrt{d}}\right)}.$
(1)

In this equation, $\mathbf{W}_{Q}$ and $\mathbf{W}_{K}$ represent the query
and key weight matrices. Essentially, this equation enables the comparison of
node feature representations for a specific node $k$ in the graph across
various embedding spaces. The weight for the representation of the $k^{th}$
node within latent space $\mathcal{M}_{i}$, denoted as
$\alpha_{\mathcal{M}_{i}}^{k}$, is determined by averaging over all latent
spaces:

$\alpha_{\mathcal{M}_{i}}^{k}=\frac{\sum_{\mathcal{M}_{j}\in\mathcal{W}}\alpha_{\mathcal{M}_{j}\mathcal{M}_{i}}^{k}}{M}$
(2)

The coefficient $\alpha_{\mathcal{M}_{i}}^{k}$ can be interpreted as the mean
importance of manifold $\mathcal{M}_{i}$ relative to other manifolds for the
$k^{th}$ node representation. Finally, the overall combined node
representation for the $k^{th}$ node, $\mathbf{x}^{\prime}_{k}$ can be
calculated via the weighted sum:

$\mathbf{x}^{\prime}_{k}=\sum_{\mathcal{M}_{i}\in\mathcal{W}}\alpha_{\mathcal{M}_{i}}^{k}\mathbf{x}_{k\mathcal{M}_{i}}.$
(3)

Once this is calculated, the resultant combined node feature matrix
$\mathbf{X}^{\prime}$ can be fed into the subsequent downstream GNN layers.

Attention-based Gradient Descent for Automatic Latent Space Selection.
Although the attentional combination of feature representations discussed
earlier provides valuable insights into selecting latent spaces, relying
solely on these weights as straightforward indicators can be problematic if
the GNN diffusion layers are not initialized carefully. Several factors,
including variations in the initialization and training states of the GNN
diffusion layers $g_{\Phi_{\mathcal{M}_{i}}}$ for each latent space
$\mathcal{M}_{i}$, can serve as confounding variables that impact the
allocation of attention weights to these spaces. To ensure an equitable
assessment of different latent spaces throughout the training process, it is
crucial to maintain consistent parameters for all the GNN diffusion layers
that operate downstream of the parallel dDGM modules. In mathematical terms,
our objective is to retain
$\Phi_{\mathcal{M}_{1}}=\Phi_{\mathcal{M}_{2}}=\dots=\Phi_{\mathcal{M}_{i}}=\dots$
for all latent spaces $\mathcal{M}_{i}\in\mathcal{W}$ in each training step.
This effectively means we assess multiple candidate latent graphs
simultaneously during the training of a single GNN, avoiding the need for an
ensemble approach. As a result, we can identify the optimal candidate geometry
for our model.

The initial step involves ensuring identical parameter initialization for each
diffusion layer downstream of every dDGM module. It is important to note that
multiple diffusion layers need to be initialized to maintain the integrity of
the computational graph. However, in essence, we are evaluating the
performance of various candidate latent graphs in parallel on a single model.
During each training step, we substitute the initial gradients of the GNN
parameters for each latent space, which are obtained through backpropagation,
with an attention-based weighted gradient formed by combining individual
gradients. We can define $\mathbf{W}_{\mathcal{M}_{i}}$ as the set of model
parameters responsible for processing the latent graph generated from the
embedding space $\mathcal{M}_{i}$. Let
$\nabla_{\mathbf{W}_{\mathcal{M}_{i}}}\mathcal{L}_{T}$ represent the gradient
of the loss concerning these parameters for the downstream task. The attention
weights for each $\mathcal{M}_{i}$, utilized to combine
$\nabla_{\mathbf{W}_{\mathcal{M}_{i}}}\mathcal{L}_{T}$, are calculated by
averaging the node attention weights $\alpha^{k}_{\mathcal{M}_{i}}$, which are
computed during the forward pass in line with Equation 2:
$\alpha_{\mathcal{M}_{i}}=\frac{\sum_{k=1}^{N}\alpha^{k}_{\mathcal{M}_{i}}}{N},$
where $N$ is the number of nodes. Finally, the attention-weighted gradient
shared across all latent spaces is computed as follows:

$\widehat{\nabla_{\mathbf{W}}\mathcal{L}_{T}}=\sum_{\mathcal{M}_{i}\in\mathcal{W}}\alpha_{\mathcal{M}_{i}}\Big{(}\nabla_{\mathbf{W}_{\mathcal{M}_{i}}}\mathcal{L}_{T}\Big{)}.$
(4)

This gradient can be employed to update all the parallel diffusion layers
using any Stochastic Gradient Descent (SGD)-based optimization method, while
ensuring uniform downstream GNN parameters across all latent spaces throughout
the training process. The attention-based feature combination and gradient
descent scheme together form our AMES latent space selection framework for
embedding space selection.

Further Details on dDGM training. When training GNN models that incorporate
dDGM latent graph inference modules, there are two distinct losses to
consider. The first one, denoted as $\mathcal{L}_{T}$, assesses the model’s
performance on downstream tasks, employing cross-entropy loss for
classification and mean squared error (MSE) for regression, among other
possibilities. Given that the GNN diffuses information using the latent graphs
generated by the latent graph inference module, it is important to note that
this loss exclusively updates the GNN parameters and does not propagate
gradients through the dDGM. Hence, an additional loss term to update the
internal learnable parameters of the dDGM is required, which is known as the
graph loss, $\mathcal{L}_{GL}$. This loss is based on the negative log
likelihood of the probability that an edge exists between two nodes:
$\mathcal{L}_{GL}=\sum_{i=1}^{N}\left(\delta(y_{i},\hat{y}_{i})\sum_{l=1}^{L}\sum_{j:(i,j)\in\varepsilon^{(l)}}\log(p_{ij}^{(l)})\right).$
The term $\delta(y_{i},\hat{y}_{i})=\mathbb{E}(a_{i})-a_{i}$ is characterized
as the disparity between two aspects: the average accuracy observed for the
$i^{th}$ node and the correctness of the current prediction for that node’s
label, represented as $a_{i}$. Here, $a_{i}$ equals 1 if $y_{i}=\hat{y}_{i}$
and 0 otherwise, where $y_{i}$ is the label prediction and $\hat{y}_{i}$ the
ground truth label. For regression tasks, the R2 score could be used in place
of accuracy. In turn, the probability of an edge existing between two given
nodes can be expressed as follows,
$p_{ij}^{(l)}(\Theta^{(l)})=\exp\left(-T\mathfrak{d}_{\mathcal{M}}\left(\overline{f_{\Theta}^{(l)}(\mathbf{x}_{i}^{(l)})},\overline{f_{\Theta}^{(l)}(\mathbf{x}_{j}^{(l)})}\right)\right),$
where $T$ is a temperature parameter, $\mathfrak{d}_{\mathcal{M}}$ is the
geodesic distance function for a given embedding space manifold $\mathcal{M}$,
$\mathbf{x}_{i}^{(l)}$ and $\mathbf{x}_{j}^{(l)}$ are the node feature
representations of nodes $i$ and $j$ in layer $l$, and
$\overline{f_{\Theta}^{(l)}(\mathbf{x}_{i}^{(l)})}$ signifies that the node
features are passed through a parametrized function $f_{\Theta}$ and later
projected $\overline{(\cdot)}$ using the exponential map onto the
corresponding manifold $\mathcal{M}$. For our attention-based model, the
overall graph loss can be computed by simply adding up individual graph losses
from each dDGM module:

$\mathcal{L}_{GL}^{\prime}=\sum_{\mathcal{M}_{i}\in\mathcal{W}}\mathcal{L}_{GL}^{(\mathcal{M}_{i})}.$
(5)

It is worth noting that while latent graph inference could theoretically be
applied to all layers of the GNN, practical experiments, as demonstrated in
Kazi et al. (2022) and Sáez de Ocáriz Borde et al. (2023c), reveal that
stacking multiple dDGM modules does not yield performance improvements,
especially when considering the computational cost involved. Consequently, in
our work, we will exclusively employ a single dDGM module for all experiments.
During the training of AMES, we optimize both $\mathcal{L}_{T}$ and
$\mathcal{L}_{GL}^{\prime}$ using the Adam optimizer (Kingma and Ba, 2014). To
ensure a fair comparison with prior work, we adopt the same number of training
epochs and GNN parameterizations as in Sáez de Ocáriz Borde et al. (2023c).
Specific details regarding the learning rate and weight decay can be found in
Appendix A.1. For additional model architecture details refer to Appendix A.2,
A.3 and A.4.

##  4 Training Interpretability Framework

In addition to numerical outcomes, we employ a gradient-based attribution
approach, drawing inspiration from Simonyan et al. (2013), to assess the
significance of each latent space’s influence on the downstream tasks. This
serves as an interpretability analysis. The saliency maps suggested in
Simonyan et al. (2013) are generated by calculating the gradient of the score
function of the label class of an input image with respect to the pixels of
the input image itself. In an intuitive sense, the gradient of a predicted
label in relation to an input image measures how the model’s prediction
changes when individual image pixels undergo slight modifications. Therefore,
a larger gradient signifies a greater influence on the predictions. In our
context, the score function is the cross-entropy loss for all the nodes in the
graph. Hence, we take the gradient of the loss $\mathcal{L}_{T}$ with respect
to $\mathbf{X}_{\mathcal{M}_{i}}\in\mathbb{R}^{N\times d}$, the feature
activations produced from individual latent graphs originating in distinct
latent spaces, denoted as $\mathcal{M}_{i}$. This returns a gradient matrix
$\nabla_{\mathbf{X}_{\mathcal{M}_{i}}}\mathcal{L}_{T}\in\mathbb{R}^{N\times
d}$ since the loss is a scalar. Given that
$\nabla_{\mathbf{X}_{\mathcal{M}_{i}}}\mathcal{L}_{T}$ is high-dimensional and
hard to visualize, we calculate its Frobenius norm:

$\left\lVert\nabla_{\mathbf{X}_{\mathcal{M}_{i}}}\mathcal{L}_{T}\right\rVert_{\mathcal{F}}=\sqrt{\sum_{i=1}^{N}\sum_{j=1}^{d}\Big{(}\nabla_{\mathbf{X}_{\mathcal{M}_{i}}}\mathcal{L}_{T}\Big{)}_{ij}^{2}}$
(6)

which gives a scalar indicator of each latent space’s attribution to the node
classification prediction and it is easier to track and visualize during
training. During training initialization, all representations stemming from
diverse latent embedding spaces are set to exert equal influence on the
downstream representations, resulting in identical Frobenius norms. As
training progresses, the attention coefficients begin to learn which latent
graphs are more suitable for computation. By monitoring the Frobenius norm of
the loss gradient, we can gauge the pace at which this adaptation occurs and
identify which embedding spaces assume a dominant role in the learning
process. We do notice the existence of other GNN interpretability approaches
(Ying et al., 2019; Luo et al., 2020; Wang et al., 2021), however, these
approaches mainly focusing on determining subgraphs that are important for
individual predictions and hence are not suitable for our task.

##  5 Results

Experimental Setup and Numerical Evaluation. In this work, we focus on
transductive node classification. We test AMES on five homophilic,
heterophilic and real-world datasets. For each dataset, we perform 10-fold
cross-validation where $90\%$ of the nodes are used for training and $10\%$
for testing in each split in line with Kazi et al. (2022). In this case, the
task-specific loss is the cross-entropy loss. In our approach, we exclusively
employ model spaces as candidate embedding spaces, which suffices to match the
performance of prior methods that utilize product manifolds. It is worth
noting that for homophilic graph benchmarks, we leverage the original dataset
graph as an inductive bias. However, when dealing with heterophilic datasets,
we have observed that starting from a point cloud yields more significant
benefits. This preference arises from the fact that we perform diffusion with
GCN layers. In the instance of TadPole, a real-world baseline dataset, we
encounter a unique scenario where no initial input graph is available.
Consequently, our latent graph inference system operates directly on the point
cloud in this case. Table 1 displays the numerical results. AMES consistently
demonstrates improved or comparable test set accuracy across various datasets
compared to earlier models, even when those models employ more intricate
embedding structures, such as product spaces. Notably, the H+S combination
consistently surpasses all other models on nearly all five datasets. The key
takeaway is that using our proposed model eliminates the need for random
exploration among latent spaces.

Table 1: Accuracy $(\%)$ $\pm$ stdev. First, Second and Third are highlighted.
k is the number of edges sampled per node when using Gumbel Top-k trick. The
homophilic graphs (Cora and CiteSeer) use the dataset graph as inductive bias,
the rest do not. E, H, S denote Euclidean, hyperbolic and spherical spaces
respectively. Note that, for example, $EH$ denotes that the embedding space is
a product, whereas $E+H$ denotes that AMES uses both model spaces
simultaneously, not a product.

| HOMOPHILIC DATASETS |  | HETEROPHILIC DATASETS |  | REAL-WORLD DATASETS
---|---|---|---|---|---
| Cora | CiteSeer |  | Chameleon | Squirrel |  | TadPole
Connections k | 7 | 5 |  | 2 | 3 |  | 3
Baseline Models |  |  |  |  |  |  |
dDGM-E | $81.89\pm 4.05$ | $73.52\pm 2.23$ |  | $44.93\pm 2.42$ | $33.17\pm 2.11$ |  | $83.21\pm 14.04$
dDGM-H | $82.41\pm 3.25$ | $73.22\pm 1.50$ |  | $48.33\pm 3.45$ | $33.29\pm 1.90$ |  | $86.96\pm 9.48$
dDGM-S | $78.59\pm 4.82$ | $70.27\pm 5.05$ |  | $43.88\pm 3.79$ | $33.04\pm 1.71$ |  | $81.07\pm 12.38$
dDGM-ES | $74.85\pm 16.53$ | $71.08\pm 7.00$ |  | $47.71\pm 2.11$ | $33.56\pm 2.39$ |  | $70.71\pm 17.22$
dDGM-EH | $84.74\pm 4.89$ | $73.55\pm 2.35$ |  | $46.78\pm 3.16$ | $33.65\pm 1.60$ |  | $86.96\pm 9.45$
dDGM-HS | $84.48\pm 4.55$ | $67.68\pm 19.12$ |  | $48.41\pm 2.86$ | $32.87\pm 1.70$ |  | $77.68\pm 15.72$
dDGM-EHH | $79.63\pm 18.23$ | $68.70\pm 16.36$ |  | $48.28\pm 3.33$ | $33.23\pm 2.66$ |  | $86.43\pm 10.01$
dDGM-EHS | $84.67\pm 6.36$ | $68.83\pm 16.14$ |  | $48.63\pm 2.30$ | $33.31\pm 1.95$ |  | $83.93\pm 14.15$
MLP | $59.52\pm 3.67$ | $58.37\pm 3.29$ |  | $42.86\pm 2.72$ | $31.12\pm 1.99$ |  | $83.57\pm 6.12$
GCN | $81.41\pm 10.45$ | $70.84\pm 2.80$ |  | $34.05\pm 4.03$ | $24.88\pm 2.78$ |  | N/A
New Models (AMES) |  |  |  |  |  |  |
AMES-H+S | $83.33\pm 5.06$ | $73.64\pm 2.45$ |  | $49.30\pm 2.33$ | $34.85\pm 2.02$ |  | $90.18\pm 4.01$
AMES-E+H+S | $83.78\pm 3.08$ | $73.43\pm 2.23$ |  | $48.02\pm 3.82$ | $33.00\pm 1.44$ |  | $87.14\pm 6.81$

Interpretability Results. We create visualizations to track the average norm
evolution across the 10 training folds. In the interest of conducting the
interpretability analysis, despite the superior performance of H+S in
principle, Figure LABEL:fig:e_h_s_no_inductive_bias_grad presents plots
illustrating the average Frobenius norm of loss gradients for E+H+S. This
allows us to monitor the learning progress across all model spaces, as opposed
to focusing solely on two of them. The trends align with the data presented in
Table 1 for all five datasets. In the case of Cora, it is noteworthy that
$\mathbb{H}$ appears to contribute the most, while $\mathbb{S}$ makes the
smallest contribution. For CiteSeer, although the gap between $\mathbb{E}$ and
$\mathbb{H}$ is not as pronounced, both curves conform to the baseline results
for single model spaces found in Table 1. For the heterophilic datasets we
observe a similar trend, the models using hyperbolic space are superior and
our framework learns to select the graph stemming from hyperbolic space as the
main contributor, see Figure 2 and 2. In summary, our model demonstrates its
ability to autonomously identify the embedding spaces that align most
favorably with the baseline results. The attention weights assigned to each
latent space closely correspond to their counterparts based on single model
spaces in the baselines. As a result, our AMES effectively assesses the
relative importance of the involved latent spaces based on their impact on
downstream tasks. Furthermore, it becomes evident that the hyperbolic space
$\mathbb{H}$ consistently emerges as the preferred choice across the majority
of the benchmark graph datasets we have employed. This intriguing finding may
merit further investigation beyond the scope of this study and seems to align
with the common belief that hyperbolic space is better suited for encoding
latent tree structures (Kratsios et al., 2023), such as those present in our
graph benchmarks.

fig:e_h_s_no_inductive_bias_grad [Cora] [CiteSeer] [Squirrel] [Chameleon]
[TadPole]

Figure 2: Gradient contribution for each model space in AMES during training.
We plot the average Frobenius norm over 10 runs against the training epochs
for Cora, CiteSeer, Squirrel, Chameleon, and TadPole. We consistently observe
that AMES gives the most importance to the hyperbolic embedding space.

##  6 Conclusion

In this work we have introduced AMES, a fully-differentiable mechanism
designed for embedding space selection within the context of latent graph
inference. In contrast to prior methods, our approach circumvents the need for
conducting random searches across the combinatorial spectrum of potential
product manifold combinations. Instead, it possesses the capacity to
dynamically learn the optimal embedding space through a single model.
Moreover, we offer interpretability techniques to monitor this learning
process. Notably, our method maintains performance levels on par with other
models in the existing literature, all while alleviating the computational
burden associated with conducting multiple experiments in pursuit of the ideal
embedding space.

Future Work. Our model selection remains grounded in constant curvature model
spaces, implying that local curvature control remains a challenge. While the
application of attention could potentially yield more flexible representations
than what constant curvature model spaces can offer on their own, additional
research is required to gain a deeper understanding of how this approach
influences the curvature of the embedding space. Note that since the method
does not distinctly select a candidate embedding space but rather combines
model spaces using attention, the model is effectively learning a new metric.

## References

  * Alon and Yahav (2021) Uri Alon and Eran Yahav.  On the bottleneck of graph neural networks and its practical implications.  In _International Conference on Learning Representations_ , 2021.  URL https://openreview.net/forum?id=i80OPhOCVH2.
  * Bahdanau et al. (2014) Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio.  Neural machine translation by jointly learning to align and translate.  _CoRR_ , abs/1409.0473, 2014.  URL https://api.semanticscholar.org/CorpusID:11212020.
  * Bronstein et al. (2021) Michael M Bronstein, Joan Bruna, Taco Cohen, and Petar Veličković.  Geometric deep learning: Grids, groups, graphs, geodesics, and gauges.  _arXiv preprint arXiv:2104.13478_ , 2021.
  * Chamberlain et al. (2021) Ben Chamberlain, James Rowbottom, Maria I Gorinova, Michael Bronstein, Stefan Webb, and Emanuele Rossi.  Grand: Graph neural diffusion.  In _International Conference on Machine Learning_ , pages 1407–1418. PMLR, 2021.
  * Chami et al. (2019) Ines Chami, Zhitao Ying, Christopher Ré, and Jure Leskovec.  Hyperbolic graph convolutional neural networks.  _Advances in neural information processing systems_ , 32, 2019.
  * Chen et al. (2020) Yu Chen, Lingfei Wu, and Mohammed Zaki.  Iterative deep graph learning for graph neural networks: Better and robust node embeddings.  _Advances in neural information processing systems_ , 33:19314–19326, 2020.
  * Cosmo et al. (2020) Luca Cosmo, Anees Kazi, Seyed-Ahmad Ahmadi, Nassir Navab, and Michael Bronstein.  Latent-graph learning for disease prediction.  In _Medical Image Computing and Computer Assisted Intervention–MICCAI 2020: 23rd International Conference, Lima, Peru, October 4–8, 2020, Proceedings, Part II 23_ , pages 643–653. Springer, 2020.
  * De Bortoli et al. (2022) Valentin De Bortoli, Emile Mathieu, Michael Hutchinson, James Thornton, Yee Whye Teh, and Arnaud Doucet.  Riemannian score-based generative modelling.  _Advances in Neural Information Processing Systems_ , 35:2406–2422, 2022.
  * Deac et al. (2022) Andreea Deac, Marc Lackenby, and Petar Velivckovi’c.  Expander graph propagation.  In _LOG IN_ , 2022.
  * Franceschi et al. (2019) Luca Franceschi, Mathias Niepert, Massimiliano Pontil, and Xiao He.  Learning discrete structures for graph neural networks.  In _International conference on machine learning_ , pages 1972–1982. PMLR, 2019.
  * Gasteiger et al. (2019) Johannes Gasteiger, Stefan Weißenberger, and Stephan Günnemann.  Diffusion improves graph learning.  _Advances in neural information processing systems_ , 32, 2019.
  * Hensel et al. (2021) Felix Hensel, Michael Moor, and Bastian Rieck.  A survey of topological machine learning methods.  _Frontiers in Artificial Intelligence_ , 4:681108, 2021\.
  * Huang et al. (2022) Chin-Wei Huang, Milad Aghajohari, Joey Bose, Prakash Panangaden, and Aaron C Courville.  Riemannian diffusion models.  _Advances in Neural Information Processing Systems_ , 35:2750–2761, 2022.
  * Jin et al. (2020) Wei Jin, Yao Ma, Xiaorui Liu, Xianfeng Tang, Suhang Wang, and Jiliang Tang.  Graph structure learning for robust graph neural networks.  In _Proceedings of the 26th ACM SIGKDD international conference on knowledge discovery & data mining_, pages 66–74, 2020.
  * Kazi et al. (2022) Anees Kazi, Luca Cosmo, Seyed-Ahmad Ahmadi, Nassir Navab, and Michael M Bronstein.  Differentiable graph module (dgm) for graph convolutional networks.  _IEEE Transactions on Pattern Analysis and Machine Intelligence_ , 45(2):1606–1617, 2022.
  * Kingma and Ba (2014) Diederik P Kingma and Jimmy Ba.  Adam: A method for stochastic optimization.  _arXiv preprint arXiv:1412.6980_ , 2014.
  * Kratsios et al. (2023) Anastasis Kratsios, Ruiyang Hong, and Haitz Sáez de Ocáriz Borde.  Capacity bounds for hyperbolic neural network representations of latent tree structures, 2023.
  * Luo et al. (2020) Dongsheng Luo, Wei Cheng, Dongkuan Xu, Wenchao Yu, Bo Zong, Haifeng Chen, and Xiang Zhang.  Parameterized explainer for graph neural network.  _Advances in neural information processing systems_ , 33:19620–19631, 2020.
  * Mathieu et al. (2019) Emile Mathieu, Charline Le Lan, Chris J Maddison, Ryota Tomioka, and Yee Whye Teh.  Continuous hierarchical representations with poincaré variational auto-encoders.  _Advances in neural information processing systems_ , 32, 2019.
  * Mettes et al. (2019) Pascal Mettes, Elise Van der Pol, and Cees Snoek.  Hyperspherical prototype networks.  _Advances in neural information processing systems_ , 32, 2019.
  * Sáez de Ocáriz Borde et al. (2023a) Haitz Sáez de Ocáriz Borde, Alvaro Arroyo, Ismael Morales, Ingmar Posner, and Xiaowen Dong.  Neural latent geometry search: Product manifold inference via gromov-hausdorff-informed bayesian optimization, 2023a.
  * Sáez de Ocáriz Borde et al. (2023b) Haitz Sáez de Ocáriz Borde, Alvaro Arroyo, and Ingmar Posner.  Projections of model spaces for latent graph inference.  In _ICLR 2023 Workshop on Physics for Machine Learning_ , 2023b.
  * Sáez de Ocáriz Borde et al. (2023c) Haitz Sáez de Ocáriz Borde, Anees Kazi, Federico Barbero, and Pietro Lio.  Latent graph inference using product manifolds.  _The Eleventh International Conference on Learning Representations_ , 2023c.
  * Simonyan et al. (2013) Karen Simonyan, Andrea Vedaldi, and Andrew Zisserman.  Deep inside convolutional networks: Visualising image classification models and saliency maps.  _arXiv preprint arXiv:1312.6034_ , 2013.
  * Vaswani et al. (2017) Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin.  Attention is all you need.  _Advances in neural information processing systems_ , 30, 2017.
  * Wang et al. (2021) Xiang Wang, Yingxin Wu, An Zhang, Xiangnan He, and Tat seng Chua.  Causal screening to interpret graph neural networks, 2021.  URL https://openreview.net/forum?id=nzKv5vxZfge.
  * Wang et al. (2019) Yue Wang, Yongbin Sun, Ziwei Liu, Sanjay E Sarma, Michael M Bronstein, and Justin M Solomon.  Dynamic graph cnn for learning on point clouds.  _Acm Transactions On Graphics (tog)_ , 38(5):1–12, 2019.
  * Ying et al. (2019) Zhitao Ying, Dylan Bourgeois, Jiaxuan You, Marinka Zitnik, and Jure Leskovec.  Gnnexplainer: Generating explanations for graph neural networks.  _Advances in neural information processing systems_ , 32, 2019.
  * Zhu et al. (2021) Yanqiao Zhu, Weizhi Xu, Jinghao Zhang, Qiang Liu, Shu Wu, and Liang Wang.  Deep graph structure learning for robust representations: A survey.  _arXiv preprint arXiv:2103.03036_ , 14, 2021.

##  Appendix A Training Statistics and Model Architecture

This section provides the training statistics and model architectures for both
the dDGM and the downstream GNN diffusion layer applied to homophilic,
heterophilic, and real-world datasets. In general, we closely follow the model
architecture and training statistics outlined in Sáez de Ocáriz Borde et al.
(2023c), with only minor differences.

###  A.1 Training Statistics

Table 2: The Training statistics of the 5 homophilic, heterophilic and real-world benchmark datasets used in this work for the baseline dDGM, MLP and GCN models. | Training Epochs | Learning Rate | Weight Decay
---|---|---|---
Cora | 1,500 | $1\times 10^{-2}$ | $1\times 10^{-4}$
CiteSeer | 1,500 | $1\times 10^{-2}$ | $1\times 10^{-4}$
Squirrel | 1,000 | $1\times 10^{-2}$ | $1\times 10^{-3}$
Chameleon | 1,000 | $1\times 10^{-2}$ | $1\times 10^{-3}$
TadPole | 800 | $1\times 10^{-3}$ | $2\times 10^{-4}$
Table 3: The Training statistics of the 5 homophilic, heterophilic and real-world benchmark datasets used in this work for AMES in this work. | Training Epochs | Learning Rate | Weight Decay
---|---|---|---
Cora | 1,500 | $5\times 10^{-3}$ | $1\times 10^{-5}$
CiteSeer | 1,500 | $4\times 10^{-3}$ | $1\times 10^{-5}$
Squirrel | 1,000 | $1\times 10^{-2}$ | $1\times 10^{-5}$
Chameleon | 1,000 | $1\times 10^{-2}$ | $1\times 10^{-5}$
TadPole | 800 | $1\times 10^{-3}$ | $2\times 10^{-4}$

###  A.2 Model Architecture for Homophilic Datasets

Table 4: Model architecture of dDGM for homophilic datasets Cora and CiteSeer. Layer size |  | dDGM
---|---|---
(No. features, 32) |  | Linear
N/A |  | ELU
(32, 16 per model space) |  | GCN Conv
N/A |  | ELU
(16 per model space, 4 per model space) |  | GCN Conv
N/A |  | Sigmoid
Table 5: Model architecture of downstream GNN layers as well as MLP baseline for homophilic datasets Cora and CiteSeer. Layer size |  | MLP | GCN | GCN-dDGM
---|---|---|---|---
|  | N/A | N/A | dDGM
(No. features, 32) |  | Linear | GCN Conv | GCN Conv
N/A |  | ELU | ELU | ELU
(32, 16) |  | Linear | GCN Conv | GCN Conv
N/A |  | ELU | ELU | ELU
(16, 8) |  | Linear | GCN Conv | GCN Conv
N/A |  | ELU | ELU | ELU
(8, 8) |  | Linear | Linear | Linear
N/A |  | ELU | ELU | ELU
(8, 8) |  | Linear | Linear | Linear
N/A |  | ELU | ELU | ELU
(8, No.classes) |  | Linear | Linear | Linear

###  A.3 Model Architecture for Heterophilic Datasets

Table 6: Model architecture of dDGM for heterophilic datasets Squirrel and Chameleon. Layer size |  | dDGM
---|---|---
(No. features, 32) |  | Linear
N/A |  | BatchNorm
N/A |  | ELU
(32, 4 per model space) |  | Linear
N/A |  | BatchNorm
N/A |  | ELU
(4 per model space, 4 per model space) |  | Linear
N/A |  | BatchNorm
N/A |  | Sigmoid
Table 7: Model architecture of downstream GNN layers as well as MLP baseline for heterophilic datasets Squirrel and Chameleon. Layer size |  | MLP | GCN-dDGM
---|---|---|---
|  | N/A | dDGM
(No. features, 16) |  | Linear | GCN Conv
N/A |  | ELU | ELU
(16, 8) |  | Linear | GCN Conv
N/A |  | ELU | ELU
(8, 8) |  | Linear | Linear
N/A |  | BatchNorm | BatchNorm
N/A |  | ELU | ELU
(8, No.classes) |  | Linear | Linear

###  A.4 Model Architecture for TadPole

Table 8: Model architecture of dDGM for TadPole. Layer size |  | dDGM
---|---|---
(No. features, 16 per model space) |  | Linear
N/A |  | BatchNorm
N/A |  | ELU
(16 per model space, 4 per model space) |  | Linear
N/A |  | BatchNorm
N/A |  | Sigmoid
Table 9: Model architecture of downstream GNN layers as well as MLP baseline for TadPole. Layer size |  | MLP | GCN-dDGM
---|---|---|---
|  | N/A | dDGM
(No. features, 32) |  | Linear | GCN Conv
N/A |  | ELU | ELU
(32, 16) |  | Linear | GCN Conv
N/A |  | ELU | ELU
(16, 8) |  | Linear | GCN Conv
N/A |  | ELU | ELU
(8, 8) |  | Linear | Linear
N/A |  | BatchNorm | BatchNorm
N/A |  | ELU | ELU
(8, 8) |  | Linear | Linear
N/A |  | BatchNorm | BatchNorm
N/A |  | ELU | ELU
(8, No.classes) |  | Linear | Linear