# A Probabilistic Approach to Learning the Degree of
Equivariance in Steerable CNNs

Lars Veefkind  Gabriele Cesa

###### Abstract

Steerable convolutional neural networks (SCNNs) enhance task performance by
modelling geometric symmetries through equivariance constraints on weights.
Yet, unknown or varying symmetries can lead to overconstrained weights and
decreased performance. To address this, this paper introduces a probabilistic
method to learn the degree of equivariance in SCNNs. We parameterise the
degree of equivariance as a likelihood distribution over the transformation
group using Fourier coefficients, offering the option to model layer-wise and
shared equivariance. These likelihood distributions are regularised to ensure
an interpretable degree of equivariance across the network. Advantages include
the applicability to many types of equivariant networks through the flexible
framework of SCNNs and the ability to learn equivariance with respect to any
subgroup of any compact group without requiring additional layers. Our
experiments reveal competitive performance on datasets with mixed symmetries,
with learnt likelihood distributions that are representative of the underlying
degree of equivariance.

Machine Learning, ICML, Geometric Deep Learning, Equivariance, Steerable CNNs,
Partial Equivariance

##  1 Introduction

Natural images and structures often exhibit various geometrical symmetries,
such as rotations, translations, and reflections. Since LeCun et al. (1995)
introduced translation-equivariant Convolutional Neural Networks (CNNs),
integrating group equivariance properties as a form of inductive bias into
deep learning architectures has been an active area of research (Weiler et
al., 2021; Bronstein et al., 2021).

While incorporating higher degrees of equivariance can serve as a powerful
inductive bias, over-constraining a model can lead to diminished performance
by preventing the use of potentially significant features (Wang et al., 2022b;
Petrache & Trivedi, 2023). Therefore, it is crucial to select an appropriate
transformation group that is neither too large nor too small when developing a
model. The required degree of equivariance often varies among features,
typically correlating with their scale (Fig. 1). In a CNN, with its
hierarchical layers processing features of varying sizes (receptive fields),
selecting the appropriate transformation group beforehand poses a challenge,
demanding substantial knowledge about the dataset and the layers’ receptive
fields.

Figure 1: Coronal CT view of left and right lung (Yang et al., 2023). Smaller
features $\bigcirc$ (i.e., textures/components) are orientation invariant in
determining whether the object is a lung, unlike the task of distinguishing
between the two lungs as a whole, as they are (approximately) mirrored
versions of each other.

To address these complexities, various strategies have been proposed to
autonomously learn the relaxation of equivariance constraints on a layer-by-
layer basis for Group CNNs (Cohen & Welling, 2016a). Among these are Residual
Pathway Priors (RPPs) (Finzi et al., 2021), non-stationary GCNNs (van der
Ouderaa et al., 2022), and Partial GCNNs (Romero & Lohit, 2022). We cover
these methods in more detail in Sec. 4. Despite the flexibility of the
framework of steerable CNNs (Cohen & Welling, 2016b) compared to GCNNs, only
very few methods are compatible with this framework. To the best of our
knowledge, RSteer (Wang et al., 2022b) is the only method directly developed
for SCNNs, while RPP provides a generalised method for modulating equivariance
regardless of the network type. However, with no separate parameterisation
describing the degree of equivariance, such as a likelihood distribution over
the transformation group as proposed by Romero & Lohit (2022), it is
impossible to obtain an interpretable degree of equivariance from these
methods without performing computationally expensive analyses. Furthermore,
RPPs achieve this modulation through residual connections that are non-
equivariant, or equivariant with respect to a smaller subgroup of the large
group. Still, accurately choosing the relevant equivariance subgroups can be
challenging: for instance, there are infinitely many $SO(2)$ subgroups in
$SO(3)$. We cover these shortcomings in more details in Sec. 4.

###  1.1 Contributions and Outline

Building upon the SCNN framework developed by Cesa et al. (2022), we propose a
general solution to obtain a learnable degree of equivariance in SCNNs.
Parameterised by an interpretable likelihood distribution over the
transformation group, our solution models partial equivariance by using
connections that are not available in an equivariant setting.

In Sec. 3.1, we exploit the interpretation of the kernel projection as an
implicit uniform averaging over the group. We show that this uniform
distribution can be modelled using Fourier coefficients in Sec. 2.2. By
updating these coefficients through backpropagation, we enable the learning of
a likelihood distribution over any compact group in $n$D, including continuous
groups. The learnt likelihood corresponds to the model’s inherent degree of
equivariance and is easily obtained with the inverse Fourier transform without
forward passes, achieving an interpretable degree of equivariance. Moreover,
to prevent a layer from erroneously regaining previously lost equivariance, we
propose employing KL-divergence (Kullback & Leibler, 1951) between subsequent
layers. Additionally, our approach allows the user to tune the band-limiting
of the Fourier coefficients as means of regularisation and improved
computational performance.

To evaluate our approaches, we conduct an extensive series of experiments in
Sec. 5, comparing our method with various baselines. These include regular
SCNNs with pre-tuned degrees of equivariance through group restrictions
(Weiler & Cesa, 2019), RPPs (Finzi et al., 2021) and Relaxed SCNNs (Wang et
al., 2022b), as these are, to our knowledge, the only approaches directly
applicable to SCNNs. Our experiments assess the performance impact on various
computer vision datasets. Additionally, we analyse the interpretability and
correctness of the learnt likelihood distributions and study the influence of
bandlimiting and KL-divergence on both performance and interpretability.
Finally, we compare data-efficiency and generalisation capabilities between
our approach and the baselines. Through these experiments, we offer the
following contributions:

  * •

We develop a probabilistic approach to model partial equivariances in SCNNs
and show that it achieves competitive performance on partially symmetric
datasets.

  * •

Our findings reveal that the learnt likelihood distributions accurately
represent the degree of equivariance, enhancing interpretability.

  * •

We demonstrate that bandlimiting serving as a useful tool to regularise the
degree of equivariance.

  * •

We show that it outperforms other methods in generalising from discrete to
continuous group symmetries.

  * •

We show that our approach achieves high data-efficiency on fairly symmetric
datasets.

##  2 Background on Steerable CNNs

This section briefly covers the framework of SCNNs. For a comprehensive
overview of all mathematical preliminaries we refer to Apx. B. We restrict
ourselves to convolutional networks acting on Euclidean spaces
$X=\mathbb{R}^{n}$ and compact groups $H\leq O(n)$, resulting in networks
equivariant to transformations in $G=(\mathbb{R}^{n},+)\rtimes H$, i.e. the
semidirect product between translations $(\mathbb{R}^{n},+)$ and $H$.

###  2.1 Steerable CNNs

As the name suggests, each layer $l$ in a $G$-equivariant neural network
satisfies the equivariance constraint:

$\forall x\in X,\forall g\in G\ \\!\colon\quad l(g\cdot x)=g\cdot l(x),$ (1)

meaning that these networks have the property to commute with the action of a
symmetry group $G$ on their input, intermediate features and output. In this
paper, we focus on SCNNs. These networks operate on feature fields, which are
comparable to feature maps, except that they are associated with a well
defined action of $G$ and a geometric type. This type is described by an
orthogonal group representation $\rho\\!\colon
H\\!\to\\!\mathbb{R}^{d_{\rho}\times d_{\rho}}$ that determines the
transformation law of the $d_{\rho}$-dimensional feature vectors in the
feature field. Specifically, a feature field of type $\rho$ is a map
$f\\!\colon\mathbb{R}^{n}\\!\to\\!\mathbb{R}^{d_{\rho}}$ that, given a group
action $g=(t,h)\\!\in\\!(\mathbb{R}^{n},+)\rtimes H$, transforms by spatially
moving feature vectors from $h^{-1}(x-t)$ to $x$ and uses $\rho(h)$ to
transform each individual feature111This is modelled via the induced
representation $\left(\left[\operatorname{Ind}_{H}^{(\mathbb{R}^{n},+)\rtimes
H}\rho\right](th)f\right)(x)=\rho(h)f(h^{-1}(x-t))$..

The simplest example of feature fields are scalar fields, used in conventional
CNNs. These fields transform according to the trivial representation
$\psi_{0}(h)=1$. GCNNs are special cases and transform according to the
regular representation of $G$. As neural networks are often composed of
multiple feature fields $f_{i}$, we often describe the entire feature field as
a direct sum $f=\bigoplus_{i}f_{i}$ of the individual feature fields, i.e. by
stacking them along the channel dimension. The resulting feature space
consequently transforms under the direct sum representation
$\rho=\bigoplus_{i}\rho_{i}$; see Def. B.27.

The goal of SCNNs is to form a general framework to obtain equivariance,
regardless of the types of the feature fields. To this end, it has been shown
that the most general equivariant maps between feature spaces are convolutions
with $H$-steerable kernels Weiler et al. (2018a). Given input and output
feature field types $\rho_{\operatorname{in}}\\!\colon
H\\!\to\\!\mathbb{R}^{d_{\operatorname{in}}\times d_{\operatorname{in}}}$ and
$\rho_{\operatorname{out}}\\!\colon
H\\!\to\\!\mathbb{R}^{d_{\operatorname{out}}\times d_{\operatorname{out}}}$
respectively, an $H$-steerable kernel is a convolution kernel
$K\\!\colon\mathbb{R}^{n}\\!\to\\!\mathbb{R}^{d_{\operatorname{out}}\times
d_{\operatorname{in}}}$ that satisfies the steerability constraint $\forall
h\in H,x\in\mathbb{R}^{n}$:

$K(x)=\rho_{\operatorname{out}}(h)K(h^{-1}x)\rho_{\operatorname{in}}(h)^{-1}.$
(2)

Therefore, in order to build SCNNs we need to find a basis of the vector space
of such $H$-steerable kernels that can be used to parameterise the
convolutions.

###  2.2 Generating Steerable Bases

Cesa et al. (2022) derives a general parameterisation of $H$-steerable
kernels. In this section, we offer a summary of an _alternative derivation_
that arrives at the same result, but with a probabilistic interpretation; see
Apx. C.2 for a detailed proof. In our derivation, we consider a general
Euclidean space $\mathbb{R}^{n}$, and compact group $H\leq O(n)$ in
$G=(\mathbb{R}^{n},+)\rtimes H$. As such, we can assume that all
representations are orthogonal, and thus
$\rho(g^{-1})=\rho(g)^{-1}=\rho(g)^{\top}$.

##### Irrep Decomposition

While $\rho_{\operatorname{in}}$ and $\rho_{\operatorname{out}}$ can be any
type of representation, through irrep decomposition (Thm. B.30) we can
decompose any orthogonal representation into a direct sum of convenient
irreducible representations (or irreps):
$\rho(h)=Q\left[\bigoplus_{{{\color[rgb]{0,0.44,0.57}\definecolor[named]{pgfstrokecolor}{rgb}{0,0.44,0.57}\pgfsys@color@cmyk@stroke{0.98}{0.13}{0}{0.43}\pgfsys@color@cmyk@fill{0.98}{0.13}{0}{0.43}i}}\in\textit{I}}{{\color[rgb]{0,0.44,0.57}\definecolor[named]{pgfstrokecolor}{rgb}{0,0.44,0.57}\pgfsys@color@cmyk@stroke{0.98}{0.13}{0}{0.43}\pgfsys@color@cmyk@fill{0.98}{0.13}{0}{0.43}\psi_{i}}}(h)\right]Q^{T}$,
where
${{\color[rgb]{0,0.44,0.57}\definecolor[named]{pgfstrokecolor}{rgb}{0,0.44,0.57}\pgfsys@color@cmyk@stroke{0.98}{0.13}{0}{0.43}\pgfsys@color<EMAIL_ADDRESS>are irreps in the set of irreps $\widehat{H}$ of $H$, $I$ is an index set
ranging over $\widehat{G}$, and $Q$ is a change of basis. Weiler & Cesa (2019)
show that the solution for any $\rho_{\operatorname{in}}$ and
$\rho_{\operatorname{out}}$ can be obtained from the solution of their irreps
decomposition. Therefore, we can assume the input and output representations
to be irreps $\psi_{l}$ and $\psi_{J}$. Furthermore, it is convenient to use
the vectorised form of our kernel:
$\kappa(\cdot)=\operatorname{vec}\left(K(\cdot)\right)\\!\colon
X\\!\to\\!\mathbb{R}^{d_{\operatorname{in}}\cdot d_{\operatorname{out}}}$.
Finally, as the constraint is linear, it can be solved by projecting
unconstrained square integrable kernel $\widehat{\kappa}\in
L^{2}(\mathbb{R}^{n})$ via integration; a similar idea was used in (Van der
Pol et al., 2020). As this is essentially performing a uniform averaging over
the group $H$, we can safely inject a uniform likelihood distribution
$\mu(h)=1\ \forall h\in H$, assuming $H$ is a compact subgroup of a group with
a finite Haar measure. Thus222$\operatorname{vec}(ABC)=(C^{\top}\otimes
A)\operatorname{vec}(B)$, with Kronecker product $\otimes$.:

$\kappa(x)=\int_{h\in
H}\mu(h)(\psi_{l}\otimes\psi_{J})(h)\widehat{\kappa}(h^{-1}x)dh.$ (3)

##### Tensor Product

 $(\psi_{l}\otimes\psi_{J})$ is a _tensor product representation_ (Def.
B.35). It can be decomposed into a sum of irreps
${{\color[rgb]{.5,0,.5}\definecolor[named]{pgfstrokecolor}{rgb}{.5,0,.5}\psi_{j^{\prime}}}}$
via Clebsch-Gordan decomposition (Thm. B.37):

$\psi_{l}\otimes\psi_{J}=\sum_{{{\color[rgb]{.5,0,.5}\definecolor[named]{pgfstrokecolor}{rgb}{.5,0,.5}j^{\prime}}}}\sum_{s}^{[{{\color[rgb]{.5,0,.5}\definecolor[named]{pgfstrokecolor}{rgb}{.5,0,.5}j^{\prime}}}(Jl)]}\left[{{\color[rgb]{.5,0,.5}\definecolor[named]{pgfstrokecolor}{rgb}{.5,0,.5}\operatorname{CG}_{s}^{{{\color[rgb]{.5,0,.5}\definecolor[named]{pgfstrokecolor}{rgb}{.5,0,.5}j^{\prime}}}(Jl)}}}\right]^{\top}{{\color[rgb]{.5,0,.5}\definecolor[named]{pgfstrokecolor}{rgb}{.5,0,.5}\psi_{j^{\prime}}}}{{\color[rgb]{.5,0,.5}\definecolor[named]{pgfstrokecolor}{rgb}{.5,0,.5}\operatorname{CG}_{s}^{{{\color[rgb]{.5,0,.5}\definecolor[named]{pgfstrokecolor}{rgb}{.5,0,.5}j^{\prime}}}(Jl)}}}.$
(4)

$[{{\color[rgb]{.5,0,.5}\definecolor[named]{pgfstrokecolor}{rgb}{.5,0,.5}j^{\prime}}}(Jl)]$
is the multiplicity of irrep
${{\color[rgb]{.5,0,.5}\definecolor[named]{pgfstrokecolor}{rgb}{.5,0,.5}\psi_{j^{\prime}}}}$.
${{\color[rgb]{.5,0,.5}\definecolor[named]{pgfstrokecolor}{rgb}{.5,0,.5}\operatorname{CG}_{s}^{{{\color[rgb]{.5,0,.5}\definecolor[named]{pgfstrokecolor}{rgb}{.5,0,.5}j^{\prime}}}(Jl)}}}$
are the Clebsch-Gordan coefficients for the $s$-th occurrence of
${{\color[rgb]{.5,0,.5}\definecolor[named]{pgfstrokecolor}{rgb}{.5,0,.5}\psi_{j^{\prime}}}}$.

##### Steerable Basis

An _unconstrained_ kernel $\widehat{\kappa}$ can be parameterised via weights
$W_{{\color[rgb]{1,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,0}j},{\color[rgb]{1,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,0}k}}\in\mathbb{R}^{d_{J}d_{l}\times
d_{{{\color[rgb]{1,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,0}j}}}}$
and an $H$-steerable basis (Def. B.46)
$\mathcal{B}=\left\\{{\color[rgb]{1,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,0}Y}_{{\color[rgb]{1,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,0}j}}^{{\color[rgb]{1,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,0}k}}:\mathbb{R}^{n}\to\mathbb{R}^{d_{{{\color[rgb]{1,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,0}j}}}}\mid{{\color[rgb]{1,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,0}\psi_{j}}}\in\widehat{H},{\color[rgb]{1,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,0}k}\right\\}$:

$\widehat{\kappa}(x)=\sum_{{\color[rgb]{1,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,0}j},{\color[rgb]{1,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,0}k}}W_{{\color[rgb]{1,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,0}j},{\color[rgb]{1,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,0}k}}{\color[rgb]{1,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,0}Y}_{{\color[rgb]{1,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,0}j}}^{{\color[rgb]{1,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,0}k}}(x).$
(5)

Let $\operatorname{unvec}(\cdot)$ be the reverse of the vectorisation. We
apply this parameterisation and the Clebsch-Gordan decomposition in Eq. 3, and
substitute
$W_{{\color[rgb]{1,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,0}j},{{\color[rgb]{.5,0,.5}\definecolor[named]{pgfstrokecolor}{rgb}{.5,0,.5}j^{\prime}}},{\color[rgb]{1,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,0}k},s}=\operatorname{vec}\left({{\color[rgb]{.5,0,.5}\definecolor[named]{pgfstrokecolor}{rgb}{.5,0,.5}\operatorname{CG}_{s}^{{{\color[rgb]{.5,0,.5}\definecolor[named]{pgfstrokecolor}{rgb}{.5,0,.5}j^{\prime}}}(Jl)}}}W_{{\color[rgb]{1,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,0}j}{\color[rgb]{1,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,0}k}}\right)$:

$\begin{split}\kappa(&x)=\sum_{{\color[rgb]{1,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,0}j},{\color[rgb]{1,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,0}k}}\sum_{{{\color[rgb]{.5,0,.5}\definecolor[named]{pgfstrokecolor}{rgb}{.5,0,.5}j^{\prime}}}}\sum_{s}^{[{{\color[rgb]{.5,0,.5}\definecolor[named]{pgfstrokecolor}{rgb}{.5,0,.5}j^{\prime}}}(Jl)]}\left[{{\color[rgb]{.5,0,.5}\definecolor[named]{pgfstrokecolor}{rgb}{.5,0,.5}\operatorname{CG}_{s}^{{{\color[rgb]{.5,0,.5}\definecolor[named]{pgfstrokecolor}{rgb}{.5,0,.5}j^{\prime}}}(Jl)}}}\right]^{\top}\\\
&\operatorname{unvec}\\!\left[\int\\!\mu(h)\left({{\color[rgb]{1,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,0}\psi_{j}}}\\!\otimes\\!{{\color[rgb]{.5,0,.5}\definecolor[named]{pgfstrokecolor}{rgb}{.5,0,.5}\psi_{j^{\prime}}}}\right)\\!(h)dh\
W_{{\color[rgb]{1,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,0}j},{{\color[rgb]{.5,0,.5}\definecolor[named]{pgfstrokecolor}{rgb}{.5,0,.5}j^{\prime}}},{\color[rgb]{1,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,0}k},s}\right]\\!{\color[rgb]{1,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,0}Y}_{{\color[rgb]{1,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,0}j}}^{{\color[rgb]{1,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,0}k}}(x)\end{split}$
(6)

Here the integral contains a tensor product, and can thus be decomposed
through the Clebsch-Gordan decomposition:

$\begin{split}\int\mu(h)&\left({{\color[rgb]{1,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,0}\psi_{j}}}\otimes{{\color[rgb]{.5,0,.5}\definecolor[named]{pgfstrokecolor}{rgb}{.5,0,.5}\psi_{j^{\prime}}}}\right)(h)dh=\\\
&Q\bigoplus_{{{\color[rgb]{0,0.44,0.57}\definecolor[named]{pgfstrokecolor}{rgb}{0,0.44,0.57}\pgfsys@color@cmyk@stroke{0.98}{0.13}{0}{0.43}\pgfsys@color@cmyk@fill{0.98}{0.13}{0}{0.43}i}}}\bigoplus_{r}^{[{{\color[rgb]{0,0.44,0.57}\definecolor[named]{pgfstrokecolor}{rgb}{0,0.44,0.57}\pgfsys@color@cmyk@stroke{0.98}{0.13}{0}{0.43}\pgfsys@color@cmyk@fill{0.98}{0.13}{0}{0.43}i}}({\color[rgb]{1,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,0}j}{{\color[rgb]{.5,0,.5}\definecolor[named]{pgfstrokecolor}{rgb}{.5,0,.5}j^{\prime}}})]}\left(\int\mu(h){{\color[rgb]{0,0.44,0.57}\definecolor[named]{pgfstrokecolor}{rgb}{0,0.44,0.57}\pgfsys@color@cmyk@stroke{0.98}{0.13}{0}{0.43}\pgfsys@color@cmyk@fill{0.98}{0.13}{0}{0.43}\psi_{i}}}(h)dh\right)Q^{\top}\end{split}$
(7)

##### Peter Weyl and inverse Fourier Transform

The Peter-Weyl theorem (Peter & Weyl, 1927) (Thm. B.42) states that the
entries of complex irreps of a compact group $H$ span the vector space
$L^{2}(H)$. However, in this work we consider real irreps, which may contain
redundant entries. A proper basis can be constructed by selecting an
independent subset of
$n_{{\color[rgb]{0,0.44,0.57}\definecolor[named]{pgfstrokecolor}{rgb}{0,0.44,0.57}\pgfsys@color@cmyk@stroke{0.98}{0.13}{0}{0.43}\pgfsys@color@cmyk@fill{0.98}{0.13}{0}{0.43}i}}=\frac{d_{{\color[rgb]{0,0.44,0.57}\definecolor[named]{pgfstrokecolor}{rgb}{0,0.44,0.57}\pgfsys@color@cmyk@stroke{0.98}{0.13}{0}{0.43}\pgfsys@color@cmyk@fill{0.98}{0.13}{0}{0.43}i}}}{[0({{\color[rgb]{0,0.44,0.57}\definecolor[named]{pgfstrokecolor}{rgb}{0,0.44,0.57}\pgfsys@color@cmyk@stroke{0.98}{0.13}{0}{0.43}\pgfsys@color@cmyk@fill{0.98}{0.13}{0}{0.43}ii}})]}$
columns of each irrep
${{\color[rgb]{0,0.44,0.57}\definecolor[named]{pgfstrokecolor}{rgb}{0,0.44,0.57}\pgfsys@color@cmyk@stroke{0.98}{0.13}{0}{0.43}\pgfsys@color<EMAIL_ADDRESS>333 This construction only works under a proper choice of basis for
${{\color[rgb]{0,0.44,0.57}\definecolor[named]{pgfstrokecolor}{rgb}{0,0.44,0.57}\pgfsys@color@cmyk@stroke{0.98}{0.13}{0}{0.43}\pgfsys@color<EMAIL_ADDRESS>$[0({{\color[rgb]{0,0.44,0.57}\definecolor[named]{pgfstrokecolor}{rgb}{0,0.44,0.57}\pgfsys@color@cmyk@stroke{0.98}{0.13}{0}{0.43}\pgfsys@color@cmyk@fill{0.98}{0.13}{0}{0.43}ii}})]\in\\{1,2,4\\}$
depending on the type of irrep
${{\color[rgb]{0,0.44,0.57}\definecolor[named]{pgfstrokecolor}{rgb}{0,0.44,0.57}\pgfsys@color@cmyk@stroke{0.98}{0.13}{0}{0.43}\pgfsys@color<EMAIL_ADDRESS>See (Cesa et al., 2022) Apx. C.3 for more details. , where
$[0({{\color[rgb]{0,0.44,0.57}\definecolor[named]{pgfstrokecolor}{rgb}{0,0.44,0.57}\pgfsys@color@cmyk@stroke{0.98}{0.13}{0}{0.43}\pgfsys@color@cmyk@fill{0.98}{0.13}{0}{0.43}ii}})]$
is the multiplicity of the trival irrep $\psi_{0}$ in
${{\color[rgb]{0,0.44,0.57}\definecolor[named]{pgfstrokecolor}{rgb}{0,0.44,0.57}\pgfsys@color@cmyk@stroke{0.98}{0.13}{0}{0.43}\pgfsys@color@cmyk@fill{0.98}{0.13}{0}{0.43}\psi_{i}}}\otimes{{\color[rgb]{0,0.44,0.57}\definecolor[named]{pgfstrokecolor}{rgb}{0,0.44,0.57}\pgfsys@color@cmyk@stroke{0.98}{0.13}{0}{0.43}\pgfsys@color<EMAIL_ADDRESS>as in Eq. 4. We denote the non-redundant columns of
${{\color[rgb]{0,0.44,0.57}\definecolor[named]{pgfstrokecolor}{rgb}{0,0.44,0.57}\pgfsys@color@cmyk@stroke{0.98}{0.13}{0}{0.43}\pgfsys@color@cmyk@fill{0.98}{0.13}{0}{0.43}\psi_{i}}}(h)$
with
$\overline{{\color[rgb]{0,0.44,0.57}\definecolor[named]{pgfstrokecolor}{rgb}{0,0.44,0.57}\pgfsys@color@cmyk@stroke{0.98}{0.13}{0}{0.43}\pgfsys@color@cmyk@fill{0.98}{0.13}{0}{0.43}\psi}}_{{\color[rgb]{0,0.44,0.57}\definecolor[named]{pgfstrokecolor}{rgb}{0,0.44,0.57}\pgfsys@color@cmyk@stroke{0.98}{0.13}{0}{0.43}\pgfsys@color@cmyk@fill{0.98}{0.13}{0}{0.43}i}}(h)\in\mathbb{R}^{d_{{\color[rgb]{0,0.44,0.57}\definecolor[named]{pgfstrokecolor}{rgb}{0,0.44,0.57}\pgfsys@color@cmyk@stroke{0.98}{0.13}{0}{0.43}\pgfsys@color<EMAIL_ADDRESS>n_{{\color[rgb]{0,0.44,0.57}\definecolor[named]{pgfstrokecolor}{rgb}{0,0.44,0.57}\pgfsys@color@cmyk@stroke{0.98}{0.13}{0}{0.43}\pgfsys@color<EMAIL_ADDRESS>and define the map
${{\color[rgb]{0,0.44,0.57}\definecolor[named]{pgfstrokecolor}{rgb}{0,0.44,0.57}\pgfsys@color@cmyk@stroke{0.98}{0.13}{0}{0.43}\pgfsys@color@cmyk@fill{0.98}{0.13}{0}{0.43}\mathcal{R}_{\psi_{i}}}}:\mathbb{R}^{d_{{\color[rgb]{0,0.44,0.57}\definecolor[named]{pgfstrokecolor}{rgb}{0,0.44,0.57}\pgfsys@color@cmyk@stroke{0.98}{0.13}{0}{0.43}\pgfsys@color<EMAIL_ADDRESS>n_{{\color[rgb]{0,0.44,0.57}\definecolor[named]{pgfstrokecolor}{rgb}{0,0.44,0.57}\pgfsys@color@cmyk@stroke{0.98}{0.13}{0}{0.43}\pgfsys@color@cmyk@fill{0.98}{0.13}{0}{0.43}i}}}\to\mathbb{R}^{d_{{\color[rgb]{0,0.44,0.57}\definecolor[named]{pgfstrokecolor}{rgb}{0,0.44,0.57}\pgfsys@color@cmyk@stroke{0.98}{0.13}{0}{0.43}\pgfsys@color<EMAIL_ADDRESS>d_{{\color[rgb]{0,0.44,0.57}\definecolor[named]{pgfstrokecolor}{rgb}{0,0.44,0.57}\pgfsys@color@cmyk@stroke{0.98}{0.13}{0}{0.43}\pgfsys@color<EMAIL_ADDRESS>that performs the inverse projection.

Using
${{\color[rgb]{0,0.44,0.57}\definecolor[named]{pgfstrokecolor}{rgb}{0,0.44,0.57}\pgfsys@color@cmyk@stroke{0.98}{0.13}{0}{0.43}\pgfsys@color<EMAIL_ADDRESS>we can describe $\mu(h)$ through its inverse Fourier Transform (IFT):

$\mu(h)=\sum_{{{\color[rgb]{0,0.44,0.57}\definecolor[named]{pgfstrokecolor}{rgb}{0,0.44,0.57}\pgfsys@color@cmyk@stroke{0.98}{0.13}{0}{0.43}\pgfsys@color@cmyk@fill{0.98}{0.13}{0}{0.43}\psi_{i}}}\in\widehat{H}}\sqrt{d_{{\color[rgb]{0,0.44,0.57}\definecolor[named]{pgfstrokecolor}{rgb}{0,0.44,0.57}\pgfsys@color@cmyk@stroke{0.98}{0.13}{0}{0.43}\pgfsys@color@cmyk@fill{0.98}{0.13}{0}{0.43}i}}}\text{Tr}\left({{\color[rgb]{0,0.44,0.57}\definecolor[named]{pgfstrokecolor}{rgb}{0,0.44,0.57}\pgfsys@color@cmyk@stroke{0.98}{0.13}{0}{0.43}\pgfsys@color@cmyk@fill{0.98}{0.13}{0}{0.43}\psi_{i}}}(h)^{\top}{{\color[rgb]{0,0.44,0.57}\definecolor[named]{pgfstrokecolor}{rgb}{0,0.44,0.57}\pgfsys@color@cmyk@stroke{0.98}{0.13}{0}{0.43}\pgfsys@color@cmyk@fill{0.98}{0.13}{0}{0.43}\mathcal{R}_{\psi_{i}}}}\left(\widehat{\mu}(\overline{{\color[rgb]{0,0.44,0.57}\definecolor[named]{pgfstrokecolor}{rgb}{0,0.44,0.57}\pgfsys@color@cmyk@stroke{0.98}{0.13}{0}{0.43}\pgfsys@color@cmyk@fill{0.98}{0.13}{0}{0.43}\psi}}_{{\color[rgb]{0,0.44,0.57}\definecolor[named]{pgfstrokecolor}{rgb}{0,0.44,0.57}\pgfsys@color@cmyk@stroke{0.98}{0.13}{0}{0.43}\pgfsys@color@cmyk@fill{0.98}{0.13}{0}{0.43}i}})\right)\right)$
(8)

Due to the orthogonality of the Fourier coefficients, by expanding $\mu$ via
its IFT, the integral in Eq. 7 simplifies to:

$\int\\!\mu(h)\\!\left({{\color[rgb]{1,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,0}\psi_{j}}}\\!\otimes\\!{{\color[rgb]{.5,0,.5}\definecolor[named]{pgfstrokecolor}{rgb}{.5,0,.5}\psi_{j^{\prime}}}}\right)(h)dh\\!=\\!Q\bigoplus_{{{\color[rgb]{0,0.44,0.57}\definecolor[named]{pgfstrokecolor}{rgb}{0,0.44,0.57}\pgfsys@color@cmyk@stroke{0.98}{0.13}{0}{0.43}\pgfsys@color@cmyk@fill{0.98}{0.13}{0}{0.43}i}}}\\!\bigoplus_{r}^{[{{\color[rgb]{0,0.44,0.57}\definecolor[named]{pgfstrokecolor}{rgb}{0,0.44,0.57}\pgfsys@color@cmyk@stroke{0.98}{0.13}{0}{0.43}\pgfsys@color@cmyk@fill{0.98}{0.13}{0}{0.43}i}}({\color[rgb]{1,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,0}j}{{\color[rgb]{.5,0,.5}\definecolor[named]{pgfstrokecolor}{rgb}{.5,0,.5}j^{\prime}}})]}\frac{{{\color[rgb]{0,0.44,0.57}\definecolor[named]{pgfstrokecolor}{rgb}{0,0.44,0.57}\pgfsys@color@cmyk@stroke{0.98}{0.13}{0}{0.43}\pgfsys@color@cmyk@fill{0.98}{0.13}{0}{0.43}\mathcal{R}_{\psi_{i}}}}\left(\widehat{\mu}(\overline{{\color[rgb]{0,0.44,0.57}\definecolor[named]{pgfstrokecolor}{rgb}{0,0.44,0.57}\pgfsys@color@cmyk@stroke{0.98}{0.13}{0}{0.43}\pgfsys@color@cmyk@fill{0.98}{0.13}{0}{0.43}\psi}}_{{\color[rgb]{0,0.44,0.57}\definecolor[named]{pgfstrokecolor}{rgb}{0,0.44,0.57}\pgfsys@color@cmyk@stroke{0.98}{0.13}{0}{0.43}\pgfsys@color@cmyk@fill{0.98}{0.13}{0}{0.43}i}})\right)}{\sqrt{d_{{\color[rgb]{0,0.44,0.57}\definecolor[named]{pgfstrokecolor}{rgb}{0,0.44,0.57}\pgfsys@color@cmyk@stroke{0.98}{0.13}{0}{0.43}\pgfsys@color<EMAIL_ADDRESS>(9)

As $\mu(h)$ is uniform, only the Fourier coefficients
$\widehat{\mu}(\psi_{0})=1$ corresponding to the trivial representation
$\psi_{0}$ with size $d_{\psi_{0}}=1$ are non-zero. As such, the summation
selects only certain relevant columns
$\\{Q_{r}=\operatorname{CG}_{r}^{0({\color[rgb]{1,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,0}j}{{\color[rgb]{.5,0,.5}\definecolor[named]{pgfstrokecolor}{rgb}{.5,0,.5}j^{\prime}}})}\\}_{r}^{[0({\color[rgb]{1,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,0}j}{{\color[rgb]{.5,0,.5}\definecolor[named]{pgfstrokecolor}{rgb}{.5,0,.5}j^{\prime}}})]}$
from $Q$:

$\int\\!\mu(h)\\!\left({{\color[rgb]{1,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,0}\psi_{j}}}\\!\otimes\\!{{\color[rgb]{.5,0,.5}\definecolor[named]{pgfstrokecolor}{rgb}{.5,0,.5}\psi_{j^{\prime}}}}\right)(h)dh=\sum_{r}^{[0({\color[rgb]{1,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,0}j}{{\color[rgb]{.5,0,.5}\definecolor[named]{pgfstrokecolor}{rgb}{.5,0,.5}j^{\prime}}})]}Q_{r}Q_{r}^{\top}.$
(10)

One can verify444 Eq. 10 essentially defines the orthogonal projection of a
(vectorised) matrix into the space of (vectorised) matrices commuting with
${{\color[rgb]{1,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,0}\psi_{j}}}$
and
${{\color[rgb]{.5,0,.5}\definecolor[named]{pgfstrokecolor}{rgb}{.5,0,.5}\psi_{j^{\prime}}}}$.
that the set
$\\{\operatorname{unvec}(Q_{r})\\}_{r}^{[0({\color[rgb]{1,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,0}j}{{\color[rgb]{.5,0,.5}\definecolor[named]{pgfstrokecolor}{rgb}{.5,0,.5}j^{\prime}}})]}$
forms a basis for all equivariant maps from
${{\color[rgb]{1,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,0}\psi_{j}}}$
to
${{\color[rgb]{.5,0,.5}\definecolor[named]{pgfstrokecolor}{rgb}{.5,0,.5}\psi_{j^{\prime}}}}$.
Hence, whenever
${{\color[rgb]{1,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,0}\psi_{j}}}\neq{{\color[rgb]{.5,0,.5}\definecolor[named]{pgfstrokecolor}{rgb}{.5,0,.5}\psi_{j^{\prime}}}}$,
$[0({{\color[rgb]{1,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,0}j}}{{\color[rgb]{.5,0,.5}\definecolor[named]{pgfstrokecolor}{rgb}{.5,0,.5}j^{\prime}}})]=0$
(_Schur’s Lemma_ Apx. B.41); in this case, Eq. 10 results in a null-map and
the corresponding weights can be discarded. Then, Eq. 6 turns into:

$\begin{split}\kappa(x)\\!=\\!\sum_{{\color[rgb]{1,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,0}j},{\color[rgb]{1,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,0}k}}\\!\sum_{s}^{[{\color[rgb]{1,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,0}j}(Jl)]}\\!\left[{{\color[rgb]{1,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,0}\operatorname{CG}_{s}^{j(Jl)}}}\right]^{\top}\\!\operatorname{unvec}\left[\sum_{r}^{[0({\color[rgb]{1,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,0}j}{\color[rgb]{1,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,0}j})]}c_{r}^{{\color[rgb]{1,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,0}j}}W_{{\color[rgb]{1,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,0}j},{\color[rgb]{1,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,0}k},s,r}\right]\\!{\color[rgb]{1,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,0}Y}_{{\color[rgb]{1,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,0}j}}^{{\color[rgb]{1,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,0}k}}(x),\end{split}$
(11)

where $Q_{r}^{\top}\in\mathbb{R}^{1\times
d_{{\color[rgb]{1,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,0}j}}d_{{{\color[rgb]{.5,0,.5}\definecolor[named]{pgfstrokecolor}{rgb}{.5,0,.5}j^{\prime}}}}}$
and
$W_{{\color[rgb]{1,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,0}j},{{\color[rgb]{.5,0,.5}\definecolor[named]{pgfstrokecolor}{rgb}{.5,0,.5}j^{\prime}}},{\color[rgb]{1,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,0}k},s}$
have been merged into a single weight
$W_{{\color[rgb]{1,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,0}j},{\color[rgb]{1,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,0}k},s,r}$
to reduce parameters.

##  3 Learning Equivariance

This section details how we obtain an interpretable and learnable degree of
equivariance in SCNNs.

###  3.1 Leveraging the Averaging Operator

In SCNNs, equivariance stems from the uniform averaging over the group $H$
performed during the projection of the unconstrained kernels, ensuring that
each element $h\in H$ is equally weighted. To learn the degree of
equivariance, we propose to replace the uniform likelihood distribution
$\mu(h)$ with a learnable likelihood distribution
${{\color[rgb]{1,0.58,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0.58,0}\pgfsys@color@cmyk@stroke{0}{0.42}{1}{0}\pgfsys@color@cmyk@fill{0}{0.42}{1}{0}\lambda}}(h)$.
As a result, we substitute $\mu(h)$ with
${{\color[rgb]{1,0.58,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0.58,0}\pgfsys@color@cmyk@stroke{0}{0.42}{1}{0}\pgfsys@color@cmyk@fill{0}{0.42}{1}{0}\lambda}}(h)$
in Eq. 9:

$\int\\!{{\color[rgb]{1,0.58,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0.58,0}\pgfsys@color@cmyk@stroke{0}{0.42}{1}{0}\pgfsys@color@cmyk@fill{0}{0.42}{1}{0}\lambda}}(h)\\!\left({{\color[rgb]{1,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,0}\psi_{j}}}\\!\otimes\\!{{\color[rgb]{.5,0,.5}\definecolor[named]{pgfstrokecolor}{rgb}{.5,0,.5}\psi_{j^{\prime}}}}\right)(h)dh\\!=\\!Q\bigoplus_{{{\color[rgb]{0,0.44,0.57}\definecolor[named]{pgfstrokecolor}{rgb}{0,0.44,0.57}\pgfsys@color@cmyk@stroke{0.98}{0.13}{0}{0.43}\pgfsys@color@cmyk@fill{0.98}{0.13}{0}{0.43}i}}}\\!\bigoplus_{r}^{[{{\color[rgb]{0,0.44,0.57}\definecolor[named]{pgfstrokecolor}{rgb}{0,0.44,0.57}\pgfsys@color@cmyk@stroke{0.98}{0.13}{0}{0.43}\pgfsys@color@cmyk@fill{0.98}{0.13}{0}{0.43}i}}({\color[rgb]{1,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,0}j}{{\color[rgb]{.5,0,.5}\definecolor[named]{pgfstrokecolor}{rgb}{.5,0,.5}j^{\prime}}})]}\frac{{{\color[rgb]{0,0.44,0.57}\definecolor[named]{pgfstrokecolor}{rgb}{0,0.44,0.57}\pgfsys@color@cmyk@stroke{0.98}{0.13}{0}{0.43}\pgfsys@color@cmyk@fill{0.98}{0.13}{0}{0.43}\mathcal{R}_{{{\color[rgb]{0,0.44,0.57}\definecolor[named]{pgfstrokecolor}{rgb}{0,0.44,0.57}\pgfsys@color@cmyk@stroke{0.98}{0.13}{0}{0.43}\pgfsys@color@cmyk@fill{0.98}{0.13}{0}{0.43}\psi_{i}}}}}}\left(\widehat{{{\color[rgb]{1,0.58,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0.58,0}\pgfsys@color@cmyk@stroke{0}{0.42}{1}{0}\pgfsys@color@cmyk@fill{0}{0.42}{1}{0}\lambda}}}(\overline{{\color[rgb]{0,0.44,0.57}\definecolor[named]{pgfstrokecolor}{rgb}{0,0.44,0.57}\pgfsys@color@cmyk@stroke{0.98}{0.13}{0}{0.43}\pgfsys@color@cmyk@fill{0.98}{0.13}{0}{0.43}\psi}}_{{\color[rgb]{0,0.44,0.57}\definecolor[named]{pgfstrokecolor}{rgb}{0,0.44,0.57}\pgfsys@color@cmyk@stroke{0.98}{0.13}{0}{0.43}\pgfsys@color@cmyk@fill{0.98}{0.13}{0}{0.43}i}})\right)}{\sqrt{d_{{\color[rgb]{0,0.44,0.57}\definecolor[named]{pgfstrokecolor}{rgb}{0,0.44,0.57}\pgfsys@color@cmyk@stroke{0.98}{0.13}{0}{0.43}\pgfsys@color<EMAIL_ADDRESS>(12)

While the uniformity of $\mu(h)$ meant only the Fourier coefficient of the
trivial representation had to be considered,
${{\color[rgb]{1,0.58,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0.58,0}\pgfsys@color@cmyk@stroke{0}{0.42}{1}{0}\pgfsys@color@cmyk@fill{0}{0.42}{1}{0}\lambda}}(h)$
is not necessarily uniform. As such, the Fourier coefficient of any
representation
${{\color[rgb]{0,0.44,0.57}\definecolor[named]{pgfstrokecolor}{rgb}{0,0.44,0.57}\pgfsys@color@cmyk@stroke{0.98}{0.13}{0}{0.43}\pgfsys@color<EMAIL_ADDRESS>can be nonzero, and the previous simplifications can no longer be performed.
Thus, using weights
$W_{{\color[rgb]{1,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,0}j},{{\color[rgb]{.5,0,.5}\definecolor[named]{pgfstrokecolor}{rgb}{.5,0,.5}j^{\prime}}},{\color[rgb]{1,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,0}{\color[rgb]{1,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,0}k}},s}\in\mathbb{R}^{d_{{\color[rgb]{1,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,0}j}}d_{{{\color[rgb]{.5,0,.5}\definecolor[named]{pgfstrokecolor}{rgb}{.5,0,.5}j^{\prime}}}}}$
the final projection becomes555Note here that the summation over
${{\color[rgb]{.5,0,.5}\definecolor[named]{pgfstrokecolor}{rgb}{.5,0,.5}j^{\prime}}}$
has returned, as the projection no longer amounts to a nullmap if
${{\color[rgb]{1,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,0}\psi_{j}}}\neq{{\color[rgb]{.5,0,.5}\definecolor[named]{pgfstrokecolor}{rgb}{.5,0,.5}\psi_{j^{\prime}}}}$.:

$\kappa(x)=\sum_{{\color[rgb]{1,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,0}j},{\color[rgb]{1,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,0}{\color[rgb]{1,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,0}k}}}\sum_{{{\color[rgb]{.5,0,.5}\definecolor[named]{pgfstrokecolor}{rgb}{.5,0,.5}j^{\prime}}},s}\left[{{\color[rgb]{.5,0,.5}\definecolor[named]{pgfstrokecolor}{rgb}{.5,0,.5}\operatorname{CG}_{s}^{{{\color[rgb]{.5,0,.5}\definecolor[named]{pgfstrokecolor}{rgb}{.5,0,.5}j^{\prime}}}(Jl)}}}\right]^{\top}\operatorname{unvec}\left[c^{{\color[rgb]{1,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,0}j}{{\color[rgb]{.5,0,.5}\definecolor[named]{pgfstrokecolor}{rgb}{.5,0,.5}j^{\prime}}}}W_{{\color[rgb]{1,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,0}j},{{\color[rgb]{.5,0,.5}\definecolor[named]{pgfstrokecolor}{rgb}{.5,0,.5}j^{\prime}}},{\color[rgb]{1,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,0}{\color[rgb]{1,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,0}k}},s}\right]{{\color[rgb]{1,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,0}Y}}_{{\color[rgb]{1,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,0}j}}^{{\color[rgb]{1,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,0}{\color[rgb]{1,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,0}k}}}(x),$
(13)

where
$c^{{\color[rgb]{1,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,0}j}{{\color[rgb]{.5,0,.5}\definecolor[named]{pgfstrokecolor}{rgb}{.5,0,.5}j^{\prime}}}}$
is defined using the Fourier coefficients:

$c^{{\color[rgb]{1,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,0}j}{{\color[rgb]{.5,0,.5}\definecolor[named]{pgfstrokecolor}{rgb}{.5,0,.5}j^{\prime}}}}=Q\left(\bigoplus_{{{\color[rgb]{0,0.44,0.57}\definecolor[named]{pgfstrokecolor}{rgb}{0,0.44,0.57}\pgfsys@color@cmyk@stroke{0.98}{0.13}{0}{0.43}\pgfsys@color@cmyk@fill{0.98}{0.13}{0}{0.43}i}}}\\!\bigoplus_{r}^{[{{\color[rgb]{0,0.44,0.57}\definecolor[named]{pgfstrokecolor}{rgb}{0,0.44,0.57}\pgfsys@color@cmyk@stroke{0.98}{0.13}{0}{0.43}\pgfsys@color@cmyk@fill{0.98}{0.13}{0}{0.43}i}}({\color[rgb]{1,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,0}j}{{\color[rgb]{.5,0,.5}\definecolor[named]{pgfstrokecolor}{rgb}{.5,0,.5}j^{\prime}}})]}\frac{{{\color[rgb]{0,0.44,0.57}\definecolor[named]{pgfstrokecolor}{rgb}{0,0.44,0.57}\pgfsys@color@cmyk@stroke{0.98}{0.13}{0}{0.43}\pgfsys@color@cmyk@fill{0.98}{0.13}{0}{0.43}\mathcal{R}_{{{\color[rgb]{0,0.44,0.57}\definecolor[named]{pgfstrokecolor}{rgb}{0,0.44,0.57}\pgfsys@color@cmyk@stroke{0.98}{0.13}{0}{0.43}\pgfsys@color@cmyk@fill{0.98}{0.13}{0}{0.43}\psi_{i}}}}}}\left(\widehat{{{\color[rgb]{1,0.58,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0.58,0}\pgfsys@color@cmyk@stroke{0}{0.42}{1}{0}\pgfsys@color@cmyk@fill{0}{0.42}{1}{0}\lambda}}}(\overline{{\color[rgb]{0,0.44,0.57}\definecolor[named]{pgfstrokecolor}{rgb}{0,0.44,0.57}\pgfsys@color@cmyk@stroke{0.98}{0.13}{0}{0.43}\pgfsys@color@cmyk@fill{0.98}{0.13}{0}{0.43}\psi}}_{{\color[rgb]{0,0.44,0.57}\definecolor[named]{pgfstrokecolor}{rgb}{0,0.44,0.57}\pgfsys@color@cmyk@stroke{0.98}{0.13}{0}{0.43}\pgfsys@color@cmyk@fill{0.98}{0.13}{0}{0.43}i}})\right)}{\sqrt{d_{{\color[rgb]{0,0.44,0.57}\definecolor[named]{pgfstrokecolor}{rgb}{0,0.44,0.57}\pgfsys@color@cmyk@stroke{0.98}{0.13}{0}{0.43}\pgfsys@color@cmyk@fill{0.98}{0.13}{0}{0.43}i}}}}\right).$
(14)

To learn the degree of equivariance we propose to initialise the Fourier
coefficients such that
${{\color[rgb]{1,0.58,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0.58,0}\pgfsys@color@cmyk@stroke{0}{0.42}{1}{0}\pgfsys@color@cmyk@fill{0}{0.42}{1}{0}\lambda}}(h)$
is a uniform distribution, and the projection is initially fully
$H$-equivariant:

$\widehat{{{\color[rgb]{1,0.58,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0.58,0}\pgfsys@color@cmyk@stroke{0}{0.42}{1}{0}\pgfsys@color@cmyk@fill{0}{0.42}{1}{0}\lambda}}}({{\color[rgb]{0,0.44,0.57}\definecolor[named]{pgfstrokecolor}{rgb}{0,0.44,0.57}\pgfsys@color@cmyk@stroke{0.98}{0.13}{0}{0.43}\pgfsys@color@cmyk@fill{0.98}{0.13}{0}{0.43}\psi_{i}}})=\begin{cases}1&{{\color[rgb]{0,0.44,0.57}\definecolor[named]{pgfstrokecolor}{rgb}{0,0.44,0.57}\pgfsys@color@cmyk@stroke{0.98}{0.13}{0}{0.43}\pgfsys@color<EMAIL_ADDRESS>\bm{0}_{d_{{\color[rgb]{0,0.44,0.57}\definecolor[named]{pgfstrokecolor}{rgb}{0,0.44,0.57}\pgfsys@color@cmyk@stroke{0.98}{0.13}{0}{0.43}\pgfsys@color@cmyk@fill{0.98}{0.13}{0}{0.43}i}}\times{n_{{\color[rgb]{0,0.44,0.57}\definecolor[named]{pgfstrokecolor}{rgb}{0,0.44,0.57}\pgfsys@color@cmyk@stroke{0.98}{0.13}{0}{0.43}\pgfsys@color@cmyk@fill{0.98}{0.13}{0}{0.43}i}}}}&{{\color[rgb]{0,0.44,0.57}\definecolor[named]{pgfstrokecolor}{rgb}{0,0.44,0.57}\pgfsys@color@cmyk@stroke{0.98}{0.13}{0}{0.43}\pgfsys@color<EMAIL_ADDRESS>0\end{cases}.$ (15)

These Fourier coefficients
$\widehat{{{\color[rgb]{1,0.58,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0.58,0}\pgfsys@color@cmyk@stroke{0}{0.42}{1}{0}\pgfsys@color@cmyk@fill{0}{0.42}{1}{0}\lambda}}}({{\color[rgb]{0,0.44,0.57}\definecolor[named]{pgfstrokecolor}{rgb}{0,0.44,0.57}\pgfsys@color@cmyk@stroke{0.98}{0.13}{0}{0.43}\pgfsys@color@cmyk@fill{0.98}{0.13}{0}{0.43}\psi_{i}}})$
are stored as learnable parameters and updated through back-propagation during
training, permitting a partially $H$-equivariant projection666In practice,
only the Fourier coefficients of irreps appearing in one of the Clebsch-Gordan
decompositions need to be considered..

###  3.2 Ensuring Interpretable Equivariance

Through IFT it is possible to construct the underlying likelihood distribution
${{{\color[rgb]{1,0.58,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0.58,0}\pgfsys@color@cmyk@stroke{0}{0.42}{1}{0}\pgfsys@color@cmyk@fill{0}{0.42}{1}{0}\lambda}}}(h)$
from the corresponding learnt Fourier coefficients
$\widehat{{{{\color[rgb]{1,0.58,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0.58,0}\pgfsys@color@cmyk@stroke{0}{0.42}{1}{0}\pgfsys@color@cmyk@fill{0}{0.42}{1}{0}\lambda}}}}({{\color[rgb]{0,0.44,0.57}\definecolor[named]{pgfstrokecolor}{rgb}{0,0.44,0.57}\pgfsys@color@cmyk@stroke{0.98}{0.13}{0}{0.43}\pgfsys@color@cmyk@fill{0.98}{0.13}{0}{0.43}\psi_{i}}})$.
However, a few steps are required to ensure interpretable likelihoods.

##### Normalisation

To ensure consistency in the magnitude of the likelihood distribution we
normalise the learnt likelihood distribution
${{{\color[rgb]{1,0.58,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0.58,0}\pgfsys@color@cmyk@stroke{0}{0.42}{1}{0}\pgfsys@color<EMAIL_ADDRESS>to a PDF. This improves interpretability, and also ensures that any potential
weight regularisation cannot simply uniformly increase the scale of the learnt
likelihood in favour of increasing the kernel weights. To achieve this, we
apply the Softmax non-linearity to ensure
${{{\color[rgb]{1,0.58,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0.58,0}\pgfsys@color@cmyk@stroke{0}{0.42}{1}{0}\pgfsys@color<EMAIL_ADDRESS>non-negativity and normalisation
$\int{{{\color[rgb]{1,0.58,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0.58,0}\pgfsys@color@cmyk@stroke{0}{0.42}{1}{0}\pgfsys@color@cmyk@fill{0}{0.42}{1}{0}\lambda}}}(h)dh=1$:

$\bm{\widehat{{{{\color[rgb]{1,0.58,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0.58,0}\pgfsys@color@cmyk@stroke{0}{0.42}{1}{0}\pgfsys@color@cmyk@fill{0}{0.42}{1}{0}\lambda}}}}}=\operatorname{FT}\left(\sigma\left(\operatorname{IFT}\left(\bm{\widehat{{{{\color[rgb]{1,0.58,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0.58,0}\pgfsys@color@cmyk@stroke{0}{0.42}{1}{0}\pgfsys@color@cmyk@fill{0}{0.42}{1}{0}\lambda}}}^{\prime}}}\right)\right)\right),$
(16)

where
$\bm{\widehat{{{{\color[rgb]{1,0.58,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0.58,0}\pgfsys@color@cmyk@stroke{0}{0.42}{1}{0}\pgfsys@color<EMAIL_ADDRESS>is the stack of Fourier coefficients of the density
${{{\color[rgb]{1,0.58,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0.58,0}\pgfsys@color@cmyk@stroke{0}{0.42}{1}{0}\pgfsys@color<EMAIL_ADDRESS>$\bm{\widehat{{{{\color[rgb]{1,0.58,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0.58,0}\pgfsys@color@cmyk@stroke{0}{0.42}{1}{0}\pgfsys@color<EMAIL_ADDRESS>the stack of learnable Fourier coefficients of its logits, and $\sigma(\cdot)$
is the Softmax non-linearity. As such, the following is computed for each $h$
in a discrete sampling set $\mathcal{H}\subset H$ of $N$ elements, using
$z_{{{\color[rgb]{1,0.58,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0.58,0}\pgfsys@color@cmyk@stroke{0}{0.42}{1}{0}\pgfsys@color@cmyk@fill{0}{0.42}{1}{0}\lambda}}}=\frac{1}{N}\sum_{n=1}^{N}e^{{{{\color[rgb]{1,0.58,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0.58,0}\pgfsys@color@cmyk@stroke{0}{0.42}{1}{0}\pgfsys@color@cmyk@fill{0}{0.42}{1}{0}\lambda}}}^{\prime}(n)-\max({{{\color[rgb]{1,0.58,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0.58,0}\pgfsys@color@cmyk@stroke{0}{0.42}{1}{0}\pgfsys@color@cmyk@fill{0}{0.42}{1}{0}\lambda}}}^{\prime})}$777We
subtract the max value to make softmax numerically stable.:

${{{\color[rgb]{1,0.58,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0.58,0}\pgfsys@color@cmyk@stroke{0}{0.42}{1}{0}\pgfsys@color@cmyk@fill{0}{0.42}{1}{0}\lambda}}}(h)=\sigma({{{\color[rgb]{1,0.58,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0.58,0}\pgfsys@color@cmyk@stroke{0}{0.42}{1}{0}\pgfsys@color@cmyk@fill{0}{0.42}{1}{0}\lambda}}}^{\prime}(h))=\frac{e^{{{{\color[rgb]{1,0.58,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0.58,0}\pgfsys@color@cmyk@stroke{0}{0.42}{1}{0}\pgfsys@color@cmyk@fill{0}{0.42}{1}{0}\lambda}}}^{\prime}(h)-\max({{{\color[rgb]{1,0.58,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0.58,0}\pgfsys@color@cmyk@stroke{0}{0.42}{1}{0}\pgfsys@color@cmyk@fill{0}{0.42}{1}{0}\lambda}}}^{\prime})}}{z_{{{\color[rgb]{1,0.58,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0.58,0}\pgfsys@color@cmyk@stroke{0}{0.42}{1}{0}\pgfsys@color<EMAIL_ADDRESS>(17)

While other normalisation methods are also valid 888E.g., the use Parseval’s
identity might be a straightforward method to normalise the Fourier
coefficients without sampling., we opted for Softmax normalisation as the
$z$-term and logits can be used directly to compute the KL-divergence
described later in this section.

##### Likelihood Alignment

While the Fourier parameterisation enables the reconstruction of the
likelihood distribution
${{{\color[rgb]{1,0.58,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0.58,0}\pgfsys@color@cmyk@stroke{0}{0.42}{1}{0}\pgfsys@color<EMAIL_ADDRESS>there is no guarantee that the weights are properly aligned with the learnt
likelihood distribution. We show in Apx. C.3 that it is possible for the model
to freely shift the likelihood distribution by shifting the weights
accordingly, diminishing the interpretability of the distributions.

To address this concern, we propose constraining the Fourier coefficients by
aligning the maximum likelihood with the identity element $e\in H$. The
theoretical justification lies in the intrinsic impossibility of violating
equivariance when the identity element is applied. Hence, we desire:

$\forall h\in
H\colon{{{\color[rgb]{1,0.58,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0.58,0}\pgfsys@color@cmyk@stroke{0}{0.42}{1}{0}\pgfsys@color@cmyk@fill{0}{0.42}{1}{0}\lambda}}}(h)\leq{{{\color[rgb]{1,0.58,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0.58,0}\pgfsys@color@cmyk@stroke{0}{0.42}{1}{0}\pgfsys@color@cmyk@fill{0}{0.42}{1}{0}\lambda}}}(e)$
(18)

Since directly enforcing this constraint on the Fourier coefficients is hard,
we opt for a soft alignment constraint:

$D_{\text{align}}({{{\color[rgb]{1,0.58,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0.58,0}\pgfsys@color@cmyk@stroke{0}{0.42}{1}{0}\pgfsys@color@cmyk@fill{0}{0.42}{1}{0}\lambda}}})=\operatorname{max}({{{\color[rgb]{1,0.58,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0.58,0}\pgfsys@color@cmyk@stroke{0}{0.42}{1}{0}\pgfsys@color@cmyk@fill{0}{0.42}{1}{0}\lambda}}})-{{{\color[rgb]{1,0.58,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0.58,0}\pgfsys@color@cmyk@stroke{0}{0.42}{1}{0}\pgfsys@color@cmyk@fill{0}{0.42}{1}{0}\lambda}}}(e).$
(19)

This error can be added as a loss term during training. Provided that the
likelihood
${{{\color[rgb]{1,0.58,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0.58,0}\pgfsys@color@cmyk@stroke{0}{0.42}{1}{0}\pgfsys@color<EMAIL_ADDRESS>has been sampled using a sufficient number of samples $N$, this error term is
equal to zero if Eq. 18 holds, and greater than zero otherwise.

While this regularisation is rather sparse, as only the largest likelihood
contributes to the loss, we found that it is sufficient in ensuring proper
alignment and therefore did not consider less sparse approaches.

##### Regaining Equivariance

Once a layer breaks the equivariance, this equivariance cannot be accurately
regained in subsequent layers. Since our approach allows independent
parameterisations of the distributions in subsequent layers, the learnt
distributions do not necessarily describe such a monotonically decreasing
equivariance. To this end, we propose to employ KL-divergence as
regularisation between the layers. As the goal is to ensure that layer $n+1$
does not regain equivariance previously lost by layer $n$, we treat layer $n$
as reference distribution for layer $n+1$. To prevent layer $n+1$ from
influencing layer $n$, we detach the likelihood of layer $n$ and treat it as
constant (see Apx D.2 for more details on the potential advantage of this
detaching, and on monotonically decreasing equivariance in general). Given the
likelihoods
${{{\color[rgb]{1,0.58,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0.58,0}\pgfsys@color@cmyk@stroke{0}{0.42}{1}{0}\pgfsys@color<EMAIL_ADDRESS>and
${{{\color[rgb]{1,0.58,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0.58,0}\pgfsys@color@cmyk@stroke{0}{0.42}{1}{0}\pgfsys@color<EMAIL_ADDRESS>we show in Apx. C.4 that KL-divergence can be computed using the $z$-terms and
Fourier coefficients of the logits and normalised distribution:

$\begin{split}&D_{KL}({{{\color[rgb]{1,0.58,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0.58,0}\pgfsys@color@cmyk@stroke{0}{0.42}{1}{0}\pgfsys@color<EMAIL_ADDRESS>||\
{{{\color[rgb]{1,0.58,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0.58,0}\pgfsys@color@cmyk@stroke{0}{0.42}{1}{0}\pgfsys@color@cmyk@fill{0}{0.42}{1}{0}\lambda}}}_{n})=\bm{\widehat{{{{\color[rgb]{1,0.58,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0.58,0}\pgfsys@color@cmyk@stroke{0}{0.42}{1}{0}\pgfsys@color@cmyk@fill{0}{0.42}{1}{0}\lambda}}}_{n+1}}}^{\top}\bm{\widehat{{{{\color[rgb]{1,0.58,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0.58,0}\pgfsys@color@cmyk@stroke{0}{0.42}{1}{0}\pgfsys@color@cmyk@fill{0}{0.42}{1}{0}\lambda}}}_{n+1}^{\prime}}}-\operatorname{max}({{{\color[rgb]{1,0.58,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0.58,0}\pgfsys@color@cmyk@stroke{0}{0.42}{1}{0}\pgfsys@color@cmyk@fill{0}{0.42}{1}{0}\lambda}}}_{n+1}^{\prime})\\\
&-\log
z_{{{\color[rgb]{1,0.58,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0.58,0}\pgfsys@color@cmyk@stroke{0}{0.42}{1}{0}\pgfsys@color@cmyk@fill{0}{0.42}{1}{0}\lambda}}_{n+1}}-\bm{\widehat{{{{\color[rgb]{1,0.58,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0.58,0}\pgfsys@color@cmyk@stroke{0}{0.42}{1}{0}\pgfsys@color<EMAIL_ADDRESS>\bm{\widehat{{{{\color[rgb]{1,0.58,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0.58,0}\pgfsys@color@cmyk@stroke{0}{0.42}{1}{0}\pgfsys@color@cmyk@fill{0}{0.42}{1}{0}\lambda}}}_{n}^{\prime}}}+\operatorname{max}({{{\color[rgb]{1,0.58,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0.58,0}\pgfsys@color@cmyk@stroke{0}{0.42}{1}{0}\pgfsys@color@cmyk@fill{0}{0.42}{1}{0}\lambda}}}_{n}^{\prime})+\log
z_{{{\color[rgb]{1,0.58,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0.58,0}\pgfsys@color@cmyk@stroke{0}{0.42}{1}{0}\pgfsys@color<EMAIL_ADDRESS>(20)

###  3.3 Bandlimiting

While our approach allows for a flexible and interpretable degree of
equivariance, it also significantly increases the number of learnable
parameters. In the equivariant setting in Eq. 11, for each irrep
${{\color[rgb]{1,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,0}\psi_{j}}}$
from the steerable basis, the kernel is parameterised by
$[0({{\color[rgb]{1,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,0}jj}})]$
single weights
$W_{{{\color[rgb]{1,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,0}j}},{\color[rgb]{1,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,0}k},s,r}$.
Instead, our projection in Eq. 13 is parameterised by a vector of size
$d_{{\color[rgb]{1,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,0}j}}d_{{\color[rgb]{.5,0,.5}\definecolor[named]{pgfstrokecolor}{rgb}{.5,0,.5}j^{\prime}}}=\sum_{{\color[rgb]{0,0.44,0.57}\definecolor[named]{pgfstrokecolor}{rgb}{0,0.44,0.57}\pgfsys@color@cmyk@stroke{0.98}{0.13}{0}{0.43}\pgfsys@color@cmyk@fill{0.98}{0.13}{0}{0.43}i}}[{{\color[rgb]{0,0.44,0.57}\definecolor[named]{pgfstrokecolor}{rgb}{0,0.44,0.57}\pgfsys@color@cmyk@stroke{0.98}{0.13}{0}{0.43}\pgfsys@color@cmyk@fill{0.98}{0.13}{0}{0.43}i}}({{\color[rgb]{1,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,0}j}}{{\color[rgb]{.5,0,.5}\definecolor[named]{pgfstrokecolor}{rgb}{.5,0,.5}j^{\prime}}})]\cdot
d_{{\color[rgb]{0,0.44,0.57}\definecolor[named]{pgfstrokecolor}{rgb}{0,0.44,0.57}\pgfsys@color@cmyk@stroke{0.98}{0.13}{0}{0.43}\pgfsys@color<EMAIL_ADDRESS>for each pair of
${{{\color[rgb]{1,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,0}\psi_{j}}}}$
and
${{\color[rgb]{.5,0,.5}\definecolor[named]{pgfstrokecolor}{rgb}{.5,0,.5}\psi_{j^{\prime}}}}$.

Recall the composition of the partially equivariant projection matrix
$c^{{{\color[rgb]{1,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,0}j}}{{\color[rgb]{.5,0,.5}\definecolor[named]{pgfstrokecolor}{rgb}{.5,0,.5}j^{\prime}}}}$
in Eq. 14. The size of the matrix is dependant on the number of irreps
${{\color[rgb]{0,0.44,0.57}\definecolor[named]{pgfstrokecolor}{rgb}{0,0.44,0.57}\pgfsys@color@cmyk@stroke{0.98}{0.13}{0}{0.43}\pgfsys@color<EMAIL_ADDRESS>appearing in the tensor product
${{\color[rgb]{1,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,0}\psi_{j}}}\otimes{{\color[rgb]{.5,0,.5}\definecolor[named]{pgfstrokecolor}{rgb}{.5,0,.5}\psi_{j^{\prime}}}}$.
For example, if we consider the group $SO(3)$, whose irreps are indexed by
their rotational frequency,
${{\color[rgb]{1,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,0}\psi_{j}}}\otimes{{\color[rgb]{.5,0,.5}\definecolor[named]{pgfstrokecolor}{rgb}{.5,0,.5}\psi_{j^{\prime}}}}$
contains all irreps
${{\color[rgb]{0,0.44,0.57}\definecolor[named]{pgfstrokecolor}{rgb}{0,0.44,0.57}\pgfsys@color@cmyk@stroke{0.98}{0.13}{0}{0.43}\pgfsys@color<EMAIL_ADDRESS>with frequencies in the range
${{\color[rgb]{0,0.44,0.57}\definecolor[named]{pgfstrokecolor}{rgb}{0,0.44,0.57}\pgfsys@color@cmyk@stroke{0.98}{0.13}{0}{0.43}\pgfsys@color@cmyk@fill{0.98}{0.13}{0}{0.43}i}}\in|{{\color[rgb]{1,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,0}j}}-{{\color[rgb]{.5,0,.5}\definecolor[named]{pgfstrokecolor}{rgb}{.5,0,.5}j^{\prime}}}|,\dots,{{\color[rgb]{1,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,0}j}}+{{\color[rgb]{.5,0,.5}\definecolor[named]{pgfstrokecolor}{rgb}{.5,0,.5}j^{\prime}}}$.
As such, we propose to reduce the number of parameters through a tunable
bandlimit $L$ on the maximum frequency of irreps
${{\color[rgb]{0,0.44,0.57}\definecolor[named]{pgfstrokecolor}{rgb}{0,0.44,0.57}\pgfsys@color@cmyk@stroke{0.98}{0.13}{0}{0.43}\pgfsys@color<EMAIL_ADDRESS>reducing the parameter space and the flexibility of the distributions.

We note that band-limiting is not strictly required to obtain a reasonable
number of parameters compared to other partially equivariant approaches.
Instead, it is an optional tool to reduce computational complexity and filter
higher frequencies as a form of regularisation (see Apx. G.5.3)

###  3.4 Implementation

Our implementation999We will publish the implementation at
https://github.com/QUVA-Lab/partial-escnn and integrate it into the escnn
library https://github.com/QUVA-Lab/escnn. relies on the escnn library (Cesa
et al., 2022), providing an intuitive environment for configuring SCNNs with
compact groups $H$ and obtaining the corresponding $H$-steerable basis
$\mathcal{B}$. For our solution, we adapt Algorithm 1 in (Cesa et al., 2022),
which we present in Algorithm 1. An important consideration is that the escnn
library is optimised for equivariant SCNNs. As such, our implementation on top
of this library is likely sub-optimal. E.g., unlike in regular SCNNs acting on
a static space $X$, our current implementation resamples the steerable basis
after each forward pass, which may not be necessary.

Algorithm 1 Generate a learnable equivariant $H$-steerable basis on space $X$

0: $\rho_{\text{in}}=\psi_{l}$ and $\rho_{\text{out}}=\psi_{J}$, $H$-steerable
basis
$\mathcal{B}=\left\\{{{\color[rgb]{1,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,0}Y_{j}^{{\color[rgb]{1,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,0}k}}}}\right\\}_{{{\color[rgb]{1,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,0}j}}}^{{\color[rgb]{1,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,0}k}}$,
$\widehat{H}=\left\\{{{\color[rgb]{1,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,0}\psi_{j}}}\right\\}_{{\color[rgb]{1,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,0}j}},\
x\in X,L\in\mathbb{N}^{+}$, stack of logit coefficients
$\bm{\widehat{{{\color[rgb]{1,0.58,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0.58,0}\pgfsys@color@cmyk@stroke{0}{0.42}{1}{0}\pgfsys@color<EMAIL_ADDRESS>
1:
${\color[rgb]{.5,0,.5}\definecolor[named]{pgfstrokecolor}{rgb}{.5,0,.5}\operatorname{CG}^{lJ}}$,
$\left\\{[{{\color[rgb]{.5,0,.5}\definecolor[named]{pgfstrokecolor}{rgb}{.5,0,.5}j^{\prime}}}(Jl)]\right\\}_{{{\color[rgb]{.5,0,.5}\definecolor[named]{pgfstrokecolor}{rgb}{.5,0,.5}j^{\prime}}}}\leftarrow$
decompose$(\psi_{l}\otimes\psi_{J})$ Eq. 4 (Thm. B.37)

2: for all
${{\color[rgb]{1,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,0}Y_{j}^{{\color[rgb]{1,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,0}k}}}}\in\mathcal{B}$
do

3: for all
${{\color[rgb]{.5,0,.5}\definecolor[named]{pgfstrokecolor}{rgb}{.5,0,.5}j^{\prime}}}\in\widehat{H}:[{{\color[rgb]{.5,0,.5}\definecolor[named]{pgfstrokecolor}{rgb}{.5,0,.5}j^{\prime}}}(Jl)]>0,s\leq[{{\color[rgb]{.5,0,.5}\definecolor[named]{pgfstrokecolor}{rgb}{.5,0,.5}j^{\prime}}}(Jl)]$
do

4:
$\left\\{\widehat{{{\color[rgb]{1,0.58,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0.58,0}\pgfsys@color@cmyk@stroke{0}{0.42}{1}{0}\pgfsys@color@cmyk@fill{0}{0.42}{1}{0}\lambda}}}(\overline{{\color[rgb]{0,0.44,0.57}\definecolor[named]{pgfstrokecolor}{rgb}{0,0.44,0.57}\pgfsys@color@cmyk@stroke{0.98}{0.13}{0}{0.43}\pgfsys@color@cmyk@fill{0.98}{0.13}{0}{0.43}\psi}}_{{\color[rgb]{0,0.44,0.57}\definecolor[named]{pgfstrokecolor}{rgb}{0,0.44,0.57}\pgfsys@color@cmyk@stroke{0.98}{0.13}{0}{0.43}\pgfsys@color@cmyk@fill{0.98}{0.13}{0}{0.43}i}})\right\\}_{{\color[rgb]{0,0.44,0.57}\definecolor[named]{pgfstrokecolor}{rgb}{0,0.44,0.57}\pgfsys@color@cmyk@stroke{0.98}{0.13}{0}{0.43}\pgfsys@color<EMAIL_ADDRESS>unstack$\left(\operatorname{FT}\left(\sigma\left(\operatorname{IFT}\left(\bm{\widehat{{{\color[rgb]{1,0.58,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0.58,0}\pgfsys@color@cmyk@stroke{0}{0.42}{1}{0}\pgfsys@color@cmyk@fill{0}{0.42}{1}{0}\lambda}}^{\prime}}}\right)\right)\right)\right)$
Eq. 17

5:
$Q,\\{[{{\color[rgb]{0,0.44,0.57}\definecolor[named]{pgfstrokecolor}{rgb}{0,0.44,0.57}\pgfsys@color@cmyk@stroke{0.98}{0.13}{0}{0.43}\pgfsys@color@cmyk@fill{0.98}{0.13}{0}{0.43}i}}({{\color[rgb]{1,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,0}j}}{{\color[rgb]{.5,0,.5}\definecolor[named]{pgfstrokecolor}{rgb}{.5,0,.5}j^{\prime}}})]\\}_{i}\leftarrow\operatorname{decompose}({{\color[rgb]{1,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,0}\psi_{j}}}\otimes{{\color[rgb]{.5,0,.5}\definecolor[named]{pgfstrokecolor}{rgb}{.5,0,.5}\psi_{j^{\prime}}}})$
Eq. 7 (Thm. B.37)

6: $Q\leftarrow$ bandlimit$(Q,L)$ Sec. 3.3

7:
$c^{{\color[rgb]{1,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,0}j}{{\color[rgb]{.5,0,.5}\definecolor[named]{pgfstrokecolor}{rgb}{.5,0,.5}j^{\prime}}}}\leftarrow
Q\cdot$
diagonal$\left(\operatorname{repeat}\left(\left\\{\widehat{{{\color[rgb]{1,0.58,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0.58,0}\pgfsys@color@cmyk@stroke{0}{0.42}{1}{0}\pgfsys@color@cmyk@fill{0}{0.42}{1}{0}\lambda}}}(\overline{{\color[rgb]{0,0.44,0.57}\definecolor[named]{pgfstrokecolor}{rgb}{0,0.44,0.57}\pgfsys@color@cmyk@stroke{0.98}{0.13}{0}{0.43}\pgfsys@color@cmyk@fill{0.98}{0.13}{0}{0.43}\psi}}_{{\color[rgb]{0,0.44,0.57}\definecolor[named]{pgfstrokecolor}{rgb}{0,0.44,0.57}\pgfsys@color@cmyk@stroke{0.98}{0.13}{0}{0.43}\pgfsys@color@cmyk@fill{0.98}{0.13}{0}{0.43}i}})\right\\}_{i}\right)\right)$
Repeats
$[{{\color[rgb]{0,0.44,0.57}\definecolor[named]{pgfstrokecolor}{rgb}{0,0.44,0.57}\pgfsys@color@cmyk@stroke{0.98}{0.13}{0}{0.43}\pgfsys@color@cmyk@fill{0.98}{0.13}{0}{0.43}i}}({{\color[rgb]{1,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,0}j}}{{\color[rgb]{.5,0,.5}\definecolor[named]{pgfstrokecolor}{rgb}{.5,0,.5}j^{\prime}}})]$
times each
${{\color[rgb]{0,0.44,0.57}\definecolor[named]{pgfstrokecolor}{rgb}{0,0.44,0.57}\pgfsys@color@cmyk@stroke{0.98}{0.13}{0}{0.43}\pgfsys@color@cmyk@fill{0.98}{0.13}{0}{0.43}\mathcal{R}_{{{\color[rgb]{0,0.44,0.57}\definecolor[named]{pgfstrokecolor}{rgb}{0,0.44,0.57}\pgfsys@color@cmyk@stroke{0.98}{0.13}{0}{0.43}\pgfsys@color@cmyk@fill{0.98}{0.13}{0}{0.43}\psi_{i}}}}}}\left(\widehat{{\color[rgb]{1,0.58,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0.58,0}\pgfsys@color@cmyk@stroke{0}{0.42}{1}{0}\pgfsys@color@cmyk@fill{0}{0.42}{1}{0}\lambda}}(\overline{{\color[rgb]{0,0.44,0.57}\definecolor[named]{pgfstrokecolor}{rgb}{0,0.44,0.57}\pgfsys@color@cmyk@stroke{0.98}{0.13}{0}{0.43}\pgfsys@color@cmyk@fill{0.98}{0.13}{0}{0.43}\psi}}_{{\color[rgb]{0,0.44,0.57}\definecolor[named]{pgfstrokecolor}{rgb}{0,0.44,0.57}\pgfsys@color@cmyk@stroke{0.98}{0.13}{0}{0.43}\pgfsys@color@cmyk@fill{0.98}{0.13}{0}{0.43}i}})\right)$
and diagonalises result (Eq. 14)

8:
$K_{{{\color[rgb]{1,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,0}j}}{{\color[rgb]{1,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,0}k}}{{\color[rgb]{.5,0,.5}\definecolor[named]{pgfstrokecolor}{rgb}{.5,0,.5}j^{\prime}}}s}(x)\leftarrow\operatorname{unvec}\left(\left[{\color[rgb]{.5,0,.5}\definecolor[named]{pgfstrokecolor}{rgb}{.5,0,.5}\operatorname{CG}_{s}^{j^{\prime}(Jl)}}\right]^{\top}\cdot
c^{{{\color[rgb]{1,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,0}j}}{{\color[rgb]{.5,0,.5}\definecolor[named]{pgfstrokecolor}{rgb}{.5,0,.5}j^{\prime}}}}\cdot{{\color[rgb]{1,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,0}Y_{{{\color[rgb]{1,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,0}j}}}^{{\color[rgb]{1,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,0}k}}}}(x)\right)$
Eq. 13

##  4 Related Work

Group CNNs (GCNNs) (Cohen & Welling, 2016a), extend the application of the
regular convolution operator to the domain of a group to obtain group
equivariance using discrete groups. Steerable CNNs (SCNNs) (Cohen & Welling,
2016b) provide a more general theory of equivariance, offering more
flexibility, the ability to model continuous groups and some computational
advantages for large groups. Previous works proposed other strategies to
construct equivariance, see for example (Worrall et al., 2017; Kondor &
Trivedi, 2018; Bekkers, 2019; Finzi et al., 2020)

Several methods have recently been proposed to obtain varying degrees of
equivariance in equivariant NNs. Weiler & Cesa (2019) offer a manual solution
via group restrictions. Other works allow the degree of equivariance to be
learnt by the network. Partial GCNNs (Romero & Lohit, 2022) parameterise a
learnable probability distribution over the elements of the subgroup $H$ of
$G$. Rather than uniformly sampling group elements as in traditional GCNNs,
Partial GCNNs use this learnt probability distribution to guide the sampling
of group elements, achieving a form of partial $H$-equivariance as a result.
Non-stationary GCNNs (van der Ouderaa et al., 2022) utilised non-stationary
kernels to achieve partial $G$-equivariance. Proposed by Finzi et al. (2021),
Residual Pathway Priors (RPP) offer a more versatile strategy that can be
applied across a variety of equivariant networks. In RPPs, each intermediate
feature field comprises (at least) two layers: one that maintains
$G$-equivariance and another with softer constraints, with a predetermined
prior controlling the contribution. Based on non-stationary GCNNs and RPPs,
van der Ouderaa et al. (2023) took a more unified approach and employed
Bayesian model selection using differentiable Laplace approximations. They
learnt layer-wise equivariances as hyper-parameters, offering a framework for
discovering symmetries at different scales. Dehmamy et al. (2021) define group
convolution in terms of the Lie-algebra: by learning the algebra basis, their
model is able to learn the required symmetries. Wang et al. (2022a) use lift
expansion to achieve partial equivariance for $Q$ learning by concatenating
encoded non-equivariant features to encoded equivariant features in an
encoder-decoder setting. Wang et al. (2022b) introduce RSteer and RGroup, SCNN
and GCNN based approaches that break equivariance through a regularised
function mapping between group element and partially equivariant weights.
Finally, Maile et al. (2023) propose an equivariance-aware neural architecture
search method to discover equivariance during training.

To learn the equivariance with respect to a specific subgroup $S$ of $H$, RPPs
require a residual connection for each of these subgroups. This is
computationally infeasible, as it would require an infinite number of residual
connections for continuous groups. Furthermore, enumerating such subgroups is
complicated for 3D rotation groups. Specifically, the collection of rotations
around any arbitrary 3D axis constitutes a subgroup of $SO(3)$. However,
sampling uniformly over the space of 3D axes is not straightforward and
requires a large number of samples and, therefore, additional residual layers.
Thus, RPPs are unfeasible for learning equivariance to an unknown subgroup $S$
of a large or continuous group $H$.

Like RPPs, our approach models partial equivariance as a combination of
equivariant and non-equivariant mappings. Indeed, RPP is comparable to our
method under a specific and non-learnable choice of distribution $\lambda$.
Furthermore, similar to Partial GCNNs, our approach uses a learnt likelihood
over the group as a sampling mechanism. Whereas Partial GCNNs directly sample
this likelihood to perform group convolutions, our solution implicitly
integrates the learnt likelihood within the SCNN framework. Through this
likelihood, we can model equivariance with respect to any subgroup $S$ of $H$,
or even subsets $S$ that are not proper groups, without additional layers. See
Apx. D.1 for an in-depth comparison between these approaches.

##  5 Experiments and Evaluation

Network Group |  Partial Equivariance | Symmetries for individual digits
---|---|---
$\bm{C_{1}}$ | $\bm{C_{4}}$ | $\bm{SO(2)}$ | $\bm{D_{1}}$ | $\bm{D_{4}}$ | $\bm{O(2)}$
CNN | N/A |  $\bm{\underline{0.962}}\ $$(0.002)$ |  $\underline{0.868}\ $$(0.011)$ |  $\underline{0.807}\ $$(0.007)$ |  $\underline{0.919}\ $$(0.006)$ |  $\underline{0.711}\ $$(0.009)$ |  $\underline{0.649}\ $$(0.019)$
$C_{4}$ | None |  $0.933\ $$(0.002)$ |  $0.484\ $$(0.008)$ |  $0.459\ $$(0.007)$ |  $0.893\ $$(0.005)$ |  $0.427\ $$(0.008)$ |  $0.405\ $$(0.008)$
Restriction |  $\underline{0.954}\ $$(0.003)$ |  $0.911\ $$(0.006)$ |  $0.877\ $$(0.009)$ |  $\underline{0.928}\ $$(0.006)$ |  $0.827\ $$(0.013)$ |  $0.776\ $$(0.018)$
RPP |  $0.937\ $$(0.006)$ |  $0.901\ $$(0.012)$ |  $0.867\ $$(0.025)$ |  $0.899\ $$(0.014)$ |  $0.821\ $$(0.023)$ |  $0.772\ $$(0.009)$
Ours |  $0.947\ $$(0.006)$ |  $\underline{0.916}\ $$(0.005)$ |  $\underline{0.891}\ $$(0.006)$ |  $0.923\ $$(0.007)$ |  $\underline{0.848}\ $$(0.007)$ |  $\underline{0.795}\ $$(0.011)$
$D_{4}$ | None |  $0.895\ $$(0.005)$ |  $0.439\ $$(0.010)$ |  $0.396\ $$(0.009)$ |  $0.473\ $$(0.010)$ |  $0.431\ $$(0.010)$ |  $0.394\ $$(0.008)$
Restriction |  $\underline{0.953}\ $$(0.004)$ |  $0.912\ $$(0.007)$ |  $\underline{0.887}\ $$(0.003)$ |  $\underline{0.930}\ $$(0.007)$ |  $0.827\ $$(0.007)$ |  $0.773\ $$(0.009)$
RPP |  $0.934\ $$(0.007)$ |  $0.888\ $$(0.014)$ |  $0.867\ $$(0.014)$ |  $0.895\ $$(0.007)$ |  $0.821\ $$(0.020)$ |  $0.775\ $$(0.013)$
Ours |  $0.949\ $$(0.005)$ |  $\underline{\bm{0.922}}\ $$(0.007)$ |  $0.885\ $$(0.012)$ |  $0.921\ $$(0.008)$ |  $\underline{0.848}\ $$(0.011)$ |  $\underline{0.801}\ $$(0.009)$
SO(2) | None |  $0.936\ $$(0.005)$ |  $0.485\ $$(0.010)$ |  $0.474\ $$(0.016)$ |  $0.890\ $$(0.006)$ |  $0.430\ $$(0.010)$ |  $0.403\ $$(0.021)$
Restriction |  $0.949\ $$(0.002)$ |  $0.911\ $$(0.010)$ |  $0.893\ $$(0.009)$ |  $0.928\ $$(0.003)$ |  $0.841\ $$(0.012)$ |  $0.796\ $$(0.011)$
RPP |  $0.935\ $$(0.008)$ |  $0.890\ $$(0.005)$ |  $0.870\ $$(0.011)$ |  $0.899\ $$(0.009)$ |  $0.821\ $$(0.022)$ |  $0.779\ $$(0.021)$
Ours |  $\underline{0.953}\ $$(0.004)$ |  $\underline{\bm{0.922}}\ $$(0.005)$ |  $\underline{\bm{0.901}}\ $$(0.005)$ |  $\bm{\underline{0.932}}\ $$(0.005)$ |  $\underline{\bm{0.863}}\ $$(0.009)$ |  $\underline{\bm{0.823}}\ $$(0.005)$
O(2) | None |  $0.881\ $$(0.005)$ |  $0.415\ $$(0.008)$ |  $0.391\ $$(0.012)$ |  $0.461\ $$(0.012)$ |  $0.424\ $$(0.009)$ |  $0.399\ $$(0.014)$
Restriction |  $0.953\ $$(0.005)$ |  $0.914\ $$(0.005)$ |  $\underline{0.894}\ $$(0.005)$ |  $\underline{0.928}\ $$(0.005)$ |  $0.845\ $$(0.011)$ |  $0.799\ $$(0.008)$
RPP |  $0.931\ $$(0.005)$ |  $0.891\ $$(0.003)$ |  $0.861\ $$(0.013)$ |  $0.891\ $$(0.004)$ |  $0.824\ $$(0.009)$ |  $0.772\ $$(0.019)$
Ours |  $\underline{0.958}\ $$(0.003)$ |  $\underline{0.919}\ $$(0.006)$ |  $\underline{0.894}\ $$(0.004)$ |  $0.927\ $$(0.004)$ |  $\underline{0.859}\ $$(0.011)$ |  $\underline{0.819}\ $$(0.010)$

Table 1: Test accuracies on various DDMNIST data symmetries. The first column
indicates the equivariance group, where CNN is a regular CNN. The second
column indicates the method used to break the equivariance. For each symmetry,
the highest accuracy for each network group is underlined, with the highest
overall accuracy in bold. Standard deviations over $5$ runs are in
parentheses.

We explore the capabilities of our approach in terms of overall performance,
generalisability, data-efficiency and interpretability on 2D and 3D
classification tasks using various groups $H\leq O(n)$ (See Apx. F.2). In our
2D experiments we use DDMNIST, our adapted double-digit version of MNIST
(LeCun et al., 1998). Here, the individual digits in each sample can be
randomly and independently transformed during training according to a chosen
subgroup; see Apx. F.1.2. For our 3D experiments we use the Organ, Synapse and
Nodule subsets from the biomedical collection MedMNIST (Yang et al., 2023), as
these datasets vary in their symmetries. In these experiments, we use 7-layer
CNNs, SCNNs, SCNNs with a group restriction to no equivariance in the last two
layers, RPPs and our Partial SCNNs (PSCNNs). Each (partially) equivariant
model outputs trivial irreps in the final convolution layer to obtain
structural invariance. See Apx. F.2 for more details.

To provide a broader comparison between our approach and alternative
approaches, we perform experiments on the JetFlow and Smoke 2D simulation
datasets with partial rotational symmetries as employed by Wang et al. (2023),
where they compare their approach to a collection of partial, fully and non-
equivariant approaches. For our experiments we simply port their best RSteer
configuration as reported by their repository, to our approach without
additional fine-tuning. See (Wang et al., 2023) and Apx. F.1.4 & F.2 for more
details on the setup and alternative approaches.

All our models are trained with the following objective:

$\mathcal{L}_{\text{total}}=\mathcal{L}_{\text{task}}+\alpha_{\text{align}}\mathcal{L}_{\text{align}}+\alpha_{\text{KL}}\mathcal{L}_{\text{KL}}.$
(21)

$\mathcal{L}_{\text{KL}}$ and $\mathcal{L}_{\operatorname{align}}$ are the
mean average KL-divergence between consecutive layer pairs, and alignment loss
for each layer, with tunable weights $\alpha_{\text{KL}}$ and
$\alpha_{\text{align}}$. Apx. G provides additional experiments and ablation
studies showing that these two terms are necessary for interpretable
likelihoods.

###  5.1 Benchmarks

##### DDMNIST

Tab. 1 contains the classification accuracies for DDMNIST. For all symmetries
and choices of $H$, non-equivariant CNNs outperform fully $H$-invariant SCNNs.
The performance gap is relatively small for $C_{1}$ symmetries, but
significantly larger for other symmetries. However, employing any of the
partially equivariant methods results in improved performance compared to the
regular CNN for all symmetries except $C_{1}$. The confusion matrices in Fig.
2(a) show that the invariant $O(2)$ SCNN is unable to distinguish between
digit reversals (e.g. $37$ and $73$). It seems that our $O(2)$ PSCNN manages
to eliminate these confusions by breaking equivariance. Furthermore, our
approach, particularly the $SO(2)$ variant, outperforms the restricted SCNN
and RPP methods across each data symmetry in most cases. Finally, taking a
group that is too large –e.g. taking $H=O(2)$ rather than $SO(2)$ for $D_{1}$
data symmetries– results in a higher drop in performance for the fully
equivariant SCNN than for the partially equivariant approaches.

(a) CNN

(b) O(2) SCNN

(c) Our O(2) PSCNN

Figure 2: Confusion matrices for DDMNIST with $O(2)$ symmetries. Labelled 0-99
from top to bottom and left to right.

| Network
---
Group
| Partial
---
Equivariance
Nodule | Synapse | Organ
CNN | N/A |  $\bm{\underline{0.873}}\ $$(0.005)$ |  $\underline{0.716}\ $$(0.008)$ |  $\underline{0.920}\ $$(0.003)$
$SO(3)$ | None |  $\bm{\underline{0.873}}\ $$(0.002)$ |  $0.738\ $$(0.009)$ |  $0.607\ $$(0.006)$
RPP |  $0.801\ $$(0.003)$ |  $0.695\ $$(0.037)$ |  $\underline{0.936}\ $$(0.002)$
Ours |  $0.871\ $$(0.001)$ |  $\bm{\underline{0.770}}\ $$(0.030)$ |  $0.902\ $$(0.006)$
$O(3)$ | None |  $0.868\ $$(0.009)$ |  $0.743\ $$(0.004)$ |  $0.592\ $$(0.008)$
RPP |  $0.810\ $$(0.013)$ |  $0.722\ $$(0.023)$ |  $\bm{\underline{0.940}}\ $$(0.006)$
Ours |  $\bm{\underline{0.873}}\ $$(0.008)$ |  $\underline{0.769}\ $$(0.005)$ |  $0.905\ $$(0.004)$

Table 2: Test accuracies on MedMNIST datasets.

##### MedMNIST

Tab. 2 contains the classification accuracies for MedMNIST. On Nodule, the
performance differences are negligible between CNNs, SCNNs and our PSCNNs,
with RPPs lagging behind. On Synapse the performance differences are larger,
with PSCNNs significantly outperforming all models, while regular SCNNs
outperform both CNNs and RPPs. Finally, on Organ the invariant SCNN is
significantly outperformed by all other models, with RPPs achieving the
highest performance. In Apx. G.2 we show that the poor SCNN performance is
largely due to confusions between left and right versions of some organs.
Overall, these results suggest that Nodule is highly symmetric, Synapse is
fairly symmetric –while still containing relevant non-invariant features– and
Organ is highly non-symmetric.

##### Smoke and JetFlow

Tab. 3 presents parameter counts and test RMSE scores for JetFlow and Smoke
using two methodologies; Future denotes testing on the same simulation as the
trainset, but in different time-steps in the simulation while domain
corresponds with testing at the same time-step but in different simulations.
All equivariant models use $C4$-equivariance, requiring no band-limiting for
our approach (See Apx. G.5.3 for band-limiting experiments). Our approach uses
fewer parameters than MLP and the partially equivariant RSteer, Lift, and RPP
approaches, with more parameters compared to CNN, Combo, RGroup and e2cnn.
Furthermore, our approach outperforms all other approaches on the Smoke
dataset, particularly using the domain testset. On JetFlow our approach
performs on-par with RSteer, while outperforming Lift and e2cnn.

| Smoke | JetFlow
---|---|---
Model | Future | Domain | Params (M) | Future | Domain | Params (k)
MLP | 1.38 $(0.06)$ | 1.34 $(0.03)$ | 8.33* | - | - | 510*
CNN | 1.21 $(0.01)$ | 1.10 $(0.05)$ | 0.25* | - | - | 10*
e2cnn | 1.05 $(0.06)$ | 0.76 $(0.02)$ | 0.62* | 0.21 $(0.02)$ | 0.27 $(0.03)$ | 21*
RPP | 0.96 $(0.10)$ | 0.82 $(0.01)$ | 4.36* | 0.16* $(0.01)$ | 0.19* $(0.01)$ | 145*
Combo | 1.07 $(0.00)$ | 0.82 $(0.02)$ | 0.53* | - | - | 19*
CLCNN | 0.96 $(0.05)$ | 0.84 $(0.10)$ | - | - | - | -
Lift | 0.82 $(0.01)$ | 0.73 $(0.02)$ | 3.32* | 0.18 $(0.02)$ | 0.21 $(0.04)$ | 479*
RGroup | 0.82 $(0.01)$ | 0.73 $(0.02)$ | 1.88* | - | - | 63*
RSteer | 0.80 $(0.00)$ | 0.67 $(0.01)$ | 5.60* | 0.17 $(0.01)$ |  0.16 $(0.01)$ | 185*
Ours |  0.77 $(0.01)$ |  0.57 $(0.00)$ | 3.12 |  0.15 $(0.00)$ | 0.17 $(0.01)$ | 105

Table 3: RMSE future and domain test scores on Smoke and JetFlow datasets.
Adapted from Tab. 1 and 2 from (Wang et al., 2022b). * Indicates our
replication attempt, see Apx. F.2.

###  5.2 Inspecting the Learnt Likelihood Distributions

Following sections are best viewed in colour. To differentiate between
elements in $O(2)$ we use colour coding: blue signifies a basic rotation, and
red a reflection and rotation.

(a) $O(2)$ symmetries

(b) $C_{1}$ symmetries (i.e. no symmetries)

Figure 3: Learnt likelihood $\lambda$ and error difference for layers 4 and 5
of our $O(2)$ PSCNN trained on DDMNIST with $O(2)$ and $C_{1}$ symmetries.
Dotted line marks the $O(2)$ reflection domain transition. Note the scale of
the error between the plots.

Fig. 3 shows the resulting likelihoods and corresponding equivariance
error101010The error is computed as the difference between the error vector
norms between our model and a fully invariant $O(2)$-SCNN. for layers 4 and 5
of our $O(2)$ PSCNN trained on DDMNIST with $O(2)$ and $C_{1}$ symmetries.
Firstly, we observe that a lower likelihood results in a higher equivariance
error. Next, in layer 4 equivariance is maintained for $\pi$ for $O(2)$
symmetries, but not $C_{1}$ symmetries. Most notably, on $O(2)$ symmetries a
high likelihood appears at $\pi$ in layer 5. As this is one of the latest
layers in the model, this corresponds to a global transformation. Indeed, a
global rotation and reflection effectively swap the digits twice, so this is
an example of correct global equivariance.

Figure 4: Likelihoods and errors of the fifth $O(2)$ PSCNN layer trained on
$SO(2)$ DDMNIST under various bandlimits $L$.

###  5.3 Investigating the Effect of Bandlimiting

Tab. 4 shows the classification accuracies on DDMNIST of our $O(2)$ PSCNN with
various levels of bandlimitting compared to an invariant SCNN. Here, we
observe that while all PSCNN models outperform the SCNN models, higher values
for $L$ achieve higher results. However, there seems to be diminishing returns
on this dataset for $L>2$. Notably, a bandlimiting of $L=0$ already yields a
significant performance improvement compared to the SCNN on $SO(2)$
symmetries, as this allows the model to reduce equivariance for reflections
that do not appear in $SO(2)$. Fig. 4 shows the likelihoods for the fifth
layer of our $O(2)$ PSCNN trained on $SO(2)$ symmetries using
$L\in\\{0,2,3\\}$. Apx. G.5.3 contains more band-limiting results, including
the effects on the parameter space, showing that band-limiting is not
necessarily required to obtain a reasonable number of parameters.

| Bandlimit $L$ |
---|---|---
$G$ | SCNN | 0 | 1 | 2 | 3 |
$SO(2)$ |  $0.391\ $$(0.012)$ |  $0.469\ $$(0.010)$ |  $\underline{0.894}\ $$(0.011)$ |  $\underline{0.894}\ $$(0.004)$ |  $0.889\ $$(0.013)$ |
$O(2)$ |  $0.399\ $$(0.014)$ |  $0.402\ $$(0.003)$ |  $0.780\ $$(0.009)$ |  $\underline{0.819}\ $$(0.010)$ |  $0.817\ $$(0.007)$ |

Table 4: $G$-DDMNIST test accuracies using our $O(2)$ PSCNN with various
bandlimits $L$ and SCNN without bandlimiting.

###  5.4 Data Ablation Study

One of the advantages of equivariant neural networks is their increased data-
efficiency. To investigate the data-efficiency of the various approaches, we
perform a data ablation study using the non-symmetric Organ and highly
symmetric Nodule datasets. See Fig. 5 for the results. On the symmetric Nodule
subset our approach consistently achieves the highest performance,
particularly on smaller train sets. Conversely, on the non-symmetric Organ
dataset both RPP and CNN outperform our PSCNN, especially on smaller train
sets. This is likely due to the fact that RPPs and CNNs, unlike our approach,
do not initialise as equivariant due to the regular convolutions. When little
data is available our approach therefore struggles to break equivariance. On
highly symmetric datasets, the reverse applies.

(a) Nodule

(b) Organ

Figure 5: Data ablation study on Organ and Nodule.

|  | Test Digit Symmetries |
---|---|---|---
| Network
---
Group
| Partial
---
Equivariance
$\bm{D_{4}}$ | $\bm{O(2)}$ | $\bm{\Delta}$
CNN | N/A |  $\underline{0.711}\ $$(0.009)$ |  $\underline{0.474}\ $$(0.037)$ | -0.237
$SO(2)$ | Restriction |  $0.842\ $$(0.015)$ |  $0.624\ $$(0.012)$ | -0.218
RPP |  $0.821\ $$(0.022)$ |  $0.569\ $$(0.030)$ | -0.252
Ours |  $\bm{\underline{0.863}}\ $$(0.009)$ |  $\bm{\underline{0.637}}\ $$(0.012)$ | -0.226
$O(2)$ | Restriction |  $0.848\ $$(0.007)$ |  $\underline{0.632}\ $$(0.007)$ | $\bm{\underline{-0.216}}$
RPP |  $0.824\ $$(0.009)$ |  $0.566\ $$(0.029)$ | -0.258
Ours |  $\underline{0.857}\ $$(0.011)$ |  $0.615\ $$(0.023)$ | -0.242

Table 5: Generalisability results for models trained on discrete $D_{4}$
symmetries. All models are evaluated on $D_{4}$ and $O(2)$ symmetries.
$\Delta$ shows the difference in performance between the two cases.

###  5.5 Generalisability

Another major appeal of equivariant neural networks is their improvement in
terms of generalisability to data outside the training data. To investigate
the influence of our approach on the generalisability we train the models on
DDMNIST with discrete $D_{4}$ symmetries and compare their $D_{4}$ and $O(2)$
performance (Tab. 5). Our models consistently outperform RPPs, both in terms
of performance and performance regression. However, restricted SCNN shows a
smaller performance regression, likely due to the fixed equivariance.

##  6 Limitations and Future Work

Although our approach offers an interpretable solution to learn the degree of
equivariance in SCNNs, it does have a few limitations. Most notable, our
approach is limited in breaking equivariances for compact groups, and thus
cannot model partial translation equivariance. Furthermore, compared to
regular SCNNs, our approach uses significantly more parameters, even with
band-limiting. Moreover, the steerable basis needs to be recomputed after each
update, unlike regular SCNNs applied fixed grids, increasing computational
cost. Besides, our approach requires an alignment and, optional, KL-divergence
term, resulting in additional hyper-parameter tuning. Nonetheless, we have
observed our approach to be fairly robust to the choice of these terms.

While we include a comparison of parameter space, due to the current sub-
optimal implementation we did not perform a more detailed computational
analysis. Similarly, we do not provide a theoretical proof that the learnt
equivariance corresponds to the true equivariance. Both would be interesting
directions for future research. Furthermore, we have only considered reducing
the degree of equivariance during training by initialising a uniform
likelihood, however, it would be interesting to initialise a non-uniform
likelihood distribution as a prior. Finally, we only consider band-limiting to
filter high frequencies, but it could also be employed to ensure a lower-limit
to the learnable equivariance.

##  7 Conclusion

This work presents a solution to generate partially $H$-steerable kernels from
a likelihood distribution over $H$ and proposes an algorithm to parameterise
this likelihood distribution in terms of its Fourier coefficient within the
flexible framework of SCNNs. Furthermore, through the employment of
normalisation, bandlimiting, KL-divergence, and our alignment loss, we ensure
an interpretable likelihood distribution with user-tunable flexibility
describing the degree of equivariance. Our experiments show that such a
learnable degree of equivariance can result in performance improvements
compared to fully equivariant approaches. Finally, in addition to theoretical
benefits, our approach shows competitive performance compared to alternative
partially equivariant approaches as well as restricted SCNNs in task-specific
performance, data-efficiency and generalisation.

## Acknowledgements

We express our gratitude towards the anonymous reviewers and Dr. Erik Bekkers
for their valuable feedback and discussions. We also want to thank SURF for
the use of the National Supercomputer Snellius. Finally, the presentation of
this paper at the conference was financially supported by the Amsterdam ELLIS
Unit and Qualcomm.

## Impact Statement

This paper presents work whose goal is to advance the field of Machine
Learning. There are many potential societal consequences of our work, none
which we feel must be specifically highlighted here.

## References

  * Armato III et al. (2011) Armato III, S. G., McLennan, G., Bidaut, L., McNitt-Gray, M. F., Meyer, C. R., Reeves, A. P., Zhao, B., Aberle, D. R., Henschke, C. I., Hoffman, E. A., et al.  The lung image database consortium (lidc) and image database resource initiative (idri): a completed reference database of lung nodules on ct scans.  _Medical physics_ , 38(2):915–931, 2011.
  * Bekkers (2019) Bekkers, E. J.  B-spline CNNs on lie groups.  _arXiv preprint arXiv:1909.12057_ , 2019.
  * Bekkers et al. (2023) Bekkers, E. J., Vadgama, S., Hesselink, R. D., van der Linden, P. A., and Romero, D. W.  Fast, expressive SE $(n)$ equivariant networks through weight-sharing in position-orientation space.  _arXiv preprint arXiv:2310.02970_ , 2023.
  * Bilic et al. (2023) Bilic, P., Christ, P., Li, H. B., Vorontsov, E., Ben-Cohen, A., Kaissis, G., Szeskin, A., Jacobs, C., Mamani, G. E. H., Chartrand, G., et al.  The liver tumor segmentation benchmark (lits).  _Medical Image Analysis_ , 84:102680, 2023.
  * Bronstein et al. (2021) Bronstein, M. M., Bruna, J., Cohen, T., and Veličković, P.  Geometric deep learning: Grids, groups, graphs, geodesics, and gauges.  _arXiv preprint arXiv:2104.13478_ , 2021.
  * Callister et al. (2015) Callister, M., Baldwin, D., Akram, A., Barnard, S., Cane, P., Draffan, J., Franks, K., Gleeson, F., Graham, R., Malhotra, P., et al.  British thoracic society guidelines for the investigation and management of pulmonary nodules: accredited by nice.  _Thorax_ , 70(Suppl 2):ii1–ii54, 2015.
  * Cesa et al. (2022) Cesa, G., Lang, L., and Weiler, M.  A program to build E(n)-equivariant steerable CNNs.  In _International Conference on Learning Representations_ , 2022.
  * Cohen & Welling (2016a) Cohen, T. and Welling, M.  Group equivariant convolutional networks.  In _International conference on machine learning_ , pp. 2990–2999. PMLR, 2016a.
  * Cohen & Welling (2016b) Cohen, T. S. and Welling, M.  Steerable CNNs.  _arXiv preprint arXiv:1612.08498_ , 2016b.
  * Dehmamy et al. (2021) Dehmamy, N., Walters, R., Liu, Y., Wang, D., and Yu, R.  Automatic symmetry discovery with lie algebra convolutional network.  _Advances in Neural Information Processing Systems_ , 34:2503–2515, 2021.
  * Finzi et al. (2020) Finzi, M., Stanton, S., Izmailov, P., and Wilson, A. G.  Generalizing convolutional neural networks for equivariance to lie groups on arbitrary continuous data.  In _International Conference on Machine Learning_ , pp. 3165–3176. PMLR, 2020.
  * Finzi et al. (2021) Finzi, M., Benton, G., and Wilson, A. G.  Residual pathway priors for soft equivariance constraints.  _Advances in Neural Information Processing Systems_ , 34:30037–30049, 2021.
  * Franzen & Wand (2021) Franzen, D. and Wand, M.  General nonlinearities in SO(2)-equivariant cnns.  In Ranzato, M., Beygelzimer, A., Dauphin, Y., Liang, P., and Vaughan, J. W. (eds.), _Advances in Neural Information Processing Systems_ , volume 34, pp. 9086–9098. Curran Associates, Inc., 2021.
  * Gould et al. (2013) Gould, M. K., Donington, J., Lynch, W. R., Mazzone, P. J., Midthun, D. E., Naidich, D. P., and Wiener, R. S.  Evaluation of individuals with pulmonary nodules: When is it lung cancer?: Diagnosis and management of lung cancer: American college of chest physicians evidence-based clinical practice guidelines.  _Chest_ , 143(5):e93S–e120S, 2013.
  * Gray (1959) Gray, E. G.  Axo-somatic and axo-dendritic synapses of the cerebral cortex: an electron microscope study.  _Journal of anatomy_ , 93(Pt 4):420, 1959.
  * Holl et al. (2020) Holl, P., Koltun, V., Um, K., and Thuerey, N.  phiflow: A differentiable pde solving framework for deep learning via physical simulations.  In _NeurIPS workshop_ , volume 2, 2020.
  * Kingma & Ba (2014) Kingma, D. P. and Ba, J.  Adam: A method for stochastic optimization.  _arXiv preprint arXiv:1412.6980_ , 2014.
  * Kingma & Welling (2013) Kingma, D. P. and Welling, M.  Auto-encoding variational bayes.  _arXiv preprint arXiv:1312.6114_ , 2013.
  * Klemann & Roubos (2011) Klemann, C. J. and Roubos, E. W.  The gray area between synapse structure and function—gray’s synapse types i and ii revisited.  _Synapse_ , 65(11):1222–1230, 2011.
  * Kondor & Trivedi (2018) Kondor, R. and Trivedi, S.  On the generalization of equivariance and convolution in neural networks to the action of compact groups.  In _International Conference on Machine Learning_ , pp. 2747–2755. PMLR, 2018.
  * Kuipers & Bekkers (2023) Kuipers, T. P. and Bekkers, E. J.  Regular SE(3) group convolutions for volumetric medical image analysis.  _arXiv preprint arXiv:2306.13960_ , 2023.
  * Kullback & Leibler (1951) Kullback, S. and Leibler, R. A.  On information and sufficiency.  _The annals of mathematical statistics_ , 22(1):79–86, 1951.
  * Lang & Weiler (2020) Lang, L. and Weiler, M.  A wigner-eckart theorem for group equivariant convolution kernels.  _arXiv preprint arXiv:2010.10952_ , 2020.
  * Larici et al. (2017) Larici, A. R., Farchione, A., Franchi, P., Ciliberto, M., Cicchetti, G., Calandriello, L., Del Ciello, A., and Bonomo, L.  Lung nodules: size still matters.  _European respiratory review_ , 26(146), 2017.
  * LeCun et al. (1995) LeCun, Y., Bengio, Y., et al.  Convolutional networks for images, speech, and time series.  _The handbook of brain theory and neural networks_ , 3361(10):1995, 1995.
  * LeCun et al. (1998) LeCun, Y., Bottou, L., Bengio, Y., and Haffner, P.  Gradient-based learning applied to document recognition.  _Proceedings of the IEEE_ , 86(11):2278–2324, 1998.
  * MacMahon et al. (2005) MacMahon, H., Austin, J. H., Gamsu, G., Herold, C. J., Jett, J. R., Naidich, D. P., Patz Jr, E. F., and Swensen, S. J.  Guidelines for management of small pulmonary nodules detected on ct scans: a statement from the fleischner society.  _Radiology_ , 237(2):395–400, 2005.
  * Maile et al. (2023) Maile, K., Wilson, D. G., and Forré, P.  Equivariance-aware architectural optimization of neural networks.  In _The Eleventh International Conference on Learning Representations_ , 2023.
  * Peter & Weyl (1927) Peter, F. and Weyl, H.  Die vollständigkeit der primitiven darstellungen einer geschlossenen kontinuierlichen gruppe.  _Mathematische Annalen_ , 97(1):737–755, 1927.
  * Petrache & Trivedi (2023) Petrache, M. and Trivedi, S.  Approximation-generalization trade-offs under (approximate) group equivariance.  _arXiv preprint arXiv:2305.17592_ , 2023.
  * Poulenard & Guibas (2021) Poulenard, A. and Guibas, L. J.  A functional approach to rotation equivariant non-linearities for tensor field networks.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pp. 13174–13183, 2021.
  * Romero & Lohit (2022) Romero, D. W. and Lohit, S.  Learning partial equivariances from data.  _Advances in Neural Information Processing Systems_ , 35:36466–36478, 2022.
  * Sheng & Kim (2011) Sheng, M. and Kim, E.  The postsynaptic organization of synapses.  _Cold Spring Harbor perspectives in biology_ , 3(12):a005678, 2011.
  * Somogyi et al. (1998) Somogyi, P., Tamas, G., Lujan, R., and Buhl, E. H.  Salient features of synaptic organisation in the cerebral cortex.  _Brain research reviews_ , 26(2-3):113–135, 1998.
  * van der Ouderaa et al. (2022) van der Ouderaa, T., Romero, D. W., and van der Wilk, M.  Relaxing equivariance constraints with non-stationary continuous filters.  _Advances in Neural Information Processing Systems_ , 35:33818–33830, 2022.
  * van der Ouderaa et al. (2023) van der Ouderaa, T. F., Immer, A., and van der Wilk, M.  Learning layer-wise equivariances automatically using gradients.  _arXiv preprint arXiv:2310.06131_ , 2023.
  * Van der Pol et al. (2020) Van der Pol, E., Worrall, D., van Hoof, H., Oliehoek, F., and Welling, M.  MDP homomorphic networks: Group symmetries in reinforcement learning.  _Advances in Neural Information Processing Systems_ , 33:4199–4210, 2020.
  * Wang et al. (2022a) Wang, D., Walters, R., Zhu, X., and Platt, R.  Equivariant $q$ learning in spatial action spaces.  In _Conference on Robot Learning_ , pp. 1713–1723. PMLR, 2022a.
  * Wang et al. (2023) Wang, D., Zhu, X., Park, J. Y., Platt, R., and Walters, R.  A general theory of correct, incorrect, and extrinsic equivariance.  _arXiv preprint arXiv:2303.04745_ , 2023.
  * Wang et al. (2022b) Wang, R., Walters, R., and Yu, R.  Approximately equivariant networks for imperfectly symmetric dynamics.  In _International Conference on Machine Learning_ , pp. 23078–23091. PMLR, 2022b.
  * Weiler & Cesa (2019) Weiler, M. and Cesa, G.  General E(2)-equivariant steerable CNNs.  _Advances in neural information processing systems_ , 32, 2019.
  * Weiler et al. (2018a) Weiler, M., Geiger, M., Welling, M., Boomsma, W., and Cohen, T. S.  3D steerable CNNs: Learning rotationally equivariant features in volumetric data.  _Advances in Neural Information Processing Systems_ , 31, 2018a.
  * Weiler et al. (2018b) Weiler, M., Hamprecht, F. A., and Storath, M.  Learning steerable filters for rotation equivariant cnns.  In _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_ , pp. 849–858, 2018b.
  * Weiler et al. (2021) Weiler, M., Forré, P., Verlinde, E., and Welling, M.  Coordinate independent convolutional networks–isometry and gauge equivariant convolutions on riemannian manifolds.  _arXiv preprint arXiv:2106.06020_ , 2021.
  * Worrall et al. (2017) Worrall, D. E., Garbin, S. J., Turmukhambetov, D., and Brostow, G. J.  Harmonic networks: Deep translation and rotation equivariance.  In _Proceedings of the IEEE conference on computer vision and pattern recognition_ , pp. 5028–5037, 2017.
  * Xu et al. (2019) Xu, X., Zhou, F., Liu, B., Fu, D., and Bai, X.  Efficient multiple organ localization in ct image using 3D region proposal network.  _IEEE transactions on medical imaging_ , 38(8):1885–1898, 2019.
  * Yang et al. (2023) Yang, J., Shi, R., Wei, D., Liu, Z., Zhao, L., Ke, B., Pfister, H., and Ni, B.  Medmnist v2-a large-scale lightweight benchmark for 2D and 3D biomedical image classification.  _Scientific Data_ , 10(1):41, 2023.

###### Table of Contents

  1. 1 Introduction
    1. 1.1 Contributions and Outline
  2. 2 Background on Steerable CNNs
    1. 2.1 Steerable CNNs
    2. 2.2 Generating Steerable Bases
  3. 3 Learning Equivariance
    1. 3.1 Leveraging the Averaging Operator
    2. 3.2 Ensuring Interpretable Equivariance
    3. 3.3 Bandlimiting
    4. 3.4 Implementation
  4. 4 Related Work
  5. 5 Experiments and Evaluation
    1. 5.1 Benchmarks
    2. 5.2 Inspecting the Learnt Likelihood Distributions
    3. 5.3 Investigating the Effect of Bandlimiting
    4. 5.4 Data Ablation Study
    5. 5.5 Generalisability
  6. 6 Limitations and Future Work
  7. 7 Conclusion
  8. A Overview of Appendix
  9. B Mathematical Preliminaries
    1. B.1 Group Theory
    2. B.2 Group Representation Theory
      1. B.2.1 Tensor Product Representation and its Decomposition
    3. B.3 Equivariance and Schur’s Lemma
    4. B.4 Peter-Weyl Theorem: Fourier Transform and Steerable Basis
  10. C Additional Derivations
    1. C.1 Irreps Decomposition
    2. C.2 Solving Equivariant Kernel Constraint
    3. C.3 Alignment Problem
    4. C.4 KL-Divergence on Fourier domain
  11. D Additional Theoretical Details
    1. D.1 Comparison with Alternative Partially Equivariant Approaches
    2. D.2 Monotonically Decreasing Equivariance
  12. E Additional Implementation Details
    1. E.1 Non-linearites
    2. E.2 Sharing Equivariance
    3. E.3 Interpretability losses
  13. F Experimental Details
    1. F.1 Datasets
      1. F.1.1 Vectors
      2. F.1.2 DDMNIST
      3. F.1.3 Medical MNIST
      4. F.1.4 Smoke and JetFlow
    2. F.2 Model Architectures
    3. F.3 Equivariance Error
    4. F.4 Training details
  14. G Additional Experiments
    1. G.1 Preliminary Approach compared to Probabilistic Approach
    2. G.2 Benchmarks on Biomedical Image Classification
    3. G.3 Inspecting the Learnt Likelihood distributions
    4. G.4 Shared Equivariance compared to Individual Equivariance
    5. G.5 Investigating the Effect of Regularisation Methods
      1. G.5.1 Alignment Loss
      2. G.5.2 KL-Divergence
      3. G.5.3 Bandlimiting
    6. G.6 Competitive Results on Image Classification Tasks

Appendix

##  Appendix A Overview of Appendix

These sections cover the supplementary materials of our work. First, in
Appendix B we provide an extensive mathematical background. Next, Appendix C
covers additional and more in-depth derivations from concepts and solutions in
the paper. In Appendix E we provide additional and more in-depth
implementation details. Appendix F contains an extensive overview of the
experimental details. Finally, in Appendix G we present additional
experiments.

##  Appendix B Mathematical Preliminaries

In this section, we cover the mathematical preliminaries required for the work
in this paper. We will start with the basics of Group Theory and
Representation Theory, work our way through equivariance and invariance after
which we close with Peter Weyl Theorem and Fourier Transforms.

###  B.1 Group Theory

######  Definition B.1.

(Group) A group is a pair $(G,\cdot)$, containing a set $G$ and a binary
operation $\cdot$, also called the group law:

$\cdot\ :G\times G\to G,(h,g)\mapsto h\cdot g$

satisfying the following axioms of group theory:

  * •

Associativity: $\forall a,b,c\in G:$ $\quad a\cdot(b\cdot c)=(a\cdot b)\cdot
c$

  * •

Identity: $\forall e\in G,\forall g\in G:$ $\quad g\cdot e=e\cdot g=g$.

  * •

Inverse: $\forall g\in G,\exists g^{-1}\in G:$ $\quad g\cdot
g^{-1}=g^{-1}\cdot g=e$.

It can be proven that the inverse element $g^{-1}$ of an element $g$, and the
identity element $e$ are unique.

To reduce notation it is common to write $hg$ instead of $h\cdot g$ and to
refer to the whole group with $G$ instead of $(G,\cdot)$ as long as this is
not ambiguous. It is also common to use the power notation to abbreviate the
combination of the element $g$ with itself $n$ times:

$g^{n}=\underbrace{g\cdot g\cdot g\dotsm g}_{n\text{ times}}$

######  Example B.2.

(General Linear Group) The general linear group
$\left(GL\left(\mathbb{R}^{n}\right),\cdot\right)$ is the group with the
matrix multiplication $\cdot$ as group law and the set $GL(\mathbb{R}^{n})$ of
all invertible $n\times n$ matrices:

$GL\left(\mathbb{R}^{n}\right)=\left\\{O\in\mathbb{R}^{n\times n}\mid
O^{-1}O=\textbf{I}_{n}\right\\}$ (22)

The identity element $e$ is the identity matrix $\textbf{I}_{n}$ of size
$n\times n$. Since the matrix multiplication is associative, this definition
satisfies the axioms from Definition B.1.

Beyond the structure of a group itself, it is useful to consider the
cardinality of the set $G$ within a group, formally termed as the order of a
group.

######  Definition B.3.

(Order of a Group) The order of a group $G$ is the cardinality of its set $G$
and it is indicated by $|G|$.

Given the order, groups naturally fall into two categories: finite and
infinite groups.

######  Definition B.4.

(Finite Group) A finite group is a group with a finite number of elements, and
therefore a finite cardinality.

Therefore, an infinite group is a group $G$ with cardinality $|G|=\infty$. An
example of a finite group is the set of integers module $n$.

######  Example B.5.

($\mathbb{Z}/n\mathbb{Z}$) The set $\left\\{e=0,1,2,\dots,n-1\right\\}$ with
the associative group law sum modulo $n$:

$+:(a,b)\mapsto(a+b)\quad\mod{n}$ (23)

forms the group $\mathbb{Z}/n\mathbb{Z}$ of integers modulo $n$.

  1. 1.

Identity: The element $0$ serves as the identity element for this group. For
any integer $a$ in $\mathbb{Z}/n\mathbb{Z}$, $a+0\equiv a\mod{n}$. Thus,
adding 0 to any element does not change its value in this set.

  2. 2.

Inverse Element: Every element $a$ in $\mathbb{Z}/n\mathbb{Z}$ has an inverse
$b$ such that $a+b\equiv 0\mod{n}$. For instance, when $n=10$, the element $4$
has the inverse $6$ because $(4+6)\mod{10}\equiv 0\mod{10}$.

The group has order $|\mathbb{Z}/n\mathbb{Z}|=n\in\mathbb{N}^{+}$, and is
therefore a finite group.

Now that we have introduced the basic concepts of group theory, we move onto
more advanced concepts. These involve relationships between groups, and
transformations of groups.

######  Definition B.6.

(Group Homomorphism) Given two groups $(G,\cdot)$ and $(H,*)$, a map $f:G\to
H$, mapping an element in group $G$ to an element in group $H$, is a group
homomorphism from $G$ to $H$ if:

$\forall a,b\in G\colon\quad f(a\cdot b)=f(a)*f(b)$

A special case of a homomorphism is an isomorphism, which adds the condition
of bijectivity.

######  Definition B.7.

(Group Isomorphism) A group homomorphism $f$ from $G$ to $H$ is a group
isomorphism if it is bijective (surjective and injective), i.e. if and only
if:

$\forall h\in H,\ \exists!\ g\in G:\quad f(g)=h.$

If we have two groups $G$ and $H$ and there exists an isomorphism between the
two, then the two groups are considered to be isomorphic.

######  Definition B.8.

(Group Automorphism) A group homomorphism $f$ from $G$ to $G$ itself is called
a group endomorphism. A group endomorphism which is also bijective (i.e., an
isomorphism) is called a group automorphism.

Groups can be useful for describing space-based symmetries. In this work we
focus on describing or negating symmetries in $\mathbb{R}^{2}$ and
$\mathbb{R}^{3}$. Mathematically, these symmetries are described as group
actions acting on a specific space.

######  Definition B.9.

(Group Action and $G$-space) Given a group $G$, a (left) $G$-space $X$ is a
set equipped with a group action $G\times X\to X$, $(g,x)\to g\cdot x$, i.e. a
map satisfying the following axioms:

  * •

Identity: $\forall x\in X\colon\quad e\cdot x=x$

  * •

Compatibility: $\forall a,b\in G,\forall x\in X\colon\quad a\cdot(b\cdot
x)=(a\cdot b)\cdot x$

If these axioms hold, $G$ acts on the $G$-space $X$.

For any group $(G,\cdot)$, its group law $\cdot:G\times G\to G$ trivially
defines a group action of the group on itself.

######  Definition B.10.

(Homogeneous Space) A homogeneous space is a $G$-space with a transitive
action on $G$, meaning that for every pair of points $x,y\in X$, there exists
an element $g\in G$ such that the action of $g$ on $x$ moves $x$ to $y$.
Formally, this is expressed as:

$\forall x,y\in X,\exists g\in G\text{ s.t. }g\cdot x=y,$ (24)

where $\cdot$ denotes the group action from Def B.9.

One of the main symmetries we are interested in are rotational symmetries. The
Cyclic Group can be used to model rotations, but it is limited to a fixed
number of discrete rotations.

######  Example B.11.

(Cyclic Group) The collection of all complex $N$-th roots of unity,
$\left\\{e^{ik\frac{2\pi}{N}}|0\leq k\leq n\right\\}$, forms a group under
multiplication. This group is isomorphic to the group
$\mathbb{Z}/n\mathbb{Z}$, as depicted in Ex. B.5. A homomorphism $f$ can be
established between these two groups and can be defined as

$f:e^{ik\frac{2\pi}{n}}\mapsto k.$ (25)

This finite group, generally denoted as $C_{N}$, is referred to as the Cyclic
Group of order $|C_{N}|=N$. On a more abstract level, this group can be
described as

$\displaystyle C_{N}=\left\\{e=g^{0},g,g^{2},\dots,g^{n-1}\mid g^{m}=g^{n}\iff
m\equiv n\text{ mod }N\right\\}.$ (26)

Here, the elements of the group are indicated using power notation.
The Cyclic Group appears frequently in this work, manifesting itself as
discrete rotations when acting on the plane $\mathbb{R}^{2}$, or as a
reflection when acting on rotation groups.

In contrast to the Cyclic Group, the Special Orthogonal Group contains
continuous rotations.

######  Example B.12.

(Special Orthogonal Group) The Special Orthogonal Group is the group of all
continuous rotations. This infinite group is denoted as $SO(n)$, where $n$
denotes the dimensionality of the space on which the group acts. The most
notable examples are $SO(2)$ and $SO(3)$, denoting planar and volumetric
rotations respectively. The action of a rotation $r_{\Theta}\in SO(n)$ with
the rotation angles (or singular angle in the case of $SO(2)$) parameterized
by the set $\Theta$ on the respective space can be defined as:

$\forall x\in X=\mathbb{R}^{n}\colon\quad r_{\Theta},x\mapsto r_{\Theta}\cdot
x=\psi(\Theta)x$

Where $\psi(\Theta)$ is the rotation matrix, i.e., for rotations in $SO(2)$
with $\Theta=\\{\theta\\}$:

$\displaystyle\psi(\Theta)$
$\displaystyle=\begin{bmatrix}\cos{\theta}&-\sin{\theta}\\\
\sin{\theta}&\phantom{-}\cos{\theta}\end{bmatrix},$ (27)

and $\psi(\Theta)x$ is the regular matrix-vector product.

The $SO(n)$ groups are therefore the group of all $n\times n$ orthogonal real
matrices with positive determinant

$\displaystyle SO(n)$ $\displaystyle=\left\\{O\in\mathbb{R}^{n\times n}\mid
O^{\top}O=\mathbf{I}_{n}\text{ and }\det(O)=1\right\\}.$ (28)

In this work, we often consider groups whose elements are completely included
in another larger group. These groups are called subgroups.

######  Definition B.13.

(Subgroup) Given a group $(G,\cdot)$, a non-empty subset $H\subseteq G$ is a
subgroup of $G$ if it forms a group $(H,*)$ under the same group law,
restricted to $H$.
For $H$ to be a subgroup of $G$, it is required and sufficient that the
restricted group law and the inverse are closed in $H$:

  * •

$\forall a,b\in H\colon\quad a*b\in H$

  * •

$\forall h\in H\colon\quad h^{-1}\in H$

In this case, it is common to write $H\leq G$.

######  Definition B.14.

(Coset) Consider the group $G$ and subgroup $H\leq G$. A left coset of $H$ in
$G$ is $gH=\\{gh\mid h\in H\\}$ for an element $g\in G$. Likewise, a right
coset of $H$ in $G$ is $Hg=\\{hg\mid h\in H\\}$ for an element $g\in G$.

Therefore, a coset of an element $g$ is the set of all $g^{\prime}\in G$ which
are obtainable through the respective left or right action of an element $h\in
H<G$ on the element $g$.

######  Example B.15.

(Left Coset of $SO(2)$ in $O(2)$) Let us consider the left coset of $SO(2)$ in
$O(2)$.

Given $SO(2)$ as our subgroup, consider a reflection $g\in O(2)$ which acts as
a pure reflection along the x-axis. The matrix representation of this
reflection is:

$g=\begin{bmatrix}-1&0\\\ 0&1\end{bmatrix}$

The left coset generated by this reflection with respect to $SO(2)$ will be:

$gSO(2)=\left\\{g\cdot r_{\Theta}\mid r_{\Theta}\in SO(2)\right\\}$

Given our representation of rotations in $SO(2)$ from Ex. B.12, for any
rotation $\psi(\Theta)\in SO(2)$, the result of the matrix multiplication
$g\cdot\psi(\Theta)$ will be a reflection followed by a rotation by $\theta$.
Since $\text{det}(g)=-1$ and $\text{det}(\psi(\Theta))=1$, the resulting coset
is as follows:

$\displaystyle gH=gSO(2)$ $\displaystyle=\left\\{O\in\mathbb{R}^{2\times
2}\mid O^{\top}O=\mathbf{I}_{n}\text{ and }\det(O)=-1\right\\}.$ (29)

######  Definition B.16.

(Normal Subgroup) Given a group $G$ with a subgroup $H\leq G$, $H$ is a normal
subgroup of $G$ iff:

$\forall g\in G\colon\quad gH=Hg$ (30)

If this holds, the fact that $H$ is a normal subgroup of $G$ can be denoted as
$H\unlhd G$.

To illustrate the concept of a subgroup, let us consider the cyclic group
$C_{N}$ and its relation to the group of all continuous rotations in a plane,
$SO(2)$.

######  Example B.17.

($C_{N}$ as a Subgroup of $SO(2)$) In Examples B.12 and B.11 we defined the
group of all continuous rotations in a plane, $SO(2)$, and the cyclic group
$C_{N}$ respectively. The cyclic group $C_{N}$, being a finite group of
rotations, is actually a subgroup of the group $SO(2)$ of infinite rotations.
It represents the group of $N$ rotations by integer multiples of an angle
equal to $\frac{2\pi}{N}$.
From Ex. B.11, we know that $C_{N}$ is isomorphic to the group of $N$-th roots
of unity under multiplication, which can be viewed as rotations in the complex
plane, denoted by $\mathbb{C}$. We can relate the elements of $C_{N}$ to
$SO(2)$ through the following mapping function:

$C_{N}\mapsto SO(2),\quad g^{k}\mapsto r_{k\frac{2\pi}{N}}.$ (31)

For example, if we take the cyclic group $C_{4}$, together with the element
$g^{3}\in C_{4}$, then this element corresponds to
$r_{3\frac{2\pi}{4}}=r_{\frac{3\pi}{2}}$, or a rotation of $270$ degrees.

Occasionally we want to describe certain groups as combinations of two or more
smaller groups. There are two types of group products allowing for such
decompositions; the direct product and semi-direct product. First, lets
consider the direct product.

######  Definition B.18.

(Direct Product) Given two groups $(K,*)$ and $(H,+)$, the direct product
group $(K\times H,\cdot)$ is defined as the Cartesian product $K\times H$ of
the two sets, along with the group law:

$(k_{1},h_{1})\cdot(k_{2},h_{2})=(k_{1}*k_{2},h_{1}+h_{2}).$ (32)

$K\times H$ is often used as the shorthand notation for the direct product
between $H$ and $K$.

Given a direct product set $K\times H$, we find that the subsets
$\\{(e_{K},h)|h\in H\\}$ and $\\{(k,e_{H})|k\in K\\}$ form distinct normal
subgroups. Furthermore, these subsets are isomorphic to $H$ and $K$,
respectively. This structure enables us to uniquely decompose any element
$(k,h)$ of the set $K\times H$ as a product of an element from $K$ and an
element from $H$. For instance, we can represent $(k,h)$ as either
$(e_{K},h)\cdot(k,e_{H})$ or $(k,e_{H})\cdot(e_{K},h)$. A notable
characteristic of these sets is that all elements of $K$ commute with the
elements of $H$. We can observe the structure of a direct-product in the
decomposition of groups of translations.

######  Example B.19.

(Decomposition of Translations) The group of translations on the plane,
denoted as $(\mathbb{R}^{2},+)$, can be decomposed into its constituent
vertical and horizontal translations. As a result, we can say that the group
$(\mathbb{R}^{2},+)$ is isomorphic to the direct product
$(\mathbb{R},+)\times(\mathbb{R},+)$, which represents two instances of the
group $(\mathbb{R},+)$, signifying translations along a line. Following the
same line of thought, we can use the direct product between more than two
groups. For example, for translations in three-dimensional space, represented
by the group $(\mathbb{R}^{3},+)$. This group is isomorphic to the direct
product $(\mathbb{R},+)\times(\mathbb{R},+)\times(\mathbb{R},+)$, which can be
viewed as three instances of the group $(\mathbb{R},+)$.

While the direct product decomposes a group in the product of two normal
subgroups whose elements commute with each other, the semi-direct product only
requires one of the subgroups to be normal subgroups.

######  Definition B.20.

(Semi-Direct Product) Considering two groups $(N,*)$ and $(H,+)$ and a group
action $\phi:H\times N\to N$ of $H$ on $N$, the semi-direct product group
$N\rtimes_{\phi}H$ is defined as the Cartesian product $N\times H$ with the
following binary operation:

$(n_{1},h_{1})\cdot(n_{2},h_{2})=(n_{1}*\phi(h_{1},n_{2}),h_{1}+h_{2})$ (33)

It is important to note here that the resulting group depends on the map
$\phi$ and that changing the map leads to a change in the group.
Like in a direct product, any element of a semi-direct product can be uniquely
identified by a pair of elements on the two subgroups.

Here, only the group $N$ is required to be a normal subgroup of $G$.
Additionally, when $\phi$ is the identity map on $N$ for any $h\in H$, we
obtain the more specific definition for the previously defined direct product.
The semi-direct product plays a role in the decomposition of the special
euclidean- and orthogonal groups.

######  Example B.21.

(Orthogonal Group $O(n)$) In Ex. B.12 we introduced the Special Euclidean
Group $SO(n)$, the group of continuous rotations in $n$-dimensional space.
Another common group in this work is the Orthogonal Group $O(n)$. In addition
to rotations, this group also contains reflections in the $n$-dimensional
space. Like in $SO(n)$, the action of a rotation $r_{\Theta}\text{ in }O(n)$
is defined as:

$\forall x\in X=\mathbb{R}^{n}\colon\quad r_{\Theta},x\mapsto r_{\Theta}\cdot
x=\psi(\Theta)x$ (34)

Where $\psi(\Theta)$ is the rotation matrix.

A potential reflection may or may not reflect the point $x$ around the first
axis. This operation can be represented by inverting the first coordinate of a
point, depending on the value of a parameter $p$ that dictates whether or not
a reflection occurs. For $p=-1$, a reflection occurs, while for $p=1$, the
original point is preserved. Thus we have that for a given a potential
reflection operator $f_{p}$ where $p\in\\{-1,1\\}$, the mapping is defined as
follows:

$\forall x\in X=\mathbb{R}^{n}\colon\quad f_{p},x\mapsto f_{p}\cdot
x=\begin{bmatrix}p&0\\\ 0&\textbf{I}_{n-1}\end{bmatrix}x$ (35)

Here, $p=-1$ results in a reflection, whereas $p=1$ leaves the point unchanged
and $\textbf{I}_{n-1}$ is the $(n-1)\times(n-1)$ identity matrix. Therefore,
this group is the group of all $n\times n$ orthogonal real matrices:

$O(n)=\left\\{O\in\mathbb{R}^{n\times n}\mid
O^{\top}O=\textbf{I}_{n}\right\\}$ (36)

With $\forall O\in O(n)\quad\det(O)=\pm 1$.

Let us consider the planar case $n=2$, hence $G=O(2)$. $O(2)$ can be
decomposed as a semi-direct product of $SO(2)$ and the cyclic group $C_{2}$.
Any element of $O(2)$ can be identified by a pair $(r_{\Theta},f_{p})$, where
$r_{\Theta}\in SO(2)$ and $f_{p}\in C_{2}$. The resulting product between
these two elements is:

$\displaystyle(r_{\Theta_{1}},f_{p_{1}})\cdot(r_{\Theta_{2}},f_{p_{2}})$
$\displaystyle=r_{\Theta_{1}}f_{p_{1}}r_{\Theta_{2}}f_{p_{2}}$ (37)
$\displaystyle=r_{\Theta_{1}}r_{\phi(f_{p_{1}})\Theta_{2}}f_{p_{1}}f_{p_{2}}$
(38)
$\displaystyle=(r_{\Theta_{1}}r_{\phi(f_{p_{1}})\Theta_{2}},f_{p_{1}}f_{p_{2}})$
(39)
$\displaystyle=(r_{\Theta_{1}}r_{\phi(f_{p_{1}})\Theta_{2}},f_{p_{1}}f_{p_{2}})$
(40)

The action $\phi$ can be identified as:

$\phi:C_{2}\times SO(2)\to(r_{\Theta_{1}},f_{p_{2}}),SO(2)\mapsto
r_{\phi(f_{p_{2}})\Theta_{1}}$ (41)

Hence, $O(n)$ can be decomposed as a semi-direct product:

$O(2)=C_{2}\rtimes_{\phi}SO(2)$ (42)

Here, the action of the $C_{2}$ group can be interpreted as a rotation of $0$
or $\pi$ around the $x$-axis in the coordinate system, where the
$2$-dimensional space is embedded in a $3$-dimensional space.

######  Example B.22.

(Special Euclidean Group $SE(n)$) The Special Euclidean Group, denoted as
$SE(n)$, is the group of all translations and rotations in the $n$ dimensional
space. Groups $SO(n)$ (Ex. B.12) and $(\mathbb{R}^{n},+)$ can be chosen as the
respective subgroup and normal subgroups of $SE(n)$. Any element of $SE(n)$
can be identified by a pair $(t_{v},r_{\Theta})$, where
$t_{v}\in\mathbb{R}^{n}$ and $r_{\Theta}\in SO(n)$. The resulting product
between these two elements is:

$\displaystyle(t_{v_{1}},r_{\Theta_{1}})\cdot(t_{v_{2}},r_{\Theta_{2}})$
$\displaystyle=t_{v_{1}}r_{\Theta_{1}}t_{v_{2}}r_{\Theta_{2}}$ (43)
$\displaystyle=t_{v_{1}}t_{\psi(\Theta_{1})v_{2}}r_{\Theta_{1}}r_{\Theta_{2}}$
(44)
$\displaystyle=(t_{v_{1}}t_{\psi(\Theta_{1})v_{2}},r_{\Theta_{1}}r_{\Theta_{2}})$
(45)
$\displaystyle=(t_{v_{1}+\psi(\Theta_{1})v_{2}},r_{\Theta_{1}}r_{\Theta_{2}})$
(46)

The action $\phi$ can be identified as:

$\phi:(\mathbb{R}^{n},+)\times
SO(n)\to(\mathbb{R}^{n},+),(t_{v_{2}},r_{\Theta_{1}})\mapsto
t_{\psi(\Theta_{1})v_{2}}$ (47)

Hence the $SE(n)$ can be decomposed as a semi-direct product:

$SE(n)=(\mathbb{R}^{n},+)\rtimes_{\phi}SO(n)$ (48)

###  B.2 Group Representation Theory

In this section, we will focus on linear group representations. These group
actions model group elements $G$ as matrices, making them suitable for acting
on $G$-spaces that are vector spaces. This is particularly useful for deep
learning, since both the data and intermediate features are generally
represented as vectors, and therefore form a vector $G$-space.

######  Definition B.23.

(Linear Group Representation) A Linear Group Representation $\rho$ of a group
$G$ on a vector space (representation space) $V$ is a group homomorphism from
$G$ to the general linear group $GL(V)$, i.e., it is a map

$\rho:G\to GL(V)\quad\text{s.t.}\quad\forall g_{1},g_{2}\in
G\colon\quad\rho(g_{1}g_{2})=\rho(g_{1})\rho(g_{2})$ (49)

In the rest of this work, the shorthand representation or group representation
will be used for linear group representation. Perhaps the simplest example of
a representation is the trivial representation.

######  Example B.24.

(Trivial Representation) The trivial representation $\rho:G\to GL(\mathbb{R})$
maps any group element to the identity. Therefore,

$\forall g\in G\colon\quad\rho(g)=1.$ (50)

The groups used in this work often contain rotations, which are modelled as
rotation matrices.

######  Example B.25.

(Rotation Matrices) As shown in Example B.12, an example of a representation
of the group $SO(2)$ are the two-dimensional rotation matrices:

$\psi:SO(2)\to
GL(\mathbb{R}^{2}),r_{\theta}\mapsto\psi(r_{\theta})=\begin{bmatrix}[r]\cos{\theta}&-\sin{\theta}\\\
\sin{\theta}&\cos{\theta}\end{bmatrix}.$ (51)

Representations that act similarly to one another are called equivalent
representations.

######  Definition B.26.

(Equivalent Representations) Two representations $\rho$ and $\rho^{\prime}$ on
a vector space $V$ are called equivalent, or isomorphic if and only if they
are related by a change of basis $Q\in GL(V)$, that is:

$\forall g\in G\colon\quad\rho^{\prime}(g)=Q\rho(g)Q^{-1}$ (52)

It is straightforward that equivalent representations behave similarly, since
their composition is independent of the basis:

$\rho^{\prime}(g_{1})\rho^{\prime}(g_{2})=Q\rho(g_{1})Q^{-1}Q\rho(g_{2})Q^{-1}=Q\rho(g_{1})\rho(g_{2})Q^{-1}$
(53)

A collection of representations can be combined into a single representation
through the direct sum.

######  Definition B.27.

(Direct Sum) Given two representations $\rho_{1}:G\to GL(V_{1})$ and
$\rho_{2}:G\to GL(V_{2})$, their direct sum $\rho_{1}\oplus\rho_{2}:G\to
GL(V_{1}\oplus V_{2})$ is defined as:

$(\rho_{1}\oplus\rho_{2})(g)=\begin{bmatrix}\rho_{1}(g)&0\\\
0&\rho_{2}(g)\end{bmatrix}.$ (54)

Here, the corresponding matrices of the two representations are combined by
simply taking the direct sum between them. Therefore, the action of the direct
sum is given by the independent actions of $\rho_{1}$ and $\rho_{2}$ on the
orthogonal subspaces $V_{1}$ and $V_{2}$, respectively, in $V_{1}\oplus
V_{2}$.

When consecutively taking the direct sum between multiple representations, we
use $\bigoplus$, e.g.:

$\bigoplus_{i}\rho_{i}(g)=\rho_{1}(g)\oplus\rho_{2}(g)\oplus\rho_{3}(g)\oplus\dots.$
(55)

Suppose $\rho:G\rightarrow GL(V)$ is a group representation and $V_{1}$ is a
subspace of $V$ such that $\forall g\in G:\rho(g)(V_{1})\subseteq V_{1}$. Then
$V_{1}$ is an invariant subspace under the action of $\rho$. We can find an
orthogonal complement $V_{2}$ of $V_{1}$ such that $V=V_{1}\oplus V_{2}$. If
we choose a basis of $V$ consistent with this decomposition, then the matrix
of $\rho(g)$ with respect to this basis has a block-diagonal form:

$\rho(g)=\begin{bmatrix}\rho_{1}(g)&0\\\ 0&\rho_{2}(g)\end{bmatrix},$ (56)

where $\rho_{1}:G\rightarrow GL(V_{1})$ and $\rho_{2}:G\rightarrow GL(V_{2})$
are the restrictions of $\rho$ to $V_{1}$ and $V_{2}$ respectively. In this
case, we say that $\rho$ is the direct sum of the representations $\rho_{1}$
and $\rho_{2}$. Representations that do not leave a subspace invariant are
called irreducible representations, which turn out to be useful building
blocks for equivariant networks.

######  Definition B.28.

(Irreducible Representation) A representation is irreducible if it does not
contain any non-trivial invariant subspaces. Irreps is often used as shorthand
for irreducible representations. We often denote irreps as $\psi$ rather than
$\rho$.

Given a group $G$, the set of irreducible representations of $G$ is often
denoted as $\hat{G}$. The trivial representation (Ex. B.24) is always in the
set of irreducible representations $\hat{G}$.

In this work, in addition to translations, we will mostly focus on the
rotation and reflection groups, both the discrete and continuous settings.
These groups are all subsets of $O(n)$. Let us provide an overview of these
subgroups in the $2$D case $H\leq O(2)$:

######  Example B.29.

(Irreducible Representations of Subgroups $H\leq O(2)$) Here we denote the
irreps of the Cyclic group $C_{N}$, Dihedral group $D_{N}$, Special Orthogonal
group $SO(2)$ and Orthogonal group $O(n)$ in the real case. For any of these
groups, we will denote the irreps as $\psi^{H}_{i}(p)$, where $H$ is the
considered subgroup, $i$ denotes some (group-specific and potentially multi-)
index, and $p$ is the parametrisation of the irrep.

##### Special Orthogonal Group $SO(2)$

The Special Orthogonal group (Ex. B.12) is the group of all continuous planar
rotations. $SO(2)$ has one $1$-dimensional representation (the trivial
representation), and infinitely many $2\times 2$ rotation matrices as irreps.

  * •

trivial representation: $\psi_{0}^{SO(2)}(r_{\theta})=1$

  * •

non-trivial irreps:
$\psi_{k}^{SO(2)}(r_{\theta})=\begin{bmatrix}\cos{(k\theta)}&-\sin{(k\theta)}\\\
\sin{(k\theta)}&\phantom{-}\cos{(k\theta)}\end{bmatrix}=\psi(k\theta),\quad
k\in\mathbb{N}^{+}$

where $\psi(k\theta)$ is the rotation matrix from Ex. B.12.

##### Orthogonal Group $O(2)$

The Orthogonal group (Ex. B.21) is the group of planar reflections and
continuous rotations with index $i=(r,k)$, where $r\in\\{0,1\\}$ denotes a
possible reflection. In addition to the trivial representation, it has another
$1$-dimensional irreducible representation that performs the reflection.

  * •

trivial representation: $\psi_{0,0}^{O(2)}(r_{\theta}p)=1$

  * •

reflection irrep: $\psi_{1,0}^{O(2)}(r_{\theta}p)=f,\quad p\in\\{-1,1\\}$

  * •

other irreps:
$\psi_{1,k}^{O(2)}(r_{\theta}p)=\begin{bmatrix}\cos{(k\theta)}&-\sin{(k\theta)}\\\
\sin{(k\theta)}&\phantom{-}\cos{(k\theta)}\end{bmatrix}\begin{bmatrix}f&0\\\
0&1\end{bmatrix}=\psi(k\theta)f_{p},\ k\in\mathbb{N}^{+},p\in\\{-1,1\\}$

Note that irreps $\psi^{O(2)}_{0,k},k>0,k\in\mathbb{N}^{+}$ do not exist.

##### Cyclic Group $C_{N}$

The Cyclic group (Ex. B.11) is the group of $N$ discrete planar rotations. Its
irreps are similar to the irreps of the special orthogonal group, only limited
to $k\leq\lfloor\frac{N}{2}\rfloor$, where $\lfloor\cdot\rfloor$ is the
flooring operation.

  * •

trivial representation: $\psi_{0}^{C_{N}}(r_{\theta}p)=1$

  * •

non-trivial irreps:
$\psi_{k}^{SO(2)}(r_{\theta})=\begin{bmatrix}\cos{(k\theta)}&-\sin{(k\theta)}\\\
\sin{(k\theta)}&\phantom{-}\cos{(k\theta)}\end{bmatrix}=\psi(k\theta),\quad
k\in\left\\{1,\dots,\lfloor\frac{N-1}{2}\rfloor\right\\}$

In the case that $N$ is even, there is an additional $1$-dimensional irrep:

  * •

$N$-even irrep: $\psi_{N/2}^{SO(2)}(r_{\theta})=\cos{(\frac{N}{2}\theta)}=\pm
1$

##### Dihedral Group $D_{N}$

The Dihehedral group is the group of planar reflections and $N$ discrete
planar rotations. Like the $O(2)$ group, it always has two $1$-dimensional
representations, with an additional two $1$-dimensional irreps if $N$ is even:

  * •

trivial representation: $\psi_{0,0}^{D_{N}}(r_{\theta}p)=1$

  * •

reflection irrep: $\psi_{1,0}^{D_{N}}(r_{\theta}p)=p,\quad p\in\\{-1,1\\}$

  * •

other irreps:
$\psi_{1,k}^{D_{N}}(r_{\theta}p)=\begin{bmatrix}\cos{(k\theta)}&-\sin{(k\theta)}\\\
\sin{(k\theta)}&\phantom{-}\cos{(k\theta)}\end{bmatrix}\begin{bmatrix}f&0\\\
0&1\end{bmatrix}=\psi(k\theta)f_{p}$,

$k\in\left\\{1,\dots,\lfloor\frac{N-1}{2}\rfloor\right\\},p\in\\{-1,1\\}$

And the $N$-even irreps:

  * •

$N$-even irrep:
$\psi_{0,N/2}^{SO(2)}(r_{\theta}p)=\cos{(\frac{N}{2}\theta)}=\pm 1$

  * •

$N$-even reflection irrep:
$\psi_{1,N/2}^{SO(2)}(r_{\theta}p)=f_{p}\psi_{0,N/2}^{SO(2)}(r_{\theta}p)=p\cos{(\frac{N}{2}\theta)}=\pm
1$

Upon closer inspection, it is notable that the irreps in the example above
correlate with some form of frequency. For each of these groups, the irrep
with index $k$ evaluated at a rotation of $\theta$ results, up to a possible
reflection, in a rotation matrix corresponding to a rotation of $k\theta$.
Therefore, we often refer to these irreps as irreps of frequency $k$. A
similar intuition applies for the irreps of subgroups $H=O(3)$ and $H=SO(3)$,
since the irreps for these groups are similar to the irreps in this example,
except that $O(3)$ also has non-reflective irreps
$\psi^{O(3)}_{0,k},k\in\mathbb{N}^{+}$. These irreps are therefore indiced by
two separate independent frequencies $r\in\\{0,1\\}$ and $k\in\mathbb{N}^{+}$
respectively. The concept of viewing these irreps as frequencies is relevant
for future sections.

It turns out that any reducible representation can be decomposed into a direct
sum of irreps.

######  Theorem B.30.

(Irreps Decomposition (Peter-Weyl theorem part 1)) Any unitary (or orthogonal)
representation $\rho:G\to V$ of a compact group $G$ over a field with
characteristic zero (e.g. the complex $\mathbb{C}^{n}$ and real
$\mathbb{R}^{n}$ fields) is a direct sum of irreducible representations. Each
irrep corresponds to an invariant subspace of the vector space $V$ with
respect to the action of $\rho$. In particular, any real linear representation
$\rho:G\to\mathbb{R}^{n}$ of a compact group $G$ can be decomposed as

$\rho(g)=Q\left[\bigoplus_{i\in\textit{I}}\psi_{i}(g)\right]Q^{-1},$ (57)

where I is an index set that specifies the irreducible representations
$\psi_{i}$ contained in $\rho$ (with possible repetitions) and $Q$ is a change
of basis. Alternatively, we can write this equation more explicitly using the
multiplicity $s$ for each irrep $\psi_{j}\in\widehat{G}$:

$\rho(g)=Q\left[\bigoplus_{j\in\widehat{G}}\bigoplus_{s}^{[j]}\psi_{j}(g)\right]Q^{-1},$
(58)

where $[j]$ denotes the multiplicity of $j$ in the decomposition.

For proofs, it is often only required to consider the irreps. A more specific
method of irreps decomposition will be introduced in Section B.2.1.

A type of representation that is particularly useful for finite groups is the
regular representation.

######  Definition B.31.

(Regular Representation) Let $G$ be a finite group, and let $\mathbb{R}^{|G|}$
be a vector space. Each basis vector $\textbf{e}_{g}$ of $\mathbb{R}^{|G|}$ is
associated with an element $g\in G$.

The regular representation of $G$, denoted as
$\rho^{G}_{\text{reg}}:G\rightarrow GL(\mathbb{R}^{|G|})$, is defined as the
map that takes an element $\widetilde{g}\in G$ and associates it with a
permutation matrix $\rho^{G}_{\text{reg}}(\widetilde{g})$ in
$GL(\mathbb{R}^{|G|})$.

The action of $\rho^{G}_{\text{reg}}(\widetilde{g})$ on a basis vector
$\textbf{e}_{g}$ is defined as:

$\rho^{G}_{\text{reg}}(\widetilde{g})\cdot\textbf{e}_{g}=\textbf{e}_{\widetilde{g}g}.$
(59)

This means that $\rho^{G}_{\text{reg}}(\widetilde{g})$ permutes the basis
vector $\textbf{e}_{g}$ to the basis vector $\textbf{e}_{\widetilde{g}g}$.

######  Example B.32.

(Regular Representation of $D_{2}$) For the Dihedral group $D_{2}$, we
represent each element of the group by a $4\times 4$ permutation matrix. The
complete regular representation of $D_{2}$ is as follows:

$g$ | $e$ | $r_{\pi}$ | $s_{x}$ | $s_{y}$
---|---|---|---|---
$\rho^{D_{2}}_{\text{reg}}$ | $\begin{bmatrix}1&0&0&0\\\ 0&1&0&0\\\ 0&0&1&0\\\ 0&0&0&1\end{bmatrix}$ | $\begin{bmatrix}0&1&0&0\\\ 1&0&0&0\\\ 0&0&0&1\\\ 0&0&1&0\end{bmatrix}$ | $\begin{bmatrix}0&0&1&0\\\ 0&0&0&1\\\ 1&0&0&0\\\ 0&1&0&0\end{bmatrix}$ | $\begin{bmatrix}1&0&0&0\\\ 0&1&0&0\\\ 0&0&0&1\\\ 0&0&1&0\end{bmatrix}$

In this representation, the $i$-th basis vector of the four-dimensional vector
space $\mathbb{R}^{4}$ is associated with the $i$-th element in the sequence
${e,\ r_{\pi},\ s,\ sr_{\pi}}$ of $D_{2}$. Specifically, the enumeration is
defined as $e\rightarrow 1,\ r_{\pi}\rightarrow 2,\ s\rightarrow 3,\