# Robust Policy Learning via Offline Skill Diffusion

Woo Kyung Kim, Minjong Yoo, Honguk Woo Honguk Woo is the corresponding author.

###### Abstract

Skill-based reinforcement learning (RL) approaches have shown considerable
promise, especially in solving long-horizon tasks via hierarchical structures.
These skills, learned task-agnostically from offline datasets, can accelerate
the policy learning process for new tasks. Yet, the application of these
skills in different domains remains restricted due to their inherent
dependency on the datasets, which poses a challenge when attempting to learn a
skill-based policy via RL for a target domain different from the datasets’
domains. In this paper, we present a novel offline skill learning framework
DuSkill which employs a guided Diffusion model to generate versatile skills
extended from the limited skills in datasets, thereby enhancing the robustness
of policy learning for tasks in different domains. Specifically, we devise a
guided diffusion-based skill decoder in conjunction with the hierarchical
encoding to disentangle the skill embedding space into two distinct
representations, one for encapsulating domain-invariant behaviors and the
other for delineating the factors that induce domain variations in the
behaviors. Our DuSkill framework enhances the diversity of skills learned
offline, thus enabling to accelerate the learning procedure of high-level
policies for different domains. Through experiments, we show that DuSkill
outperforms other skill-based imitation learning and RL algorithms for several
long-horizon tasks, demonstrating its benefits in few-shot imitation and
online RL.

##  1 Introduction

Skill-based learning demonstrates the potentials in accelerating the
adaptation to complex long-horizon tasks by leveraging pretrained skill
representations on behavior patterns from the offline datasets. However,
existing approaches in skill-based reinforcement learning (RL) (e.g., Pertsch,
Lee, and Lim 2020; Pertsch et al. 2021) and skill-based few-shot imitation
learning (e.g., Hakhamaneshi et al. 2022; Du et al. 2023) often operate under
the premise that the target domain for a downstream task was present during
skill pretraining. Thus, policies learned with the pretrained skills might
lead to sub-optimal performance, particularly when the target domain diverges
from the domains of the given datasets.

Figure 1: Concept of Offline Skill Diffusion: When a downstream task belongs
to the domain different from those of the training datasets, conventional
skill-based learning approaches struggle in learning and choosing suitable
skills. In contrast, our offline skill diffusion expands the skill diversity
that goes beyond the training datasets, enabling the execution of compatible
skills for the downstream task. The skills are discretely represented for
visual illustration.

As shown in the left side of Figure 1, where each small circle denotes a
specific skill, conventional skill learning approaches might experience low
performance in the high-level policy for a downstream task, if the task calls
for skills that differ from the pretrained ones. For instance, suppose that
robotic manipulation skills are learned from the datasets in the safety-first
domain; then, they might heavily lean towards “slow” speed manipulation. In
that case, a high-level policy learned with these skills might fail to adapt
efficiently to the downstream task that involves stringent time constraints.
These situations often arise in the environment encompassing diverse domains,
as a single task can require different skills depending on the domain it is
in. The dependency of conventional skill-based learning approaches on the
specificity of the datasets exacerbates these challenges. Moreover, it is
practically difficult to obtain comprehensive datasets that span all potential
skills for diverse domains.

To tackle these challenges in skill-based learning, we take a novel approach,
offline skill diffusion, aiming to broaden skill diversity beyond mere
imitation from the datasets. Given that diffusion models have been recognized
for their efficacy in generating human-like images with conditional values
(Rombach et al. 2022; Ho and Salimans 2022; Preechakul et al. 2022), we
leverage a guided diffusion model for the skill decoder to generate diverse
skills. The right side of Figure 1 illustrates the benefits of our framework
approach, where the red-colored dotted quadrangle represents an expanded skill
set encompassing all the skills that are required to solve the downstream task
in a different domain.

In this paper, we present the DuSkill framework, designed to generate diverse
skills for downstream tasks that can belong to the domains different from the
source domains in the training datasets. Recognizing that certain aspects of a
skill remain consistent despite domain variations, we view each skill as a
composition of domain-invariant and domain-variant features. Then, we employ a
guided diffusion-based decoder along with a hierarchical domain encoder so as
to effectively disentangle each skill into two separate embedding spaces. The
hierarchical domain encoder systematically segments skills into domain-
invariant and domain-variant embeddings by conditioning only the lower encoder
on domain variations. With two distinct embeddings, the conditional generation
process of the guided diffusion-based decoder enables distinct modulation of
each embedding, thereby facilitating the execution of a wide range of skills
in different domains.

For downstream tasks in different domains, we train a high-level policy which
produces both domain-variant and domain-invariant embeddings. The high-level
policy operates alongside the frozen guided diffusion-based decoder, which
encompasses the necessary knowledge for generating a broad range of skills,
adaptable to various domains. As such, our proposed framework stands apart
from existing skill-based learning approaches (Pertsch, Lee, and Lim 2020;
Pertsch et al. 2021), as it enables the generation of diverse skills that
extend beyond the training datasets.

The contributions of our work are summarized as follows.

  * •

We present the DuSkill framework, which facilitates robust skill-based policy
learning for downstream tasks in different domains.

  * •

We develop the offline skill diffusion model, incorporating the hierarchical
domain encoder and guided diffusion-based decoder. The model enables the
diverse skill generation that extends beyond the training datasets.

  * •

We test the framework with long-horizon tasks in various domains,
demonstrating its capability to adapt to different domains in both few-shot
imitation and online RL settings.

##  2 Preliminaries and Problem Formulation

Given the training datasets $\mathcal{D}=\\{\tau_{i}\\}_{i\leq n}$, where each
trajectory $\tau_{i}$ in the datasets is represented as a sequence of state
and action pairs $\\{(s_{t},a_{t})\\}_{t\leq H}$, the objective of skill
representation learning is to accelerate the adaptation to long-horizon tasks
by leveraging pretrained skill representations. Here, a skill is defined as a
sequence of $h$ consecutive actions $\bm{a}=\\{a_{t},...,a_{t+h}\\}$ (Pertsch,
Lee, and Lim 2020; Hakhamaneshi et al. 2022). Through the joint learning of
both a skill encoder $q(z|\bm{a})$ and a skill decoder $\epsilon(\bm{a}|z)$ on
the datasets, we are able to obtain a skill embedding $z\in\mathcal{Z}$. The
learning objective involves optimizing the evidence lower bound (ELBO), which
consists of a reconstruction term and a regularization term,

$\mathcal{L}_{\text{VAE}}=\mathbb{E}_{\bm{a}\sim\mathcal{D}}\left[-\log\epsilon(\bm{a}|z)+\beta
D_{\text{KL}}(p(z)||q(z|\bm{a})\right]$ (1)

where $D_{\text{KL}}$ is the Kullback-Leibler (KL) divergence, $p(z)$ is a
prior following a unit Gaussian distribution $\mathcal{N}(0,I)$ and $\beta$ is
a hyperparameter for regularization (Higgins et al. 2016). Then, this skill
representation is leveraged to accelerate the downstream task adaptation.

Problem Formulation. In our problem formulation, we operate under the premise
that training datasets $\mathcal{D}$ are available. As skills required for a
task might vary depending on the domain it belongs to, we further assume that
the datasets are collected from multiple source domains. The variations across
the domains are parameterized by $\omega\in\Omega$, and this parameter
information is incorporated in the datasets. Then, we aim to tackle downstream
tasks in a range of domains distinct from the source domains. Successfully
achieving this requires more than merely imitating the skills present in the
given datasets, due to the varied nature of the tasks across different
domains.

We formulate a downstream task as a goal-conditioned Markov decision process
(MDP) $\mathcal{M}$ combined with its domain $\Omega$. The goal-conditioned
MDP $\mathcal{M}$ is denoted as $(S,A,P,r,\mathcal{G},\gamma,\mu_{0})$ where
$s\in S$ is a state space, $a\in A$ is an action space, $\mathcal{G}$ is a
goal space, $P:S\times A\times\Omega\rightarrow[0,1]$ is a transition
probability, $r:S\times A\times\mathcal{G}\times\Omega\rightarrow\mathbb{R}$
is a reward function, $\gamma\in[0,1]$ is a discount factor, and
$\mu_{0}:S\rightarrow[0,1]$ is an initial state distribution. The domain
variations may affect either the reward function or the transition probability
in an MDP, while the goal space remains consistent. For instance, on the top
of Figure 1, a task may have the same goal space as slide puck in the
goalposts, but the domain related to the time constraint results in different
reward functions.

Then, our objective is to maximize the discounted cumulative sum of rewards
for downstream tasks in different domains, through a high-level policy
$\pi(z|s)$,

$\pi^{*}=\operatorname*{argmax}_{\pi}\mathbb{E}_{z\sim\pi(\cdot|s),\bm{a}\sim\epsilon(\cdot|z)}\left[\sum_{t=0}^{T-1}\gamma^{t}r\left(s_{t},a_{t})\right)\right]$
(2)

where $\epsilon(\bm{a}|z)$ is a skill decoder and $T$ is the maximum length
for an episode.

Figure 2: Offline Skill Diffusion and Downstream Policy Learning in DuSkill:
(i) In the offline skill diffusion phase, a skill is decomposed into domain-
invariant and domain-variant embeddings, and then they are combined through
the guided diffusion based decoder to generate diverse skills. (ii) In the
downstream policy learning phase, a high-level policy is learned for a task in
different domains either through few-shot imitation or online RL.

##  3 Approach

###  3.1 DuSkill Framework

Figure 3: DuSkill Framework: In (i-1), the hierarchical domain encoder
disentangles the domain-invariant and domain-variant embeddings. At the same
time, the domain-invariant and domain-variant priors are jointly learned with
these encoders. For diverse skill generation, the encoders are trained in
conjunction with the guided diffusion-based decoder in (i-2). Here, the
domain-invariant decoder and domain-variant decoder are responsible for
reconstructing actions based on the domain-invariant and domain-variant
embeddings, respectively. In (ii), the high-level policy is learned to solve
the task in different domain either through few-shot imitation or online RL.

To facilitate the diverse skill generation, we propose the DuSkill framework
consisting of two main phases: (i) the offline skill diffusion phase, and (ii)
the downstream policy learning phase, as illustrated in Figure 2.

In the offline skill diffusion phase, we employ a guided diffusion-based
decoder in conjunction with a hierarchical domain encoder to disentangle
skills into domain-invariant and domain-variant embeddings. Specifically, the
hierarchical domain encoder consists of a domain-invariant encoder and a
domain-variant encoder. The domain-invariant encoder processes a sequence of
state and action, resulting in domain-invariant embedding. Meanwhile, the
domain-variant encoder takes the domain-invariant embedding and the domain
parameterization as input to produce the domain-variant embedding. In this
way, these encoders play distinct roles, in which domain-invariant encoder is
responsible for encapsulating features necessary to reconstruct fundamental
action sequences pertinent to achieving goals, while the domain-variant
encoder is tailored to capture the features related to domain variations. To
effectively disentangle skill features via the hierarchical domain encoder, we
utilize a guided diffusion-based decoder with two components, where one is
conditioned on the domain-invariant embedding, while the other focuses on the
domain-variant embedding. This conditional generation mechanism facilitates a
distinct influence of the domain-invariant and domain-variant embeddings on
the separate segments of generated action sequences. By employing the
hierarchical domain encoder and guided diffusion-based decoder, our framework
is capable of generating diverse action sequences that encompass different
combinations of the domain-invariant and domain-variant features.

In the downstream policy learning phase, we exploit the disentangled
embeddings via a high-level policy, which produces both domain-invariant and
domain-variant embeddings. These embeddings are then fed into the frozen
guided diffusion-based decoder, which generates executable skills. In this
phase, we consider few-shot imitation and online RL adaptation scenarios,
where the high-level policy adapts to the tasks in different domains through
either fine-tuning on a limited number of trajectories or online RL
interactions.

###  3.2 Offline Skill Diffusion

Hierarchical domain encoder. To disentangle domain-invariant and domain-
variant features from skills, we introduce a hierarchical encoding approach.
This enables learning in two distinct embedding spaces: the domain-invariant
embedding space $\mathcal{Z}_{\rho}$ and the domain-variant embedding space
$\mathcal{Z}_{\sigma}$. Specifically, we employ a domain-invariant encoder
$q_{\rho}$ which maps a sequence of states and actions to the domain-invariant
embedding. We also use a domain-variant encoder $q_{\sigma}$ which maps the
domain-invariant embedding and the domain parameterization $\omega$ to the
domain-variant embedding, i.e.,

$z_{\rho}\sim q_{\rho}(\bm{s},\bm{a}),z_{\sigma}\sim
q_{\sigma}(z_{\rho},\omega)$ (3)

where $\bm{s}=\\{s_{t},...,s_{t+h}\\}$ is a sequence of states,
$\bm{a}=\\{a_{t},...,a_{t+h}\\}$ is a sequence of actions,
$z_{\rho}\in\mathcal{Z}_{\rho}$ is the domain-invariant embedding, and
$z_{\sigma}\in\mathcal{Z}_{\sigma}$ is the domain-variant embedding. To
optimize the encoders and the skill decoder $\epsilon$, we employ the evidence
lower bound (ELBO) loss using (1), similar to (Pertsch et al. 2021;
Hakhamaneshi et al. 2022), i.e.,

$\displaystyle\mathcal{L}_{\text{HVAE}}=$
$\displaystyle\mathbb{E}_{(\bm{s},\bm{a})\sim\mathcal{D}}\bigg{[}{-\prod_{t=0}^{h}\log\epsilon(a_{t}|s_{t},z_{\rho},z_{\sigma})}$
(4)
$\displaystyle+\beta_{\rho}D_{\text{KL}}(p(z_{\rho})||q_{\rho}(z_{\rho}|\bm{s},\bm{a}))$
$\displaystyle+\beta_{\sigma}D_{\text{KL}}(p(z_{\sigma}|z_{\rho})||q(z_{\sigma}|z_{\rho},\bm{s},\bm{a}))\bigg{]}$

where $q(z_{\sigma}|z_{\rho},\bm{s},\bm{a})$ is a naive modeling for the
domain-variant encoder, and $\beta_{\rho}$ and $\beta_{\sigma}$ are
regularization hyperparameters. The prior $p(z_{\sigma})$ and
$p(z_{\rho}|z_{\sigma})$ are set to be unit Gaussian. To disentangle skill
features, we modify the regularization term of the domain-variant encoder in
(4) as

$\mathcal{L}_{\text{aspect}}=\mathbb{E}_{(\bm{s},\bm{a})\sim\mathcal{D}}\left[\log\frac{q(z_{\sigma}|z_{\rho},\bm{s},\bm{a})}{q_{\sigma}(z_{\sigma}|z_{\rho},\omega)}\right].$
(5)

This loss term enables the domain-variant encoder to construct the distinct
embedding space $\mathcal{Z}_{\sigma}$, capturing domain-variant features
conditioned on the domain-invariant embedding. By combining (4) and (5), we
rewrite the loss as

$\displaystyle\mathcal{L}_{\text{DHVAE}}=$
$\displaystyle\mathbb{E}_{(\bm{s},\bm{a})\sim\mathcal{D}}\bigg{[}-\prod_{t=0}^{h}\log\epsilon(a_{t}|s_{t},z_{\rho},z_{\sigma})$
(6)
$\displaystyle+\beta_{\rho}D_{\text{KL}}(p(z_{\rho})||q_{\rho}(z_{\rho}|\bm{s},\bm{a}))$
$\displaystyle+\beta_{\sigma}D_{\text{KL}}(p(z_{\sigma}|z_{\rho})||q_{\sigma}(z_{\sigma}|z_{\rho},\omega))\bigg{]}.$

For downstream policy learning, we employ a domain-invariant prior $p_{\rho}$
and a domain-variant prior $p_{\sigma}$.

$z_{\rho}\sim p_{\rho}(s_{t}),z_{\sigma}\sim p_{\sigma}(z_{\rho})$ (7)

The domain-invariant prior $p_{\rho}$ is conditioned on the current state
$s_{t}$, facilitating the selection of suitable domain-invariant embedding,
while the domain-variant prior $p_{\sigma}$ provides a prior distribution over
the domain-variant embedding. By designing a domain-variant prior conditioned
solely on the domain-invariant embedding, our model achieves the flexibility
to explore a wide range of parameterizations across different domains. These
priors are jointly trained with the hierarchical domain encoder by minimizing
the KL divergence between each respective encoder.

$\displaystyle\mathcal{L}_{\text{prior}}=$ $\displaystyle\
\mathbb{E}_{(\bm{s},\bm{a})\sim\mathcal{D}}[D_{\text{KL}}\left(p_{\rho}(z_{\rho}|s_{t})||q_{\rho}(z_{\rho}|\bm{s},\bm{a})\right)$
(8)
$\displaystyle+D_{\text{KL}}\left(p_{\sigma}(z_{\sigma}|z_{\rho})||q_{\sigma}(z_{\sigma}|z_{\rho},\omega)\right)]$

Guided diffusion-based decoder. To generate diverse skills from domain-
invariant embedding $z_{\rho}$ and domain-variant embedding $z_{\sigma}$, we
employ the diffusion model (HO, Jain, and Abbeel 2020; Pearce et al. 2023;
Wang, Hunt, and Zhou 2023; Ajay et al. 2023). In particular, we adopt the
denoising diffusion probabilistic model (DDPM) (HO, Jain, and Abbeel 2020) to
represent our skill decoder. The decoder reconstructs an action $a_{t}$ from a
noisy input $x^{K}\sim\mathcal{N}(0,I)$ by sequentially predicting
$x^{K-1},x^{K-2},...,x^{0}(=a_{t})$, with each iteration being a marginally
denoised version of its predecessor. During the training phase, the noisy
input $x^{k}$ is generated by progressively adding the Gaussian noise to the
original action $a_{t}(=x^{0})$ over $K$ steps as

$x^{k}=\sqrt{\bar{\alpha}^{k}}a_{t}+\sqrt{1-\bar{\alpha}^{k}}\eta$ (9)

where $\eta\sim\mathcal{N}(0,I)$ and $\bar{\alpha}^{k}$ is a variance
schedule. As our skill decoder is designed to predict the noise $\eta$, the
reconstruction loss in (6) becomes an $L_{2}$ distance between the decoder’s
output and the noise.

$\mathcal{L}_{\text{rec}}=\mathbb{E}_{k\sim[1,K],\eta\sim\mathcal{N}(0,I)}\left[||\epsilon(x^{k},k,s_{t},z_{\sigma},z_{\rho})-\eta||_{2}^{2}\right]$
(10)

To further align with the objective in (6), we slightly modify the classifier-
free guidance (Ho and Salimans 2022) to divide the decoder into two separate
parts: domain-invariant decoder $\epsilon_{\rho}(x^{k},k,s_{t},z_{\rho})$ and
domain-variant decoder $\epsilon_{\sigma}(x^{k},k,s_{t},z_{\sigma})$. Thus,
our decoder is redefined as the combination of both a domain-invariant decoder
and a domain-variant decoder, i.e.,

$\displaystyle\epsilon(x^{k},k,$ $\displaystyle s_{t},z_{\rho},z_{\sigma}):=$
(11)
$\displaystyle(1-\delta)\epsilon_{\rho}(x^{k},k,s_{t},z_{\sigma})+\delta\epsilon_{\sigma}(x^{k},k,s_{t},z_{\rho})$

where $\delta>0$ serves as a guidance weight that determines the degree of
adjustment towards the domain-variant decoder. This approach allows our
domain-invariant decoder to reconstruct actions that consistently execute the
designated task across domain features. Simultaneously, the domain-variant
decoder is seamlessly incorporated by generating guidance that encapsulates
domain-variant features.

Input: Trainig Datasets $\mathcal{D}$, total denoise step $K$, guidance weight
$\delta$, hyperparameters $\beta_{\rho}$, $\beta_{\sigma}$

1: Initialize encoders $q_{\rho}$, $q_{\sigma}$, priors $p_{\rho}$,
$p_{\sigma}$, decoders $\epsilon_{\rho}$, $\epsilon_{\sigma}$

2: while not converge do

3: Sample a batch $\\{(\bm{s},\bm{a},\omega)\\}_{i}\sim\mathcal{D}$

4: Update $q_{\rho}$ and $q_{\sigma}$ using $\mathcal{L}_{\text{DHVAE}}$ in
(6)

5: Update $p_{\rho}$ and $p_{\sigma}$ using $\mathcal{L}_{\text{prior}}$ in
(8)

6: Update $\epsilon_{\rho}$ and $\epsilon_{\sigma}$ using
$\mathcal{L}_{\text{rec}}$ in (10)

7: end while

8: return
$q_{\rho},q_{\sigma},p_{\rho},p_{\sigma},\epsilon_{\rho},\epsilon_{\sigma}$

Algorithm 1 Offline Skill Diffusion

In our framework, we employ the hierarchical domain encoder to establish
distinct embedding spaces for domain-invariant and domain-variant features,
optimized by the loss defined in (6). Concurrently, the respective priors for
the downstream task are jointly trained, optimized by the loss in (8), as
depicted in Figure 3 (i-1), Furthermore, to effectively disentangle the
domain-invariant and domain-variant embeddings, we jointly train the guided
diffusion-based decoder in conjunction with the encoders, optimized by the
loss (10), as illustrated in Figure 3 (i-2). Algorithm 1 lists the learning
procedures of DuSkill.

###  3.3 Downstream Policy Learning

For efficient policy learning on downstream tasks, we adopt a hierarchical
learning scheme akin to other skill-based approaches (Pertsch, Lee, and Lim
2020; Hakhamaneshi et al. 2022). In this scheme, the higher-level policy is
employed to produce the skill embedding as output rather than directly
generating executable actions. Specifically, we train a policy
$\pi(z_{\rho},z_{\sigma}|s_{t})$, designed to align with the hierarchical
domain encoder, which generates domain-invariant and domain-variant
embeddings,

$\pi(z_{\rho},z_{\sigma}|s_{t})=\pi_{\rho}(z_{\rho}|s_{t})\cdot\pi_{\sigma}(z_{\sigma}|z_{\rho}).$
(12)

Subsequently, these embeddings are fed into the learned guided diffusion-based
decoder, which remains frozen to decode them into a sequence of executable
actions. To decode an action from the guided diffusion-based decoder, the
process starts with sampling a noisy input $x^{K}\sim\mathcal{N}(0,I)$. Then,
the decoder iteratively denoises the input while conditioning the decoder on
the disentangled embeddings to generate an action,

$x^{k-1}=\frac{1}{\sqrt{\alpha^{k}}}\left(x^{k}-\frac{1-\alpha^{k}}{\sqrt{1-\bar{\alpha}^{k}}}\epsilon(x^{k},k,s_{t},z_{\rho},z_{\sigma})\right)+\zeta^{k}\eta$
(13)

where $\eta\sim\mathcal{N}(0,I)$, $\zeta^{k}$ and $\alpha^{k}$ are parameters
for variance schedule. As it is empirically observed that the low-temperature
sampling leads to improved performance, similar to (Ajay et al. 2023), we set
$\zeta^{k}=0$.

For downstream task adaptation, we explore two different scenarios: few-shot
imitation and online RL adaptation. For the few-shot imitation, we initialize
the high-level policy with the learned domain-invariant and domain-variant
priors, and then we fine-tune the high-level policy using (10) along with the
learned guided diffusion-based decoder. Likewise, for online RL adaptation, we
adopt the soft actor-critic (SAC) algorithm, where the learned priors guide
the high-level policy as in (Pertsch, Lee, and Lim 2020). In both scenarios,
we freeze the decoder and solely fine-tune the high-level policy. Even in the
absence of decoder updates, our framework manages to attain robust performance
on downstream tasks in different domains, as demonstrated in Section 4.2.
Figure 3 (ii) illustrates the procedure of downstream policy learning.

Domain | Level | BC | SPiRL* | SPiRL-c | SPiRL-c* | FIST* | DuSkill
---|---|---|---|---|---|---|---
Speed | Source | $2.66\pm 0.73$ | $2.16\pm 0.63$ | $3.62\pm 0.35$ | $3.72\pm 0.19$ | $3.98\pm 0.01$ | $\textbf{3.99}\pm\textbf{0.00}$
Level 1 | $2.71\pm 0.28$ | $1.83\pm 0.50$ | $3.41\pm 0.50$ | $3.51\pm 0.25$ | $3.86\pm 0.05$ | $\textbf{3.92}\pm\textbf{0.07}$
Level 2 | $2.62\pm 0.54$ | $2.16\pm 0.62$ | $3.33\pm 0.44$ | $3.24\pm 0.23$ | $3.51\pm 0.12$ | $\textbf{3.83}\pm\textbf{0.06}$
Level 3 | $2.22\pm 0.23$ | $2.22\pm 0.49$ | $2.87\pm 0.47$ | $3.12\pm 0.22$ | $3.28\pm 0.19$ | $\textbf{3.81}\pm\textbf{0.06}$
Energy | Source | $1.49\pm 0.35$ | $0.67\pm 0.07$ | $2.83\pm 0.39$ | $1.60\pm 0.33$ | $3.08\pm 0.19$ | $\textbf{3.85}\pm\textbf{0.08}$
Level 1 | $1.26\pm 0.32$ | $0.56\pm 0.07$ | $1.90\pm 0.45$ | $1.19\pm 0.13$ | $2.53\pm 0.25$ | $\textbf{3.87}\pm\textbf{0.09}$
Level 2 | $0.90\pm 0.15$ | $0.47\pm 0.08$ | $1.77\pm 0.26$ | $0.91\pm 0.19$ | $1.85\pm 0.34$ | $\textbf{3.71}\pm\textbf{0.10}$
Level 3 | $0.53\pm 0.09$ | $1.69\pm 0.25$ | $0.89\pm 0.18$ | $2.02\pm 0.24$ | $1.04\pm 0.20$ | $\textbf{3.67}\pm\textbf{0.14}$
Wind | Source | $3.32\pm 0.41$ | $3.78\pm 0.10$ | $2.83\pm 0.39$ | $3.92\pm 0.10$ | $\textbf{4.00}\pm\textbf{0.00}$ | $3.98\pm 0.02$
Level 1 | $2.99\pm 0.52$ | $3.14\pm 0.54$ | $1.90\pm 0.45$ | $3.71\pm 0.29$ | $\textbf{3.89}\pm\textbf{0.15}$ | $3.78\pm 0.13$
Level 2 | $2.19\pm 0.42$ | $2.62\pm 0.33$ | $1.77\pm 0.26$ | $3.24\pm 0.24$ | $3.24\pm 0.24$ | $\textbf{3.48}\pm\textbf{0.21}$
Level 3 | $2.41\pm 0.45$ | $2.63\pm 0.46$ | $0.89\pm 0.18$ | $3.04\pm 0.29$ | $3.04\pm 0.29$ | $\textbf{3.44}\pm\textbf{0.18}$
Table 1: Few-shot Imitation Performance in Multi-stage Meta-World: The
performance of the baselines and DuSkill is measured in achieved rewards. For
each domain, we categorize domain disparity between the training datasets and
downstream tasks into four different levels. The baselines marked with an
asterisk (*) indicate that both the high-level policy and decoder are fine-
tuned, while the baselines without an asterisk and DuSkill only fine-tune the
high-level policy. Each is evaluated with 3 random seeds, and the highest
performance is highlighted in bold.

##  4 Evaluations

###  4.1 Experiment Settings

Environments. For evaluation, we use the multi-stage Meta-World, which is
implemented based on the Meta-World simulated benchmark (Yu et al. 2019). Each
multi-stage task is composed of a sequence of existing Meta-World tasks (sub-
tasks). In these multi-stage tasks, an agent is required to maneuver a robotic
arm to complete a series of sub-tasks, such as slide puck, close drawer, and
etc. To emulate different domains in the environment, we deliberately add
variations to either reward functions or transition dynamics. Specifically, we
modify the reward function by setting time constraints (i.e., speed domains)
and by incorporating the energy usage consideration (energy domains).
Furthermore, to emulate varying conditions of transition dynamics in the
environment, we manipulate kinematic parameters such as wind (wind domains).

Datasets. For datasets, we implement several rule-based expert policies
tailored to each domain-specific environment. For offline training datasets,
we collect 20 trajectories for each source domain (of $6\sim 16$ domains). For
few-shot imitation datasets, we collect another 3 trajectories for each target
domain.

Baselines. For baselines, we implement several imitation learning and skill-
based RL algorithms.

  * •

BC (Esmaili, Sammut, and Shirazi 1998) is a widely used supervised behavior
cloning method. A policy is learned on the training datasets and then fine-
tuned for the downstream tasks.

  * •

SPiRL (Pertsch, Lee, and Lim 2020) is a state-of-art skill-based RL algorithm
that employs a hierarchical structure to embeds skills into a latent space,
thereby accelerating the downstream adaptation.

  * •

SPiRL-c is a variant of SPiRL that uses the closed-loop skill decoder, used in
(Pertsch et al. 2021).

  * •

FIST (Hakhamaneshi et al. 2022) is a state-of-art few-shot skill-based
imitation algorithm that employs a semi-parametric approach for skill
determination.

To obtain few-shot imitation, the baselines such as BC, SPiRL and SPiRL-c are
pretrained on the training datasets and then fine-tuned on the few-shot
imitation datasets.

###  4.2 Few-shot Imitation Performance

Table 1 compares the few-shot imitation performance in rewards achieved by our
framework (DuSkill) and other baselines (BC, SPiRL, SPiRL-c, FIST). Among the
baselines, those denoted with an asterisk (*) signify that both the high-level
policy and the decoder are fine-tuned, while those without an asterisk
indicate fine-tuning solely of the high-level policy. An important distinction
is that DuSkill exclusively focuses on fine-tuning the high-level policy.
Based on the domain disparities between the datasets and the downstream tasks,
we categorize them into four different levels. At the source level, downstream
tasks remain same with the source domains. As we move to higher levels, the
domain disparities become more pronounced.

As shown, the baselines exhibit a notable decline in performance as the domain
dissimilarity increases from source to level 4, where even the most
competitive baseline FIST achieves an average degradation of $25.3\%$. In
contrast, DuSkill consistently maintains robust performance across all domains
and levels, achieving a small degradation of $7.6\%$ at average.

In this experiment, SPiRL-c demonstrates relatively low performance, primarily
because its decoder can only generate skills present in its training data.
Consequently, this poses a challenge in attaining robust performance when
solely relying on fine-tuning the high-level policy. Meanwhile, SPiRL-c* is
expected to yield higher performance than SPiRL-c as it fine-tunes both the
high-level policy and the decoder; yet, in some cases, SPiRL-c surpasses
SPiRL-c*. This is because fine-tuning the entire model with a few samples
might cause a covariant shift, a phenomenon commonly observed in the few-shot
imitation studies (Hakhamaneshi et al. 2022; Nasiriany et al. 2022). FIST*
adopts a different strategy, involving the fine-tuning of the entire model
along with the utilization of a semi-parametric method to retrieve the future
state $s_{t+H}$ that it aims to reach from the training datasets. While the
semi-parametric method leads to improved performance for tasks in source
domains, FIST* is prone to fail in producing skills for downstream tasks in
different domains. This is because the training datasets do not cover the
skills required for different domains. In contrast, DuSkill disentangles the
domain-invariant and -variant features to effectively generate the skills
through the guided diffusion-based decoder. This allows for robust performance
in few-shot adaptation across different domains, through fine-tuning solely
the high-level policy.

###  4.3 Analysis

Online RL. Table 2 compares online RL adaptation performance in reward
achieved by our framework (DuSkill) and other baselines (BC+SAC, SPiRL,
SPiRL-c). Both the baselines and DuSkill fine-tune the high-level policy via
the SAC algorithm. Here, we categorize domain disparities into source and
target domains, where target domains correspond to the level 3 in Table 1. For
more stable learning in target domains, we warm-up the high-level policy with
a single trajectory for DuSkill and other baselines. The performance gap
between SPiRL-c and DuSkill is not remarkable in source domain tasks, as
expected. In contrary, DuSkill exhibits superior performance compared to the
baselines for tasks in different domains, outperforming SPiRL-c by $89.16\%$.
This highlights the capability of our guided diffusion-based decoder in
generating diverse skills that extend beyond the limitations of the given
datasets.

Level | BC+SAC | SPiRL-c | DuSkill
---|---|---|---
Source | $0.00\pm 0.00$ | $4.00\pm 0.00$ | $\textbf{4.00}\pm\textbf{0.00}$
Target | $0.00\pm 0.00$ | $0.36\pm 0.02$ | $\textbf{3.32}\pm\textbf{0.20}$
Table 2: Online RL Performance in Speed Domain

Embedding Visualization. Here, we verify the efficacy of our DuSkill framework
in disentangled embeddings. Figure 4 visualizes the domain-invariant and
domain-variant embeddings generated by DuSkill separately in two distinct
speed domains (fast and slow). In the figure, the labels T1 to T4 correspond
to sub-tasks numbered from 1 to 4. Regarding the domain-invariant embeddings,
we observe that the identical tasks are paired together, thereby establishing
the domain-invariant knowledge. On the other hand, the domain-variant
embeddings are grouped with respect to the domains, implying the encapsulation
of domain-variant knowledge. This indicates the effectiveness of our offline
skill diffusion process, which disentangles domain-invariant and variant
features.

Figure 4: Visualization of Domain-invariant and Domain-variant embeddings

Sample Efficiency. Figure 5 shows the performance with respect to samples (or
timesteps) used by DuSkill and the baselines (SPiRL-c, SPiRL-c*, FIST) for
downstream policy learning in few-shot imitation and online RL scenarios. For
the few-shot imitation learning, we test with different numbers of few-shot
trajectories $(1\sim 20)$. As shown in Figure 5(a), DuSkill exhibits robust
performance with only a $4.89\%$ drop from $1$ to $20$ trajectories, whereas
the most competitive baseline FIST shows a notable performance drop of
$13.96\%$. Furthermore, as shown in Figure 5(b), DuSkill efficiently adapts to
the downstream task in online settings, enhancing performance with only $50$k
samples, while SPiRL-c rarely achieves improvement with those samples.

(a) Few-shot imitation

(b) Online RL

Figure 5: Sample Efficiency of Downstream Policy Learning

Ablation on Skill Diffusion. Table 3 provides an ablation study of DuSkill,
focusing on the impact of the hierarchical embedding structure and guided
diffusion-based decoder. In this study, we conduct few-shot imitation
scenarios with two ablated variants. DU is a variant of SPiRL-c, utilizing a
naive diffusion model for the skill decoder, while HDU employs the
hierarchical embedding structure like DuSkill and a naive diffusion model for
the skill decoder. The results show that the combination of hierarchical
domain encoder and guided diffusion based decoder in DuSkill yields improved
performance compared to the other variants, showing $8.77\sim 8.92\%$ average
performance gains. This is not surprising since the guided diffusion based
decoder promotes the learning of disentangled representations, as the domain-
invariant and domain-variant decoders are conditioned on each embedding to
generate executable actions effectively. Therefore, it is crucial to employ
both hierarchical embedding structure and guided diffusion based decoder for
achieving better few-shot imitation learning performance.

Domain | DU | HDU | DuSkill
---|---|---|---
Speed | $3.59\pm 0.19$ | $3.49\pm 0.68$ | $\textbf{3.81}\pm\textbf{0.06}$
Energy | $3.16\pm 0.35$ | $3.39\pm 0.36$ | $\textbf{3.67}\pm\textbf{0.14}$
Wind | $3.19\pm 0.17$ | $3.08\pm 0.36$ | $\textbf{3.44}\pm\textbf{0.18}$
Table 3: Performance by Encoder and Decoder Types

##  5 Related Work

Skill-based Learning. To leverage offline datasets for long-horizon tasks,
hierarchical skill representation learning techniques have been investigated
in the context of online RL (Pertsch, Lee, and Lim 2020; Pertsch et al. 2021)
and imitation learning (Hakhamaneshi et al. 2022; Nasiriany et al. 2022; Du et
al. 2023). Pertsch, Lee, and Lim (2020) proposed the hierarchical skill
learning structure to accelerate the downstream task adaptation by guiding a
high-level policy with learned skill priors. Meanwhile, Hakhamaneshi et al.
(2022) exploited a semi-parametric approach within this hierarchical skill
structure, focusing on the few-shot imitation. In our work, we also utilize
skill embedding techniques, but we tackle the challenge of adapting to
downstream tasks in different domains. Unlike the prior work, our DuSkill
adapts a robust generative model such as diffusion with the technique of
disentangled skill embeddings, enabling the effective generation of diverse
skills in offline.

Diffusion for RL. Given the remarkable success of diffusion models in the
field of computer vision (HO, Jain, and Abbeel 2020; Rombach et al. 2022),
their application has been extended to sequential decision-making problems in
recent years. Pearce et al. (2023) utilized diffusion models to imitate human
demonstrations, capitalizing on their capacity to represent highly multi-modal
data. Wang, Hunt, and Zhou (2023) adopted diffusion models in the context of
offline RL to implement policy regularization. Furthermore, Liang et al.
(2023) leveraged diffusion models to generate diverse synthetic trajectories
on limited training data, aiming to have self-evolving offline RL for goal-
conditioned RL tasks. Recently, Ajay et al. (2023) proposed a general
framework for sequential decision-making problems using diffusion models. It
allows for dynamic recombination of behaviors at test time by conditioning the
diffusion models on several factors such as returns, constraints, or skills.
To the best of our knowledge, our DuSkill framework is the first to integrate
a diffusion model with skill embedding techniques, providing a novel
hierarchical RL method to generate diverse skills on limited datasets and
achieving robust performance for downstream tasks in different domains.

##  6 Conclusion

In this work, we presented the DuSkill framework, a novel approach designed to
bridge the gap between the given datasets and downstream tasks that exist
within different domains. We devised the offline skill diffusion, which
employs the guided diffusion-based decoder in conjunction with the
hierarchical encoders to effectively disentangles domain-invariant and
-variant features from skills. This enables the generation of diverse skills
capable of addressing tasks in different domains. Our framework stands apart
from existing skill-based learning approaches, which are typically limited to
adapt within the domains encountered during skill pretraining. In our future
work, we aim to extend our framework to address challenging cross-domain
situations with significant domain shifts, such as entirely different tasks,
robot embodiment variations, or different simulation environments.

## Acknowledgments

We would like to thank anonymous reviewers for theri valuable comments and
suggestions. This work was supported by Institute of Information &
communications Technology Planning & Evaluation (IITP) grant funded by the
Korea government (MSIT) (No. 2022-0-01045, 2022-0-00043, 2019-0-00421,
2021-0-00875) and by the National Research Foundation of Korea (NRF) grant
funded by the MSIT (No. RS-2023-00213118).

## References

  * Ajay et al. (2023) Ajay, A.; Du, Y.; Gupta, A.; Tenenbaum, J. B.; Jaakkola, T. S.; and Agrawal, P. 2023.  Is Conditional Generative Modeling All You Need for Decision-Making?  In _Proceedings of the 11th International Conference on Learning Representations_.
  * Du et al. (2023) Du, M.; Nair, S.; Sadigh, D.; and Finn, C. 2023.  Behavior Retrieval: Few-Shot Imitation Learning by Querying Unlabeled Datasets.  arXiv:2304.08742.
  * Esmaili, Sammut, and Shirazi (1998) Esmaili, N.; Sammut, C.; and Shirazi, G. M. 1998.  Behavioural cloning in control of a dynamic system.  In _Proceedings of the IEEE International Conference on Systems, Man and Cybernetics._ , 2904–2909.
  * Hakhamaneshi et al. (2022) Hakhamaneshi, K.; Ruihan Zhao, A. Z.; Abbeel, P.; and Laskin, M. 2022.  Hierarchical Few-Shot Imitation with Skill Transition Models.  In _Proceedings of the 10th International Conference on Learning Representations_.
  * Higgins et al. (2016) Higgins, I.; Matthey, L.; Pal, A.; Burgess, C. P.; Glorot, X.; Botvinick, M. M.; Mohamed, S.; and Lerchner, A. 2016.  beta-VAE: Learning Basic Visual Concepts with a Constrained Variational Framework.  In _Proceedings of the 4th International Conference on Learning Representations_.
  * HO, Jain, and Abbeel (2020) HO, J.; Jain, A.; and Abbeel, P. 2020.  Denoising Diffusion Probabilistic Models.  In _Proceedings of the 34th Conference on Neural Information Processing System_.
  * Ho and Salimans (2022) Ho, J.; and Salimans, T. 2022.  Classifier-Free Diffusion Guidance.  arXiv:2207.12598.
  * Liang et al. (2023) Liang, Z.; Mu, Y.; Ding, M.; Ni, F.; Tomizuka, M.; and Luo, P. 2023.  AdaptDiffuser: Diffusion Models as Adaptive Self-evolving Planners.  In _Proceedings of the 40th International Conference on Machine Learning_.
  * Nasiriany et al. (2022) Nasiriany, S.; Gao, T.; Mandlekar, A.; and Zhu, Y. 2022.  Learning and Retrieval from Prior Data for Skill-based Imitation Learning.  In _Proceedings of the 6th Conference on Robot Learning_ , 2181–2204.
  * Pearce et al. (2023) Pearce, T.; Rashid, T.; Kanervisto, A.; Bignell, D.; Sun, M.; Georgescu, R.; Macua, S. V.; Tan, S. Z.; Momennejad, I.; Hofmann, K.; and Devlin, S. 2023.  Imitating Human Behaviour with Diffusion Models.  In _Proceedings of the 11th International Conference on Learning Representations_.
  * Pertsch, Lee, and Lim (2020) Pertsch, K.; Lee, Y.; and Lim, J. J. 2020.  Accelerating Reinforcement Learning with Learned Skill Priors.  In _Proceedings of the 4th Conference on Robot Learning_ , 188–204.
  * Pertsch et al. (2021) Pertsch, K.; Lee, Y.; Wu, Y.; and Lim, J. J. 2021.  Demonstration-Guided Reinforcement Learning with Learned Skills.  In _Proceedings of the 5th Conference on Robot Learning_ , 729–739.
  * Preechakul et al. (2022) Preechakul, K.; Chatthee, N.; Wizadwongsa, S.; and Suwajanakorn:, S. 2022.  High-Resolution Image Synthesis with Latent Diffusion Models.  In _Proceedings of the Conference on Computer Vision and Pattern Recorgnition_ , 10609–10619.
  * Rombach et al. (2022) Rombach, R.; Blattmann, A.; Lorenz, D.; Esser, P.; and Ommer, B. 2022.  High-Resolution Image Synthesis with Latent Diffusion Models.  In _Proceedings of the Conference on Computer Vision and Pattern Recorgnition_ , 10674–10685.
  * Wang, Hunt, and Zhou (2023) Wang, Z.; Hunt, J. J.; and Zhou, M. 2023.  Diffusion Policies as an Expressive Policy Class for Offline Reinforcement Learning.  In _Proceedings of the 11th International Conference on Learning Representations_.
  * Yu et al. (2019) Yu, T.; Quillen, D.; He, Z.; Julian, R.; Hausman, K.; Finn, C.; and Levine, S. 2019.  Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learning.  In _Proceedings of the 3rd Conference on Robot Learning_ , 1094–1100.


# What Affects Learned Equivariance in Deep Image Recognition Models?

Robert-Jan Bruintjes∗,1, Tomasz Motyka ,2, Jan van Gemert1
1Computer Vision Lab, Delft University of Technology, 2Synerise
<EMAIL_ADDRESS><EMAIL_ADDRESS>Equal
contribution.

###### Abstract

Equivariance w.r.t. geometric transformations in neural networks improves data
efficiency, parameter efficiency and robustness to out-of-domain perspective
shifts. When equivariance is not designed into a neural network, the network
can still learn equivariant functions from the data. We quantify this learned
equivariance, by proposing an improved measure for equivariance. We find
evidence for a correlation between learned translation equivariance and
validation accuracy on ImageNet. We therefore investigate what can increase
the learned equivariance in neural networks, and find that data augmentation,
reduced model capacity and inductive bias in the form of convolutions induce
higher learned equivariance in neural networks.

##  1 Introduction

Figure 1: Neural networks can learn features that are invariant or equivariant
w.r.t. a geometric transformation of the data, such as rotation. We measure
learned equivariance w.r.t. translation and rotation in neural networks.

Equivariance in neural network features allows invariance to geometric
transformations [23, 6], making such networks more data efficient [38, 40,
25], parameter efficient [6] and robust to out-of-distribution transformations
[1, 13, 33].

Equivariance with respect to specific geometric transformations can be
designed into the neural network architecture [6, 37, 5]. However, even with
careful design, it may happen that the resulting architecture is not as
equivariant as intended [13, 45, 19, 17]. An example is the convolution
operator in Convolutional Neural Networks (CNNs) for translation equivariance,
which can be broken by border effects [19, 17] or pooling [45]. On the other
hand, even if neural networks are not designed to be equivariant, they can
still learn equivariance naturally. Existing works demonstrate qualitative
examples of learned equivariant features [2, 10, 29]. However, how much
equivariance is learned, and which factors affect equivariance, are open
questions.

In this work, we quantify learned equivariance in image recognition neural
networks that have and have not been explicitly designed for equivariance.
Where existing works [15, 46, 18, 3, 27] typically only measure equivariance
at the output of the network, we measure equivariance for all intermediate
layers. To do so, we deviate from existing measures of learned equivariance
which are inconsistent across network depths, and we design a consistent
measure.

Using our measure for learned equivariance, we find evidence that learned
translation equivariance in intermediate features of neural networks
correlates with increased validation accuracy on ImageNet. We therefore
investigate how we can increase learned equivariance by changing how we train
neural networks. In particular, we find that 1) making the task equivariant
does not increased learned equivariance; 2) data augmentations designed for
invariance indeed increase learned equivariance, even in early and middle
layers; 3) reducing model capacity increases learned equivariance, suggesting
that equivariant features arise from a need to compress representations; 4)
CNNs learn more translation and rotation equivariance in intermediate features
than the Vision Transformers (ViTs).

We make the following contributions:

  * •

We propose a new measure for learned equivariance that is allows comparing
learned equivariance of features at different depths of the network.

  * •

We show evidence for a positive correlation between learned translation
equivariance in intermediate features and validation accuracy on ImageNet.

  * •

We test how several aspects of neural network training affect learned
equivariance. In summary, we find that data augmentation, reduced model
capacity and the inductive biases of CNNs positively affect learned
equivariance.

##  2 Related Works

Neural networks can learn equivariant features from data [24, 28, 29].
Particularly inspiring is the work by Olah _et al_. [29], that demonstrates by
precise and meticulous manual investigations that learned equivariant features
exist in networks that were not designed to be equivariant. Inspired by this
work, we here investigate how to move beyond laborous manual qualitative
investigations, and instead offer a quantitative approach, by giving an
automatic measure for learned equivariance.

A number of existing works measure equivariance in neural networks. [15] study
models from the pre-Deep Learning era which have since been superseded by the
models we study. More recent works measure equivariance in Convolutional
Neural Networks, with KL divergence on class probabilities [46], with
Euclidean distance [18], or cosine similarity [3, 27] on feature maps. In our
work, we show that the cosine similarity is not appropriate for measuring
equivariance in intermediate feature maps, and offer a correlation-based
measure.

Several works study how neural network hyperparameters and datasets affect
learned translation equivariance in the final output of the network. The
kernel and padding sizes of the architecture affect translation invariance
[27], although data augmentation might have a bigger effect on translation
invariance than the network architecture [18]. Similar conclusions about the
importance of the data were found by others [3, 4]. Here, we follow these
investigations, and extend them by analyzing the impact on the intermediate
layers.

There are some works that study equivariant properties of intermediate
features. Recently, [26] proposed a method to detect invariance to any learned
Lie group for intermediate features. However, they do not study equivariance,
like we do. Other works study only the transformation group of translations
($\mathbb{Z}^{2}$). [45] measures the translation equivariance by computing
cosine similarity between feature maps to show how max pooling violates the
translation equivariance property. [19, 17] show that some padding methods
disrupt the translation equivariance property in CNNs. [32] measure the
invariance of intermediate representations using normalized cosine similarity
to study the effect of pooling on deformation stability. Where these works
diagnose issues with designed equivariance and test for their effects, we
consider learned equivariance in a more general sense, including
transformation groups not designed into the network, such as rotations.

##  3 Method

Neural networks can learn to be equivariant in two ways: either by learning
invariant features or by learning equivariant groups of features, as shown in
Fig 1. In this section we detail how we can measure the quantity of invariant
features and equivariant groups of features. We discuss which similarity
measure is appropriate for measurements of learned equivariance in features at
different depths of a neural network. Finally, we verify our measures using
artificially engineered equivariant CNNs.

In the following we will refer to invariant features and equivariant groups of
features under the single predicate ”learned equivariance”, as invariance is a
special case of equivariance.

###  3.1 Invariant features

We derive a measure of learned equivariance from inspecting the definition of
equivariance [6] applied to a single neural network layer:

$f(T_{g}(X))=T_{g}^{\prime}(f(X)),$ (1)

where $X\in\mathbb{R}^{C_{\text{in}}\times H\times W}$ is an image or a
feature map, $f(X)\in\mathbb{R}^{C\times H\times W}$ is the output of a neural
network operation with $C$ output features and $T_{g}$ is the application of a
transformation $g$ from a transformation group $G$. For example, if
$G=\mathbb{Z}^{2}$, then $g$ is a translation with a particular integer-valued
$(x,y)$ offset. If $T_{g}^{\prime}$ is the identity function for all $g$, the
layer $f$ is invariant w.r.t. transformation $T$:

$f(T(X))=f(X).$ (2)

Without designing invariance to $T$ into neural network layer $f$, each
individual feature in $f(X)$ can learn to behave invariant or not invariant
with respect to $T$. We therefore define invariance for each feature $c\in C$
independently:

$f(T(X))_{c}=f(X)_{c}.$ (3)

In Fig. 1 we show an example where $T$ is a $90^{\circ}$ rotation.

To measure a feature’s invariance w.r.t. $g$, we compute the similarity
between $f(T(X))_{c}$ and $f(X)_{c}$:

$\text{Invariance}(f_{c},g)=S(f(T(X))_{c},f(X)_{c}),$ (4)

given a similarity function $S:\mathbb{R^{H\times W}}\times\mathbb{R^{H\times
W}}\to[0,1]$. Given invariance measures for each feature, we can average these
measures for all features in a layer to compute a layer’s invariance.

###  3.2 Equivariant features

(a) Comparison of different similarity measures on random feature maps.
Feature maps on the bottom are shifted by 0.5 with respect to the top ones.
Cosine similarity is sensitive to such shifts while correlation is invariant.

(b) Comparison between magnitude of the activations (red) and equivariance
computed using correlation (blue) and cosine similarity (orange). We compute
correlation between the magnitude of the activations and equivariance scores
computed with correlation (0.11) and cosine similarity (0.63).

Figure 2: Analysis of the influence of magnitude of weights on different
similarity measures.

A group of features $C_{G}\subseteq C$ in a neural network layer is
equivariant with respect to a transformation group $G$ if each feature
$c^{\prime}\in C_{G}$ activates for a different transformation $g$ from the
group. In other words, for a sample $T_{g}(X)$ transformed with any
transformation from the group, the group of feature maps $f(X)_{c^{\prime}\in
C_{G}}$ will have one feature $c^{\prime}$ whose transformed feature map
$T_{g}(f(X))_{c^{\prime}}$ matches $f(T_{g}(X))_{c^{\prime}}$:

$f(T_{g}(X))_{c^{\prime}}=T_{g}(f(X))_{c^{\prime}},\quad\exists c^{\prime}\in
C_{T}.$ (5)

In Fig. 1 we show an example where $g$ is a $90^{\circ}$ rotation.

To fit this with the definition of equivariance (Eq. 1) we define
$T^{\prime}_{g}$:

$f(T_{g}(X))_{c^{\prime}\in C_{G}}=T^{\prime}_{g}(f(X))_{c^{\prime}\in C_{G}}$
(6)

where $T^{\prime}_{g}$ transforms with $T_{g}$ and selects feature
$c^{\prime}$ that matches the transformation $g$. When this equation holds,
the feature group $C_{G}$ is equivariant w.r.t. $T$.

To measure the equivariance of a feature $c$ we find the maximum similarity
between $f(T_{g}(x))_{c}$ and $T_{g}(f(x))_{c^{\prime}}$ over all features
$c^{\prime}\in C$, for a given transformation $g$:

$\displaystyle\text{Equivariance}(f_{c},g)$ $\displaystyle=\max_{c^{\prime}\in
C}S(f(T_{g}(X))_{c},T_{g}(f(X))_{c^{\prime}})$ (7)

given a similarity function $S:\mathbb{R^{H\times W}}\times\mathbb{R^{H\times
W}}\to[0,1]$. Given equivariance measures for each feature, we can average
these measures for all features in a layer to compute a layer’s equivariance.
Note that if a feature is invariant w.r.t. $T_{g}$, we will measure an
equivariance score that is at least as high as the invariance score. As
invariance is a special case of equivariance, this behavior of our measure is
intended.

###  3.3 Measuring similarity

(a) Rotation invariance

(b) Rotation equivariance

Figure 3: We create two controlled toy CNNs, each designed to be perfectly
invariance and equivariant respectively, to test if our method measures
equivariance correctly, which it does.

We need to choose a similarity measure $S:\mathbb{R^{H\times
W}}\times\mathbb{R^{H\times W}}\to[0,1]$ with which to compare feature maps to
measure invariance and equivariance. In existing works, cosine similarity is
commonly used as a similarity measure used to compute invariance or
equivariance of the networks representations [4, 27, 45]. However, cosine
similarity is sensitive to the mean values of its input vectors. This
behaviour is depicted in Figure 2(a). As different layers in a neural network
have different mean activation values (see Fig. 2(b)), this biases the
similarity measure.

We propose to use Pearson correlation [14] instead. Pearson correlation, also
called centered cosine similarity, is a similarity measure that does not
suffer from sensitivity to the mean of the inputs, as it computes the
covariance of the inputs normalized by their standard deviations. It is the
basis of many methods for comparing network representations [20, 22, 31].

To motivate our choice, we visualize the difference between using cosine
similarity and correlation for measuring equivariance in the following
example. We train ResNet-44 [16] model on CIFAR-10 [21] dataset and compute
invariance w.r.t. to $90^{\circ}$ rotation after each residual block. In
Figure 2 we show the qualitative comparison between the scores, computed using
cosine similarity and correlation, and the mean of the activations.
Additionally, we compute a correlation between the magnitude of the
activations and equivariance scores computed with cosine similarity (0.63) and
correlation (0.11). Scores computed using cosine similarity correlate visibly
with the mean of the activation while, for the scores computed using
correlation, this effect is less prevalent. In our experiments, we therefore
use correlation as a measure to quantify equivariance.

##  4 Experiments

###  4.1 Controlled experiments

To verify that our method captures equivariance, we apply it to two controlled
toy settings. We create two 3-layer CNNs with hand-crafted filters such that
we expect to measure perfect learned rotation invariance and equivariance
respectively. For the invariant model, we set all the filters to be
rotationally symmetric, using a 2D isotropic Gaussian function, and measure
the invariance after each layer (Fig. 3(a)). For the equivariant model, we cut
out corners of the filters from the invariant model such that all the filters
are rotations of one another (Fig. 3(b)). Our measure finds both models
capture exactly the intended learned equivariances, demonstrating the validity
of our measure.

###  4.2 Does learned equivariance improve accuracy?

We study the relationship between the validation accuracy and the amount of
learned equivariance in large-scale seminal models. For each part of each
trained model we compute Spearman’s rank correlation between the amount of
invariance or equivariance and the ImageNet validation accuracy of the model.

We test four CNNs (EfficientNet-B6 & EfficientNet-B7 [35], ResNeXT-101 [41]
and Inception-V3 [34]) and two Vision Transformer variants (Vision Transformer
[11] and MLP-Mixer [36]). We measure invariance and equivariance for both
translation and rotation for 2000 images from the ImageNet validation set. We
do not train the models ourselves but instead use available checkpoints from
torchvison [30] or timm [39]. Since the studied model families do not have the
same exact number of layers, we divide each model into depth-wise parts and
report the average equivariance measures over all layers in each part. Feature
maps from the beginning of the network until the global average pooling (GAP)
layer are uniformly partitioned into Early, Middle and Late parts. Pool
captures the feature maps directly after the GAP layer and Final is the
feature map directly before the softmax layer. We discriminate between the
Pool part and the Final part to identify what role in achieving equivariance
the global pooling and final classifier have.

Figure 4 shows there is some correlation between translation equivariance in
Early and Middle layers and accuracy on ImageNet, while attaining almost
perfect correlation in the Final part. In contrast, for rotations there is
little correlation between the equivariance in the representation before
global pooling and the validation accuracy.

Even though the sample size (six models) for this correlation test is small,
we conclude that there is some evidence for the benefit of learning
translation equivariance in intermediate features of neural networks trained
on ImageNet. In the following we therefore study what can increase the learned
equivariance in such networks.

###  4.3 Equivariance in the data

On tasks where invariant responses are beneficial to solve the task, e.g.
translation invariance in image recognition, one may wonder how this
invariance is achieved. We study how learned equivariance in intermediate
features is affected by adding transformations to the data and therefore into
the task. We choose to study rotation transformations on CNNs, as rotation
equivariance is not designed into CNNs. We study whether there is a difference
if the task is invariant or equivariant with respect to introduced
transformations.

We train a 7-layer CNN taken from [6], consisting of 7 layers of $3\times 3$
convolutions, 20 channels in each layer, ReLU activation functions, batch
normalization, and max-pooling after layer 2, on 3 different datasets. We test
on three different datasets. The first dataset is MNIST6, which is the regular
MNIST[9] without $\\{0,1,6,8\\}$ classes, to get rid of rotational
transformations that these classes have. For example, digit 8 is very similar
to its $180^{\circ}$ rotation, so, by default, this class would introduce some
rotation invariance, which is undesirable as we want to control for rotation
invariance in this setting. Second is the MNIST6-Rot-Inv where every digit in
MNIST6 is randomly rotated by
$r\in\\{0^{\circ},90^{\circ},180^{\circ},270^{\circ}\\}$ upfront. This dataset
imposes invariance into the task as, for every transformation, the predicted
class should be the same. The last dataset, MNIST6-Rot-Eq, is created in the
same way as MNIST6-Rot-Inv, but now the classes are made up of all
combinations of digit number and rotation (e.g. class 2 is
$(\text{digit}0^{\circ},180^{\circ})$). This dataset imposes equivariance into
the task. We compute in- and equivariance of the trained model for 2000 images
from the validation set. We average the score over
${90^{\circ},180^{\circ},270^{\circ}}$ rotations. Each experiment is repeated
three times using different random seed. We train for 100 epochs using Adam
with a batch size of 128 and a learning rate of 0.01, L2 regularization at
0.0005 and weight decay at epochs 25 and 50 with a factor of 10.0.

In Figure 5 we show the learned rotation equivariance. Firstly, we observe
that the equivariance decreases with the depth, up to the final part after
global average pooling (GAP), regardless of the task. For the tasks where the
equivariance or invariance is imposed in the task, we see an increase in the
final part, which suggests that GAP plays a significant role in achieving
equivariance. Secondly, we do not see any significant differences between
MNIST6, MNIST6-Rot-Inv and MNIST6-Rot-Eq, up to a later stage, which may
indicate that early convolutional layers learn features with some amount of
rotation equivariance regardless of the rotation invariance of the task.
Finally, we observe that rotation equivariance is much larger then rotation
invariance in the early and middle layers, which shows that CNNs do learn more
rotated versions of the same feature in different channel rather then learning
invariant, symmetrical features in a single channel. We conclude that
introducing equivariance into the task does not significantly affect the
learned equivariance of intermediate features.

###  4.4 Data augmentations

Figure 4: Spearman’s rank correlation between learned equivariance and
ImageNet validation accuracy. Translation equivariance in intermediate
features correlates with increased accuracy on ImageNet, while rotation
equivariance does not. Figure 5: Learned rotation equivariance for rotation
invariance/equivariance in the data. Invariance or equivariance in the task
does not induce learning more equivariant features up until the late part of
the network. Also there is no visible difference, up until the late part of
the network, in the learned equivariance between the invariant and equivariant
tasks.

(a) Data augmentations

(b) Model capacity: number of channels $s$

Figure 6: Measuring learned translation equivariance for (a) data
augmentations and (b) model capacity. For data augmentations (a), random crops
and RandAugment increase channel equivariance the most, while other strategies
have no discernible improvements. For model capacity (b), smaller models learn
more in- and equivariance, although the amount of in- and equivariance in the
end is similar.

(a) Learned translation equivariance

(b) Learned rotation equivariance

Figure 7: Measuring learned equivariance for inductive biases. For translation
(a), the CNN variants exhibit more equivariance in the intermediate
representation then the Vision Transformer variants. Global pooling seems to
play an important role in achieving invariance. For rotation (b), the CNN
variants exhibit more equivariance in the intermediate representation then the
Vision Transformer variants. The Early and Middle parts have more equivariance
than invariance.

By duplicating input samples under some transformation, data augmentation can
induce invariance in the neural network. We study what the effect of data
augmentation is on learned equivariance in intermediate representations: does
data augmentation result in more invariant or equivariant intermediate
features? For example, we know that the random crops data augmentation method,
which essentially introduces random translations into the data, increases the
model performance and translation invariance at the last layer [18]. The
question is whether random crops increase the learned equivariance of
intermediate features as well. In this experiment we study how translation
equivariance is affected by different data augmentations.

We train ResNet-44[16], adapted for CIFAR-10[21], on the CIFAR-10 dataset
using one of the following augmentations: random crops, horizontal flips,
CutMix [43], RandAugment [7]. In each experiment we compute equivariance of
the trained model over 2000 images from the validation set and average the
score over diagonal shifts from one to 16 pixels. Each experiment is repeated
three times by training the network different random seeds. We train for 200
epochs using SGD with a batch size of 128 and a learning rate of 0.1 and a
momentum of 0.9, L2 regularization at 0.0001 and weight decay at epochs 100
and 150 with a factor of 10.0.

In principle we expect the in- and equivariance to be the same since
translation equivariance should be provided by the convolution. However, we
include equivariance in our experiments since there are works showing that the
information about location can be encoded in different channels [17, 19].

In Figure 6(a) we show learned translation equivariance for the tested data
augmentations. Random crops and RandAugment increase the equivariance of
learned features in the Middle, Late and Final parts, while the other data
augmentation methods do not have any significant effect, with CutMix even
having less equivariance than the baseline in the Middle part. We complement
the finding of [18] by showing that random crops increase not only translation
invariance but also translation equivariance in the intermediate layers. Also,
we do not see any difference between invariance and equivariance for any data
augmentation, which means that any equivariance learned is just invariance.

###  4.5 Model capacity

We hypothesize that a smaller model in principle benefits from a more
efficient representation and hence may learn more equivariant features. We
therefore study whether model capacity influences learning translation
equivariant representations. We train WideResNet-40 (WRN-40) [44] models,
where we scale the number of channels (the ”width”) in each layer by a factor
$s\in\\{1,2,4,8\\}$. We train on the CIFAR-10 dataset and measure learned
translation equivariance. The hyperparameters used for training are the same
as in the data augmentation experiment of Sec 4.4.

In Figure 6(b) we show learned translation equivariance for different model
capacities. We observe that the amount of translation equivariance is lower
for the wider models, even though the amount of invariance in the final part
is the same, which matches our hypothesis: an efficient representation learns
to be equivariant.

###  4.6 Architectures

The architecture of a neural network determines which biases can be learned in
training. Vision Transformers (ViTs) [11] lack certain inductive biases
present in CNNs, which has been linked to their reduced data efficiency [42,
12]. We are interested to what extent the difference in inductive bias between
CNNs and Vision Transformers (ViTs) affects learned equivariance.

We test architectures as they were designed for the ImageNet dataset [8], to
faithfully represent their intended inductive bias. We use the same
architectures and pre-trained model weights as tested in Sec. 4.2: four CNNs
(EfficientNet-B6 & EfficientNet-B7 [35], ResNeXT-101 [41] and Inception-V3
[34]) and two Vision Transformer variants (Vision Transformer [11] and MLP-
Mixer [36]). We measure both translation and rotation equivariance on trained
models for 2000 images from the ImageNet validation set. We also use the same
depth-wise partitioning of feature maps into parts as used in Sec. 4.2. We
measure translation equivariance over diagonal shifts of size 1 to 32 and
rotation equivariance for ${90\textdegree,180\textdegree,270\textdegree}$
rotations.

In Figure 7(a) we present the results for learned translation equivariance. We
can see that ViT and MLP-Mixer have less translation equivariance than CNNs in
Early and Middle layers. This is not unexpected, as convolutions directly
integrate translation equivariance, whereas Vision Transformers have to learn
position embeddings that are translation equivariant. This reduced translation
equivariance could be the reason for the poor data efficiency of ViT and MLP-
Mixer [11, 36] since translation equivariance improves data efficiency [19].
Finally, we note that learned invariance and equivariance are identical for
the tested models, meaning that these networks do not learn to represent
different translations in different channels.

In Figure 7(b) we present the results for learned rotation equivariance. We
observe that the ViT and MLP-Mixer have lower rotation equivariance than the
CNNs in intermediate features, while after the GAP layer the ViT exhibits the
most rotation equivariance out of all the models. Secondly, we note that early
parts of all networks learn equivariant features that are not invariant, more
so than in late parts of the networks. In contrast to the results for
translation equivariance, we see that models with low rotation equivariance
throughout Early, Middle and Late parts (ViT, Efficient-Net B6/B7) have the
highest rotation equivariance in the Final part, while the models with highest
equivariance in Early, Middle and Late parts (ResNeXT-101, Inception-v3) have
the least equivariance in the Final part. This shows that high learned
equivariance in the final model representation does not imply that
intermediate representations are also highly equivariant.

##  5 Conclusion

We conduct a quantitative study on learned equivariance in intermediate
features of CNNs and Vision Transformers trained for image recognition, using
an improved measure of equivariance. We find evidence that translation
equivariance in intermediate representations correlates with ImageNet
validation accuracy. We show that data augmentations and reduced model
capacity can increase learned equivariance in intermediate features. Also, the
CNNs we test learn more translation and rotation equivariance in intermediate
features than the ViTs we test.

Limitations. Our method allows to measure equivariance w.r.t. affine
transformations only. The reason for that is the transformation $g$ with
respect to which we measure the equivariance has to be a map from and to an
identical discrete domain, e.g. feature maps. This restriction disqualifies
continuous transformations such as rotations with any other resolution than 90
degrees, or scaling with non-integer scaling factors.

Future work. Learned equivariance benefits image recognition models. However,
applying equivariant priors usually adds additional cost in terms of memory or
computation. Future work could study whether one can apply equivariant priors
selectively within a neural network, saving computing cost where networks
already learn to be equivariant. Additionally, we show that Vision
Transformers learn less translation equivariance than CNNs. Future work could
explore methods to increase translation invariance in Vision Transformers, to
aid in their data efficiency.

## Acknowledgements

Robert-Jan Bruintjes and Jan van Gemert are financed by the Dutch Research
Council (NWO) (project VI.Vidi.192.100). All authors sincerely thank everyone
involved in funding this work.

## References

  * [1] Thomas Altstidl, An Nguyen, Leo Schwinn, Franz Köferl, Christopher Mutschler, Björn Eskofier, and Dario Zanca.  Just a matter of scale? reevaluating scale equivariance in convolutional neural networks.  arXiv preprint arXiv:2211.10288, 2022.
  * [2] James Bergstra, Aaron Courville, and Yoshua Bengio.  The statistical inefficiency of sparse coding for images (or, one gabor to rule them all).  arXiv preprint arXiv:1109.6638, 2011.
  * [3] Valerio Biscione and Jeffrey Bowers.  Learning translation invariance in cnns.  arXiv preprint arXiv:2011.11757, 2020.
  * [4] Diane Bouchacourt, Mark Ibrahim, and Ari S. Morcos.  Grounding inductive biases in natural images: invariance stems from variations in data.  arXiv preprint arXiv:2211.10288, 2021.
  * [5] Taco Cohen, Maurice Weiler, Berkay Kicanaoglu, and Max Welling.  Gauge equivariant convolutional networks and the icosahedral cnn.  In International conference on Machine learning, pages 1321–1330. PMLR, 2019.
  * [6] Taco S. Cohen and Max Welling.  Group equivariant convolutional networks.  In Proceedings of the 33rd International Conference on International Conference on Machine Learning - Volume 48, ICML’16, page 2990–2999. JMLR.org, 2016.
  * [7] Ekin D. Cubuk, Barret Zoph, Jonathon Shlens, and Quoc V. Le.  Randaugment: Practical automated data augmentation with a reduced search space.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, June 2020.
  * [8] Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei.  Imagenet: A large-scale hierarchical image database.  In 2009 IEEE Conference on Computer Vision and Pattern Recognition, pages 248–255, 2009.
  * [9] Li Deng.  The mnist database of handwritten digit images for machine learning research.  IEEE Signal Processing Magazine, 29(6):141–142, 2012.
  * [10] Sander Dieleman, Jeffrey De Fauw, and Koray Kavukcuoglu.  Exploiting cyclic symmetry in convolutional neural networks.  In International conference on machine learning, pages 1889–1898. PMLR, 2016.
  * [11] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al.  An image is worth 16x16 words: Transformers for image recognition at scale.  arXiv preprint arXiv:2010.11929, 2020.
  * [12] Stéphane d’Ascoli, Hugo Touvron, Matthew L Leavitt, Ari S Morcos, Giulio Biroli, and Levent Sagun.  Convit: Improving vision transformers with soft convolutional inductive biases.  In International Conference on Machine Learning, pages 2286–2296. PMLR, 2021.
  * [13] Tom Edinxhoven.  Using and abusing equivariance.  Master’s thesis, Delft University of Technology, 2023.
  * [14] David Freedman, Robert Pisani, and Roger Purves.  Statistics (international student edition).  Pisani, R. Purves, 4th edn. WW Norton & Company, New York, 2007\.
  * [15] Ian Goodfellow, Honglak Lee, Quoc Le, Andrew Saxe, and Andrew Ng.  Measuring invariances in deep networks.  Advances in neural information processing systems, 22, 2009.
  * [16] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun.  Deep residual learning for image recognition.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016.
  * [17] Md Amirul Islam, Matthew Kowal, Sen Jia, Konstantinos G Derpanis, and Neil DB Bruce.  Global pooling, more than meets the eye: Position information is encoded channel-wise in cnns.  In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 793–801, 2021.
  * [18] Eric Kauderer-Abrams.  Quantifying translation-invariance in convolutional neural networks, 2017\.
  * [19] Osman Semih Kayhan and Jan C van Gemert.  On translation invariance in cnns: Convolutional layers can exploit absolute spatial location.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14274–14285, 2020.
  * [20] Simon Kornblith, Mohammad Norouzi, Honglak Lee, and Geoffrey Hinton.  Similarity of neural network representations revisited.  In International Conference on Machine Learning, pages 3519–3529. PMLR, 2019.
  * [21] Alex Krizhevsky, Geoffrey Hinton, et al.  Learning multiple layers of features from tiny images.  2009\.
  * [22] Malte Kuss and Thore Graepel.  The geometry of kernel canonical correlation analysis.  2003\.
  * [23] Yann LeCun, Yoshua Bengio, et al.  Convolutional networks for images, speech, and time series.  The handbook of brain theory and neural networks, 3361(10):1995, 1995.
  * [24] Karel Lenc and Andrea Vedaldi.  Understanding image representations by measuring their equivariance and equivalence.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 991–999, 2015.
  * [25] Attila Lengyel and Jan van Gemert.  Exploiting learned symmetries in group equivariant convolutions.  In 2021 IEEE International Conference on Image Processing (ICIP), pages 759–763. IEEE, 2021.
  * [26] Artem Moskalev, Anna Sepliarskaia, Ivan Sosnovik, and Arnold Smeulders.  Liegg: Studying learned lie group generators.  arXiv preprint arXiv:2210.04345, 2022.
  * [27] Johannes C Myburgh, Coenraad Mouton, and Marelie H Davel.  Tracking translation invariance in cnns.  In Artificial Intelligence Research: First Southern African Conference for AI Research, SACAIR 2020, Muldersdrift, South Africa, February 22-26, 2021, Proceedings 1, pages 282–295. Springer, 2020.
  * [28] Chris Olah, Nick Cammarata, Ludwig Schubert, Gabriel Goh, Michael Petrov, and Shan Carter.  An overview of early vision in inceptionv1.  Distill, 5(4):e00024–002, 2020.
  * [29] Chris Olah, Nick Cammarata, Chelsea Voss, Ludwig Schubert, and Gabriel Goh.  Naturally occurring equivariance in neural networks.  Distill, 5(12):e00024–004, 2020.
  * [30] Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang, Zachary DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala.  Pytorch: An imperative style, high-performance deep learning library.  In Advances in Neural Information Processing Systems 32, pages 8024–8035. Curran Associates, Inc., 2019.
  * [31] Maithra Raghu, Justin Gilmer, Jason Yosinski, and Jascha Sohl-Dickstein.  Svcca: Singular vector canonical correlation analysis for deep learning dynamics and interpretability.  Advances in neural information processing systems, 30, 2017.
  * [32] Avraham Ruderman, Neil C Rabinowitz, Ari S Morcos, and Daniel Zoran.  Pooling is neither necessary nor sufficient for appropriate deformation stability in cnns.  arXiv preprint arXiv:1804.04438, 2018.
  * [33] Mateus Sangalli, Samy Blusseau, Santiago Velasco-Forero, and Jesus Angulo.  Scale equivariant u-net.  arXiv preprint arXiv:2210.04508, 2022.
  * [34] Christian Szegedy, Vincent Vanhoucke, Sergey Ioffe, Jon Shlens, and Zbigniew Wojna.  Rethinking the inception architecture for computer vision.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 2818–2826, 2016.
  * [35] Mingxing Tan and Quoc Le.  Efficientnet: Rethinking model scaling for convolutional neural networks.  In International conference on machine learning, pages 6105–6114. PMLR, 2019.
  * [36] Ilya O Tolstikhin, Neil Houlsby, Alexander Kolesnikov, Lucas Beyer, Xiaohua Zhai, Thomas Unterthiner, Jessica Yung, Andreas Steiner, Daniel Keysers, Jakob Uszkoreit, et al.  Mlp-mixer: An all-mlp architecture for vision.  Advances in neural information processing systems, 34:24261–24272, 2021.
  * [37] Maurice Weiler and Gabriele Cesa.  General e (2)-equivariant steerable cnns.  Advances in Neural Information Processing Systems, 32, 2019.
  * [38] Maurice Weiler, Fred A Hamprecht, and Martin Storath.  Learning steerable filters for rotation equivariant cnns.  In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 849–858, 2018.
  * [39] Ross Wightman.  Pytorch image models.  https://github.com/rwightman/pytorch-image-models, 2019.
  * [40] Marysia Winkels and Taco S Cohen.  3d g-cnns for pulmonary nodule detection.  arXiv preprint arXiv:1804.04656, 2018.
  * [41] Saining Xie, Ross Girshick, Piotr Dollár, Zhuowen Tu, and Kaiming He.  Aggregated residual transformations for deep neural networks.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1492–1500, 2017.
  * [42] Yufei Xu, Qiming Zhang, Jing Zhang, and Dacheng Tao.  Vitae: Vision transformer advanced by exploring intrinsic inductive bias.  Advances in Neural Information Processing Systems, 34:28522–28535, 2021.
  * [43] Sangdoo Yun, Dongyoon Han, Seong Joon Oh, Sanghyuk Chun, Junsuk Choe, and Youngjoon Yoo.  Cutmix: Regularization strategy to train strong classifiers with localizable features.  In Proceedings of the IEEE/CVF international conference on computer vision, pages 6023–6032, 2019.
  * [44] Sergey Zagoruyko and Nikos Komodakis.  Wide residual networks.  arXiv preprint arXiv:1605.07146, 2016.
  * [45] Richard Zhang.  Making convolutional networks shift-invariant again.  CoRR, abs/1904.11486, 2019.
  * [46] Allan Zhou, Fahim Tajwar, Alexander Robey, Tom Knowles, George J Pappas, Hamed Hassani, and Chelsea Finn.  Do deep networks transfer invariances across classes?  arXiv preprint arXiv:2203.09739, 2022.