# Can Transformers Jump Around Right in Natural Language?
Assessing Performance Transfer from SCAN

Rahma Chaabouni∗†
Ecole Normale Superieure
{chaabounirahma<EMAIL_ADDRESS><EMAIL_ADDRESS>&Roberto Dessì
Facebook AI & Pompeu Fabra &Eugene Kharitonov
Facebook AI  Equal contribution. † work was done while R.C. was at Facebook
AI.

###### Abstract

Despite their failure to solve the compositional SCAN dataset, seq2seq
architectures still achieve astonishing success on more practical tasks. This
observation pushes us to question the usefulness of SCAN-style compositional
generalization in realistic NLP tasks. In this work, we study the benefit that
such compositionality brings about to several machine translation tasks. We
present several focused modifications of Transformer that greatly improve
generalization capabilities on SCAN and select one that remains on par with a
vanilla Transformer on a standard machine translation (MT) task. Next, we
study its performance in low-resource settings and on a newly introduced
distribution-shifted English-French translation task.

Overall, we find that improvements of a SCAN-capable model do not directly
transfer to the resource-rich MT setup. In contrast, in the low-resource
setup, general modifications lead to an improvement of up to 13.1% BLEU score
w.r.t. a vanilla Transformer. Similarly, an improvement of 14% in an accuracy-
based metric is achieved in the introduced compositional English-French
translation task. This provides experimental evidence that the compositional
generalization assessed in SCAN is particularly useful in resource-starved and
distribution-shifted scenarios.

##  1 Introduction

While sequence-to-sequence (seq2seq) models achieve remarkable performance in
many tasks (Sutskever et al., 2014; Raffel et al., 2019; Adiwardana et al.,
2020), they often fail to generalize in a systematic way Baroni (2019); McCoy
et al. (2020); Hupkes et al. (2020); Kharitonov and Chaabouni (2021); Dankers
et al. (2021). These shortcomings are particularly obvious in the experiments
on the SCAN domain Lake and Baroni (2018); Loula et al. (2018); Bastings et
al. (2018).

In SCAN, inputs are instructions that describe trajectories and outputs define
sequences of actions to follow them (see Table 1). To illustrate how SCAN
probes a model for compositional generalization, imagine that we train it on a
set of instructions {jump, run, turn, turn twice, run twice}, but test it on
jump twice. Strictly speaking, nothing in the training data indicates that the
model must output JUMP JUMP instead of e.g. JUMP. However, it is hypothesised
that a bias for such compositional, human-like induction is beneficial Lake
and Baroni (2018); Lake (2019).

This hypothesis stumbles into a perplexing situation: despite failing at
compositional generalization, considered as a core requirement for language
understanding, seq2seq models have tremendous success in practice. Is the type
of compositional generalization, that SCAN probes for, useful for NLP tasks?
If so, in what scenarios?

In this work, we aim to answer this question. Firstly, we introduce focused
modifications to Transformer that greatly improve accuracy performance on
SCAN. To build such modifications, we exploit two observations: (i)
Transformer’s architecture is very similar to convolution-based seq2seq models
(ConvS2S) (Gehring et al., 2017), (ii) ConvS2S performs well on SCAN (Dessì
and Baroni, 2019). This capability (ii) is hypothesized to be due to
explicitly localized representation of the sequences, where only deep layers
of the model can access more distant tokens Dessì and Baroni (2019); Hupkes et
al. (2020). Such a capability should also benefit to natural language
processing as human languages are proved to favor local syntactic
constructions Futrell et al. (2015). Motivated by these observations, we focus
on the major differences between ConvS2S and Transformer, namely the localized
receptive field span and the gating mechanism, to inject into Transformer
inductive biases useful for SCAN.

As a testbed, we use the machine translation (MT) domain as one of the most
popular applications for seq2seq models. We consider both resource-rich
(IWSLT’14 German$\rightarrow$English) and low-resource
(Nepali&Sinhala$\leftrightarrow$English) setups. Finally, to evaluate SCAN-
style capabilities in natural language tasks, we build a dataset that probes
whether models can systematically generalize w.r.t. noun-adjective ordering
while translating from English to French. We construct this dataset by
filtering the EuroParl corpus (a part of WMT’14).

Our results indicate that combining two ConvS2S-inspired changes improves
accuracy on one SCAN split (SCAN-jump) from 3.4% to 43.0%, while maintaining a
high accuracy on the other splits (SCAN-simple and SCAN-around-right). As
expected, given that SCAN is an artificial diagnostic dataset, not all
modifications lead to equal improvements on an MT task. We select one of the
considered modifications that performs on par with the vanilla Transformer on
IWSLT’14.

Testing the selected modification on low-resource data, we observe that it
provides between 3.6% and 13.1% BLEU improvements over Transformer. On the
noun-adjective ordering dataset, we find that our modification results into
gains in generalization of 14%.

This leads to the following picture: the localized attention, augmented by a
gating mechanism, provides a useful inductive bias that proves to be
beneficial for SCAN-style generalization. Additionally, it turns out useful in
low-resource and distribution-shifted settings. Thus, testing seq2seq models
on SCAN while controlling for a non-degraded performance leads to improvement
in domains where syntactic compositionality is crucial for a task success.

jump | $\Rightarrow$ | JUMP
---|---|---
jump around right | $\Rightarrow$ | RTURN JUMP RTURN JUMP RTURN JUMP RTURN JUMP
turn left twice | $\Rightarrow$ | LTURN LTURN
jump opposite left after walk around left | $\Rightarrow$ | LTURN WALK LTURN WALK LTURN WALK LTURN WALK LTURN LTURN JUMP

Table 1: Examples of SCAN trajectories and instructions, adopted from (Lake
and Baroni, 2018).

##  2 Transformers and ConvS2S

Architecture overview Both Transformer and ConvS2S are encoder-decoder
architectures Sutskever et al. (2014), where the decoder has an attention
mechanism to peek into the encoder’s representation Bahdanau et al. (2014).
These representations are obtained by embedding the inputs, adding a
positional embedding, and passing them through a sequence of layers.

In Transformer’s encoder, the output representations are the result of a
sequential application of two (sub)layer types: self-attention and fully
connected layers. The input representation can “skip” any sublayer via a
residual connection. The output of the sublayer is passed through a dropout
mechanism and added to the residual. This sum is then layer-normalized. Any
relation between input tokens is modeled solely by self-attention modules.

In ConvS2S, the encoder is also a sequence of identical blocks. The inter-term
dependencies are modeled by 1D convolutions with GLU activation functions
Dauphin et al. (2017).111GLU was introduced as a combination of a convolution
network and an activation; we follow the Pytorch convention and consider it as
separate blocks for convenience. In contrast to self-attention, convolutions
have a finite kernel size, thus effectively capping the maximal distance of
intermediate dependencies that can be modeled. The GLU activation function
serves as a gate, allowing ConvS2S to control the balance between residuals
and the output of the convolution. After the GLU operation, the intermediate
representation is added to the residual and scaled. The output of the final
convolution is then passed into a fully connected layer. In ConvS2S and
Transformer, decoders have similar structures to those of encoders, with an
additional decoder$\rightarrow$encoder attention layer after the convolution
and self-attention blocks, respectively.

Despite the similarities, there are numerous low-level differences between the
two architectures: normalization (layer norm Ba et al. (2016) vs. weight
normalization Salimans and Kingma (2016)), optimization (Adam Kingma and Ba
(2014) with a ramp-up vs. NAG Sutskever et al. (2013)), etc. A priori, any of
those can affect models’ inductive biases. However, we concentrate on some of
the most obvious architectural differences: the limited convolution span and
GLU activations.

We believe these features can greatly affect models’ performance on SCAN.
Indeed, SCAN has only local dependencies between tokens, thus the ability to
avoid spurious correlations with more distant tokens can be useful. Similarly,
the ability to weight contributions from the token interactions into the
intermediate representation is intuitively prerequisite to build compositional
representation.

GLU Dauphin et al. (2017) Given a vector input ${\bm{x}}$, GLU splits it in
two equally sized halves ${\bm{x}}_{1}$ and ${\bm{x}}_{2}$; one is passed
through a sigmoid ($\sigma(x)=(1+e^{-x})^{-1}$). Then both parts are pointwise
multiplied:

$GLU({\bm{x}})\coloneqq\sigma({\bm{x}}_{1})\odot{\bm{x}}_{2}$ (1)

This allows a network to implement a gating mechanism, where one half of its
output gates the signal from the second.

Self-attention Instead of convolutions, Transformer uses multi-headed self-
attention to model interactions between tokens. Given $n$ embeddings,
${\bm{x}}_{1},{\bm{x}}_{2},...{\bm{x}}_{n}$ of dimensionality $d$, the self-
attention layer transforms them in the following way.

Firstly, each embedding ${\bm{x}}_{i}$ is projected by three matrices $Q$,
$K$, and $V$ to get query ${\bm{q}}_{i}$, key ${\bm{k}}_{i}$, and value
${\bm{v}}_{i}$ representations, respectively:
${\bm{q}}_{i},{\bm{k}}_{i},{\bm{v}}_{i}\leftarrow
Q{\bm{x}}_{i},K{\bm{x}}_{i},V{\bm{x}}_{i}$. Next, a scaled dot-product between
query ${\bm{q}}_{i}$ and key ${\bm{k}}_{j}$ is calculated as follow:

$\alpha_{ij}=\frac{1}{\sqrt{d}}{\bm{q}}_{i}\cdot{\bm{k}}_{j}^{T}$ (2)

This dot-product defines the attention weights $w_{ij}=e^{\alpha_{ij}}/\sum
e^{\alpha_{ij}}$ which are used to get the output representations:
${\bm{o}}_{i}=\sum_{j}w_{ij}{\bm{v}}_{j}$. This process is done in parallel
for multiple heads, acting on independent slices of the input embeddings;
their outputs are concatenated and passed through a fully connected layer.

##  3 Transformer Modifications

Self-attention gate (SAG) The simplest way to imitate the effect of GLU
activation (Eq. 1) is to weight (gate) the output of self-attention by a
learned scalar parameter. To ensure that it is non-negative and is scaled in
$[0,1]$, we parameterize it as a sigmoid of a real-valued learned parameter
$\beta$. Algorithm 1 illustrates the introduced change. In comparison to
Transformer, SAG adds one scalar parameter for each encoder and decoder layer.

We treat $\beta_{0}$, the value $\beta$ is initialized with before training,
as a hyperparameter. In the preliminary experiments, we found that after
training, encoder layers often have small negative $\beta$ values
($-2..-0.5$), while decoder layers have positive values ($0.2..4.5$) that grow
monotonically for higher layers.

A similar modification was considered in an effort to stabilize Transformer
training in the Reinforcement Learning domain Parisotto et al. (2020).

Convolution as self-attention In the limit case, we can entirely replace self-
attention with convolutions. This modification introduces one hyperparameter
(kernel size). However, convolutional layers have fewer parameters than the
self-attention mechanism. One might consider this not to be a Transformer
variant due to the lack of self-attention, but as self-attention generalizes
convolutions Cordonnier et al. (2020), we consider this as an extreme form of
regularization.

Fixed-span self-attention A less extreme modification would be to use the
regular multi-head self-attention mechanism, but without allowing attention to
peek beyond some distance. This mimics the limited kernel size of convolutions
in ConvS2S. We achieve this by adding a fixed bias term $b_{ij}$ to the self-
attention logits (Eq. 2):

$\alpha_{ij}=\frac{1}{\sqrt{d}}{\bm{q}}_{i}\cdot{\bm{k}}_{j}^{T}+b_{ij}$ (3)

Setting $b_{ij}$ to $-\infty$ when the difference $|i-j|$ exceeds some fixed
value $s$ and to 0 otherwise prevents the self-attention to look beyond
distance $s$.

Fixed-span self-attention with a span parameter $s$ has the same “receptive
field” as 1D convolution with kernel size $2s+1$. This modification adds one
hyperparameter (span size), but does not introduce new learned parameters.

T5 attention Further relaxing constraints on self-attention, we consider the
case where we allow Transformer to learn how to (soft-)limit its self-
attention. We introduce the bias term $b_{ij}$ that is learned as a function
of a (signed) difference $i-j$, capping it to $[-s,+s]$ (e.g., positions with
difference above $s$ would have the same bias $b_{s}$).

This modification is similar to one introduced by Raffel et al. (2019) in T5,
with the only exception that we allow each head to have its own bias. Again,
the span size is a new hyperparameter. In a model with $n_{h}$ heads and
$n_{l}$ layers, this modification requires $(2s+1)\times n_{l}\times n_{h}$
new parameters, which is negligible in comparison with the sizes of fully
connected layers. Examples of the learned $b_{ij}$ parameters are shown in
Supplementary when training on SCAN.

Implementation details We used the fairseq Ott et al. (2019) implementation of
Transformer seq2seq as a foundation, with its initialization and default
parameters. T5 and fixed-span attentions are implemented by providing additive
attention masks to Transformer.

1:procedure compute_SelfAttention

2: $res\leftarrow x$

3: $x\leftarrow self\\_attn(x)$

4: $x\leftarrow x*\sigma(\beta)$

5: $x\leftarrow layer\\_norm(res+dropout(x))$

Algorithm 1 Self-attention gate (SAG). The only introduced change is on line
4. $\beta$ is a learned per-layer scalar parameter.

##  4 Datasets

###  4.1 SCAN

Introduced by Lake and Baroni (2018), SCAN is a collection of tasks used for
studying systematic generalization of seq2seq models (see Table 1 for some
input-output examples). A set of 4 primitive verbs are combined with different
modifiers generating around 27k unique samples. Lake and Baroni (2018) and,
later, Loula et al. (2018) prepared several non-i.i.d. splits of the data into
training and test sets. To successfully generalize on such non-i.i.d. splits,
a model has to generalize systematically, in a compositional way.

We experiment with three tasks, often focused on in the literature222We have
also ensured that our best modification performs on par ($\approx$ 10%) with
Transformer on SCAN-length; however SCAN-length is believed to require a
different type of generalization Gordon et al. (2019). Dessì and Baroni
(2019). SCAN-simple splits all sequences in train and test sets uniformly at
random. Hence, both train and test are identically distributed. Typically
models succeed at it easily. In SCAN-jump, the test set contains all
compositional uses of one of the primitives, jump. The train set contains all
uses of other primitives, and inputs where jump is used in isolation. SCAN-
around-right tests if a model is capable to generalize to combinations of two
modifiers, around and right, that never co-occur in the training data. The
test data contain all examples where the two modifiers are combined.

###  4.2 Machine Translation

We hypothesize that the type of systematic generalization that SCAN probes for
could be most useful in data-poor tasks or tasks with train-test distribution
shift. Hence, we complement the standard IWSLT’14 En-De dataset with a low-
resource task, FLoRes. To study whether our models can perform SCAN-style
generalization on natural language data, we also build a dataset that probes
for compositional generalization in noun-adjective ordering in French, when
translating from English.

IWSLT’14 En-De This is a standard MT dataset, that includes train, validation,
& test sets. We apply preprocessing as in the fairseq
example.333https://github.com/pytorch/fairseq/tree/master/examples/translation

FLoRes Guzmán et al. (2019) FloRes is a low-resource dataset for
English$\leftrightarrow$ Nepali and English $\leftrightarrow$ Sinhala
translation. The dataset is split into dev, devtest, and test subsets. We only
use the provided supervised data.

Noun-adjective ordering We take inspiration from SCAN-jump to construct an MT
dataset that probes for compositional generalization using noun-adjective
ordering in French. In French, both adjective noun (forward) and noun
adjective (backward) orders are used, unlike English that only has the forward
order. Which order is used largely depends on the adjective. For example, to
refer to a specific response, French speakers say résponse spécifique
(backward order), while new response would be nouvelle résponse (forward).

To draw a parallel with SCAN-jump, we consider the nouns as primitives and
adjectives as modifiers. Modifiers appear with different primitives, however,
some primitives appear with only one modifier. For instance, if, in the
training set, response only appears with specific (backward), we test models
on translating sentences containing new response, where new modifies many
other nouns in the training set in the forward order. Such generalization is
required by humans when dealing with rare or too specific nouns.

To construct our dataset, we start from the English-French Europarl dataset (a
part of WMT’14 En-Fr)444http://www.statmt.org/europarl/ and select $8$ nouns,
$\mathcal{N}=${response, institution, organisation, solution, source,
decision, responsibility, population}. We constrain our train set so that each
of the nouns in $\mathcal{N}$ appears only with one adjective (hence in one
particular order) as shown in Table 2. For example, the noun response will
only be composed with the adjective specific. However, specific (and all other
adjectives in Table 2) appears with other nouns. To select these sentences, we
use the CoreNLP parser Manning et al. (2014).555https://stanfordnlp.github.io/
Finally, all sentences with nouns that are not among the selected ones are
kept. In other words, the training set may contain sentences that have neither
the selected adjectives nor the selected nouns. This results to $1641681$
sentence pairs split into train ($1478195$ pairs) and validation ($163486$
pairs) sets.
The test set is composed of the filtered sentences of the original Europarl
dataset: we select sentences where nouns in the backward column of Table 2
({response, institution, organism, solution}) are only modified by the
adjectives in the forward column ({new, good, big, first}). Similarly, we also
consider the sentences where the nouns of the forward column are composed with
the adjectives of the backward column of the Table.666We use the Stanford
parser to select these sentences. This process will ensure that in the test
set, the noun-adjective only appears in the reverse order compared to the
train set. Unlike the training data, the test data contains only sentences
with the target nouns and adjectives. In total, we test models on $659$
sentences. Note that the train and validation sets are identically
distributed, however, the test set is distribution-shifted w.r.t. train, akin
to SCAN-jump.
We follow the preprocessing steps on the fairseq example page for WMT’14
English to French.3

backward order | forward order
---|---
(‘specific’, ‘response’) (‘particular’, ‘institution’) (‘effective’, ‘organisation’) (‘permanent’, ‘solution’) |  (‘new’, ‘source’) (‘good’, ‘decision’) (‘big’, ‘responsibility’) (‘first’, ‘population’)

Table 2: (adjective, noun) pairs in the train set of the noun-adjective
ordering dataset, classified by their order in French language.

| jump | around-right | simple
---|---|---|---
Transformer | $3.4_{\pm 2.0}$ | $97.6_{\pm 1.5}$ | $100.0_{\pm 0.0}$
self-att. gate (SAG) | $17.2_{\pm 5.8}$ | $85.2_{\pm 10.0}$ | $100.0_{\pm 0.0}$
\+ Conv as s.-a. | $25.7_{\pm 20.4}$ | $38.4_{\pm 7.8}$ | $99.8_{\pm 0.0}$
\+ Fixed-span | $33.6_{\pm 9.5}$ | $97.6_{\pm 1.3}$ | $100.0_{\pm 0.0}$
\+ T5 | $43.0_{\pm 9.5}$ | $92.6_{\pm 2.8}$ | $100.0_{\pm 0.0}$
LSTM seq2seq Lake and Baroni (2018) | 1.2 | 2.5 | 99.8
ConvS2S Dessì and Baroni (2019) | $69.2_{\pm 8.2}$ | $56.7_{\pm 10.2}$ | $100.0_{\pm 0.0}$

Table 3: Accuracy on SCAN tasks, %. For each architecture and task, we report
the mean accuracy of the best hyperparameter configuration. $\pm$ denotes 1
SEM.

##  5 Methodology

SCAN Lake and Baroni (2018) were concerned by the feasibility of systematic
generalization in seq2seq models. Hence, in their experiments, they tuned the
models on the train data and then directly evaluated them on test set,
reporting test scores. We follow the same protocol: given a grid of
hyperparameters, we fit models on the training data. Next, for each
hyperparameter configuration, we average the performance of the models across
random seeds. Such a setup demonstrates that, at least for some hyperparameter
configurations, the introduced models can learn to generalize systematically.
At evaluation time, we decode greedily.

IWSLT’14 De-En We run a grid search on train data; next we select the best
performing checkpoint on the validation dataset. We report performance on the
test data. We use the same training and evaluation protocols as suggested on
the fairseq MT example page.3 We use beam size 5.

FLoRes This dataset has dev, devtest, and test splits provided. We run a
hyperparameter grid search training on the dev data. Next, we select the
hyperparameter configuration that has the best average (across seeds)
performance on devtest. We report the performance of the selected
hyperparameter configuration on the test set, averaged across seeds. We use
the training/evaluation scripts, tokenization and other parameters suggested
on the dataset page: beam size 5 and length penalty 1.2.

Noun-adjective ordering We run the hyperparameter search similarly to IWSLT’14
De-En. The training and evaluation protocols are the ones suggested by the
fairseq page for WMT’14 En-Fr.3 We also use beam size 5.

As we aim to probe abilities for compositional generalization, we introduce an
accuracy-based measure, _COMP_. When analyzing models’ errors, we encountered
3 common errors: (1) removing the adjective (example 1 in Table 6), (2)
replacing the adjective with a synonym and reversing the order (examples 2 and
3 in Table 6), and (3) producing a completely wrong generalization while
removing the adjective. While (2) provides a good enough translation, it is a
mistake in the noun-adjective order. However, when outputting the right noun
and adjective, the order is always preserved. Hence, to measure if a model is
compositional, we only look if both the target adjective and the target noun
appear in the prediction, irrespective of their order. We define thus COMP as
the ratio of predicted sentences that include _both_ the target adjective and
noun.777It happens that models use a synonym in the right order as shown in
SAG+T5’s prediction 2 in Table 6. In that case, models had generalized well
but are still penalized by COMP. COMP is hence only a proxy measure for
compositional generalization based on the common failures.

Hyperparameter search Transformer models have multiple hyperparameters
(embeddings dimensionality, number of layers and attention heads, dropout
probabilities, etc.). On top of those, our introduced models add the attention
span $s$, and the initial gate state $\beta_{0}$. For MT tasks, we start from
the existing strong baseline hyperparameter configurations (FLoRes: specified
by Guzmán et al. (2019), De-En & En-Fr: following the fairseq example page)
and only tune (a) the parameters introduced by our architectures, and (b) the
attention dropout parameter (for all architectures, including Transformer).
For SCAN, there is no baseline hyperparameter configuration, so we start with
tuning Transformer and then base hyperparameters of the introduced
architectures on it. We report full hyperparameter grids in Supplementary.

##  6 SCAN experiments

In our preliminary experiments, we found that our modifications of the self-
attention mechanism do not lead to improvements over the standard Transformer
when they are not combined with the self-attention gate (SAG). Hence, we focus
our experiments on architectures that include SAG.

We report our results in Table 3. We also include results for LSTM- and Conv-
based seq2seq models that were reported in earlier work Lake and Baroni
(2018); Dessì and Baroni (2019). From Table 3, we see that the unmodified
Transformer has very low accuracy on jump (3.4%), which is only slightly above
that of LSTM seq2seq (1.2%) and well below ConvS2S (69.2%). This indicates
that Transformer models are indeed failing in compositional generalization on
jump. However, they have a very high score on the around-right split (97.6%)
and simple ($\geq$ 99.8%). By introducing the different modifications
described in Section 3, making Transformers closer to ConvS2S, we aim at
preserving the high performance of Transformers on around-right and simple
while significantly improving it on jump.

Adding SAG increases accuracy on jump 5-fold (17.2%) at the expense of a small
drop in around-right scores (not stat. sig.).

Further, we observe that changes of the self-attention mechanism (replacing it
with Convs, limiting its span, and adding a relative position-dependent bias),
can further increase the performance on jump. Apart from SAG+Conv as s.-a, the
self-attention modifications do not significantly alter the performance on
around-right.

We see that the architectural changes that we proposed improve the
compositional capabilities of the Transformer models. As expected, the
introduced hybrid architectures reach significantly better performance on jump
(up to 12x improvements for SAG+T5) while keeping high performance on the
around-right & simple tasks.

Transformer | SAG | \+ Conv s.-a. | \+ fix. span | \+ T5
---|---|---|---|---
$34.64_{\pm 0.03}$ | $34.28_{\pm 0.08}$ | $33.44_{\pm 0.04}$ | $34.32_{\pm 0.01}$ | $34.66_{\pm 0.04}$

Table 4: BLEU on test set. IWSLT’14 German to English dataset. $\pm$ denotes 1
SEM.

##  7 Machine Translation experiments

IWSLT’14 De-En In Table 4, we report BLEU scores on German-English
translation. SAG + T5 performs slightly better (0.02 BLEU, not stat. sig.),
but other modifications underperform w.r.t. Transformer. Replacing self-
attention with convolutions resulted in the largest drop, 3%. Other
differences are smaller, $\leq$ 1%. For the following parts, we only
experiment with the SAG + T5 model as the only non-degraded one. However,
results with the remaining models on FLoRes and the Noun-adjective ordering
datasets are reported in Supplementary.

FLoRes, Si/Ne $\leftrightarrow$ En We report results on
English$\leftrightarrow$Nepali and English$\leftrightarrow$Sinhala translation
in Table 5. Following Guzmán et al. (2019), we use tokenized BLEU when
translating from English. We run standard Transformer models as specified in
Guzmán et al. (2019), but adding a search over the attention dropout
probability. We verify that we have close results compared to Guzmán et al.
(2019).888We got better BLEU scores due to the extra grid search. Table 5
shows that SAG + T5 outperforms Transformer on all language pairs and
directions with relative improvements between 3.6% (si-en) and 13.1% (en-ne).

| FLoRes (BLEU) |  | Noun-Adj. ordering
---|---|---|---
| ne-en | si-en | en-ne | en-si |  | BLEU | COMP
Transformer | $7.94_{\pm 0.05}$ | $7.15_{\pm 0.07}$ | $4.43_{\pm 0.01}$ | $2.32_{\pm 0.08}$ |  | $40.86_{\pm 0.34}$ | $0.64_{\pm 0.01}$
SAG + T5 | $\mathbf{8.40_{\pm 0.02}}$ | ${7.41_{\pm 0.10}}$ | $\mathbf{5.01_{\pm 0.10}}$ | $\mathbf{2.54_{\pm 0.03}}$ |  | $41.43_{\pm 0.29}$ | $\mathbf{0.73_{\pm 0.01}}$

Table 5: Models performance on FLoRes and Noun-adjective ordering (English to
French) dataset. For FLoRes, we report the BLEU dev-test scores for the
different translation directions. For the Noun-adjective ordering dataset, we
report both BLEU and COMP measures on the test set. In bold are values that
stat. sig. improve over Transformer. $\pm$ denotes 1 SEM.

Noun-adjective ordering BLEU scores on the test set are reported in Table 5.
SAG + T5 leads to a relative improvement of $1.39\%$ compared to standard
Transformer. BLEU, however, is not informative about the particular noun-
adjective generalization. We hence also report COMP scores. From Table 5, we
see that SAG + T5 demonstrates a significant improvement with $14\%$ relative
gain compared to the standard Transformer architecture. Our follow-up
experiments show that the hybrid model recovers an average of 43.3% of cases
where the best Transformer model (best seed w.r.t. COMP) failed in
compositional generalization, whereas Transformer is only correct at 21.5% of
SAG + T5’s errors. We report in Table 6 examples comparing SAG + T5 and
Transformer translations.

Discussion Upon analyzing our experiments on SCAN and machine translation
tasks, we see the following picture. Indeed the hybrid models that we
described in Section 2 have considerably higher accuracy on SCAN-jump w.r.t.
Transformer and a comparable performance on the other SCAN splits. Hence, our
results suggest the importance of both gating and (the ability of) limiting
the attention span for SCAN generalization.

As expected, the improvement on SCAN do not consistently entail improvements
on the resource-rich dataset, and only the combination of SAG and T5 showed a
tiny improvement. This emphasizes the importance of testing models on
realistic setups to model from being too SCAN-tailored.

Finally, we test SAG + T5 on low-resource and compositional tasks. The hybrid
architecture shows consistent improvements on FLoRes for all translation
directions, with at up to 13.1% relative improvement, and on the the natural
language compositional task with 14% relative improvement on COMP. Our
qualitative analysis also showed that SAG + T5 correctly handles noun-
adjective ordering in most cases, while Transformer makes more mistakes.

Target: Nous sommes face à une responsabilité politique particulière.
---
Prediction SAG+T5: Nous sommes accablés par une responsabilité politique
particulière.
Prediction Transformer: Nous sommes accablés par une responsabilité politique.
Target: Nous voulons trouver une bonne solution à ce problème.
Prediction SAG+T5: Nous voulons trouver une bonne solution à ce problème.
Prediction Transformer: Nous voulons trouver une solution adéquate à ce
problème.
Target: Ce qui nous déçoit par rapport à cette décision particulière, c’est
que le projet aurait pu clairement voir le jour.
Prediction SAG+T5: Ce qui est triste dans cette décision précise, c’est que le
projet aurait été clairement réalisé.
Prediction Transformer: Ce qui est triste dans cette mauvaise décision, c’est
que le projet aurait clairement été.
Table 6: Generation Examples for Noun-adjective ordering dataset. Models are
tested on the underlined and italic (adjective, noun). For the first 2
examples, SAG+T5 predicted the right (adjective, noun) translation. In the
last one, SAG+T5 replaced the adjective with a synonym but in the right target
order (the one not seen in the training set). In the first example,
Transformer removed the adjective particulière. In the two following examples,
Transformer replaced the right adjective with a close synonym adjective to be
conform with the training order. For instance, in the second example, bonne
(an adjective that appears in the forward order) was replaced by adéquate (an
adjective that appears in the backward order) as the solution appears only in
the backward order at training.

##  8 Related Work

Compositionally-biased models Several approaches were proposed to build SCAN-
capable architectures. They span from meta-learning Lake (2019), disentangling
syntax and semantics Russin et al. (2019), learning equivariant Gordon et al.
(2019) and disentangled representations Li et al. (2019) or combining neural &
symbolic computations Chen et al. (2020). In contrast, we do not build new
models that are specialized to SCAN. Instead, we show that a standard model
can be incrementally modified so that performs well on SCAN and still performs
well on a standard MT task. Having such incrementally improved models allows
us to step back and wonder if SCAN (or similar artificial tasks) should be
used as a guidance when developing new models.

Bastings et al. (2018) raised concerns due to SCAN being too artificial by
showing that even degenerate architectures can perform well on some SCAN
tasks. Our results echo their findings: by developing architectures tailored
for SCAN, one can easily come up with models that perform worse on general
tasks. However, we find that if one avoids this “SCAN overfitting” and endows
a model with capabilities that SCAN probes for without harming its general
performance, they can gain in low-resource scenarios and better handle
relevant phenomena in language.

Changing attention mechanisms Self- and cross-attention mechanisms were
tweaked in earlier work in order to inject useful biases, e.g., by adding
information of relative positions of tokens Shaw et al. (2018); Raffel et al.
(2019) or accounting for the locality bias in cross-attention Yang et al.
(2018). Sukhbaatar et al.  and Rae and Razavi (2020) demonstrated that having
a short attention span on the lower layers of Transformer models is enough for
good language modeling performance.

##  9 Conclusion

In this work, we primarily focused on whether and in which scenarios the
inductive bias for compositional generalization, that SCAN looks for, can be
useful in natural language tasks.

We ran study in two steps. As the first step, by exploiting
ConvS2S/Transformer similarities, we came up with a modification of the
Transformer architecture that performs considerably better than vanilla
Transformer on SCAN-jump (43.0% vs 3.4% accuracy) and performs equally well on
SCAN-simple, SCAN-around-right, and on a standard resource-rich MT task
(IWSLT’14 De-En).

Next, we tested this modification in low-resource and distribution-shifted
setups. In the low-resource MT setup (FLoRes Si/Ne$\leftrightarrow$En), we
found that our considered architecture improves by up to 13.1% in BLEU score
over the vanilla Transformer. Then, we introduced a new dataset that probes
specifically for compositional reasoning in natural language. Unlike SCAN, our
compositional dataset is built by filtering an existing natural language
corpus (EuroParl En-Fr) to probe how models perform noun-adjective ordering
under a (minimal) distribution shift. Thus, we are largely closer to testing
the compositional generalization required by humans compared to SCAN, and
succeeding on the test set requires both compositional reasoning and good
language model performance (see examples in Table 6). We believe that such a
dataset is beneficial for future research to test more complex
compositionality skills. Finally, our experiments on our dataset demonstrated
that better SCAN generalization leads to better results on noun-adjective
ordering (14% on COMP).

Our findings indicate the following. Firstly, as hypothesized before Dessì and
Baroni (2019); Hupkes et al. (2018), the limited attention span provides a
useful inductive bias that allows models to perform better on compositional
generalization induction, that SCAN probes for. Further, endowing a model with
SCAN-style generalization capabilities can lead to improvements in low-
resource and distribution-shifted scenarios as long as we ensure that we do
not overfit to SCAN.

We believe that the contribution of diagnostic datasets like SCAN is of great
value. As performance grows on tasks such as MT, identifying gaps where a
model’s performance lags will become fundamental and will guide us to develop
architectures that cover genuine new linguistic grounds and not just overfit
to peculiarities of standard datasets.

## Acknowledgments

The authors are grateful to Marco Baroni and the reviewers for feedback that
helped us to improve our work.

## References

  * Adiwardana et al. (2020) Daniel Adiwardana, Minh-Thang Luong, David R So, Jamie Hall, Noah Fiedel, Romal Thoppilan, Zi Yang, Apoorv Kulshreshtha, Gaurav Nemade, Yifeng Lu, et al. 2020\.  Towards a human-like open-domain chatbot.  _arXiv preprint arXiv:2001.09977_.
  * Ba et al. (2016) Jimmy Lei Ba, Jamie Ryan Kiros, and Geoffrey E Hinton. 2016.  Layer normalization.  _arXiv preprint arXiv:1607.06450_.
  * Bahdanau et al. (2014) Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2014.  Neural machine translation by jointly learning to align and translate.  _arXiv preprint arXiv:1409.0473_.
  * Baroni (2019) Marco Baroni. 2019.  Linguistic generalization and compositionality in modern artificial neural networks.  _Philosophical Transactions of the Royal Society B: Biological Sciences_ , 375(1791):20190307.
  * Bastings et al. (2018) Jasmijn Bastings, Marco Baroni, Jason Weston, Kyunghyun Cho, and Douwe Kiela. 2018\.  Jump to better conclusions: Scan both left and right.  In _MNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP_.
  * Chen et al. (2020) Xinyun Chen, Chen Liang, Adams Wei Yu, Dawn Song, and Denny Zhou. 2020.  Compositional generalization via neural-symbolic stack machines.
  * Cordonnier et al. (2020) Jean-Baptiste Cordonnier, Andreas Loukas, and Martin Jaggi. 2020.  On the relationship between self-attention and convolutional layers.  In _ICLR_.
  * Dankers et al. (2021) Verna Dankers, Elia Bruni, and Dieuwke Hupkes. 2021.  The paradox of the compositionality of natural language: a neural machine translation case study.
  * Dauphin et al. (2017) Yann N Dauphin, Angela Fan, Michael Auli, and David Grangier. 2017.  Language modeling with gated convolutional networks.  In _ICML_.
  * Dessì and Baroni (2019) Roberto Dessì and Marco Baroni. 2019.  CNNs found to jump around more skillfully than RNNs: Compositional generalization in seq2seq convolutional networks.  In _ACL_.
  * Futrell et al. (2015) Richard Futrell, Kyle Mahowald, and Edward Gibson. 2015.  Large-scale evidence of dependency length minimization in 37 languages.  _PNAS_ , 112(33):10336–10341.
  * Gehring et al. (2017) Jonas Gehring, Michael Auli, David Grangier, Denis Yarats, and Yann N. Dauphin. 2017\.  Convolutional sequence to sequence learning.  In _ICML_.
  * Gordon et al. (2019) Jonathan Gordon, David Lopez-Paz, Marco Baroni, and Diane Bouchacourt. 2019.  Permutation equivariant models for compositional generalization in language.  In _ICLR_.
  * Guzmán et al. (2019) Francisco Guzmán, Peng-Jen Chen, Myle Ott, Juan Pino, Guillaume Lample, Philipp Koehn, Vishrav Chaudhary, and Marc’Aurelio Ranzato. 2019.  Two new evaluation datasets for low-resource machine translation: Nepali-english and sinhala-english.
  * Hupkes et al. (2020) Dieuwke Hupkes, Verna Dankers, Mathijs Mul, and Elia Bruni. 2020.  Compositionality decomposed: How do neural networks generalise?  _Journal of Artificial Intelligence Research_ , 67:757–795.
  * Hupkes et al. (2018) Dieuwke Hupkes, Anand Singh, Kris Korrel, German Kruszewski, and Elia Bruni. 2018\.  Learning compositionally through attentive guidance.  _arXiv preprint arXiv:1805.09657_.
  * Kharitonov and Chaabouni (2021) Eugene Kharitonov and Rahma Chaabouni. 2021.  What they do when in doubt: a study of inductive biases in seq2seq learners.  In _ICLR_.
  * Kingma and Ba (2014) Diederik P Kingma and Jimmy Ba. 2014.  Adam: A method for stochastic optimization.  _arXiv preprint arXiv:1412.6980_.
  * Lake (2019) Brenden Lake. 2019.  Compositional generalization through meta sequence-to-sequence learning.  In _NeurIPS_.
  * Lake and Baroni (2018) Brenden Lake and Marco Baroni. 2018.  Generalization without systematicity: On the compositional skills of sequence-to-sequence recurrent networks.  In _ICML_.
  * Li et al. (2019) Yuanpeng Li, Liang Zhao, Jianyu Wang, and Joel Hestness. 2019.  Compositional generalization for primitive substitutions.  _arXiv preprint arXiv:1910.02612_.
  * Loula et al. (2018) Joao Loula, Marco Baroni, and Brenden M Lake. 2018.  Rearranging the familiar: Testing compositional generalization in recurrent networks.  _EMNLP 2018_.
  * Manning et al. (2014) Christopher Manning, Mihai Surdeanu, John Bauer, Jenny Finkel, Steven Bethard, and David McClosky. 2014.  The Stanford CoreNLP natural language processing toolkit.  In _ACL: System Demonstrations_.
  * McCoy et al. (2020) R Thomas McCoy, Robert Frank, and Tal Linzen. 2020.  Does syntax need to grow on trees? sources of hierarchical inductive bias in sequence-to-sequence networks.  _TACL_ , 8:125–140.
  * Ott et al. (2019) Myle Ott, Sergey Edunov, Alexei Baevski, Angela Fan, Sam Gross, Nathan Ng, David Grangier, and Michael Auli. 2019.  fairseq: A fast, extensible toolkit for sequence modeling.  In _NAACL-HLT 2019: Demonstrations_.
  * Parisotto et al. (2020) Emilio Parisotto, Francis Song, Jack Rae, Razvan Pascanu, Caglar Gulcehre, Siddhant Jayakumar, Max Jaderberg, Raphael Lopez Kaufman, Aidan Clark, Seb Noury, et al. 2020.  Stabilizing transformers for reinforcement learning.  In _ICLR_.
  * Rae and Razavi (2020) Jack Rae and Ali Razavi. 2020.  Do transformers need deep long-range memory?  In _ACL_.
  * Raffel et al. (2019) Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. 2019.  Exploring the limits of transfer learning with a unified text-to-text transformer.  _arXiv preprint arXiv:1910.10683_.
  * Russin et al. (2019) Jake Russin, Jason Jo, Randall C O’Reilly, and Yoshua Bengio. 2019.  Compositional generalization in a deep seq2seq model by separating syntax and semantics.  _arXiv preprint arXiv:1904.09708_.
  * Salimans and Kingma (2016) Tim Salimans and Durk P Kingma. 2016.  Weight normalization: A simple reparameterization to accelerate training of deep neural networks.  In _NeurIPS_.
  * Shaw et al. (2018) Peter Shaw, Jakob Uszkoreit, and Ashish Vaswani. 2018.  Self-attention with relative position representations.  In _NAACL_ , pages 464–468.
  * (32) Sainbayar Sukhbaatar, Édouard Grave, Piotr Bojanowski, and Armand Joulin.  Adaptive attention span in transformers.
  * Sutskever et al. (2013) Ilya Sutskever, James Martens, George Dahl, and Geoffrey Hinton. 2013.  On the importance of initialization and momentum in deep learning.  In _ICML_.
  * Sutskever et al. (2014) Ilya Sutskever, Oriol Vinyals, and Quoc V Le. 2014.  Sequence to sequence learning with neural networks.  In _NeurIPS_.
  * Yang et al. (2018) Baosong Yang, Zhaopeng Tu, Derek F Wong, Fandong Meng, Lidia S Chao, and Tong Zhang. 2018.  Modeling localness for self-attention networks.  _arXiv preprint arXiv:1810.10182_.

##  Appendix A Hyperparameter grids

###  A.1 SCAN

For each architecture, we used the same hyperparameter grids for all splits of
SCAN.

All models were trained by Adam with default $\beta_{1}$ and $\beta_{2}$
parameters, for 250 epochs, with batch size 256, learning rate $5\cdot
10^{-4}$, dropout and attention dropout 0.1, random seeds $\\{0,1,2\\}$. We
vary the Encoder and Decoder parameters independently: number of attention
heads $\\{4,8\\}$, embedding dimensions $\\{128,256\\}$, FFN dimensions
$\\{256,512\\}$, and the number of layers $\\{4,6,8\\}$; clip norm 1.0.

For hybrids models, add the following parameters.

#### SAG

To reduce the search space, we did not vary $\beta_{0}$, setting it to $-1$.

#### SAG + CNN as self-attention

Span: $\\{2,4,6\\}$, $\beta_{0}=-1$, number of layers: {4, 6}.

#### SAG + fixed span self-attention

Span: $\\{2,4,6\\}$, $\beta_{0}=-1$, number of layers: 4.

#### SAG + T5

Span: $\\{2,4,6\\}$, $\beta_{0}=-1$, number of layers: 4.

###  A.2 Machine Translation

#### De-En

We start from the standard architecture suggested by Fairseq examples for
IWSLT’14 De-En. That is we share decoder input and output embeddings. Both
Encoder and decoder have an embedding size of 512 FFN dimensions of 1024, 4
attention heads, 6 encoder & decoder layers. We used adam optimizer with
learning rate of 5e-4, no clip norm, warm-up learning rate 1e-7, inverse
square root learning rate scheduler, 4000 warm-up updates, dropout 0.3, weight
decay 1e-4, label smoothing 0.1, max. tokens per batch per GPU: 1024, 40
epochs. We used 4 GPUs for training.

For SAG-enabled architectures, we additionally searched for Encoder’s
$\beta_{0}$ in $\\{-1,0\\}$ and $\\{1,0\\}$ for Decoder. We varied attention
span in $\\{2,4,6\\}$, ttention dropout in $\\{0.0,0.2\\}$ and pre-block
encoder and decoder normalization.

For model selection, we also follow Fairseq example and checkpoint the best
model on validation set based on BLEU score. BLEU score is computed with a
beam of size 5.

#### FLoRes

We used shared embeddings between Encoder and Decoder, embedding dimenions of
512, FFN dimensions of 2048, 2 attention heads, 5 encoder & decoder layers. We
used pre-block normalization999\--encoder-normalize-before and \--decoder-
normalize-before in fairseq., learning rate of 1e-3, no clip norm, warm-up
learning rate 1e-7, inverse square root learning rate scheduler, 4000 warm-up
updates, dropout 0.4, activation dropout 0.2, weight decay 1e-4, label
smoothing 0.2, max. tokens per batch per GPU: 4000, 100 epochs. We searched
for attention dropout in $\\{0.0,0.2\\}$. We used 4 GPUs for training.

For SAG-enabled arcchitectures, we additionally searched for Encoder’s
$\beta_{0}$ in $\\{-2,-1,0\\}$ and $\\{2,1,0\\}$ for Decoder. We varied
attention span in $\\{2,4,6\\}$.

#### Noun-adjective order agreement

We start from the standard architecture suggested by Fairseq examples for
WMT’14 En-Fr. That is we share encoder, decoder and output embeddings. Both
Encoder and decoder have an embedding size of 1024 FFN dimensions of 4096, 16
attention heads, 6 encoder & decoder layers. We used adam optimizer with
learning rate of 7e-4, no clip norm, warm-up learning rate 1e-7, inverse
square root learning rate scheduler, 4000 warm-up updates, dropout 0.1, label
smoothing 0.1, max. tokens per batch per GPU: 4000, 30 epochs. We used 6 GPUs
for training.

For SAG-enabled architectures, we additionally searched for Encoder’s
$\beta_{0}$ in $\\{-1,0\\}$ and $\\{1,0\\}$ for Decoder. We varied attention
span in $\\{2,4,6\\}$, ttention dropout in $\\{0.0,0.2\\}$ and pre-block
encoder and decoder normalization.

Best checkpoint is based on the loss of the validation set.

##  Appendix B Other modifications on FloRes and Noun-adjective ordering
datasets

In the main paper, we only experimented with SAG + T5 as the only non-degraded
modification on the IWSLT’14 En-De dataset. Our intuition is that the
remaining hybrid models are SCAN-tailored and would not lead to any
improvement in the low-resource (FloRes) and domain-shifted (Noun-adjective
ordering dataset) settings. In this section, we verify our intuition and
report the results of all the introduced variants. The hyper-parameters search
is reported in Section A.

#### FloRes

We report results on English$\leftrightarrow$Nepali and
English$\leftrightarrow$Sinhala translation in Table 7. We also report Guzmán
et al. (2019) results under “Baseline”.

Analyzing results of SAG, we notice that it is usually very close to
Transformer’s results on all tasks, apart from Nepali$\rightarrow$English,
where it lags behind. The fixed-span modification performs worse than
Transformer on in all directions. Replacing self-attention with convolutions
results in better scores on En$\rightarrow$Ne and worse scores on
Ne$\rightarrow$En/En$\rightarrow$Si.

Hence, as expected, only the SAG + T5 model outperforms Transformer on all
language pairs and directions, highlighting the importance of verifying the
generality of the model on realistic datasets.

| Baseline | Transformer | SAG | \+ Conv s.-a. | \+ fixed span | \+ T5
---|---|---|---|---|---|---
ne-en | 7.6 | $7.94_{\pm 0.05}$ | $7.58_{\pm 0.06}$ | $7.59_{\pm 0.02}$ | $7.44_{\pm 0.08}$ | $\mathbf{8.40_{\pm 0.02}}$
si-en | 7.2 | $7.15_{\pm 0.07}$ | $7.14_{\pm 0.10}$ | $7.18_{\pm 0.10}$ | $.78_{\pm 0.07}$ | ${7.41_{\pm 0.10}}$
en-ne | 4.3 | $4.43_{\pm 0.01}$ | $4.36_{\pm 0.08}$ | $\mathbf{4.63_{\pm 0.03}}$ | $4.12_{\pm 0.05}$ | $\mathbf{5.01_{\pm 0.10}}$
en-si | 1.2 | $2.32_{\pm 0.08}$ | $2.37_{\pm 0.10}$ | $2.14_{\pm 0.03}$ | $2.12_{\pm 0.04}$ | $\mathbf{2.54_{\pm 0.03}}$

Table 7: BLEU dev-test scores on FLoRes. Baseline scores are taken from Guzmán
et al. (2019). In bold are values that stat. sig. improve over Transformer
($p<10^{-3}$). $\pm$ indicates 1 SEM.

#### Noun-adjective order agreement

BLEU scores on the test set are reported in Table 8. SAG leads to a relative
improvement of $1.44\%$ compared to standard Transformer, closely followed by
SAG + T5. Still, in total, the differences are very small across all models.
On the other hand, all introduced variants outperform standard Transformer on
COMP. However, only SAG + T5 demonstrates a significant improvement with
$14\%$ relative gain.

Overall, we observe that the SCAN-tailored variants do not degrade
performances on the Noun-adjective order agreement dataset, but still do not
lead to any significant improvement, contrary to SAG + T5.

| Transformer | SAG | \+ Conv s.-a. | \+ fixed span | \+ T5
---|---|---|---|---|---
BLEU | $40.86_{\pm 0.34}$ | $41.45_{\pm 0.14}$ | $39.89_{\pm 0.27}$ | $41.01_{\pm 0.24}$ | $41.43_{\pm 0.29}$
COMP | $0.64_{\pm 0.01}$ | $0.70_{\pm 0.03}$ | $0.67_{\pm 0.01}$ | $0.68_{\pm 0.01}$ | $\mathbf{0.73_{\pm 0.01}}$

Table 8: BLEU and COMP measures on test sets: compositional English to French
dataset. In bold are values that stat. sig. improve over Transformer
($p<0.05)$. $\pm$ denotes 1 SEM.

##  Appendix C Visualizing attention biases

In this Section, we illustrate how a successful SAG + T5 model uses its
$b_{ij}$ terms (Eq. 1) to control its attention.

We take the most successful hyperparameter combination on SCAN-jump in Table 3
and select a model instance that has the best accuracy ($\approx$ 60%). Next,
for each attention head of each encoder and decoder layer, we retrieve its
learned relative-position bias $b_{d}$, where $d$ is a (signed) relative
distance between positions $i$ and $j$, that is capped to be within $[-s,+s]$
(see Section 3). For each head, we apply a softmax, to find its “preference”
$\hat{b}_{d}$ over relative positions $d$:

$\hat{b}_{d}=\frac{\exp(b_{d})}{\sum_{d}\exp(b_{d})}$

We report the results in Figure 1. Interestingly, quite a few attention heads
have very strong preferences for fixed relative positions and some are even
dominantly focused on particular positions (Encoder: head 7 in the layer 0;
heads 4, 5 in layer 1, heads 3, 7 in layer 2; head 2 in layer 3; Decoder: head
4 in layer 0, head 2 in layer 1, heads 3,4,5 in layer 2; heads 2, 6, 7 in
layer 3)101010T5 reduces to the vanilla Transformer if all $b_{d}$ are equal
to zero. That corresponds to the uniform bias $\hat{b}_{d}$.. More often than
not, those “specialized” heads look within the span and not on the “border”
values of $d$ (due to $d$ being capped, they also correspond to arbitrary
distant positions to the left and right).

Hence we conclude that in a T5 model (among most successful on SCAN-jump),
several heads leverage the ability pay attention locally; supporting our
finding that local attention is connected with the compositional
generalization needed to succeed at SCAN. At the same time, some heads have
large relative-position bias for distant positions ($[s,+\infty[$ or
$]-\infty,-s]$). This general ability to optionally look beyond a fixed span
in T5 could be responsible for its better performance compared to the fixed
span modification.

(a) Encoder layer 0.

(b) Encoder layer 1.

(c) Encoder layer 2.

(d) Encoder layer 3.

(e) Decoder layer 0.

(f) Decoder layer 1.

(g) Decoder layer 2.

(h) Decoder layer 3.

Figure 1: Relative attention biases for T5 + SAG architecture (after a
softmax). Each cell indicates preference of a head to a position at a signed
relative distance. The relative distances are capped. For the decoder we only
represent relative attention biases for $d\leq 0$, as positions with positive
relative distance are masked in the autoregressive decoder.


††thanks: These two authors contributed equally.††thanks: These two authors
contributed equally.††thanks: e-mail<EMAIL_ADDRESS>
# Supplementary Information:
Escape dynamics of active particles in multistable potentials

A. Militaru  Photonics Laboratory, ETH Zurich, CH-8093 Zurich, Switzerland  M.
Innerbichler  Faculty of Physics, University of Vienna, 1090 Wien, Austria  M.
Frimmer  Photonics Laboratory, ETH Zurich, CH-8093 Zurich, Switzerland  F.
Tebbenjohanns  Photonics Laboratory, ETH Zurich, CH-8093 Zurich, Switzerland
L. Novotny  Photonics Laboratory, ETH Zurich, CH-8093 Zurich, Switzerland  C.
Dellago  Faculty of Physics, University of Vienna, 1090 Wien, Austria

main

##  I Extraction of Rate Coefficients

Let us explicitly consider a two-state system characterized by the
concentrations/population fractions $c_{A}$ and $c_{B}$. In the context of
single particle systems one can define $c_{A}(t)$ via an ensemble of a large
number of replicas. Then, $c_{A}(t)$ corresponds to the fraction of replicas
in state $A$ at time $t$. Subjected to reactive dynamics, $c_{A}$ evolves
according to the rate equation

$\dot{c}_{A}=-k_{AB}c_{A}+k_{BA}c_{B}=k_{BA}-(k_{AB}+k_{BA})c_{A}.$ (S1)

The rate constants $k_{AB}$ and $k_{BA}$ respectively prescribe the speed of
the reaction from $A$ to $B$ and from $B$ to $A$. Population/particle
conservation enforces a constraint of the form $\sum_{X}c_{X}=1$, used above
to arrive at the rightmost expression. It is straightforward to determine the
solution as

$c_{A}(t)=c_{A}(0)e^{-(k_{AB}+k_{BA})t}+\frac{k_{BA}}{k_{AB}+k_{BA}}\left[1-e^{-(k_{AB}+k_{BA})t}\right].$
(S2)

Hence, the populations approach their equilibrium values $c_{A,eq}$ and
$c_{B,eq}$ exponentially with a decay-time controlled by the sum of the rate
constants $k=k_{AB}+k_{BA}$. We underline this point by reformulating the
above equation in terms of the equilibrium/stationary state concentration
$c_{A,eq}$

$c_{A}(t)=\left[c_{A}(0)-c_{A,eq}\right]e^{-kt}+c_{A,eq},$ (S3)

where the equilibrium population is given by
$c_{A,eq}=k_{BA}/(k_{AB}+k_{BA})$. The rate constants can therefore be
extracted from the exponential decay/growth of the concentration. For the
initial condition $c_{A}(0)=1$, the population $c_{A}(t)$ can be interpreted
as the probability of finding the particle in well A at time t provided it was
located there at time $t=0$. Contact between the phenomenological rate
equation (S1) and the experimental time traces of the system can be made by
considering the indicator function $h_{A}$, which is a function equal to unity
inside well $A$ and zero otherwise. Then, the average $\left\langle
h_{A}(t)\right\rangle$ of the indicator function corresponds to the population
$c_{A}(t)$.

In terms of the indicator function $h_{A}$, the conditional probability to
find the system in $A$ at time $t$, provided it was there at time $t_{0}=0$,
can be expressed using the time correlation function of the population as
$\left\langle h_{A}(0)h_{A}(t)\right\rangle/\left\langle h_{A}\right\rangle$.
Here, the average is taken over a long (equilibrium/stationary state)
trajectory. Assuming that for sufficiently long times the kinetics of the
system is described by the rate equation, one obtains

$\displaystyle\left\langle h_{A}(0)h_{A}(t)\right\rangle$
$\displaystyle=c_{A,eq}(1-c_{A,eq})e^{-kt}+c_{A,eq}^{2},$ (S4)
$\displaystyle\left\langle\Delta h_{A}(0)\Delta h_{A}(t)\right\rangle$
$\displaystyle=\left\langle\Delta h_{A}(0)^{2}\right\rangle e^{-kt},$ (S5)

where $\Delta h_{A}$ designates $h_{A}-\left\langle h_{A}\right\rangle$ and we
took advantage of the relations $\left\langle h_{A}\right\rangle=c_{A,eq}$ and
$h_{A}^{2}=h_{A}$. Consequently, one can extract the rate coefficient $k$ by
performing a single parameter fit to the (normalized) autocorrelation
function. In reality, however, the rate equation only represents a convenient
simplification of the system at hand and deviations from purely exponential
decays are expected, for instance due to short-time correlations. In order to
avoid including short-time correlations in the rate estimations, we fit
equation (S4) only over time scales where their effect has disappeared.

##  II Activity and Effective Barrier Height

To supplement the experimental findings delineated in the main part of the
document, we provide additional context by studying the transition dynamics
theoretically and computationally. For clarity, we direct our focus mainly on
the phenomenology and select a particularly simple and frequently used form of
the potential, i.e.

$U(x)=h\left(1-\frac{x^{2}}{\sigma^{2}}\right)^{2}.$ (S6)

The parameter $h$ gives the height of the barrier and the minima of the
symmetric potential are located at $\pm\sigma$. All quantities with dimensions
of inverse time are subsequently given in terms of
$k_{0}=t_{0}^{-1}=\sqrt{k_{\text{B}}T/m\sigma^{2}}$. We investigate the shape
of the resulting transition rate landscape as function of the translational
damping $\Gamma$ and rotational diffusivity $D_{R}$. We generate and analyse
individual long trajectories for each set of $\Gamma$ and $D_{R}$ on a
logarithmically spaced grid, whilst all other parameters remain fixed. Further
details about the simulation are delineated in Sec. V.

(a)  (b)

(c)  (d)

Figure S1: Transition rate as a function of the translational damping and
rotational diffusion for four activity magnitudes
$A\sigma/k_{\text{B}}T=0.25,\,1.00,\,4.00,\,16.0$. As $A$ increases the active
turnover appears and becomes increasingly more prominent, whereas the Kramers
turnover retains its magnitude. Simulation parameters: barrier height
$h=4k_{\text{B}}T$, time step $${10}^{-2}$t_{0}$, trajectory length per data
point $T=$2\text{\times}{10}^{4}$t_{0}$.

Some illustrative examples of the resulting rate landscapes with various
activity values are depicted in Fig. S1. Starting our discussion from low
(almost vanishing) activity, one observes the standard Kramers turnover only.
As one would expect in this case, the transition rate rises to its maximum at
some moderate value of $\Gamma$ and remains virtually unaffected by the active
contributions. Stronger activities, however, lead to data exhibiting some
interesting features: a second line of maxima appears in the direction of the
rotational diffusion axis $D_{R}$. This line seems to stop abruptly at the
position of the standard Kramers turnover. Further increasing $A$ results in
an evermore prominent active turnover, completely superseding its original
counterpart at its horizontal position. The Kramers turnover has not
disappeared, yet its magnitude is far too small to be seen on a linear scale.
The newly found active turnover is unable to push past the limit imposed by
the passive one and disappears gradually as the damping is increased. For even
higher $A$, it becomes possible for the active turnover to stretch farther
into the overdamped regime, however, simultaneously becoming more prominent at
lower $\Gamma$ values.

We have provided an intuitive motivation of the novel features arising due to
active propulsion in the main text. In the following, we proceed to
investigate the active turnover more quantitatively. Following Kramers’
example, we demonstrate the existence of a turnover by examining the limits of
high and low rotational diffusion individually. Instead of directly jumping to
these limiting scenarios, however, it is instructive to discuss the activity’s
non-equilibrium nature.

Because it continuously feeds energy into the system, active propulsion
constitutes an example of a non-conservative force. As such, the system’s
stationary state $p_{s}(x,v,\varphi)$ is not described by the Boltzmann
distribution. The stationary state becomes explicitly dependent on the damping
and rotational diffusion, distinctly setting the underlying physics apart from
the properties generating the Kramers turnover: In conservative settings, the
rate exhibits a maximum at moderate friction due to purely dynamical effects,
while the probability density in phase space is independent of the damping
$\Gamma$.

(a)  (b)

Figure S2: Left: Free energy barrier height $\beta\Delta F$ as a function of
rotational diffusivity $D_{R}$. Right: Transition rate $k$ as a function of
free energy barrier height $\beta\Delta F$. The data points stem from
trajectory simulations in the quartic potential of equation (S6) with barrier
height $h=6k_{\text{B}}T$, activity $A=3k_{\text{B}}T/\sigma$ and at a damping
of $\Gamma=0.2k_{0}$. The trajectory length per point equals
$T=${10}^{6}$t_{0}$. The showcased data encompass two separate sets of
simulations for the high and low $D_{R}$ regime only differing in the size of
time step $\Delta t$. We used $\Delta t=0.01t_{0}$ for the low $D_{R}$ part
(blue) and $\Delta t=0.001t_{0}$ for the fast rotation section (orange), as a
finer discretization is needed to thoroughly resolve the evolution of the
angle. The two gray lines accompanying the generalized Arrhenius plot indicate
the slope expected theoretically. A switch in the rate’s prefactor seems to
emerge around $4k_{\text{B}}T$ \- about a factor two in magnitude - not
attributable to stationary effects. Nonetheless, the data’s general
consistency with the theoretical approximation points to the barrier height’s
central role in determining $k$.

While usually introduced in equilibrium thermodynamics and statistical
physics, it is useful to define the free energy $F$ for non-conservative
settings analogously as the negative natural logarithm of the stationary state
via $\beta F=-\ln p_{s}(x,v,\varphi)$ Sivak and Crooks (2012). Adopting the
view of transition state theory Dellago and Bolhuis (2009), the rate depends
exponentially on the height of the free energy barrier $\Delta F$, i.e. it is
proportional to the probability to reside atop the barrier

$k\propto e^{-\beta\Delta F}.$ (S7)

Even small modifications of $\Delta F$ affect the rate landscape considerably.
Studying the variation of $\Delta F$ as a function of $A$, $\Gamma$, and
$D_{R}$ leads to a more quantitative understanding of the emergence of the
active turnover. The Arrhenius plot depicted in Fig. S2 serves to establish
$\beta\Delta F$ as the key quantity in shaping the activity-dominated part of
the rate landscape. The fact that only comparatively small deviations from the
postulated exponential decay arise suggests that most relevant effects are
indeed summarized into this single measure. This circumstance constitutes the
basis of the fairly simple quantification approaches laid out below.

Our physical model is described by a set of Langevin equations. Therefore the
evolution of its phase space density $p(x,v,\varphi,t)$ is governed by the
associated Fokker-Planck equation (we set the mass to unity for notational
simplicity)

$\partial_{t}p(x,v,\varphi)=-v\partial_{x}p+\partial_{v}(\Gamma
v+U^{\prime}(x)-A\cos\varphi)p+D\partial_{vv}^{2}p+D_{R}\partial_{\varphi\varphi}^{2}p.$
(S8)

The stationary distribution $p_{s}$ and free energy $F$ are in principle
obtained by imposing $\partial_{t}p=0$. Unfortunately, as of now a closed form
solution of the resulting FPE remains elusive, forcing us to resort to
numerical or approximate treatments. The upcoming derivations for the high and
low $D_{R}$ limits are inherently based on the FPE, although some aspects are
understood more easily in the Langevin picture. The theories successfully
reproduce the transition rates obtained computationally, capturing the
essential aspects to affect the free energy barrier height.

##  III High Rotational Diffusion Limit

Let us first consider the limit of large $D_{R}$, in which the persistence
time of the active force is far shorter than all remaining physical
timescales. This effectively implies that before the particle changes its
position in phase space by an appreciable amount, the direction of its active
propulsion completely decorrelates, allowing it to perform several stochastic
rotations on the spot. Therefore, at the timescale of physical motion the
active noise behaves as if subsequent realizations were statistically
independent, effectively putting it into the same memory class as white noise.
Based on this reasoning, a modification of the diffusion constant or,
equivalently, the introduction of an effective temperature provides an
adequate approximation of the impact of the active force on the system. To
justify this approximation quantitatively, let us quickly consider the limit
$D_{R}\to\infty$. For an orientation changing infinitely fast, the angle
distribution must be uniform at each point in reduced phase space $(x,v)$. The
effective dynamics in reduced phase space then results from replacing all
$\varphi$-dependent terms with their local averages. The expectation value of
the active force $A\cos\varphi$ vanishes given a uniformly distributed angle,
leading us to the same equation of motion as in the absence of activity:

$\dot{v}=-\Gamma v-U^{\prime}(x)+\sqrt{2m\Gamma k_{\text{B}}T}\eta(t).$ (S9)

Next, we consider the equation of motion at the translational timescale for a
finite (but still very high) rotational diffusion. We emphasise that for fast
rotational diffusion it is possible to choose a duration $\Delta t$ that
allows for substantial changes in $\varphi$ whilst keeping $x$ and $v$
effectively constant. The deterministic displacement in $x$ and $v$ throughout
the time step $\Delta t$ should then be virtually unaffected by the details of
the noise history in-between. This allows us to investigate the displacements
attributed to the (active) noise terms without the presence of any
deterministic drifts due to potential forces and friction. Our argument should
even work in the presence of multiplicative noise as the current phase point,
for all intents and purposes, turns into a parameter.

The total velocity displacement due to stochastic forces $\Delta_{s}v$ over
$\Delta t$ can be expressed as

$\Delta_{s}v=\int_{0}^{\Delta
t}dt\left(A\cos\varphi(t)+\sqrt{2D}\eta(t)\right).$ (S10)

We consider velocity displacements because the active turnover resides
primarily in the underdamped regime. The present argumentation can be applied
to the overdamped approximation as well. We will do so when studying its
disappearance for increasing $\Gamma$. We compute the mean square displacement
associated with equation (S10) to obtain

$\displaystyle\left\langle\Delta_{s}v^{2}\right\rangle$
$\displaystyle=\left\langle\int_{0}^{\Delta t}dt\int_{0}^{\Delta
t}dt^{\prime}\left(A\cos\varphi(t)+\sqrt{2D}\eta(t)\right)\left(A\cos\varphi(t^{\prime})+\sqrt{2D}\eta(t^{\prime})\right)\right\rangle=$
$\displaystyle=\int_{0}^{\Delta t}dt\int_{0}^{\Delta
t}dt^{\prime}A^{2}\left\langle\cos\varphi(t)\cos\varphi(t^{\prime})\right\rangle+2D\left\langle\eta(t)\eta(t^{\prime})\right\rangle=$
$\displaystyle=\int_{0}^{\Delta t}dt\int_{0}^{\Delta
t}dt^{\prime}\left(\frac{A^{2}}{2}e^{-D_{R}|t-t^{\prime}|}+2D\delta(t-t^{\prime})\right).$
(S11)

The expectation values of the mixed terms disappear due to statistical
independence, splitting them into a product of expectation values equal to
zero. The autocorrelation of the white noise yields a Dirac-delta by
definition and the cosine term is derived in Sec. VI. The final result reads

$\displaystyle\left\langle\Delta_{s}v^{2}\right\rangle$
$\displaystyle=2D\Delta t+\frac{A^{2}}{D_{R}}\left(\Delta
t+\frac{1}{D_{R}}\left(e^{-D_{R}\Delta t}-1\right)\right)=$
$\displaystyle=2\left(D+\frac{A^{2}}{2D_{R}}\right)\Delta
t+\mathcal{O}(D_{R}^{-2}).$ (S12)

Comparing equation (III) with inactive dynamics, we obtain an effective
translational diffusion equal to $D_{\text{eff}}=D+A^{2}/(2D_{R})$, hereby
neglecting the small terms of order $D_{R}^{-2}$. These higher order terms
emerge from the fact that the active noise is inherently not a Gaussian white
noise even in the limit considered. For instance, white noise can induce
arbitrarily large displacements throughout $\Delta t$, whereas active noise
has an upper bound in magnitude at $A\Delta t$. Nonetheless, these differences
become negligible for very high $D_{R}$. In terms of an effective temperature
one obtains (now including the mass for completeness)

$D_{\text{eff}}=m\Gamma k_{\text{B}}T_{\text{eff}}=m\Gamma
k_{\text{B}}T\left(1+\frac{A^{2}}{2m\Gamma k_{\text{B}}TD_{R}}\right).$ (S13)

Attributing all relevant modifications between the active and inactive
stationary distribution to this effective temperature leads us to the relation

$\beta\Delta F=\frac{\Delta U}{k_{\text{B}}T_{\text{eff}}},$ (S14)

where $\Delta U$ denotes the energy barrier of the trapping potential.

At this point we are able to approximate the high rotational diffusion part of
the landscape with a single parameter fit of the form

$k(D_{R})=c_{0}e^{-\Delta U/k_{\text{B}}T_{\text{eff}}(D_{R})}.$ (S15)

(a)  (b)

Figure S3: Average kinetic energy and transition rate constant as a function
of the rotational diffusion. The parameters used are the same as those of Fig.
S2. The solid lines represent the theoretical approximations of equation
(S13), with the approximation for the kinetic energy being parameter-free. The
solid curve on the right is fitted to the range of high $D_{R}$ only,
amounting to a vertical displacement on the logarithmic scale. Both
theoretical predictions are consistent with the data within their designated
regime. Fit parameter: $c_{0}=0.648k_{0}$.

The applicability of our findings is highlighted in Fig. (S3). The average
kinetic energy $K$ shown is related to the effective temperature via
$K=k_{\text{B}}T_{\text{eff}}/2$ and is approximated extremely well by
equation (S13). It remains quite consistent even in close proximity to the
active turnover, where the assumptions of our simplification no longer apply.
A similar quality is achieved in the parameter-fit showcased by the right
panel of Fig. (S3). The $D_{R}$ dependence beyond the active turnover seems
well-explained by an Arrhenius-like estimate along with the effective
temperature provided.

### Termination in the Overdamped Regime

The termination of the active turnover at higher dampings $\Gamma$ shown in
Fig. S1 can be derived in a similar vein to what we discussed in the previous
section. The central premise leading to the activity’s inclusion in the
diffusive term consists of the persistence time being far smaller than the
translational timescale. Whether this goal is achieved by speeding up the
rotation or inhibiting translational movement is irrelevant. Increasing the
friction serves to gradually arrest the dynamics and we end up with the same
result as before even without considering the Langevin equation’s overdamped
limit explicitly. In what follows, the diffusivity still pertains to velocity
and not to the position ($D=m\Gamma k_{\text{B}}T$). The expression for the
effective temperature

$k_{\text{B}}T_{\text{eff}}=k_{\text{B}}T\left(1+\frac{A^{2}}{2m\Gamma
k_{\text{B}}TD_{R}}\right)$ (S16)

gives a quantitative estimate about the influence of the active forces on the
overall rate landscape at high $\Gamma$, gradually decaying as $A^{2}/\Gamma$
approaches zero. Irrespective of the activity $A$, one can always select a
sufficiently high damping for the active turnover to effectively disappear. On
the other hand raising $A$ necessitates a quadratic increase of $\Gamma$ to
suppress its effects to the same extent in this approximation. This aspect
supports the further expansion of the active turnover into the overdamped
regime with increasing $A$ as well as the rate’s overall increase, clearly
visible in the final panel of Fig. S1.

##  IV Low Rotational Diffusion Limit

### Infinitely Slow Rotation

In the extreme case in which the rotational diffusion is infinitesimal, we can
consider the angle $\varphi$ as a parameter and the active force as a
modification of the potential equal to $-Ax\cos\varphi$. This new, effective
potential exhibits different left and right barrier heights that are functions
of $\varphi$ \- we denote them with $\Delta U_{L}(\varphi)$ and $\Delta
U_{R}(\varphi)$ for the left and right well respectively. Analogously, we
define the left and right transition rate as $k_{L}(\varphi)$ and
$k_{R}(\varphi)$.

First of all, let us introduce an expression for the effective transition rate
derived from an ensemble containing all possible modified potentials: For a
fixed angle the probability to stay in the left well decays per definition
like $e^{-k_{L}(\varphi)t}$. The long-time decay of a cumulative measure or
population that has components individually decaying exponentially is
dominated by the slowest rate present. Nonetheless, we need to extract an
effective rate that circumvents addressing these individual components
directly - a point of view that summarizes all potential tilts into a single
rate equation.

To this end, consider the average transition probability $p_{L,\text{cm}}$ at
low waiting times $\Delta t$ (i.e. the probability to observe a jump between
wells within this duration)

$p_{L,\text{cm}}=\int_{0}^{2\pi}d\varphi\;\left(1-e^{-k_{L}(\varphi)\Delta
t}\right)p_{L}(\varphi)=:1-e^{-k_{\text{eff}}\Delta t},$ (S17)

where $p_{L}$ denotes the probability density to find the particle in
orientation $\varphi$ under the constraint that it currently resides in the
left well. The integral introduces an effective rate $k_{\text{eff}}$ that
generally depends on $\Delta t$. Nonetheless, at $\Delta t$ much smaller than
the multiplicative inverse of all rates present, one can conveniently expand
the above expression to obtain

$k_{\text{eff}}=\int_{0}^{2\pi}d\varphi\;k_{L}(\varphi)p_{L}(\varphi),$ (S18)

equalling the standard expectation value. The short time behaviour (i.e.,
shorter than any rate and of course diffusion timescales) of the left and
right well concentrations is then consistently represented by the original ODE
as

$\dot{c}_{L}=-k_{L,\text{eff}\;}c_{L}+k_{R,\text{eff}\;}c_{R}.$ (S19)

In the case of infinitely slow rotational diffusion we are in the fortunate
position to know a closed form of the marginal angular distribution. Under
this condition each angle can be treated as a separate case, allowing us to
recycle the result shown in Eq. (S2). Up to a normalization factor it reads

$p_{L}(\varphi)\propto\dfrac{k_{R}(\varphi)}{k_{L}(\varphi)+k_{R}(\varphi)}.$
(S20)

In its current form, our result is unfortunately independent of $D_{R}$. In
the next section we introduce a straightforward variant of the above model
that amends this shortcoming and discuss the conditions for its applicability.

### Reaction-Diffusion Model

Rate equations are typically not concerned with details about the velocity and
position beyond classifying the current state, allowing us to discard spurious
details. We can thus integrate the irrelevant variables over the phase space
portion corresponding to the state. Proceeding with this line of thought
transforms the FPE into a reaction-diffusion equation as shown in the
following.

Without loss of generality, we place the boundary between the two wells at
$x=0$ and define the left-well population as

$\displaystyle
c_{L}(\varphi,t):=\int_{-\infty}^{\infty}dv\;\int_{-\infty}^{0}dx\;p(x,v,\varphi,t).$
(S21)

We apply the same integrations to the FPE in (S8) to obtain:

$\displaystyle\partial_{t}c_{L}$
$\displaystyle=D_{R}\partial_{\varphi\varphi}^{2}c_{L}+\int_{-\infty}^{\infty}dv\;\int_{-\infty}^{0}dx\;\left[-v\partial_{x}p+\partial_{v}(\Gamma
v+U^{\prime}(x)-A\cos\varphi)p+D\partial_{vv}^{2}p\right]=$
$\displaystyle=D_{R}\partial_{\varphi\varphi}^{2}c_{L}-\int_{-\infty}^{\infty}dv\;vp(x=0,v,\varphi,t).$
(S22)

Under the reasonable assumption that the probability distribution decays
faster than any linear (typically polynomial) function, all terms containing a
derivative of the velocity drop out. A similar idea was applied to the lower
limit of the position integral. Alternatively, one can assume a
confined/compact initial condition, which generates a time-dependent
distribution that vanishes at infinity anyhow. Our expressions are still exact
at this point under these conditions. We shall now proceed to apply physically
motivated approximations for the remaining integral term of equation (IV),
which describes the net probability flux through the dividing surface.

We exclude stochastic recrossings where the particle velocity changes its sign
in the immediate proximity of the dividing surface. These cover a very small
fraction of all reactive paths and contribute practically nothing to the
integral for one of two reasons: Either the velocity multiplied to the local
distribution is already very small or the sign reversal becomes extremely
unlikely. From this standpoint, almost all instances of negative velocities on
the barrier come from the right well and vice versa, splitting the integral
into two terms associated with the particle origin and thus proportional to
the respective local concentration $c_{L}$ or $c_{R}$.

In the next step, we use the fact that since the rotational diffusion is
small, one can treat the angle as constant at least over the duration of the
average transition path time. The probability to visit the barrier top is then
approximately proportional to the Boltzmann factor of the effective potential
including its linear tilt due to active propulsion
$U_{\text{eff}}(x,\varphi)=U(x)-Ax\cos\varphi$. In the context of this
conservative-like dynamics, the velocity part of the distribution should
factorize as usual and appear in terms of the kinetic energy. The velocity
integral thus becomes an additional prefactor, similar to the normalization of
the local Boltzmann distribution. Therefore we arrive at

$\displaystyle\partial_{t}c_{L}(\varphi,t)$
$\displaystyle=D_{R}\partial_{\varphi\varphi}^{2}c_{L}-z_{L}e^{-\beta\Delta
U_{\text{eff},L}}c_{L}+z_{R}e^{-\beta\Delta U_{\text{eff},R}}c_{R}$
$\displaystyle=D_{R}\partial_{\varphi\varphi}^{2}c_{L}-k_{L}(\varphi)c_{L}+k_{R}(\varphi)c_{R}.$
(S23)

The velocity parts and normalizations are absorbed into the dynamic prefactors
$z_{L},z_{R}$. In a symmetric potential these quantities become equivalent and
once again constitute the only parameter(s) of the model. Together with the
Boltzmann factor we can identify their product as angle-dependent rates,
resulting in a reaction diffusion equation much simpler than the original FPE
as it depends on only two variables instead of the original four.

Deriving the analogous expression for the right well and using the constraint
of an overall uniform marginal angle distribution in the interval $[0,2\pi)$,
we end up with a single equation governing the stationary state

$D_{R}\partial_{\varphi\varphi}^{2}c_{L}=[k_{L}(\varphi)+k_{R}(\varphi)]c_{L}-\frac{1}{2\pi}k_{R}(\varphi).$
(S24)

In presence of the boundary conditions of positivity and periodicity, this
ordinary linear differential equation has, in principle, a unique solution. It
can even be further simplified to a system of first-order ODEs by defining
$w_{L}=\partial_{\varphi}c_{L}$

$\frac{d}{d\varphi}\left(\begin{array}[]{c}c_{L}\\\
w_{L}\end{array}\right)=\left(\begin{array}[]{cc}0&1\\\
k_{L}+k_{R}&0\end{array}\right)\left(\begin{array}[]{c}c_{L}\\\
w_{L}\end{array}\right)-\frac{1}{2\pi}\left(\begin{array}[]{c}0\\\
k_{R}\end{array}\right).$ (S25)

(a)  (b)

Figure S4: Left: Example of a $\varphi$ distribution under the condition that
the particle sojourns in the right well
($D_{R}=$1.35\text{\times}{10}^{-2}$k_{0}$). Right: Comparison of transition
rates extracted from the autocorrelation method (blue circles) against
Boltzmann-factor averages using the observed distributions (orange crosses).
The same simulation parameters were used as in the high $D_{R}$ investigation,
but the time step was increased to obtain longer trajectories at the same
expense (each by a factor ten). Additional estimates computed via the reaction
diffusion model are included as well, depicted as solid orange lines in the
distributions and rate estimates. Using a constant prefactor
$z_{L}=z_{R}=0.3k_{0}$ as the only parameter in the model, we attempt to
reproduce the real angle distributions and therefrom estimated rates. The
assumption of constant prefactors $z_{L},z_{R}$ appears to hold within the
regime of validity of the averaging approximation.

For finite (and low) rotational diffusivity, the solution of Eq. (S25)
represents the stationary distribution to calculate the effective rate as seen
in Eq. (S18). To reiterate, the central requirement for the validity of our
effective model is only a matter of the rotational diffusion’s relative
smallness and our expressions finally include $D_{R}$ explicitly.
Unfortunately, even this simplified model does not seem to have an easily
accessible closed solution due to the complex form of our rate function. The
theory’s performance can be checked by means of computational data, however,
highlighted in Fig. (S4). The showcased example of an angle distribution is
fully in line with our expectations, with angles favouring transitions into
the respective well becoming increasingly more prominent at lower $D_{R}$.

Furthermore, the right panel depicts the slow rotation part of the rate
landscape from last section. Besides increasing the time step and trajectory
length by a factor ten, the simulation parameters have been left they as were.
The angular distributions along the active modification of the potential are
used to compute the rate up to a common prefactor (amounts to a simple
displacement in logarithmic scale, the value of the lowest point is made to
coincide with the autocorrelation data). Our predictions turn out to be very
accurate for an extensive regime of rotational diffusivity values and capture
the rate’s initial increase up until the theory reaches its high $D_{R}$
plateau. At that point we are faced with a situation where our initial
premises and conditions fall apart, making this discrepancy fully expected.
The theory seems like an adequate approximation up until one to two orders of
magnitude before reaching the active turnover, falling short of the fast
rotation limit in terms of accurate coverage.

Figure S5: Side-by-side depiction of the high and low $D_{R}$ data and plots.
Outside the regime of the turnover itself, our models make very accurate
predictions about the landscape’s shape, capturing its key features.

Finally, Fig. S5 showcases the results of both models alongside another,
accurately replicating the landscape over all possible $D_{R}$ safe for the
immediate proximity of the active turnover. Both approaches only rely on the
dynamic prefactor as parameter, corresponding to simple displacements of the
curves on a logarithmic scale. The good consistency between theory and data
hints at both theories capturing the essential details affecting the rate.

##  V Notes on the Simulation

In this section, we summarize the general outline of the simulations
conducted. We propagate trajectories in one spatial dimension by using the
OVRVO integrator first devised by Sivak et al. Sivak  _et al._ (2014) and
based on the well-known Liouville framework for symplectic integrators.
Compared to a standard velocity Verlet integrator, this method additionally
encompasses appropriate Ornstein-Uhlenbeck steps placed at the beginning and
the end of each iteration to act as the system’s thermostat. Each data point
in the transition landscapes results from a trajectory of duration $T$
generated with time step $\Delta t$, specified in the description of the
respective figures. As all rate coefficients shown were extracted from long
trajectories and reflect the stationary state, the initial condition becomes
irrelevant. For completeness, however, let us note we generally start in one
of the minima with a velocity drawn from the Maxwell distribution. Next, we
proceed to delineate the analysis methods applied to both numerical and
experimental trajectories.

The first step of any rate extraction algorithm consists in the selection of
an appropriate dividing surface. Let us direct our attention to Fig. (S6),
showcasing a segment of a simulated trajectory. The peaks in the distribution
corresponding to the metastable states are separated by a range of low
probability. For signals of this kind, the extracted rate does not depend
sensibly on the dividing surface’s exact location. Shifting it slightly to the
left or right influences our results only negligibly, which is why a
straightforward selection method fully serves our purposes. We locate the
approximate position of the maxima left and right by generating histograms
with a prescribed bin width and place our surface in the minimum between those
points. This approach relies on sufficiently smooth histograms to yield
appropriate results, which all investigated trajectories (experimental or
computational ones) were able to provide.

(a)  (b)

Figure S6: Segment of a trajectory in a bistable trap (left) and corresponding
full-length histogram (right), belonging to the data shown in Fig. S3 at
$D_{R}=$4.64\text{\times}{10}^{2}$k_{0}$. The dashed line demonstrates the
dividing surface’s location.

Having placed the dividing surface, our signal is converted into an indicator
function to subsequently evaluate its normalized autocorrelation, an example
of which is shown in Fig. (S7). We invoke the Wiener-Khinchin theorem and
compute all autocorrelations via FFTs for computational efficiency. The
initial behaviour of our autocorrelations is very well-represented by a simple
exponential decay except a very brief period at the very beginning attributed
to short-time correlations. The curves’ roughness increases as the
autocorrelation approaches zero as a consequence of fewer uncorrelated
indicator pairs, but can be diminished by analyzing longer trajectories (i.e.,
gathering more statistics). Excluding the initial non-exponential deviation,
we fit an exponential of the form $c_{0}e^{-kt}$ to these curves, limiting the
regime to the part of the normalized autocorrelation larger than a prescribed
threshold, here $0.10$. The parameter $k$ should then correspond to the sum of
left and right transition rates, completing the extraction.

Figure S7: Normalized autocorrelation functions for five values of the
rotational diffusion. The fits (dashed lines) exhibit an excellent agreement
with the computational data (gray lines). At very short timescales deviations
from the predicted exponential decay occur, which are excluded from the
fitting regime.

Uncertainties have only been computed for the data presented in the main text,
evaluated by splitting the trajectories in ten segments each. We extract the
rate for each segment and compute the standard deviation of the mean results
as our error estimates.

### Replication Study

In this section we describe how we computationally replicated the experimental
findings seen in the main text, specifically the rate landscape visualized in
Fig. 3. Such an endeavour provides additional insight by unearthing
differences and physical contributions not captured by the simple Langevin
model. Since the signal mainly serves to distinguish between wells rather than
providing an accurate position at any point of time, it is not inherently
well-suited to infer the experimental potential. Unable to directly extract
the shape of the physical potential from the signals, we opt for a model that
allows to prescribe all well-known information along with the remaining
parameters to be optimized.

The key quantities of interest are the approximate widths of the two wells and
of the barrier (or alternatively their frequencies) to account for entropic
effects, the barrier height $\Delta U$, and the activity $A$. The two well-
frequencies have been measured experimentally, amounting to
$\omega_{L}=2\pi\times(73.0\pm 0.5)$\text{\,}\mathrm{k}\mathrm{H}\mathrm{z}$$
and $\omega_{R}=2\pi\times(82\pm 4)$\text{\,}\mathrm{k}\mathrm{H}\mathrm{z}$$.
The remaining three parameters, i.e. activity, well-separation, and barrier
height are tuned to find the best replication of the experimental landscape as
presented in the main text (Fig. 3). Initial guesses for the activity and
barrier height have been extracted directly from the rate landscape by means
of parametric fits. The rate approximation in the regime of high rotational
diffusion directly contains both $A$ as well as $\Delta U$ within the
expression for the expected rate Eq. (S16). As the experimental data does not
indicate a significant difference between left and right barrier height, we
assume them to be equal.

As for the conservative trapping/barrier force we employ a piecewise parabolic
potential consisting of three segments, straightforwardly allowing us to
prescribe the well frequencies:

$U(x)=\begin{cases}\dfrac{m\omega_{L}^{2}}{2}(x-a_{L})^{2}&\quad\text{for
}x<b_{L},\\\\[10.0pt] -\dfrac{m\omega_{B}^{2}}{2}x^{2}+\Delta U&\quad\text{for
}b_{L}\leq x\leq b_{R},\\\\[10.0pt]
\dfrac{m\omega_{R}^{2}}{2}(x-a_{R})^{2}&\quad\text{for }x>b_{R}.\end{cases}$
(S26)

Imposing the aforementioned parameters and additionally demanding continuity
of the potential and its first derivative fully determines the location of
minima $a_{L},a_{R}$ and connection points between segments $b_{L},b_{R}$. The
expressions for $a_{L},b_{L}$ read

$\displaystyle a_{L}$ $\displaystyle=-\sqrt{\frac{2\Delta
U(\omega_{L}^{2}+\omega_{B}^{2})}{m\omega_{B}^{2}\omega_{L}^{2}}},$
$\displaystyle b_{L}$
$\displaystyle=\frac{\omega_{L}^{2}a_{L}}{\omega_{B}^{2}+\omega_{L}^{2}}.$
(S27)

The expressions for the right side are obtained by replacing the respective
index and changing the sign for $a_{R}$.

As this approximate one-dimensional potential represents a simplification of
the experimental system, we aim to locate a point in parameter space that
generates a rate landscape as consistent as possible in selected key aspects.
The key aspects considered include the general magnitude of rates, the height,
position and width (at half height) of the passive and active turnovers on a
logarithmic scale. To this end we apply a bisection-like approach on a
discrete grid of parameter values. We consider $A$, $\omega_{B}$, and $\Delta
U$ in steps of $0.05\text{\,}\mathrm{f}\mathrm{N}$, $0.05\omega_{L}$, and
$0.01k_{\text{B}}T$, respectively. Since the Kramers turnover in the regime of
fast rotation is largely independent of $A$, the respective experimental data
can be used to infer the barrier height and frequency/well-separation
separately. Leaving the range of potential values more liberty than would be
necessary from these initial estimates, we limit the search intervals to
$\Delta U/k_{\text{B}}T\in[1.9,2.3]$ and $\omega_{B}/\omega_{L}\in[0.5,2.]$.
The permissible activity values are restricted by the experimental estimate in
conjunction with its symmetric uncertainty. We find excellent agreement
between simulation and experiment in terms of turnover heights and widths with
the parameter set $A=$9.35\text{\,}\mathrm{f}\mathrm{N}$$,
$\omega_{B}=0.60\omega_{L}$ (translating into a minima separation of
$0.66\text{\,}\mu\mathrm{m}$), and $\Delta U=2.065k_{\text{B}}T$.

The position of the calculated Kramers turnover coincides with the
experimental result within its resolution and uncertainties as well, leaving
the position of the active turnover in terms of $D_{R}$ as the only clear
discrepancy despite the ad hoc nature of the potential’s functional form. The
ratio of the simulated and the experimental position of the turnover on the
$D_{R}$-axis amounts to approximately $0.447$, with the simulation placing it
at a lower diffusivity. Simulations performed in- and outside the delineated
optimization range do not hint at the possibility of situating the two lines
of maxima at their targeted positions simultaneously. It appears that the
ratio between the active and passive turnover positions remains roughly
constant over an extensive regime of parameter values. A possible explanation
of this phenomenon lies in both being closely tied to the average transition
path time. The Kramers turnover occurs when the particle loses energy on the
scale of one $k_{\text{B}}T$ throughout a transition. Similarly, the active
turnover stems from the propulsion decorrelating over the duration of a
typical transition. Both turnover locations are thus expected to be inversely
proportional to the average transition path time in good approximation. In one
dimension it becomes therefore difficult to change the position of one maximum
independently from the other. Multidimensional potential landscapes, on the
other and, offer additional freedom by allowing for a multitude of transition
paths that differ already in position space. Transitions induced purely by
thermal fluctuations as captured and collected in the Kramers phenomenology
typically look for paths crossing the lowest point in the barrier as preferred
transition channels, even if this point does not lie on a straight line with
the minima. On the contrary, activity-induced transitions events are likely to
follow a more ”direct” approach, offsetting an increased barrier height in
proportion to the magnitude of $A$. The different channels associated with
active and passive transitions generally translate in distinct transition path
times and thus location of the turnover maxima. As the experiment naturally
unfolds in a three-dimensional setup, and not all coupling effects between
dimensions can be erased in practice, it seems sensible to assume that the
lowest energy paths would neither follow a straight line nor perfectly
coincide with the active force axis. Any small misalignments of the active
force, well- and barrier minima, major axes of the trapping forces in their
parabolic approximation etc. could induce this effect.

Finally, it is important to once again underline the consistency of the
simulation results with the experimental rates despite the usage of an ad hoc
potential. Even the most strongly deviating property, the active turnover’s
position, only does so by a factor of $c=0.447$, residing in the same order of
magnitude. Given a rotational diffusion axis rescaled by this calibration
factor, the obtained replicated landscape becomes quantitatively consistent
with the experiment everywhere, supporting the models discussed in this
document.

##  VI Active Noise

### Statistical properties

In a reduced phase space consisting of only position $x$ and velocity $v$ (but
not the angle $\varphi$), active propulsion is treated as a source of colored
noise with persistence time $\tau_{A}$. In this section, we derive the most
central properties of the active force $n(t)=A\cos(\varphi(t))$ from equation
(1). These also serve as consistency tests for the experimentally implemented
noise source.

We start by computing the stationary distribution $p(n)$ of $n(t)$. Noting
that $\varphi$ evolves randomly according to Brownian motion on the interval
$[0,2\pi)$ with periodic boundary conditions, the stationary distribution of
$\phi$ must be uniform, $p(\varphi)=1/(2\pi)$. Applying a change of variable
from $\varphi$ to $n$ yields

$p(n)=\int_{0}^{2\pi}d\varphi\;\frac{1}{2\pi}\delta(n-A\cos\varphi)=\frac{1}{A\pi|\sin\varphi(n)|}=\frac{1}{\pi\sqrt{A^{2}-n^{2}}}.$
(S28)

Next, we direct our attention to the autocorrelation $R_{nn}(t)$ and to the
power spectral density $S_{nn}(\omega)$. As the angle evolution represents a
Wiener process rescaled by the prefactor $\sqrt{2D_{R}}$, increments of the
angle throughout a time step $\Delta t$ are independently distributed
according to a Gaussian with variance $D_{R}\Delta t$

$\Delta\varphi=\varphi(t+\Delta t)-\varphi(t)\propto\frac{1}{\sqrt{2\pi\Delta
tD_{R}}}e^{-\frac{\Delta\varphi^{2}}{4\Delta tD_{R}}}.$ (S29)

In equation (S29), the periodicity of $\varphi$ has not been taken into
account yet. We impose without loss of generality $\Delta t\geq 0$. For any
real stochastic process, we recover the negative time intervals from the
property $R_{nn}(\Delta t)=R_{nn}(-\Delta t)$. The activity’s autocorrelation
can be evaluated as

$\displaystyle R_{nn}(\Delta t):=\langle n(t+\Delta
t)n(t)\rangle=A^{2}\int_{0}^{2\pi}d\varphi\;\int_{-\infty}^{\infty}d\Delta\varphi\;\cos(\varphi)\cos(\varphi+\Delta\varphi)p(\varphi)p(\Delta\varphi).$
(S30)

Applying Euler’s relation $\cos x=(e^{ix}+e^{-ix})/2$ and

$\displaystyle\left\langle e^{i\varphi}\right\rangle=0,\qquad\left\langle
e^{i\Delta\varphi}\right\rangle=e^{-D_{R}\Delta t},$

leads us to an exact expression for the autocorrelation

$\displaystyle R_{nn}(\Delta t)=\frac{A^{2}}{2}e^{-D_{R}|\Delta t|}.$ (S31)

Consequently invoking the Wiener-Khintchin theorem we obtain the power
spectral density $S_{nn}(\omega)$ (the shorthand $c.c.$ denotes the complex
conjugate):

$S_{nn}(\omega)=\frac{1}{2\pi}\int_{\mathbb{R}}\text{d}t\ R_{nn}(t)e^{-i\omega
t}=\frac{A^{2}}{2\pi}\int_{\mathbb{R}^{+}}\text{d}t\
e^{-\left(D_{R}+i\omega\right)t}+c.c.=\frac{A^{2}}{\pi}\frac{D_{R}}{\omega^{2}+D_{R}^{2}}.$
(S32)

### Experimental Characterization

Figure 2b in the main text depicts a histogram of the active force signal at
the output of the FPGA. The output range of the FPGA is
$[$-1\text{\,}\mathrm{V}$,$1\text{\,}\mathrm{V}$]$, meaning that
$A=$1\text{\,}\mathrm{V}$$. The histogram in Fig. 2b agrees very well with
equation (S28) for $A=1$. Figure 2c shows the power spectral density $S_{nn}$
of the active force for three different values of the rotational diffusion
$D_{R}$. We fit $S_{nn}$ with Eq. (S32) and extract the cutoff frequency
$f_{c}$. The cutoff frequency allows us to calibrate the physical value of the
rotational diffusion using $D_{R}=2\pi\times f_{c}$. The value of $A$ measured
in Newton is estimated from the position response of a particle to a known
modulated voltage. The experimental value of $A$ equals $6.8\pm
3.4\text{\,}\mathrm{f}\mathrm{N}$. The uncertainty in the estimation of the
activity is limited by the uncertainty in the calibration factor of the
position measurement Hebestreit  _et al._ (2018). Specifically, in order to
determine a force in Newton, we make use of the mass-dependent transfer
function of the particle. An uncertainty up to $50\%$ in the mass of the
particle, the main precision bottleneck of the position calibration,
translates into the same uncertainty for the estimation of the activity. Note,
however, that the estimate of $A/m$ does not suffer from such uncertainty.

## References

  * Sivak and Crooks (2012) D. A. Sivak and G. E. Crooks, Phys. Rev. Lett. 108, 150601 (2012).
  * Dellago and Bolhuis (2009) C. Dellago and P. G. Bolhuis, “Transition Path Sampling and Other Advanced Simulation Techniques for Rare Events,” in  _Adv. Comput. Simul. Approaches Soft Matter Sci. III_, edited by C. Holm and K. Kremer (Springer Berlin Heidelberg, 2009) pp. 167–233.
  * Sivak  _et al._ (2014) D. A. Sivak, J. D. Chodera, and G. E. Crooks, J. Phys. Chem. B 118, 6466 (2014).
  * Hebestreit  _et al._ (2018) E. Hebestreit, M. Frimmer, R. Reimann, C. Dellago, F. Ricci, and L. Novotny, Rev. Sci. Instrum. 89, 033111 (2018).