# Probing for targeted syntactic knowledge through grammatical error detection

Christopher Davis$\dagger$ Christopher Bryant$\dagger$ Andrew Caines$\dagger$
Marek Rei$\ddagger$$\dagger$ Paula Buttery$\dagger$
$\dagger$ALTA Institute, Department of Computer Science & Technology,
University of Cambridge, U.K.
$\ddagger$Imperial College London, U.K.
<EMAIL_ADDRESS>
<EMAIL_ADDRESS>
###### Abstract

Targeted studies testing knowledge of subject-verb agreement (SVA) indicate
that pre-trained language models encode syntactic information. We assert that
if models robustly encode subject-verb agreement, they should be able to
identify when agreement is correct and when it is incorrect. To that end, we
propose grammatical error detection as a diagnostic probe to evaluate token-
level contextual representations for their knowledge of SVA. We evaluate
contextual representations at each layer from five pre-trained English
language models: bert, xlnet, gpt-2, roberta, and electra. We leverage public
annotated training data from both English second language learners and
Wikipedia edits, and report results on manually crafted stimuli for subject-
verb agreement. We find that masked language models linearly encode
information relevant to the detection of SVA errors, while the autoregressive
models perform on par with our baseline. However, we also observe a divergence
in performance when probes are trained on different training sets, and when
they are evaluated on different syntactic constructions, suggesting the
information pertaining to SVA error detection is not robustly encoded.

##  1 Introduction

Recent work investigates whether linguistic information is encoded in pre-
trained transformer-based language models Peters et al. (2018); Devlin et al.
(2019). Research using diagnostic methods Shi et al. (2016); Alain and Bengio
(2017); Adi et al. (2017); Conneau et al. (2018); Hupkes et al. (2018)
indicates models encode syntax via experiments targeting, for example, part-
of-speech and dependency labelling Tenney et al. (2019); Jawahar et al.
(2019); Hewitt and Manning (2019), while targeted syntactic evaluation studies
show models encode a large amount of hierarchical syntactic information in
tests for subject-verb agreement Linzen et al. (2016); Marvin and Linzen
(2018); Goldberg (2019). Although previous research has covered a large number
of probing tasks Tenney et al. (2019); Liu et al. (2019a), no one has yet
fully explored grammatical error detection (GED) as a probe. We assert that
the ability to detect ungrammatical tokens serves as a complementary
evaluation to assess linguistic competence.

GED is a natural and complex NLP task that assesses a model’s ability to
detect which tokens in a sentence are grammatically incorrect. Ungrammatical
tokens may be categorised within a taxonomy111Much recent research in GED uses
error-type labels based on ERRANT Bryant et al. (2017). comprising three
operational categories (replacement, unnecessary, and missing) and twenty-five
categories based on parts-of-speech. For example:

. [Replacement subject-verb agreement] _The train are a good option for long
trips._

. [Replacement pronoun] _Everybody must have free time for yourself._

. [Missing determiner] _The birth of [ a] new star._

. [Unnecessary preposition] _Public transport means travelling around […] by
using trains, buses, and planes._

To do well in the task, a model must encode and make use of a wide array of
linguistic information. For example, detecting subject-verb agreement errors
in English tests a model’s capacity to identify i) verbs, ii) the subjects of
the verbs, iii) the grammatical number (singular/plural) of both, and iv)
whether their number agrees.

The above makes the task a very interesting test-bed for evaluating a model’s
syntactic knowledge. We operationalise the GED task and train probes to detect
replacement subject-verb agreement errors (as in Example 1) using contextual
representations from different hidden layers from five pre-trained English
language models – bert Devlin et al. (2019), xlnet Yang et al. (2019), gpt-2
Radford et al. (2019), roberta Liu et al. (2019b), and electra Clark et al.
(2020).

To ensure a robust and thorough evaluation, we leverage existing publicly
annotated data from two domains for training: essays by learners of English as
a second language from both the Cambridge English Write & Improve + LOCNESS
(W&I) corpus (Bryant et al., 2019) and the First Certificate in English corpus
(FCE) Yannakoudakis et al. (2011), along with a corpus of automatically
extracted edited sentences from native English Wikipedia edit histories
Grundkiewicz and Junczys-Dowmunt (2014). For evaluation, we re-frame the
minimal-pair dataset from Marvin and Linzen (2018) to create targeted
evaluation sets annotated for GED. In doing so, we demonstrate how existing
minimal-pair datasets can be leveraged to create challenging and interpretable
test sets for GED models222In principle these minimal-pair datasets can also
be used to evaluate grammatical error correction systems. Hu et al. (2020).

We find that electra, bert, and roberta linearly encode information for the
detection of SVA errors in the contextual representations of verbs, however,
we observe a gap in performance when probes are trained on data from different
domains, implying the information is not encoded consistently or robustly. The
results show consistent patterns across layers: both bert and electra encode
information related to SVA errors in the middle-to-late layers, while roberta
seems to encode information earlier in the model. Probes trained on
representations from gpt-2 and xlnet (with uni- and bi-directional decoding)
perform poorly on the evaluation set, indicating a fundamental difference from
either the training objective or pre-training data. Finally, we show that GED
probes can complement existing tools for syntactic evaluation: our results
suggest that although neural language models perform well on targeted
syntactic evaluation tasks, their encoding of SVA does not robustly extend to
the detection of SVA errors.333We release our code at
https://github.com/chrisdavis90/ged-syntax-probing

Syntactic Construction | Example
---|---
Simple agreement | The author laughs/laugh∗
Agreement in a sentential complement | The bankers knew the officer smiles/smile∗
Agreement across a prepositional phrase | The farmer near the parents smiles/smile∗
Agreement across a subject relative clause | The officers that love the skater smile/smiles∗
Short verb-phrase coordination | The senator smiles and laughs/laugh∗
Long verb-phrase coordination | The manager writes in a journal every day and
|  likes/like∗ to watch television shows
Agreement across an objective relative clause | The farmer that the parents love swims/swim∗
Agreement within an objective relative clause | The farmer that the parents love/loves∗ swims
Table 1: Examples for the main syntactic constructions from the subject-verb-
agreement stimuli from Marvin and Linzen (2018). Bold indicates the subject-
noun, and underlined tokens indicate the grammatical/ungrammatical∗ verb.

##  2 Token-level grammaticality

We motivate the use of GED-probes by first reviewing previous literature
involving grammaticality judgements and tests for subject-verb agreement, then
discuss the advantages in tests for GED.

Boolean acceptability judgements have long been used as a primary behavioural
measure to observe humans’ grammatical knowledge Chomsky (1957); Pater (2019),
and have recently been employed in computational linguistics to evaluate
grammatical knowledge in neural models. For example, Warstadt et al. (2019)
train classifiers to predict sentence-level Boolean acceptability judgements
on example sentences from the linguistics literature. As each sentence is
designed to demonstrate a particular grammatical construction, performance on
the task is interpreted as a reflection of the implicit knowledge of the
classifier.

An alternative approach frames acceptability as a choice between minimal pairs
of sentences – one grammatical and another ungrammatical, where the difference
between the two is typically one or two tokens. Marvin and Linzen (2018)
evaluate linguistic knowledge by testing whether a language model assigns
higher probability to a grammatical sentence relative to its minimally
different ungrammatical counterpart. Similar to Warstadt et al. (2019), fine-
grained grammatical knowledge is evaluated by controlling the evaluation
stimuli, with the hypothesis that models must have implicit knowledge of the
underlying grammatical concept to succeed.

Rather than evaluating sentence-level scores, Linzen et al. (2016) compare
predicted probabilities assigned to target verbs in minimal pair sentences,
where each sentence in a pair uses a different form of the verb. Goldberg
(2019) extends this to masked language models where he replaces a target verb
with the [MASK] token and feeds the entire sentence to a bert model. A model
is considered successful, and thereby has knowledge related to SVA, if it
assigns higher probability to the correct form of the verb.

Our work differs from the above in three important ways. First, we don’t
assume to know where the incorrect token is – the probe is trained to detect
errors for all tokens in a sentence, given each token’s contextual
representation. This is a more fine-grained evaluation compared to sentence-
level judgements and tests whether probes know where the error is located.
Second, instead of targeting information in the masked token, we investigate
whether the model implicitly encodes SVA information in a token’s contextual
representation. Third, we test for knowledge of SVA without comparing to the
counterpart token or sentence. We argue that if a model has knowledge of SVA,
it should be able to detect SVA-errors without requiring a comparison.

##  3 Data

###  3.1 Second language learner corpora

Following previous work in grammatical error correction and detection, we use
the Cambridge English Write & Improve + LOCNESS corpus (Bryant et al., 2019)
and the First Certificate in English Yannakoudakis et al. (2011), hereinafter
w&i-fce.444Public data for W&I and the FCE are available at:
https://www.cl.cam.ac.uk/research/nl/bea2019st#data

The edit annotations in these corpora were pre-processed and standardised
using the ERRANT annotation framework Bryant et al. (2017). One advantage of
this framework is that error types are modular, and consist of “operation” +
“main” type tags. This provides us flexibility to target grammatical errors at
different levels of granularity. E.g. all NOUN errors or only R:NOUN for
replacement nouns. In addition, we can take advantage of the corrections
provided with each edit annotation to control the number and variation of
grammatical errors. Since we focus only on replacement subject-verb-agreement
errors, R:VERB:SVA, we correct all other error types and keep only those
sentences containing at least one grammatical error. This leaves 1936
sentences for training and 142 sentences for validation.

###  3.2 Dataset of Wikipedia edits

As an alternative to the learner corpora, we additionally experiment with a
corpus of automatically extracted edited sentences from native English
Wikipedia edit histories (wiked) Grundkiewicz and Junczys-Dowmunt (2014). We
use the clean and preprocessed version of English Wikipedia edits, consisting
of ~29 million sentences.555https://github.com/snukky/wikiedits We follow the
same procedure as above and retain only sentences with R:VERB:SVA errors by
correcting all other error types, and keep only the sentences containing at
least one error. This leaves ~233K sentences, from which we sample five
training sets each with 1936 sentences each to match the amount of sentences
in the learner corpora after processing, and 5839 sentences for the validation
set. Statistics for both corpora are given in Appendix A. We refer to the
sampled training sets as wiked-s.

###  3.3 Minimal-pair datasets

We use the manually constructed subject-verb agreement stimuli from Marvin and
Linzen (2018) (m&l) to evaluate the GED-probes – this enables a more
controlled evaluation compared to the naturally occurring sentences in w&i-fce
and wiked. The dataset consists of seven main syntactic constructions, shown
in Table 1. In addition to those shown in the table, sentences with multiple
nouns (except for those testing VP coordination) include instances with two
nouns and one acts as a distractor, potentially agreeing with the verb even
though it is not the subject:

. . The farmer near the parent smiles/smile∗. .̱ The farmer near the parents
smiles/smile∗. .̧ The farmers near the parent smiles∗/smile. .̣ The farmers
near the parents smiles∗/smile.

In the above sentences, the verbs marked with an asterisk are ungrammatical.
The dataset also expands on sentences testing agreement with object relative
clauses: agreement is tested across and within the clause, using animate and
inanimate main subjects, and with and without the _that_ -complementizer.

We process the m&l minimal pairs to create token-level GED annotations, where
the ungrammatical verbs are tagged as R:VERB:SVA.666In principle, any minimal-
pair dataset can be converted to token-level annotations using ERRANT, but not
all grammatical errors map cleanly to ERRANT categories. For example,
replacement pronouns (R:PRON) includes reflexive anaphor gender- and number-
agreement errors. We include all of the ungrammatical and grammatical
sentences for evaluation – to be successful, the GED-probe should recognise
when the agreement is correct and label all tokens as grammatical.777While the
evaluation stimuli consists of minimal pairs, the training data does not.
Finally, we capitalise the first word in each sentence and add a full stop if
one doesn’t already exist. Appendix B contains details about the processed
dataset.

##  4 Experiment 1: Per Layer Probes

We first investigate whether models encode SVA-errors by examining probing
performance at each layer; we want to test whether models encode this
information in the final layer, the token representation, but also how this
information develops across the layers. We then break-down performance by
syntactic construction to better understand how the SVA-error encoding
generalizes. Finally, we carry out a follow-up experiment to investigate the
impact of training data size and verb frequency.

###  4.1 Experimental setup

For each model, we extract contextual representations for every token in a
sentence for every one of the twelve layers in the model. We then train a
linear probe Tenney et al. (2019); Hupkes et al. (2018); Liu et al. (2019a)
per layer to predict whether each token-level contextual representation is
ungrammatical. We train two versions of each probe: one trained using w&i-fce,
and the other using wiked-s. Every probe is evaluated on the m&l stimuli

Since the probe is trained to detect R:VERB:SVA errors, high probing
performance would indicate the probe has learned to extract features to
identify subject-verb agreement errors from the contextual representations of
the verbs. This implicitly includes sub-tasks to identify the verb, the
subject noun, the number of both the verb and noun, and that their number
disagrees. Furthermore, as this is a token labelling probe, high probing
performance would indicate the pre-trained model has encoded the relevant
features in the contextual representations of the verbs.

We evaluate probes using F1 on the evaluation stimuli from m&l containing an
equal number of grammatical and ungrammatical sentences.888This departs from
F0.5 used in the GED literature, which was motivated from educational
applications where high precision is preferred over recall because false-
positives can be more harmful for language learners compared to false-
negatives. We compare probes to a verb-only baseline which incorrectly tags
all verbs as ungrammatical. The number of verbs per sentence varies across
syntactic constructions; constructions with one verb have an equal number of
grammatical and ungrammatical verbs, and therefore have a baseline score of
0.67. Constructions with two and three verbs have scores of 0.40 and 0.30,
respectively. Evaluating the baseline over all constructions yields a score of
0.43.

We evaluate five pre-trained models: bert-base-cased, gpt-2 (small), roberta-
base, xlnet with both uni-directional (xlnet-uni) and bi-directional xlnet-bi
decoding, and electra-base (discriminator). As all five models use sub-word
tokenisation, we follow Liu et al. (2019a) and use the last sub-word unit for
token classification. We train the probes for 50 epochs with a patience of 10
epochs for early stopping based on in-domain validation sets.

Four of the five models were pre-trained with a language modelling objective:
either masked language modelling (MLM) or autoregressive language modelling
(ALM). Whereas electra is the exception – the _replaced token detection_
training objective is somewhat aligned with GED and therefore we may expect
representations to encode grammatically discriminative information.999The
electra discriminator model is trained to detect substituted tokens in a
grammatical sentence, where an original token is substituted with a plausible
alternative from a masked language model. Indeed, Yuan et al. (2021) find
electra outperforms bert when fine-tuned for binary GED targeting a wide range
of error-types. bert and roberta also detect replaced tokens during training,
but only on 1.5% of tokens.

(a) w&i-fce

(b) wiked-s

Figure 1: F1 scores for probes trained on contextual representations at different layers from bert, electra, roberta, xlnet with bidirectional decoding (xlnet-bi), xlnet with unidirectional decoding (xlnet-uni), and gpt-2. 1(a) and 1(b) show results for probes trained on w&i-fce and the wiked-s training sets respectively. The verb-only baseline scores are illustrated using grey horizontal lines. 1(b) shows the mean and standard deviation across the five training sets (section 3.2). All probes are evaluated on the m&l stimuli (section 3.3). | _w &i-fce_ | _wiked-s_
---|---|---
Model | Layer | F1 | Layer | F1
electra | 9 | 0.95 | 10 | 0.89±0.02
bert | 12 | 0.89 | 12 | 0.73±0.03
roberta | 5 | 0.80 | 5 | 0.70±0.04
xlnet-bi | 6 | 0.59 | 6 | 0.51±0.01
xlnet-uni | 3 | 0.48 | 3 | 0.43±0.02
gpt-2 | 6 | 0.45 | 6 | 0.48±0.02
verb-only | - | 0.43 | - | 0.43
Table 2: Top F1 scores on the m&l evaluation set, for probes trained on either
w&i-fce or wiked-s. Scores for probes trained on wiked-s are reported as the
_mean_ $\pm$1 standard deviation over the five sampled training sets.

###  4.2 Results

Figure 1 shows F1 scores for probes trained on w&i-fce and wiked-s, and
evaluated on the m&l stimuli.101010We additionally evaluate probes against the
other error types in the W&I dataset and verify that probes only detect SVA
errors. Probes trained on either bert or electra obtain mean scores of 0.04
($\sigma$=0.04), verifying that information extracted by the probe is isolated
to subject-verb agreement errors. For the latter, we plot the mean and
standard deviation evaluated over the five sampled training sets. We
illustrate the verb-only baseline score with a grey horizontal line. Table 2
shows layers which obtained the top F1 score per model.

The figure and table shows electra encodes the most salient information for
SVA error detection, with probes obtaining maximum scores of 0.95 and 0.89
($\sigma$=0.02) when trained on w&i-fce and wiked-s, respectively. Though this
may not be surprising given the _replaced token detection_ pre-training
objective, it illustrates that probes trained on representations from a model
capable of SVA error detection can obtain high performance using both training
sets.

We observe a divergence in performance between MLM-probes and ALM-probes; the
MLM-probes tend to perform better, obtaining maximum scores between 0.70 and
0.89 F1, while the ALM-probes don’t score above 0.59 on either training set.
In fact, probes trained on representations from gpt-2 and xlnet-uni often
perform worse than the verb-only baseline at 0.43 and don’t score above 0.50
F1. These results imply that gpt-2 and xlnet-uni representations do not
linearly encode enough information to differentiate between grammatical and
ungrammatical verbs in SVA.

The MLM-ALM performance gap could be due to the language model directionality:
the two uni-directional models (gpt-2 and xlnet-uni) do perform the worst, but
this fails to account for the performance of xlnet-bi – a bi-directional
language model which does not perform much better. It may be that the MLM
training objective helps to imbue contextual representations with information
useful for detecting SVA errors, but we cannot discount the inclusion of the
replaced token detection objective, even though it is rarely included.
Finally, we note key differences in the pre-training data used by the models:
bert and electra use the BooksCorpus and English Wikipedia, gpt-2 uses web-
scraped data, and xlnet and roberta use a combination of BooksCorpus,
Wikipedia, and web-scraped data.

When we examine performance across layers we see that electra\- and bert-
probes follow a similar trajectory, with performance on par with the baseline
from layers 1-5 and higher performance only in layers 8-12. SVA-error
information is highest in the final layer for bert (the token representation)
while layers 9 and 10 seem to encode the most useful information for electra.
In contrast, probes trained with representations from roberta peak at layer 5
but have mostly consistent performance until 12, apart from a drop in layer 7.

The results for bert support those from Jawahar et al. (2019), where they find
probes encode elements of syntax in the middle to late layers. Liu et al.
(2019a) also find that later layers obtain the best performance for a general
GED probe, though they experiment on a single dataset (FCE Yannakoudakis et
al. (2011)) and include all grammatical error types. Recent results from Lasri
et al. (2022b) show that removing _number_ information from nouns at different
layers has a detrimental effect on the number-agreement task up until layer 8.
They hypothesise that some transfer of noun-_number_ takes place in the
previous layers. Our results seem to support this: low probing performance in
layers 1-5 (when the noun-_number_ has yet to be transferred to the verb), and
high probing performance in layers 8-12 after transfer has taken place.
Interestingly, performance for electra takes the same shape, suggesting that
the pre-training objective (_replace token detection_ versus MLM) may not have
an important role in how models encode SVA information, especially given that
roberta, trained using masked-language modelling, displays a different pattern
across layers. Finally, if GED-probe-performance can be taken as a proxy for
noun-_number_ transfer, then results for roberta-probes suggest that noun-
_number_ information is transferred to the target verb earlier, possibly due
to the more robust optimization.

Turning to probe performance across training sets, we find that probes trained
on w&i-fce consistently perform better than those trained on wiked-s, for all
models tested apart from gpt-2. This could be due to a domain mismatch, where
learner writing may be more similar to the m&l stimuli than data from wiked-s.
Though, at the very least this indicates that information pertaining to SVA
errors is not consistently encoded.

Previous work finds some evidence that bert’s representations encode knowledge
of SVA Goldberg (2019); Jawahar et al. (2019); Newman et al. (2021); Lasri et
al. (2022a) but that this knowledge is based on heuristics rather than robust
SVA rule learning Chaves and Richter (2021); McCoy et al. (2019). Our results
indicate that information encoded in contextual representations extends to the
detection of SVA errors in electra, bert, and to a lesser extent, roberta.
However, we find the encoding is not robust across domains, supporting the
heuristic-learning claim. These results illustrate the importance of utilising
training and evaluation datasets from disparate domains to evaluate probes.

Figure 2: F1 scores for probes trained on contextual representations from bert
and roberta, using both the w&i-fce and wiked-s training sets. The probes are
evaluated on m&l stimuli. The verb-only baseline is illustrated using grey
horizontal lines.

###  4.3 Results per syntactic construction

We break down the performance of probes for each syntactic construction in the
m&l dataset. Figure 2 illustrates F1 scores for probes trained on w&i-fce and
wiked-s using representations from each layer of bert and roberta.111111We
select bert and roberta because they are both MLMs, whereas the electra-
discriminator is trained using a _replaced token detection_ objective. The
verb-only baseline is shown as grey horizontal lines. For brevity, we present
results on sentences with simple agreement, sentential complements,
prepositional phrases, subject relative clauses, and object relative clauses
(agreement within and across the clause). Results for the other models and
syntactic constructions are included in Appendix E.

The trends across layers observed in Figure 1 are generally consistent within
syntactic constructions: probes trained on bert representations improve in the
later layers, while layers 5 and 8-12 seem to be the most salient for probes
trained on roberta.

Probes for both models detect SVA errors in the simple agreement constructions
– with only the bert-probe trained on wiked-s scoring less that 0.90 F1. We
find bert-probes perform well for most constructions, especially in layer 12,
but performance for roberta-probes drops for sentences with subject-relative
clauses, prepositional phrases, and object-relative clasues (agreement within
the clause), particularly when trained on wiked-s data. This suggests that the
token representation from bert models, potentially used in downstream tasks,
already encodes a lot of information related to SVA before any fine-tuning.

When comparing performance between probes across training sets, we observe a
noticeable difference in performance between bert-probes trained on w&i-fce
versus those trained on wiked-s, most evidently in five out of the six
syntactic constructions shown. On the other hand, probes trained on roberta
don’t always display a performance gap – performance is more comparable
between probes on sentences testing simple agreement, sentential complements,
and agreement across object relative clauses. For sentences with subject
relative clauses we find the probes trained on wiked-s outperform those
trained on w&i-fce. These results may indicate that information pertaining to
the detection of SVA errors is more robustly encoded in roberta than bert –
that is, the information is more invariant to the choice of probe training
set.

Figure 3: F1 scores for probes trained on bert representations, with varying
amounts of data (left and centre); 1x=1936, 4x=7744, 8x=15488 sentences. The
plots show the mean and standard deviation across five training sets (section
5). On the left, probes are trained on data including m&l verbs, while the
centre shows scores for probes trained on data without m&l verbs (apart from
“is” and “are”, as described in section 5). The right-hand plot shows the mean
and standard deviation of differences in F1 scores between probes trained on
the two types of dataset: with and without m&l verbs. The verb-only baseline
is shown as a grey horizontal line.

##  5 Experiment 2: Generalisation to unseen verbs

In our first experiment, we observe that although the MLMs encode more
information for SVA error detection compared to the ALMs, the information does
not always generalize across domains or syntactic constructions. We carry out
a follow-up experiment to investigate whether the information generalizes
across verbs using probes trained on bert representations and wiked data. We
focus on layers 6 to 12 as these were the layers where performance was above
the baseline. There are 13 target verbs in the m&l stimuli, of which “to be”
is the most frequent with 946 occurrences in wiked-s. The remaining verbs
appear very infrequently – for example, eight verbs have frequencies less than
30. To test generalization across verbs we remove all sentences from the
training and development sets which contain a verb from the m&l stimuli except
for “to be”. We then re-sample sentences from the full wiked data to maintain
1936 sentences as in the first experiment. Due to the infrequency of many
verbs and to ensure a more thorough evaluation, we also increase the training
set size by 4- and 8-times to yield training sets with 7744 and 15488
sentences, respectively. We refer to the three sizes as small, medium, and
large. This results in paired training sets: for each training set size, there
is one set “with m&l verbs” and a set “without m&l verbs”. We sample each
training set five times and report the mean and standard deviation over the
samples. For example, we sample five training sets with 7744 sentences “with
m&l verbs”, and another five “without m&l verbs”. Since we are only interested
in the performance of 12 verbs, we modify the evaluation stimuli to remove a)
sentences containing only “to be” verbs, and for sentences with multiple verbs
we remove the “to be” token from evaluation. For example, in the sentence “The
movie the security guards like is good”, we remove “is” from evaluation.

Figure 3 illustrates the F1 scores: the left and centre plots show results for
probes trained on data with and without m&l verbs, respectively. The plot on
the right presents the mean and standard deviation for the pairwise
differences between probes trained on datasets of the same size. The right-
hand plot shows that for the smallest training set size we observe a slight
benefit when including the m&l verbs, but this is limited to ~0.05 F1 and
restricted to layers 8-10. We generally observe no difference in performance
for the medium and large training sets, indicating that SVA-error information
does generalize across verbs. Furthermore, we observe no difference when
comparing results across training set sizes in the left and central plots,
except for probes trained on the small training set without m&l verbs. These
results indicate that SVA-error information is linearly accessible and
generalizable across across the verbs we test, even when probes are trained
with limited data. Future work may expand the investigation to cover more
verbs, though we expect performance to deteriorate as verbs become infrequent
in the pre-training data Wei et al. (2021).

##  6 Discussion

Our experiments test whether information for SVA errors is implicitly encoded
in the contextual representations of verbs, but they don’t provide any
indication as to _how_ the information is encoded: is grammaticality encoded
atomically or compositionally? Furthermore, we note that selecting the
“erroneous token” can be an ambiguous choice between the noun and the verb,
for example in “The authors laughs”. Yet, the probes we evaluate never tag the
nouns. This could indicate that a) the probes learn to only tag verbs, and/or
b) that SVA-grammaticality is disparately encoded between nouns and verbs. A
compositional account of grammatical encoding is a plausible explanation given
the results provided in Lasri et al. (2022b) – that nouns and verbs have
different encodings for _number_. We plan to investigate how grammaticality is
encoded in future work, both in pre-trained language models as well as models
trained specifically for GED.

##  7 Conclusion

We analyse whether pre-trained transformer-based language models implicitly
encode knowledge of SVA errors using GED probes. We carry out a thorough
evaluation on five models, using two public training sets from different
domains, and evaluate on a manually constructed evaluation set. This enables
us to get a more complete and reliable picture of a models’ performance.

Grammatical error detection is a challenging and linguistically aligned task
to assess the knowledge of neural language models; we show that GED-probes can
be used as a complementary analysis tool to evaluate a models’ linguistic
capabilities.

Our results show that electra, bert, and roberta encode information for SVA-
error detection, but gpt-2 and xlnet do not. For bert and roberta, we find
that the SVA-error encoding is not robust across all syntactic constructions
or training set domains, though we do find some evidence that the encoding
generalizes across verbs for bert. Furthermore, a layer-wise analysis reveals
the final layers in electra and bert are the most salient for SVA-error
detection.

## Acknowledgements

We thank Guy Aglionby, Rami Aly, Paula Czarnowska, and Tiago Pimentel for
helpful discussions and feedback on early drafts of this work. We also thank
the anonymous reviewers for their helpful feedback. This paper reports on
research supported by Cambridge University Press & Assessment. We thank the
NVIDIA Corporation for the donation of the Titan X Pascal GPU used in this
research.

## References

  * Adi et al. (2017) Yossi Adi, Einat Kermany, Yonatan Belinkov, Ofer Lavi, and Yoav Goldberg. 2017.  Fine-grained analysis of sentence embeddings using auxiliary prediction tasks.  In _5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings_. OpenReview.net.
  * Alain and Bengio (2017) Guillaume Alain and Yoshua Bengio. 2017.  Understanding intermediate layers using linear classifier probes.  In _5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Workshop Track Proceedings_. OpenReview.net.
  * Bryant et al. (2019) Christopher Bryant, Mariano Felice, Øistein E Andersen, and Ted Briscoe. 2019\.  The BEA-2019 shared task on grammatical error correction.  In _Proceedings of the Fourteenth Workshop on Innovative Use of NLP for Building Educational Applications_ , pages 52–75.
  * Bryant et al. (2017) Christopher Bryant, Mariano Felice, and Edward Briscoe. 2017.  Automatic annotation and evaluation of error types for grammatical error correction.  Association for Computational Linguistics.
  * Chaves and Richter (2021) Rui P Chaves and Stephanie N Richter. 2021.  Look at that! BERT can be easily distracted from paying attention to morphosyntax.  _Proceedings of the Society for Computation in Linguistics_ , 4(1):28–38.
  * Chomsky (1957) Noam Chomsky. 1957.  _Syntactic structures’s_.  Mouton: The Hague, The Netherlands.
  * Clark et al. (2020) Kevin Clark, Minh-Thang Luong, Quoc V. Le, and Christopher D. Manning. 2020.  ELECTRA: Pre-training Text Encoders as Discriminators Rather Than Generators.  In _International Conference on Learning Representations_.
  * Conneau et al. (2018) Alexis Conneau, German Kruszewski, Guillaume Lample, Loïc Barrault, and Marco Baroni. 2018.  What you can cram into a single $&!#* vector: Probing sentence embeddings for linguistic properties.  In _Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pages 2126–2136, Melbourne, Australia. Association for Computational Linguistics.
  * Devlin et al. (2019) Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019.  BERT: pre-training of deep bidirectional transformers for language understanding.  In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2019, Minneapolis, MN, USA, June 2-7, 2019, Volume 1 (Long and Short Papers)_ , pages 4171–4186. Association for Computational Linguistics.
  * Goldberg (2019) Yoav Goldberg. 2019.  Assessing BERT’s syntactic abilities.  _arXiv preprint arXiv:1901.05287_.
  * Grundkiewicz and Junczys-Dowmunt (2014) Roman Grundkiewicz and Marcin Junczys-Dowmunt. 2014.  The WikEd Error Corpus: A corpus of corrective Wikipedia edits and its application to grammatical error correction.  In _Advances in Natural Language Processing – Lecture Notes in Computer Science_ , volume 8686, pages 478–490. Springer.
  * Hewitt and Manning (2019) John Hewitt and Christopher D. Manning. 2019.  A structural probe for finding syntax in word representations.  In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)_ , pages 4129–4138, Minneapolis, Minnesota. Association for Computational Linguistics.
  * Hu et al. (2020) Jennifer Hu, Jon Gauthier, Peng Qian, Ethan Wilcox, and Roger Levy. 2020.  A systematic assessment of syntactic generalization in neural language models.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 1725–1744, Online. Association for Computational Linguistics.
  * Hupkes et al. (2018) Dieuwke Hupkes, Sara Veldhoen, and Willem Zuidema. 2018.  Visualisation and ‘diagnostic classifiers’ reveal how recurrent and recursive neural networks process hierarchical structure.  _Journal of Artificial Intelligence Research_ , 61:907–926.
  * Jawahar et al. (2019) Ganesh Jawahar, Benoît Sagot, and Djamé Seddah. 2019.  What does BERT learn about the structure of language? In _ACL 2019 - 57th Annual Meeting of the Association for Computational Linguistics_ , Florence, Italy.
  * Lasri et al. (2022a) Karim Lasri, Alessandro Lenci, and Thierry Poibeau. 2022a.  Does BERT really agree? fine-grained analysis of lexical dependence on a syntactic task.  In _Findings of the Association for Computational Linguistics: ACL 2022_ , pages 2309–2315.
  * Lasri et al. (2022b) Karim Lasri, Tiago Pimentel, Alessandro Lenci, Thierry Poibeau, and Ryan Cotterell. 2022b.  Probing for the usage of grammatical number.  In _Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pages 8818–8831.
  * Linzen et al. (2016) Tal Linzen, Emmanuel Dupoux, and Yoav Goldberg. 2016.  Assessing the ability of LSTMs to learn syntax-sensitive dependencies.  _Transactions of the Association for Computational Linguistics_ , 4:521–535.
  * Liu et al. (2019a) Nelson F Liu, Matt Gardner, Yonatan Belinkov, Matthew E Peters, and Noah A Smith. 2019a.  Linguistic knowledge and transferability of contextual representations.
  * Liu et al. (2019b) Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019b.  RoBERTa: A robustly optimized BERT pretraining approach.  _arXiv preprint arXiv:1907.11692_.
  * Marvin and Linzen (2018) Rebecca Marvin and Tal Linzen. 2018.  Targeted syntactic evaluation of language models.  In _Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing_ , pages 1192–1202, Brussels, Belgium. Association for Computational Linguistics.
  * McCoy et al. (2019) Tom McCoy, Ellie Pavlick, and Tal Linzen. 2019.  Right for the wrong reasons: Diagnosing syntactic heuristics in natural language inference.  In _Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics_ , pages 3428–3448.
  * Newman et al. (2021) Benjamin Newman, Kai-Siang Ang, Julia Gong, and John Hewitt. 2021.  Refining targeted syntactic evaluation of language models.  In _Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_ , pages 3710–3723.
  * Pater (2019) Joe Pater. 2019.  Generative linguistics and neural networks at 60: Foundation, friction, and fusion.  _Language_ , 95(1):e41–e74.
  * Peters et al. (2018) Matthew E. Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018.  Deep contextualized word representations.  In _Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers)_ , pages 2227–2237, New Orleans, Louisiana. Association for Computational Linguistics.
  * Radford et al. (2019) Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. 2019.  Language models are unsupervised multitask learners.  _OpenAI blog_ , 1(8):9.
  * Shi et al. (2016) Xing Shi, Inkit Padhi, and Kevin Knight. 2016.  Does string-based neural MT learn source syntax?  In _Proceedings of the 2016 conference on empirical methods in natural language processing_ , pages 1526–1534.
  * Tenney et al. (2019) Ian Tenney, Patrick Xia, Berlin Chen, Alex Wang, Adam Poliak, R Thomas McCoy, Najoung Kim, Benjamin Van Durme, Samuel R Bowman, Dipanjan Das, et al. 2019.  What do you learn from context? probing for sentence structure in contextualized word representations.  _arXiv preprint arXiv:1905.06316_.
  * Warstadt et al. (2019) Alex Warstadt, Amanpreet Singh, and Samuel R Bowman. 2019.  Neural network acceptability judgments.  _Transactions of the Association for Computational Linguistics_ , 7:625–641.
  * Wei et al. (2021) Jason Wei, Dan Garrette, Tal Linzen, and Ellie Pavlick. 2021.  Frequency effects on syntactic rule learning in transformers.  In _Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing_ , pages 932–948.
  * Yang et al. (2019) Zhilin Yang, Zihang Dai, Yiming Yang, Jaime Carbonell, Russ R Salakhutdinov, and Quoc V Le. 2019.  XLNet: Generalized autoregressive pretraining for language understanding.  _Advances in neural information processing systems_ , 32.
  * Yannakoudakis et al. (2011) Helen Yannakoudakis, Ted Briscoe, and Ben Medlock. 2011.  A new dataset and method for automatically grading ESOL texts.  In _Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies_ , pages 180–189, Portland, Oregon, USA. Association for Computational Linguistics.
  * Yuan et al. (2021) Zheng Yuan, Shiva Taslimipoor, Christopher Davis, and Christopher Byrant. 2021.  Multi-Class Grammatical Error Detection for Correction:A Tale of Two Systems.  In _Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing (EMNLP-2021)_.

##  Appendix A Corpus statistics

| Corpus | # sentences |  $\mu$ sent. length ($\sigma$) |  $\mu$ errors per sentence ($\sigma$)
---|---|---|---|---
Original | FCE-train | 28K | 16 (11) | 1.9 (2.5)
W&I-train | 34K | 18 (12) | 2.0 (2.8)
BEA-dev | 4K | 20 (12) | 1.9 (2.8)
| WikEd (total) | 28M | 22 (12) | 1.6 (1.6)
Processed | FCE-train | 626 | 25 (14) | 1.1 (0.3)
W&I-train | 1310 | 27 (21) | 1.1 (0.3)
BEA-dev | 142 | 26 (19) | 1.1 (0.3)
| WikEd-train | 1936 | 24 (12) | 1.0 (0.2)
| WikEd-dev | 5839 | 23 (11) | 1.0 (0.2)
Table 3: Corpus statistics.

##  Appendix B Marvin & Linzen statistics

Statistics per construction.

Construction | # sentences |  $\mu$ sent. length ($\sigma$)
---|---|---
Simple agr. | 280 | 4.57 (0.49)
In sent. comp. | 3360 | 7.57 (0.49)
Across prep. | 44800 | 8.85 (1.17)
Across subj. rel. | 22400 | 8.77 (0.64)
Short VP coord | 1680 | 7.14 (0.64)
Long VP coord | 800 | 14.40 (0.49)
Across obj. rel. | 44800 | 9.18 (0.86)
Across obj. rel. (no comp) | 44800 | 8.18 (0.86)
Within obj. rel. | 44800 | 9.18 (0.86)
Within obj. rel. (no comp) | 44800 | 8.18 (0.86)
Table 4: Details for the evaluation stimuli from Marvin and Linzen (2018).

##  Appendix C Results for probes trained on w&i-fce

Model |  10.111.11 |  20.221.22  |  30.331.33  |  40.441.44  |  50.551.55  |  60.661.66  |  70.771.77  |  80.881.88  |  90.991.99  |  10.111.110  |  10.111.111  |  10.111.112
---|---|---|---|---|---|---|---|---|---|---|---|---
bert |  00.001.00.40  |  00.001.00.43  |  00.001.00.43  |  00.001.00.44  |  00.001.00.48  |  00.001.00.50  |  00.001.00.61  |  00.001.00.74  |  00.001.00.84  |  00.001.00.82  |  00.001.00.84  |  00.001.00.89
electra |  00.001.00.43  |  00.001.00.46  |  00.001.00.47  |  00.001.00.50  |  00.001.00.50  |  00.001.00.53  |  00.001.00.76  |  00.001.00.88  |  00.001.00.95  |  00.001.00.88  |  00.001.00.90  |  00.001.00.91
roberta |  00.001.00.51  |  00.001.00.50  |  00.001.00.66  |  00.001.00.67  |  00.001.00.80  |  00.001.00.71  |  00.001.00.65  |  00.001.00.69  |  00.001.00.73  |  00.001.00.74  |  00.001.00.74  |  00.001.00.72
gpt-2 |  00.001.00.37  |  00.001.00.35  |  00.001.00.37  |  00.001.00.37  |  00.001.00.36  |  00.001.00.45  |  00.001.00.43  |  00.001.00.34  |  00.001.00.38  |  00.001.00.31  |  00.001.00.21  |  00.001.00.25
xlnet-bi |  00.001.00.39  |  00.001.00.44  |  00.001.00.46  |  00.001.00.50  |  00.001.00.59  |  00.001.00.59  |  00.001.00.52  |  00.001.00.48  |  00.001.00.47  |  00.001.00.48  |  00.001.00.45  |  00.001.00.41
xlnet-uni |  00.001.00.31  |  00.001.00.43  |  00.001.00.48  |  00.001.00.46  |  00.001.00.44  |  00.001.00.39  |  00.001.00.36  |  00.001.00.35  |  00.001.00.33  |  00.001.00.34  |  00.001.00.29  |  00.001.00.31
Table 5: F1 scores for probes trained on contextual representations at
different layers from bert, electra, roberta, xlnet with bidirectional
decoding, xlnet with unidirectional decoding, and gpt-2. Probes were trained
on learner data described in section 3.1, and evaluated on the Marvin and
Linzen (2018) stimuli (section 3.3).

##  Appendix D Results for probes trained on wiked-s

Model |  10.111.11  |  20.221.22  |  30.331.33  |  40.441.44  |  50.551.55  |  60.661.66  |  70.771.77  |  80.881.88  |  90.991.99  |  10.111.110  |  10.111.111  |  10.111.112
---|---|---|---|---|---|---|---|---|---|---|---|---
bert |  00.001.00.39  |  00.001.00.39  |  00.001.00.40  |  00.001.00.43  |  00.001.00.43  |  00.001.00.47  |  00.001.00.53  |  00.001.00.62  |  00.001.00.68  |  00.001.00.65  |  00.001.00.65  |  00.001.00.73
electra |  00.001.00.42  |  00.001.00.43  |  00.001.00.45  |  00.001.00.48  |  00.001.00.48  |  00.001.00.49  |  00.001.00.66  |  00.001.00.83  |  00.001.00.87  |  00.001.00.89  |  00.001.00.88  |  00.001.00.84
roberta |  00.001.00.46  |  00.001.00.48  |  00.001.00.60  |  00.001.00.62  |  00.001.00.70  |  00.001.00.65  |  00.001.00.59  |  00.001.00.63  |  00.001.00.70  |  00.001.00.65  |  00.001.00.68  |  00.001.00.69
gpt-2 |  00.001.00.37  |  00.001.00.39  |  00.001.00.42  |  00.001.00.44  |  00.001.00.45  |  00.001.00.48  |  00.001.00.44  |  00.001.00.41  |  00.001.00.40  |  00.001.00.38  |  00.001.00.37  |  00.001.00.34
xlnet-uni |  00.001.00.38  |  00.001.00.41  |  00.001.00.43  |  00.001.00.43  |  00.001.00.42  |  00.001.00.38  |  00.001.00.35  |  00.001.00.34  |  00.001.00.35  |  00.001.00.35  |  00.001.00.33  |  00.001.00.35
xlnet-bi |  00.001.00.40  |  00.001.00.43  |  00.001.00.45  |  00.001.00.48  |  00.001.00.51  |  00.001.00.51  |  00.001.00.48  |  00.001.00.44  |  00.001.00.43  |  00.001.00.43  |  00.001.00.42  |  00.001.00.43
Table 6: F1 scores for probes trained on contextual representations at
different layers from bert, electra, roberta, xlnet with bidirectional
decoding, xlnet with unidirectional decoding, and gpt-2. Probes were trained
on wikipedia data described in section 3.2, and evaluated on the Marvin and
Linzen (2018) stimuli (section 3.3).

##  Appendix E Results across syntactic constructions

Figure 4: F1 scores for probes trained the w&i-fce and training set. The
probes are evaluated on m&l stimuli. The verb-only baseline is illustrated
using grey horizontal lines. Figure 5: F1 scores for probes trained the wiked
and training set. The probes are evaluated on m&l stimuli. The verb-only
baseline is illustrated using grey horizontal lines.


# Current-induced H-shaped-skyrmion creation and their dynamics in the helical
phase

R. Knapman1, D. R. Rodrigues1, J. Masell2, and K. Everschor-Sitte3 1 Institute
of Physics, Johannes Gutenberg University Mainz, 55128 Mainz, Germany  2 RIKEN
Center for Emergent Matter Science (CEMS), Wako 351-0198, Japan  3 Faculty of
Physics, University of Duisburg-Essen, 47057 Duisburg, Germany  knapman@uni-
mainz.de

###### Abstract

Inevitable for the basic principles of skyrmion racetrack-like applications is
not only their confined motion along one-dimensional channels but also their
controlled creation and annihilation. Helical magnets have been suggested to
naturally confine the motion of skyrmions along the tracks formed by the
helices, which also allow for high-speed skyrmion motion. We propose a
protocol to create topological magnetic structures in a helical background. We
furthermore analyse the stability and current-driven motion of the skyrmions
in a helical background with in-plane uniaxial anisotropy fixing the
orientation of the helices.

Keywords: skyrmions, racetrack, non-volatile memory, helimagnets

##  1 Introduction

Magnetic skyrmions are topologically nontrivial textures that arise in
magnetic materials. [1, 2, 3, 4] They are typically considered to be vortex-
like solitons in a ferromagnetic background. Owing to their small size and
high stability against thermal fluctuations, they are considered as possible
candidates for information units with applications in devices such as logic
gates [5, 6] and racetrack memory devices. [7, 8, 9] The latter have gained
considerable attention as candidates for low-input non-volatile memory. A
major challenge for skyrmion-based racetracks is that the magnetic whirls
typically not only move along the tracks but additionally experience current-
induced forces in a perpendicular direction. [10, 11] This skyrmion Hall
effect [12, 13] might lead to the annihilation of a skyrmion as it moves
towards the edges of the racetrack. Several studies have been proposed to
suppress or eliminate the skyrmion Hall effect which basically can be grouped
into two categories. One exploits the coupling between two skyrmions of
opposite polarity, as they arise for example in (artificial) antiferromagnets
[14, 15, 16], such that the Magnus forces acting on the individual skyrmions
effectively cancel each other. The other one exploits an additional symmetry
breaking which eliminates the skyrmion Hall effect along certain directions or
by engineered material properties. [17, 18, 19, 20, 21, 22] Another approach
to avoid the annihilation of the skyrmion at the edge of the nanowire is to
engineer the potential at the edges. The latter creates a repulsive force that
counteracts the current-induced perpendicular force on the skyrmion. [23, 24,
25, 26, 27, 28, 29, 30].

The essential goal for racetrack memories is to devise a mechanism to
constrain skyrmions to certain well-defined lanes. While all of the suggested
methods above counteract the Magnus force in some form, they often require
fine-tuned material engineering and restrict the motion of the skyrmions along
tracks only by means of geometrical confinement of the nanowire. A different
suggestion to obtain a skyrmion transport along 1D channels, which so far has
not received much attention, is to exploit the tracks that are naturally
formed in helical states. [31, 32, 33]. In particular, it has been proposed
that currents perpendicular to these formed tracks allow for high-speed
skyrmion motion in materials with low damping. Furthermore, skyrmions and
anti-skyrmions can coexist in helical magnets, and move along different tracks
in the helical background. Any application, however, first requires the
controlled creation of the topological objects.

In this work, we propose a protocol to create skyrmions in the helical
background by electrical means. We show by numerical simulations that
skyrmion-antiskyrmion pairs are nucleated at impurities, exploiting a concept
that previously has been applied for ferromagnetic states. [34, 35, 36, 37,
38]. The generated topological structures immediately enter “lanes” that act
as a natural confining potential along which the (anti-)skyrmions can be
driven.

The manuscript is organized as follows. In Sec. 2, we consider the helical
state in a thin film helimagnet with an in-plane easy axis. We discuss the
helical ground state and different skyrmion solutions on top of the helical
ground state. In Sec. 3 we analyse the current-induced dynamics of helical
states and derive current-driven instabilities. Furthermore, we discuss the
current-driven motion of skyrmions in the helical background. In Sec. 4 we
present an all-electrical protocol for creating skyrmions in the helical
background. The manuscript ends with a discussion and conclusion in Sec. 5.

##  2 Topological excitations in helical states

We consider a model for a smooth thin film helimagnet with an in-plane easy
axis. The energy functional for this system is given by

$E[{\bm{m}}({\bm{r}})]=\int\rmd^{2}r\,\left[\frac{J}{2}(\nabla{\bm{m}})^{2}+D{\bm{m}}\cdot\left(\hat{{\bm{y}}}\times\partial_{x}{\bm{m}}-\hat{{\bm{x}}}\times\partial_{y}{\bm{m}}\right)-K({\bm{m}}\cdot\hat{{\bm{y}}})^{2}\right],$
(1)

where ${\bm{m}}({\bm{r}})$ characterizes the normalized magnetization at
position ${\bm{r}}=(x,y)$. The first term corresponds to the exchange
interaction with strength $J$. The second term is the chiral interaction
corresponding to an interfacial Dzyaloshinskii-Moriya interaction (DMI) with
strength $D$. The third term is the easy axis anisotropy along the in-plane
direction $\hat{{\bm{y}}}$. By choosing appropriate units Eq. (1) can be
transformed to a single-parameter theory. For this, we reparametrize energy
and length by $J$ and $J/D$, respectively, to obtain

$\displaystyle
E[{\bm{m}}({\bm{r}})]=\int\rmd^{2}r\,\left[\frac{1}{2}\left(\partial_{x}{\bm{m}}-\hat{{\bm{y}}}\times{\bm{m}}\right)^{2}+\frac{1}{2}\left(\partial_{y}{\bm{m}}+\hat{{\bm{x}}}\times{\bm{m}}\right)^{2}+\frac{1}{2}m_{x}^{2}\right.$
(2) $\displaystyle\left.+\left(\frac{1}{2}-\kappa\right)m_{y}^{2}\right],$

with the effective parameter $\kappa=JK/D^{2}$. Note that we have rewritten
the expression in such a way that the squares are completed. This
representation shows the chirality induced by the Néel DMI and the existence
of an effective anisotropy. For large
$\kappa>\kappa_{\mathrm{c}}\approx\pi^{2}/8\approx 1.23$ the model is
dominated by the uniaxial anisotropy and the ground state corresponds to
ferromagnetic domains oriented along $\pm\hat{{\bm{y}}}$. In the following, we
will focus on the region $\kappa<\kappa_{\mathrm{c}}$ where the ground state
is a helical configuration.

Figure 1: (Meta-) stable magnetic states in helimagnets. a)-c) show the
magnetic configurations of a) the helical ground state, b) for the H-shaped
skyrmion (HSk) and c) for the interstitial skyrmion (iSk) for $\kappa=0$. The
out of plane component is shown by the contrast $m_{z}=1$ ($m_{z}=-1$)
corresponds to white (black). The different colours encode the in-plane
component of the magnetization as shown by the colour wheel on the right and
indicated additionally by the small white arrows. d) Rescaled energy as a
function of the effective anisotropy $\kappa$ for the two different metastable
localized topological states and their skyrmion number density
$\rho_{\mathrm{sk}}$ distribution (encoded in colour) is shown in e) and f),
respectively. The white arrows indicate the in-plane component of the
magnetization at this point, to better compare with the corresponding panels
above.

###  2.1 Helical ground state

For $\kappa<\kappa_{\mathrm{c}}$ the ground state of the model of Eq. (2) is
given by a helical state of the form

${\bm{m}}({\bm{r}})=\cos\theta({\bm{r}}-{\bm{r}}_{0})\,\hat{\bm{q}}+\sin\theta({\bm{r}}-{\bm{r}}_{0})\,\hat{{\bm{z}}}.$
(3)

Here the normalized vector $\hat{{\bm{q}}}$ characterizes the orientation of
the helix, the function $\theta({\bm{r}})$ describes the profile of the helix,
and ${\bm{r}}_{0}$ its spatial displacement. As the considered model, Eq. (2)
is translational invariant, in the following we will choose ${\bm{r}}_{0}=0$.
In the absence of anisotropy ($\kappa=0$), the minimal energy configuration is
described by the profile function

$\theta({\bm{r}})\equiv\theta_{\mathrm{h}}^{0}({\bm{r}})=\hat{{\bm{q}}}\cdot{\bm{r}}.$
(4)

This corresponds to a helix with wavelength $\lambda_{0}=2\pi$ in the rescaled
units. Notice that no orientation of the helix is favoured, i.e.
$\hat{{\bm{q}}}$ is only specified to be in the $xy$ plane. In the presence of
a small easy-axis anisotropy ($0<\kappa<\kappa_{\mathrm{c}}$), the helical
wave vector is favoured to be parallel to the easy axis, i.e.
$\hat{{\bm{q}}}\,||\,\hat{{\bm{y}}}$. In this case, the lanes are oriented
along the $\hat{{\bm{x}}}$-direction, see Fig. 1a) and the profile function
only depends on the $y$-coordinate. 111Note that a perpendicular magnetic
anisotropy favours no particular orientation of the helix, and thus
potentially results in current-driven reorientations of the helix direction at
boundaries and impurities. [39, 40, 41] Furthermore, the helix deforms and its
profile is given by

$\theta({\bm{r}})\equiv\theta_{\mathrm{h}}(y)=\textrm{am}\left[\sqrt{2\kappa
c}\,y,\frac{1}{c}\right],$ (5)

where $\textrm{am}(u,m)$ is the Jacobi amplitude. [31, 42] The constant $c$
characterizes the conserved linear momentum corresponding to the translational
symmetry along $\hat{{\bm{y}}}$-direction.222The conserved linear momentum
$c=(\partial_{y}\theta)^{2}/(2\kappa)+\sin^{2}\theta$ is independent of the
coordinate $y$. It is bound by $1<c<1+1/(2\kappa)$ due to the DMI-induced open
boundary condition $\partial_{y}\theta_{\mathrm{h}}(y)=1$ while enforcing that
at somewhere along the helix the magnetization is in-plane.333The non-
linearity of the solution Eq. 5 is an obstacle to obtain an approximate
analytical expression for the $c$ dependence on $\kappa$ that minimizes the
energy density. The natural wavelength of the distorted helix is
$\lambda_{\mathrm{h}}=4\,\textrm{F}(\pi/2,1/c)/\sqrt{2\kappa c}$, where
$\textrm{F}(\pi/2,1/c)$ is the elliptic integral of the first kind. [31, 42]
As a disclaimer, note that contrary to the ferromagnetic state, the helical
state is very much affected by geometrical and boundary effects as well as
impurities. Thus, the precise helical structure and its wavelength is, in
general, quite complicated for confined samples.

Analytically, we can consider two limits for infinite samples i) $\kappa=0$
and ii) $\kappa\rightarrow\kappa_{\mathrm{c}}$. The limit $\kappa\rightarrow
0$ corresponds to $c\rightarrow\infty$ for which
$\textrm{F}(\pi/2,1/c)\rightarrow\pi/2$ and $\sqrt{2\kappa c}\rightarrow 1$.
With this, one obtains back the helical configuration with the profile
described in Eq. (4). In the limit $\kappa\rightarrow\kappa_{\mathrm{c}}$ the
constant $c$ approaches 1, the wavelength $\lambda_{\mathrm{h}}$ diverges and
the magnetic configuration described by Eq. (3) turns into a single domain
wall with $\theta_{\mathrm{h}}(y)=\arccos\left(\tanh\sqrt{2\kappa}y\right)$.

Our numerical results obtained by simulations with MuMax3 are shown in Fig. 2,
for details see App. 7. In Fig. 2a) we show the wavelength dependence on the
effective parameter $\kappa$ and in Fig. 2b) the profiles of the helices for
different values of $\kappa$. The numerical simulations reproduce the main
qualitative features from the analytics: i) for $\kappa\rightarrow 0$, the
domain wall profile obtains a regular cosine pattern as described by Eq. 4 and
ii) that for $\kappa\rightarrow\kappa_{\mathrm{c}}$, the wavelength diverges
and the profile converges to that of a domain wall.

Figure 2:  Characteristics of the helical state as a function of the effective
parameter $\kappa$; a) The helical wavelength increases as a function of
$\kappa$ . Insets show a section of the helical state for $\kappa=0$ and close
to the critical value $\kappa_{\mathrm{c}}$ indicated by the grey dashed line.
Besides the wavelength also the profile of the helices changes as a function
of $\kappa$ which is shown explicitly in panel b). The magnetization component
along the helical wave vector, here $m_{y}$ changes from a regular cosine
behavior for $\kappa=0$ towards a domain wall-like profile for
$\kappa\rightarrow\kappa_{\mathrm{c}}$.

###  2.2 Skyrmions in the helical ground state

It has been shown that on top of the helical ground state, localized
metastable topological magnetic structures exist [31, 32]. In a ferromagnetic
state, a local configuration with unitary topological winding number

$N_{\mathrm{sk}}=\frac{1}{4\pi}\int_{\Omega}\rmd^{2}r\,\rho_{\mathrm{sk}}\equiv\frac{1}{4\pi}\int_{\Omega}\rmd^{2}r\,{\bm{m}}\cdot\left(\partial_{x}{\bm{m}}\times\partial_{y}{\bm{m}}\right),$
(6)

is typically associated to a magnetic skyrmion. In the helical background
there are two distinct configurations associated with the topological charge
$N_{\mathrm{sk}}=1$: _interstitial skyrmions_ (iSk) and _“H-shaped” skyrmions_
(HSk), see Fig. 1b) and c). The former resemble skyrmions in ferromagnets,
around which the helical background bends. [32] The latter are bound meron
pairs, where each meron has a half-integer topological charge. [31, 32, 38].
Their corresponding skyrmion densities $\rho_{\mathrm{sk}}$ are shown in Fig.
1e) and f). In agreement with Ref. [32], we find that HSk are energetically
favourable compared to iSk, see Fig. 1d).

##  3 Current-driven dynamics in helical states

In the following, we discuss the current-induced dynamics of helical states
subject to spin-transfer torques, [43, 44] which is well described in the
framework of the Landau-Lifshitz-Gilbert (LLG) equation

$\dot{{\bm{m}}}=-{\bm{m}}\times{\bm{B}}_{\mathrm{eff}}-\left({\bm{v}}_{\mathrm{s}}\cdot\nabla\right){\bm{m}}+\alpha{\bm{m}}\times\dot{{\bm{m}}}+\beta{\bm{m}}\times({\bm{v}}_{\mathrm{s}}\cdot\nabla){\bm{m}}.$
(7)

Here, ${\bm{B}}_{\mathrm{eff}}=-\delta E[{\bm{m}}]/\delta{\bm{m}}$ and
$\alpha,\beta$ are the damping parameters. Time has been rescaled by
$\tau=JM_{\mathrm{s}}/\gamma D^{2}$ where $\gamma$ is the (positive)
gyromagnetic ratio and $M_{\mathrm{s}}$ is the saturation magnetization. The
strength of the spin-transfer torque is characterized by the velocity
${\bm{v}}_{\mathrm{s}}$ which is proportional to the electrical current
density ${\bm{j}}_{\mathrm{e}}$ as
${\bm{v}}_{\mathrm{s}}=-{\bm{j}}_{\mathrm{e}}P\mu_{\mathrm{B}}/(e\gamma
D(1+\beta^{2}))$, where $P$ is the polarisation, $\mu_{\mathrm{B}}$ is the
Bohr magneton, and $e>0$ is the electron charge.

To analyze the effects of spin-transfer torques, which couple only to
gradients of the magnetic structure, one has to consider several regimes. In a
real sample, pinning forces favour static equilibrium configurations while
spin-torques inject energy into the system and drive the system away from
equilibrium. Their interplay divides the dynamical phase-space into three
regimes ordered by increasing driving strength in i)
$v_{\mathrm{s}}<v_{\mathrm{pin}}$ the magnetic configuration is static; ii)
$v_{\mathrm{pin}}<v_{\mathrm{s}}<v_{\mathrm{C}}$ the magnetic configuration
evolves controllably; iii) $v_{\mathrm{s}}>v_{\mathrm{C}}$ the magnetic
configuration is unstable and there is no long-range magnetic order. While the
strengths of $v_{\mathrm{pin}}$ and $v_{\mathrm{C}}$ depend on the particular
system, more specifically on the magnetization configuration, geometry and
potential disorder, a few general remarks can be made. In homogeneous systems
an example of dynamics in region (ii) is the steady translation of rigid
configurations, which can be described by a Thiele equation [45, 11]. For non-
homogeneous systems the magnetization dynamics becomes more complex [36, 46,
34, 35, 47, 48] and gives rise to interesting phenomena such as the shedding
of topological textures. [34, 35, 36, 48]

###  3.1 Instabilities of the helical state

We consider first the _pure_ helical state with open boundary conditions in
the absence of disorder and at zero temperature. In this case, the
magnetization gradient only couples to the $\hat{{\bm{y}}}$-component of the
driving. Furthermore, $v_{\mathrm{pin}}=0$, and spin-transfer torques lead to
a global translation of the helices along the $\hat{{\bm{y}}}$-direction. For
drivings above a strength $v_{\mathrm{C}}\equiv v_{\mathrm{W}}$ the helical
state becomes unstable, and the magnetisation may start to precess, leading to
a Walker-breakdown instability [41], similar to the one for a domain
wall.444In the absence of anisotropy $v_{\mathrm{W}}=1$ and the Walker
breakdown of the helix is the transition from driven conical to polarized.

In a typical experiment, however, the helical phase is very sensitive to
geometry, boundary effects and disorder. Therefore, $v_{\mathrm{pin}}$ can
only be determined for the specific considered configuration. Below we will
consider a regime where we prevent the global translation of the helices along
the $\hat{{\bm{y}}}$-direction by pinning the helices on the boundaries, as
the key idea is to work in a regime where only local structures, i.e. the
skyrmions, move along the tracks and not the full helical system, see Sec.
3.2. For such pinned helical states with wave vector along $\hat{{\bm{y}}}$,
the $y$-component of the drive will deform the profile of the helices while
the $x$-component couples with any perturbations of the helical background. It
turns out that this pinned helix regime splits further into two cases, one in
which the applied drive is small in the sense that perturbations only act
locally on a size much smaller than the helical wavelength
$\lambda_{\mathrm{h}}$ and one where perturbations are on the order of
$\lambda_{\mathrm{h}}$. The latter allows to change the structure of the
helical state and produce metastable local excitations such as the HSk and
iSk. We denote the drive above which a local excitation modifies the helical
state by $v_{\mathrm{c}}$.

Analytically we can obtain an estimation of $v_{\mathrm{c}}$ for
$\kappa\lesssim\kappa_{\mathrm{c}}$, where the helical state is distorted in
such a way that it can be interpreted as a sequence of ferromagnetic domains
periodically separated by domain walls, see inset of Fig. 2a) for $\kappa$
close to $\kappa_{\mathrm{c}}$. The lowest excitation modes of such a state
can be interpreted as a local translation of a domain wall with a rigid
profile [49] within the length scale of a helical period to not shift the full
helical state. Therefore, we consider the magnetization locally given in terms
of the spherical angles as

$\displaystyle{\bm{m}}({\bm{r}})=\sin\theta({\bm{r}})\hat{{\bm{z}}}+\cos\theta({\bm{r}})\left[\cos\phi({\bm{r}})\,\hat{{\bm{x}}}+\sin\phi({\bm{r}})\,\hat{{\bm{y}}}\right],$
(8a) $\displaystyle\text{with}\ \
\theta({\bm{r}})=\arccos\left(\tanh\left(\sqrt{2\kappa}(y-y_{0}(x))\right)\right),\
\mathrm{and}\ \ \phi({\bm{r}})=\frac{\pi}{2}+\varphi(x),$ (8b)

in a region $y-y_{0}\ll\lambda_{\mathrm{h}}/2$. Here $y_{0}(x)$ and
$\varphi(x)$ describe the fluctuations along the tracks that might arise due
to some local perturbation in this region. We consider the magnetization to be
static everywhere else corresponding to the helical background. In this
approximation, we claim that the energy associated to perturbations of the
helical background is equivalent to the energy associated to a perturbation of
a domain wall in a confining potential, i.e.
$E_{\mathrm{cp}}[{\bm{m}}({\bm{r}})]\approx E[{\bm{m}}_{DW}({\bm{r}})]+\int
dx\,2y_{0}^{2}/\lambda_{\mathrm{h}}^{2}$, and yields

$E[{\bm{m}}({\bm{r}})]\approx\int
dx\left(\frac{1}{\sqrt{2\kappa}}(\partial_{x}\varphi)^{2}+\sqrt{2\kappa}(\partial_{x}y_{0})^{2}-\frac{\pi}{2}\varphi\partial_{x}y_{0}+\left(\frac{\pi}{2}+\sqrt{2\kappa}\right)\varphi^{2}+\frac{2y_{0}^{2}}{\lambda_{\mathrm{h}}^{2}}\right).$
(8i)

The last two terms correspond to confining potentials that fix the helicity
and position of the domain wall, respectively. Notice that the potential for
$\varphi$ is positive for $\kappa<\kappa_{\mathrm{c}}$, and the constraining
potential for $y_{0}$ vanish as $\kappa$ approaches the critical value.

For a current applied along the $\hat{{\bm{x}}}$-direction and in the absence
of non-adiabatic damping ($\beta=0$), from Eq. (7) and Eq. (8i) we deduce the
linearized equations of motions for $y_{0}$ and $\varphi$,

$\displaystyle\partial_{t}\varphi=\left(v_{x}+\frac{\pi}{2}\right)\partial_{x}\varphi-\sqrt{2\kappa}\partial_{x}^{2}y_{0}+\frac{2}{\lambda_{\mathrm{h}}^{2}}y_{0}-\sqrt{2\kappa}\alpha\partial_{t}y_{0},$
(8ja)
$\displaystyle\partial_{t}y_{0}=\left(v_{x}+\frac{\pi}{2}\right)\partial_{x}y_{0}+\frac{1}{\sqrt{2\kappa}}\partial_{x}^{2}\varphi-\left(\frac{\pi}{2}+\sqrt{2\kappa}\right)\varphi+\frac{1}{\sqrt{2\kappa}}\alpha\partial_{t}\varphi.$
(8jb)

Analysing the stability of the system, in the limit of small damping $\alpha$,
we find that for $\kappa\lesssim\kappa_{\mathrm{c}}$ perturbations in the
system damp out for drives below

$v_{c}^{\kappa\lesssim\kappa_{\mathrm{c}}}\approx\sqrt{\frac{2}{\lambda_{\mathrm{h}}^{2}}+\frac{\pi}{2}+\sqrt{2\kappa}}-\frac{\pi}{2}.$
(8jk)

which in the limit $\kappa=\kappa_{\mathrm{c}}$ yields
$v_{\mathrm{c}}^{\kappa_{\mathrm{c}}}\approx 0.20$. Moreover, in this
approximation, one notices that as $\kappa\rightarrow 0$ the helical state
becomes unstable at any non-zero drive, $|v_{s}|>0$. [41] Above this critical
current, any perturbation tends to grow exponentially. Depending on the
detailed energy landscape, such perturbations span a great variety of possible
magnetic configurations and may allow for the creation of metastable states.
This regime, for example, allows for the creation of HSk or iSk and will be
explored in Sec. 4.

To numerically study the critical current density above which the helical
state significantly deforms, we consider an impurity as a circular region of
radius $R_{\mathrm{i}}$ with an easy-axis anisotropy along $\hat{{\bm{n}}}$
perpendicular to $\hat{{\bm{y}}}$. The energy contribution for the impurity to
the functional in Eq. (2) is given by

$E_{i}[{\bm{m}}({\bm{r}})]=-\int\rmd^{2}r\,\kappa_{\mathrm{i}}(\hat{{\bm{n}}}\cdot{\bm{m}})^{2}\,\Theta\left(R_{\mathrm{i}}^{2}-(x-x_{i})^{2}-(y-y_{i})^{2}\right),$
(8jl)

where $\kappa_{\mathrm{i}}$ is the strength of the local easy-axis anisotropy
along $\hat{{\bm{n}}}$, $\Theta(x)$ is the Heaviside step function being 1 for
$x\geq 0$ and $0$ for $x<0$, the coordinates $x_{i},y_{i}$ are the position of
the inhomogeneity region. The inhomogeneity leads to a local deformation of
the helix in this region, see Fig. 3a) and d). When applying a current, the
deformation enhances and above a certain drive $v_{\mathrm{c}}$ does no longer
allow for a statically stable solution. In Fig. 3 we show the results obtained
with MuMax3 for systematically computing the critical drive $v_{\mathrm{c}}$
as a function of the ratio of the local and global anisotropy strengths
$\kappa_{\mathrm{i}}/\kappa$, the radius $R_{\mathrm{i}}$ and for
$\hat{{\bm{n}}}$ along $\pm\hat{{\bm{x}}}$ and $\pm\hat{{\bm{z}}}$. The
results shown in Fig. 3 are computed for $\kappa=0.31$, $\alpha=0.05$ and
$\beta=0$, for details see App. 7.

Figure 3: Current-induced instability of the helical phase due to an
inhomogeneity. a) and d) show an example magnetisation configuration around
the impurity region for $\hat{{\bm{n}}}=\hat{{\bm{x}}}$ and
$\hat{{\bm{n}}}=\hat{{\bm{z}}}$, respectively. b), c), e), f) show the
critical drives $v_{\mathrm{c}}$ as a function of inhomogeneity radius and
relative anisotropy strength $\kappa_{\mathrm{i}}/\kappa$. For all panels
$\kappa\approx 0.31$, $\alpha=0.05$ and $\beta=0$ and in a) and d) we used
$2R_{\mathrm{i}}=\lambda_{0}$ and $\kappa_{\mathrm{i}}/\kappa=10$. We note
that the graphs e) and f) are identical due to the mirror symmetry along
$\hat{{\bm{x}}}$ for the case of an impurity along $\hat{{\bm{z}}}$.

The presence of the inhomogeneity modifies the magnetization locally as the
magnetization inside the inhomogeneity tends to rotate towards
$\hat{{\bm{n}}}$ see Fig. 3a) and, d). The stronger the inhomogeneity the more
the magnetization aligns with the anisotropy direction of the impurity, and
the domain wall stripes are pushed out of the inhomogeneity region. As the
initial perturbation is a bit stronger for a larger
$\kappa_{\mathrm{i}}/\kappa$, the critical current decreases as a function of
$\kappa_{\mathrm{i}}/\kappa$. The most important factor for the deformation of
the helical state, however, is the size of the inhomogeneity, which
essentially determines the curvature of the domain wall. We find that the
critical drive generally decreases with an increasing size since a bigger
impurity typically produces a large initial perturbation that couples more
strongly to the spin-transfer torque.

For the impurity region with $\hat{\bm{n}}=\hat{\bm{z}}$, the set-up is still
axial symmetric and therefore a current along $+\hat{\bm{x}}$ or
$-\hat{{\bm{x}}}$-direction yields the same critical drive. For
$\hat{\bm{n}}=\hat{\bm{x}}$, the impurity breaks the axial symmetry and it
induces the magnetization to align perpendicularly to the plane defined by the
helical configuration, generating an extra gradient component. Therefore,
reversing the current direction alters the size of the critical current.

###  3.2 Current-Driven Motion of Skyrmions in the Helical Background

A key prerequisite of spintronic-based racetrack memories is the possibility
to move the rigid localized magnetic configurations by applying a current. To
obtain the motion of the skyrmions in the helical background, we consider that
in the current-driven steady motion the magnetization is given by
${\bm{m}}({\bm{r}},t)\equiv{\bm{m}}({\bm{r}}-{\bm{v}}_{\mathrm{s}}t)$. Given
this assumption and projecting Eq. (7) along
${\bm{m}}\times\partial_{x}{\bm{m}}$ and ${\bm{m}}\times\partial_{y}{\bm{m}}$
leads to the Thiele equation [45, 11]

${\bm{\mathcal{G}}}\times({\bm{v}}-{\bm{v}}_{s})+\mathcal{D}(\alpha{\bm{v}}-\beta{\bm{v}}_{\mathrm{s}})-{\bm{F}}={\bm{0}},$
(8jm)

where ${\bm{\mathcal{G}}}=4\pi\hat{{\bm{z}}}$ is the gyrovector,
$\mathcal{D}_{ij}=\int\rmd^{2}x\partial_{i}{\bm{m}}\cdot\partial_{j}{\bm{m}}$
is the dissipative tensor, and
${\bm{F}}=\partial_{x}E[{\bm{m}}({\bm{r}})]\hat{{\bm{x}}}+\partial_{y}E[{\bm{m}}({\bm{r}})]\hat{{\bm{y}}}$
is a force due to space variations of the energy density functional. The first
term in the equation is responsible for the skyrmion Hall angle, it produces a
movement of the skyrmion that is perpendicular to the perturbation. The second
term is a dissipative term whose direction of motion depends on the shape of
the skyrmion. The third term pushes the skyrmion towards a stable position in
the energy landscape. The helical background has a significant impact on the
last two terms by producing a non vanishing ${\bm{F}}$ and deforming the
skyrmion as it is displaced from a path of minimal energy. The helical state
also confines the motion of the skyrmions along the helical tracks, i.e.
${\bm{v}}\parallel\hat{{\bm{x}}}$ in a steady motion. Solving Eq. (8jm) for
the velocity of the skyrmion thus yields for the steady motion

${\bm{v}}=\left(\frac{4\pi({\bm{v}}_{s}\cdot\hat{{\bm{y}}})}{\alpha\mathcal{D}_{xx}}+\frac{\beta}{\alpha}{\bm{v}}_{s}\cdot\frac{(\mathcal{D}_{xx}\hat{{\bm{x}}}+\mathcal{D}_{xy}\hat{{\bm{y}}})}{\mathcal{D}_{xx}}\right)\hat{{\bm{x}}}.$
(8jn)

For small drives where this equation holds, the velocity grows linearly with
the applied current. Note that Eq. (8jn) holds for all metastable magnetic
structures in the helical background, including the HSk and iSk. Furthermore,
in the absence of non-adiabatic spin-transfer torques, i.e. $\beta=0$, only a
current along $\hat{\bm{y}}$ will induce a motion of the skyrmion.

Figure 4: Properties of HSk and iSk subject to an applied drive
$v_{\mathrm{s}}$ along the $\hat{{\bm{y}}}$-direction. a) and b) show the
skyrmion speed $v$ as a function of driving strength $v_{\mathrm{s}}$ and
inverse damping $1/\alpha$, respectively. For small drives the corresponding
relation is linear and becomes sub linear for larger drives. High skyrmion
speeds can be obtained in the regime of small damping and large drives. Note
that because of the HSk being more stable it generally allows for higher
drives than the iSk. For large drives both skyrmions deforms as exemplarily
shown in the insets. The dotted data points have been obtained by numerical
simulations and the cross-shaped data points have been obtained using Eq. 8jn
while micromagnetically computing the dissipative tensor $\mathcal{D}$, the
drive dependence of which is shown in c). For this plot we used $\kappa\approx
0.31$, and $\beta=0$. In a) and c) $\alpha=0.1$, and in b)
$v_{\mathrm{s}}\approx$2.1\text{\times}{10}^{-3}$$.

For larger drives, where the skyrmion tends to move away from the local
minimum given by the helical track, the skyrmion gets deformed with a
deformation that depends on the strength of the applied current. By means of
micromagnetic simulations we show that, in this case, the relation of the
skyrmion speed and the drive becomes sublinear, see Fig. 4a). In Fig. 4 we
show the speed and deformation of the HSk and iSk skyrmion as a function of
the applied drive and inverse damping in the absence of non-adiabatic spin-
transfer torques obtained by micromagnetic simulations as well as the results
of Eq. (8jn) when computing the dissipative tensor $\mathcal{D}$
micromagnetically, shown in panel c). High-speed skyrmion motion arises for
high drives and low damping. We also find that, as expected, the HSk is more
robust, and thus allows for higher velocities compared to the iSk which
annihilates when pushed too much away from its minimal energy position.

##  4 Shedding of Skyrmions in the Helical State

Figure 5: Creation of a Sk/anti-Sk pair in a helical background through the
interplay of an inhomogeneity and an applied current. The top (bottom) row
shows the magnetization configuration (skyrmion number density) around the
impurity at different times during the creation process. Panel e) shows the
created HSK/anti-HSk pair. The colour coding is identical to the one used in
Fig. 1. The material parameters are discussed in App. 7.

For the controllable injection of skyrmions we consider the design with a
localized inhomogeneity in the sample, as described by Eq. (8jl) with
$\hat{\bm{n}}=\hat{\bm{x}}$, $\kappa_{\mathrm{i}}=10\,\kappa$ and
$2R_{\mathrm{i}}/\lambda_{0}=1.0$. Applying a drive along the
$\hat{{\bm{x}}}$-direction above $v_{\mathrm{c}}$ deforms the magnetization
around the impurity, and, due to the conservation of topological charge
$N_{\mathrm{sk}}$, not only creates a HSk but a HSk/anti-HSk pair on opposite
sides of the impurity, see Fig. 5. After the creation of this pair,
unfortunately the magnetization configuration around the impurity is in a
metastable state which does not immediately allow for another pair creation.
Combining this fact with the current-driven skyrmion motion discussed in Sec.
3.2, we developed the following all-electrical three-step protocol for
creating skyrmions in the helical background, shown in Fig. 6:

  * •

Step1: Create a HSk/anti-HSk pair: Starting from the helical state with an
impurity, use a current pulse along $-\hat{{\bm{x}}}$ with strength above the
critical current.

  * •

Step2: Move HSk and anti-HSk away from the impurity region: Apply a current
along $\hat{{\bm{y}}}$ to move the skyrmions. Notice that the HSk and anti-HSk
move in opposite directions.

  * •

Step3: Restore the impurity configuration: Apply a short pulse of current
along $\hat{{\bm{x}}}$ with a strength above the critical current in that
direction.

Once the magnetization at the impurity is reinitiated to its lowest energy
configuration, the protocol can be repeated to create another Sk/anti-Sk pair.
We note that the size and duration of the pulses must be tailored for the size
and strength of the impurity as discussed in Fig. 3 This enhances the control
over the skyrmion creation since different impurities will not be able to
generate and be reset by the same applied current pulses allowing for precise
creation of skyrmions at the desired locations. Furthermore, the skyrmions are
moved with much smaller currents applied along the $\hat{{\bm{y}}}$-direction,
allowing the process of creation and skyrmion transport to be carried out
rather independently from each other.

Figure 6: All electrical three-step HSk creation protocol. a) shows the
applied current pulses as a function of time. The labels A-H mark the times
where we show the corresponding magnetization configurations. The details of
the HSk/anti-HSk creation (A-E) are shown in Fig. 5. In b) we first repeat the
configuration of the HSk/anti-HSk after its creation (E) and then we focus on
the HSk that has been moved away from the impurity region with a pulse in
$\hat{{\bm{y}}}$-direction. Under the influence of such a pulse the anti-HSk
moves in the opposite direction, i.e. out of the focus region here. In c) we
show the result of step 3, the restored impurity (G), which then allows for
the production of another HSk/anti-HSk pair. The material parameters are
discussed in App. 7.

##  5 Discussion and Conclusion

We studied the stability and current-driven dynamics of helical states with
in-plane uniaxial anisotropy in the presence of perturbations and
inhomogeneities. Compared to the ferromagnetic state, the helical state has
more degrees of freedom making its physics richer. Besides naturally providing
tracks for skyrmion motion, advantages of helical phases are that they exist
at room temperature in absence of external magnetic fields, and that they are
very robust to external perturbations. [50, 51]

As a central result, we developed a theory to explain for which current ranges
it is possible to produce and manipulate metastable topological structures in
the helical background. Based on these findings we developed an all-electrical
three-step protocol that independently allows for the controlled creation and
the controllable motion of HSks. Our results obtained by micromagnetic
simulations are supported by analytical calculations where the latter are
mainly performed in the limits of small or high in-plane anisotropy. For
simplicity, we neglected stray fields in our calculations. Generally, we
expect their influence not to be too important, as the helical state itself
has zero net magnetization. We expect that their main effect is to distort the
helical profile mildly, and enhance the stability of the HSk and iSk, as
magneto-static interactions favour the formation of twisted structures.
Furthermore, our numerical results were obtained in the absence of damping-
like spin-transfer torques, i.e. $\beta=0$. For $\beta\neq 0$ we expect the
current to couples more effectively to the magnetization and lead to lower
critical currents $v_{\mathrm{c}}$ and higher skyrmion speeds, similar as in
Ref. [48]. Also, all our simulations are performed at zero temperature.
Temperature induces perturbations in the well-ordered states, and thus might
alter the stability of the twisted magnetic structures as well as lower the
critical current densities.

As a final remark, in the absence of a magnetic field, the helical background
does not favour HSk over anti-HSks. During the three-step protocol, the anti-
HSk was not in our focus, as the idea was to show a protocol to create
multiple HSks for a racetrack-like device. But since HSk/anti-HSk pairs are
created on the opposite side of the impurity and moved away from the impurity
in opposite directions, this protocol allows for information to be created and
moved along two directions simultaneously, going even beyond the concept of a
HSk-based helical magnetization racetrack memory.

##  6 Acknowlegdments

We thank O. Gomonay for fruitful discussions and J. Sinova for the constant
great support. We acknowledge funding from the German Research Foundation
(DFG) under Project Nos. 403233384 (SPP Skyrmionics), 320163632 (Emmy Noether)
and 268565370 (TRR 173, project B12). R. K. is supported by a scholarship from
the Studienstiftung des deutschen Volkes. J. M. is supported by JSPS (project
No.19F19815) and the Alexander von Humboldt foundation.

##  7 Supplementary Information

The micromagnetic simulations were performed with MuMax3 [52]. For all
simulations we used
$M_{\mathrm{s}}=$10^{6}\text{\,}\mathrm{A}\text{\,}{\mathrm{m}}^{-1}$$, spin
polarization $P=1$, $J=$20\text{\,}\mathrm{pJ}\text{\,}{\mathrm{m}}^{-1}$$ and
$D\approx$2.55\text{\times}{10}^{-3}\text{\,}\mathrm{J}\text{\,}{\mathrm{m}}^{-2}$$,
yielding a pure helical wave length
($K=$0\text{\,}\mathrm{J}\text{\,}{\mathrm{m}}^{-3}$$) of $\lambda_{0}=2\pi
J/D\approx$49.3\text{\,}\mathrm{nm}$$ and a characteristic time scale of
$\tau=JM_{s}/\gamma D^{2}\approx$0.0175\text{\,}\mathrm{ns}$$. The choice of
the DMI strength is motivated by choosing $D=2D_{\mathrm{c}}$, where
$D_{\mathrm{c}}=4\sqrt{JK/2}/\pi$, [53] i.e. a regime where we found the
skyrmions to be metastable. To obtain
$D\approx$2.55\text{\times}{10}^{-3}\text{\,}\mathrm{J}\text{\,}{\mathrm{m}}^{-2}$$
we used $K=$10^{5}\text{\,}\mathrm{J}\text{\,}{\mathrm{m}}^{-3}$$ for which we
obtain an effective parameter of $\kappa=JK/D^{2}\approx 0.31$. The effective
parameter $\kappa$ is only ever varied by varying $K$, and unless otherwise
specified, $K=$10^{5}\text{\,}\mathrm{J}\text{\,}{\mathrm{m}}^{-3}$$.
Furthermore, we chose the damping constants $\alpha=10^{-3}$ and $\beta=0$
unless otherwise specified. Note that we made sure to adjust the system size
to allow for the natural helical solution avoiding geometrical influence of
finite size samples.

To pin the helical state in the simulations with applied current, the spins at
the top and bottom of the system were frozen. To compensate for current-
induced effects at the sample boundaries, we suppressed the spin dynamics at
the boundary by setting a high damping constant at the left and right ends of
the system, $\alpha=5$.

The proportionality factor between the drive velocity ${\bm{v}}_{\mathrm{s}}$
and the electrical current density is $P\mu_{B}/(e\gamma
D(1+\beta^{2}))\approx$1.29\cdot
10^{-13}\text{\,}{\mathrm{m}}^{2}\text{\,}{\mathrm{A}}^{-1}$$. A typical used
drive value of $v_{\mathrm{s}}=0.025$ to move the skyrmions corresponds to
$j_{e}\approx$1.94\cdot
10^{11}\text{\,}\mathrm{A}\text{\,}{\mathrm{m}}^{-2}$$, and
$v_{\mathrm{c}}=0.25$ to create the skyrmions corresponds to
$j_{c}\approx$1.94\cdot
10^{12}\text{\,}\mathrm{A}\text{\,}{\mathrm{m}}^{-2}$$. And a dimensionless
skyrmion speed of $v=0.2$ corresponds to the dimensionful skyrmion speed
$v_{\mathrm{d}}=\frac{\gamma
D}{M_{\mathrm{s}}}v\approx$89.7\text{\,}\mathrm{m}\text{\,}{\mathrm{s}}^{-1}$$.

##  8 References

## References

  * [1] Nagaosa N and Tokura Y 2013 Nature Nanotechnology 8 899–911 ISSN 17483395 URL http://www.nature.com/articles/nnano.2013.243
  * [2] Fert A, Reyren N and Cros V 2017 Nature Reviews Materials 2 17031 ISSN 2058-8437 URL http://www.nature.com/articles/natrevmats201731
  * [3] Everschor-Sitte K, Masell J, Reeve R M and Kläui M 2018 Journal of Applied Physics 124 240901 ISSN 0021-8979 URL http://aip.scitation.org/doi/10.1063/1.5048972
  * [4] Back C, Cros V, Ebert H, Everschor-Sitte K, Fert A, Garst M, Ma T, Mankovsky S, Monchesky T L, Mostovoy M, Nagaosa N, P Parkin S S, Pfleiderer C, Reyren N, Rosch A, Taguchi Y, Tokura Y, von Bergmann K, Zang J, Parkin S S P, Pfleiderer C, Reyren N, Rosch A, Taguchi Y, Tokura Y, von Bergmann K and Zang J 2020 Journal of Physics D: Applied Physics J. Phys. D: Appl. Phys 53 363001 URL https://doi.org/10.1088/1361-6463/ab8418
  * [5] Zhang X, Ezawa M and Zhou Y 2015 Scientific Reports 5 9400 ISSN 2045-2322 URL http://www.nature.com/articles/srep09400
  * [6] Luo S, Song M, Li X, Zhang Y, Hong J, Yang X, Zou X, Xu N and You L 2018 Nano Letters 18 1180–1184 ISSN 1530-6984 URL https://pubs.acs.org/doi/10.1021/acs.nanolett.7b04722
  * [7] Fert A, Cros V and Sampaio J 2013 Nature Nanotechnology 8 152–156 ISSN 17483395 URL http://www.nature.com/articles/nnano.2013.29
  * [8] Tomasello R, Martinez E, Zivieri R, Torres L, Carpentieri M and Finocchio G 2014 Scientific Reports 4 6784 ISSN 20452322 (Preprint 1409.6491)
  * [9] Lee K, Han D S and Kläui M 2021 Chiral magnetic domain wall and skyrmion memory devices Emerging Non-volatile Memory Technologies (Springer) pp 175–201
  * [10] Jonietz F, Muhlbauer S, Pfleiderer C, Neubauer A, Munzer W, Bauer A, Adams T, Georgii R, Boni P, Duine R A, Everschor K, Garst M and Rosch A 2010 Science 330 1648–1651 ISSN 0036-8075 URL http://www.sciencemag.org/cgi/doi/10.1126/science.1195709https://www.sciencemag.org/lookup/doi/10.1126/science.1195709
  * [11] Everschor K, Garst M, Duine R A and Rosch A 2011 Physical Review B 84 064401 ISSN 1098-0121 URL https://link.aps.org/doi/10.1103/PhysRevB.84.064401
  * [12] Litzius K, Lemesh I, Krüger B, Bassirian P, Caretta L, Richter K, Büttner F, Sato K, Tretiakov O A, Förster J, Reeve R M, Weigand M, Bykova I, Stoll H, Schütz G, Beach G S D and Kläui M 2017 Nature Physics 13 170–175 ISSN 1745-2473 URL http://www.nature.com/articles/nphys4000
  * [13] Jiang W, Chen G, Liu K, Zang J, te Velthuis S G and Hoffmann A 2017 Physics Reports 704 1–49 ISSN 03701573 URL https://www.sciencedirect.com/science/article/pii/S0370157317302934?via%3Dihubhttps://linkinghub.elsevier.com/retrieve/pii/S0370157317302934
  * [14] Zhang X, Zhou Y and Ezawa M 2016 Scientific Reports 6 24795 ISSN 20452322 (Preprint 1504.01198)
  * [15] Barker J and Tretiakov O A 2016 Physical Review Letters 116 147203 ISSN 0031-9007 URL https://link.aps.org/doi/10.1103/PhysRevLett.116.147203
  * [16] Zhang X, Zhou Y and Ezawa M 2016 Nature Communications 7 10293 ISSN 2041-1723 URL http://www.nature.com/doifinder/10.1038/ncomms10293
  * [17] Hoffmann M, Zimmermann B, Müller G P, Schürhoff D, Kiselev N S, Melcher C and Blügel S 2017 Nature Communications 8 308 ISSN 2041-1723 URL http://www.nature.com/articles/s41467-017-00313-0
  * [18] Huang S, Zhou C, Chen G, Shen H, Schmid A K, Liu K and Wu Y 2017 Physical Review B 96 144412 ISSN 2469-9950 URL https://link.aps.org/doi/10.1103/PhysRevB.96.144412
  * [19] Kim K W, Moon K W, Kerber N, Nothhelfer J and Everschor-Sitte K 2018 Physical Review B 97 224427 ISSN 2469-9950 URL https://link.aps.org/doi/10.1103/PhysRevB.97.224427
  * [20] Göbel B, Mook A, Henk J, Mertig I and Tretiakov O A 2019 Physical Review B 99 060407 ISSN 2469-9950 URL https://link.aps.org/doi/10.1103/PhysRevB.99.060407
  * [21] Zarzuela R, Bharadwaj V K, Kim K w, Sinova J and Everschor-Sitte K 2020 Physical Review B 101 054405 ISSN 2469-9950 URL https://link.aps.org/doi/10.1103/PhysRevB.101.054405
  * [22] Zhang X, Xia J, Shen L, Ezawa M, Tretiakov O A, Zhao G, Liu X and Zhou Y 2020 Physical Review B 101 1–14 ISSN 24699969
  * [23] Zhang X, Zhao G P, Fangohr H, Liu J P, Xia W X, Xia J and Morvan F J 2015 Scientific Reports 5 7643 ISSN 2045-2322 (Preprint 1403.7283) URL http://www.nature.com/articles/srep07643
  * [24] Purnama I, Gan W L, Wong D W and Lew W S 2015 Scientific Reports 5 10620 ISSN 2045-2322 URL http://www.nature.com/articles/srep10620
  * [25] Müller J 2017 New Journal of Physics 19 025002 ISSN 1367-2630 (Preprint 1704.05757) URL http://stacks.iop.org/1367-2630/19/i=2/a=025002?key=crossref.abd24846f26363c5230044ff8e42ae2b
  * [26] Lai P, Zhao G P, Tang H, Ran N, Wu S Q, Xia J, Zhang X and Zhou Y 2017 Scientific Reports 7 45330 ISSN 2045-2322 URL http://www.nature.com/articles/srep45330
  * [27] Tomasello R, Komineas S, Siracusano G, Carpentieri M and Finocchio G 2018 Physical Review B 98 024421 ISSN 2469-9950 URL https://link.aps.org/doi/10.1103/PhysRevB.98.024421
  * [28] Fook H T, Gan W L, Purnama I and Lew W S 2015 IEEE Transactions on Magnetics 51 1–4
  * [29] Song K M, Jeong J S, Pan B, Zhang X, Xia J, Cha S, Park T E, Kim K, Finizio S, Raabe J, Chang J, Zhou Y, Zhao W, Kang W, Ju H and Woo S 2020 Nature Electronics 3 148–155 ISSN 2520-1131 URL http://www.nature.com/articles/s41928-020-0385-0
  * [30] Göbel B and Mertig I 2021 Scientific Reports 11 3020 ISSN 2045-2322 URL https://doi.org/10.1038/s41598-021-81992-0http://www.nature.com/articles/s41598-021-81992-0
  * [31] Ezawa M 2011 Physical Review B 83 100408 ISSN 10980121
  * [32] Müller J, Rajeswari J, Huang P, Murooka Y, Rønnow H M, Carbone F and Rosch A 2017 Physical Review Letters 119 137201 ISSN 0031-9007 URL https://link.aps.org/doi/10.1103/PhysRevLett.119.137201
  * [33] Stepanova M, Masell J, Lysne E, Schoenherr P, Köhler L, Qaiumzadeh A, Kanazawa N, Rosch A, Tokura Y, Brataas A, Garst M and Meier D 2021 1–10 (Preprint 2103.14449) URL http://arxiv.org/abs/2103.14449
  * [34] Sitte M, Everschor-Sitte K, Valet T, Rodrigues D R, Sinova J and Abanov A 2016 Physical Review B 94 064422 ISSN 24699969 (Preprint 1607.03336)
  * [35] Müller J, Rosch A and Garst M 2016 New Journal of Physics 18 065006 ISSN 1367-2630 URL http://stacks.iop.org/1367-2630/18/i=6/a=065006?key=crossref.befe24f2f8d0d953b85156f18feeaf32
  * [36] Everschor-Sitte K, Sitte M, Valet T, Abanov A and Sinova J 2017 New Journal of Physics 19 092001 ISSN 1367-2630 URL http://stacks.iop.org/1367-2630/19/i=9/a=092001?key=crossref.e2551bf2f58df6de92422e2ffda1dc44https://iopscience.iop.org/article/10.1088/1367-2630/aa8569
  * [37] Stier M, Häusler W, Posske T, Gurski G and Thorwart M 2017 Physical Review Letters 118 267203 ISSN 0031-9007 (Preprint 1701.07256) URL http://link.aps.org/doi/10.1103/PhysRevLett.118.267203
  * [38] Gao N, Je S G, Im M Y, Choi J W, Yang M, Li Q, Wang T Y, Lee S, Han H S, Lee K S, Chao W, Hwang C, Li J and Qiu Z Q 2019 Nature Communications 10 1–9 ISSN 20411723 URL https://doi.org/10.1038/s41467-019-13642-z
  * [39] Everschor K 2012 Current-induced dynamics of chiral magnetic structures Ph.D. thesis Universität zu Köln URL https://kups.ub.uni-koeln.de/4811/
  * [40] Hals K M D and Everschor-Sitte K 2019 Physical Review B 99 104422 ISSN 2469-9950 URL https://link.aps.org/doi/10.1103/PhysRevB.99.104422
  * [41] Masell J, Yu X, Kanazawa N, Tokura Y and Nagaosa N 2020 Physical Review B 102 180402
  * [42] Abramowitz M, Stegun I A et al. 1972 Handbook of mathematical functions: with formulas, graphs, and mathematical tables vol 55 (National bureau of standards Washington, DC)
  * [43] Gilbert T 2004 IEEE Transactions on Magnetics 40 3443–3449 ISSN 0018-9464 URL http://ieeexplore.ieee.org/document/1353448/
  * [44] Li Z and Zhang S 2004 Physical Review Letters 92 207203 ISSN 00319007
  * [45] Thiele A A 1973 Physical Review Letters 30 230–233 ISSN 0031-9007 URL https://link.aps.org/doi/10.1103/PhysRevB.7.391https://link.aps.org/doi/10.1103/PhysRevLett.30.230
  * [46] Tatara G and Kohno H 2004 Physical Review Letters 92 086601 ISSN 10797114
  * [47] Masell J, Rodrigues D R, McKeever B F and Everschor-Sitte K 2020 Physical Review B 101 214428 ISSN 2469-9950 URL https://link.aps.org/doi/10.1103/PhysRevB.101.214428
  * [48] Rodrigues D R, Sommer N and Everschor-Sitte K 2020 Physical Review B 101 224410 ISSN 0031-899X (Preprint 2004.07546) URL https://doi.org/10.1103/PhysRevB.101.224410
  * [49] Rodrigues D R, Abanov A, Sinova J and Everschor-Sitte K 2018 Physical Review B 97 134414 ISSN 2469-9950 URL https://link.aps.org/doi/10.1103/PhysRevB.97.134414
  * [50] Adams T, Chacon A, Wagner M, Bauer A, Brandl G, Pedersen B, Berger H, Lemmens P and Pfleiderer C 2012 Physical Review Letters 108 1–5 ISSN 00319007
  * [51] Zhang S, Stasinopoulos I, Lancaster T, Xiao F, Bauer A, Rucker F, Baker A, Figueroa A, Salman Z, Pratt F et al. 2017 Scientific reports 7 1–10
  * [52] Vansteenkiste A, Leliaert J, Dvornik M, Helsen M, Garcia-Sanchez F and Van Waeyenberge B 2014 AIP Advances 4 0–22 ISSN 21583226
  * [53] Rohart S and Thiaville A 2013 Physical Review B 88 184422 ISSN 1098-0121 URL https://link.aps.org/doi/10.1103/PhysRevB.88.184422