# Beyond Hard Samples: Robust and Effective Grammatical Error Correction with
Cycle Self-Augmenting

Zecheng Tang Kaiqi Feng Juntao Li Min Zhang
School of Computer Science and Technology, Soochow University
{zctang<EMAIL_ADDRESS><EMAIL_ADDRESS>Corresponding
author.

###### Abstract

Recent studies have revealed that grammatical error correction methods in the
sequence-to-sequence paradigm are vulnerable to adversarial attack, and simply
utilizing adversarial examples in the pre-training or post-training process
can significantly enhance the robustness of GEC models to certain types of
attack without suffering too much performance loss on clean data. In this
paper, we further conduct a thorough robustness evaluation of cutting-edge GEC
methods for four different types of adversarial attacks and propose a simple
yet very effective Cycle Self-Augmenting (CSA) method accordingly. By
leveraging the augmenting data from the GEC models themselves in the post-
training process and introducing regularization data for cycle training, our
proposed method can effectively improve the model robustness of well-trained
GEC models with only a few more training epochs as an extra cost. More
concretely, further training on the regularization data can prevent the GEC
models from over-fitting on easy-to-learn samples and thus can improve the
generalization capability and robustness towards unseen data (adversarial
noise/samples). Meanwhile, the self-augmented data can provide more high-
quality pseudo pairs to improve model performance on the original testing
data. Experiments on four benchmark datasets and seven strong models indicate
that our proposed training method can significantly enhance the robustness of
four types of attacks without using purposely built adversarial examples in
training. Evaluation results on clean data further confirm that our proposed
CSA method significantly improves the performance of four baselines and yields
nearly comparable results with other state-of-the-art models. Our code is
available at https://github.com/ZetangForward/CSA-GEC.

##  1 Introduction

Grammatical error correction (GEC) is one of the most essential application
tasks in the NLP community for its crucial values in many scenarios including,
but not limited to, writing assistant (Napoles et al., 2019; Fitria, 2021),
automatic speech recognition (Karat et al., 1999; Namazifar et al., 2021; Zhao
et al., 2021; Wang et al., 2021; Zhang et al., 2021), information retrieval
(Gao et al., 2010; Duan & Hsu, 2011; Hagen et al., 2017; Zhuang & Zuccon,
2021), which mainly aims to detect and correct various textual errors, such as
spelling, punctuation, grammatical, word choice, and other article mistakes
(Wang et al., 2020). Existing solutions to tackle this task can be roughly
divided into two categories, i.e., sequence-to-sequence generation (Seq2Seq)
(Ji et al., 2017; Chollampatt & Ng, 2018) and sequence-to-editing (Seq2Edits)
(Stahlberg & Kumar, 2020; Awasthi et al., 2019; Li & Shi, 2021). The former
group performs the translation from ungrammatical sentences to the
corresponding error-free sentences, while the latter introduces tagging or
sequence labeling to merely edit a small proportion of the input sentences,
remaining the rest part unchanged.

With the well-tested encoder-decoder framework (Sutskever et al., 2014;
Vaswani et al., 2017) as the backbone, GEC methods in the Seq2Seq paradigm can
achieve promising performance but is sensitive to the quality and scale of
training data. Thus, many recent works have studied the problem of
automatically obtaining high-quality paired data to compensate for the lack of
human-labeled data pairs (Zhao et al., 2019; Kiyono et al., 2019; Yasunaga et
al., 2021). As for the Seq2Edits group, it generally achieves a faster
inference speed than Seq2Seq methods by decoding the target text in parallel
and meanwhile obtaining very competitive performance (Awasthi et al., 2019;
Omelianchuk et al., 2020)111According to the leaderboard of CoNLL-14 shared
task, three of the top-5 best performed GEC systems belong to the Seq2Edits
paradigm. The link is:
http://nlpprogress.com/english/grammatical_error_correction.html. Existing
literature has also revealed that incorporating large-scale pre-trained
language models (PLMs) can enhance the GEC performance of both Seq2Seq (Kaneko
et al., 2020) and Seq2Edits (Malmi et al., 2019; Omelianchuk et al., 2020)
methods. However, recent studies have disclosed that Seq2Seq GEC models (even
with data augmentation) are vulnerable to adversarial examples, which are
purposely constructed to confuse a converged model to generate wrong
predictions by perturbing its inputs (Wang & Zheng, 2020). Fooling a model by
perturbing its inputs, which is also called an adversarial attack, has become
an essential means of exploring the model vulnerabili- ties. Studies on other
classification tasks and PLMs further hint at the possible vulnerability of
PLMs-based GEC methods (Li et al., 2021). In view of the above-mentioned
facts, it is imperative to conduct a systematical evaluation of existing GEC
methods to adversarial attacks, especially for the under-explored Seq2Edits
paradigm and PLMs-based models.

To fill this gap, we propose to evaluate the robustness of cutting-edge GEC
models to different adversarial attacks. More concretely, we introduce three
discrete adversarial attack strategies and one continuous attack method to
obtain adversarial examples. These three discrete variations are motivated by
an existing GEC work (Wang & Zheng, 2020) to detect the vulnerable
tokens/positions that are most likely to cause the failure of the GEC models
once they are substituted with the grammatical errors people may make. In this
paper, we implement three different substitution strategies, i.e., rule-based
perturbations that follow the rules of Wang & Zheng (2020) and relatively
imperceptible grammatical errors people may make, including synonyms and
antonyms based on WordNet222https://wordnet.princeton.edu/. As for the
continuous adversarial noise, back-translation (Sennrich et al., 2016) is
widely used for neural GEC models before the era of large-size pre-trained
language models (PLMs), and we leverage this type of perturbation to test
whether GEC models constructed on powerful (PLMs) are robust enough to such
simple adversarial noise. We also propose one evaluation metric Recovery Rate
with one associated attack set which contains fixed number of attack per
sentence. Resembling the observation on the previous Seq2Seq method attacked
by mapping & rules, cutting-edge GEC models are also susceptible to the
introduced attacks. Taking the BART-based method (Lewis et al., 2020;
Katsumata & Komachi, 2020) for example, its performance ($F_{0.5}$) on
CoNLL-2014 (Ng et al., 2014) decreases sharply, from 62.6 to 36.8.
Intuitively, the dramatic performance decline can be mitigated by pre-training
or post-training with a great number of adversarial examples for a certain
type of attack (Wang & Zheng, 2020). However, such methods require preparing
considerable data for each attack type in advance, which is infeasible for
real-world scenarios. Another minor flaw of these methods is that the
significant improvement in robustness is possibly accompanied by a performance
decrease on the original testing data.

To avoid these problems, we introduce the concept of regularization data which
is a kind of strict hard sample, and propose a very effective cycle self-
augmenting (CSA) method. Concretely, our proposed CSA is only introduced in
the post-training process of a converged GEC model and merely needs the
original training data. The self-augmented data can provide more high-quality
pseudo pairs to improve model performance on the original testing data, and
meanwhile, further training on the regularization data can prevent the GEC
models from over-fitting on easy-to-learn samples. Thus, our proposed CSA
method can significantly improve model robustness with only a few more
training epochs as the extra cost. Since our CSA no longer requires well-
crafted adversarial examples for model training, it is more feasible in
applications and can generalize well to different GEC frameworks. Experimental
results on seven strong models (e.g., BERT-fuse, BART, RoBERTa, XLNET) and
four benchmark datasets (i.e., BEA, CoNLL, FCE, JFLEG) demonstrate the
effectiveness of our proposed CSA method. Our CSA method achieves significant
robustness improvement on all settings and, at the same time, yields
meaningful performance improvement on the original testing data (four out of
seven tested models), with nearly comparable results for the left three SOTA
baselines. By further analyzing the effects of hard samples and regularization
data, we observe the advancement of the regularization data. Besides, we also
find that the trade-off between the robustness to attack and the performance
on the original testing data is associated with regularization examples, where
more regularization pairs in training lead to better robustness but with
performance decline on the original testing data, and vice versa. Furthermore,
we find a deeper relationship between the performance of the GEC model and
data characteristics by conducting quantitative experiments on the two fine-
grained data components. As for the anomalies of the slight improvement of the
original testing data or attack sets for some models, we take a thorough
analysis of this phenomenon and summarize a paradigm of utilizing
regularization data for different model architectures in the end.

It is worthy noting that the recent work (Zhang et al., 2023) also explores
the robustness of the GEC systems when nuanced modifications irrelevant to
errors are introduced by users, but our work is more inclined towards using
CSA methods to enhance the robustness of the GEC system.

##  2 Preliminary

In this section, we summarize the key components of cutting-edge GEC methods
and present a few representative works correlated with the robustness of GEC
models against adversarial attacks. We firstly give the definition of the
grammatical error correction (GEC) task. Then, we review some typical methods
for obtaining synthetic data and introduce two most popular GEC model
architectures in existing literature, i.e., Seq2Seq and Seq2Edits, In the end,
we present the definition of textual adversarial attacks in NLP, following
with the pilot studies of adversarial attacks in GEC and a few widely-used
attack methods for other NLP tasks.

Level | Example
---|---
Lexical | The best place for young people in our aree is without doubt the lake .
The best place for young people in our area is without doubt the lake .
Syntactic | I don’t recommend it to children lower than thirteen years old .
I don’t recommend it to children under thirteen years old .
Semantic | I had a big conversation yesterday in that house .
I had a long conversation yesterday in that house .
Discourse | Water is made up of one elements, hydrogen and oxygen.
Water is made up of two elements, hydrogen and oxygen.
Pragmatic | I need sunscreen because it rains so hard.
I need an umbrella because it rains so hard.
Table 1: Different levels of errors existed in the text. Texts colored with
red are wrongly written and corrected into blue ones.

###  2.1 Task Definition

Grammatical error correction (GEC) is the task of converting an ungrammatical
writing text into a grammatical one. Specifically, giving a text
$x_{1},\dots,x_{n}\in\mathcal{X}$, this task is to build a system
$\mathcal{F}$ which can detect and correct the ungrammatical context without
changing the meaning of the original text, and finally return the grammatical
text $y_{1},\dots,y_{m}\in\mathcal{Y}$ ($m$ maybe not equal with $n$). From a
linguistic point of view, errors existing in the text can be classified at
five levels (Kukich, 1992) which are listed with their explanations below, and
Table 1 provides some cases of these five error types.

  * •

Lexical errors refer to misspelling a word into a non-existent one.

  * •

Syntactic errors violate some syntactic rules, e.g., subject-verb agreement,
which causes contextual or rules mismatching.

  * •

Semantic errors are caused by contextual spelling mistakes. Although there is
no syntactic error existing, the sentence’s meaning may be changed, or there
are collocation/co-occurrence errors.

  * •

Discourse errors break inherent coherence relations in a text, which may cause
temporal or other conflicts in or among the sentences.

  * •

Pragmatic errors correlate to the disobedience of common sense in the text.

In application, GEC systems mainly focus on the first three types of errors
because correcting errors from the last two types requires auxiliary tasks,
e.g., discourse analysis.

###  2.2 Data Expansion

The recent success of GEC models highly relies on the availability of massive
training data pairs (Awasthi et al., 2019; Kiyono et al., 2019; Omelianchuk et
al., 2020; Rothe et al., 2021). Considering that human-labeled pairs are
expensive to obtain, many efforts have been devoted to exploring the
generation of pseudo data pairs for GEC (Lichtarge et al., 2019; Grundkiewicz
et al., 2019; Náplava & Straka, 2019), and the combination of synthetically
generated data has almost been indispensable for recently proposed GEC models
(Kiyono et al., 2019). Furthermore, in addition to generating and utilizing a
large amount of synthetic data for improving the model performance under the
supervised setting, how to efficiently use different data variations for
probing model capability or improving model performance under an unsupervised
setting during the fine-tuning stage also catches much research interests. We
classify the prevalent operations of synthesizing data in GEC tasks into two
categories, i.e., vocabulary-based perturbation and generation-based
perturbation, which are detailed below.

##### Vocabulary-Based Perturbation

Representative vocabulary-based perturbation methods are based on three basic
noising operations: deletion, insertion, and replacement. The first two
operations can be applied directly to the original grammatical sentences,
while the replacement operation requires a confusion set that contains words
that people often make mistakes with (Bryant & Briscoe, 2018; Rozovskaya et
al., 2014). We introduce a few of them as follows:

  * •

With the convenient access to large scale general-domain or out-of-domain
corpora, e.g., Wikipedia, some works (Lichtarge et al., 2018; Awasthi et al.,
2019; Katsumata & Komachi, 2020) utilize vocabulary-based perturbation for
these datasets to acquire massive data with relatively low-quality for pre-
training a GEC model, in which the pre-training process can be further
enhanced by strategies from language model pre-training, e.g., masking
operation (Kiyono et al., 2019).

  * •

Many other works explore constructing high-quality data samples. For instance,
Yasunaga et al. (2021); Náplava et al. (2022); Mita & Yanaka (2021); Zhao et
al. (2019) focus on how to build better obfuscation sets to improve the
performance further or prob the characteristics of the model by manually
annotating or utilizing linguistic rules such as POS tagging.

##### Generation-Based Perturbation

Although the vocabulary-based perturbation is convenient and straightforward,
some unrealistic error patterns that do not resemble those observed in the
actual data may occur in the results (Koyama et al., 2021). Instead, some
researchers utilize the neural network to learn error distributions and
automatically generate errors:

  * •

Ge et al. (2018) propose a fluency metric to evaluate the quality and
correctness of sentences. For each ungrammatical-corrected sentence pair, they
utilize a sequence-to-sequence error generation model to create n-best pseudo
pairs during inference, which are further sorted in ascending order of
fluency. Then, different fluency boost learning strategies are introduced to
enhance the training process.

  * •

Wan et al. (2020) perform data augmentation in a controllable manner.
Specifically, they train a model which can reconstruct a grammatical sentence
into an error-specific one based on the given error type.

  * •

Back-translation (Sennrich et al., 2016; Lample et al., 2018) is an effective
method to improve neural machine translation with monolingual data by
augmenting the parallel training corpus with target language sentences. It can
also be utilized for the GEC task as we can expand the training corpus with
the back-translations of grammatical texts (Lichtarge et al., 2019) or texts
in other languages (Zhou et al., 2020). Different from the above two methods,
the back-translation method is more direct and typically introduced in the
post-processing procedure, i.e., modifying the decoding strategy (Xie et al.,
2018; Kasewa et al., 2018).

  * •

To improve the performance of GEC systems under the unsupervised setting,
Yasunaga et al. (2021) apply the BIFI framework (Yasunaga & Liang, 2021) to
the GEC task, which contains a critic module to evaluate the outputs from the
back-translation model to acquire more realistic error distributions.

Some works also pursue improving the data quality in addition to expanding the
data quantity. Rothe et al. (2021) release a CLANG-8 dataset by using a
multilingual model (mT5 (Xue et al., 2021)) to automatically clean and relabel
the original LANG-8 corpus (Mizumoto et al., 2011; Tajiri et al., 2012). Wan &
Wan (2021) propose a syntax-guided model to make use of the syntactic
knowledge from raw data, which can achieve comparable performance without the
enhancement of pre-trained models.

(a) Seq2Seq model architecture

(b) Seq2Edits model architecture

Figure 1: Two prevalent GEC model architectures. Figure (a) illustrates the
Seq2Seq framework, where $x_{i}$, $h_{i}$ and $y_{i}$ represents the input
token, the correlated probability distribution outputs by the decoder and the
output token, respectively. The output tokens are generated by beam search
(Tillmann & Ney, 2003) or other decoding strategies (Holtzman et al., 2019).
Figure (b) illustrates the Seq2Edits framework, where $e_{i}$ represents an
editing operation for the correlated input token $x_{i}$. The output token
$y_{i}$ is then generated by editing the corresponding input token.

###  2.3 Adversarial Attack

####  2.3.1 Textual Adversarial Attacks Definition

The core of building textual adversarial examples is to confuse the NLP
models. As mentioned in Section 2.1, a GEC system $\mathcal{F}$ can map a
ungrammatical sentence $x\in\mathcal{X}$ to a correct sentence
$y\in\mathcal{Y}$, while an adversarial example
$x^{\prime}=x^{\prime}_{1},\dots,x^{\prime}_{n}\notin\mathcal{X}$ is built to
satisfy the following paradigm:

$\mathcal{F}(x^{\prime})\neq y,\quad sim(x,x^{\prime})\geq\delta,$ (1)

where $sim$ represents a metric function that calculates the similarity of the
original $x$ and the associated adversarial example $x^{\prime}$, and $\delta$
is the minimum similarity.

####  2.3.2 Application of Adversarial Attack

In actual application scenarios, there exist different types of noise in text,
and recent studies on the GEC task have revealed that existing Seq2Seq methods
are quite vulnerable to adversarial examples under the white-box setting (Wang
& Zheng, 2020). In other words, a well-trained off-the-shelf model may
collapse when facing texts which happen to be adversarial examples. As a
result, building a GEC system with robustness and generalization capability
over noisy examples is critical. Inspired by adversarial attack and defense in
NLP (Jia & Liang, 2017; Zhao et al., 2018), we explore the model performance
on both adversarial attack sets and regular benchmarking datasets of the GEC
task.

To obtain adversarial examples, Wan et al. (2020) propose to first identify
the weak spots of a model and then replace the vulnerable tokens with two
different strategies. One is to create a correct-to-error mapping from the GEC
training set. Another is to present a series of substitution rules if there is
no candidate in the mapping. Hereafter, we denote this method as Mapping &
Rules for short. There are also other popular adversarial example construction
methods for PLMs and other tasks but are less explored in GEC, e.g., word
substitutions (Ma, 2019; Dong et al., 2021; Li et al., 2021).

###  2.4 Model Architecture

As mentioned above, the goal of GEC task is to map an ungrammatical piece
($x_{1},\dots,x_{n}$) into a grammatical one ($y_{1},\dots,y_{m}$) ($n$ may
not be equal with $m$) from the dataset $\mathcal{D}$ with the use of GEC
system $\mathcal{F}$. As illustrated in Figure 1, there are two main model
architectures, i.e., the Seq2Seq framework (Sutskever et al., 2014; Vaswani et
al., 2017; Lewis et al., 2020) and the Seq2Edits framework (Awasthi et al.,
2019; Devlin et al., 2019).

####  2.4.1 Seq2Seq framework

Many works (Xie et al., 2016; Yuan & Briscoe, 2016; Xie et al., 2018; Junczys-
Dowmunt et al., 2018; Zhao et al., 2019; Sun et al., 2021; Kaneko et al.,
2020) formulate GEC as a natural language generation (NLG) task and utilize an
encoder-decoder structure to complete the sequence-to-sequence (Seq2Seq)
generation task.

Given an input sentence $\bm{x}$ of $n$ tokens, the encoder first encodes it
into the hidden representation $h^{s}_{1:n}$, and then the decoder outputs
each token in an auto-regressive fashion. The output distribution over the
vocabulary at the $k$-th decoding step is conditioned on $h^{s}_{1:n}$ from
the encoder and the summarized representation of previously generated $k$-1
tokens $h^{t}_{1:k-1}$ from the decoder, formulated as
Pr$(y_{k}|\bm{y}_{<k},\bm{x})$ = Pr$(y_{k}|h^{t}_{1:k-1},h^{s}_{1:n})$. The
training objective of Seq2Seq architecture is the negative log-likelihood,
written by

$\mathcal{L}(\theta)=-\frac{1}{|\mathcal{D}|}\sum_{x,y\in\mathcal{D}}log(p(y|x))$
(2)

where $\theta$ refers to trainable model parameters. To get a optimal output,
beam search decoding (Yuan & Briscoe, 2016; Chollampatt & Ng, 2018) and its
variations are also utilized (Sun et al., 2021). This architecture can achieve
promising performance with massive data but will sacrifice inference
efficiency due to the iterative decoding.

####  2.4.2 Seq2Edits framework

To alleviate the embarrassed situation of inference speed and large decoding
space problems in Seq2Seq model architecture, Seq2Edits provides another
alternative that casts GEC into a tagging problem (Awasthi et al., 2019;
Omelianchuk et al., 2020; Malmi et al., 2019) along with the non-
autoregressive sequence prediction (Li & Shi, 2021). Instead of directly
predicting the token, Seq2Edits architecture first predicts the edit operation
type $e_{i}\in E$ for each input token $x_{i}$ and then performs a series of
transformation operations based on the predicted edit to realize the
grammatical output. The training objective of tagging is formulated as:

$\mathcal{C}(\phi)=-\frac{1}{|\mathcal{D}|}\sum_{x\in D,e\in E}log(p(e|x))$
(3)

where $\phi$ corresponds to model parameters to be trained. This architecture
can achieve competitive performance and faster inference speed with limited
data but requires heuristic prior and human efforts to obtain labeled data for
the tagging task.

##  3 Cycle Self-Augmenting Method

In this section, we introduce our Cycle Self-Augmenting method (CSA).
Specifically, we elaborate on how Self-Augmenting and Cycle Training work
under our settings in Section 3.1 and Section 3.2 respectively, following with
a comparison between our method and conventional data augmentation methods in
Section 3.3. In the cycle training process, we will present the concept of
regularization data for GEC, which is the key to the robustness against
adversarial attacks.

Figure 2: The overall framework of our proposed Cycle Self-Augmenting (CSA).
The Self-Augmenting mechanism corresponds with step ①-④. We launch cycle
training in step ⑤, along with the utilization of regularization data. It is
worth noting that the types of augmenting pairs are numerous, and we utilize
hard sample $(x_{h},y)$ and $(y^{{}^{\prime}},y)$ in our experiment.

###  3.1 Self-Augmenting

To enhance model robustness, existing works mainly utilize numerous well-
crafted adversarial examples for certain types of adversarial attacks in the
pre-training or/and post-training stages (Wang & Zheng, 2020; Li et al.,
2021). Instead of elaborately designing adversarial example generation
strategies for each type of attack, we make the GEC model itself perform self-
augmenting to defend against various types of attack, which is more efficient
and adaptive to varied GEC models. To better exploit the capability of GEC
models, we introduce the self-augmenting mechanism for post-training.

Concretely, the crux of Self-Augmenting is to obtain augmenting data pairs
$\mathcal{D}_{Aug}$ which consists of extra pairs $\mathcal{D}_{self}$
constructed by the model itself. The detailed process is illustrated in Figure
2. Given a well-trained GEC model $f(\cdot)$ and the original training dataset
$\mathcal{D}$=$\\{(x^{(i)},y^{(i)})\\}_{i=1\cdots|\mathcal{D}|}$, we feed each
input $x$ into $f(\cdot)$ to obtain the corresponded output $y^{{}^{\prime}}$
(step ①-② in Figure 2), and compare $y^{{}^{\prime}}$ with the golden $y$ word
for word (step ③). If there is any difference between $y^{{}^{\prime}}$ and
$y$, we will add $(y^{{}^{\prime}},y)$ into $\mathcal{D}_{self}$, and utilize
these extra data for post-training (step ④).

One merit of this is enabling the model to iteratively refine the outputs,
which is a popular paradigm in many other situations, e.g., non-auto
regressive machine translation task (Lee et al., 2018; Geng et al., 2021;
Huang et al., 2021), and also consistent with the GEC task. However, it may
cause a dramatic forgetting problem. Thus, we also collect another augmenting
data variant $\mathcal{D}_{hard}$ as $\mathcal{D}_{Aug}$ which consists of
hard samples directly extracted from the original training set. The
construction of $\mathcal{D}_{hard}$ is similar to the $\mathcal{D}_{self}$,
except that we insert each original training pair $(x,y)$ into
$\mathcal{D}_{hard}$. To better distinguish the hard samples from the original
training set, we denote pairs in $\mathcal{D}_{hard}$ as $(x_{h},y)$ in this
paper. We will compare these two strategies in our experiment (Section 5).

Besides, one can collect $(x,y^{{}^{\prime}})$ for self-distillation proposed
by Kim et al., which is a regularization method to mitigate the over-fitting
problem for the situation of mismatch between the model capacity and the
number of data. However, such method is unsuitable for the GEC task since the
the quantity of training data is sufficient and the the target of the GEC
model should be grammatical texts.

Instead, utilizing $D_{hard}$ or $D_{self}$ for post-training can provide more
feasible training pairs and is more tally with the GEC task, i.e., only part
of the input sentence is edited. We will show the superiority of our self-
augmenting in Section 6.3.

###  3.2 Cycle Training

To effectively utilize augmenting pairs from the above introduced self-
augmenting process, we further present a cycle training strategy, which is
sketched out in step ⑤ of Figure 2. Specifically, we use the aforementioned
self-augmenting strategy to construct a new dataset $\mathcal{D}_{Aug}^{k}$ in
each cycle $k$, where $0\leq$$k$$\leq\epsilon$. Thus, we can leverage
$\epsilon$ augmenting datasets
($\mathcal{D}_{Aug}^{1}$,$\dots$,$\mathcal{D}_{Aug}^{k}$,$\dots$,$\mathcal{D}_{Aug}^{\epsilon}$)
in cycle training, where these datasets are utilized differently in two
training stages.

In Stage I, the obtained augmenting datasets contain many unseen data pairs in
the original training dataset, which can be simply used by conducting further
training to improve both model performance and robustness. Accordingly, we
adopt the following training process for each cycle at the early stage, i.e.,
when $0\leq$$k$$\leq\mathcal{P}$:

  * •

Perform training on $\mathcal{D}_{Aug}^{k}$ until convergence.

  * •

Conduct further tuning on a small high-quality GEC dataset
$\mathcal{D}_{tune}$ to prevent over-fitting on the augmenting dataset.

Note that the improvement of performance and robustness is not caused by
merely using the small dataset, which is discussed in Section 6.4.

Along with the model training, there are fewer and fewer unseen data pairs in
the augmenting datasets. Simply utilizing the augmenting dataset in each cycle
for model training might yield over-fitting on these datasets. Thus, we turn
to focus on these hard-to-learn data, i.e., these data pairs that have not
been learned after $\mathcal{P}$ cycles. Inspired by previous work (Zhou et
al., 2021) that names some specific samples that are negatively associated
with the performance of knowledge distillation as regularization examples, we
treat these hard-to-learn data as Regularization Data for the GEC task. When
$\mathcal{P}\leq$$k$$\leq\mathcal{\epsilon}$, the regularization data of the
$k$-th cycle is obtained as
$\mathcal{D}_{Reg}^{k}$=$\mathcal{D}_{Aug}^{k-p+1}\cap\dots\cap\mathcal{D}_{Aug}^{k}$.
In this stage (Stage II), the trained GEC model from Stage I is further
trained as below:

  * •

Perform training on $\mathcal{D}_{Reg}^{k}$ until convergence.

  * •

Conduct further tuning on a small high-quality GEC dataset
$\mathcal{D}_{tune}$.

We summarize the whole procedure of CSA method into algorithm 1.

Algorithm 1 Procedure of CSA method

1:max cycle times of Stage I : $\mathcal{P}$; max cycle times of Stage II :
$\epsilon$; original training dataset : $(x_{i},x_{i})$; a small high-quality
GEC dataset : $\mathcal{D}_{tune}$; a well-trained GEC model : $g(\theta)$;

2:a converged GEC model $g(\theta)$ after cycle self-augmenting training

3:function CSA($\mathcal{P},x_{i},y_{i},g(\theta),\epsilon$)

4: $k\leftarrow 0$

5: $converged\leftarrow False$

6: $p\leftarrow True$

7: while $k<\mathcal{P}$ and $converged==False$ do

8: if $k==0$ then

9: $\mathcal{D}^{k}_{Aug}\leftarrow$ Compare$(y_{i},g(x_{i};\theta))$

10: else

11: utilize $\mathcal{D}^{k}_{Aug}$ and $\mathcal{D}_{tune}$to train model
$g(\theta)$

12: if performance of $g(\theta)$ does not improve then

13: $converged\leftarrow True$

14: end if

15: end if

16: end while

17: $converged\leftarrow False$

18: while $k<\epsilon$ and $converged==False$ do

19:
$\mathcal{D}^{k}_{Reg}\leftarrow\mathcal{D}_{Aug}^{k-p+1}\cap\dots\cap\mathcal{D}_{Aug}^{k}$

20: utilize $\mathcal{D}^{k}_{Reg}$ and $\mathcal{D}_{tune}$ to train model
$g(\theta)$

21: if performance of $g(\theta)$ does not improve then

22: $converged\leftarrow True$

23: end if

24: $k\leftarrow k+1$

25: end while

26: return $g(\theta)$

27:end function

The benefits of launching further training on regularization data are four-
folds: 1) it prevents over-fitting on the easy-to-learn data pairs in the
augmenting datasets; 2) it can reduce model capacity to improve its
generalization ability and robustness; 3) it gives more opportunities for the
model to address hard-to-learn pairs; 4) it can accelerate each training cycle
by using fewer data pairs. More analysis of regularization data is given in
Section 6.5.

###  3.3 Comparison with Previous Works

Our CSA method contains a certain degree of correlation in Fluency Boost
Learning(Ge et al., 2018) methods and one may view our method is a traditional
data augmentation method. We will elaborate on the advancement of our method
from two aspects: (1) continual improvement on a well-trained model (2) a
efficient way to filter regularization Data.

##### Continual Improvement

As shown in Figure 3, unlike the previous work (Ge et al., 2018) which utilize
dual learning method333There are actually three methods proposed in their
work: self-boost, back-boost and dual-boost. However, dual-boost method is
finally put into implementation. to improve the GEC model, our method mainly
focus on self-iterative refinement on one well trained model. From the
implementation perspective, we do not modify the original model structure
(including decoding strategy) and do not add an auxiliary module. This plug-
and-play method is significant, especially in some environments where the
model architecture remains unchanged or has a strict requirement for the
number of model parameters.

(a) Dual-boost method

(b) CSA method

Figure 3: Comparison between our CSA with previous data augmentation method.
Picture (a) shows the framework of the fluency boost learning method proposed
by Ge et al., which contains a generator that produces error data and a GEC
model trained with both real data and the data produced by the generator. The
fluency score is introduced to sample the fluency boost pairs output by
models. Picture (b) illustrates our CSA method, which contains $k$ cycles, and
in each cycle, the GEC model is trained with both read data and the
regularization data. The regularization data is selected among the cycles.

##### Regularization Data Extraction

A critical merit of our method is to select regularization data that
simultaneously improves model performance and robustness among the cyclic
procedure. Following the aforementioned data augmentation paradigm shown in
equation 1 for improving robustness, the regularization data can improve both
the model robustness and the performance on the original testing data, as it
contains two main components:

  * •

$X_{unl}$: A set of data where the model keeps generating the same errors or
is unable to edit correctly mainly contributes to improving the model’s
robustness on attack sets.

  * •

$X_{unc}$: A set of data where the model keeps generating different errors or
is unable to edit correctly mainly contributes to improving the model’s
performance on the original testing data.

We will analyse the influence brought by regularization data in Section 6.5,
and illustrate the procedure of generating attack sets in Section 4. Moreover,
we select different variants of augmenting pairs for post-tuning, i.e.,
$(x^{{}^{\prime}},y)$ and $(y^{{}^{\prime}},y)$, other than simply filtering
one type of data like the fluency boost learning method.

##  4 Adversarial Data

Since adversarial data set is the cornerstone of testing the defense ability
of the GEC model, we introduce four textual adversarial attack methods to
construct different variants for each original test sets, including back-
translation (Xie et al., 2018), mapping & rules (Wang & Zheng, 2020), antonym
substitution (Ma, 2019) and synonyms substitution (Li et al., 2021). We divide
these four ways of constructing adversarial data into two categories: discrete
attack and continuous attack, which are described in Section 4.1 and 4.2,
respectively. In Section 4.3, we introduce Recovery Rate, one evaluation
metric that tests the model’s robustness.

(a) Synonym

(b) Antonym

(c) Mapping & Rules

(d) Back-Translation

Figure 4: Comparison between our CSA and the adversarial training method on
four different attack sets.

###  4.1 Discrete Attack

Inspired by the previous work (Wan et al., 2020) which identifies the
vulnerable tokens according to the positional scores, we locate the weak spots
with a probability distribution. We formally describe the implementation
details of identification operation for Seq2Seq model and Seq2Edits model in
Section 4.1.1 and 4.1.2 respectively. Then, we introduce three discrete types
of substitution operations in Section 4.1.3.

####  4.1.1 Constructing data for Seq2Seq model architecture

For Seq2Seq model architecture, we use beam search (Tillmann & Ney, 2003)
decoding strategy and set beam size as 5. After the original ungrammatical
sentence $X=\\{x_{1},\dots,x_{n}\\}$ passes through the well-trained cutting-
edge GEC system $\mathcal{F}$, we can obtain 5 candidate sentences, as well as
token-level prediction probability for $i$-th token $p^{t}_{i}$. Then, we
select the sentence with the highest generation probability, and use the
alignment function implemented in fairseq toolkit (Ott et al., 2019) to align
the source and target sentences. We average all the token-level prediction
probability to get a threshold $\epsilon$, we take $x_{k}$ as a candidate to
be perturbed if its prediction probability $p^{t}_{k}$ is less than the
threshold $\epsilon$.

####  4.1.2 Constructing data for Seq2Edits model architecture

For Seq2Edits model architecture, there is token-level predicting probability
for editing token. Since the Tagger module (as shown in Figure 1(b)) often
consists of several linear layers with softmax layers on the top (Omelianchuk
et al., 2020; Awasthi et al., 2019; Malmi et al., 2019), each token-level
prediction probability of tag $p^{e}_{i}$ is obtained by applying an argmax
over the encoder logits. Owing to the natural one-to-one correspondence
between the original ungrammatical sentence and the tags, we can calculate the
threshold $\epsilon$ and obtain the vulnerable candidates directly without
aligning operation.

####  4.1.3 Substitution Operation

After detecting the vulnerable spots for each sentence, a substitution
operation is followed. Details of three common attack strategies, which we
call mapping & rules, antonym substitution, and synonyms substitution, are
listed below.

##### Mapping & Rules

We firstly build a $Good\mapsto Poor$ mapping $\mathcal{M}$ from training
datasets. Specifically, given one ungrammatical sentence and the corresponding
correct sentence, we can get the alignment between two sentences by utilizing
ERRor ANnotation Toolkit444https://github.com/chrisjbryant/errant (Bryant et
al., 2017; Felice et al., 2016).

Then, we compare the aligned pieces from the source sentence and the golden
sentence, respectively, and add the inconsistent piece pairs to the mapping
set $\mathcal{M}$. We extend such operation to the scope of all public
annotated corpus (as shown in Table 2). After acquiring the mapping
$\mathcal{M}$, we apply a weighted random sampling (Efraimidis & Spirakis,
2006) method (Equation 4) to select the perturbations for the vulnerable spots
in the original sentence step by step:

$e\sim(\mathbb{U}_{i})^{1/c_{i}}$ (4)

where $\mathbb{U}_{i}$ is the random numbers between 0 and 1 for $i$-th token,
and $c_{i}$ is calculated by Equation 5:

$c_{i}=\left\\{\begin{aligned}
&sim(s_{i},s_{i}^{\prime})\quad&sim(s_{i},s_{i}^{\prime})&\textgreater\zeta\\\
&\quad\quad\zeta\quad&sim(s_{i},s_{i}^{\prime})&\leq\zeta\end{aligned}\right.,$
(5)

where $s_{i}$ is the original ungrammatical sentence and its corresponding
perturbation is $s_{i}^{\prime}$. Following the previous work (Wang & Zheng,
2020), we select edit distance (Ristad & Yianilos, 1998) as $sim$ function
here, and $\zeta$ is a threshold which controls similarity between two
sentences. In order to keep the semantic consistency between the original text
and adversarial text as much as possible, we must denote that if the edit
distance $sim(s_{i},s_{i}^{\prime})$ continually greater than $\zeta$ for
three perturbation steps(Wang & Zheng, 2020), this strategy ends. We set
$\zeta$ as 0.1 by default.

##### Antonym and Synonyms Substitutions.

We detect the vulnerable tokens by using the method proposed above and simply
use the open-source tools NLPAug (Ma,
2019)555https://github.com/makcedward/nlpaug to substitute opposite meaning
word according to WordNet antonym (Miller, 1995) or substitute similar words
according to WordNet/PPDB (of Hertfordshire, 2007) synonym. The sampling
method and default settings of these two perturbation strategies keep up with
the Mapping & Rules strategy.

Algorithm 2 Procedure of Recovery Rate metric (SR)

1:source sentences set: $S$; attacked position set: $P$; golden sentences set:
$Y$; model outputs set: $Y^{\prime}$

2:error_recovery $er$

3:function ErrorRecovery($S,P,Y,Y^{\prime}$)

4: $r\leftarrow 0$

5: $e\leftarrow 0$

6: $LY\leftarrow\textbf{Length}(Y)$

7: $i\leftarrow 0$

8: while $i<LY$ do

9: $s\leftarrow S[i]$

10: $pl\leftarrow P[i]$

11: $y\leftarrow Y[i]$

12: $y^{\prime}\leftarrow Y^{\prime}[i]$

13: $editSeq_{1}\leftarrow$ Align $(y,s)$

14: $editSeq_{2}\leftarrow$ Align $(y^{\prime},y)$

15: $LP\leftarrow$ Length$(pl)$

16: $j\leftarrow 0$

17: $Flag\leftarrow$ TRUE

18: while $j<LP$ do

19: $s\\_pos\leftarrow pl[j]$

20: $pos_{1}\leftarrow$ getPos$(editSeq_{1},s\\_pos)$

21: $pos_{2}\leftarrow$ getPos$(editSeq_{2},pos_{1})$

22: if $y[pos_{1}]\neq y^{\prime}[pos_{2}]$ then

23: $e\leftarrow e+1$

24: $Flag\leftarrow$ FALSE

25: Break

26: end if

27: $j\leftarrow j+1$

28: end while

29: if $Flag==$ TRUE then

30: $r\leftarrow r+1$

31: end if

32: $i\leftarrow i+1$

33: end while

34: $er\leftarrow r/(e+r)$

35: return $er$

36:end function

###  4.2 Continuous Attack

Besides, we also want to explore the robustness of the GEC models by
perturbing the inputs in the representation space. We utilize the back-
translation method (Sennrich et al., 2016) and the implementation details are
illustrated below.

##### Back-Translation

Inspired by the previous works (Sennrich et al., 2016; Edunov et al., 2018),
back-translation plays an important role in enlarging the monolingual data and
boosting fluency for phrase-based statistical machine translation, as well as
GEC tasks. We reverse the examples from pre-training datasets to train a back-
translation model which can generate errorful examples from a clean corpus.
However, this method generates too few errors when using a traditional
decoding strategy, and the outputs cannot confuse the model easily as it
samples the most likely token in each decoding step. Then we implement a
variant back-translation method (Xie et al., 2018) which adds
$r\beta_{random}$ to penalize every hypothesis during the beam search step,
where $r$ is drawn uniformly from the interval [0, 1] and $\beta_{random}$ is
a hyper-parameter sampling from $\mathbb{R}$. We follow the previous work
(Kiyono et al., 2019) to set $\beta_{random}=6$.

###  4.3 Recovery Rate to Statistics

As long as the current prevalent evaluation metric of the GEC task mainly
focuses on the global performance of the post-editing sentences, a specific
metric for evaluating partial changes is missing. Following the previous work
(Wang & Zheng, 2020), we propose an evaluation metric Recovery Rate to measure
the correction rate of the attacked position. The entire procedure is
described in the algorithm 2 and the following notations are defined for
clearer illustration:

  * •

$P$ represents attacked positions of each sentence, which can be automatically
generated or manually defined. As for discrete attacks, we regard vulnerable
spots of the original text as $P$, while for continuous attacks, $P$ is
automatically generated by aligning the attack texts and the original texts.

  * •

$r$ represents the number of recovery sentences. It is worth noting that if
all the attack positions are recovered in one sentence, this sentence can be
judged as recovery.

  * •

$e$ represents the number of unrecovered sentences. If some attack positions
are missed to be recovered, this sentence is judged to be unrecovered.

  * •

Align($X$,$Y$) is an aligning function implemented by ERRor ANnotation Toolkit
which can produce the editing operation sequence between two sentences, i.e.,
sentence $X$ can transfer to sentence $Y$ by following the editing operation
sequence.

  * •

getPos($editSeq$,$pos$) can locate the recovery position by simulating the
operations in the editing operation sequence $editSeq$ until reaching the
attack position $pos$ and calculating the simulation result’s length.

We can see that the Recovery Rate metric dictates that all the attack
positions per sentence need to be recovered while ignoring the number of
attacks which indicates the global modification information and can be
abbreviated into SR (Sentence level Recovery) rate. As we are concerned about
the local modification, we take each recovery position into consideration,
i.e, calculating the Recovery Rate of each attacked token rather than totally
recovery sentence, and we abbreviate this metric into TR (Token level
Recovery) rate.

To unify the evaluation settings, we also build corresponding datasets which
contain a fixed number of errors per sentence. Specifically, based on the
Mapping & Rules substitution operation mentioned in Section 4.1.3, we perturb
the fixed number of positions for each sentence by ranking the generation
probabilities, selecting the tokens with lower scores and recording the
positions $P$.

##  5 Experiments

We conduct experiments on both original testing sets and attack sets. We first
present necessary details about train sets and evaluation metrics, followed
with the description of baselines and concrete implementation settings of all
models. We then give the evaluation results on original testing data and
attack sets.

###  5.1 Datasets and Evaluations

In this section, we summarize the training data in Section 5.1.1 as well as
introduce the construction details for attack data in Section 5.1.2, and
describe our evaluation metric in Section 5.1.3.

####  5.1.1 Training Datasets

Dataset | #Sentences | Errors (%) | Usage
---|---|---|---
PIE-synthetic | 9,000,000 | 100.0 % | Pre-training
Pseudo-generate | 15,000,000 | 100.0 % | Pre-training
Lang-8 | 1,102,868 | 51.1 % | Fine-tuning †
FCE | 34,490 | 62.6 % | Fine-tuning †
NUCLE | 57,151 | 38.2 % | Fine-tuning †
W&I+LOCNESS | 34,308 | 66.3 % | Fine-tuning ‡
W&I-Dev (BEA-test) | 4,384 | - | Evaluation
CoNLL-2014 (test) | 1,312 | - | Evaluation
FCE-test | 2,695 | - | Evaluation
JFLEG | 1,951 | - | Evaluation
Table 2: Statistics of datasets used in our experiments. The top group shows
the data for pre-training. The second group shows the data for fine-tuning.
The third group shows the data for evaluation. “$\dagger$” denotes data used
in fine-tuning stage I, while “$\ddagger$” refers to data used in both fine-
tuning stage I and II.

The first and second group of Table 2 describes all the datasets that are
utilized in model training. Following the previous study (Omelianchuk et al.,
2020), we leverage these datasets in two different training phases:

##### Pre-Training

In this phase, we use 9M pseudo parallel sentences with synthetic errors
(Awasthi et al., 2019)666https://drive.google.com/open?id=1bl5reJ-
XhPEfEaPjvO45M7w0yN-0XGOA and pseudo-generated data (Kiyono et al., 2019) for
pre-training. Specifically, as for pseudo-generated data, we follow the
operation mentioned in (Kiyono et al., 2019) and use unlabeled corpus from
Gigaword777English Gigaword Fourth Edition (LDC Catalog No.LDC2009T13)(Graff
et al., 2003) as seed corpus. The setting of ($\mu_{mask}$, $\mu_{deletion}$,
$\mu_{insertion}$, $\mu_{keep}$) is (0.5, 0.15, 0.15, 0.2). We randomly sample
10000 sentences from the training data as the development set and pre-train
the model with 15 epochs with the batch size of 9012 tokens. Based on the
results on the development set, we select the best checkpoint for the
following stages.

##### Fine-Tuning

During the fine-tuning phase, we use the official corpora from BEA-2019 shared
task (Bryant et al., 2019)888https://www.cl.cam.ac.uk/research/nl/bea2019st/
for fine-tuning, which comprises four datasets, i.e., Lang-8 Corpus of Learner
English (Lang-8) (Mizumoto et al., 2011; Tajiri et al., 2012), National
University of Singapore Corpus of Learner English (NUCLE) (Dahlmeier et al.,
2013), the First Certificate in English (FCE) (Yannakoudakis et al., 2011),
and Cambridge English Write & Improve + LOCNESS Corpus (W&I+LOCNESS) (Granger,
1998; Yannakoudakis et al., 2018). We split out validation data by random
sampling from the official training corpora with a ratio of 2/98 and decompose
the fine-tuning phase into two stages. In stage I, the model is fine-tuned on
errorful-only sentences. In stage II, the model is tuned on a high-quality and
more realistic dataset as in (Kiyono et al., 2019; Omelianchuk et al., 2020).
In each stage, we set the max training epoch as 20 and select the best
checkpoint according to the result on the validation set.

####  5.1.2 Attack Datasets

To evaluate the defense capability of different models, we utilize four
perturbation methods mentioned in Section 4.1.3 for the official evaluation
data (the third group in Table 2) to generate corresponding attack sets and
exploit the obtained attack sets for testing. Specifically, for the back-
translation strategy, we utilize the PIE-synthetic dataset for training the
model and the training details are listed in Table 3.

Model architecture | transformer_vaswani_wmt_en_de_big Vaswani et al. (2017)
---|---
Number of epochs | 25
Batch Size (Tokens) | 9012
Max Sentence Length | 512
Optimizer | Adam Kingma & Ba (2014)
Adam setting | ($\beta_{1}$=0.9, $\beta_{2}$=0.98)
Learning rate | 3e-4
Lr-Scheduler | inverse_sqrt
Warm-up | 4000
Dropout | 0.1
Loss function | label smoothed cross-entropy Szegedy et al. (2016)
Label Smoothing | 0.1
Table 3: Hyper-parameters of back-translation model. The whole training
strategy follows the pre-training and fine-tuning paradigm, and we utilize the
PIE-synthetic datasets in pre-training stage and Lang-8 datasets in fine-
tuning stage. We split the validation data with a ratio of 2/98 for model
selection. The hyper-parameters of the back-translation model are shown in
this table.

Moreover, to report the Recovery Rate, i.e., TR and SR, we stipulate the
number of errors in each sentence from one to three and build three
corresponding variants of CoNLL-2014 evaluation set 999We ignore the sentences
in the test set which are shorter than two, e.g., one single quotation mark,
both in the perturbation and the calculation procedure. by applying the method
mentioned in Section 4.3. The datasets created for calculating the Recovery
Rate are marked as ${\rm ATK}_{i}$, where $i\in\\{1,2,3\\}$ represents for the
number of attack positions per sentence.

####  5.1.3 Evaluations

For official evaluation data (original testing sets) and attack datasets build
by ourselves (attack datasets), we utilize different evaluation metrics shown
below:

##### Original Testing Sets

We report results on the official evaluation datasets of BEA, CoNLL-2014 (Ng
et al., 2014), FCE, and JFLEG (Napoles et al., 2017). We measure the results
of CoNLL-2014 and FCE by $M^{2}$ scorer (Dahlmeier & Ng, 2012). For JFLEG
results, we use the GLEU metric (Napoles et al., 2015; 2016). We report the
scores measured by ERRANT (Bryant et al., 2017; Felice et al., 2016) for BEA-
test. As the reference of the BEA-test is unavailable, we report results from
CodaLab101010https://competitions.codalab.org/competitions/20228.

##### Attack Sets

As each variant of the attack set is constructed from the original test set,
we leverage the same metrics to calibrate model robustness, i.e., $M^{2}$
scorer, GLEU metric, and ERRANT. We also report TR and SR score for ${\rm
ATK}_{i}$ $(i\in\\{1,2,3\\})$ set.

###  5.2 Baselines and Settings

Note that our proposed CSA method is a post-training strategy, which can be
utilized upon any neural GEC model. We leverage seven cutting-edge models as
our baselines to conduct experiments under the supervised setting. To make
sure all the baselines are well fine-tuned, if there exists a publicly
available checkpoint for each baseline model, we will use it directly.
Otherwise, we will follow the original settings to train a model by
ourselves.111111To ensure that the baseline models are fully fine-tuned, we
conduct experiments on the original testing datasets and compare the results
reported in the original paper. Detailed results are presented in Section
5.3.1 As for the gradient-based adversarial attack, we set $\tau$ as 1 and
$\gamma$ as 0.5. Specifically, we carry out experiments on four Seq2Seq
Models, including Transformer (Kiyono et al.,
2019)121212https://github.com/butsugiri/gec-pseudodata, BERT-fuse (Kaneko et
al., 2020)131313https://github.com/bert-nmt/bert-nmt, BART large (Katsumata &
Komachi, 2020)141414https://github.com/Katsumata420/generic-pretrained-GEC,LM-
Critic method (Yasunaga et al., 2021)151515We apply LM-Critic under the
unsupervised setting and the basic model architecture is Transformer. The
implementation is in the https://github.com/michiyasunaga/LM-Critic , and
three Seq2Edit model variants (RoBERTa, BERT, XLNet) based on large-scale pre-
trained language models in GECToR (Omelianchuk et al.,
2020)161616https://github.com/grammarly/gector.

Besides, to show the superiority of our CSA method, we utilize one gradient-
based defence method (Yasunaga et al., 2017) for comparison. Followed by the
previous work, we inject the noise into the embedding of each model during the
training stage. More details are shown in Appendix D.

As for the CSA method, we set max cycle times $\epsilon$ = 5 and patience
$\mathcal{P}$ = 2. If the model performance does not improve over two
consecutive cycles, the training process is stopped. Note that we use the same
official development set throughout the fine-tuning process of baselines and
the cycle training process of our CSA
method171717https://www.cl.cam.ac.uk/research/nl/bea2019st/ for checking
training convergence. During the post-training stage, all hyper-parameter
settings are the same with baselines.

###  5.3 Main Results

We report three results, including the performance on the original testing
sets (Section 5.3.1), the performance on attack sets (Section 5.3.2), and the
value of Recovery Rate on ${\rm ATK}_{i}$ $(i\in\\{1,2,3\\})$ sets (Section
5.3.3). For each baseline, we compare the performance with two CSA variants,
i.e., $y^{{}^{\prime}}\mapsto y$ and $x_{h}\mapsto y$, and one gradient-based
method ($\nabla_{ATK}$). Moreover, to explore whether our proposed method can
adapted to non-English GEC task, we also conduct some experiments on the
Chinese GEC task and report the results in Appendix D.

Model |  BEA (ERRANT) |  CoNLL-2014 ($M^{2}$) |  FCE ($M^{2}$) | JELEG
---|---|---|---|---
Prec. | Rec. | F_0.5 | Prec. | Rec. | F_0.5 | Prec. | Rec. | F_0.5 | GLEU
Transformer | 65.5 | 59.4 | 64.2 | 68.9 | 43.9 | 61.8 | 59.4∗ | 39.5∗ | 54.0∗ | 59.7
$y^{{}^{\prime}}\mapsto y$ (4 Cycles) | 69.6 | 64.7 | 68.6 | 69.5 | 49.5 | 64.3 | 63.2 | 43.3 | 57.9 | 62.7
$x_{h}\mapsto y$ (3 Cycles) | 67.9 | 64.6 | 67.2 | 69.0 | 50.1 | 64.2 | 63.0 | 43.9 | 58.0 | 61.8
$\nabla_{ATK}$ (2 Cycles) | 66.4 | 62.4 | 65.6 | 69.0 | 48.7 | 63.7 | 61.1 | 42.5 | 56.2 | 61.2
BERT-fuse | 67.1 | 60.1 | 65.6 | 69.2 | 45.6 | 62.6 | 59.8 | 46.9 | 56.7 | 61.3
$y^{{}^{\prime}}\mapsto y$ (3 Cycles) | 68.9 | 64.5 | 68.0 | 69.4 | 49.8 | 64.4 | 64.4 | 46.6 | 59.9 | 62.5
$x_{h}\mapsto y$ (2 Cycles) | 68.1 | 64.0 | 67.2 | 67.5 | 49.4 | 62.9 | 63.7 | 46.9 | 59.4 | 62.0
$\nabla_{ATK}$ (1 Cycle) | 68.1 | 62.6 | 66.9 | 68.1 | 50.4 | 63.6 | 61.4 | 43.1 | 56.6 | 61.4
BART | 68.3 | 57.1 | 65.6 | 69.3 | 45.0 | 62.6 | 59.6∗ | 40.3∗ | 54.4∗ | 57.3
$y^{{}^{\prime}}\mapsto y$ (2 Cycles) | 70.9 | 61.9 | 68.9 | 70.4 | 46.7 | 63.9 | 65.2 | 34.4 | 55.3 | 59.4
$x_{h}\mapsto y$ (1 Cycle) | 66.8 | 61.2 | 65.6 | 66.3 | 45.7 | 60.8 | 60.1 | 39.8 | 54.5 | 58.6
$\nabla_{ATK}$ (2 Cycles) | 73.4 | 56.5 | 69.2 | 71.6 | 42.6 | 63.0 | 62.9 | 34.7 | 54.1 | 57.4
LM-Critic | 51.6 | 24.7 | 42.4 | 64.4 | 35.6 | 55.5 | 49.6∗ | 24.6∗ | 41.2∗ | 51.4∗
$y^{{}^{\prime}}\mapsto y$ (2 Cycles) | 68.4 | 61.6 | 66.9 | 65.7 | 47.4 | 61.0 | 58.0 | 39.6 | 53.1 | 59.1
$x_{h}\mapsto y$ (1 Cycle) | 69.2 | 49.6 | 64.2 | 64.8 | 36.4 | 56.1 | 59.9 | 30.7 | 50.3 | 55.1
$\nabla_{ATK}$ (1 Cycle) | 63.2 | 56.0 | 61.6 | 61.9 | 42.4 | 56.7 | 57.0 | 35.3 | 50.7 | 56.6
BERT | 71.5 | 55.7 | 67.6 | 72.1 | 42.0 | 63.0 | 66.2∗ | 42.0∗ | 59.4∗ | 57.5∗
$y^{{}^{\prime}}\mapsto y$ (1 Cycle) | 67.7 | 57.2 | 65.3 | 70.0 | 44.3 | 62.3 | 64.0 | 43.1 | 58.3 | 57.8
$x_{h}\mapsto y$ (1 Cycle) | 66.4 | 56.0 | 64.0 | 68.7 | 42.8 | 61.3 | 63.8 | 42.8 | 58.1 | 58.4
$\nabla_{ATK}$ (1 Cycle) | 66.1 | 57.8 | 64.3 | 66.3 | 44.5 | 60.4 | 57.9 | 36.5 | 51.8 | 57.5
RoBERTa | 68.4 | 60.8 | 66.8 | 68.7 | 47.2 | 62.9 | 61.6∗ | 45.3∗ | 57.5∗ | 59.1∗
$y^{{}^{\prime}}\mapsto y$ (1 Cycle) | 68.8 | 60.3 | 66.9 | 68.0 | 46.9 | 62.4 | 62.7 | 44.8 | 58.0 | 58.6
$x_{h}\mapsto y$ (1 Cycle) | 66.2 | 60.4 | 64.9 | 66.3 | 47.7 | 61.5 | 61.4 | 44.8 | 57.2 | 59.2
$\nabla_{ATK}$ (1 Cycle) | 64.2 | 61.7 | 63.7 | 64.4 | 49.1 | 60.6 | 56.2 | 40.1 | 52.0 | 59.0
XLNet | 79.2 | 53.9 | 72.4 | 77.5 | 40.1 | 65.3 | 71.9∗ | 41.3∗ | 62.7∗ | 56.0∗
$y^{{}^{\prime}}\mapsto y$ (1 Cycle) | 77.8 | 55.0 | 71.8 | 75.3 | 41.6 | 64.8 | 71.5 | 42.7 | 63.1 | 56.5
$x_{h}\mapsto y$ (1 Cycle) | 65.9 | 62.7 | 65.3 | 66.7 | 48.6 | 62.1 | 64.5 | 51.1 | 61.3 | 60.1
$\nabla_{ATK}$ (1 Cycle) | 67.9 | 60.0 | 66.1 | 67.2 | 47.5 | 62.0 | 59.4 | 38.5 | 53.6 | 59.0

Table 4: Evaluation results on the original testing data. The numbers labeled
with “*” refer to the results tested by ourselves with the released
checkpoints from the original papers, while all the left numbers are copied
from the original papers. We report the performance of two variants of
regularization data for each model and the corresponding best cycle times,
where $\mapsto$ represents the direction of data flow. The bold fonts are the
best performance in each comparison. Note that all the non-CSA baselines here
are fine-tuned on the high-quality fine-tuning set (except for LM-Critic,
which is an unsupervised method). The CSA and its variants are trained from
the same checkpoints as baselines.

Model |  BEA  (ERRANT) |  CoNLL-2014  ($M^{2}$) |  FCE  ($M^{2}$) | JFLEG
---|---|---|---|---
Prec. | Rec. | F_0.5 | Prec. | Rec. | F_0.5 | Prec. | Rec. | F_0.5 | GLEU
Transformer | 21.0 | 48.0 | 23.4 | 34.1 | 39.7 | 34.9 | 29.7 | 34.2 | 30.3 | 45.4
$y^{{}^{\prime}}\mapsto y$ (4 Cycles) | 23.9 | 53.3 | 26.6 | 37.6 | 45.5 | 38.8 | 32.5 | 38.8 | 33.4 | 46.4
$x_{h}\mapsto y$ (3 Cycles) | 23.8 | 53.6 | 26.5 | 38.2 | 45.9 | 39.4 | 32.5 | 38.8 | 33.8 | 47.2
$\nabla_{ATK}$ (2 Cycles) | 22.9 | 51.8 | 25.6 | 34.9 | 39.7 | 35.7 | 30.1 | 35.2 | 30.6 | 44.8
BERT-fuse | 20.4 | 46.1 | 22.6 | 33.5 | 38.2 | 34.1 | 31.0 | 34.5 | 31.4 | 45.4
$y^{{}^{\prime}}\mapsto y$ (3 Cycles) | 23.9 | 53.7 | 26.6 | 37.9 | 45.4 | 39.0 | 33.7 | 39.9 | 34.6 | 47.0
$x_{h}\mapsto y$ (2 Cycles) | 23.8 | 53.6 | 26.5 | 37.4 | 45.5 | 38.7 | 33.9 | 40.6 | 34.9 | 47.2
$\nabla_{ATK}$ (1 Cycles) | 24.5 | 54.8 | 27.3 | 38.6 | 46.1 | 39.7 | 33.1 | 39.5 | 33.9 | 47.3
BART | 20.9 | 44.7 | 23.0 | 34.5 | 38.8 | 35.0 | 30.1 | 31.5 | 30.0 | 43.8
$y^{{}^{\prime}}\mapsto y$ (2 Cycles) | 25.0 | 53.9 | 27.7 | 39.1 | 46.0 | 40.1 | 32.1 | 37.5 | 32.9 | 45.8
$x_{h}\mapsto y$ (1 Cycle) | 23.4 | 51.4 | 25.9 | 36.7 | 43.4 | 37.7 | 31.8 | 36.3 | 32.3 | 45.1
$\nabla_{ATK}$ (2 Cycles) | 23.8 | 50.8 | 26.2 | 36.5 | 43.3 | 37.6 | 31.0 | 32.6 | 30.8 | 44.6
LM-Critic | 18.6 | 39.0 | 20.5 | 34.5 | 35.9 | 34.5 | 23.6 | 24.7 | 23.5 | 41.1
$y^{{}^{\prime}}\mapsto y$ (2 Cycles) | 24.5 | 52.1 | 27.1 | 41.1 | 46.0 | 41.8 | 31.9 | 36.3 | 32.5 | 46.1
$x_{h}\mapsto y$ (1 Cycle) | 20.6 | 43.3 | 22.6 | 31.7 | 35.3 | 32.1 | 28.2 | 29.4 | 28.1 | 42.9
$\nabla_{ATK}$ (1 Cycles) | 22.8 | 41.3 | 24.5 | 36.3 | 37.7 | 36.3 | 30.0 | 32.7 | 30.2 | 44.1
BERT | 23.1 | 50.2 | 25.7 | 35.6 | 42.9 | 37.4 | 33.2 | 39.4 | 34.2 | 45.7
$y^{{}^{\prime}}\mapsto y$ (1 Cycle) | 23.4 | 51.3 | 26.1 | 37.4 | 41.9 | 38.0 | 33.7 | 40.1 | 34.6 | 45.8
$x_{h}\mapsto y$ (1 Cycle) | 22.0 | 47.8 | 24.4 | 35.2 | 40.4 | 35.9 | 32.5 | 37.5 | 33.2 | 45.3
$\nabla_{ATK}$ (1 Cycles) | 23.8 | 50.8 | 26.3 | 36.8 | 43.0 | 37.7 | 30.2 | 35.2 | 31.0 | 45.6
RoBERTa | 24.8 | 52.4 | 27.4 | 38.2 | 44.1 | 39.0 | 33.9 | 39.9 | 34.8 | 46.3
$y^{{}^{\prime}}\mapsto y$ (1 Cycle) | 25.0 | 52.7 | 27.6 | 38.7 | 44.6 | 39.5 | 34.3 | 40.3 | 35.1 | 46.5
$x_{h}\mapsto y$ (1 Cycle) | 24.7 | 52.6 | 27.3 | 38.5 | 45.0 | 39.4 | 34.0 | 40.2 | 34.9 | 46.4
$\nabla_{ATK}$ (3 Cycles) | 25.0 | 52.5 | 27.6 | 38.2 | 44.6 | 39.6 | 32.2 | 37.0 | 32.9 | 46.3
XLNet | 25.7 | 54.6 | 28.4 | 38.9 | 46.6 | 40.1 | 36.5 | 44.9 | 37.7 | 47.3
$y^{{}^{\prime}}\mapsto y$ (2 Cycles) | 25.8 | 54.8 | 28.5 | 39 | 46.6 | 40.1 | 36.6 | 44.9 | 37.9 | 47.5
$x_{h}\mapsto y$ (1 Cycle) | 25.0 | 53.0 | 27.6 | 38.1 | 45.0 | 39.1 | 36.3 | 44.0 | 37.4 | 47.3
$\nabla_{ATK}$ (1 Cycles) | 27.2 | 50.0 | 28.3 | 38.9 | 45.8 | 39.8 | 32.5 | 36.5 | 33.1 | 46.5

Table 5: The average of evaluation results on four attack sets and each test set corresponds to four variants for attack, including Mapping & Rules, Synonyms, Back-Translation and Antonym. We report the performance of two variants of regularization data for each model and the corresponding best cycle times, where $\mapsto$ represents the direction of data flow. The bold fonts indicate the optimal performance of each comparison. Model |  ATK 1 ($\uparrow$) |  ATK 2 ($\uparrow$) |  ATK 3 ($\uparrow$)
---|---|---|---
TR(%) | SR(%) | TR(%) | SR(%) | TR(%) | SR(%)
Transformer | 32.6 | 32.6 | 23.5 | 13.3 | 16.1 | 4.1
$y^{{}^{\prime}}\mapsto y$ (4 Cycles) | 37.7 | 37.7 | 26.7 | 15.9 | 18.7 | 5.4
$x_{h}\mapsto y$ (4 Cycles) | 35.6 | 35.6 | 28.2 | 17.6 | 18.2 | 5.4
BERT-fuse | 33.6 | 33.6 | 25.0 | 15.0 | 15.1 | 3.7
$y^{{}^{\prime}}\mapsto y$ (3 Cycles) | 36.0 | 36.0 | 26.6 | 15.4 | 17.9 | 4.9
$x_{h}\mapsto y$ (2 Cycles) | 35.1 | 35.1 | 26.5 | 15.4 | 17.8 | 4.9
BART | 38.2 | 38.2 | 27.2 | 16.6 | 18.1 | 5.3
$y^{{}^{\prime}}\mapsto y$ (2 Cycles) | 38.5 | 38.5 | 28.6 | 18.1 | 18.6 | 5.3
$x_{h}\mapsto y$ (2 Cycles) | 38.2 | 38.2 | 27.0 | 16.9 | 17.8 | 5.2
LM-critic | 34.5 | 34.5 | 23.5 | 12.9 | 15.8 | 4.6
$y^{{}^{\prime}}\mapsto y$ (2 Cycles) | 34.9 | 34.9 | 24.4 | 14.4 | 16.4 | 5.0
$x_{h}\mapsto y$ (1 Cycle) | 34.5 | 34.5 | 26.0 | 15.8 | 16.1 | 4.2
RoBERTa | 36.9 | 36.9 | 26.5 | 16.1 | 16.9 | 4.9
$y^{{}^{\prime}}\mapsto y$ (1 Cycle) | 36.9 | 36.9 | 25.3 | 15.1 | 16.6 | 4.9
$x_{h}\mapsto y$ (1 Cycle) | 37.1 | 37.1 | 26.9 | 16.3 | 17.9 | 6.0
BERT | 33.4 | 33.4 | 23.2 | 13.4 | 15.9 | 4.1
$y^{{}^{\prime}}\mapsto y$ (1 Cycle) | 33.3 | 33.3 | 23.3 | 13.7 | 16.7 | 4.5
$x_{h}\mapsto y$ (1 Cycle) | 31.7 | 31.7 | 22.8 | 13.3 | 15.8 | 4.2
XLNet | 35.5 | 35.5 | 25.8 | 15.4 | 17.7 | 5.1
$y^{{}^{\prime}}\mapsto y$ (2 Cycles) | 36.3 | 36.3 | 26.3 | 15.7 | 8.1 | 5.0
$x_{h}\mapsto y$ (2 Cycles) | 35.7 | 35.7 | 25.4 | 15.4 | 17.7 | 4.9
Table 6: Recovery Rate for ${\rm ATK}_{i}$ $(i\in\\{1,2,3\\})$ sets. We report
both the token level recovery rate (TR) and the sentence level recovery rate
(SR), and the cycle times are correlated with the results on clean data (Table
4). We report the performance of two variants of regularization data for each
model and the corresponding best cycle times, where $\mapsto$ represents the
direction of data flow. The bold fonts indicate the optimal performance of
each comparison.

####  5.3.1 GEC Results

We first present the experimental results on four original testing sets to
calibrate the influence of our proposed CSA to baseline models, where the
detailed numbers are shown in Table 4. It can be seen that with several
additional cycles, our proposed CSA method yields impressive performance
improvement on four baselines, i.e., Transformer, BERT-fuse, BART, and LM-
Critic. Specifically, the improvement is about two points for the supervised
setting (Transformer, BERT-fuse, BART). For the unsupervised setting (LM-
Critic), the improvement is dramatic, e.g., 19.2 for the BEA test set. For the
rest three strong baselines, our proposed method can achieve nearly comparable
results181818Our method achieves better performance on six out of twelve
settings, where each setting refers to the combination of a specific baseline
and a dataset, e.g., CSA achieves higher F_0.5 score than the BERT model on
the JELEG dataset..

####  5.3.2 Attack Results

The results of evaluation results on four different attack sets are put in
Appendix A. In Table 5, we report the average evaluation score on attack sets.
Recall that we construct four variants of attack sets with different methods
for each original test set. To better show the effectiveness of our proposed
CSA method, we utilize the averaged results of four attack sets for each
original test set. It can be observed that our proposed CSA method yields
robustness improvement on all baseline methods. In particular, our CSA methods
leads to improvements on all attack sets, and there existing dramatical
increase of the average score under the $y^{{}^{\prime}}\mapsto y$ setting,
i.e., the improvements of 4.9 ($F_{0.5}$) points over BERT-fuse and 5.1
($F_{0.5}$) points over the BART model on the CoNLL-2014 attack sets. However,
it is worth noting that the original Seq2Edit models all have a better
performance on attack sets than the Seq2Seq models and even can compete with
the performance of some Seq2Seq models, which are reinforced by CSA method.

####  5.3.3 Recovery Rate Results

The results of Recovery Rate are shown in Table 6. We report the TR (token
level recovery rate) and SR (sentence level recovery rate) for each model. It
can be observed that the performance of the Seq2Seq model (Transformer, BERT-
fuse, BART and LM-Critic) is obviously better than that of the Seq2Edit model
(i.e., including RoBERTa, BERT, and XLNet ). We can also see that, with the
increasing number of attack positions, the results of both TR and SR decrease
a lot, and the improvement of CSA method is diminishing. Although the
performance degradation in this Seq2Edit model is not extensive in magnitude,
the slight improvement and unstable performance of Seq2Edit models attract our
interests, and we will dig for the reasons behind it subsequently.

(a) Antonym

(b) Synonym

(c) Mapping & Rules

(d) Back-Translation

Figure 5: Comparison between our CSA and the adversarial training method on
four different attack sets. In each sub-figure, the orange straight square
represents for the traditional defence method, while blue one represents for
our CSA method.

##  6 Analysis and Discussion

In this section, we conduct extensive studies from different perspectives to
better understand our CSA method and regularization data. Since the most
important merit of our CSA method is simplicity and effectiveness in improving
the robustness against multiple types of attack, we first evaluate its defense
capability of CSA with a recently proposed defense method for the GEC task
(Wan et al., 2020) in Section 6.2. Then, we conduct experiments to study the
effects of self-augmenting and hard samples in Section 6.3, followed by hyper-
parameter analysis in Section 6.4. Afterward, we conduct a case study of
regularization data, elaborate the influence brought by two data components
$X_{unl}$ and $X_{unc}$ decomposed from regularization data, and try different
variations of our CSA method in Section 6.5. Finally, we analyze the reason
behind the slight improvement of Seq2Edits framework in Section 6.5.3. These
studies are mainly taken on CoNLL-2014 dataset and the attack set is
constructed by the Mapping & Rules method unless there is a clear explanation,
and all the experiments are launched on the Transformer.

###  6.1 Large Langeage Models Robustness

We evaluate the robustness of GPT-3.5 (text-davinci-003) using the original
CoNLL-2014 testing data and its three types of attack sets. Given that the
format and the exact wording of GPT-3.5’s prompts can significantly impact the
task performance, we conduct zero-shot and few-shot prompt setting proposed by
previous work (Coyne & Sakaguchi, 2023), which performs best in prompt
engineering experiments. The results are listed in Table 7, indicate that
GPT-3.5 suffers from a significant decrease in performance on the attack sets,
e.g., from 34.8 to 28.4 on the Vector-based set in few-shot setting. Despite
being trained on a vast amount of data, GPT-3.5 is still susceptible to a
reduction in its robustness.

Data | Transformer | Davinci-003 $\dagger$ | Davinci-003 $\ddagger$
---|---|---|---
Prec. | Rec. | F_0.5 | Prec. | Rec. | F_0.5 | Prec. | Rec. | F_0.5
Origin | 68.9 | 43.9 | 61.8 | 31.5 | 52.3 | 34.2 | 32.1 | 52.3 | 34.8
Mapping & Rules | 36.5 | 37.8 | 36.7 | 27.6 | 48.2 | 28.1 | 28.0 | 48.0 | 30.6
Vector-Based | 26.5 | 36.0 | 28.0 | 27.6 | 48.2 | 30.2 | 25.9 | 46.6 | 28.4
Back-Translation | 33.0 | 48.9 | 35.3 | 26.4 | 50.0 | 29.1 | 26.5 | 49.7 | 29.2
Table 7: Experimental results on CoNLL-2014 testing data and its correlated
attack set, where $\dagger$ represents the zero-shot setting and $\ddagger$
represents the few-shot setting.

###  6.2 Defense Capability Comparison

previous adversarial training method (Wan et al., 2020), which add Mapping &
Rules attack samples into the training set. Figure 5 presents the evaluation
results on four test sets under the four types mentioned above of adversarial
attack. Our CSA has better defense capability than the baseline model under
three types of attack and achieves comparable results under the Mapping &
Rules attack. In other words, our CSA can achieve competitive results with the
previous defense method, which uses many well-crafted adversarial examples for
training. Our CSA achieves much better model robustness for other attacks
without specifically designed adversarial training examples. These results
demonstrate the effectiveness and generalization ability of our method.

Strategies | #Cycles | #Pairs | CoNLL-2014 ($M^{2}$)
---|---|---|---
Prec. | Rec. | F_0.5
$x\mapsto y^{{}^{\prime}}$ | 0 | 625,467 | 68.9 | 43.9 | 61.8
1 | 511,006 | 66.5 | 51.1 | 62.6
2 | 436,229 | 65.2 | 46.3 | 60.3
$y^{{}^{\prime}}\mapsto y$ | 0 | 625,467 | 68.9 | 43.9 | 61.8
1 | 506,572 | 67.2 | 49.4 | 62.6
2 | 263,993 | 68.3 | 50.3 | 63.8
$x_{h}\mapsto y$ | 0 | 625,467 | 68.9 | 43.9 | 61.8
1 | 24,213 | 68.0 | 50.9 | 63.7
2 | 466,295 | 68.6 | 50.1 | 63.9
Table 8: Results of two strategies for self-augmenting. The first group refers
to the strategy of self-distillation by using failed pairs ($x\mapsto
y^{{}^{\prime}}$) to re-train the model. The second and third group
corresponds to our strategy of using the model outputs to construct
($y^{{}^{\prime}}\mapsto y$) pairs and using hard samples to construct
($x_{h}\mapsto y$).

Model | CoNLL (CLN) | CoNLL (ATK) | ATK 1 | ATK 2 | ATK 3
---|---|---|---|---|---
Prec. | Rec. | F_0.5 | Prec. | Rec. | F_0.5 | TR | SR | TR | SR | TR | SR
Transformer | 68.9 | 43.9 | 61.8 | 34.1 | 39.7 | 34.9 | 32.6 | 32.6 | 23.5 | 13.3 | 16.1 | 4.1
$y^{{}^{\prime}}\mapsto y$ (CSA) | 69.5 | 49.5 | 64.3 | 37.7 | 45.5 | 38.9 | 37.7 | 37.7 | 26.7 | 15.9 | 18.7 | 5.4
$x_{h}\mapsto y$ (CSA) | 69.0 | 50.1 | 64.2 | 38.1 | 45.6 | 39.2 | 35.6 | 35.6 | 28.2 | 17.6 | 18.2 | 5.4
$x_{h}\mapsto y$ (DIR) | 69.4 | 50.8 | 64.6 | 38.6 | 46.1 | 39.7 | 35.9 | 35.9 | 27.2 | 16.1 | 18.3 | 5.8
BERT-fuse | 69.2 | 45.6 | 62.6 | 33.5 | 38.2 | 34.1 | 33.6 | 33.6 | 25.0 | 15.0 | 15.1 | 3.7
$y^{{}^{\prime}}\mapsto y$ (CSA) | 69.4 | 49.8 | 64.4 | 37.9 | 45.5 | 39.0 | 36.0 | 36.0 | 26.6 | 15.4 | 17.9 | 4.9
$x_{h}\mapsto y$ (CSA) | 67.5 | 49.4 | 62.9 | 37.4 | 45.5 | 38.7 | 35.1 | 35.1 | 26.5 | 15.4 | 17.8 | 4.9
$x_{h}\mapsto y$ (DIR) | 67.2 | 50.2 | 62.9 | 37.7 | 45.9 | 38.9 | 35.2 | 35.2 | 26.9 | 15.7 | 18.3 | 5.1
BART | 69.3 | 45.0 | 62.6 | 34.5 | 38.8 | 35.0 | 38.2 | 38.2 | 27.2 | 16.6 | 18.1 | 5.3
$y^{{}^{\prime}}\mapsto y$ (CSA) | 70.5 | 46.7 | 64.0 | 39.1 | 46.1 | 40.1 | 38.5 | 38.5 | 28.6 | 18.1 | 18.6 | 5.3
$x_{h}\mapsto y$ (CSA) | 66.3 | 45.7 | 60.8 | 36.7 | 43.4 | 37.7 | 38.2 | 38.2 | 27.0 | 16.9 | 17.8 | 5.2
$x_{h}\mapsto y$ (DIR) | 65.1 | 47.4 | 60.6 | 37.6 | 45.1 | 37.5 | 39.2 | 39.2 | 27.8 | 17.7 | 19.2 | 5.9
LM-critic | 64.4 | 35.6 | 55.5 | 34.5 | 35.9 | 34.5 | 34.5 | 34.5 | 23.5 | 12.9 | 15.8 | 4.6
$y^{{}^{\prime}}\mapsto y$ (CSA) | 65.7 | 47.4 | 61.0 | 41.1 | 46.1 | 41.8 | 34.9 | 34.9 | 24.4 | 14.4 | 16.4 | 5.0
$x_{h}\mapsto y$ (CSA) | 64.8 | 36.4 | 56.1 | 31.7 | 35.3 | 32.1 | 34.5 | 34.5 | 26.0 | 15.8 | 16.1 | 4.2
$x_{h}\mapsto y$ (DIR) | 58.5 | 35.8 | 51.9 | 31.0 | 35.1 | 31.5 | 34.3 | 34.3 | 24.4 | 14.4 | 16.1 | 5.0
BERT | 72.1 | 42.0 | 63.0 | 35.6 | 42.9 | 37.4 | 33.4 | 33.4 | 23.2 | 13.4 | 15.9 | 4.1
$y^{{}^{\prime}}\mapsto y$ (CSA) | 70.0 | 44.3 | 62.3 | 37.3 | 41.8 | 38.3 | 33.3 | 33.3 | 23.3 | 13.7 | 16.7 | 4.5
$x_{h}\mapsto y$ (CSA) | 68.7 | 42.8 | 61.3 | 35.2 | 40.4 | 35.9 | 31.7 | 31.7 | 22.8 | 13.3 | 15.8 | 4.2
$x_{h}\mapsto y$ (DIR) | 68.0 | 43.6 | 61.2 | 35.6 | 41.2 | 36.3 | 32.6 | 32.6 | 23.4 | 13.5 | 16.2 | 4.4
RoBERTa | 68.7 | 47.2 | 62.9 | 38.2 | 44.1 | 39.0 | 36.9 | 36.9 | 26.5 | 16.1 | 16.9 | 4.9
$y^{{}^{\prime}}\mapsto y$ (CSA) | 68.0 | 46.9 | 62.4 | 38.7 | 44.5 | 39.5 | 36.9 | 36.9 | 25.3 | 15.1 | 16.6 | 4.9
$x_{h}\mapsto y$ (CSA) | 66.3 | 47.7 | 61.5 | 38.5 | 45.0 | 39.4 | 37.1 | 37.1 | 26.9 | 16.3 | 17.9 | 6.0
$x_{h}\mapsto y$ (DIR) | 66.4 | 47.5 | 61.5 | 38.5 | 44.8 | 39.4 | 36.5 | 36.5 | 26.8 | 15.9 | 18.1 | 6.0
XLNet | 77.5 | 40.1 | 65.3 | 38.9 | 46.6 | 40.1 | 35.5 | 35.5 | 25.8 | 15.4 | 17.7 | 5.1
$y^{{}^{\prime}}\mapsto y$ (CSA) | 75.3 | 41.6 | 64.8 | 39.0 | 46.6 | 40.1 | 36.3 | 36.3 | 26.3 | 15.7 | 18.1 | 5.0
$x_{h}\mapsto y$ (CSA) | 66.7 | 48.6 | 62.1 | 38.1 | 45.0 | 39.1 | 35.7 | 35.7 | 25.4 | 15.4 | 17.7 | 4.9
$x_{h}\mapsto y$ (DIR) | 67.8 | 49.1 | 63.0 | 38.6 | 45.7 | 39.6 | 35.6 | 35.6 | 25.9 | 15.6 | 18.1 | 5.0

Table 9: Comparison among different augmenting methods. For each group, the
first three settings were the same as in the previous experiment, while the
last one which is denoted by DIR is conducting the Stage I of the CSA method
directly without filtering the regularization samples. The bold fonts are the
optimal performance of each comparison.

###  6.3 The Effect of Self-Augmenting and Hard Samples

As mentioned before, there are different strategies during the self-augmenting
process. One is to follow the self-distillation method which combines each
incorrect output $y^{{}^{\prime}}$ generated by GEC model with the
corresponding source data $x$ from the original training datasets to post-
train the model. The other strategy is to utilize $D_{self}$ or $D_{hard}$ as
new training set. To illustrate the advancement of our CSA method, we compare
these training strategies by reporting the results on the original testing
data and the number of augmenting pairs per cycle in table 8. Afterward, to
figure out the effect of the regularization data, we utilize augmenting pairs
$D_{Aug}$ to retrain the model directly by applying the Stage I of the CSA
method and report the results in Table 9.

####  6.3.1 Effect of Self-Augmenting

We find that the performance of the baseline model can be improved in the
first training cycle but will decrease after the second cycle under the self-
distillation setting. As for our introduced strategy in the self-augmenting
process, the model performance rises continuously after two training cycles,
with fewer training pairs. This is because the GEC model is sensitive to the
golden data, and the incorrect targets will make the model think that the
syntactically incorrect parts are “correct”. This also confirms the above
statement that the goal of self-distillation method is contrary to the GEC
task.

####  6.3.2 Effect of Regularization data

We can conclude that the over-fitting problem of the GEC model hardly comes up
when simply post-training the model with hard samples since the quantity of
data is sufficient. Specifically, for the Transformer model, we can observe
that the hard samples can simultaneously improve the performance on the
original testing sets and attack sets. Three out of five experiments
(CoNLL(CLN), CoNLL(ATK), ATK 3) indicate the improvement brought by directly
utilizing hard samples for training can even surpass the CSA method. However,
in terms of the overall results, the effects brought by regularization data
are more prominent. In short, on the one hand, we can infer that the success
of our CSA method derives from the hard samples to some extent. On the other
hand, the effects of regularization are beyond hard samples.

###  6.4 Hyper-Parameter Analysis

In the aforementioned experiments, we ignore analysing whether the improvement
comes from the small high-quality data $\mathcal{D}_{tune}$ or the
regularization data $\mathcal{D}_{Reg}$, and the influence of the hyper-
parameter $\epsilon$ and $\mathcal{P}$ remains to be discovered. As a result,
we train the model by simply utilizing the original training data or
regularization data as well as compare the performance under the different
settings of $\epsilon$ in Section 6.4.1. Meanwhile, we strictly control the
training steps and the data selection strategy to study the influence of
hyper-parameter $\mathcal{\bm{P}}$ in Section 6.4.2. In the end, we gradually
reduce the proportion of the regularization data by controlling the reserving
rate $\mathcal{\bm{r}}$ during the training step and compare the performance
of the GEC model in the original testing set and attack test set
simultaneously in Section 6.4.3.

#Cycles | 0 | 1 | 2 | 3 | 4 | 5
---|---|---|---|---|---|---
Ori Set | 61.8 | 56.2 | 57.0 | 56.5 | 57.1 | 55.2
\+ Ours | 61.8 | 62.5 | 62.6 | 62.7 | 62.9 | 62.7
Attack Set | 36.7 | 37.4 | 37.0 | 36.6 | 37.0 | 37.1
\+ Ours | 36.7 | 42.2 | 41.3 | 41.6 | 41.8 | 42.2
Table 10: Comparison between re-training the GEC model on $\mathcal{D}\cup\mathcal{D}_{tune}$ with the same epochs as our CSA, where Ori Set means the original testing set. $\mathcal{\bm{P}}$ | CoNLL-2014 | CoNLL-2014 $(ATK)$
---|---|---
Prec. | Rec. | F_0.5 | Prec. | Rec. | F_0.5
- | 67.9 | 44.1 | 61.3 | 34.1 | 39.7 | 34.9
2 | 67.2 | 49.4 | 62.6 | 40.6 | 44.3 | 41.3
3 | 68.1 | 48.5 | 63.2 | 40.3 | 43.6 | 40.9
4 | 68.2 | 48.9 | 63.2 | 40.6 | 43.7 | 41.2
5 | 68.6 | 48.6 | 63.4 | 40.0 | 43.4 | 40.7
6 | 66.2 | 48.9 | 61.8 | 40.2 | 43.2 | 40.8
Table 11: The influence of $\mathcal{P}$ to model performance. “-” denotes baseline, and $(ATK)$ refers to the attack set. Reserving Rates (%) | CoNLL-2014 $(M^{2})$ | CoNLL-2014 $(ATK)$
---|---|---
P | R | F_0.5 | P | R | F_0.5
0 % | 68.6 | 48.6 | 63.4 | 40.0 | 43.4 | 40.7
25 % | 68.5 | 48.6 | 63.3 | 40.1 | 43.3 | 40.7
50 % | 68.3 | 48.8 | 63.2 | 40.3 | 43.7 | 40.9
75 % | 68.3 | 48.5 | 63.1 | 40.5 | 43.8 | 41.1
100 % | 67.5 | 49.4 | 62.9 | 40.9 | 44.5 | 41.6
Table 12: The influence of regularization data amount to model performance and
defence capability.

####  6.4.1 The Influence of Threshold $\epsilon$

Table 10 presents the results of the model with different training cycles,
correlating with the hyper-parameters of $\epsilon$. With the increase of
training cycles, we can observe that our CSA method can surpass the baseline
on the attack set in each cycle (except for Cycle 5) and significantly improve
the original test set, i.e., around 5 points improvement in Cycle 1. To
explore whether the improvement in cycle training comes from the introduced
small dataset $\mathcal{D}_{tune}$, we train the baseline model on
$\mathcal{D}\cup\mathcal{D}_{tune}$ with the same training epochs as our CSA
method. The dramatic decrease of the performance along with the cycle training
proves that the improvement of robustness and performance on the original
testing set is not simply brought by $\mathcal{D}_{tune}$ in the cycle
training process.

####  6.4.2 The Influence of Patience $\mathcal{P}$

Table 11 shows the performance of the model along with the different settings
of patience $\mathcal{P}$. With the increase of $\mathcal{P}$, the model can
achieve better performance on the original testing set in five cycles, and the
robustness is unstable but much better than the baseline. It can be seen that
$\mathcal{P}$=2 is sufficient to achieve competitive performance on the
original testing set and best performance on the attack set, which is also
used as the standard-setting in our implementations.

####  6.4.3 The Influence of Reserving Rate

We launch a preliminary experiment to show the relationship between the
quantity of regularization data and the model’s performance. Table 12 presents
the experimental results of removing different proportions of regularization
data in the last training cycle. It can be seen that more regularization data
can improve the model robustness but suffer from the decreased performance on
the original testing data. This seemingly incompatible phenomenon validates
previous work (Li et al., 2021) that there is a relative balance between the
performance on the original training set and robustness of the model, and also
reminds us of the possibility of utilizing fewer augmented data to achieve
better results.

###  6.5 The Influence of Regularization Data

In this section, We’ll take a thorough analysis of the impact of the two
components $X_{unl}$ and $X_{unc}$ in the regularization data $D_{Reg}$. We
select some cases of regularization examples and make a case study in Section
6.5.1. To solve the problem of why regularization data can improve the
performance on both attack set and original testing set mentioned in Section
3.3, we utilize $X_{unl}$ and $X_{unc}$ to fine-tune a plug-and-play GEC pre-
trained model (transformer-big) to probe the latent capacity brought by the
two data components in Section 6.5.2. Since there are some unexpected
phenomena of the Seq2Edits models, i.e., performing badly after post-training
with regularization data, we will analyze such phenomena and propose a set of
paradigms for different models about how to utilize regularization data
efficiently in Section 6.5.3 and 6.5.4, respectively.

####  6.5.1 Case Study

Examples of Regularization Data
---
Poor: | I hope you ’ll attend my suggestions .
Reg 1,2,3,4: | I hope you ’ll follow my suggestions .
Good: | I hope you ’ll act on my suggestions .
Poor: | So It was very derty.
Reg 1,2,3,4: | So it was very dirty.
Good: | So It was very dirty.
Poor: | I was very frethend, but I knew , that I should do something .
Reg 1,2,3,4: | I was very free, but I knew that I should do something .
Good: | I was very frightened, but I knew that I had to do something .
Poor: | But I do n’t stop to walk .
Reg 1: | But I did n’t stop walking .
Reg 2,3,4: | But I do n’t stop walking .
Good: | But I did n’t stop walking .
Poor: | Today, It ’s fist time to entry my message .
Reg 1: | Today is my first time to write a message .
Reg 2,3,4: | Today, It ’s my first time to write a message .
Good: | Today is the first time for me to enter a message .
Poor: | I thank you about my normal day .
Reg 1: | Thank you for my normal day .
Reg 2,3: | I thank you for my normal day .
Reg 4: | I am thankful for my normal day .
Good: | I thank you for my normal day .
Poor: | Hiroshima Carp lost the game, but it was very exciting game .
Reg 1: | Hiroshima Carp lost the game, but it was a very exciting game .
Reg 2,3: |  The Hiroshima Carp lost the game, but it was a very exciting game .
Reg 4: | Hiroshima Carp lost the game, but it was a very exciting game .
Good: | Hiroshima Carp lose the game, but it was very exciting game .
Table 13: Examples of regularization data, where Poor, Reg k and Good
represent for ungrammatical sentence, regularization data from $k$-th cycle
and grammatical sentence respectively. We annotate the different positions of
each sentence for three kinds of data by utilizing red, blue and olive color.
We sample the sentences from four cycles of regularization data generated by
Seq2Seq model. To save space for the table, if there are multiple cycles of
regularization with the same data, we will put them on the same row.

As mentioned in Section 3.3, we decompose the regularization data into two
components $X_{unl}$ and $X_{unc}$, and utilize those datasets to fine-tune
the GEC model directly. Similar to the process of building $D_{Reg}$, we take
the intersection among different regularization data sets to extract
$X_{unl}$, which can be formally written as
$X_{unl}^{k}=\mathcal{D}_{Reg}^{1}\bigcap\mathcal{D}_{Reg}^{2}\dots\bigcap\mathcal{D}_{Reg}^{k}$
$(k\in\\{2,3,4\\})$. As for $X_{unc}^{k}$ of the $k$-th cycle, we first
combine all the regularization data of $k$ cycles and subtract the
corresponding $X_{unl}^{k}$ set, which can be formally written as
$X_{unc}^{k}=\mathcal{D}_{Reg}^{1}\bigcup\mathcal{D}_{Reg}^{2}\dots\bigcup\mathcal{D}_{Reg}^{k}-X_{unl}^{k}$
$(k\in\\{2,3,4\\})$. Afterward, we select some representative cases from
$X_{unl}$ and $X_{unc}$, and present them in Table 23, where the first three
cases are $X_{unl}$ cases, and the others are $X_{unc}$ cases. We summarize
five main types of regularization data as well as show some examples below:

  * Type 1

The intermediate results are syntactic and semantic correct, while the editing
positions are replaced by synonyms texts. For example, in the first case, the
regularization sentences have no syntactic error and these two phrases “follow
my suggestion” and “act on my suggestion” have the same meaning.

  * Type 2

The target sentence is mislabeled, i.e., in the second case which convert “it”
to “It”. Such mislabeled samples impair the correction ability of the GEC
model as they confuse the optimization goal, i.e., transferring
$\textit{Poor}\rightarrow\textit{Good}$ to
$\textit{Good}\rightarrow\textit{Poor}$.

  * Type 3

The intermediate results are syntactic correct but semantic incorrect. For
example, in the third case, the goal is to correct the spelling of the word
“frightened” but the regularization sentence change the meaning of original
sentence from “being frightened” to “being free”.

  * Type 4

Because of the lack of prompt message, the intermediate results may have
syntactic errors. For example, in the forth case, the original sentence (Poor)
is in the simple present tense and the target sentence is in the simple past
tense, but there is no obvious hint, which cause the changing of the generated
sentence’s tense.

  * Type 5

For sentences which do not require modification, the intermediate results are
excessively modified as shown in the last case.

From the above classification and explanation, we can summarize that the
intermediate results are mostly syntactic correct but semantic incorrect,
which impairs the performance. From the perspective of data quality, the
reasons behind this are the lack of enough context for Seq2Seq models, and
some mislabeled data interfere with optimization objectives. From another
aspect, we also think the shortcomings inherent in the Casual Language Model
(CLM) count for this problem. Specifically, Seq2Seq model suffers from the
degeneration (Li et al., 2016) and over-reasoning problems191919The generating
model tends to generate the generic, safe, universal text., and that is why it
generates different variants of sentences even there is no need to modify the
Poor sentences.

To verify this opinion, we collect the regularization data among different
cycles and plot the recall and precision of editing operation in operation
tier (Bryant, 2019) in Figure 6, where the above three sub-figures describe
the recall and the bottom three describe the precision of editing operation,
respectively.

(a) Missing(R)

(b) Replacement(R)

(c) Unnecessary(R)

(d) Missing(P)

(e) Replacement(P)

(f) Unnecessary(P)

Figure 6: The trend of error type distribution along with the number of
cycles. The above three figures describes the recall (R) of each error types
while the bottom three figures describes the precision (P) of each error
types. The error types are classified in terms of edit operation tier, i.e.,
whether tokens are missing (M), replaced (R) or unnecessary (U).

As we can infer from the recall rate that the editing results (intermediate
results) of Seq2Seq model changes considerably with the increase of the number
of cycles, especially for Unnecessary and Missing error types, which
represents that the model tends to add or delete different tokens when
generating the results. Compared with recall for each error type, the
precision tends to be stable, which means many editing operations are
redundant. In general, the editing ability is enhanced by the addition of
regularization data, which counts for the improving performance of the GEC
model.

####  6.5.2 Comparison between $X_{unl}$ and $X_{unc}$

Model |  ATK 1 ($\uparrow$) |  ATK 2 ($\uparrow$) |  ATK 3 ($\uparrow$)
---|---|---|---
TR(%) | SR(%) | TR(%) | SR(%) | TR(%) | SR(%)
Transformer (pre-train) | 35.4 | 35.4 | 27.5 | 16.1 | 19.2 | 6.1
Transformer (benchmark) | 32.6 | 32.6 | 23.5 | 13.3 | 16.1 | 4.1
+$X_{unl}$ (2 Cycles) | 34.4 | 34.4 | 26.8 | 15.7 | 17.4 | 4.4
+$X_{unl}$ (3 Cycles) | 34.0 | 34.0 | 27.2 | 16.1 | 17.2 | 4.4
+$X_{unl}$ (4 Cycles) | 33.4 | 33.4 | 26.7 | 15.3 | 17.2 | 4.6
+$X_{unl}$ (2 Cycles / reverse) | 34.5 | 34.5 | 27.7 | 17.2 | 17.8 | 4.9
+$X_{unl}$ (3 Cycles / reverse) | 34.4 | 34.4 | 27.4 | 16.9 | 17.1 | 4.0
+$X_{unl}$ (4 Cycles / reverse) | 34.6 | 34.6 | 27.1 | 16.5 | 18.1 | 5.1
+$X_{unc}$ (2 Cycles / seed 1) | 34.9 | 34.9 | 27.0 | 16.2 | 18.2 | 5.0
+$X_{unc}$ (2 Cycles / seed 2) | 35.1 | 35.1 | 27.4 | 16.5 | 18.9 | 5.4
+$X_{unc}$ (2 Cycles / seed 3) | 34.6 | 34.6 | 27.5 | 16.8 | 18.0 | 5.0
+$X_{unc}$ (2 Cycles / seed 4) | 35.1 | 35.1 | 27.5 | 16.7 | 18.4 | 5.3
+$X_{unc}$ (2 Cycles / seed 5) | 34.0 | 34.0 | 27.2 | 16.3 | 18.2 | 5.3
+$X_{unc}$ (2 Cycles / Avg. ) | 34.7 | 34.7 | 27.3 | 16.5 | 18.3 | 5.2
+$X_{unc}$ (3 Cycles / seed 1) | 35.0 | 35.0 | 27.3 | 16.7 | 18.4 | 5.4
+$X_{unc}$ (3 Cycles / seed 2) | 35.1 | 35.1 | 26.8 | 16.0 | 18.7 | 5.5
+$X_{unc}$ (3 Cycles / seed 3) | 34.7 | 34.7 | 27.5 | 16.9 | 18.0 | 5.0
+$X_{unc}$ (3 Cycles / seed 4) | 34.7 | 34.7 | 27.2 | 16.3 | 18.2 | 5.0
+$X_{unc}$ (3 Cycles / seed 5) | 34.7 | 34.7 | 27.8 | 17.1 | 18.8 | 5.5
+$X_{unc}$ (3 Cycles / Avg. ) | 34.8 | 34.8 | 27.3 | 16.6 | 18.4 | 5.3
+$X_{unc}$ (4 Cycles / seed 1) | 35.0 | 35.0 | 27.6 | 17.3 | 18.7 | 5.4
+$X_{unc}$ (4 Cycles / seed 2) | 35.1 | 35.1 | 26.8 | 16.1 | 18.2 | 5.3
+$X_{unc}$ (4 Cycles / seed 3) | 34.0 | 34.0 | 27.3 | 16.8 | 17.7 | 4.8
+$X_{unc}$ (4 Cycles / seed 4) | 33.9 | 33.9 | 27.3 | 16.8 | 18.4 | 5.2
+$X_{unc}$ (4 Cycles / seed 5) | 34.4 | 34.4 | 26.4 | 15.9 | 18.2 | 5.1
+$X_{unc}$ (4 Cycles / Avg. ) | 34.5 | 34.5 | 27.1 | 16.6 | 18.2 | 5.2
Table 14: Comparison of the recovery rate among the regularization data variants. ATK $i$ ($i\in\\{1,2,3\\}$) represents for the evaluation sets with fixed number of attack positions per sentence, i.e., $i$ represents for the number of attack positions. We report the TR and SR score for each model, and annotate the cycle time $k$ of filtered regularization data. The bold fonts indicate the optimal performance of each comparison or the average score of five seeds. Table 15: Comparison of model performance among regularization data variants. CLN and ATK represents for original test data and standard attack set of CoNLL-2014 corpus respectively. We report the F_0.5 score and the corresponding #IPS value. We regard the F_0.5 of Transformer (pre-train) model as $score^{i}$ for calculating #IPS. The bold fonts indicate the optimal performance of each comparison or the average score of five. It is worth noting that we keep two decimal places in the actual calculation, but we only keep one decimal place in this report. Model | #Pairs | CoNLL-2014 (CLN) | CoNLL-2014 (ATK)
---|---|---|---
F_0.5 ($\uparrow$) |  #IPS ($\downarrow$) |  F_0.5 ($\uparrow$) |  #IPS ($\downarrow$)
Transformer (pre-train) | 9,000,000 | 50.1 | - | 36.0 | -
Transformer (benchmark) | 659,775 | 61.8 | 5,658.4 | 34.9 | -59,979.5
+$X_{unl}$ (2 Cycles) | 298,301 | 62.7 | 2,375.0 | 38.9 | 10,198.3
+$X_{unl}$ (3 Cycles) | 271,499 | 62.6 | 2,179.0 | 38.8 | 9,696.4
+$X_{unl}$ (4 Cycles) | 259,780 | 62.8 | 2,052.0 | 38.8 | 9,195.8
+$X_{unl}$ (2 Cycles / reverse) | 298,301 | 62.5 | 2,413.4 | 38.6 | 11,584.5
+$X_{unl}$ (3 Cycles / reverse) | 271,499 | 62.9 | 2,127.7 | 38.7 | 10,245.3
+$X_{unl}$ (4 Cycles / reverse) | 259,780 | 62.7 | 2,068.3 | 38.7 | 9,711.4
+$X_{unc}$ (2 Cycles / seed 1) | 298,301 | 62.4 | 2,433.1 | 38.8 | 10,847.3
+$X_{unc}$ (2 Cycles / seed 2) | 298,301 | 62.8 | 2,356.2 | 38.9 | 10,286.2
+$X_{unc}$ (2 Cycles / seed 3) | 298,301 | 62.7 | 2,375.0 | 38.8 | 10,653.6
+$X_{unc}$ (2 Cycles / seed 4) | 298,301 | 62.6 | 2,394.1 | 38.7 | 10,946.8
+$X_{unc}$ (2 Cycles / seed 5) | 298,301 | 62.6 | 2,394.1 | 38.7 | 11,151.4
+$X_{unc}$ (2 Cycles / Avg. ) | 298,301 | 62.6 | 2,390.5 | 38.8 | 10,777.1
+$X_{unc}$ (3 Cycles / seed 1) | 271,499 | 62.4 | 2,214.5 | 38.5 | 10,752.4
+$X_{unc}$ (3 Cycles / seed 2) | 271,499 | 62.8 | 2,144.5 | 38.8 | 9,783.7
+$X_{unc}$ (3 Cycles / seed 3) | 271,499 | 61.9 | 2,308.7 | 38.5 | 10,860.0
+$X_{unc}$ (3 Cycles / seed 4) | 271,499 | 62.7 | 2,161.6 | 38.7 | 10,245.3
+$X_{unc}$ (3 Cycles / seed 5) | 271,499 | 62.8 | 2,144.5 | 38.8 | 9,872.7
+$X_{unc}$ (3 Cycles / Avg. ) | 271,499 | 62.5 | 2,144.8 | 38.6 | 10,302.8
+$X_{unc}$ (4 Cycles / seed 1) | 259,780 | 62.6 | 2,084.9 | 38.7 | 9,533.2
+$X_{unc}$ (4 Cycles / seed 2) | 259,780 | 63.0 | 2,020.1 | 38.7 | 9,621.5
+$X_{unc}$ (4 Cycles / seed 3) | 259,780 | 62.9 | 2,035.9 | 38.8 | 9,631.4
+$X_{unc}$ (4 Cycles / seed 4) | 259,780 | 62.3 | 2,136.3 | 38.6 | 9,991.5
+$X_{unc}$ (4 Cycles / seed 5) | 259,780 | 62.3 | 2,136.3 | 38.5 | 10,603.3
+$X_{unc}$ (4 Cycles / Avg. ) | 259,780 | 62.6 | 2,082.7 | 38.7 | 9,822.2

To figure out the effect of two data components $X_{unl}$ and $X_{unc}$, we
utilize those data sets respectively to fine-tune the plug-and-play GEC pre-
trained model (Kiyono et al., 2019)202020https://gec-pseudo-data.s3-ap-
northeast-1.amazonaws.com/ldc_giga.pret.checkpoint_last.pt and compare the
model’s robustness and performance on the original testing set. We divide the
experiments into two groups: one is conducted on ATK $i$ ($i\in\\{1,2,3\\}$)
to compare the robustness, and the other one is conducted on original testing
set and standard CoNLL-2014 attack set. Considering that the $X_{unl}^{k}$ set
always contains more data than the $X_{unc}^{k}$ set, we count the number of
pairs in the $X_{unl}^{k}$ set and randomly sample the same number of
sentences from the $X_{unc}^{k}$ set. In order to reduce the randomness of
sampling, we perform the above sampling operation five times with different
random seeds and report the average result.

To calibrate the data quality, we also propose an evaluation metric #IPS
(Improvement Per Score) which can be formally written as:
$\text{\\#IPS}=\frac{\\#N_{sen}}{(score^{t}-score^{i})/\gamma}$, where
$\\#N_{sen}$ represents for the number of sentences, $score^{t}-score^{i}$
represents for the difference value of scores, and $\gamma$ represents for the
zoom factor. We set $\gamma$ as 0.1 and utilize the #IPS score to calibrate
the quality of GEC data.

The results are shown in Table 14 and 15. We report the performance of plug-
and-play GEC model in the first group of each table. We can see that the pre-
trained GEC model has surprising strong robustness as it has the highest
Recovery Rate on ATK 1 and ATK 2 sets. Although the performance of fine-tuned
GEC model on the original testing data is much higher than the pre-trained
one, it has weak resistance to the attack data and even gets a negative value
of #IPS score.

##### The Influence of $X_{unl}$

The results of $X_{unl}$ are shown in the second group and third group of each
table. With the increase of $k$ (Cycles), the quality of $X_{unl}$ becomes
higher since the model achieve a relatively high Recovery Rate and the #IPS
score is the lowest in the forth cycle compared with other datasets, i.e., we
can utilize fewer data to achieve a comparable performance. Moreover, as
mentioned above, the type of errors in $X_{unl}$ is generally semantic but not
syntactic, so we reverse the data pairs and use these data to fine-tune the
pre-trained GEC model. The results are shown in the third group of the table.
We surprisingly find that using reversed $X_{unl}$ can also improve the model
robustness and performance on the original testing data and has the same trend
as the original $X_{unl}$ with the increase of $k$. One explanation for this
phenomenon is that most of the source sentence (Poor) in $X_{unl}$ set is
grammatical in the syntactic level, and the reversed pairs can also make up
for mislabeling problem as mentioned in Section 6.5.1.

##### The Influence of $X_{unc}$

The results of $X_{unc}$ are shown from the fourth to the fifth group of each
table. Each group shows the results of five seeds and one average at the
bottom. We can infer that the $X_{unc}$ is the main component for improving
the model robustness as it achieves better Recovery Rate scores than $X_{unl}$
in ATK sets. Moreover, although the F_0.5 score on standard CoNLL-2014 (ATK)
set of $X_{unc}^{k}$ and corresponding $X_{unl}^{k}$ is similar, Recovery Rate
of $X_{unc}^{k}$ on ATK $i$ set are higher than the F_0.5 of $X_{unl}^{k}$,
which means the GEC model trained with $X_{unc}^{k}$ is more sensitive to the
robustness of local attacks. In other words, to improve the robustness of the
local attaks, more quantity of data is required as the #IPS score of
$X_{unc}^{k}$ on CoNLL-2014 (ATK) set in Table 15 is much higher than the #IPS
of the corresponding $X_{unl}^{k}$. As for the model performance on the
original testing data, utilizing $X_{unc}$ does improve it and #IPS score for
each $X_{unc}^{k}$ is close to the #IPS of the corresponding $X_{unl}^{k}$ as
shown in Table 15, i.e., 2,82.7 for $X_{unc}^{k}$ and 2,052.0 for
$X_{unl}^{k}$.

####  6.5.3 Analysis of Anomalies on Seq2Edit models

Examples of Regularization Data
---
Poor: | I was very frethend, but I knew , that I should do something .
BERT: | I was very frightened, but I knew that I should do something .
Transformer: | I was very free, but I knew that I should do something .
Good: | I was very frightened, but I knew that I had to do something .
Poor: | It is a free soft wear which is serviced by Google .
BERT: | It is a free software which is provided by Google .
Transformer: | It is a free software which is serviced by Google .
Good: | It is a free software provided by Google .
Poor: | I thank you about my normal day .
BERT: | I thank you for my normal day .
Transformer: | I am thankful for my normal day .
Good: | I thank you for my normal day .
Poor: | I have every day, holding a pillow in bed .
BERT: | I have every day, holding a pillow in bed .
Transformer: | I have a pillow every day in bed .
Good: | I have the whole day to hold a pillow in my bed .
Table 16: Comparison of regularization data between Seq2Edit model and Seq2Seq
model, where Poor and Good represent for ungrammatical sentence and
grammatical sentence respectively. We use BERT and Transformer as the
representatives of the Seq2Edit and Seq2Seq model architecture. We annotate
the different positions by utilizing red and blue colors for Poor and Good
sentences. As for regularization data belong to different model architectures,
we utilize orange and brown colors to annotate the differences.

In order to figure out the slight improvement and unstable performance of the
Seq2Edit model in the previous experiments, we firstly compare the generated
intermediate data between the Seq2Edit model and Seq2Seq model by utilizing
Transformer and BERT as representatives. We selected some representative cases
from each intermediate data after two cycles of our CSA method and presented
them in Table 16. Combined with the six main types of regularization data
generated by Seq2Seq model (Section 6.5.1), We can find some characteristics
of the results generated by Seq2Edit model:

  * •

The Seq2Edit model tends to modify more for syntactic errors than for the
semantics of the Poor sentences. For example, in the first case, the Seq2Seq
model changes the original semantics while Seq2Edit just corrects the wrong
spelling of ’frightened’.

  * •

Compared with Seq2Seq model, Seq2Edit model seems to be less context-dependent
when correcting syntax errors. For example, in the second case, both
’serviced’ and ’provided’ can be used in this sentence if there is no context,
while Seq2Edit can choose the latter one which matches the Good sentences.

  * •

The over-reasoning problem is mitigated in the Seq2Edit model, and editing
positions are fewer than Seq2Seq model generally. In the third case, the
outcome by Seq2Edit model only edits one position while the Seq2Seq generates
another piece of content, so is the fourth case.

From the above case study, we can infer that Seq2Edit tends to keep the
structure and semantics of the original sentence. Although this editing-based
method is highly accurate, controllable, and interpretable, it to some extent
goes against our CSA method since the regularization data generated by
Seq2Edit model is not as informative as the Seq2Seq model. Moreover, since the
set of tags is generally not too large for Seq2Edit model, it works better on
closed sets, i.e., the official test datasets, but the complexities and long
texts on open sets are challenging to deal with. In other words, the
limitation of tags, on the one hand, can improve the accuracy of editing
operation on what the model has seen. On the other hand, it hurts the
improvement in robustness since some co-occurrence tags never appeal in the
dictionary, and the addition of regularization cannot change the distribution
of tags. Even worse, the released checkpoints of such model (Seq2Edit) have
already been meticulously trained on existing data, and any further post-
training may hurt their performance.

To verify the above conjecture, we investigate the correction results for each
error type. Specifically, We use ERRANT(Felice et al., 2016; Bryant et al.,
2017) to measure the precision and recall of the model for each error type. We
compare the original plug-and-play GEC models with their enhanced versions,
which are reinforced with our CSA method. The results are shown in Table 17.

Error Type | Transformer | \+ CSA (2 cycles) | BERT | \+ CSA (2 cycles)
---|---|---|---|---
Prec. | Rec. | Prec. | Rec. | Prec. | Rec. | Prec. | Rec.
Missing | 50.3 | 30.2 | 55.8 | 51.4 | 57.1 | 50.2 | 55.4 | 47.5
Replacing | 53.9 | 35.5 | 53.9 | 40.3 | 51.9 | 33.2 | 54.2 | 35.5
Unnecessary | 44.7 | 40.1 | 48.4 | 44.0 | 49.6 | 38.9 | 45.4 | 36.3
PUNCT | 61.5 | 25.5 | 61.7 | 56.5 | 60.3 | 55.6 | 58.8 | 51.6
OTHER | 31.1 | 15.6 | 30.9 | 17.5 | 29.7 | 12.0 | 24.1 | 11.5
DET | 52.8 | 46.6 | 55.8 | 53.1 | 57.2 | 53.1 | 56.1 | 48.7
PREP | 52.8 | 38.9 | 52.3 | 45.4 | 52.3 | 41.6 | 51.7 | 38.0
VERB:TENSE | 49.8 | 33.0 | 52.9 | 40.4 | 51.9 | 37.0 | 50.7 | 31.9

Table 17: Comparison of the Seq2Seq model and Seq2Edit model on most error
types including all the top-5 frequent types of errors in W&I-Dev evaluation
set. We report the precision and recall for each error type. The first group
represents the operation tier error types, and the second group shows the
top-5 frequent types of errors. The bold fonts indicate the optimal
performance of each comparison.

As we can see from the Table, the original Seq2Edit model does much better
than the original Seq2Seq model in the missing and unnecessary error types
which are corresponding to insert and delete operations, and the editing-based
operation also exceeds the generating-based operation on the four out of the
top-5 frequent types of errors. However, after the CSA method, We can also
surprisingly find the considerable improvement of Seq2Seq model on the top-5
frequent types of errors, and it exceeds Seq2Edit model on all the error
types. We can also find that the performance of Seq2Edit model gets worse with
the addition of the regularization data.

####  6.5.4 Paradigms for Utilizing Regularization Data

From the above experiments and analysis, we find different effects of
$X_{unc}$ set and $X_{unl}$ set on improving the performance, and we also
compare the influence of regularization data between the Seq2Seq model and the
Seq2Edit model.

In this section, we summarize the use of regularization data for different
model architectures. For Seq2Seq model, the $X_{unc}$ set in regularization
data contains much information which can make the model more robust and
generalized since such model tends to generate diverse outcomes due to the
decoding strategies, while the $X_{unl}$ set serves to compensate for the
model’s natural shortcomings because this data component force the model to
reduces the diversity and reinforce the distributions which are not fully
learned.

For Seq2Edit model, lack of information in regularization data and the
limitation of tags result in the side effects of regularization data on the
performance and robustness. However, there is no need to worry about this
situation, as the original Seq2Edit models work well on the closed sets of
evaluation data due to the powerful encoder of PLMs.

##  7 Conclusion

Recent works have revealed that Seq2Seq GEC models (even with data
augmentation) are vulnerable to adversarial examples Wan et al. (2020).
Nevertheless, those previous works mainly rested on simplex attack sets and
tedious defense methods, i.e., training with adversarial samples, and the
scope of the evaluation is narrow. In this paper, we further explore the
robustness of the GEC models by thoroughly evaluating various types of
adversarial attacks and implementing an effective cycle self-augmenting method
to improve model robustness. Specifically, with our method, the model can be
improved by 1.0 point (F_0.5) on the original testing set and 3.9 points
(F_0.5) on the attack set by utilizing only about $39.4\%$ of the original
data without requiring well-crafted adversarial examples at scale for a
specific type of adversarial attack. Experimental results on seven strong
baselines, four benchmark test sets, five types of adversarial attacks, and
two newly introduced evaluation metrics confirm the effectiveness of our
proposed method, which can generalize well to various GEC models with only a
few more training epochs as the extra cost. Moreover, we also indicate that
the reason behind the improvement is attributed to two data components in the
regularization data, where one data component contributes to improving the
correction ability while the other component focuses on improving the
robustness. In the future, we will explore how to improve the performance of
the Seq2Edit model on the open attack sets and some more efficient ways to use
regularization data.

## Acknowledge

We would like to thank the efforts of anonymous reviewers for improving this
paper.

## References

  * Awasthi et al. (2019) Abhijeet Awasthi, Sunita Sarawagi, Rasna Goyal, Sabyasachi Ghosh, and Vihari Piratla.  Parallel iterative edit models for local sequence transduction.  In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)_ , pp. 4260–4270, 2019.
  * Bryant & Briscoe (2018) Christopher Bryant and Ted Briscoe.  Language model based grammatical error correction without annotated training data.  In _Proceedings of the thirteenth workshop on innovative use of NLP for building educational applications_ , pp. 247–253, 2018.
  * Bryant et al. (2017) Christopher Bryant, Mariano Felice, and Ted Briscoe.  Automatic annotation and evaluation of error types for grammatical error correction.  In _Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pp. 793–805, 2017.
  * Bryant et al. (2019) Christopher Bryant, Mariano Felice, Øistein E. Andersen, and Ted Briscoe.  The BEA-2019 shared task on grammatical error correction.  In _Proceedings of the Fourteenth Workshop on Innovative Use of NLP for Building Educational Applications_ , pp. 52–75, Florence, Italy, August 2019. Association for Computational Linguistics.  doi: 10.18653/v1/W19-4406.  URL https://aclanthology.org/W19-4406.
  * Bryant (2019) Christopher Jack Bryant.  _Automatic annotation of error types for grammatical error correction_.  PhD thesis, University of Cambridge, 2019.
  * Chollampatt & Ng (2018) Shamil Chollampatt and Hwee Tou Ng.  A multilayer convolutional encoder-decoder neural network for grammatical error correction.  In _Proceedings of the AAAI Conference on Artificial Intelligence_ , volume 32, 2018.
  * Coyne & Sakaguchi (2023) Steven Coyne and Keisuke Sakaguchi.  An analysis of gpt-3’s performance in grammatical error correction.  _arXiv preprint arXiv:2303.14342_ , 2023.
  * Dahlmeier & Ng (2012) Daniel Dahlmeier and Hwee Tou Ng.  Better evaluation for grammatical error correction.  In _Proceedings of the 2012 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_ , pp. 568–572, 2012.
  * Dahlmeier et al. (2013) Daniel Dahlmeier, Hwee Tou Ng, and Siew Mei Wu.  Building a large annotated corpus of learner english: The nus corpus of learner english.  In _Proceedings of the eighth workshop on innovative use of NLP for building educational applications_ , pp. 22–31, 2013.
  * Devlin et al. (2019) Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova.  Bert: Pre-training of deep bidirectional transformers for language understanding.  In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)_ , pp. 4171–4186, 2019.
  * Dong et al. (2021) Xinshuai Dong, Anh Tuan Luu, Rongrong Ji, and Hong Liu.  Towards robustness against natural language word substitutions.  _International Conference of Representation Learning_ , 2021.
  * Duan & Hsu (2011) Huizhong Duan and Bo-June Hsu.  Online spelling correction for query completion.  In _Proceedings of the 20th international conference on World wide web_ , pp. 117–126, 2011.
  * Edunov et al. (2018) Sergey Edunov, Myle Ott, Michael Auli, and David Grangier.  Understanding back-translation at scale.  In _Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing_ , pp. 489–500, 2018.
  * Efraimidis & Spirakis (2006) Pavlos S Efraimidis and Paul G Spirakis.  Weighted random sampling with a reservoir.  _Information Processing Letters_ , 97(5):181–185, 2006.
  * Felice et al. (2016) Mariano Felice, Christopher Bryant, and Ted Briscoe.  Automatic extraction of learner errors in esl sentences using linguistically enhanced alignments.  In _Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers_ , pp. 825–835, 2016.
  * Fitria (2021) Tira Nur Fitria.  Grammarly as ai-powered english writing assistant: Students’ alternative for writing english.  _Metathesis: Journal of English Language, Literature, and Teaching_ , 5(1):65–78, 2021.
  * Gao et al. (2010) Jianfeng Gao, Xiaolong Li, Daniel Micol, Chris Quirk, and Xu Sun.  A large scale ranker-based system for search query spelling correction.  In _Proceedings of the 23rd International Conference on Computational Linguistics (Coling 2010)_ , pp. 358–366, 2010.
  * Ge et al. (2018) Tao Ge, Furu Wei, and Ming Zhou.  Fluency boost learning and inference for neural grammatical error correction.  In _Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pp. 1055–1065, 2018.
  * Geng et al. (2021) Xinwei Geng, Xiaocheng Feng, and Bing Qin.  Learning to rewrite for non-autoregressive neural machine translation.  In _Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing_ , pp. 3297–3308, 2021.
  * Graff et al. (2003) David Graff, Junbo Kong, Ke Chen, and Kazuaki Maeda.  English gigaword.  _Linguistic Data Consortium, Philadelphia_ , 4(1):34, 2003.
  * Granger (1998) Sylviane Granger.  Prefabricated patterns in advanced efl writing: Collocations and lexical phrases.  _Phraseology: Theory, analysis and applications_ , pp. 145–160, 1998.
  * Grundkiewicz et al. (2019) Roman Grundkiewicz, Marcin Junczys-Dowmunt, and Kenneth Heafield.  Neural grammatical error correction systems with unsupervised pre-training on synthetic data.  In _Proceedings of the Fourteenth Workshop on Innovative Use of NLP for Building Educational Applications_ , pp. 252–263, 2019.
  * Hagen et al. (2017) Matthias Hagen, Martin Potthast, Marcel Gohsen, Anja Rathgeber, and Benno Stein.  A large-scale query spelling correction corpus.  In _Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval_ , pp. 1261–1264, 2017.
  * Holtzman et al. (2019) Ari Holtzman, Jan Buys, Li Du, Maxwell Forbes, and Yejin Choi.  The curious case of neural text degeneration.  _arXiv preprint arXiv:1904.09751_ , 2019.
  * Huang et al. (2021) Xiao Shi Huang, Felipe Perez, and Maksims Volkovs.  Improving non-autoregressive translation models without distillation.  In _International Conference on Learning Representations_ , 2021.
  * Ji et al. (2017) Jianshu Ji, Qinlong Wang, Kristina Toutanova, Yongen Gong, Steven Truong, and Jianfeng Gao.  A nested attention neural hybrid model for grammatical error correction.  In _Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pp. 753–762, 2017.
  * Jia & Liang (2017) Robin Jia and Percy Liang.  Adversarial examples for evaluating reading comprehension systems.  In _Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing_ , pp. 2021–2031, 2017.
  * Junczys-Dowmunt et al. (2018) Marcin Junczys-Dowmunt, Roman Grundkiewicz, Shubha Guha, and Kenneth Heafield.  Approaching neural grammatical error correction as a low-resource machine translation task.  In _Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers)_ , pp. 595–606, 2018.
  * Kaneko et al. (2020) Masahiro Kaneko, Masato Mita, Shun Kiyono, Jun Suzuki, and Kentaro Inui.  Encoder-decoder models can benefit from pre-trained masked language models in grammatical error correction.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pp. 4248–4254, 2020.
  * Karat et al. (1999) Clare-Marie Karat, Christine Halverson, Daniel Horn, and John Karat.  Patterns of entry and correction in large vocabulary continuous speech recognition systems.  In _Proceedings of the SIGCHI conference on Human Factors in Computing Systems_ , pp. 568–575, 1999.
  * Kasewa et al. (2018) Sudhanshu Kasewa, Pontus Stenetorp, and Sebastian Riedel.  Wronging a right: Generating better errors to improve grammatical error detection.  In _Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing_ , pp. 4977–4983, 2018.
  * Katsumata & Komachi (2020) Satoru Katsumata and Mamoru Komachi.  Stronger baselines for grammatical error correction using a pretrained encoder-decoder model.  In _Proceedings of the 1st Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 10th International Joint Conference on Natural Language Processing_ , pp. 827–832, 2020.
  * Kim et al. (2020) Kyungyul Kim, ByeongMoon Ji, Doyoung Yoon, and Sangheum Hwang.  Self-knowledge distillation: A simple way for better generalization.  _arXiv preprint arXiv:2006.12000_ , 3, 2020.
  * Kingma & Ba (2014) Diederik P Kingma and Jimmy Ba.  Adam: A method for stochastic optimization.  _arXiv preprint arXiv:1412.6980_ , 2014.
  * Kiyono et al. (2019) Shun Kiyono, Jun Suzuki, Masato Mita, Tomoya Mizumoto, and Kentaro Inui.  An empirical study of incorporating pseudo data into grammatical error correction.  In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)_ , pp. 1236–1242, 2019.
  * Koyama et al. (2021) Aomi Koyama, Kengo Hotate, Masahiro Kaneko, and Mamoru Komachi.  Comparison of grammatical error correction using back-translation models.  In _Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Student Research Workshop_ , pp. 126–135, 2021.
  * Kukich (1992) Karen Kukich.  Techniques for automatically correcting words in text.  _Acm Computing Surveys (CSUR)_ , 24(4):377–439, 1992.
  * Lample et al. (2018) Guillaume Lample, Myle Ott, Alexis Conneau, Ludovic Denoyer, and Marc’Aurelio Ranzato.  Phrase-based & neural unsupervised machine translation.  In _Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing_ , pp. 5039–5049, 2018.
  * Lee et al. (2018) Jason Lee, Elman Mansimov, and Kyunghyun Cho.  Deterministic non-autoregressive neural sequence modeling by iterative refinement.  In _Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing_ , pp. 1173–1182, 2018.
  * Lewis et al. (2020) Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Veselin Stoyanov, and Luke Zettlemoyer.  Bart: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pp. 7871–7880, 2020.
  * Li et al. (2016) Jiwei Li, Michel Galley, Chris Brockett, Jianfeng Gao, and William B Dolan.  A diversity-promoting objective function for neural conversation models.  In _Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_ , pp. 110–119, 2016.
  * Li & Shi (2021) Piji Li and Shuming Shi.  Tail-to-tail non-autoregressive sequence prediction for Chinese grammatical error correction.  In _Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)_ , pp. 4973–4984, 2021.
  * Li et al. (2021) Zongyi Li, Jianhan Xu, Jiehang Zeng, Linyang Li, Xiaoqing Zheng, Qi Zhang, Kai-Wei Chang, and Cho-Jui Hsieh.  Searching for an effective defender: Benchmarking defense against adversarial word substitution.  In _Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing_ , pp. 3137–3147, 2021.
  * Lichtarge et al. (2018) Jared Lichtarge, Christopher Alberti, Shankar Kumar, Noam Shazeer, and Niki Parmar.  Weakly supervised grammatical error correction using iterative decoding.  _arXiv preprint arXiv:1811.01710_ , 2018.
  * Lichtarge et al. (2019) Jared Lichtarge, Chris Alberti, Shankar Kumar, Noam Shazeer, Niki Parmar, and Simon Tong.  Corpora generation for grammatical error correction.  In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)_ , pp. 3291–3301, 2019.
  * Ma (2019) Edward Ma.  Nlp augmentation.  https://github.com/makcedward/nlpaug, 2019.
  * Malmi et al. (2019) Eric Malmi, Sebastian Krause, Sascha Rothe, Daniil Mirylenka, and Aliaksei Severyn.  Encode, tag, realize: High-precision text editing.  In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)_ , pp. 5054–5065, 2019.
  * Miller (1995) George A Miller.  Wordnet: a lexical database for english.  _Communications of the ACM_ , 38(11):39–41, 1995.
  * Mita & Yanaka (2021) Masato Mita and Hitomi Yanaka.  Do grammatical error correction models realize grammatical generalization?  _arXiv preprint arXiv:2106.03031_ , 2021.
  * Mizumoto et al. (2011) Tomoya Mizumoto, Mamoru Komachi, Masaaki Nagata, and Yuji Matsumoto.  Mining revision log of language learning sns for automated japanese error correction of second language learners.  In _Proceedings of 5th International Joint Conference on Natural Language Processing_ , pp. 147–155, 2011.
  * Namazifar et al. (2021) Mahdi Namazifar, John Malik, Li Erran Li, Gokhan Tur, and Dilek Hakkani Tür.  Correcting Automated and Manual Speech Transcription Errors Using Warped Language Models.  In _Proc. Interspeech 2021_ , pp. 2037–2041, 2021.  doi: 10.21437/Interspeech.2021-591.
  * Náplava & Straka (2019) Jakub Náplava and Milan Straka.  Grammatical error correction in low-resource scenarios.  In _Proceedings of the 5th Workshop on Noisy User-generated Text (W-NUT 2019)_ , pp. 346–356, 2019.
  * Náplava et al. (2022) Jakub Náplava, Milan Straka, Jana Straková, and Alexandr Rosen.  Czech grammar error correction with a large and diverse corpus.  _arXiv preprint arXiv:2201.05590_ , 2022.
  * Napoles et al. (2015) Courtney Napoles, Keisuke Sakaguchi, Matt Post, and Joel Tetreault.  Ground truth for grammatical error correction metrics.  In _Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 2: Short Papers)_ , pp. 588–593, 2015.
  * Napoles et al. (2016) Courtney Napoles, Keisuke Sakaguchi, Matt Post, and Joel Tetreault.  Gleu without tuning.  _arXiv preprint arXiv:1605.02592_ , 2016.
  * Napoles et al. (2017) Courtney Napoles, Keisuke Sakaguchi, and Joel Tetreault.  Jfleg: A fluency corpus and benchmark for grammatical error correction.  _EACL 2017_ , pp. 229, 2017.
  * Napoles et al. (2019) Courtney Napoles, Maria Nădejde, and Joel Tetreault.  Enabling robust grammatical error correction in new domains: Data sets, metrics, and analyses.  _Transactions of the Association for Computational Linguistics_ , 7:551–566, 2019.
  * Ng et al. (2014) Hwee Tou Ng, Siew Mei Wu, Ted Briscoe, Christian Hadiwinoto, Raymond Hendy Susanto, and Christopher Bryant.  The conll-2014 shared task on grammatical error correction.  In _Proceedings of the Eighteenth Conference on Computational Natural Language Learning: Shared Task_ , pp. 1–14, 2014.
  * of Hertfordshire (2007) University of Hertfordshire.  Ppdb: Pesticide properties database.  2007\.
  * Omelianchuk et al. (2020) Kostiantyn Omelianchuk, Vitaliy Atrasevych, Artem Chernodub, and Oleksandr Skurzhanskyi.  Gector–grammatical error correction: Tag, not rewrite.  _ACL 2020_ , pp. 163, 2020.
  * Ott et al. (2019) Myle Ott, Sergey Edunov, Alexei Baevski, Angela Fan, Sam Gross, Nathan Ng, David Grangier, and Michael Auli.  fairseq: A fast, extensible toolkit for sequence modeling.  In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics (Demonstrations)_ , pp. 48–53, 2019.
  * Ristad & Yianilos (1998) Eric Sven Ristad and Peter N Yianilos.  Learning string-edit distance.  _IEEE Transactions on Pattern Analysis and Machine Intelligence_ , 20(5):522–532, 1998.
  * Rothe et al. (2021) Sascha Rothe, Jonathan Mallinson, Eric Malmi, Sebastian Krause, and Aliaksei Severyn.  A simple recipe for multilingual grammatical error correction.  In _Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 2: Short Papers)_ , pp. 702–707, 2021.
  * Rozovskaya et al. (2014) Alla Rozovskaya, Dan Roth, and Vivek Srikumar.  Correcting grammatical verb errors.  In _Proceedings of the 14th Conference of the European Chapter of the Association for Computational Linguistics_ , pp. 358–367, 2014.
  * Sennrich et al. (2016) Rico Sennrich, Barry Haddow, and Alexandra Birch.  Improving neural machine translation models with monolingual data.  In _Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pp. 86–96, 2016.
  * Stahlberg & Kumar (2020) Felix Stahlberg and Shankar Kumar.  Seq2edits: Sequence transduction using span-level edit operations.  _arXiv preprint arXiv:2009.11136_ , 2020.
  * Sun et al. (2021) Xin Sun, Tao Ge, Furu Wei, and Houfeng Wang.  Instantaneous grammatical error correction with shallow aggressive decoding.  _arXiv preprint arXiv:2106.04970_ , 2021.
  * Sutskever et al. (2014) Ilya Sutskever, Oriol Vinyals, and Quoc V Le.  Sequence to sequence learning with neural networks.  In _Advances in neural information processing systems_ , pp. 3104–3112, 2014.
  * Szegedy et al. (2013) Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus.  Intriguing properties of neural networks.  _arXiv preprint arXiv:1312.6199_ , 2013.
  * Szegedy et al. (2016) Christian Szegedy, Vincent Vanhoucke, Sergey Ioffe, Jon Shlens, and Zbigniew Wojna.  Rethinking the inception architecture for computer vision.  In _Proceedings of the IEEE conference on computer vision and pattern recognition_ , pp. 2818–2826, 2016.
  * Tajiri et al. (2012) Toshikazu Tajiri, Mamoru Komachi, and Yuji Matsumoto.  Tense and aspect error correction for esl learners using global context.  In _Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)_ , pp. 198–202, 2012.
  * Tang et al. (2021) Zecheng Tang, Yixin Ji, Yibo Zhao, and Junhui Li.  Chinese grammatical error correction enhanced by data augmentation from word and character levels.  In _Proceedings of the 20th Chinese National Conference on Computational Linguistics, Hohhot, China_ , pp. 13–15, 2021.
  * Tillmann & Ney (2003) Christoph Tillmann and Hermann Ney.  Word reordering and a dynamic programming beam search algorithm for statistical machine translation.  _Computational linguistics_ , 29(1):97–133, 2003.
  * Vaswani et al. (2017) Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin.  Attention is all you need.  In _Proceedings of the 31st International Conference on Neural Information Processing Systems_ , pp. 6000–6010, 2017.
  * Wallace et al. (2019) Eric Wallace, Shi Feng, Nikhil Kandpal, Matt Gardner, and Sameer Singh.  Universal adversarial triggers for attacking and analyzing nlp.  _arXiv preprint arXiv:1908.07125_ , 2019.
  * Wan & Wan (2021) Zhaohong Wan and Xiaojun Wan.  A syntax-guided grammatical error correction model with dependency tree correction.  _arXiv preprint arXiv:2111.03294_ , 2021.
  * Wan et al. (2020) Zhaohong Wan, Xiaojun Wan, and Wenguang Wang.  Improving grammatical error correction with data augmentation by editing latent representation.  In _Proceedings of the 28th International Conference on Computational Linguistics_ , pp. 2202–2212, 2020.
  * Wang & Zheng (2020) Lihao Wang and Xiaoqing Zheng.  Improving grammatical error correction models with purpose-built adversarial examples.  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , pp. 2858–2869, 2020.
  * Wang et al. (2021) Xiaoqiang Wang, Yanqing Liu, Sheng Zhao, and Jinyu Li.  A Light-Weight Contextual Spelling Correction Model for Customizing Transducer-Based Speech Recognition Systems.  In _Proc. Interspeech 2021_ , pp. 1982–1986, 2021.  doi: 10.21437/Interspeech.2021-379.
  * Wang et al. (2020) Yu Wang, Yuelin Wang, Jie Liu, and Zhuo Liu.  A comprehensive survey of grammar error correction.  _arXiv preprint arXiv:2005.06600_ , 2020.