# Improved Text Classification via Contrastive Adversarial Training

Lin Pan†, Chung-Wei Hang†, Avirup Sil§, Saloni Potdar†, Mo Yu¶
IBM Watson†
IBM Research AI§
MIT-IBM Watson AI Lab¶
{panl, hangc, avi, potdars<EMAIL_ADDRESS>
###### Abstract

We propose a simple and general method to regularize the fine-tuning of
Transformer-based encoders for text classification tasks. Specifically, during
fine-tuning we generate adversarial examples by perturbing the word embeddings
of the model and perform contrastive learning on clean and adversarial
examples in order to teach the model to learn noise-invariant representations.
By training on both clean and adversarial examples along with the additional
contrastive objective, we observe consistent improvement over standard fine-
tuning on clean examples. On several GLUE benchmark tasks, our fine-tuned
BERT${}_{\textrm{Large}}$ model outperforms BERT${}_{\textrm{Large}}$ baseline
by $1.7\%$ on average, and our fine-tuned RoBERTa${}_{\textrm{Large}}$
improves over RoBERTa${}_{\textrm{Large}}$ baseline by $1.3\%$. We
additionally validate our method in different domains using three intent
classification datasets, where our fine-tuned RoBERTa${}_{\textrm{Large}}$
outperforms RoBERTa${}_{\textrm{Large}}$ baseline by $1$–$2\%$ on average.

##  1 Introduction

Adversarial training (AT) introduced in [14] provides an effective means of
regularization and improving model robustness against adversarial examples
[37] for computer vision (CV) tasks such as image classification. In AT of
this form, a small, gradient-based perturbation is added to the original
example, and a model is trained on both clean and perturbed examples. Due to
the discrete nature of textual data, this method is not directly applicable to
NLP tasks. [28] extends this method to NLP and propose to apply perturbation
to the word embeddings of an LSTM-based model [19] on text classification
tasks. Since the word embeddings after perturbation do not map to new words in
the vocabulary, the method is proposed exclusively as a means of
regularization.

In this work, we present CAT, contrastive adversarial training for text
classification. We build upon [28] to regularize the fine-tuning of
Transformer-based [39] encoders on text classification tasks. Additionally, we
encourage the model to learn noise-invariant representations by introducing a
contrastive objective [38] that pushes clean examples and their corresponding
perturbed examples close to each other in the representation space, while
pushing apart examples not from the same pair. We evaluate our method on a
range of natural language understanding tasks including the standard GLUE [40]
benchmark as well as three intent classification tasks for dialog systems. On
GLUE tasks, we compare our fine-tuning method against strong baselines of
fine-tuning BERT${}_{\textrm{Large}}$ [10] and RoBERTa${}_{\textrm{Large}}$
[25] on clean examples with the cross-entropy loss. Our method outperforms
BERT${}_{\textrm{Large}}$ by $1.7\%$ on average and
RoBERTa${}_{\textrm{Large}}$ by $1.3\%$. On intent classification tasks, our
fine-tuned RoBERTa${}_{\textrm{Large}}$ outperforms
RoBERTa${}_{\textrm{Large}}$ baseline by $1\%$ on the full test sets and $2\%$
on the difficult test sets. We further perform sample efficiency tests, where
we use only half of the training data (per intent) and achieve near identical
accuracy compared to the baseline trained using full training data.

##  2 Related Work

###  2.1 Adversarial Training

Adversarial Training (AT) has been explored in many supervised classification
tasks that include object detection [5, 34, 45], object segmentation [1, 45]
and image classification [14, 31, 35]. AT can be defined as the process in
which a system is trained to defend against malicious “attacks” and increase
network robustness, by training the system with adversarial examples and
optionally with clean examples. Typically these attacks are produced by
perturbing the input (clean) examples, which makes the system predict the
wrong class label [4, 46]. [28] extends the _Fast Gradient Sign Method_ (FGSM)
proposed in [14] to NLP tasks by perturbing word embeddings, and applies the
method to both supervised and semi-supervised settings with Virtual
Adversarial Training (VAT) [29] for the latter. [43] applies AT to relation
extraction. Recent works [21, 22, 47] propose to apply perturbations to the
attention mechanism in Transformer-based encoders. Compared to single-step
FGSM, [26] demonstrates the superior effectiveness of the multi-step approach
to generate perturbed examples with projected gradient descent, which comes at
a greater computational cost due to the inner loop that iteratively calculates
the perturbations. [33] proposes “free” adversarial training. In the inner
loop where perturbations are calculated, gradients with respect to model
parameters are also calculated and updated. The number of training epochs are
also reduced to achieve comparable complexity with natural training. [47]
adopts the “free” AT algorithm and further add gradient accumulation to
achieve a larger effective batch. Similar to [28], perturbations are applied
to word embeddings of LSTM and BERT-based models. In our work, we use the
simpler one-step FGSM to generate perturbed examples and perform contrastive
learning with clean examples.

###  2.2 Contrastive Learning

Recent advances in self-supervised contrastive learning, such as MoCo [17] and
SimCLR [6] have bridged the gap in performance between self-supervised
learning and fully-supervised methods on the ImageNet [9] dataset. Several
works have successfully applied this representation learning paradigm to
various NLP tasks. A key component in contrastive learning is how to create
positive pairs. [12] uses back-translation to generate another view of the
original English data. [44] applies word and span deletion, reordering, and
substitution. [27] uses sequence cropping and masked sequence from an
auxiliary Transformer. [15] treats training examples of the same class as
positive pairs and performs supervised contrastive learning [20]. [13] uses
different dropout masks on the same data to generate positive pairs. As a
supervised alternative, they leverage NLI datasets [2, 42] and treat premises
and their corresponding hypotheses as positive pairs and contradictions as
hard negatives. In our work, we treat an original example, and its adversarial
example as a positive pair, and a contrastive loss is used as an additional
regularizer during fine-tuning. For multilingual NLP, [7, 30, 41] leverage
parallel data and perform contrastive learning on parallel sentences for
cross-lingual representation learning.

###  2.3 Improving BERT for Text Classification

As a general method to improve Transformer-based model performance on
downstream tasks, [36] and [16] propose further language model pretraining in
the target domain before the final fine-tuning. [11] proposes to use self-
training as another way to leverage unlabeled data, where a teacher model is
first trained on labeled data, and is then used to label large amount of in-
domain unlabeled data for the student model to learn from. Recent developments
in language model pretraining have also advanced state-of-the-art results on a
wide range of NLP tasks. ELECTRA [8] uses a generator to generate noisy text
via the mask language modeling objective and a discriminator is used to
classify each input token as original or replaced. The model shows strong
performance on downstream tasks by fine-tuning the discriminator. DeBERTa [18]
proposes a disentangled attention mechanism and a new form of VAT, where
perturbations are applied to the normalized word embeddings.

##  3 Method

In this section we first briefly describe the standard fine-tuning procedure
for Transformer-based encoders on text classification tasks. We then introduce
our method of generating adversarial examples and propose our method CAT that
uses these examples to perform contrastive learning with clean examples.
Figure 1 shows our overall model architecture.

Figure 1: Model architecture for our proposed method to fine-tune Transformer-
based encoders on text classification tasks. We use the Fast Gradient Sign
Method to generate adversarial examples by perturbing the word embedding
matrix $V$ of the encoder. We then train on both clean and perturbed examples
with the cross-entropy loss. Additionally, we introduce a third, contrastive
loss that brings the representations of clean examples and their corresponding
perturbed examples close to each other in order for the model to learn noise-
invariant representations.

###  3.1 Preliminaries

Our learning setup is based on a standard multi-class classification problem
with input training examples $\\{x_{i},y_{i}\\}_{i=1,\ldots,N}$. We assume
access to a Transformer-based pre-trained language model (PLM), such as BERT,
and RoBERTa. Given a token sequence $x_{i}=[CLS,t_{1},t_{2},\ldots,t_{T},SEP]$
111In the case of sequence pairs, another [SEP] token is added in between the
sequences., the PLM outputs a sequence of contextualized token representations
$H^{L}=[h_{[CLS]}^{L},h_{1}^{L},h_{2}^{L},\ldots,h_{T}^{L},h_{[SEP]}^{L}]$.

$h_{[CLS]}^{L},h_{1}^{L},\ldots,h_{T}^{L},h_{[SEP]}^{L}=PLM([CLS],t_{1},\ldots,t_{T},[SEP]),$

where $L$ denotes the number of model layers 222We drop the layer superscript
from here on for notation convenience..

The standard practice for fine-tuning these large PLMs is to add a softmax
classifier on top of the model’s sentence-level representations, such as the
final hidden state $h_{[CLS]}$ of the $[CLS]$ token in BERT:

$p(y_{c}|h_{[CLS]})=\mathit{softmax}(Wh_{[CLS]})\;\;\;c\in C,$ (1)

where $W\in\mathbb{R}^{d_{C}\times d_{h}}$, and C denotes the number of
classes. A model is trained by minimizing the cross entropy loss:

$\mathcal{L}_{CE}=-\frac{1}{N}\sum_{i=1}^{N}\sum_{c=1}^{C}y_{i,c}\log(p(y_{i,c}|h_{[CLS]}^{i})),$
(2)

where $N$ is batch size.

###  3.2 Adversarial Examples

Adversarial examples are imperceptibly perturbed input to a model that causes
misclassification. [14] proposes the _Fast Gradient Sign Method_ (FGSM) to
generate such examples, and training on both clean and adversarial examples as
an efficient way to improve model robustness against adversaries. Formally,
given a loss function $\mathcal{L}(f_{\theta}(x_{i}+r),y_{i})$, where
$f_{\theta}$ is a neural network parameterized by $\theta$, and $x_{i}$ the
input example, we maximize the loss function subject to the max norm
constraint on the perturbation $r$.

$\max_{r}\mathcal{L}(f_{\theta}(x_{i}+r),y_{i}),s.t.\left\|r\right\|_{\infty}<\epsilon,\;\;\;\text{where
$\epsilon>0$}$ (3)

Using first-order approximation, the loss function is approximately equivalent
to the following:

$\mathcal{L}(f_{\theta}(x_{i}+r),y_{i})\approx\mathcal{L}(f_{\theta}(x_{i}),y_{i})+\nabla_{x_{i}}\mathcal{L}(f_{\theta}(x_{i}),y_{i})^{T}r$
(4)

Solving for (3) and (4) yields perturbation in the following form:

$r=-\epsilon\>\mathit{sign}(\nabla_{x_{i}}\mathcal{L}(f_{\theta}(x_{i}),y_{i}))$
(5)

Alternatively, using $l_{2}$-norm constraint on the perturbation $r$ in (3)
yields:

$r=-\epsilon\frac{\nabla_{x_{i}}\mathcal{L}(f_{\theta}(x_{i}),y_{i})}{\left\|\nabla_{x_{i}}\mathcal{L}(f_{\theta}(x_{i}),y_{i})\right\|_{2}}$
(6)

AT in [14] uses both the clean example $x_{i}$ and perturbed example $x_{i}+r$
to train a model. For NLP problems where input is usually discrete, FGSM is
not directly applicable. [28] proposes to apply perturbation to the word
embedding $v_{i}$ from the corresponding row in embedding matrix
$V\in\mathbb{R}^{d_{v}\times d_{h}}$, where $d_{v}$ is vocabulary size and
$d_{h}$ hidden size. We follow this approach, but instead of perturbing the
word embeddings, we directly perturb the word embedding matrix of Transformer-
based encoders to generate our adversarial examples. Specifically, after each
forward pass with clean examples, we calculate the gradient of the loss
function in (2) with respect to the word embedding matrix $V$ instead of word
embedding in (5) to calculate the perturbation. Empirically, we find that
perturbing the word embedding matrix performs better than word embeddings (see
a comparison on GLUE in Table 9, Appendix A). For task classification tasks,
we train on clean and adversarial examples with the cross entropy loss in (2).
Additionally, we experiment with different forms of perturbation in (5) and
(6), as well as randomly sampling between the two for each batch of data. We
observe that using $r$ with the max norm constraint consistently leads to the
best result. In Section 4, we report results from using this form of
perturbation.

###  3.3 Contrastive Learning

Intuitively, given a pair of clean and adversarial examples, we want their
encoded sentence-level representation to be as similar to each other as
possible so that our trained model will be more noise-invariant. At the same
time, examples not from the same pair should be farther away in the
representation space. To model this relationship, we leverage contrastive
learning as an additional regularizer during the fine-tuning process. Recent
works on contrastive learning, such as MoCo [17], and SimCLR [6] use various
forms of data augmentation, e.g., random cropping, and random color
distortion, as the first step to create positive pairs. MoCo uses a queue
structure to store negative examples, while SimCLR performs in-batch negative
example sampling. In our work, we employ the SimCLR formulation of positive
and negative pairs, and its loss function to implement our contrastive
objective.

Concretely, given the final hidden state $h_{[CLS]}^{i}$ of the $[CLS]$ token
for a clean example, and $h_{[CLS]}^{j}$ for its corresponding adversarial
example, we treat ($h_{[CLS]}^{i}$, $h_{[CLS]}^{j}$) as a pair of positive
examples. Following [6], we add a non-linear projection layer on top of them:

$\displaystyle z_{i}$ $\displaystyle=W_{2}\mathit{ReLU}(W_{1}h_{[CLS]}^{i}),$
(7) $\displaystyle z_{j}$
$\displaystyle=W_{2}\mathit{ReLU}(W_{1}h_{[CLS]}^{j})$ (8)

where $W_{1}\in\mathbb{R}^{d_{h}\times d_{h}}$,
$W_{2}\in\mathbb{R}^{d_{k}\times d_{h}}$, and $d_{k}$ is set to $300$. With a
batch of $N$ clean examples and their corresponding adversarial examples, for
each positive pair, there are $2(N-1)$ negative pairs, i.e., all the rest of
the examples in the batch are negative examples. The contrastive objective is
to identify the positive pair:

$\mathcal{L_{\text{ctr}}}=-\log\frac{\exp(sim(z_{i},z_{j}/\tau))}{\sum_{k=1}^{2N}\mathbb{1}_{[k\neq
i]}\exp(sim(z_{i},z_{k}/\tau))},$ (9)

where $\mathit{sim(u,v)}$ denotes cosine similarity between two vectors, and
$\tau$ a temperature hyperparameter.

Finally, we perform fine-tuning in a multi-task manner and take a weighted
average of the two classification losses and the contrastive loss 333In our
experiments, we always assign equal weights to the two classification losses.
It is possible that a different weight distribution yields better results.:

$\mathcal{L}=\frac{(1-\lambda)}{2}(\mathcal{L}_{CE}^{V}+\mathcal{L}_{CE}^{V+r})+\lambda\mathcal{L}_{ctr}$
(10)

##  4 Experiment

###  4.1 Datasets

We conduct experiments on seven tasks of the GLUE benchmark, including textual
entailment (MNLI, RTE), question answering/entailment (QNLI), question
paraphrase (QQP), paraphrase (MRPC), grammatical correctness (CoLA), and
sentiment analysis (SST-2). Table 1 summarizes the statistics of the GLUE
tasks.

Dataset | Task | Labels | Training | Metric
---|---|---|---|---
MNLI | Textual entailment | 3 | 393k | Accuracy
QQP | Question paraphrase | 2 | 364k | Accuracy
QNLI | Question answering & textual entailment | 2 | 105k | Accuracy
MRPC | Paraphrase | 2 | 3.7k | F1
RTE | Textual entailment | 2 | 2.5k | Accuracy
CoLA | Grammatical correctness | 2 | 8.5k | MCC
SST-2 | Sentiment analysis | 2 | 67k | Accuracy
Table 1: GLUE sequence classification datasets statistics

We additionally experiment on three commonly used intent classification
datasets—clinc150 [23], banking77 [3] and hwu64 [24]. Intent classification is
the process of identifying the class (intent) of any utterance in a task-
oriented dialog system. These three datasets largely represent a short-text
classification task in real-world settings. Table 2 summarizes the statistics
of the three datasets.

#### clinc150

covers $150$ intents in $10$ domains (e.g., banking, work, auto, travel). The
dataset is designed to capture the breadth of topics that a production task-
oriented chatbot handles. The dataset also comes with $1,200$ out-of-scope
examples. In this work, we focus on in-scope examples only.

#### banking77

is a single domain dataset created for fine-grained intent classification. The
dataset consists of customer service queries in the banking domain, covering
$77$ intents across $10,003$ training examples and $3080$ test examples.

#### hwu64

covers $64$ intents in $21$ domains (e.g., alarm, email, game, news). The
dataset is created in the context of a real-world use case of a home assistant
bot. We use the one fold train-test split with $9,957$ training examples and
$1,076$ test examples for our experiments.

For the three intent classification datasets, in addition to the original
evaluation data, we also evaluate on a difficult subset of each test set
described in [32]. The difficult subsets are constructed by comparing the TF-
IDF vector of each test example to those of the training examples for a given
intent. The test examples that are most dissimilar to the corresponding
training examples are selected for inclusion to the difficult subset. The
evaluation metric for all intent classification datasets is accuracy.

|  |  | Examples
---|---|---|---
Dataset | Intents | Domains | Training | Test | Test (difficult)
clinc150 | 150 | 10 | 17,999 | 4,500 | 750
banking77 | 77 | 1 | 10,003 | 3,080 | 770
hwu64 | 64 | 21 | 9,957 | 1,076 | 620
Table 2: Intent classification dataset statistics

###  4.2 Training Details

We apply CAT to the fine-tuning of two backbone PLMs,
BERT${}_{\textrm{Large}}$ and RoBERTa${}_{\textrm{Large}}$. For all
experiments, we use AdamW optimizer with $0.01$ weight decay and a linear
learning rate scheduler. We set max sequence length to $128$ and learning rate
warmup for the first $10\%$ of the total iterations. On GLUE tasks, we largely
follow the hyperparameter settings reported in [10] and [25] to generate our
BERT${}_{\textrm{Large}}$ and RoBERTa${}_{\textrm{Large}}$ baselines. For
BERT${}_{\textrm{Large}}$, we set batch size to $32$ and fine-tune for $3$
epochs. Grid search is performed over $lr\in\\{0.00001,0.00002,0.00003\\}$.
For RoBERTa${}_{\textrm{Large}}$, we sweep over the same learning rates as
BERT${}_{\textrm{Large}}$ and batch size $\in\\{16,32\\}$. On intent
classification datasets, we use a batch size of $32$ and fine-tune for 5
epochs [32], and search over $lr\in\\{0.00003,0.00004,0.00005\\}$. For fine-
tuning with CAT, we use the exact same hyperparameter settings as the
baseline, and further perform grid search over
$\epsilon\in\\{0.0001,0.001,0.005,0.02\\}$,
$\tau\in\\{0.05,0.06,0.07,0.08,0.09,0.1\\}$, and
$\lambda\in\\{0.1,0.2,0.3,0.4,0.5\\}$. On the three intent classification
datasets, we follow the hyperparameter settings reported in [32]. We use a
batch size of $32$ and fine-tune RoBERTa${}_{\textrm{Large}}$ for $5$ epochs
with learning rate set to $0.00004$. All our experiments were run on a single
32 GB V100 GPU.

###  4.3 GLUE Results

On GLUE tasks, we fine-tune BERT${}_{\textrm{Large}}$ and
RoBERTa${}_{\textrm{Large}}$ using our method with two classification losses
(on clean examples, and adversarial examples, respectively) and the
contrastive loss. We compare them with the BERT${}_{\textrm{Large}}$ and
RoBERTa${}_{\textrm{Large}}$ baseline, which are conventionally fine-tuned
with classification loss on clean examples. For all experiments, we fine-tune
BERT${}_{\textrm{Large}}$ and RoBERTa${}_{\textrm{Large}}$ from their original
checkpoints with no task-wise transfer learning involved.

We accompany each set of experiments with statistical significance test. For
tasks evaluated with accuracy, we use McNemar’s test. For CoLA, which is
evaluated with Matthews correlation coefficient (MCC), and MRPC with F1, we
use Fisher Randomization test.

Table 3 shows the dev set results. In summary, CAT fine-tuning approach
consistently outperforms the standard fine-tuning approach for both
BERT${}_{\textrm{Large}}$ and RoBERTa${}_{\textrm{Large}}$. CAT leads to a
$1.7\%$ improvement on average over conventionally fine-tuned
BERT${}_{\textrm{Large}}$. The largest improvement is observed on the CoLA
task (i.e., $4.0\%$). For the stronger baseline of
RoBERTa${}_{\textrm{Large}}$, we observe an improvement of $1.3\%$ on average
with our method. On MRPC and CoLA, our result improves over
RoBERTa${}_{\textrm{Large}}$ baseline by $2.4\%$, showing the effectiveness
our method on both single sequence, as well as sequence pair classification
tasks. For statistical significance, we note that our improved results over
the baseline are not significant on some smaller datasets, e.g., improved
BERT${}_{\textrm{Large}}$ on MRPC and RTE, and RoBERTa${}_{\textrm{Large}}$ on
MRPC, RTE and CoLA.

Model | MNLI | QQP | QNLI | MRPC | RTE | CoLA | SST-2 | Avg
---|---|---|---|---|---|---|---|---
BERT${}_{\textrm{Large}}$ (our impl) | 86.6 | 91.4 | 92.0 | 90.0 | 69.3 | 61.8 | 93.6 | 83.5
BERT${}_{\textrm{Large}}$ \+ AT + CTR |  87.4$\dagger$ |  92.2$\dagger$ |  93.0$\dagger$ | 91.6 | 71.5 |  65.8$\dagger$ |  95.2$\dagger$ | 85.2
RoBERTa${}_{\textrm{Large}}$ (our impl) | 90.5 | 91.8 | 94.5 | 90.6 | 85.9 | 67.0 | 96.1 | 88.1
RoBERTa${}_{\textrm{Large}}$ \+ AT + CTR | 91.1$\dagger$ | 92.5$\dagger$ | 95.1$\dagger$ | 93.0$\dagger$ | 87.4 | 69.4 | 97.0 | 89.4

Table 3: Results on the dev sets of GLUE benchmark. AT refers to the
adversarial training component of our system and CTR the contrastive learning
component. On average, our fine-tuned BERT${}_{\textrm{Large}}$ model
outperforms BERT${}_{\textrm{Large}}$ baseline by $1.7\%$, and our fine-tuned
RoBERTa${}_{\textrm{Large}}$ improves over RoBERTa${}_{\textrm{Large}}$
baseline by $1.3\%$. We fine-tune BERT${}_{\textrm{Large}}$ and
RoBERTa${}_{\textrm{Large}}$ from their original checkpoints with no task-wise
transfer learning involved. $\dagger$ indicates statistically significant
improvement over the baseline. For CoLA and MRPC, we use Fisher Randomization
test, and McNemar’s test for all the other tasks.

###  4.4 Intent Classification Results

On the three intent classification datasets, clinc150, banking77, and hwu64,
we experiment with the stronger RoBERTa${}_{\textrm{Large}}$ baseline. Table 4
summarizes the results. On average, CAT outperforms the fine-tuned
RoBERTa${}_{\textrm{Large}}$ baseline by $1\%$, when evaluated on the full
test sets of the three datasets. The largest improvement is made on the hwu64
dataset ($1.4\%$).

Furthermore, we show that by training with adversarial examples and
contrastive learning, CAT makes RoBERTa${}_{\textrm{Large}}$ work better on
the difficult test subsets of the three intent classification tasks. As shown
in Table 5, our approach improves over standard RoBERTa${}_{\textrm{Large}}$
fine-tuning by $2\%$ on average. On banking77, our method results in a large
improvement of $3.5\%$. For statistical significance test on intent
classification datasets, we use McNemar’s test and observe significant
improvement over baseline results for all datasets and evaluation settings.

Model | clinc150 | banking77 | hwu64 | Average
---|---|---|---|---
RoBERTa${}_{\textrm{Large}}$ | 97.4 | 93.9 | 92.4 | 94.6
RoBERTa${}_{\textrm{Large}}$ \+ AT + CTR | 98.0$\dagger$ | 95.0$\dagger$ | 93.8$\dagger$ | 95.6
Table 4: Results on full test sets of intent classification datasets. AT refers to the adversarial training component of our system and CTR the contrastive learning component. On average, our fine-tuned RoBERTa${}_{\textrm{Large}}$ improves over RoBERTa${}_{\textrm{Large}}$ baseline by $1\%$. $\dagger$ indicates statistically significant improvement over the baseline using McNemar’s test. Model | clinc150 | banking77 | hwu64 | Average
---|---|---|---|---
RoBERTa${}_{\textrm{Large}}$ | 91.1 | 83.8 | 89.5 | 88.1
RoBERTa${}_{\textrm{Large}}$ \+ AT + CTR | 92.1$\dagger$ | 87.3$\dagger$ | 90.8$\dagger$ | 90.1
Table 5: Results on difficult test sets of intent classification datasets. On
average, our fine-tuned RoBERTa${}_{\textrm{Large}}$ improves over
RoBERTa${}_{\textrm{Large}}$ baseline by $2\%$. $\dagger$ indicates
statistically significant improvement over the baseline using McNemar’s test.

###  4.5 Sample Efficiency

Next, we demonstrate that CAT has better sample efficiency compared to
standard fine-tuning. We design the experiment on the three intent
classification datasets. Specifically, we use about half of the training data
(per intent) from each dataset to fine-tune RoBERTa${}_{\textrm{Large}}$ with
CAT. As Table 6 shows, with the usage of only half of the training data, our
method achieves nearly the same results, compared to standard fine-tuning
using all the training data. This result confirms the sample efficiency of our
proposed CAT, and also indicates the advantage of our approach in the
challenging low-resource scenarios.

Model | Training data | clinc150 | banking77 | hwu64 | Average
---|---|---|---|---|---
RoBERTa${}_{\textrm{Large}}$ | full | 97.4 | 93.9 | 92.4 | 94.6
RoBERTa${}_{\textrm{Large}}$ \+ AT + CTR | half | 97.1 | 94.2 | 92.3 | 94.5
Table 6: Sample efficiency test results on the full test sets of intent
classification datasets. With our proposed fine-tuning method, we can use
about half of the training data (per intent) and achieve near identical
accuracy compared to baseline using full training data.

Model | MNLI | QQP | QNLI | MRPC | RTE | CoLA | SST-2 | Average
---|---|---|---|---|---|---|---|---
RoBERTa${}_{\textrm{Large}}$ | 90.5 | 91.8 | 94.5 | 90.6 | 85.9 | 67.0 | 96.1 | 88.1
RoBERTa${}_{\textrm{Large}}$ \+ AT | 90.7 | 92.1 | 94.9 | 92.4 | 85.6 | 69.9 | 96.6 | 88.9
RoBERTa${}_{\textrm{Large}}$ \+ AT + CTR | 91.1 | 92.5 | 95.1 | 93.0 | 87.4 | 69.4 | 97.0 | 89.4

Table 7: Ablation results on the dev sets of GLUE benchmark. AT refers to the
adversarial training component of our system and CTR the contrastive learning
component. Adding AT leads to an improvement of $0.8\%$ over the baseline.
Further adding the contrastive objective contributes to an additional $0.5\%$
improvement.

###  4.6 Ablation

Finally, we perform ablation experiments on both GLUE benchmark and the three
intent classification datasets. Results are presented in Table 7 and Table 8.
On GLUE, by removing the contrastive loss, i.e., fine-tuning with clean and
adversarial examples, we observe an accuracy drop of $0.5\%$ on average,
compared to the full system. Compared to the baseline, this setup makes an
average improvement of $0.8\%$. Our full system performs the best on all tasks
except for CoLA, on which removing the contrastive loss yields the best
performance.

On intent classification datasets, we use the difficult test sets for
evaluation. Here, we observe a much larger effect from the additional
contrastive loss, improving over (RoBERTa${}_{\textrm{Large}}$ \+ AT) by
$1.3\%$ on average, while AT alone improves over the baseline by $0.7\%$. For
all ablation experiments with the (RoBERTa${}_{\textrm{Large}}$ \+ AT) setup,
we perform a grid search over $\epsilon\in\\{0.0001,0.001,0.005,0.02\\}$.

Model | clinc150 | banking77 | hwu64 | Average
---|---|---|---|---
RoBERTa${}_{\textrm{Large}}$ | 91.1 | 83.8 | 89.5 | 88.1
RoBERTa${}_{\textrm{Large}}$ \+ AT | 90.8 | 85.8 | 89.8 | 88.8
RoBERTa${}_{\textrm{Large}}$ \+ AT + CTR | 92.1 | 87.3 | 90.8 | 90.1
Table 8: Ablation results on difficult test sets of intent classification
datasets. AT refers to the adversarial training component of our system and
CTR the contrastive learning component. Adding AT improves over the baseline
by $0.7\%$. Further adding the contrastive objective contributes to an
additional $1.3\%$ improvement.

##  5 Conclusion

In this paper, we describe CAT, a simple and effective method for regularizing
the fine-tuning of Transformer-based encoders. By leveraging adversarial
training and contrastive learning, our system consistently outperforms the
standard fine-tuning method for text classification. We use strong baseline
models and evaluate our method on a range of GLUE benchmark tasks and three
intent classification datasets in different settings. Sample efficiency and
ablation tests show the positive effects of combining our adversarial and
contrastive objectives for improved text classification. In the future, we
plan to study additional word-level objectives to complement the sentence-
level contrastive learning objective, in order to extend our method to other
NLP tasks.

## References

  * [1] Anurag Arnab, Ondrej Miksik, and Philip HS Torr.  On the robustness of semantic segmentation models to adversarial attacks.  In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 888–897, 2018.
  * [2] Samuel R. Bowman, Gabor Angeli, Christopher Potts, and Christopher D. Manning.  A large annotated corpus for learning natural language inference.  In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 632–642, Lisbon, September 2015. Association for Computational Linguistics.
  * [3] Iñigo Casanueva, Tadas Temčinas, Daniela Gerz, Matthew Henderson, and Ivan Vulić.  Efficient intent detection with dual sentence encoders.  In Proceedings of the 2nd Workshop on Natural Language Processing for Conversational AI, pages 38–45, Online, July 2020.
  * [4] Anirban Chakraborty, Manaar Alam, Vishal Dey, Anupam Chattopadhyay, and Debdeep Mukhopadhyay.  Adversarial attacks and defences: A survey.  arXiv preprint arXiv:1810.00069, 2018.
  * [5] Shang-Tse Chen, Cory Cornelius, Jason Martin, and Duen Horng Chau.  ShapeShifter: Robust physical adversarial attack on faster r-cnn object detector. corr abs/1804.05810 (2018).  arXiv preprint arXiv:1804.05810, 2018.
  * [6] Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton.  A simple framework for contrastive learning of visual representations.  In Proceedings of the International Conference on Machine Learning (ICML), pages 1597–1606, Virtual, July 2020.
  * [7] Zewen Chi, Li Dong, Furu Wei, Nan Yang, Saksham Singhal, Wenhui Wang, Xia Song, Xian-Ling Mao, Heyan Huang, and Ming Zhou.  InfoXLM: An information-theoretic framework for cross-lingual language model pre-training.  arXiv preprint arXiv:2007.07834, pages 1–11, 2020.
  * [8] Kevin Clark, Minh-Thang Luong, Quoc V. Le, and Christopher D. Manning.  ELECTRA: Pre-training text encoders as discriminators rather than generators.  In Proceedings of the 8th International Conference on Learning Representation (ICLR), Addis Ababa, Ethiopia, 2020.
  * [9] Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei.  ImageNet: A large-scale hierarchical image database.  In Processings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 248–255, Miami, June 2009.
  * [10] Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova.  BERT: Pre-training of deep bidirectional transformers for language understanding.  In Proceedings of the 20th Annual Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT), pages 4171–4186, Minneapolis, June 2019. The Association for Computational Linguistics.
  * [11] Jingfei Du, Edouard Grave, Beliz Gunel, Vishrav Chaudhary, Onur Celebi, Michael Auli, Ves Stoyanov, and Alexis Conneau.  Self-training improves pre-training for natural language understanding.  arXiv preprint arXiv:2010.02194, pages 1–8, 2021.
  * [12] Hongchao Fang, Sicheng Wang, Meng Zhou, Jiayuan Ding, and Pengtao Xie.  Cert: Contrastive self-supervised learning for language understanding.  arXiv preprint arXiv:2005.12766, pages 1–16, 2020.
  * [13] Tianyu Gao, Xingcheng Yao, and Danqi Chen.  SimCSE: Simple contrastive learning of sentence embeddings.  arXiv preprint arXiv:2104.08821, pages 1–16, 2021.
  * [14] Ian J. Goodfellow, Jonathon Shlens, and Christian Szegedy.  Explaining and harnessing adversarial examples.  In Proceedings of the 3rd International Conference on Learning Representation (ICLR), San Diego, 2015.
  * [15] Beliz Gunel, Jingfei Du, Alexis Conneau, and Veselin Stoyanov.  Supervised contrastive learning for pre-trained language model fine-tuning.  In Proceedings of the 9th International Conference on Learning Representation (ICLR), Virtual, 2021.
  * [16] Suchin Gururangan, Ana Marasović, Swabha Swayamdipta, Kyle Lo, Iz Beltagy, Doug Downey, and Noah A. Smith.  Don’t stop pretraining: Adapt language models to domains and tasks.  In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics (ACL), pages 8342–8360, Online, July 2020.
  * [17] Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross B. Girshick.  Momentum contrast for unsupervised visual representation learning.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 9726–9735, Seattle, June 2020.
  * [18] Pengcheng He, Xiaodong Liu, Jianfeng Gao, and Weizhu Chen.  DeBERTa: Decoding-enhanced BERT with disentangled attention.  In Proceedings of the 9th International Conference on Learning Representation (ICLR), Virtual, 2021.
  * [19] Sepp Hochreiter and Jürgen Schmidhuber.  Long short-term memory.  Neural computation, 9(8):1735–1780, November 1997.
  * [20] Prannay Khosla, Piotr Teterwak, Chen Wang, Aaron Sarna, Yonglong Tian, Phillip Isola, Aaron Maschinot, Ce Liu, and Dilip Krishnan.  Supervised contrastive learning.  In Advances in Neural Information Processing Systems, pages 18661–18673, Vancouver, 2020.
  * [21] Shunsuke Kitada and Hitoshi Iyatomi.  Attention meets perturbations: Robust and interpretable attention with adversarial training.  arXiv preprint arXiv:2009.12064, pages 1–1, 2020.
  * [22] Shunsuke Kitada and Hitoshi Iyatomi.  Making attention mechanisms more robust and interpretable with virtual adversarial training for semi-supervised text classification.  arXiv preprint arXiv:2104.08763, pages 1–12, 2021.
  * [23] Stefan Larson, Anish Mahendran, Joseph J. Peper, Christopher Clarke, Andrew Lee, Parker Hill, Jonathan K. Kummerfeld, Kevin Leach, Michael A. Laurenzano, Lingjia Tang, and Jason Mars.  An evaluation dataset for intent classification and out-of-scope prediction.  In Proceedings of the Conference on Empirical Methods in Natural Language Processing and the International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 1311–1316, Hong Kong, November 2019\.
  * [24] Xingkun Liu, Arash Eshghi, Pawel Swietojanski, and Verena Rieser.  Benchmarking natural language understanding services for building conversational agents.  In Proceedings of the International Workshop on Spoken Dialogue Systems Technology (IWSDS), pages 165–183, Siracusa, Italy, April 2019.
  * [25] Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov.  RoBERTa: A robustly optimized BERT pretraining approach.  arXiv preprint arXiv:1907.11692, pages 1–13, 2019.
  * [26] Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu.  Towards deep learning models resistant to adversarial attacks.  In Proceedings of the 6th International Conference on Learning Representation (ICLR), Vancouver, 2018.
  * [27] Yu Meng, Chenyan Xiong, Payal Bajaj, Saurabh Tiwary, Paul Bennett, Jiawei Han, and Xia Song.  Coco-lm: Correcting and contrasting text sequences for language model pretraining.  arXiv preprint arXiv:2102.08473, pages 1–13, 2021.
  * [28] Takeru Miyato, Andrew M. Dai, and Ian Goodfellow.  Adversarial training methods for semi-supervised text classification.  In Proceedings of the 5th International Conference on Learning Representation (ICLR), Toulon, France, 2017.
  * [29] Takeru Miyato, Shin ichi Maeda, Masanori Koyama, Ken Nakae, and Shin Ishii.  Distributional smoothing with virtual adversarial training.  In Proceedings of the 4th International Conference on Learning Representation (ICLR), San Juan, Puerto Rico, 2016.
  * [30] Lin Pan, Chung-Wei Hang, Haode Qi, Abhishek Shah, Saloni Potdar, and Mo Yu.  Multilingual BERT post-pretraining alignment.  In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 210–219, Online, June 2021.
  * [31] Nicolas Papernot, Patrick McDaniel, Somesh Jha, Matt Fredrikson, Z Berkay Celik, and Ananthram Swami.  The limitations of deep learning in adversarial settings.  In 2016 IEEE European symposium on security and privacy (EuroS&P), pages 372–387. IEEE, 2016.
  * [32] Haode Qi, Lin Pan, Atin Sood, Abhishek Shah, Ladislav Kunc, Mo Yu, and Saloni Potdar.  Benchmarking commercial intent detection services with practice-driven evaluations.  In Proceedings of the 21st Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies: Industry Papers (NAACL), pages 304–310, Online, June 2021.
  * [33] Ali Shafahi, Mahyar Najibi, Amin Ghiasi, Zheng Xu, John Dickerson, Christoph Studer, Larry S Davis, Gavin Taylor, and Tom Goldstein.  Adversarial training for free!  arXiv preprint arXiv:1904.12843, 2019.
  * [34] Dawn Song, Kevin Eykholt, Ivan Evtimov, Earlence Fernandes, Bo Li, Amir Rahmati, Florian Tramer, Atul Prakash, and Tadayoshi Kohno.  Physical adversarial examples for object detectors.  In 12th USENIX Workshop on Offensive Technologies (WOOT), 2018\.
  * [35] Jiawei Su, Danilo Vasconcellos Vargas, and Kouichi Sakurai.  One pixel attack for fooling deep neural networks.  IEEE Transactions on Evolutionary Computation, 23(5):828–841, 2019\.
  * [36] Chi Sun, Xipeng Qiu, Yige Xu, and Xuanjing Huang.  How to fine-tune bert for text classification?  arXiv preprint arXiv:1905.05583, pages 1–10, 2020.
  * [37] Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus.  Intriguing properties of neural networks.  In Proceedings of the 2nd International Conference on Learning Representation (ICLR), Banff, AB, Canada, 2014.
  * [38] Aaron van den Oord, Yazhe Li, and Oriol Vinyals.  Representation learning with contrastive predictive coding.  arXiv preprint arXiv:1807.03748, pages 1–13, 2019.
  * [39] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Łukasz Kaiser, and Illia Polosukhin.  Attention is all you need.  In Advances in Neural Information Processing Systems (NIPS), pages 5998–6008, Long Beach, CA, December 2017.
  * [40] Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel R. Bowman.  GLUE: A multi-task benchmark and analysis platform for natural language understanding.  In Proceedings of the 7th International Conference on Learning Representation (ICLR), New Orleans, 2019.
  * [41] Xiangpeng Wei, Yue Hu, Rongxiang Weng, Luxi Xing, Heng Yu, and Weihua Luo.  On learning universal representations across languages.  arXiv preprint arXiv:2007.15960, pages 1–13, 2020.
  * [42] Adina Williams, Nikita Nangia, and Samuel Bowman.  A broad-coverage challenge corpus for sentence understanding through inference.  In Proceedings of the 19th Annual Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (HLT-NAACL), pages 1112–1122, New Orleans, June 2018. The Association for Computational Linguistics.
  * [43] Yi Wu, David Bamman, and Stuart Russell.  Adversarial training for relation extraction.  In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1778–1783, Copenhagen, September 2017.
  * [44] Zhuofeng Wu, Sinong Wang, Jiatao Gu, Madian Khabsa, Fei Sun, and Hao Ma.  CLEAR: Contrastive learning for sentence representation.  arXiv preprint arXiv:2012.15466, pages 1–10, 2020.
  * [45] Cihang Xie, Jianyu Wang, Zhishuai Zhang, Yuyin Zhou, Lingxi Xie, and Alan Yuille.  Adversarial examples for semantic segmentation and object detection.  In Proceedings of the IEEE International Conference on Computer Vision, pages 1369–1378, 2017.
  * [46] Xiaoyong Yuan, Pan He, Qile Zhu, and Xiaolin Li.  Adversarial examples: Attacks and defenses for deep learning.  IEEE transactions on neural networks and learning systems, 30(9):2805–2824, 2019.
  * [47] Chen Zhu, Yu Cheng, Zhe Gan, Siqi Sun, Tom Goldstein, and Jingjing Liu.  FreeLB: Enhanced adversarial training for natural language understanding.  In Proceedings of the 8th International Conference on Learning Representation (ICLR), Addis Ababa, Ethiopia, 2020.

##  Appendix A Additional Results

Model | MNLI | QQP | QNLI | MRPC | RTE | CoLA | SST-2 | Avg
---|---|---|---|---|---|---|---|---
BERT${}_{\textrm{Large}}$ \+ $AT^{x}$ \+ CTR | 87.1 | 91.9 | 92.8 | 91.3 | 71.8 | 63.3 | 94.2 | 84.6
BERT${}_{\textrm{Large}}$ \+ $AT^{V}$ \+ CTR | 87.4 | 92.2 | 93.0 | 91.6 | 71.5 | 65.8 | 95.2 | 85.2
RoBERTa${}_{\textrm{Large}}$ $AT^{x}$ \+ CTR | 90.9 | 92.4 | 94.9 | 92.6 | 86.3 | 69.4 | 96.6 | 89.0
RoBERTa${}_{\textrm{Large}}$ \+ $AT^{V}$ \+ CTR | 91.1 | 92.5 | 95.1 | 93.0 | 87.4 | 69.4 | 97.0 | 89.4

Table 9: Comparison between perturbation applied to word embeddings and word
embedding matrix of BERT${}_{\textrm{Large}}$ and RoBERTa${}_{\textrm{Large}}$
during CAT fine-tuning. Results are on the dev sets of the GLUE benchmark. We
use $AT^{x}$ to denote adversarial training with perturbation applied to word
embeddings and $AT^{V}$ to the word embedding matrix. CTR refers to the
contrastive learning component. On average, perturbing word embedding matrix
outperforms perturbing word embeddings by $0.6\%$ for
BERT${}_{\textrm{Large}}$ and $0.4\%$ for RoBERTa${}_{\textrm{Large}}$.