# LM-CPPF: Paraphrasing-Guided Data Augmentation
for Contrastive Prompt-Based Few-Shot Fine-Tuning

Amirhossein Abaskohi1, Sascha Rothe2, Yadollah Yaghoobzadeh1,3
1School of Electrical and Computer Engineering
College of Engineering, University of Tehran, Tehran, Iran
2Google DeepMind, Zürich, Switzerland
3 Tehran Institute for Advanced Studies, Khatam University, Iran
<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>
###### Abstract

In recent years, there has been significant progress in developing pre-trained
language models for NLP. However, these models often struggle when fine-tuned
on small datasets. To address this issue, researchers have proposed various
adaptation approaches. Prompt-based tuning is arguably the most common way,
especially for larger models. Previous research shows that adding contrastive
learning to prompt-based fine-tuning is effective as it helps the model
generate embeddings that are more distinguishable between classes, and it can
also be more sample-efficient as the model learns from positive and negative
examples simultaneously. One of the most important components of contrastive
learning is data augmentation, but unlike computer vision, effective data
augmentation for NLP is still challenging. This paper proposes LM-CPPF,
Contrastive Paraphrasing-guided Prompt-based Fine-tuning of Language Models,
which leverages prompt-based few-shot paraphrasing using generative language
models, especially large language models such as GPT-3 and OPT-175B, for data
augmentation. Our experiments on multiple text classification benchmarks show
that this augmentation method outperforms other methods, such as easy data
augmentation, back translation, and multiple templates.111Our implementation
is publicly available at: https://github.com/AmirAbaskohi/LM-CPPF

##  1 Introduction

Pre-trained language models (PLMs) are trained on large-scaled corpora in a
self-supervised fashion. They have fundamentally changed the NLP community in
the past few years by achieving impressive results in various Tasks Devlin et
al. (2018); Radford et al. (2018); Yang et al. (2019); Chiang et al. (2022).
However, when PLMs are fine-tuned on small datasets, their performance
declines. Researchers have proposed various techniques to adapt PLMs to these
scenarios Snell et al. (2017); Sung et al. (2018). In addition to performance,
fine-tuning PLMs to learn a new task is parameter inefficient, because an
entirely new model is required for every task Houlsby et al. (2019).

By the introduction of GPT-3 Brown et al. (2020b) with 175B parameters, it has
been shown that Large Language Models (LLMs) are efficient few-shot learners
as they can use their knowledge more effectively. One of the key features of
these LLMs is their ability to perform multiple tasks using prompts. A
language prompt is a piece of text that is added to the input query to help
the model make more accurate predictions. In addition, LLMs can be fine-tuned
for specific tasks using few examples. This has made them powerful tools for
NLP tasks, especially in few-shot scenarios. However, that might not be
practical for many situations because of the model size. Therefore, there is a
need to adapt smaller PLMs to work in a similar way to LLMs.

Prompt-based fine-tuning is a method for adapting PLMs to specific tasks or
domains by providing a prompt Schick and Schütze (2020a, b). This approach has
been shown to be effective in various NLP tasks, including text classification
Han et al. (2021); Wang et al. (2022) and question answering Yao et al.
(2022). However, it can be challenging to achieve strong performance when only
a few examples are available for each task. Gao et al. (2020) introduced a
prompt-based fine-tuning method called LM-BFF for RoBERTa Liu et al. (2019) to
tackle this issue. Their approach includes automated prompt generation and a
more effective way of using task examples in fine-tuning.

Building on the success of LM-BFF and considering contrastive learning’s
promising results both in computer vision Chen et al. (2020) and NLP Chen et
al. (2020); Miao et al. (2021), Jian et al. (2022) present a contrastive
learning framework to improve LM-BFF. They propose a Supervised Contrastive
Learning (SCL) approach Khosla et al. (2020) that classifies inputs using
different augmented views of the data. These views are created using different
templates for their demonstrations when building prompts.

In this paper, we show that while SCL at the feature space can be beneficial,
the use of different templates can limit the full potential of this approach.
We propose LM-CPPF (Contrastive Paraphrasing-guided Prompt-based Fine-tuning
of Language Models), in which we integrate the knowledge of LLMs like GPT-3
and OPT-175B Zhang et al. (2022) to build different views using paraphrasing.
These models can generate paraphrases of a sentence with different syntax, not
just by changing the lexicalization. Previous studies have considered
generating paraphrases a challenging and costly NLP task Siddique et al.
(2020); Garg et al. (2021); Zhou and Bhat (2021). However, PLMs can generate
paraphrases easily and effectively using in-context learning with few
examples. Although prior research has studied paraphrase generation with PLMs
Roy and Grangier (2019); Hegde and Patil (2020), to the best of our knowledge,
this is the first time that large LLMs are utilized to generate paraphrases
with prompts as an augmentation method. Our experiments on six different text
classification tasks demonstrate that LM-CPPF outperforms the previous SOTA
methods of data augmentation in prompt-based fine-tuning, including Easy Data
Augmentation (EDA) Wei and Zou (2019), Back Translation (BT) Sugiyama and
Yoshinaga (2019), and multiple templates Jian et al. (2022).

##  2 Related Works

LLMs like GPT-3 Brown et al. (2020a) can perform NLP tasks with few examples
and natural prompts. But smaller models are not efficient with this approach
and there are data sparsity and prompt sensitivity issues. To address these
challenges, Gao et al. (2021) propose LM-BFF, a framework that leverages a
large PLM to automatically generate task-specific prompts for smaller models.
It improves their few-shot performance on different NLP tasks. Some work have
enhanced LM-BFF with different prompt tuning methods. For example, Zhou et al.
(2022) present a dual context-guided continuous prompt tuning method that uses
the language context and connects discrete and continuous prompt tuning. Jian
et al. (2022) integrate contrastive learning and data augmentation with LM-
BFF. In their contrastive part, in addition to comparing different instances
from the same or different classes, they introduced a novel prompt-specific
augmentation method. In their approach, they change the template of the
prompt. In this paper, we use few-shot paraphrasing with LLMs for contrastive
prompt-tuning, which fine-tunes models with natural prompts.

Paraphrasing is the task of expressing the same meaning with different words
or structures. It can be used to create training data with increased diversity
and naturalness for NLP tasks, such as text classification Xie et al. (2020),
natural language inference Kumar et al. (2019), and text summarization Loem et
al. (2022), surpassing the limitations of traditional approaches. Paraphrasing
helps with data scarcity and model generalization. There are different ways to
generate paraphrases for data augmentation. One is back-translation Sennrich
et al. (2016), which uses a translation system to convert a sentence to
another language and back. Another is to use paraphrasing models trained on
parallel paraphrase datasets Wieting and Gimpel (2018); Zhu et al. (2022).
PLMs can also generate paraphrases by using large-scale corpora, but they may
produce paraphrases that are not semantically consistent or relevant. LLMs can
reduce this problem as they encode and generate language better. In this
paper, we generate paraphrases by carefully prompting LLMs and then use them
for data augmentation.

##  3 Method

Figure 1:  Our method, LM-CPPF, includes two objectives: (I) MLM and (II)
Supervised Contrastive Learning. The target sentence is the first sentence in
each prompt with a [MASK] token. The target sentence of Sent_0 is used to
train our model and calculate the MLM loss. We build Sent_3, whose target
sentence is a paraphrase of Sent_0’s target sentence. Sent_1 and Sent_2,
sampled from the dataset, have target sentences in the same and different
classes as Sent_0, respectively.

#### Background

Contrastive learning’s success relies on data augmentation, which creates new
views of the input data. Contrastive learning has been utilized for various
tasks in deep learning Le-Khac et al. (2020); Conde and Turgutlu (2021);
Abaskohi et al. (2022); however, most NLP data augmentation methods may
influence semantics which results in limited improvement. For instance, EDA’s
synonym substitution may create entirely new samples since words do not have
equal senses Keselj (2009). In addition to these augmentation methods, the
approach used in Jian et al. (2022) cannot be counted as data augmentation as
the sample is still the same and only the template for the verbalizer changes.
Although it is a creative approach designed specifically for the prompt-based
method of LM-BFF, it is limited in performance even compared to EDA in several
benchmarks. Furthermore, it requires an expert to create multiple templates
for each task, which makes it challenging for newly emerged tasks. Here we
propose leveraging LLMs to generate paraphrases and introduce LM-CPPF, a novel
approach aimed at addressing the challenges associated with contrastive
prompt-based fine-tuning of PLMs.

#### Few-shot paraphrasing

Paraphrasing is one of the best methods for data augmentation in NLP. One of
the most popular approaches for paraphrasing is back-translation (BT) Sugiyama
and Yoshinaga (2019) due to its simplicity and efficiency. Nonetheless, BT’s
performance depends a lot on the intermediary language. In this paper, we,
instead, use a combination of prompt-learning and LLMs for paraphrasing. In
few-shot paraphrasing, an LLM rewrites a sentence given an instruction and a
few examples. We believe that LLMs generate high-quality paraphrases due to
their encoded semantic and sentence structure knowledge. We utilize GPT-3
Brown et al. (2020b) or OPT-175B Zhang et al. (2022) via their official APIs
222OPT-175B: opt.alpa.ai and GPT-3: openai.com/api for generating paraphrases.

To avoid violating the prompt-based fine-tuning settings, we do not include
any additional task data in generating our paraphrases. Following the few-shot
setting in LM-BFF, we assume to have access to a PLM $M$, datasets
$\mathcal{D}_{train}$, and $\mathcal{D}_{test}$ with label space $\mathcal{Y}$
where there are only $\mathcal{K}=16$ examples per class in
$\mathcal{D}_{train}$. We use this setting for both prompt-based few-shot
paraphrasing and fine-tuning. To generate paraphrases, excluding the one
sample that we want to paraphrase, we use QuillBot333quillbot.com to create
paraphrases for our prompts for the remaining 15 samples in the same class of
$\mathcal{D}_{train}$. We leverage two types of prompts for paraphrasing: (I)
Only Demonstration: Here, the samples and their paraphrased versions are given
using the templates in Table C.3 to demonstrate the task of paraphrasing. (II)
Demonstrations with Instruction: In addition to the previous method, this one
includes instructions at the beginning of the prompt, defining paraphrasing
before demonstrations. These instructions can be seen in Table C.4.

Task | LM-BFF | LM-BFF+ | LM-BFF+ | LM-CPPF | LM-CPPF | LM-CPPF | LM-CPPF
---|---|---|---|---|---|---|---
SupConLoss | Multi-templates | GPT-3 | OPT | GPT-2 | FT GPT-2
SST-2 | 89.5 | 90.3 | 91.0 | 92.3 | 91.8 | 91.1 | 91.4
SST-5 | 48.5 | 49.6 | 50.3 | 52.8 | 52.2 | 51.4 | 51.6
MNLI | 62.3 | 63.2 | 64.8 | 68.4 | 66.2 | 65.6 | 65.8
CoLA | 6.9 | 9.6 | 11.6 | 14.1 | 13.3 | 10.7 | 11.8
QNLI | 61.2 | 65.4 | 67.2 | 69.2 | 68.5 | 67.5 | 67.8
CR | 89.7 | 89.9 | 90.2 | 91.4 | 91.1 | 90.2 | 90.7
Table 1:  Performance of LM-CPPF and our baselines in six datasets. LM-
BFF+Multi-templates refers to Jian et al. (2022). LM-BFF+SupConLoss uses the
same architecture of LM-BFF+Multi-templates, but without any data
augmentation, just integrating supervised contrastive and MLM loss functions.
Two cases are available for GPT-2: the pre-trained model and the GPT-2 fine-
tuned (FT) on ParaNMT-50M Wieting and Gimpel (2018) dataset. LM-BFF, LM-
BFF+Multi-template, and LM-CPPF (on average for all models used for
paraphrasing) have 0.77 and 1.02, and 1.65 standard deviations on average for
each task, respectively.

#### Contrastive prompt-based fine-tuning

LM-CPPF consists of two steps. The first step involves calculating the Masked
Language Modeling (MLM) loss by using the target sentence in the given
template, the specific demonstrations in the prompt, and the verbalizer
matched with the target sentence’s label. We calculate the supervised
contrastive loss in the second step by comparing the target prompt with
another sample with the same template but different random demonstrations.
This comparison sample can be in the same or a different class as the target
prompt. When the comparison sample belongs to a different class, it is
randomly sampled from the dataset. However, in cases where the comparison
sample belongs to the same class, an alternative approach is employed. This
involves either selecting another sample from the same class within the
dataset or applying data augmentation techniques, paraphrasing in our case, to
augment the target sample in order to create a new view of it. In both of
these cases, the demonstrations are not the same. Figure 1 illustrates the
fine-tuning process, and Algorithm D.1 shows our methodology when paraphrasing
creates a new view of the target sample. See Appendix D for more information.

##  4 Experiments

#### Evaluation datasets and protocol

Our method is evaluated on six different classification tasks from LM-BFF Liu
et al. (2021). The reported numbers represent the average accuracy from five
runs using Roberta-base Liu et al. (2019). In Section 4.1 where LLMs are
compared for paraphrasing, we also employed pre-trained and fine-tuned GPT-2
as an additional model for paraphrasing, allowing us to leverage smaller
models in our experiments. For the fine-tuning of GPT-2 specifically for
paraphrasing, we utilized the ParaNMT-50M Wieting and Gimpel (2018) dataset.
More details regarding the training process can be found in Appendix A.

###  4.1 Paraphrasing in Prompt Fine-tuning

This section presents the results of our fine-tuning approach using
paraphrasing on various NLP tasks. As shown in Table 1, LM-CPPF improves the
model’s accuracy on all tasks compared to the baseline method of LM-BFF+Multi-
templates Jian et al. (2022). Comparing the standard deviation of our model in
five runs and the standard deviations of LM-BFF and LM-BFF + Multi-templates,
we see that LM-CPPF has a higher standard deviation as it uses an intermediary
model for generating paraphrases. In contrast, LM-BFF + Multi-templates
integrates templates that have nearly equal performance Jian et al. (2022).

We also compare the effect of using GPT-3, OPT-175B, and GPT-2 as our language
model for few-shot paraphrasing. We did two experiments with GPT-2 large: (I)
Using a pre-trained version of GPT-2 where the weights are not tuned at all
(II) Fine-tuned GPT-2 where the model has been fine-tuned on the ParaNMT-50M
dataset. The results in Table 1 indicate that GPT-3 outperforms OPT-175B in
all tasks and GPT-2 has a lower performance, which was predictable since it
has significantly fewer parameters. Also, fine-tuned GPT-2 shows a better
performance which suggests that GPT-2’s knowledge after pre-training is not
enough for doing a task like paraphrasing. About the LLMs, although both
models have 175B parameters, OPT-175B has a $1/7$ carbon footprint of GPT-3,
and it is also freely available Zhang et al. (2022). Consequently, we base our
further analysis on OPT-175B.

Task | Few-shot | Back Traslation | SR | RI | RS | RD | EDA
---|---|---|---|---|---|---|---
Paraphrasing | AR | FR | DE | ZH | HI
SST-2 | 91.8 | 90.8 | 90.6 | 90.4 | 90.7 | 90.3 | 90.5 | 89.5 | 90.8 | 91.3 | 90.4
SST-5 | 52.2 | 49.2 | 49.3 | 49.1 | 49.6 | 48.3 | 47.9 | 49.3 | 49.3 | 48.2 | 48.2
MNLI | 66.2 | 64.3 | 63.1 | 63.8 | 65.4 | 62.2 | 62.9 | 63.2 | 61.7 | 60.2 | 60.3
CoLA | 13.3 | 6.7 | 6.8 | 6.4 | 7.1 | 5.9 | 6.3 | 5.8 | 5.8 | 5.1 | 5.1
QNLI | 68.5 | 66.5 | 66.2 | 65.8 | 66.6 | 64.3 | 66.1 | 65.9 | 66.3 | 65.6 | 63.3
CR | 91.1 | 88.5 | 88.6 | 88.4 | 88.7 | 87.9 | 89.8 | 89.1 | 89.3 | 89.6 | 89.7
Table 2:  Comparing the accuracy of our few-shot paraphrasing approach with
the Back Translation (BT) and Easy Data Augmentation (EDA) methods. EDA
includes Synonym Replacement (SR), Random Insertion (RI), Random Swap (RS),
and Random Deletion (RD). EDA in the results is combination of all of the four
mentioned methods. BT and EDA standard deviations are 1.31 and 1.4 on average,
respectively, while our approach has a standard deviation of 1.65.

###  4.2 Few-shot Paraphrasing vs. Other Data Augmentation Methods

Task | Template Number
---|---
1 | 2 | 3 | 4 | 5 | 6
SST-2 | 91.8 | 91.2 | 91.4 | 89.1 | 92.1 | 92.4
SST-5 | 52.2 | 53.1 | 52.7 | 53.4 | 53.6 | 54.1
MNLI | 66.2 | 65.9 | 66.9 | 66.1 | 66.2 | 66.4
CoLA | 13.3 | 12.7 | 13.2 | 13.8 | 13.4 | 13.6
QNLI | 68.5 | 68.4 | 68.6 | 68.5 | 68.8 | 69.3
CR | 91.1 | 91.2 | 91.3 | 91.5 | 91.7 | 92.2
Table 3: Performance of different paraphrasing prompt demonstration templates.

In this section, we present an experimental comparison of the performance of
the few-shot paraphrasing approach and other data augmentation methods,
including BT and EDA. The results are shown in Table 2. The BT approach is
evaluated using different intermediary languages (Arabic, French, Deutsch,
Chinese, and Hindi). The results indicate that BT’s performance is slightly
different across languages, with Chinese showing the highest performance. In
general, paraphrasing approaches, including BT, are better in comparison to
EDA. In SST-2 and CR, where the samples are usually simple sentences, BT shows
weaker performance than EDA. We believe the reason is that BT can be more
effective for longer sequences because longer sequences usually contain more
context and nuanced meaning. Moreover, EDA employs additional knowledge from
another PLM in certain actions, such as synonym substitution, similar to BT
and few-shot paraphrasing.

The few-shot paraphrasing approach introduced in this work outperforms both BT
and EDA. This confirms that using PLM’s knowledge properly in paraphrasing is
an effective and efficient data augmentation method. In few-shot paraphrasing,
we instruct the model to generate paraphrases that differ in lexicalization
and sentence structure.

###  4.3 Prompt Template Evaluation

As the heart of our method is the few-shot paraphrase generation done by LLMs,
we investigate the impact of different paraphrasing prompt demonstrations and
instruction templates on the performance of our model. Table 3 shows that the
last template presented in Table C.3 is better in almost all tasks. This
template, “<Original Text>, in other words <Paraphrased>”, uses a complete and
concrete sentence, unlike other templates, which use specific tokens, such as
“[Original]”, to distinguish between the original and the paraphrased version.
Also, we compare different instruction templates presented in Table C.4. As we
aimed to report our best result in each task here, we used the best
demonstration template for any particular task, which was determined in Table
3. Table 4 shows that the fourth template achieves the best performance, as it
precisely describes the task with its instruction “Generate a paraphrase of
the following text using different words and sentence structures while still
conveying the same meaning”.

Task | w/o Instruct | Template Number
---|---|---
1 | 2 | 3 | 4 | 5
SST-2 | 92.4 | 93.1 | 93 | 92.8 | 93.2 | 92.7
SST-5 | 54.1 | 54.7 | 54.5 | 54.2 | 54.9 | 54.3
MNLI | 66.9 | 67.8 | 67.5 | 67.1 | 68.2 | 67.2
CoLA | 13.6 | 13.1 | 13.2 | 12.6 | 13.3 | 12.8
QNLI | 69.3 | 69.8 | 70.1 | 69.5 | 70.2 | 69.6
CR | 92.2 | 93.1 | 92.8 | 92.6 | 93.3 | 92.4
Table 4: Performance of different paraphrasing prompt instruction templates on
various NLP tasks.

##  5 Conclusion

Our experiments demonstrated the effectiveness of using few-shot paraphrasing
as a data augmentation method for contrastive prompt-based fine-tuning of
PLMs. It outperformed other data augmentation methods in text classification
tasks, such as EDA, multiple templates, and back translation. We also found
that our approach is effective with GPT-3 or OPT-175b models in generating
paraphrases. Overall, LM-CPPF improves the performance of LM-BFF by large
margins using contrastive learning applied on paraphrases generated by LLMs.

## Limitations

Our approach relies on the performance of the few-shot paraphrasing. This
results in two limitations for our approach. One limitation is the difficulty
in accessing GPT-3 and OPT-175b models. These models currently need to be more
widely available. OPT-175B has a free version but it is very slow. Another
limitation is the need for annotated demonstrations for few-shot paraphrasing.
While there are available models and tools, like QuillBot, that can be used
for this purpose, their quality is not comparable to GPT-3 and OPT-175b. This
can limit the power of these tools in our approach. Using human knowledge to
paraphrase the demonstration can help these large models generate high-quality
paraphrases but it is expensive.

## Ethics Statement

The research conducted in this paper has been carried out in accordance with
the ethical principles of ACL. We have ensured that our experiments do not
harm any individuals or groups and have obtained informed consent from all
participants. As mentioned in the paper, we also tried to base our main
experimentation on the more environmentally-friendly option, OPT-175B.

## References

  * Abaskohi et al. (2022) Amirhossein Abaskohi, Fatemeh Mortazavi, and Hadi Moradi. 2022.  Automatic speech recognition for speech assessment of persian preschool children.  _arXiv preprint arXiv:2203.12886_.
  * Brown et al. (2020a) Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020a.  Language models are few-shot learners.  _Advances in neural information processing systems_ , 33:1877–1901.
  * Brown et al. (2020b) Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. 2020b.  Language models are few-shot learners.
  * Chen et al. (2020) Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. 2020.  A simple framework for contrastive learning of visual representations.  In _International conference on machine learning_ , pages 1597–1607. PMLR.
  * Chiang et al. (2022) Cheng-Han Chiang, Yung-Sung Chuang, and Hung-yi Lee. 2022.  Recent advances in pre-trained language models: Why do they work and how do they work.  In _Proceedings of the 2nd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 12th International Joint Conference on Natural Language Processing: Tutorial Abstracts_ , pages 8–15, Taipei. Association for Computational Linguistics.
  * Conde and Turgutlu (2021) Marcos V Conde and Kerem Turgutlu. 2021.  Clip-art: Contrastive pre-training for fine-grained art classification.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pages 3956–3960.
  * Devlin et al. (2018) Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018.  Bert: Pre-training of deep bidirectional transformers for language understanding.  _arXiv preprint arXiv:1810.04805_.
  * Gao et al. (2020) Tianyu Gao, Adam Fisch, and Danqi Chen. 2020.  Making pre-trained language models better few-shot learners.  _arXiv preprint arXiv:2012.15723_.
  * Gao et al. (2021) Tianyu Gao, Adam Fisch, and Danqi Chen. 2021.  Making pre-trained language models better few-shot learners.  In _Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)_ , pages 3816–3830.
  * Garg et al. (2021) Sonal Garg, Sumanth Prabhu, Hemant Misra, and G Srinivasaraghavan. 2021.  Unsupervised contextual paraphrase generation using lexical control and reinforcement learning.  _arXiv preprint arXiv:2103.12777_.
  * Han et al. (2021) Xu Han, Weilin Zhao, Ning Ding, Zhiyuan Liu, and Maosong Sun. 2021.  Ptr: Prompt tuning with rules for text classification.
  * Hegde and Patil (2020) Chaitra Hegde and Shrikumar Patil. 2020.  Unsupervised paraphrase generation using pre-trained language models.  _arXiv preprint arXiv:2006.05477_.
  * Houlsby et al. (2019) Neil Houlsby, Andrei Giurgiu, Stanislaw Jastrzebski, Bruna Morrone, Quentin De Laroussilhe, Andrea Gesmundo, Mona Attariyan, and Sylvain Gelly. 2019.  Parameter-efficient transfer learning for nlp.  In _International Conference on Machine Learning_ , pages 2790–2799. PMLR.
  * Jian et al. (2022) Yiren Jian, Chongyang Gao, and Soroush Vosoughi. 2022.  Contrastive learning for prompt-based few-shot language learners.  In _Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_ , pages 5577–5587, Seattle, United States. Association for Computational Linguistics.
  * Keselj (2009) Vlado Keselj. 2009.  Speech and language processing daniel jurafsky and james h. martin (stanford university and university of colorado at boulder) pearson prentice hall, 2009, xxxi+ 988 pp; hardbound, isbn 978-0-13-187321-6.
  * Khosla et al. (2020) Prannay Khosla, Piotr Teterwak, Chen Wang, Aaron Sarna, Yonglong Tian, Phillip Isola, Aaron Maschinot, Ce Liu, and Dilip Krishnan. 2020.  Supervised contrastive learning.  _Advances in Neural Information Processing Systems_ , 33:18661–18673.
  * Krizhevsky (2014) Alex Krizhevsky. 2014.  One weird trick for parallelizing convolutional neural networks.  _arXiv preprint arXiv:1404.5997_.
  * Kumar et al. (2019) Ashutosh Kumar, Satwik Bhattamishra, Manik Bhandari, and Partha Talukdar. 2019.  Submodular optimization-based diverse paraphrasing and its effectiveness in data augmentation.  In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)_ , pages 3609–3619, Minneapolis, Minnesota. Association for Computational Linguistics.
  * Le-Khac et al. (2020) Phuc H Le-Khac, Graham Healy, and Alan F Smeaton. 2020.  Contrastive representation learning: A framework and review.  _Ieee Access_ , 8:193907–193934.
  * Liu et al. (2021) Shikun Liu, Shuaifeng Zhi, Edward Johns, and Andrew J Davison. 2021.  Bootstrapping semantic segmentation with regional contrast.  _arXiv preprint arXiv:2104.04465_.
  * Liu et al. (2019) Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019.  Roberta: A robustly optimized bert pretraining approach.  _arXiv preprint arXiv:1907.11692_.
  * Loem et al. (2022) Mengsay Loem, Sho Takase, Masahiro Kaneko, and Naoaki Okazaki. 2022.  ExtraPhrase: Efficient data augmentation for abstractive summarization.  In _Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies: Student Research Workshop_ , pages 16–24, Hybrid: Seattle, Washington + Online. Association for Computational Linguistics.
  * Miao et al. (2021) Deshui Miao, Jiaqi Zhang, Wenbo Xie, Jian Song, Xin Li, Lijuan Jia, and Ning Guo. 2021.  Simple contrastive representation adversarial learning for nlp tasks.  _arXiv preprint arXiv:2111.13301_.
  * Radford et al. (2018) Alec Radford, Karthik Narasimhan, Tim Salimans, Ilya Sutskever, et al. 2018.  Improving language understanding by generative pre-training.
  * Roy and Grangier (2019) Aurko Roy and David Grangier. 2019.  Unsupervised paraphrasing without translation.  _arXiv preprint arXiv:1905.12752_.
  * Schick and Schütze (2020a) Timo Schick and Hinrich Schütze. 2020a.  Exploiting cloze questions for few shot text classification and natural language inference.  _arXiv preprint arXiv:2001.07676_.
  * Schick and Schütze (2020b) Timo Schick and Hinrich Schütze. 2020b.  It’s not just size that matters: Small language models are also few-shot learners.  _arXiv preprint arXiv:2009.07118_.
  * Sennrich et al. (2016) Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016.  Improving neural machine translation models with monolingual data.  In _Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pages 86–96, Berlin, Germany. Association for Computational Linguistics.
  * Siddique et al. (2020) AB Siddique, Samet Oymak, and Vagelis Hristidis. 2020.  Unsupervised paraphrasing via deep reinforcement learning.  In _Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining_, pages 1800–1809.
  * Snell et al. (2017) Jake Snell, Kevin Swersky, and Richard Zemel. 2017.  Prototypical networks for few-shot learning.  _Advances in neural information processing systems_ , 30.
  * Sugiyama and Yoshinaga (2019) Amane Sugiyama and Naoki Yoshinaga. 2019.  Data augmentation using back-translation for context-aware neural machine translation.  In _Proceedings of the Fourth Workshop on Discourse in Machine Translation (DiscoMT 2019)_ , pages 35–44.
  * Sung et al. (2018) Flood Sung, Yongxin Yang, Li Zhang, Tao Xiang, Philip HS Torr, and Timothy M Hospedales. 2018.  Learning to compare: Relation network for few-shot learning.  In _Proceedings of the IEEE conference on computer vision and pattern recognition_ , pages 1199–1208.
  * Tian et al. (2020) Yonglong Tian, Dilip Krishnan, and Phillip Isola. 2020.  Contrastive multiview coding.  In _European conference on computer vision_ , pages 776–794. Springer.
  * Wang et al. (2022) Jianing Wang, Chengyu Wang, Fuli Luo, Chuanqi Tan, Minghui Qiu, Fei Yang, Qiuhui Shi, Songfang Huang, and Ming Gao. 2022.  Towards unified prompt tuning for few-shot text classification.
  * Wei and Zou (2019) Jason Wei and Kai Zou. 2019.  Eda: Easy data augmentation techniques for boosting performance on text classification tasks.  _arXiv preprint arXiv:1901.11196_.
  * Wieting and Gimpel (2018) John Wieting and Kevin Gimpel. 2018.  ParaNMT-50M: Pushing the limits of paraphrastic sentence embeddings with millions of machine translations.  In _Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pages 451–462, Melbourne, Australia. Association for Computational Linguistics.
  * Xie et al. (2020) Qizhe Xie, Zihang Dai, Eduard Hovy, Minh-Thang Luong, and Quoc V. Le. 2020.  Unsupervised data augmentation for consistency training.
  * Yang et al. (2019) Zhilin Yang, Zihang Dai, Yiming Yang, Jaime Carbonell, Russ R Salakhutdinov, and Quoc V Le. 2019.  Xlnet: Generalized autoregressive pretraining for language understanding.  _Advances in neural information processing systems_ , 32.
  * Yao et al. (2022) Yuan Yao, Bowen Dong, Ao Zhang, Zhengyan Zhang, Ruobing Xie, Zhiyuan Liu, Leyu Lin, Maosong Sun, and Jianyong Wang. 2022.  Prompt tuning for discriminative pre-trained language models.  _arXiv preprint arXiv:2205.11166_.
  * Zhang et al. (2022) Susan Zhang, Stephen Roller, Naman Goyal, Mikel Artetxe, Moya Chen, Shuohui Chen, Christopher Dewan, Mona Diab, Xian Li, Xi Victoria Lin, et al. 2022.  Opt: Open pre-trained transformer language models.  _arXiv preprint arXiv:2205.01068_.
  * Zhou and Bhat (2021) Jianing Zhou and Suma Bhat. 2021.  Paraphrase generation: A survey of the state of the art.  In _Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing_ , pages 5075–5086.
  * Zhou et al. (2022) Jie Zhou, Le Tian, Houjin Yu, Zhou Xiao, Hui Su, and Jie Zhou. 2022.  Dual context-guided continuous prompt tuning for few-shot learning.  In _Findings of the Association for Computational Linguistics: ACL 2022_ , pages 79–84, Dublin, Ireland. Association for Computational Linguistics.
  * Zhu et al. (2022) Hongyu Zhu, Yan Chen, Jing Yan, Jing Liu, Yu Hong, Ying Chen, Hua Wu, and Haifeng Wang. 2022.  DuQM: A Chinese dataset of linguistically perturbed natural questions for evaluating the robustness of question matching models.  In _Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing_ , pages 7782–7794, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.

##  Appendix A Evaluation Setting

We used a learning rate of $1e^{-5}$ for MLM loss like LM-BFF. Although
contrastive learning algorithms often perform better with larger batch
training, due to resource limitations, we had to use half the batch size
suggested in Jian et al. (2022) for various tasks in the SCL phase. As
recommended in Krizhevsky (2014), we used $sqrt(0.5)\approx 0.7$ of the
learning rates mentioned in Jian et al. (2022) for this phase. Therefore, we
report baselines with our smaller batch size. Our method uses a single
template for each task’s prediction. The primary prompts are listed in
Appendix B. For the prompts used in the paraphrasing phase, with the exception
of experiments in Section 4.3, we used randomly selected templates from the
suggested prompts listed in Table C.3. In all of the experiments, we used
OPT-175B, except one of the results mentioned in Section 4.1, where we
compared OPT-175B and GPT-3 in paraphrasing.

We show the batch size and learning rate for SupCon in Table A.1. It is
important to note that the results of LM-BFF presented in the main paper were
obtained using the same large batch size as our method to ensure fair
comparisons.

We fine-tuned with a batch size that fits into GPU memory and is divisible by
the total number of examples in the task. Experiments were conducted on one
NVIDIA RTX-3090 with 24 GB memory using the RoBERTa-base model. Furthermore,
as per LM-BFF, we fine-tuned for a maximum of 1000 steps.

Task | Batch Size | Learning Rate
---|---|---
SST-2 | 8 | $7e^{-7}$
SST-5 | 20 | $7e^{-6}$
MNLI | 12 | $7e^{-6}$
CoLA | 8 | $7e^{-6}$
QNLI | 8 | $7e^{-6}$
CR | 16 | $7e^{-6}$
Table A.1: Batch size and learning rate for SupCon loss used for each task.

For the GPT-2 experiments in Table 1, we followed the same intructions for
generating paraphrases as we used for GPT-3 and OPT-175. In fine-tuning GPT-2,
we fine-tuned our model on ParaNMT-50M Wieting and Gimpel (2018) with the
batch size of 32 and learning rate of $1e^{-3}$ for 5 epochs.

##  Appendix B Task Prompts

The primary prompts utilized for each task in our experiments are displayed in
Table B.2. They were handpicked by LM-BFF Gao et al. (2021).

Task | Template | Verbalizers
---|---|---
SST-2 | <S1>It was [MASK] . | positive: great, negative: terrible
SST-5 | <S1>It was [MASK] . | v.positive: great, positive: good, neutral: okay, negative: bad, v.negative: terrible
MNLI | <S1>? [MASK] , <S2> | entailment: Yes, netural: Maybe, contradiction: No
CoLA | <S1>This is [MASK] . | grammatical: correct, not_grammatical: incorrect
QNLI | <S1>? [MASK] , <S2> | entailment: Yes, not_entailment: No
CR | <S1>It was [MASK] . | positive: great, negative: terrible
Table B.2: Primary templates and verbalizers (label words) used in our
experiments.

##  Appendix C Paraphrasing Prompts

To find the best prompt for paraphrasing, we checked different corpus
available online and found out how the paraphrasing examples are introduced.
We generated our prompts by using this information and our manual modification
in these templates.

In this demonstration prompt, we did not provide any explanations or
descriptions for the specific transformation applied to the input to produce
the output. Instead, we labeled the original sample and its paraphrase. For
instance, we used the token [Original] to indicate the original sentence in
the dataset and the token [Paraphrase] to indicate the paraphrased sample.
Table C.3 shows the templates we used for this approach.

Demonstration Template
---
Original:<Original Text>
Paraphrase:<Paraphrased Text>
[Original]:<Original Text>
[Paraphrase]:<Paraphrased Text>
Original:<Original Text>
Rewrite:<Paraphrased Text>
[Original]:<Original Text>
[Rewrite]:<Paraphrased Text>
Here is the original source: <Original Text>
Here is the paraphrase: <Paraphrased Text>
<Original Text>, in other words <Paraphrased Text>
Table C.3: The templates that were used to give examples of how the
paraphrasing should be done to the pre-trained language model.

In instruction for prompts, we provided examples and simple instructions to
the language models. The instructions were used to ask the model to generate
paraphrases before presenting them with examples. Table C.4 shows the
instructions we used to explain the task to the model at the beginning of our
prompts.

Instructions
---
Summarize the following text in your own words
Rewrite the following text that expresses the same idea in a different way
Generate a paraphrase of the following text that expresses the same ideas in a
different way
Generate a paraphrase of the following text using different words and sentence
structures while still conveying the same meaning
Generate a summary or paraphrase of the following text that captures the
essence of the ideas in a concise manner
Table C.4: The instructions that were used before giving examples to the
language model to describe the paraphrasing task.

##  Appendix D Contrastive Prompt-based Fine-tuning Details

Contrastive prompt-based fine-tuning contains two main steps: (1) Masked
Language Modeling and (2) Contrastive Learning.

Algorithm D.1 Learning from MLM and SupCon with Paraphrasing

1:Input:

2:Training set: $\mathcal{D}_{train}$

3:MLM model: $\mathcal{M}$

4:Function to concatenate two strings: $Concat$

5:Cross Entropy loss: $CE$

6:Supervised Contrastive loss: $SupCon$

7:Paraphrase function: $Paraphrase$

8:Function that samples from a dataset and puts it in the specific template:
$Sample$

9:// The third parameter of this function specifies

10:// whether to pus [MASK]or the verbalizer of

11:// the label

12:Template For Prompts: $Template$

13:$MaxStep=1000$

14:Preparing Samples:

15:for i < MaxStep do

16: $sent,y$=Sample($\mathcal{D}_{train}$, $Template$, false)

17: $demo_{1}$=Sample($\mathcal{D}_{train}$, $Template$, true)

18: $demo_{2}$=Sample($\mathcal{D}_{train}$, $Template$, true)

19: $demo_{3}$=Sample($\mathcal{D}_{train}$, $Template$, true)

20: $demo_{4}$=Sample($\mathcal{D}_{train}$, $Template$, true)

21: $demo_{in_{1}}$=Concat($demo_{1}$, $demo_{2}$,)

22: $demo_{in_{2}}$=Concat($demo_{3}$, $demo_{4}$,)

23: $x_{in_{1}}$=Concat($\mathcal{T}(sent),\mathcal{T}(demo_{in_{1}})$)

24:
$x_{in_{2}}$=Concat($\mathcal{T}$(Par($sent$))$,\mathcal{T}(demo_{in_{2}})$)

25: $\vartriangleright$ MLM Learning:

26: $output_{1}$ = $\mathcal{M}(x_{in_{1}})$

27: $\mathcal{L}_{MLM}$ = CE($output_{1},y$)

28: $\mathcal{L}_{MLM}$.backward()

29: optimizer.step()

30: $\vartriangleright$ Contrastive Learning:

31: $output_{2}$ = $\mathcal{M}(x_{in_{2}})$

32: $\mathcal{L}_{SupCon}$ = SupCon($output_{1},output_{2}$)

33: $\mathcal{L}_{SupCon}$.backward()

34: optimizer.step()

35:end for

#### Masked Language Modeling (MLM) Loss.

A classification task is approached as a Masked Language Modeling(MLM) problem
in prompt-based methods. The input consists of a sentence (sent) and a
template with a mask (temp) (i.e., $x_{prompt}=sent,temp([$MASK$])$), and the
goal is to determine the best token to fill in the $[$MASK$]$. This results in
a MLM loss, represented as $\mathcal{L}_{MLM}=MLM(x_{prompt},y)$, where $y$ is
the word label associated with $x_{prompt}$. LM-BFF Gao et al. (2021) uses
demonstrations of label words to improve the results. The input for this
approach includes the sentence ($sent_{0}$) and the masked template
($temp_{0}$) with a mask ([MASK]. The input also contains an additional
sentence ($sent_{i}$) with the same template ($temp_{0}$) with its own
verbalizer ($word_{i}$) for those sentences. The label words are sampled from
the training set. The classification loss is then calculated using this input.

The language model first encodes the input sentence $x_{in}$ into a sequence
of tokens, which are then mapped to a sequence of hidden states
${h_{1},h_{2},...,h_{L}}$. $L$ denotes the length of the sequence, and the
dimension of the hidden states is denoted by $d$. For example, in prompt-based
fine-tuning, if the input sentence ($x_{in}$) is “France missed the world cup
in penalties,” the corresponding prompt $x_{prompt}$ would be $[$CLS$]$
$x_{in}$, $[$MASK$]$.$[$SEP$]$. The model then determines whether it is more
likely to place the appropriate verbalizer at the [MASK] position. It has been
found that fine-tuning with this fill-in-the-blank framework is superior to
standard fine-tuning. The prediction of the model $\mathcal{M}$ for a class
$y\in\mathcal{Y}$ can be expressed by mapping the label space Y to the label
words, where $\mathcal{V}(y)$ represents the label word for class $y$. This
can be written as:

$\begin{split}p(y|x_{in})=p([MASK]=\mathcal{V}(y)|x_{in})\\\
=\frac{exp(w_{\mathcal{V}(y)}.h_{[MASK]})}{\sum_{y^{\prime}\in\mathcal{Y}}exp(w_{\mathcal{V}(y^{\prime})}.h_{[MASK]})}\end{split}$
(1)

where the weight vector of the MLM head is denoted by $w$.

In LM-BFF, the authors add demonstrations to the input $x_{prompt}$ to improve
the model’s understanding of verbalizers. As a result, the input to LM-BFF is
in the following form:

$\mathcal{T}(x_{in})\oplus\mathcal{T}(x_{in}^{1},y^{1})\oplus...\oplus\mathcal{T}(x_{in}^{k},y^{k})$
(2)

where $\mathcal{T}(x_{in}^{i},y^{i})$ illustrates the $i$-th demonstration in
the template $mathcal{T}$ with where the actual verbalizer of the samples
replaces the [MASK]. Also, $k$ is the number of demonstrations we want to use
in our prompts. This paper uses random sampling to select demonstrations from
the training set. The MLM loss is calculated as follows:

$\mathcal{L}_{MLM}=\sum_{(x_{in},y)\in\mathcal{D}_{train}}-log[p(y|x_{in})]$
(3)

#### Supervised Contrastive Loss.

Supervised Contrastive Learning is a specific form of contrastive learning
Chen et al. (2020); Tian et al. (2020); Liu et al. (2021) that clusters two
augmented batches at the class level in the feature space and calculates the
contrastive loss using Equation 4:

$\mathcal{L}_{SupCon}=(x^{\prime}_{1},x^{\prime}_{2},y)$ (4)

where $x^{\prime}_{1}$ and $x^{\prime}_{2}$ are the augmented version of the
input batch $x$ and $y$ is the actual label of the batch.

To use SupCon on multiple views of an input text, we first need to obtain two
views of the text:

$x_{in_{1}}=\mathcal{T}(sent)\oplus\mathcal{T}(demo_{1})\oplus\mathcal{T}(demo_{2})$
(5)
$x_{in_{2}}=\mathcal{T}(Par(sent))\oplus\mathcal{T}(demo_{3})\oplus\mathcal{T}(demo_{4})$
(6)

where $x_{in_{1}}$ is the same as $x_{prompt+demo}$ in LM-BFF and
$\mathcal{T}$ is a function that formats the sentence according to a specific
template. Instead of using a new template in which the newly generated sample
does not provide a new perspective, we use the few-shot paraphrasing ($Par$)
function. Also, $verb$ stands for the verbalizer used for the actual label of
the sample. Now using Equation 4 on two views, we can calculate the total
loss:

$\mathcal{L}_{Total}=\mathcal{L}_{SupCon}+\mathcal{L}_{MLM}$ (7)

Algorithm D.1 shows an overview of our method which uses contrastive few-shot
fine-tuning with few-shot paraphrasing. It is important to mention that
learning from $\mathcal{L}_{SupCon}$ requires one additional forward and
backward pass, which increases the computational cost by a factor of 1.5.
However, the cost is still the same as Jian et al. (2022)’s model due to the
$O(1)$ time complexity of the $Paraphrase$ function. Figure 1 shows the fine-
tuning procedure for one prompt sample and its new view created using few-shot
paraphrasing.


# P-Tuning v2: Prompt Tuning Can Be
Comparable to Fine-tuning Universally Across Scales and Tasks

Xiao Liu1,2∗, Kaixuan Ji1∗, Yicheng Fu1∗, Weng Lam Tam1, Zhengxiao Du1,2,
Zhilin Yang1,3†, Jie Tang1,2†
1Tsinghua University, KEG 2Beijing Academy of Artificial Intelligence (BAAI)
3Shanghai Qi Zhi Institute
<EMAIL_ADDRESS>

###### Abstract

Prompt tuning, which only tunes continuous prompts with a frozen language
model, substantially reduces per-task storage and memory usage at training.
However, in the context of NLU, prior work reveals that prompt tuning does not
perform well for normal-sized pretrained models. We also find that existing
methods of prompt tuning cannot handle hard sequence labeling tasks,
indicating a lack of universality. We present a novel empirical finding that
properly optimized prompt tuning can be universally effective across a wide
range of model scales and NLU tasks. It matches the performance of finetuning
while having only 0.1%-3% tuned parameters. Our method P-Tuning v2 is an
implementation of Deep Prompt Tuning Li and Liang (2021); Qin and Eisner
(2021) optimized and adapted for NLU. Given the universality and simplicity of
P-Tuning v2, we believe it can serve as an alternative to finetuning and a
strong baseline for future research.111Our code and data are released at
https://github.com/THUDM/P-tuning-v2.

††footnotetext:  † corresponding to: Zhilin Yang<EMAIL_ADDRESS>and
Jie Tang<EMAIL_ADDRESS>††footnotetext:  ∗ indicates equal
contribution.

##  1 Introduction

Pretrained language models Radford et al. (2019); Devlin et al. (2018); Yang
et al. (2019); Raffel et al. (2019) improve performance on a wide range of
natural language understanding (NLU) tasks. A widely-used method, fine-tuning,
updates the entire set of model parameters for a target task. While fine-
tuning obtains good performance, it is memory-consuming during training
because gradients and optimizer states for all parameters must be stored.
Moreover, keeping a copy of model parameters for each task during inference is
inconvenient since pre-trained models are usually large.

Prompting, on the other hand, freezes all parameters of a pre-trained model
and uses a natural language prompt to query a language model Brown et al.
(2020). For example, for sentiment analysis, we can concatenate a sample
(e.g., "Amazing movie!") with a prompt “This movie is [MASK]” and ask the pre-
trained language model to predict the probabilities of masked token being
“good” and “bad” to decide the sample’s label. Prompting requires no training
at all and stores one single copy of model parameters. However, discrete
prompting Shin et al. (2020); Gao et al. (2020) can lead to suboptimal
performance in many cases compared to fine-tuning.

Figure 1: Average scores on RTE, BoolQ and CB of SuperGLUE dev. With 0.1%
task-specific parameters, P-tuning v2 can match fine-tuning across wide scales
of pre-trained models, while Lester et al. (2021) & P-tuning can make it
conditionally at 10B scale.

Prompt tuning222We use “prompt tuning” to refer to a class of methods rather
than a particular method. is an idea of tuning only the continuous prompts.
Specifically, Liu et al. (2021); Lester et al. (2021) proposed to add
trainable continuous embeddings (also called continuous prompts) to the
original sequence of input word embeddings. Only the continuous prompts are
updated during training. While prompt tuning improves over prompting on many
tasks Liu et al. (2021); Lester et al. (2021); Zhong et al. (2021), it still
underperforms fine-tuning when the model size is not large, specifically less
than 10 billion parameters Lester et al. (2021). Moreover, as shown in our
experiments, prompt tuning performs poorly compared to fine-tuning on several
hard sequence labeling tasks such as extractive question answering (Cf.
Section 4.2).

Our main contribution in this paper is a novel empirical finding that properly
optimized prompt tuning can be comparable to fine-tuning universally across
various model scales and NLU tasks. In contrast to observations in prior work,
our discovery reveals the universality and potential of prompt tuning for NLU.

Technically, our approach P-tuning v2 is not conceptually novel. It can be
viewed as an optimized and adapted implementation of Deep Prompt Tuning Li and
Liang (2021); Qin and Eisner (2021) designed for generation and knowledge
probing. The most significant improvement originates from appling continuous
prompts for every layer of the pretrained model, instead of the mere input
layer. Deep prompt tuning increases the capacity of continuous prompts and
closes the gap to fine-tuning across various settings, especially for small
models and hard tasks. Moreover, we present a series of critical details of
optimization and implementation to ensure finetuning-comparable performance.

Experimental results show that P-tuning v2 matches the performance of fine-
tuning at different model scales ranging from 300M to 10B parameters and on
various hard sequence tagging tasks such as extractive question answering and
named entity recognition. P-tuning v2 has 0.1% to 3% trainable parameters per
task compared to fine-tuning, which substantially reduces training time memory
cost and per-task storage cost.

##  2 Preliminaries

NLU Tasks. In this work, we categorize NLU challenges into two families:
simple classification tasks and hard sequence labeling tasks.333Note that the
notions of “simple” and “hard” are specific to prompt tuning, because we find
sequence labeling tasks are more challenging for prompt tuning. Simple
classification tasks involve classification over a label space. Most datasets
from GLUE Wang et al. (2018) and SuperGLUE Wang et al. (2019) are in this
category. Hard sequence labeling tasks involve classification over a sequence
of tokens, such as named entity recognition and extractive question answering.

Prompt Tuning. Let $\mathcal{V}$ be the vocabulary of a language model
$\mathcal{M}$ and let $\mathbf{e}$ be the embedding layer of $\mathcal{M}$. In
the case of discrete prompting Schick and Schütze (2020), prompt tokens {"It",
"is", "[MASK]"} $\subset\mathcal{V}$ can be used to classify a movie review.
For example, given the input text $\mathbf{x}=$"Amazing movie!", the input
embedding sequence is formulated as
$[\mathbf{e}(\mathbf{x}),\mathbf{e}(\text{"It"}),\mathbf{e}(\text{"is"}),\mathbf{e}(\text{"[MASK]"})]$.

Lester et al. (2021) and Liu et al. (2021) introduce trainable continuous
prompts as a substitution to natural language prompts for NLU with the
parameters of pretrained language models frozen. Given the trainable
continuous embeddings $[h_{0},...,h_{i}]$, the input embedding sequence is
written as
$[\mathbf{e}(\mathbf{x}),h_{0},...,h_{i},\mathbf{e}(\text{"[MASK]"})]$, as
illustrated in Figure 2. Prompt tuning has been proved to be comparable to
fine-tuning on 10-billion-parameter models on simple classification tasks
Lester et al. (2021); Kim et al. (2021); Liu et al. (2021).

Figure 2: From Lester et al. (2021) & P-tuning to P-tuning v2. Orange blocks
(i.e., $h_{0},...,h_{i}$) refer to trainable prompt embeddings; blue blocks
are embeddings stored or computed by frozen pre-trained language models.

##  3 P-Tuning v2

###  3.1 Lack of Universality

Lester et al. (2021); Liu et al. (2021) have been proved quite effective in
many NLP applications Wang et al. (2021a, b); Chen et al. (2021); Zheng et al.
(2021); Min et al. (2021), but still fall short at replacing fine-tuning due
to lack of universality, as discussed below.

Lack of universality across scales. Lester et al. (2021) shows that prompt
tuning can be comparable to fine-tuning when the model scales to over 10
billion parameters. However, for medium-sized models (from 100M to 1B) that
are widely used, prompt tuning performs much worse than fine-tuning.

Lack of universality across tasks. Though Lester et al. (2021); Liu et al.
(2021) have shown superiority on some of the NLU benchmarks, the effectiveness
of prompt tuning on hard sequence tagging tasks is not verified. Sequence
tagging predicts a sequence of labels for each input token, which can be
harder and incompatible with verbalizers Schick and Schütze (2020). In our
experiments (Cf. Section 4.2 and Table 3), we show that Lester et al. (2021);
Liu et al. (2021) perform poorly on typical sequence tagging tasks compared to
fine-tuning.

Considering these challenges, we propose P-tuning v2, which adapts deep prompt
tuning Li and Liang (2021); Qin and Eisner (2021) as a universal solution
across scales and NLU tasks.

###  3.2 Deep Prompt Tuning

In Lester et al. (2021) and Liu et al. (2021), continuous prompts are only
inserted into the input embedding sequence (Cf. Figure 2 (a)). This leads to
two challenges. First, the number of tunable parameters is limited due to the
constraints of sequence length. Second, the input embeddings have relatively
indirect impact on model predictions.

To address these challenges, P-tuning v2 employs the idea of deep prompt
tuning Li and Liang (2021); Qin and Eisner (2021). As illustrated in Figure 2,
prompts in different layers are added as prefix tokens. On one hand, P-tuning
v2 have more tunable task-specific parameters (from 0.01% to 0.1%-3%) to allow
more per-task capacity while being parameter-efficient; on the other hand,
prompts added to deeper layers have more direct impact on model predictions
(see analysis in Appendix B).

###  3.3 Optimization and Implementation

There are a few useful details of optimization and implementation for
achieving the best performance.

Reparameterization. Prior works usually leverage a reparameterization encoder
such as an MLP Li and Liang (2021); Liu et al. (2021) to transform trainable
embeddings. However, for NLU, we discover that its usefulness depends on tasks
and datasets. For some datasets (e.g., RTE and CoNLL04), MLP brings a
consistent improvement; for the others, MLP leads to minimal or even negative
effects on the results (e.g., BoolQ and CoNLL12). See Appendix B for more
analysis.

Prompt Length. The prompt length plays a critical role in P-Tuning v2. We find
that different NLU tasks usually achieve their best performance with different
prompt lengths (Cf. Appendix B). Generally, simple classification tasks prefer
shorter prompts (less than 20); hard sequence labeling tasks prefer longer
ones (around 100).

Multi-task Learning. Multi-task learning jointly optimizes multiple tasks with
shared continuous prompts before fine-tuning for individual tasks. Multi-task
is optional for P-Tuning v2 but can be used for further boost performance by
providing a better initialization Gu et al. (2021).

Classification Head. Using a language modeling head to predict verbalizers
Schick and Schütze (2020) has been central for prompt tuning Liu et al.
(2021), but we find it unnecessary in a full-data setting and incompatible
with sequence labeling. P-tuning v2 instead applies a randomly-initialized
classification head on top of the tokens as in BERT Devlin et al. (2018) (Cf.
Figure 2).

To clarify P-tuning v2’s major contribution, we present a conceptual
comparison to existing prompt tuning approaches in Table 1.

Method | Task |  Re- param. |  Deep PT |  Multi- task |  No verb.
---|---|---|---|---|---
P-tuning Liu et al. (2021) |  KP NLU | LSTM | - | - | -
PromptTuning Lester et al. (2021) | NLU | - | - | ✓ | -
Prefix Tuning Li and Liang (2021) | NLG | MLP | ✓ | - | -
Soft Prompts Qin and Eisner (2021) | KP | - | ✓ | - | -
P-tuning v2 (Ours) |  NLU SeqTag | (depends) | ✓ | ✓ | ✓
Table 1: Conceptual comparison between P-tuning v2 and existing Prompt Tuning
approaches (KP: Knowledge Probe; SeqTag: Sequence Tagging; Re-param.:
Reparameterization; No verb.: No verbalizer).

##  4 Experiments

|  #Size | BoolQ | CB | COPA | MultiRC (F1a)
---|---|---|---|---|---
|  |  FT |  PT |  PT-2 |  FT |  PT |  PT-2 |  FT |  PT |  PT-2 |  FT |  PT |  PT-2
BERTlarge |  335M |  77.7 |  67.2 |  75.8 |  94.6 |  80.4 |  94.6 |  69.0 |  55.0 |  73.0 |  70.5 |  59.6 |  70.6
RoBERTalarge |  355M |  86.9 |  62.3 |  84.8 |  98.2 |  71.4 |  100 |  94.0 |  63.0 |  93.0 |  85.7 |  59.9 |  82.5
GLMxlarge |  2B |  88.3 |  79.7 |  87.0 |  96.4 |  76.4 |  96.4 |  93.0 |  92.0 |  91.0 |  84.1 |  77.5 |  84.4
GLMxxlarge |  10B |  88.7 |  88.8 |  88.8 |  98.7 |  98.2 |  96.4 |  98.0 |  98.0 |  98.0 |  88.1 |  86.1 |  88.1

|  #Size | ReCoRD (F1) | RTE | WiC | WSC
---|---|---|---|---|---
|  |  FT |  PT |  PT-2 |  FT |  PT |  PT-2 |  FT |  PT |  PT-2 |  FT |  PT |  PT-2
BERTlarge |  335M |  70.6 |  44.2 |  72.8 |  70.4 |  53.5 |  78.3 |  74.9 |  63.0 |  75.1 |  68.3 |  64.4 |  68.3
RoBERTalarge |  355M |  89.0 |  46.3 |  89.3 |  86.6 |  58.8 |  89.5 |  75.6 |  56.9 |  73.4 |  63.5 |  64.4 |  63.5
GLMxlarge |  2B |  91.8 |  82.7 |  91.9 |  90.3 |  85.6 |  90.3 |  74.1 |  71.0 |  72.0 |  95.2 |  87.5 |  92.3
GLMxxlarge |  10B |  94.4 |  87.8 |  92.5 |  93.1 |  89.9 |  93.1 |  75.7 |  71.8 |  74.0 |  95.2 |  94.2 |  93.3

Table 2: Results on SuperGLUE development set. P-tuning v2 surpasses P-tuning
& Lester et al. (2021) on models smaller than 10B, matching the performance of
fine-tuning across different model scales. (FT: fine-tuning; PT: Lester et al.
(2021) & P-tuning; PT-2: P-tuning v2; bold: the best; underline: the second
best).

|  #Size | CoNLL03 | OntoNotes 5.0 | CoNLL04
---|---|---|---|---
|  |  FT |  PT |  PT-2 |  MPT-2 |  FT |  PT |  PT-2 |  MPT-2 |  FT |  PT |  PT-2 |  MPT-2
BERTlarge |  335M |  92.8 |  81.9 |  90.2 |  91.0 |  89.2 |  74.6 |  86.4 |  86.3 |  85.6 |  73.6 |  84.5 |  86.6
RoBERTalarge |  355M |  92.6 |  86.1 |  92.8 |  92.8 |  89.8 |  80.8 |  89.8 |  89.8 |  88.8 |  76.2 |  88.4 |  90.6
DeBERTaxlarge |  750M |  93.1 |  90.2 |  93.1 |  93.1 |  90.4 |  85.1 |  90.4 |  90.5 |  89.1 |  82.4 |  86.5 |  90.1

|  #Size | SQuAD 1.1 dev (EM / F1) | SQuAD 2.0 dev (EM / F1)
---|---|---|---
|  | FT | PT | PT-2 | MPT-2 | FT | PT | PT-2 | MPT-2
BERTlarge |  335M |  84.2 |  91.1 |  1.0 |  8.5 |  77.8 |  86.0 |  82.3 |  89.6 |  78.7 |  81.9 |  50.2 |  50.2 |  69.7 |  73.5 |  72.7 |  75.9
RoBERTalarge |  355M |  88.9 |  94.6 |  1.2 |  12.0 |  88.5 |  94.4 |  88.0 |  94.1 |  86.5 |  89.4 |  50.2 |  50.2 |  82.1 |  85.5 |  83.4 |  86.7
DeBERTaxlarge |  750M |  90.1 |  95.5 |  2.4 |  19.0 |  90.4 |  95.7 |  89.6 |  95.4 |  88.3 |  91.1 |  50.2 |  50.2 |  88.4 |  91.1 |  88.1 |  90.8

|  #Size | CoNLL12 | CoNLL05 WSJ | CoNLL05 Brown
---|---|---|---|---
|  |  FT |  PT |  PT-2 |  MPT-2 |  FT |  PT |  PT-2 |  MPT-2 |  FT |  PT |  PT-2 |  MPT-2
BERTlarge |  335M |  84.9 |  64.5. |  83.2 |  85.1 |  88.5 |  76.0 |  86.3 |  88.5 |  82.7 |  70.0 |  80.7 |  83.1
RoBERTalarge |  355M |  86.5 |  67.2 |  84.6 |  86.2 |  90.2 |  76.8 |  89.2 |  90.0 |  85.6 |  70.7 |  84.3 |  85.7
DeBERTaxlarge |  750M |  86.5 |  74.1 |  85.7 |  87.1 |  91.2 |  82.3 |  90.6 |  91.2 |  86.9 |  77.7 |  86.3 |  87.0

Table 3: Results on Named Entity Recognition (NER), Question Answering
(Extractive QA), and Semantic Role Labeling (SRL). All metrics in NER and SRL
are micro-f1 score. (FT: fine-tuning; PT: P-tuning & Lester et al. (2021);
PT-2: P-tuning v2; MPT-2: Multi-task P-tuning v2; bold: the best; underline:
the second best).

We conduct extensive experiments over different commonly-used pre-trained
models and NLU tasks to verify the effectiveness of P-tuning v2. In this work,
all methods except for fine-tuning are conducted with frozen language model
backbones, which accords with Lester et al. (2021)’s setting but differs from
Liu et al. (2021)’s tuned setting. Ratios of task-specific parameters (e.g.,
0.1%) are derived from comparing continuous prompts’ parameters with
transformers’ parameters. Another thing to notice is that our experiments are
all conducted in the fully-supervised setting rather than few-shot setting.

NLU Tasks. First, we include datasets from SuperGLUE Wang et al. (2019) to
test P-tuning v2’s general NLU ability. Additionally, we introduce a suite of
sequence labeling tasks, including named entity recognition Sang and De
Meulder (2003); Weischedel et al. (2013); Carreras and Màrquez (2004),
extractive Question Answering Rajpurkar et al. (2016), and semantic role
labeling Carreras and Màrquez (2005); Pradhan et al. (2012)).

Pre-trained Models. We include BERT-large Devlin et al. (2018), RoBERTa-large
Liu et al. (2019), DeBERTa-xlarge He et al. (2020), GLM-xlarge/xxlarge Du et
al. (2021) for evaluation. They are all bidirectional models designed for NLU
tasks, covering a wide range of sizes from about 300M to 10B.

Multitask Learning. For the multi-task setting, we combine the training sets
of the datasets in each task type (e.g., combing all training sets of semantic
role labeling). We use separate linear classifiers for each dataset while
sharing the continuous prompts (Cf. Appendix A).

###  4.1 P-tuning v2: Across Scales

Table 2 presents P-tuning v2’s performances across model scales. In SuperGLUE,
performances of Lester et al. (2021) and P-tuning at smaller scales can be
quite poor. On the contrary, P-tuning v2 matches the fine-tuning performance
in all the tasks at a smaller scale. P-tuning v2 even significantly
outperforms fine-tuning on RTE.

In terms of larger scales (2B to 10B) with GLM Du et al. (2021), the gap
between Lester et al. (2021); Liu et al. (2021) and fine-tuning is gradually
narrowed down. On 10B scale, we have a similar observation as Lester et al.
(2021) reports, that prompt tuning becomes competitive to fine-tuning. That
said, P-tuning v2 is always comparable to fine-tuning at all scales but with
only 0.1% task-specific parameters needed comparing to fine-tuning.

###  4.2 P-tuning v2: Across Tasks

From Table 3, we observe that P-tuning v2 can be generally comparable to fine-
tuning on all tasks. P-tuning and Lester et al. (2021) show much poorer
performance, especially on QA, which might be the most challenging of the
three tasks. We also notice that there are some abnormal results of Lester et
al. (2021) and P-tuning on SQuAD 2.0. This is probably because SQuAD 2.0
contains unanswerable questions, which causes optimization challenges for
single-layer prompt tuning. Multi-task learning generally brings significant
improvements to P-Tuning v2 over most tasks except for QA.

###  4.3 Ablation Study

|  SST-2 |  RTE |  BoolQ |  CB
---|---|---|---|---
CLS & linear head |  96.3 |  88.4 |  84.8 |  96.4
Verbalizer & LM head |  95.8 |  86.6 |  84.6 |  94.6
Table 4: Comparison between [CLS] label with linear head and verbalizer with
LM head on RoBERTa-large.

Verbalizer with LM head v.s. [CLS] label with linear head. Verbalizer with LM
head has been a central component in previous prompt tuning approaches.
However, for P-tuning v2 in a supervised setting, it is affordable to tune a
linear head with about several thousand parameters. We present our comparison
in Table 4, where we keep other hyper-parameters and only change [CLS] label
with linear head to verbalizer with LM head. Here, for simplicity, we use
“true” and “false” for SST-2, RTE and BoolQ; “true”, “false” and “neutral” for
CB. Results indicate that there is no significant difference between
performances of verbalizer and [CLS].

(a) RTE

(b) BoolQ

Figure 3: Ablation study on prompt depth using BERT-large. “[x-y]" refers to
the layer-interval we add continuous prompts (e.g., “21-24” means we are add
prompts to transformer layers from 21 to 24). Same amount of continuous
prompts added to deeper transformer layers (i.e., more close to the output
layer) can yield a better performance than those added to beginning layers.

Prompt depth. The main difference between Lester et al. (2021); Liu et al.
(2021) and P-tuning v2 is the multi-layer continuous prompts. To verify its
exact influence, given a certain number of $k$ layers to add prompts, we
select them in both ascending and descending order to add prompts; for the
rest layers, we left them untouched. As shown in Figure 3, with the same
amount of parameters (i.e., num of transformer layers to add prompts), adding
them in the descending order is always better than in the ascending order. In
the RTE case, only adding prompts to layers 17-24 can yield a very close
performance to all layers.

##  5 Conclusions

We present P-tuning v2, a prompt tuning method. Despite its relatively limited
technical novelty, it contributes to a novel finding that prompt tuning can be
comparable to fine-tuning universally across scales (from 330M to 10B
parameters) and tasks. With high accuracy and parameter efficiency, P-Tuning
v2 can be a potential alternative for fine-tuning and a strong baseline for
future work.

## ACKNOWLEDGEMENT

We would like to thank the anonymous reviewers for their suggestions and
comments. Jie Tang is supported by the NSFC for Distinguished Young Scholar
(61825602) and NSFC (61836013). Kaixuan Ji is supported by Tsinghua University
Initiative Scientific Research Program and DCST Student Academic Training
Program.

## References

  * Brown et al. (2020) Tom B Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020.  Language models are few-shot learners.  _arXiv preprint arXiv:2005.14165_.
  * Carreras and Màrquez (2004) Xavier Carreras and Lluís Màrquez. 2004.  Introduction to the CoNLL-2004 shared task: Semantic role labeling.  In _Proceedings of the Eighth Conference on Computational Natural Language Learning (CoNLL-2004) at HLT-NAACL 2004_ , pages 89–97, Boston, Massachusetts, USA. Association for Computational Linguistics.
  * Carreras and Màrquez (2005) Xavier Carreras and Lluís Màrquez. 2005.  Introduction to the CoNLL-2005 shared task: Semantic role labeling.  In _Proceedings of the Ninth Conference on Computational Natural Language Learning (CoNLL-2005)_ , pages 152–164, Ann Arbor, Michigan. Association for Computational Linguistics.
  * Chen et al. (2021) Xiang Chen, Xin Xie, Ningyu Zhang, Jiahuan Yan, Shumin Deng, Chuanqi Tan, Fei Huang, Luo Si, and Huajun Chen. 2021.  Adaprompt: Adaptive prompt-based finetuning for relation extraction.  _arXiv preprint arXiv:2104.07650_.
  * Devlin et al. (2018) Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018.  Bert: Pre-training of deep bidirectional transformers for language understanding.  _arXiv preprint arXiv:1810.04805_.
  * Devlin et al. (2018) Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018.  Bert: Pre-training of deep bidirectional transformers for language understanding.  _arXiv e-prints_.
  * Du et al. (2021) Zhengxiao Du, Yujie Qian, Xiao Liu, Ming Ding, Jiezhong Qiu, Zhilin Yang, and Jie Tang. 2021.  All nlp tasks are generation tasks: A general pretraining framework.  _arXiv preprint arXiv:2103.10360_.
  * Gao et al. (2020) Tianyu Gao, Adam Fisch, and Danqi Chen. 2020.  Making pre-trained language models better few-shot learners.  _arXiv preprint arXiv:2012.15723_.
  * Gu et al. (2021) Yuxian Gu, Xu Han, Zhiyuan Liu, and Minlie Huang. 2021.  Ppt: Pre-trained prompt tuning for few-shot learning.  _arXiv preprint arXiv:2109.04332_.
  * He et al. (2020) Pengcheng He, Xiaodong Liu, Jianfeng Gao, and Weizhu Chen. 2020.  Deberta: Decoding-enhanced bert with disentangled attention.
  * Kim et al. (2021) Boseop Kim, HyoungSeok Kim, Sang-Woo Lee, Gichang Lee, Donghyun Kwak, Dong Hyeon Jeon, Sunghyun Park, Sungju Kim, Seonhoon Kim, Dongpil Seo, et al. 2021\.  What changes can large-scale language models bring? intensive study on hyperclova: Billions-scale korean generative pretrained transformers.  _arXiv preprint arXiv:2109.04650_.
  * Lester et al. (2021) Brian Lester, Rami Al-Rfou, and Noah Constant. 2021.  The power of scale for parameter-efficient prompt tuning.  _arXiv preprint arXiv:2104.08691_.
  * Li and Liang (2021) Xiang Lisa Li and Percy Liang. 2021.  Prefix-tuning: Optimizing continuous prompts for generation.  _arXiv preprint arXiv:2101.00190_.
  * Liu et al. (2021) Xiao Liu, Yanan Zheng, Zhengxiao Du, Ming Ding, Yujie Qian, Zhilin Yang, and Jie Tang. 2021.  Gpt understands, too.  _arXiv:2103.10385_.
  * Liu et al. (2019) Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019.  RoBERTa: A Robustly Optimized BERT Pretraining Approach.  _arXiv e-prints_.
  * Min et al. (2021) Sewon Min, Mike Lewis, Hannaneh Hajishirzi, and Luke Zettlemoyer. 2021.  Noisy channel language model prompting for few-shot text classification.  _arXiv preprint arXiv:2108.04106_.
  * Pradhan et al. (2012) Sameer Pradhan, Alessandro Moschitti, Nianwen Xue, Olga Uryupina, and Yuchen Zhang. 2012.  CoNLL-2012 shared task: Modeling multilingual unrestricted coreference in OntoNotes.  In _Joint Conference on EMNLP and CoNLL - Shared Task_ , pages 1–40, Jeju Island, Korea. Association for Computational Linguistics.
  * Qin and Eisner (2021) Guanghui Qin and Jason Eisner. 2021.  Learning how to ask: Querying lms with mixtures of soft prompts.  _arXiv preprint arXiv:2104.06599_.
  * Radford et al. (2019) Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019.  Language models are unsupervised multitask learners.  _OpenAI blog_ , 1(8):9.
  * Raffel et al. (2019) Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. 2019.  Exploring the limits of transfer learning with a unified text-to-text transformer.  _arXiv preprint arXiv:1910.10683_.
  * Rajpurkar et al. (2016) Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. 2016\.  Squad: 100,000+ questions for machine comprehension of text.
  * Sang and De Meulder (2003) Erik F Sang and Fien De Meulder. 2003.  Introduction to the conll-2003 shared task: Language-independent named entity recognition.  _arXiv preprint cs/0306050_.
  * Schick and Schütze (2020) Timo Schick and Hinrich Schütze. 2020.  It’s not just size that matters: Small language models are also few-shot learners.  _arXiv preprint arXiv:2009.07118_.
  * Shin et al. (2020) Taylor Shin, Yasaman Razeghi, Robert L Logan IV, Eric Wallace, and Sameer Singh. 2020.  Autoprompt: Eliciting knowledge from language models with automatically generated prompts.  _arXiv preprint arXiv:2010.15980_.
  * Wang et al. (2019) Alex Wang, Yada Pruksachatkun, Nikita Nangia, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel R. Bowman. 2019.  SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systems.  In _NeurIPS 2019_ , pages 3261–3275.
  * Wang et al. (2019) Alex Wang, Yada Pruksachatkun, Nikita Nangia, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel R. Bowman. 2019.  Superglue: A stickier benchmark for general-purpose language understanding systems.  _arXiv e-prints_.
  * Wang et al. (2018) Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel R. Bowman. 2018.  Glue: A multi-task benchmark and analysis platform for natural language understanding.  _arXiv e-prints_.
  * Wang et al. (2021a) Hongru Wang, Mingyu Cui, Zimo Zhou, Gabriel Pui Cheong Fung, and Kam-Fai Wong. 2021a.  Topicrefine: Joint topic prediction and dialogue response generation for multi-turn end-to-end dialogue system.  _arXiv preprint arXiv:2109.05187_.
  * Wang et al. (2021b) Shuo Wang, Zhaopeng Tu, Zhixing Tan, Wenxuan Wang, Maosong Sun, and Yang Liu. 2021b.  Language models are good translators.  _arXiv preprint arXiv:2106.13627_.
  * Weischedel et al. (2013) Ralph Weischedel, Martha Palmer, Mitchell Marcus, Eduard Hovy, Sameer Pradhan, Lance Ramshaw, Nianwen Xue, Ann Taylor, Jeff Kaufman, Michelle Franchini, et al. 2013.  Ontonotes release 5.0 ldc2013t19.  _Linguistic Data Consortium, Philadelphia, PA_ , 23.
  * Xu et al. (2021) Lu Xu, Zhanming Jie, Wei Lu, and Lidong Bing. 2021.  Better feature integration for named entity recognition.  _arXiv preprint arXiv:2104.05316_.
  * Yang et al. (2019) Zhilin Yang, Zihang Dai, Yiming Yang, Jaime Carbonell, Ruslan Salakhutdinov, and Quoc V Le. 2019.  Xlnet: Generalized autoregressive pretraining for language understanding.  _arXiv preprint arXiv:1906.08237_.
  * Zheng et al. (2021) Yanan Zheng, Jing Zhou, Yujie Qian, Ming Ding, Jian Li, Ruslan Salakhutdinov, Jie Tang, Sebastian Ruder, and Zhilin Yang. 2021.  Fewnlu: Benchmarking state-of-the-art methods for few-shot natural language understanding.  _arXiv preprint arXiv:2109.12742_.
  * Zhong et al. (2021) Zexuan Zhong, Dan Friedman, and Danqi Chen. 2021.  Factual probing is [mask]: Learning vs. learning to recall.  _arXiv preprint arXiv:2104.05240_.

(a) NLI: RTE

(b) NER: CoNLL04

(c) MQA: BoolQ

(d) SRL: CoNLL12

Figure 4: Ablation study on prompt length and reparamerization using RoBERTa-
large. The conclusion can be very different given certain NLU task and
dataset. (MQA: Multiple-choice QA)

##  Appendix A Problem Formulation on Sequence Tagging

Name entity recognition (NER). NER aims to predict all spans of words that
represent some given classes of entity with a sentence. We adopted CoNLL03
Sang and De Meulder (2003), OntoNotes 5.0 Weischedel et al. (2013) and CoNLL04
Carreras and Màrquez (2004). For CoNLL03 and CoNLL04, we trained our model on
the standard train-develop-test split. For OntoNotes 5.0, we use the same
train, develop, test split as Xu et al. (2021). All the datasets are labeled
in IOB2 format. We use sequence tagging to solve NER tasks by assigning labels
marking the beginning and inside some classes of entity. The language models
generate a representation for each token, and we use a linear classifier to
predict the labels. We use the official scripts to evaluate the results. For
the multi-task setting, we combine the training set of the three datasets for
pre-training. We use different linear classifiers for each dataset while
sharing the continuous prompts.

(Extractive) Question Answering (QA). Extractive QA is designed to extract the
answer from the context given the context and a question. We use SQuAD
Rajpurkar et al. (2016) 1.1 and 2.0, in which each answer is within a
continuous span of the context. Following tradition, we formulate the problem
as sequence tagging by assigning one of the two labels: ‘start’ or ‘end’ to
each token and at last selecting the span of the most confident start-end pair
as the extracted answer. If the probability of the most confident pair is
lower than a threshold, the model will assume the question unanswerable. For
the multi-task setting, our training set for pre-training combines the
training sets of SQuAD 1.1 and 2.0. When pre-training, we assume that all the
questions, regardless of their origin, are possibly unanswerable.

Semantic Role Labeling (SRL). SRL assigns labels to words or phrases in a
sentence that indicate their semantic roles in the sentence. We evaluate
P-tuning v2 on CoNLL05 Carreras and Màrquez (2005) and CoNLL12 Pradhan et al.
(2012). Since a sentence can have multiple verbs, we add the target verb token
to the end of each sentence to help recognize which verb is used for
prediction. We classify each word with a linear classifier based on the
corresponding semantic role representation. For multi-task setting, the pre-
train training set is a combination of the training set of CoNLL05 Carreras
and Màrquez (2005), CoNLL12 Pradhan et al. (2012) and propbank-release (a
common extend data used for training SRL). The multi-task training strategy is
similar to NER.

##  Appendix B More Ablation Study

Due to the page limit, we present hyper-parameters and architecture designs
ablations regarding reparameterization and prompt length in this section.

Embedding v.s. MLP reparameterization. In both prefix-tuning Li and Liang
(2021) and P-tuning Liu et al. (2021), authors discover the reparameterization
to be useful in improving training speed, robustness and performance. However,
we conduct experiments to show that the reparameterization effect is
inconsistent across different NLU tasks and datasets.

As shown in Figure 4, in RTE and CoNLL04, MLP reparameterization generally
indicates better performance than embedding for almost all prompt lengths.
However, in BoolQ, MLP and embedding’s results are competitive; in CoNLL12,
the embedding consistently outperforms MLP.

Prompt Length. Prompt length is yet another influential hyper-parameter for
P-tuning v2, and its optimal value varies from task to task. From Figure 4, we
observe that for simple NLU tasks, usually, a shorter prompt is enough for the
best performance; for hard sequence tasks, usually, a longer prompt than 100
would be helpful.

We also discover that reparameterization has a close bond with optimal prompt
length. For example, in RTE, CoNLL04, and BoolQ, MLP reparameterization
achieves its optimal result earlier than embedding. This conclusion may
contribute some thoughts on P-tuning’s optimization properties.