# Noise Stability Regularization for Improving BERT Fine-tuning

Hang Hua1*†, Xingjian Li2,3*, Dejing Dou${}^{2\textsuperscript{\Letter}}$,
Chengzhong Xu3, Jiebo Luo1

1University of Rochester, Rochester, NY, USA
{hhua2<EMAIL_ADDRESS>
2Big Data Lab, Baidu Research, Beijing, China
{lixingjian<EMAIL_ADDRESS>
3Department of Computer Science, University of Macau, Macau, China
<EMAIL_ADDRESS>
###### Abstract

Fine-tuning pre-trained language models such as BERT has become a common
practice dominating leaderboards across various NLP tasks. Despite its recent
success and wide adoption, this process is unstable when there are only a
small number of training samples available. The brittleness of this process is
often reflected by the sensitivity to random seeds. In this paper, we propose
to tackle this problem based on the noise stability property of deep nets,
which is investigated in recent literature Arora et al. (2018); Sanyal et al.
(2020). Specifically, we introduce a novel and effective regularization method
to improve fine-tuning on NLP tasks, referred to as Layer-wise Noise Stability
Regularization (LNSR). We extend the theories about adding noise to the input
and prove that our method gives a stabler regularization effect. We provide
supportive evidence by experimentally confirming that well-performing models
show a low sensitivity to noise and fine-tuning with LNSR exhibits clearly
higher generalizability and stability. Furthermore, our method also
demonstrates advantages over other state-of-the-art algorithms including
$\text{L}^{2}$-SP Li et al. (2018), Mixout Lee et al. (2020) and SMART Jiang
et al. (2020).

††footnotetext: †Contribution during internship at Baidu
Research.††footnotetext: * Equal contributions. 🖂 Correspondence.

##  1 Introduction

Large-scale pre-trained language models such as BERT Devlin et al. (2019) have
been widely used in natural language processing tasks Guu et al. (2020); Liu
(2019); Wadden et al. (2019); Zhu et al. (2020b). A typical process of
training a supervised downstream dataset is to fine-tune a pre-trained model
for a few epochs. In this process, most of the model’s parameters are reused,
while a random initialized task-specific layer is added to adapt the model to
the new task.

Figure 1: Attenuation of injected noise on the BERT-Large-Uncased model on the
MRPC task (X-axis: the layer index. Y-axis: the $L^{2}$ norm between the
original output and noise perturbed output). A curve starts at the layer where
a scaled Gaussian noise is injected to its output whose $l_{2}$ norm is set to
$5\%$ of the norm of its original output. As it propagates up, the injected
noise has a rapidly decreasing effect on the lower layers but becomes volatile
on the higher layers, which indicates the poor generalizability and
brittleness of the BERT top layers. Moreover, models with higher accuracies
(marked in the upper right) usually have lower error ratios or higher noise
stability in top layers.

Fine-tuning BERT has significantly boosted the state of the art performance on
natural language understanding (NLU) benchmarks such as GLUE Wang et al.
(2018) and SuperGLUE Wang et al. (2019). However, despite the impressive
empirical results, this process remains unstable due to the randomness
involved by data shuffling and the initialization of the task-specific layer.
The observed instability in fine-tuning BERT was first discovered by Devlin et
al. (2019); Dodge et al. (2020), and several approaches have been proposed to
solve this problem Lee et al. (2020); Zhang et al. (2020); Mosbach et al.
(2020).

In this study, we consider the fine-tuning stability of BERT from the
perspective of the sensitivity to input perturbation. This is motivated by
Arora et al. (2018) and Sanyal et al. (2020) who show that noise injected at
the lower layers has very little effect on the higher layers for neural
networks with good generalizability. However, for a well pre-trained BERT, we
find that the higher layers are still very sensitive to the lower layer’s
perturbation (as shown in Figure 1), implying that the high level
representations of the pre-trained BERT may not generalize well on
downstreaming tasks and consequently lead to instability. This phenomenon
coincides with the observation that transferring the top pre-trained layers of
BERT slows down learning and hurts performance Zhang et al. (2020). In
addition, Yosinski et al. (2014) also point out that in transfer learning
models for object recognition, the lower pre-trained layers learn more general
features while the higher layers closer to the output specialize more to the
pre-training tasks. We argue that this result also applies to BERT.
Intuitively, if a trained model is insensitive to the perturbation of the
lower layers’ output, then the model is confident about the output, and vice
versa. Based on the above theoretical and empirical results, we propose a
simple and effective regularization method to reduce the noise sensitivity of
BERT and thus improve the stability and performance of fine-tuned BERT.

To verify our approach, we conduct extensive experiments on different few-
sample (fewer than 10k training samples) NLP tasks, including CoLA Warstadt et
al. (2019), MRPC Dolan and Brockett (2005), RTE Wang et al. (2018); Dagan et
al. (2005); Bar-Haim et al. (2006); Giampiccolo et al. (2007), and STS-B Cer
et al. (2017). With the layer-wise noise stability regularization, we obtain
strong empirical performance. Compared with other state-of-the-art models, our
approach not only improves the fine-tuning stability (with a smaller standard
deviation) but also consistently improve the overall performance (with a
larger mean, median and maximum).

In summary, our main contributions are:

  * •

We propose a lightweight and effective regularization method, referred to as
Layer-wise Noise Stability Regularization (LNSR) to improve the local
Lipschitz continuity of each BERT layer and thus ensure the smoothness of the
whole model. The empirical results show that the fine-tuned BERT models
regularized with LNSR obtain significantly more accurate and stable results.
LNSR also outperforms other state-of-the-art methods aiming at stabilizing
fine-tuning such as $\text{L}^{2}$-SP Li et al. (2018), Mixout Lee et al.
(2020) and SMART Jiang et al. (2020).

  * •

We are the first to study the effect of noise stability in NLP tasks. We
extend classic theories of adding noise to explicitly constraining the output
consistency when adding noise to the input. We theoretically prove that our
proposed layer-wise noise stability regularizer is equivalent to a special
case of the Tikhonov regularizer, which serves as a stabler regularizer than
simply adding noise to the input Rifai et al. (2011).

  * •

We investigate the relation of the noise stability property to the
generalizability of BERT. We find that in general, models with good
generalizability tend to be insensitive to noise perturbation; the lower
layers of BERT show a better error resilience property but the higher layers
of BERT remain sensitive to the lower layers’ perturbation (as is depicted in
Figure 1).

##  2 Related Work

###  2.1 Pre-training

Pre-training has been well studied in machine learning and natural language
processing Erhan et al. (2009, 2010). Mikolov et al. (2013) and Pennington et
al. (2014) proposed to use distributional representations (i.e., word
embeddings) for individual words. Dai and Le (2015) proposed to train a
language model or an auto-encoder with unlabeled data and then leveraged the
obtained model to finetune downstream tasks. Recently, pre-trained language
models, like ELMo Peters et al. (2018), GPT/GPT-2 Radford (2018); Radford et
al. (2019), BERT Devlin et al. (2019), cross-lingual language model (briefly,
XLM) Lample and Conneau (2019), XLNet Yang et al. (2019), RoBERTa Liu et al.
(2019) and ALBERTLan et al. (2020) have attracted more and more attention in
natural language processing communities. The models are first pre-trained on
large amount of unlabeled data to capture rich representations of the input,
and then applied to the downstream tasks by either providing context-aware
embeddings of an input sequence Peters et al. (2018), or initializing the
parameters of the downstream model Devlin et al. (2019) for fine-tuning. Such
pre-training approaches deliver decent performance on natural language
understanding tasks.

###  2.2 Instability in Fine-tuning

Fine-tuning instability of BERT has been reported in various previous works.
Devlin et al. (2019) report instabilities when fine-tuning BERT on small
datasets and resort to performing multiple restarts of fine-tuning and
selecting the model that performs best on the development set. Dodge et al.
(2020) performs a large-scale empirical investigation of the fine-tuning
instability of BERT. They found dramatic variations in fine-tuning accuracy
across multiple restarts and argue how it might be related to the choice of
random seed and the dataset size. Lee et al. (2020) propose a new
regularization method named Mixout to improve the stability and performance of
fine-tuning BERT. Zhang et al. (2020) evaluate the importance of debiasing
step empirically by fine-tuning BERT with both BERTAdam and standard Adam
optimizer Kingma and Ba (2015) and propose a re-initialization method to get a
better initialization point for fine-tuning optimization. Mosbach et al.
(2020) analyses the cause of fine-tuning instability and propose a simple but
strong baseline (small learning rate combined with bias correction).

###  2.3 Regularization

There has been several regularization approaches to stabilizing the
performance of models. Loshchilov and Hutter (2019) propose a decoupled weight
decay regularizer integrated in Adam Kingma and Ba (2015) optimizer to prevent
neural networks from being too complicate. Gunel et al. (2020) use contrastive
learning method to augment training set to improve the generalization
performance. In addition, spectral norm Yoshida and Miyato (2017); Roth et al.
(2019) serves as a general method can also be used to constrain the Lipschitz
continuous of matrix, which can increase the stability of generalized neural
networks.

There are also several noise-based methods have been proposed to improve the
generalizability of pre-trained language models, including SMART Jiang et al.
(2020), FreeLB Zhu et al. (2020a) and R3F Aghajanyan et al. (2020). They
achieves state of the art performance on GLUE, SNLI Bowman2015ALA, SciTail
Khot et al. (2018), and ANLI Nie et al. (2020) NLU benchmarks. Most of these
algorithms employ adversarial training method to improve the robustness of
language model fine-tuing. SMART uses an adversarial methodology to encourage
models to be smooth within a neighborhoods of all the inputs; FreeLB optimizes
a direct adversarial loss through iterative gradient ascent steps; R3F removes
the adversarial nature of SMART and optimize the smoothness of the whole model
directly. Different from these methods, our proposed method does not adopt the
adversarial training strategy, we optimize the smoothness of each layer of
BERT directly and thus improve the stability of whole model.

##  3 Using Noise Stability as a Regularizer

One of the central issues in neural network training is to determine the
optimal degree of complexity for the model. A model which is too limited will
not sufficiently capture the structure in the data, while one which is too
complex will model the noise on the data (the phenomenon of over-fitting). In
either case, the performance on new data, that is the ability of the network
to generalize, will be poor. The problem can be regarded as one of finding the
optimal trade-off between the high bias of a model which is too inflexible and
the high variance of a model with too much freedom Geman et al. (1992); Bishop
(1995); Novak et al. (2018); Bishop (1991). To control the trade-off of bias
against variance of BERT models, we impose an explicit noise regularization
method.

###  3.1 Introduction of Our Method

Denoting the training set as $D$, we give the general form of optimization
objective for a BERT model $f(\cdot;\theta)$ with $L$ layers, as following:

$\theta^{*}=\mathop{\arg\min}_{\mathbf{\theta}}\underset{(x,y)\sim
D}{\mathbb{E}}[\mathcal{L}(f(x;\theta),y)+\hat{\mathcal{R}}(\theta)].$ (1)

To represent $\hat{\mathcal{R}}(\theta)$, we first define the injection
position as the _input_ of layer $b$ which is denoted as $x^{b}$. If the
regularization is operated at the _output_ of layer $r$, we can further denote
the function between layer $b$ and $r$ as $f^{b,r}$, satisfying that
$1<=b<=r<=L$. To implement the noise stability regularization, we inject a
Gaussian-like noise vector $\varepsilon$ to $x^{b}$ and get a neighborhood
$x^{b}+\varepsilon$. Specifically, each element $\varepsilon_{i}$ is
independently randomly sampled from a Gaussian distribution with the mean of
zero and the standard deviation of $\sigma$ as
$\varepsilon_{i}\sim\mathcal{N}(0,\sigma^{2})$. The probability density
function of the noise distribution can be written as
$p(\varepsilon_{i})=\frac{1}{\sqrt{2\pi}\sigma}e^{-\frac{\varepsilon_{i}^{2}}{2\sigma^{2}}}$.
Our goal is to minimize the discrepancy between their outputs over $f^{b,r}$
defined as $||f^{b,r}(x^{b}+\varepsilon)-f^{b,r}(x^{b})||^{2}$. In our
framework, we use a fixed position $b$ as the position of noise injection and
constrain the output distance on all layers following layer $b$. Denoting the
regularization weight corresponding to each $f^{b,r}$ as $\lambda^{b,r}$,
given a sample $(x,y)\sim D$, the regularization term is represented by the
following formulas:

$\hat{\mathcal{R}}(\theta)=\sum_{r=b}^{L}\lambda^{b,r}||f^{b,r}(x^{b}+\varepsilon)-f^{b,r}(x^{b})||^{2}.$
(2)

An overall algorithm is represented in Algorithm 1.

###  3.2 Theoretical Analysis

Regularzation is a kind of commonly used techniques to reduce the function
complexity and, as a result, to make the learned model generalize well on
unseen examples. In this part, we theoretically prove that the proposed LNSR
algorithm has the effects of encouraging the local Lipschitz continuity and
imposing a Tikhonov regularizer under different assumptions. For simplicity,
we omit the notations about the layer number in this part, denoting $f$ as the
target function and $x$ as the input of $f$ parameterized by $\theta$. Given a
sample $(x,y)\sim D$, we discuss the general form of the noise stability
defined as following:

$\mathcal{R}(\theta)=\underset{\varepsilon}{E}\\{\|f(x+\varepsilon)-f(x)\|^{2}\\}.$
(3)

Lipschitz continuity. The Lipschitz property reflects the degree of smoothness
for a function. Recent theoretical studies on deep learning has revealed the
close connection between Lipschitz property and generalization Bartlett et al.
(2017); Neyshabur et al. (2017).

Given a sampled $\varepsilon$, minimizing $\|f(x+\varepsilon)-f(x)\|^{2}$ is
equivalent to minimizing:

$\frac{\|f(x+\varepsilon)-f(x)\|^{2}}{\|x+\varepsilon-x\|^{2}}.$ (4)

Thus the noise stability regularization can be regarded as minimizing the
Lipschitz constant in a local region around the input $x$.

Tikhonov regularizer. The Tikhonov regularizer Willoughby (1979) involves
constraints on the derivatives of the objective function with respect to
different orders. For the simplest first-order case, it can be regarded as
imposing robustness and shaping a flatter loss surface at input, which makes
the learned function smoother.

Assuming that the magnitude of $\varepsilon$ is small, we can expand the first
term as a Taylor approximation as:

$f(x+\varepsilon)=f(x)+\varepsilon\cdot
J_{f}(x)+\frac{1}{2}\varepsilon^{T}\cdot
H_{f}(x)\cdot\varepsilon+O(\varepsilon^{3}),$ (5)

where $J_{f}(x)$ and $H_{f}(x)$ refer to the Jacobian and Hessian of $f$ with
respect to the input $x$ respectively.

Ignoring the higher order term $O(\mathbf{\varepsilon}^{3})$ and denoting
$f_{k}$ as the k-th output of the function $f$, we can rewrite the regularizer
by substituting Eq. 5 in Eq. 3 as:

$\displaystyle\mathcal{R}(\theta)$
$\displaystyle\\!=\\!\underset{\mathbf{\varepsilon}}{E}\\{\|\mathbf{\varepsilon}\cdot
J_{f}(x)+\frac{1}{2}\mathbf{\varepsilon}^{T}\cdot
H_{f}(x)\cdot\mathbf{\varepsilon}\|^{2}\\}$ (6)
$\displaystyle=\\!\int\|\mathbf{\varepsilon}\\!\cdot\\!J_{f}(x)+\frac{1}{2}\mathbf{\varepsilon}^{T}\\!\cdot\\!H_{f}(x)\\!\cdot\\!\mathbf{\varepsilon}\|^{2}p(\mathbf{\varepsilon})d\mathbf{\varepsilon}$
$\displaystyle=\\!\sum_{k}\\!\int\\!\|\mathbf{\varepsilon}\\!\cdot\\!J_{f_{k}}(x)\\!+\\!\frac{1}{2}\\!\mathbf{\varepsilon}^{T}\\!\cdot\\!H_{f_{k}}(x)\\!\cdot\\!\mathbf{\varepsilon}\|^{2}p(\mathbf{\varepsilon})d\mathbf{\varepsilon}.$

We define the input vector $x$ as $(x_{1},x_{2},......,x_{d_{1}})$ and noise
vector $\mathbf{\varepsilon}$ as
$\mathbf{\varepsilon}=(\varepsilon_{1},\varepsilon_{2},......,\varepsilon_{d_{1}})$.
Assuming that distributions of the noise and the input are irrelevant, and the
derivative of $f$ with respect to different elements of the input vector is
independent with each other, we expand the second order term corresponding to
the Jacobian as:

$\displaystyle\Omega_{J}(f_{k})$
$\displaystyle=\int\|\mathbf{\varepsilon}\cdot
J_{f_{k}}(x)\|^{2}p(\mathbf{\varepsilon})d\mathbf{\varepsilon}$ (7)
$\displaystyle=\int\sum_{i}(\varepsilon_{i}\frac{\partial f_{k}}{\partial
x_{i}})^{2}p(\mathbf{\varepsilon})d\mathbf{\varepsilon}$
$\displaystyle=\sum_{i}(\frac{\partial f_{k}}{\partial
x_{i}})^{2}\int\varepsilon_{i}^{2}p(\mathbf{\varepsilon})d\mathbf{\varepsilon}$
$\displaystyle=\sigma^{2}\|J_{f_{k}}(x)\|^{2}.$

According to the characteristics of the Gaussian distribution, we also have

$\int\varepsilon_{i}\varepsilon_{j}p(\mathbf{\varepsilon})d\mathbf{\varepsilon}=0\text{\
\ \ for any\ }i\neq j.$ (8)

Thus, we can rewrite the second order term corresponding to the Hessian in Eq.
6 as:

$\displaystyle\Omega_{H}(f_{k})$
$\displaystyle=\int\|\frac{1}{2}\mathbf{\varepsilon}^{T}\cdot
H_{f_{k}}(x)\cdot\mathbf{\varepsilon}\|^{2}p(\mathbf{\varepsilon})d\mathbf{\varepsilon}$
(9)
$\displaystyle=\int\frac{1}{4}\sum_{i}(\varepsilon_{i}^{2}\frac{\partial^{2}f_{k}}{\partial
x_{i}^{2}})^{2}p(\mathbf{\varepsilon})d\mathbf{\varepsilon}$
$\displaystyle=\frac{1}{4}\sum_{i}(\frac{\partial^{2}f_{k}}{\partial
x_{i}^{2}})^{2}\int\varepsilon_{i}^{4}p(\mathbf{\varepsilon})d\mathbf{\varepsilon}$
$\displaystyle=C\|\mathrm{Tr}(H_{f_{k}}(x)^{T}H_{f_{k}}(x))\|^{2},$

Where $C$ is a constant independent of the input $x$. The third term generated
from the expansion of Eq. 6 is zero as we have
$\int\mathbf{\varepsilon}^{3}p(\mathbf{\varepsilon})d\mathbf{\varepsilon}=0$.
Thus we get

$\mathcal{R}(\theta)=\sum_{k}\\{\Omega_{J}(f_{k})+\Omega_{H}(f_{k})\\}.$ (10)

Considering that the input and output of the function $f$ are both scalar
variable, the Tikhonov regularization Willoughby (1979) takes the general form
as:

$\mathcal{R_{T}}(\theta)=\sum_{r}\int h_{r}(x)(\frac{\partial^{r}f}{\partial
x^{r}})^{2}dx.$ (11)

Eq. 10 shows that our proposed regularizer ensuring the noise stability is
equivalent to a special case of the Tikhonov regularizer, where we involve the
first and second order derivatives of the objective function $f$.

An alternative for improving the robustness is to directly add noise to the
input, without explicitly constraining the output stability. Rifai et al.
(2011) has derived that adding noise to the input has the effect of penalizing
both the $L_{2}$-norm of the Jacobian $\|J_{f}(x)\|^{2}$ and the trace of the
Hessian $\mathrm{Tr}(H_{f}(x))$, whereas the Hessian term is not constrained
to be positive. While the regularizer brought by our proposed LNSR is
guaranteed to be positive by involving the sum of squares of the first and
second order derivatives. Moreover, our work relaxes the assumption of MSE
regression loss required by Rifai et al. (2011). By imposing the explicit
constraint of noise stability on middle layer representations, we extend the
theoretical understanding of noise stability into deep learning algorithms.

Algorithm 1 Layer-wise Noise Stability Regularization (LNSR)

0: Training set $D$, perturbation bound $\delta$, learning rate $\tau$, number
of layers $L$, number of training epochs $N$, function $f$ and its
corresponding parameters $\theta$, the position of noise injection $b$, and
regularization weights for each layer $\\{\lambda^{b},...,\lambda^{L}\\}$.

1: Initialize $\theta$

2: for epoch=$1,2,...,N$ do

3: for minibatch $B\sim D$ do

4: $\hat{\mathcal{R}}\leftarrow 0$

5: for each $x\in B$ do

6: $\varepsilon\sim\mathcal{N}(0,\sigma^{2})$

7: $\tilde{x}\leftarrow x+\varepsilon$

8: forward pass given $x$ and $\tilde{x}$ as inputs

9: for $r=b,b+1,...,L$ do

10:
$\hat{\mathcal{R}}\leftarrow\hat{\mathcal{R}}+\lambda^{b,r}||f^{b,r}(x)-f^{b,r}(\tilde{x})||^{2}$

11: end for

12: end for

13:
$g\leftarrow\frac{1}{|B|}\sum_{(x,y)}\nabla[\mathcal{L}(f(x;\theta),y)+\hat{\mathcal{R}}]$

14: $\theta\leftarrow\theta-\tau g$

15: end for

16: end for

16: $\theta$

| RTE | MRPC | CoLA | STS-B
---|---|---|---|---
mean | std | max | mean | std | max | mean | std | max | mean | std | max
FT Devlin et al. (2019) | $70.13$ | $1.84$ | $72.56$ | $87.57$ | $0.92$ | $89.16$ | $60.54$ | $1.49$ | $62.59$ | $89.38$ | $0.53$ | $90.23$
$\text{L}^{2}\text{-SP}$ Li et al. (2018) | $70.58$ | $\bf{1.29}$ | $73.28$ | $87.74$ | $0.86$ | $88.95$ | $60.19$ | $1.42$ | $63.89$ | $89.25$ | $0.62$ | $90.14$
MixoutLee et al. (2020) | $71.35$ | $1.66$ | $74.36$ | $87.63$ | $0.62$ | $88.91$ | $63.12$ | $1.68$ | $65.12$ | $89.58$ | $0.35$ | $90.11$
SMARTJiang et al. (2020) | $72.23$ | $2.41$ | $75.45$ | $87.86$ | $0.63$ | $89.09$ | $63.16$ | $1.17$ | $65.21$ | $90.11$ | $0.33$ | $90.83$
LNSR(ours) | $\bf{73.31}$ | $1.55$ | $\bf{76.17}$ | $\bf{88.50}$ | $\bf{0.56}$ | $\bf{90.02}$ | $\bf{63.35}$ | $\bf{1.05}$ | $\bf{65.99}$ | $\bf{90.23}$ | $\bf{0.31}$ | $\bf{90.97}$

Table 1: The mean, standard deviation, and maximum performance on the
development set of RTE, MRPC, CoLA, and STS-B tsak across 25 random seeds when
fine-tuning the BERT-Large model with various regularization methods. FT
refers to the standard BERT fine-tuning. Standard deviation: lower is better.

Figure 2: Performance distribution box plot of each model on the four tasks
from 25 random seeds.

##  4 Experiments

In this section, we experimentally demostrate the effectiveness of LNSR method
on text classification tasks over other regularization methods, and confirm
that the insensitivity to noise promotes the generalizability and stability of
BERT.

###  4.1 Data

We conduct experiments on four few-sample (less than 10k training samples)
text classification tasks of GLUE 111https://gluebenchmark.com/, the datasets
are described below and summarized in Appendix A Table 4.

Corpus of Linguistic Acceptability (CoLA Warstadt et al. (2019)) consists of
English acceptability judgments drawn from books and journal articles on
linguistic theory. Each example is a sequence of words annotated with whether
it is a grammatical English sentence. This is a binary classification task and
Matthews correlation coefficient (MCC) Matthews (1975) is used to evaluate the
performance.

Microsoft Research Paraphrase Corpus (MRPC Dolan and Brockett (2005)) is a
corpus of sentence pairs with human annotations for whether the sentences in
the pair are semantically equivalent. The evaluation metrics is the average of
F1 and Accuracy.

Recognizing Textual Entailment (RTE Wang et al. (2018)) Dagan et al. (2005)
Bar-Haim et al. (2006) Giampiccolo et al. (2007) is a corpus of textual
entailment, and each example is a sentence pair annotated whether the first
entails the second. The evaluation metrics is Accuracy.

Semantic Textual Similarity Benchmark (STS-B Cer et al. (2017))is a regression
task. Each example is a sentence pair and is human-annotated with a similarity
score from 1 to 5; the task is to predict these scores. The evaluation metrics
is the average of Pearson and Spearman correlation coefficients.

###  4.2 Baseline Models

We use BERT Devlin et al. (2019), a large-scale bidirectional pre-trained
language model as the base model in all experiments. We adopt pytorch edition
implemented by Wolf et al. (2019).

Fine-tuning. We use the standard BERT fine-tuning method described in Devlin
et al. (2019).

$\text{L}^{2}$-SP Li et al. (2018) is a regularization scheme that explicitly
promotes the similarity of the final solution with the initial model. It is
usually used for preventing pre-trained models from catastrophic forgetting.
We adopt the form of
$\Omega(w)=\frac{\alpha}{2}||w_{s}-w_{s}^{0}||+\frac{\beta}{2}||w_{\bar{s}}||$.

Mixout Lee et al. (2020) is a stochastic regularization technique motivated by
Dropout Srivastava2014DropoutAS and DropConnect Wan et al. (2013). At each
training iteration, each model parameter is replaced with its pre-trained
value with probability $p$. The goal is to improve the generalizability of
pre-trained language models.

SMART Jiang et al. (2020) imposes an smoothness regularizer inducing an
adversarial manner to control the model complexity at the fine-tuning stage.
It also employs a class of Bregman proximal point optimization methods to
prevent the model from aggressively updating during fine-tuning.

###  4.3 Experimental Setup

Our model is implemented using Pytorch based on Transformers framework
222https://huggingface.co/transformers/index.html. Specifically, we use the
learning setup and hyperparameters recommended by Devlin et al. (2019). We use
Huggingface edition Adam Kingma and Ba (2015) optimizer (without bias
correction) with learning rate of $2\times
10^{-5}$,$\beta_{1}=0.9,\beta_{2}=0.999$, and warmup over the first 10% steps
of the total steps. We fine-tune the entire model (340 million parameters), of
which the vast majority start as pre-trained weights (BERT-Large-Uncased) and
the classification layer (2048 parameters). Weights of the classification
layer are initialized with $\mathcal{N}(0,0.02^{2})$. We train with a batch
size of 32 for 3 epochs. More details of our experimental setup are described
in Appendix A.

| RTE | MRPC | CoLA | STS-B
---|---|---|---|---
mean | std | max | mean | std | max | mean | std | max | mean | std | max
FT | $70.13$ | $1.84$ | $72.56$ | $87.57$ | $0.92$ | $89.16$ | $61.56$ | $1.34$ | $64.10$ | $89.38$ | $0.53$ | $90.23$
FT (4 Epochs) | $70.69$ | $1.97$ | $73.65$ | $88.15$ | $0.65$ | $89.21$ | $60.69$ | $1.24$ | $62.09$ | $89.29$ | $0.56$ | $90.12$
FT+Noise | $70.62$ | $1.56$ | $72.93$ | $87.95$ | $0.83$ | $89.33$ | $60.18$ | $1.58$ | $62.59$ | $89.34$ | $0.51$ | $90.11$
LNSR(ours) | $\bf{73.31}$ | $\bf{1.55}$ | $\bf{76.17}$ | $\bf{88.50}$ | $\bf{0.56}$ | $\bf{90.02}$ | $\bf{63.35}$ | $\bf{1.05}$ | $\bf{65.99}$ | $\bf{90.23}$ | $\bf{0.31}$ | $\bf{90.97}$

Table 2: Ablation study of LNSR on each task, we report the mean evaluation
scores and standard deviation and max value across 25 random seeds. FT refers
to standard BERT fine-tuning.

| RTE | MRPC | CoLA | STS-B
---|---|---|---|---
train / eval / gap | train / eval / gap | train / eval / gap | train / eval / gap
FT Devlin et al. (2019) |  $\bf{95.89}$/$70.13$/$25.76$ |  $96.57$/$87.57$/$9.00$ |  $\bf{97.71}$/$61.56$/$36.25$ |  $98.31$/$89.38$/ $8.93$
LNSR(ours) |  $90.72$/$\bf{73.31}$/$\bf{17.41}$ |  $\bf{96.68}$/$\bf{88.50}$/$\bf{8.18}$ |  $93.44$/$\bf{63.35}$/$\bf{30.09}$ |  $\bf{98.45}$/$\bf{90.23}$/ $\bf{8.22}$

Table 3: Comparison of the generalizability performance of different models.
We report the mean training Acc and evaluation Acc and the generalizability
gap (training Acc - evaluation Acc) of each model across 20 random seeds.

###  4.4 Overall Performance

Table 1 shows the results of all the models on selected GLUE datasets. We
train each dataset over 25 random seeds. To implement our LNSR, we uniformly
inject noise at the first layer on BERT-large for the comparison with baseline
models. As we can see from the table, our model outperforms all the baseline
models in mean and max values, which indicates the stronger generalizability
of our model against other baseline models. The p-values between the accuracy
distributions of standard BERT fine-tuning and our model are calculated to
verify whether the improvements are significant. We obtain very small p-values
in all tasks: RTE: $9.7\times 10^{-7}$, MRPC: $2.3\times 10^{-4}$, CoLA:
$4.7\times 10^{-8}$, STS-2: $3.3\times 10^{-8}$.

Standard deviation is an indicator of the stability of models’ performance and
higher std means more sensitive to random seeds. Our model shows a lower
standard deviation on each task, which means our model is less sensitive to
random seeds than other models. Figure 2 presents a clearer illustration. To
sum up, our proposed method can effectively improve the performance and
stability of fine-tuning BERT.

##  5 Analysis

###  5.1 Ablation Study

To verify the effectiveness of our proposed LNSR model, we conduct several
ablation experiments including fine-tuning with more training epochs and noise
perturbation without regularization (we inject noise directly to the output of
a specific layer, and then use the perturbed representation to conduct
propagation and then calculate loss, this process is similar to a vector-space
represent augmentation). The results are shown in Table 2. We observe that
benefit obtained by longer training is limited. Similarly, fine-tuning with
noise perturbation only achieves slightly better results on two of these
tasks, showing that simply adding noise without an explicit restriction on
outputs may not be sufficient to obtain good generalizability. While BERT
models with LNSR perform better on each task. This verifies our claim that
LNSR can promote the stability of BERT fine-tuning and meanwhile improve the
generalizability of the BERT model.

Figure 3: The top-1 accuracy (top) and loss (bottom) curve for fine-tuning
BERT with and w/o LNSR.

###  5.2 Effects on the Generalizability of Models

We verify the effects of our proposed method on the generalizability of BERT
models in two ways – generalization gap and models’ performance on fewer
training samples. Due to the limited data and the extremely high complexity of
BERT model, bad fine-tuning start point makes the adapted model overfit the
training data and does not generalize well to unseen data. Generalizability of
models can be intuitively reflected by generalization gap and models’
performance on fewer training samples.

Table 3 shows the mean training Acc, mean evaluation Acc and generalization
gap of different models on each task. As we can see from the table, fine-
tuning with LNSR can effectively narrow the generalization gap, and achieve
higher evaluation score. The effect of narrowing generalization gap is also
reflected in Figure 3 where we can see the higher evaluation accuracy and
lower evaluation loss.

We sample subsets from the two relatively larger datasets CoLA (8.5k training
samples) and STS-B (7k training samples) with the sampling ratio of 0.15, 0.3
and 0.5. As is shown in Figure 4, fine-tuning with LNSR shows clear advantage
on fewer training samples, suggesting LNSR can effectively promote the model’s
generalizability.

Figure 4: Mean evaluation score comparison of fine-tuning BERT with and w/o
LNSR on fewer training samples of the CoLA and STS-B tasks. The mean values
are calculated over 20 random seeds.

###  5.3 Sensitivity to the Position of Noise Injection

We briefly discuss about the sensitivity to the position of noise injection as
it is a pre-determined hyperparameter of our method. As is shown in Figure 5
in Appendix A, we observe that the performance of LNSR does not fluctuate much
as the position of noise injection changes. All injection positions bring
significant improvements over vanilla fine-tuning. Note that, with LNSR, noise
injection to the lower layers usually leads to relatively higher accuracy and
stability, implying that LNSR may be more effective when it affects both the
lower and higher layers of the network.

###  5.4 Relationship to Previous Noise-based Approaches

Our method is related to SMART Jiang et al. (2020), FreeLB Zhu et al. (2020a)
and R3F Aghajanyan et al. (2020). As is mentioned before, most of these
approaches employ adversarial training strategies to improve the robustness of
BERT fine-tuing. SMART solves supremum by using an adversarial methodology to
achieve the largest KL divergence with an $\epsilon$-ball, FreeLB optimizes a
direct adversarial loss
$\mathcal{L}_{FreeLB}(\theta)=\text{sup}_{\Delta\theta:|\Delta\theta|\leq\epsilon}\mathcal{L}(\theta+\Delta\theta)$
through iterative gradient ascent steps, while R3F removes the adversarial
nature of SMART and optimize the smoothness of the whole model directly.

Compared with this sort of adversarial based algorithms, our method is easier
to implement and provides a relatively rigorous theoretical guarantee. The
design of layer-wise regularization is sensible that it exploits the
characteristics of hierarchical representations in modern deep neural
networks. Studies in knowledge distillation have shown similar experience that
imitating through middle layer representations Adriana et al. (2015);
Zagoruyko and Komodakis (2016) performs better than aligning the final outputs
Hinton et al. (2015). Moreover, LNSR allows us to use different regularization
weights for different layers (we use fixed weight 1 on all layers in this
paper). We will leave the exploitation in future work.

##  6 Conclusion

In this paper, we propose the Layer-wise Noise Stability Regularization (LNSR)
as a lightweight and effective method to improve the generalizability and
stability when fine-tuning BERT on few training samples. Our proposed LNSR
method is a general technique that improves model output stability while
maintaining or improving the original performance. Furthermore, we provide a
theoretically analysis of the relationship of our model to the Lipschitz
continuity and Tikhonov regularizer. Extensive empirical results show that our
proposed method can effectively improve the generalizability and stability of
the BERT model.

##  7 Acknowledgements

Hang Hua would like to thank Jeffries for supporting his research.

## References

  * Adriana et al. (2015) Romero Adriana, Ballas Nicolas, K Samira Ebrahimi, Chassang Antoine, Gatta Carlo, and B Yoshua. 2015.  Fitnets: Hints for thin deep nets.  _Proc. ICLR_.
  * Aghajanyan et al. (2020) Armen Aghajanyan, Akshat Shrivastava, Anchit Gupta, Naman Goyal, Luke Zettlemoyer, and Sonal Gupta. 2020.  Better fine-tuning by reducing representational collapse.  _ArXiv_ , abs/2008.03156.
  * Arora et al. (2018) Sanjeev Arora, R. Ge, Behnam Neyshabur, and Yi Zhang. 2018.  Stronger generalization bounds for deep nets via a compression approach.  _ArXiv_ , abs/1802.05296.
  * Bar-Haim et al. (2006) Roy Bar-Haim, I. Dagan, B. Dolan, Lisa Ferro, Danilo Giampiccolo, and B. Magnini. 2006.  The second pascal recognising textual entailment challenge.
  * Bartlett et al. (2017) Peter Bartlett, Dylan J Foster, and Matus Telgarsky. 2017.  Spectrally-normalized margin bounds for neural networks.  _arXiv preprint arXiv:1706.08498_.
  * Bishop (1991) Chris Bishop. 1991.  Improving the generalization properties of radial basis function neural networks.  _Neural computation_ , 3(4):579–588.
  * Bishop (1995) Chris M Bishop. 1995.  Training with noise is equivalent to tikhonov regularization.  _Neural computation_ , 7(1):108–116.
  * Cer et al. (2017) Daniel Matthew Cer, Mona T. Diab, Eneko Agirre, I. Lopez-Gazpio, and Lucia Specia. 2017.  Semeval-2017 task 1: Semantic textual similarity multilingual and crosslingual focused evaluation.  _ArXiv_ , abs/1708.00055.
  * Dagan et al. (2005) I. Dagan, Oren Glickman, and B. Magnini. 2005.  The pascal recognising textual entailment challenge.  In _MLCW_.
  * Dai and Le (2015) Andrew M. Dai and Quoc V. Le. 2015.  Semi-supervised sequence learning.  In _NIPS_.
  * Devlin et al. (2019) J. Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019.  Bert: Pre-training of deep bidirectional transformers for language understanding.  In _NAACL-HLT_.
  * Dodge et al. (2020) Jesse Dodge, Gabriel Ilharco, Roy Schwartz, Ali Farhadi, Hannaneh Hajishirzi, and Noah A. Smith. 2020.  Fine-tuning pretrained language models: Weight initializations, data orders, and early stopping.  _ArXiv_ , abs/2002.06305.
  * Dolan and Brockett (2005) W. Dolan and Chris Brockett. 2005.  Automatically constructing a corpus of sentential paraphrases.  In _IWP@IJCNLP_.
  * Erhan et al. (2010) D. Erhan, Aaron C. Courville, Yoshua Bengio, and Pascal Vincent. 2010.  Why does unsupervised pre-training help deep learning?  _J. Mach. Learn. Res._ , 11:625–660.
  * Erhan et al. (2009) D. Erhan, Pierre-Antoine Manzagol, Yoshua Bengio, S. Bengio, and P. Vincent. 2009\.  The difficulty of training deep architectures and the effect of unsupervised pre-training.  In _AISTATS_.
  * Geman et al. (1992) Stuart Geman, Elie Bienenstock, and René Doursat. 1992.  Neural networks and the bias/variance dilemma.  _Neural computation_ , 4(1):1–58.
  * Giampiccolo et al. (2007) Danilo Giampiccolo, B. Magnini, I. Dagan, and W. Dolan. 2007.  The third pascal recognizing textual entailment challenge.  In _ACL-PASCAL@ACL_.
  * Gunel et al. (2020) Beliz Gunel, Jingfei Du, Alexis Conneau, and Ves Stoyanov. 2020.  Supervised contrastive learning for pre-trained language model fine-tuning.  _ArXiv_ , abs/2011.01403.
  * Guu et al. (2020) Kelvin Guu, Kenton Lee, Zora Tung, Panupong Pasupat, and Ming-Wei Chang. 2020.  Realm: Retrieval-augmented language model pre-training.  _ArXiv_ , abs/2002.08909.
  * Hinton et al. (2015) Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. 2015.  Distilling the knowledge in a neural network.  _arXiv preprint arXiv:1503.02531_.
  * Jiang et al. (2020) Haoming Jiang, Pengcheng He, Weizhu Chen, Xiaodong Liu, Jianfeng Gao, and Tuo Zhao. 2020.  Smart: Robust and efficient fine-tuning for pre-trained natural language models through principled regularized optimization.  In _ACL_.
  * Khot et al. (2018) Tushar Khot, A. Sabharwal, and Peter Clark. 2018.  Scitail: A textual entailment dataset from science question answering.  In _AAAI_.
  * Kingma and Ba (2015) Diederik P. Kingma and Jimmy Ba. 2015.  Adam: A method for stochastic optimization.  _CoRR_ , abs/1412.6980.
  * Lample and Conneau (2019) Guillaume Lample and Alexis Conneau. 2019.  Cross-lingual language model pretraining.  _ArXiv_ , abs/1901.07291.
  * Lan et al. (2020) Zhenzhong Lan, Mingda Chen, Sebastian Goodman, Kevin Gimpel, Piyush Sharma, and Radu Soricut. 2020.  Albert: A lite bert for self-supervised learning of language representations.  _ArXiv_ , abs/1909.11942.
  * Lee et al. (2020) Cheolhyoung Lee, Kyunghyun Cho, and Wanmo Kang. 2020.  Mixout: Effective regularization to finetune large-scale pretrained language models.  _ArXiv_ , abs/1909.11299.
  * Li et al. (2018) Xuhong Li, Y. Grandvalet, and F. Davoine. 2018.  Explicit inductive bias for transfer learning with convolutional networks.  In _ICML_.
  * Liu et al. (2019) Y. Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, M. Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019.  Roberta: A robustly optimized bert pretraining approach.  _ArXiv_ , abs/1907.11692.
  * Liu (2019) Yang Liu. 2019.  Fine-tune bert for extractive summarization.  _ArXiv_ , abs/1903.10318.
  * Loshchilov and Hutter (2019) I. Loshchilov and F. Hutter. 2019.  Decoupled weight decay regularization.  In _ICLR_.
  * Matthews (1975) Brian W Matthews. 1975.  Comparison of the predicted and observed secondary structure of t4 phage lysozyme.  _Biochimica et Biophysica Acta (BBA)-Protein Structure_ , 405(2):442–451.
  * Mikolov et al. (2013) Tomas Mikolov, Ilya Sutskever, Kai Chen, G. S. Corrado, and J. Dean. 2013.  Distributed representations of words and phrases and their compositionality.  _ArXiv_ , abs/1310.4546.
  * Mosbach et al. (2020) Marius Mosbach, Maksym Andriushchenko, and D. Klakow. 2020.  On the stability of fine-tuning bert: Misconceptions, explanations, and strong baselines.  _ArXiv_ , abs/2006.04884.
  * Neyshabur et al. (2017) Behnam Neyshabur, Srinadh Bhojanapalli, David McAllester, and Nathan Srebro. 2017\.  Exploring generalization in deep learning.  _arXiv preprint arXiv:1706.08947_.
  * Nie et al. (2020) Yixin Nie, Adina Williams, Emily Dinan, Mohit Bansal, Jason Weston, and Douwe Kiela. 2020.  Adversarial NLI: A new benchmark for natural language understanding.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_. Association for Computational Linguistics.
  * Novak et al. (2018) Roman Novak, Y. Bahri, D. Abolafia, Jeffrey Pennington, and Jascha Sohl-Dickstein. 2018.  Sensitivity and generalization in neural networks: an empirical study.  _ArXiv_ , abs/1802.08760.
  * Pennington et al. (2014) Jeffrey Pennington, R. Socher, and Christopher D. Manning. 2014.  Glove: Global vectors for word representation.  In _EMNLP_.
  * Peters et al. (2018) Matthew E. Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018.  Deep contextualized word representations.  _ArXiv_ , abs/1802.05365.
  * Radford (2018) A. Radford. 2018.  Improving language understanding by generative pre-training.
  * Radford et al. (2019) A. Radford, Jeffrey Wu, R. Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019\.  Language models are unsupervised multitask learners.
  * Rifai et al. (2011) Salah Rifai, Xavier Glorot, Yoshua Bengio, and Pascal Vincent. 2011.  Adding noise to the input of a model trained with a regularized objective.  _arXiv preprint arXiv:1104.3250_.
  * Roth et al. (2019) K. Roth, Yannic Kilcher, and Thomas Hofmann. 2019.  Adversarial training generalizes data-dependent spectral norm regularization.  _ArXiv_ , abs/1906.01527.
  * Sanyal et al. (2020) Amartya Sanyal, P. Torr, and Puneet K. Dokania. 2020.  Stable rank normalization for improved generalization in neural networks and gans.  _ArXiv_ , abs/1906.04659.
  * Vaswani et al. (2017) Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, L. Kaiser, and Illia Polosukhin. 2017.  Attention is all you need.  In _NIPS_.
  * Wadden et al. (2019) David Wadden, Ulme Wennberg, Yi Luan, and Hannaneh Hajishirzi. 2019.  Entity, relation, and event extraction with contextualized span representations.  In _EMNLP/IJCNLP_.
  * Wan et al. (2013) Li Wan, Matthew D. Zeiler, Sixin Zhang, Y. LeCun, and R. Fergus. 2013.  Regularization of neural networks using dropconnect.  In _ICML_.
  * Wang et al. (2019) Alex Wang, Yada Pruksachatkun, Nikita Nangia, Amanpreet Singh, Julian Michael, F. Hill, Omer Levy, and Samuel R. Bowman. 2019.  Superglue: A stickier benchmark for general-purpose language understanding systems.  _ArXiv_ , abs/1905.00537.
  * Wang et al. (2018) Alex Wang, Amanpreet Singh, Julian Michael, F. Hill, Omer Levy, and Samuel R. Bowman. 2018.  Glue: A multi-task benchmark and analysis platform for natural language understanding.  _ArXiv_ , abs/1804.07461.
  * Warstadt et al. (2019) Alex Warstadt, Amanpreet Singh, and Samuel R. Bowman. 2019.  Neural network acceptability judgments.  _Transactions of the Association for Computational Linguistics_ , 7:625–641.
  * Willoughby (1979) Ralph A Willoughby. 1979.  Solutions of ill-posed problems (an tikhonov and vy arsenin).  _SIAM Review_ , 21(2):266.
  * Wolf et al. (2019) Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lhoest, and Alexander M. Rush. 2019.  Huggingface’s transformers: State-of-the-art natural language processing.  _ArXiv_ , abs/1910.03771.
  * Yang et al. (2019) Z. Yang, Zihang Dai, Yiming Yang, J. Carbonell, R. Salakhutdinov, and Quoc V. Le. 2019.  Xlnet: Generalized autoregressive pretraining for language understanding.  In _NeurIPS_.
  * Yoshida and Miyato (2017) Y. Yoshida and Takeru Miyato. 2017.  Spectral norm regularization for improving the generalizability of deep learning.  _ArXiv_ , abs/1705.10941.
  * Yosinski et al. (2014) J. Yosinski, J. Clune, Yoshua Bengio, and Hod Lipson. 2014.  How transferable are features in deep neural networks?  In _NIPS_.
  * Zagoruyko and Komodakis (2016) Sergey Zagoruyko and Nikos Komodakis. 2016.  Paying more attention to attention: Improving the performance of convolutional neural networks via attention transfer.  _arXiv preprint arXiv:1612.03928_.
  * Zhang et al. (2020) Tianyi Zhang, Felix Wu, Arzoo Katiyar, Kilian Q. Weinberger, and Yoav Artzi. 2020\.  Revisiting few-sample bert fine-tuning.  _ArXiv_ , abs/2006.05987.
  * Zhu et al. (2020a) C. Zhu, Y. Cheng, Zhe Gan, S. Sun, T. Goldstein, and Jing jing Liu. 2020a.  Freelb: Enhanced adversarial training for natural language understanding.  _arXiv: Computation and Language_.
  * Zhu et al. (2020b) Jinhua Zhu, Yingce Xia, Lijun Wu, Di He, Tao Qin, W. Zhou, H. Li, and T. Liu. 2020b.  Incorporating bert into neural machine translation.  _ArXiv_ , abs/2002.06823.

##  Appendix A Experimental Details

The model we use for experiments in section 4 is the standard BERT large model
with 24 layers staked Transformers Vaswani et al. (2017) encoder, 1024 hidden
size, and 16 self-attention heads. We initialize the pre-trained part of the
model with BERT-Large-Uncased-Whole-Word-Masking weight. The final layer is a
classification layer with 2048 parameters which contains $0.0006\%$ of the
total number of parameters in the model. We initialize the last layer with
$\mathcal{N}(0,0.02^{2})$ and each bias is 0. For the position of noise
injection, we uniformly chose the first layer as the noise regularization
start point. In the sensitivity to the position of noise injection analysis
section, we also try injecting noise from the different layers as is shown in
Figure 5. As for the baseline model Mixout, we use the code from the Github
repository https://github.com/bloodwass/mixout.git. The other baseline models
are implemented by ourselves.

Table 4 summarizes dataset statistics used in this work. We use the standard
GLUE benchmark datasets downloaded from https://gluebenchmark.com/tasks.

##  Appendix B Other Experimental Reports

We also report the maximum value we get during fine-tuning BERT with our
proposed LNSR regularizer among a large number of random seeds and several
noise injection position, since the maximum value can also reflect the ability
of the learning algorithm to reach an optimal point. The results are shown in
Table 5, and we can see that on some tasks, fine-tuning BERT with LNSR is even
competitive with fine-tuning state-of-the-art models which adopt more powerful
modern architectures and pre-training strategies.

|  | RTE | MRPC | CoLA | STS-B
---|---|---|---|---|---
Task | NLI | Paraphrase | Acceptability | Similarity
| Metrics | Accuracy | $\frac{\text{Accuracy+F1}}{2}$ | Matthews corr | Pearson/Spearman corr
|  $\\#$ of labels | 2 | 2 | 2 | 1
|  $\\#$ of training samples | 2.5k | 3.7k | 8.6k | 7k
|  $\\#$ of validation samples | 276 | 408 | 1k | 1.5k
|  $\\#$ of test samples | 3k | 1.7k | 1k | 1.4k
Table 4: The summarization of the datasets used in this work.

|  | RTE | MRPC | CoLA | STS-B
---|---|---|---|---|---
BERT Devlin et al. (2019) | 70.4 | 88.0 | 60.6 | 90.0
| BERT Phang2018SentenceEO | 70.0 | 90.7 | 62.1 | 90.9
| LNSR (ours) | $\bf{79.1}$ | $90.4$ | $\bf{68.1}$ | $\bf{91.0}$
| XLNet Yang et al. (2019) | $83.8$ | $89.2$ | $63.6$ | $91.8$
| RoBERTa Liu et al. (2019) | $86.6$ | $90.9$ | $68.0$ | $92.4$
| ALBERT Lan et al. (2020) | $89.2$ | $90.9$ | $71.4$ | $93.0$

Table 5: We report the maximum value we get when fine-tuning the LNSR model on
different noise injection position and random seeds on the four tasks. On some
tasks, BERT (standard BERT-large-uncased Devlin et al. (2019)) with LNSR even
become competitive with some newly proposed powerful models (bottom rows)

.

Figure 5: Performance distribution box plot of each model on the four tasks
across 25 random seeds.


# Interface characteristics in an $\alpha$+$\beta$ titanium alloy

Abigail K. Ackerman  Corresponding author<EMAIL_ADDRESS>Department of Materials, Royal School of Mines, Imperial College London,
Prince Consort Road, London, SW7 2BP, UK  Vassili A. Vorontsov  Department of
Materials, Royal School of Mines, Imperial College London, Prince Consort
Road, London, SW7 2BP, UK  Ioannis Bantounas  Department of Materials, Royal
School of Mines, Imperial College London, Prince Consort Road, London, SW7
2BP, UK  Yufeng Zheng  Center for the Accelerated Maturation of Materials and
Department of Materials Science and Engineering, The Ohio State University,
Columbus, OH 43212, USA  Department of Chemical and Materials Engineering,
University of Nevada Reno, Reno, NV 89557, USA  Yanhong Chang  Max-Planck-
Institut für Eisenforschung GmbH, Max-Planck-Str. 1, 40237 Düsseldorf, Germany
Thomas McAuliffe  Department of Materials, Royal School of Mines, Imperial
College London, Prince Consort Road, London, SW7 2BP, UK  William A. Clark
Center for the Accelerated Maturation of Materials and Department of Materials
Science and Engineering, The Ohio State University, Columbus, OH 43212, USA
Hamish L. Fraser  Center for the Accelerated Maturation of Materials and
Department of Materials Science and Engineering, The Ohio State University,
Columbus, OH 43212, USA  Baptiste Gault  Max-Planck-Institut für
Eisenforschung GmbH, Max-Planck-Str. 1, 40237 Düsseldorf, Germany  Department
of Materials, Royal School of Mines, Imperial College London, Prince Consort
Road, London, SW7 2BP, UK  David Rugg  Rolls-Royce plc., Elton Road, Derby,
DE24 8BJ, UK  David Dye<EMAIL_ADDRESS>Department of Materials,
Royal School of Mines, Imperial College London, Prince Consort Road, London,
SW7 2BP, UK

(August 0 d , 2024)

###### Abstract

The $\alpha$/$\beta$ interface in Ti-6Al-2Sn-4Zr-6Mo (Ti-6246) was
investigated _via_ centre of symmetry analysis, both as-grown and after 10%
cold work. Semi-coherent interface steps are observed at a spacing of 4.5
$\pm$1.13 atoms in the as-grown condition, in good agreement with theory.
Lattice accommodation is observed, with elongation along
[$\mathrm{bar}{1}$2$\mathrm{bar}{1}$0]α and contraction along
[10$\mathrm{bar}{1}$0]α. Deformed $\alpha$ exhibited larger, less coherent
steps with slip bands lying in $\\{110\\}_{\beta}$. This indicates dislocation
pile-up at the grain boundary, a precursor to globularisation during heat
treatment. Atom probe tomography measurements of secondary $\alpha$ plates in
the fully heat treated condition showed a Zr excess at the interface, which
was localised into regular structures indicative of Zr association with
interface defects, such as dislocations. Such chemo-mechanical stabilisation
of the interface defects would both inhibit plate growth during elevated
temperature service and the interaction of interface defects with gliding
dislocations during deformation.

Titanium alloys, STEM HAADF, Interface structure, Modelling, Atom probe
tomography (APT)

##  I Introduction

Titanium alloys are widely used in safety-critical aerospace applications due
to their exceptional specific fatigue-allowable strengths Lütjering, G.
Williams (2003). The solid-state transformation from the high temperature
_bcc_ $\beta$ phase to the _hcp_ $\alpha$ phase offers the opportunity for
microstructural tailoring through solid-state processing, which allows fine-
grained microstructures to be produced Lütjering (1999). The orientation
relationship between the phases obtained by minimising the interface strain
and maximising interface coherency is approximately
$\\{110\\}_{\beta}\\!\parallel\\!\\{0002\\}_{\alpha}$ $\langle
111\rangle_{\beta}\\!\parallel\\!\langle 11\mathrm{bar}{2}0\rangle_{\alpha}$
Furuhara  _et al._ (1996). In $\alpha-\beta$ alloys, naturally-grown $\alpha$
forms as plates nucleated from pre-existing $\alpha$ or from $\beta$ grain
boundaries, _e.g._ in a Widmanstätten morphology; equiaxed $\alpha$ can then
be produced by hot working and globularisation Stefansson  _et al._ (2002);
Park  _et al._ (2012).

The low energy, broad face habit plane of $\alpha$ plates is commonly held to
be $\\{112\\}_{\beta}\parallel\\{10\mathrm{bar}{1}0\\}_{\alpha}$ Lütjering, G.
Williams (2003), although other works have also reported
$\\{11\,11\,\overline{13}\\}_{\beta}\\!\parallel\\!\\{\overline{27}\,20\,7\,0\\}_{\alpha}$
Furuhara  _et al._ (1991). Detailed Transmission Kikuchi Diffraction (TKD)
analysis has recently observed both possibilities in the same specimen Tong
_et al._ (2017). In order to accommodate the interface misfit between the two
phases, the plates contain interface defects, including interfacial
dislocations Aaronson  _et al._ (1989); Furuhara and Aaronson (1991),
structural ledges Shiflet  _et al._ (1998); Cabibbo  _et al._ (2013) and
misfit-compensating ledges. These give rise to plate thickening through the
classical terrace-ledge-kink model, where the ledges are incoherent and
therefore mobile whereas the terraces are coherent and therefore immobile.
Pond and Hirth Pond  _et al._ (2003); Hirth  _et al._ (2007) analysed the
ledges as being composed of both dislocation character and a step associated
with the core, terming the overall defect a _disconnection_.

Models of the $\alpha$/$\beta$ interface have therefore been constructed to
predict the interface, based on both the O-lattice theory of Bollmann Bollmann
(1970) and the topological model of Hirth and Pond Hirth and Pond (1996); Pond
_et al._ (2003), with the topological approach generally finding more
applications. Recently, Zheng _et al._ Zheng  _et al._ (2018) examined the
interfaces formed in refined Ti-5553, validating the calculated disconnection
spacing and direction in the interfaces observed.

The disconnection character and spacing is of technological significance
because these will determine the mobility of the interface and therefore the
plate thickening rates that arise. This will in turn be important to the plate
formation kinetics. Therefore these defects determine refinement of the
microstructures achieved by heat treatment, given the mobilities of the rate-
controlling solutes at the $\alpha$/$\beta$ interface. It can also be
hypothesised that the interface defects and their mobility are in some way
related to the nucleation of branching side-plates where fine-scale secondary
$\alpha$ are formed in heavily $\beta$-stabilised alloys such as
Ti-6Al-2Sn-4Zr-6Mo (Ti-6246).

Another aspect that is likely critical to interface mobility is local chemical
composition and the possible segregation of particular species to such an
interphase or grain boundary interface. The general subject of segregation to
interfaces was discussed, for instance, by Raabe _et al._ Raabe  _et al._
(2014), where the authors describe how the decoration of grain boundaries by
solutes or secondary phases can be used to benefit the performance of
materials. Atom probe tomography (APT) is often employed in such
investigations Breen  _et al._ (2016); Herbig  _et al._ (2014). In recent
years, there have been many reports of such segregation to interfaces in
materials other than titanium, such as aluminium. For example Pandey _et al._
Pandey  _et al._ (2019) observed Zr segregation to the eutectic
Al3Ni/$\alpha$-Al interface in a Al rich Al-Ni eutectic alloy with the
addition of 0.15 wt.% Zr. It was suggested that the presence of Zr at this
interface reduces the interfacial energy. Solute segregation has also been
observed in a model Al-Zn-Mg-Cu alloy Zhao  _et al._ (2018), where
10$\usk\mathrm{n}\mathrm{m}$ precipitate-free zones had formed adjacent to
enriched grain boundaries. Zn is frequently depleted at Al grain boundaries,
in regions 7-15$\usk\mathrm{n}\mathrm{m}$ wide Sha  _et al._ (2018). It has
also been proposed that oxygen may pin the grain boundary in nanocrystalline
Al Tang  _et al._ (2012). In steels, Miyamoto _et al._ Miyamoto  _et al._
(2018) observed boron segregation in a low carbon steel, the extent of which
reduced at low angle grain boundaries. In an Fe(Cr) nanocrystalline alloy,
solute segregation was found to be dependent on grain boundary type Zhou  _et
al._ (2016). However, grain and phase boundary solute segregation has not yet
received a similar level of attention in Ti alloys.

In this article, we investigate the interface of primary $\alpha$ plates
formed in Ti-6246 with basketweave primary $\alpha$, in order to examine the
generality of the findings of Zheng _et al._ Zheng  _et al._ (2018). We then
examine the effect of deformation on the interface structure , and discuss the
implications of these changes for the subsequent evolution of the
microstructure during processing to produce fine grained primary $\alpha$ in
commercial product. In addition, atom probe tomography data are presented that
measure chemical segregation in the vicinity of $\alpha$ interfaces in the
aged condition. We finally discuss how our findings on the deformation and
segregation might then facilitate _e.g._ the nucleation of side-plates and the
remodelling of $\alpha$ plates during globularisation.

##  II Experimental methods

Specimens were prepared from a high pressure compressor disc of nominal
composition Ti-6Al-2Sn-4Zr-6Mo (wt.%), Ti-10.8Al-0.8Sn-2.1Zr-3.0Mo (at.%)
supplied by Rolls-Royce plc. Derby. A $10\times 10\times
10\usk\mathrm{m}\mathrm{m}$ sample was heat treated for $30\usk\mathrm{min}$
at $960\mathrm{\SIUnitSymbolCelsius}$ then cooled at
$7\mathrm{\SIUnitSymbolCelsius}\usk\mathrm{min}$-1 to
$800\mathrm{\SIUnitSymbolCelsius}$, followed by manual water quenching. The
resulting microstructure featured primary $\alpha$ laths within a $\beta$
matrix, to allow a clean interface to be investigated via electron microscopy.
The material was then prepared using a standard metallographic process.
Specimens were then etched using Kroll’s solution
($100\usk\mathrm{m}\mathrm{l}$ H2O, 6$\usk\mathrm{m}\mathrm{l}$ HNO3,
3$\usk\mathrm{m}\mathrm{l}$ HF), to image grain boundaries in the scanning
electron microscope (SEM), prior to lift-out.

To produce a deformed microstructure, a bar $15\times 15\times
120\usk\mathrm{m}\mathrm{m}$ was flat rolled at room temperature to 90$\%$ of
its starting thickness in one pass. A section cut from the centre of the bar
was then prepared for electron backscatter diffraction (EBSD). EBSD
orientation mapping was performed using a Zeiss Auriga FEG-SEM with an Bruker
HKL eFlash EBSD detector to locate grains for ion milling, using an FEI Helios
NanoLab 600 DualBeam system with an OmniprobeTM micromanipulator using a
standard lift-out procedure Giannuzzi (1999). Scanning transmission electron
microscopy (STEM) was performed using a probe-corrected FEI Titan3TM 80-300.

Lattice parameters were found by X-ray diffraction (XRD) using a Panalytical
X’Pert Pro diffractometer system, with Cu Kα radiation. Spectrum patterns were
recorded in the range 5-80$\mathrm{\SIUnitSymbolDegree}$ 2$\theta$, with a
step size of 0.0334$\mathrm{\SIUnitSymbolDegree}$. Peak fitting was performed
using Topas-Academic.

To characterise the interface structure, image masks were placed over the FFT
pattern spots to refine the image, using Gatan Digital Micrograph software.
Atom column positions were calculated using a MATLAB script to find nearest
neighbour coordinates for symmetry analysis

As-received, fully heat treated needle-shaped specimens for APT were prepared
by the cryogenic focused ion beam (cryo-FIB) milling protocol described by
Chang et al. Chang  _et al._ (2019) to prevent the undesired H pick-up from
the environment. Site-specific lift-outs were conducted with a 30 kV, 6–9 nA
Xenon plasma source at ambient temperature, and the subsequent annular milling
was done using a 30 kV, 0.46 nA to 24 pA Xe beam after the stage was cooled to
$\sim$135$\degreecelsius$ using the setup described in Stephenson  _et al._
(2018). The final milling was conducted with a 2 kV, 24 pA Xe ion beam under
cryogenic conditions. APT measurements were performed using a Cameca LEAP 5000
XR operated in high-voltage pulsing mode with 20% pulse fraction, 250 kHz
pulse frequency and a target detection rate of 5 ions per 1000 pulses at a
base temperature of 50 K. The pressure in the ultra-high vacuum chamber was
consistently below $4\times 10^{-9}$ Pa. Isocomposition surfaces Hellman  _et
al._ (2003) of 2 at.% were then used to analyse Zr concentration at the
interface, where a set of 3D surfaces delineate regions within the point cloud
that contain over 2 at.% Zr.

##  III Modelling

###  III.1 Phenomenological Theory of Martensitic Transformations (PTMT)

Figure 1: (a) Schematic illustration of the broad interface between the
$\alpha$ and $\beta$ phases, with the corresponding (ideal) axes in each
phase. (b) The cubic lattice (purple) and the hexagonal lattice (pink),
illustrating the Burgers orientation relationship. There is a fixed angle
$\chi$ of each lattice, and lattice parameter, a. (c) The trigonometric
relation of the interface parameters, and (d) the lattice matching defining
the angles $\omega$ and $\phi$.

According to the phenomenological theory of martensite
crystallography/transformations (PTMC/PTMT) Pond  _et al._ (2003), a
deformation, $\mathbf{S}$, transforms the cubic $\beta$ lattice into the
hexagonal $\alpha$ lattice at the Burgers orientation;

$\begin{split}\mathbf{S}=\Omega&\begin{bmatrix}\,\,\,\,\,\Omega^{-1}&0&0\\\
0&1&(\cos\chi_{\beta}-\cos\chi_{\alpha})/\sin\chi_{\beta}\,\,\\\
0&0&\sin\chi_{\alpha}/\sin\chi_{\beta}\end{bmatrix}\\\
=&\begin{bmatrix}1&0&0\\\ 0&1.0420&-0.1842\\\ 0&0&0.9572\\\
\end{bmatrix}\end{split}$ (1)

where $\Omega=2a_{\alpha}/\sqrt{3}a_{\beta}$ and $a_{\alpha}$ and $a_{\beta}$
are the lattice parameters of each phase, obtained by X-ray diffraction. These
were measured to be $a_{\alpha}$ =
0.29346$\pm$0.0004$\usk\mathrm{n}\mathrm{m}$, $c_{\alpha}$ =
0.4693$\pm$0.0006$\usk\mathrm{n}\mathrm{m}$ and $a_{\beta}$ =
0.32519$\pm$0.0004$\usk\mathrm{n}\mathrm{m}$. No deformation is assumed along
the $x$ axis (_hcp_ $c$ direction) as the coherency strain is small in this
direction, so the lattice invariant deformation (LID) does not need to be
considered Bywater and Christian (1972); Pond  _et al._ (2003), and therefore
this can be treated as a 2-dimensional problem.
$\chi_{\beta}=70.53\mathrm{\SIUnitSymbolDegree}$ and
$\chi_{\alpha}=60\mathrm{\SIUnitSymbolDegree}$ are the angles between lattice
planes, which are set for _hcp_ and _bcc_ , Figure 1(b and c). Wayman Wayman
(1964) decomposed the deformation tensor $\mathbf{S}$ into
$\mathbf{R}$1$\mathbf{B}$, where $\mathbf{B}$ is the lattice transformation
(Bain strain, which must be symmetric) and $\mathbf{R_{1}}$ is a rigid body
rotation.

$\mathbf{B}=\begin{bmatrix}1&0&0\\\ 0&1.0376&-0.0956\\\ 0&-0.0956&0.9700\\\
\end{bmatrix}$ (2)

$\mathbf{R}_{1}$ is then:

$\begin{split}\mathbf{R}_{1}&=\mathbf{S}\mathbf{B}^{-1}=\mathbf{S}\left(\mathbf{S}^{\text{T}}\mathbf{S}\right)^{-\frac{1}{2}}\\\
&=\begin{bmatrix}1&0&0\\\ 0&0.9958&-0.0918\\\ 0&0.0918&0.9958\\\
\end{bmatrix}\end{split}$ (3)

The matrix $\mathbf{B}$ is multiplied by an invariant vector in the invariant
plane, $\mathbf{v}$. $\mathbf{v\cdot Bv}$ can be used to find the angle of
rotation,
${\color[rgb]{1,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,0}\psi_{P}}$,
where $\mathbf{Bv}$ is the undistorted but rotated vector. To discover the
orientation of the invariant plane, solutions are found for equation 4,  where
$\mathbf{v^{\prime}}$ is the transpose of the vector v.

$\mathbf{v^{\prime}B^{2}v}=\mathbf{v^{\prime}v}$ (4)

As the unit vector $\mathbf{v}$ is in the $y-z$ plane, $x=0$, equation 4
becomes:

$\begin{split}y^{2}(B_{22}^{2}+B_{23}B_{32})+2yz(B_{22}B_{23}+B_{23}B_{33})+\\\
z^{2}(B_{23}B_{32}+B_{33}^{2})=y^{2}+z^{2}\end{split}$ (5)

where $B_{23}=B_{32}$. Values from equation 2 can now be inserted.
Additionally, it can then be said that $y^{2}+z^{2}$ = 1 as $\mathbf{v}$ is a
unit vector so that:

$0.0793(1-z^{2})-0.0556z^{2}-0.3816z(\sqrt{1-z^{2}})=0$ (6)

This can then be solved numerically to give:

$\mathbf{v}=\left(\begin{array}[]{c}0\\\ 0.9771\\\ 0.2141\end{array}\right)$
(7)

The angle of inclination of the invariant plane, IP, with respect to the
terrace is then given by:

${\color[rgb]{1,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,0}\omega_{P}}=\tan^{-1}(v_{y}/v_{z})=12.27\usk\mathrm{\SIUnitSymbolDegree}$
(8)

where $v_{y}$ and $v_{z}$ are the $y$ and $z$ components of $\mathbf{v}$.

The rotation between $\mathbf{v}$ (invariant vector) and $\mathbf{Bv}$
(rotated but undistorted vector) is denoted $\mathbf{R_{2}}$. To calculate
this and the accompanying angle of rotation, a standard rotation matrix is
used for the angle between $\mathbf{v}$ and $\mathbf{Bv}$, $\psi_{P}$.

${\color[rgb]{1,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,0}\psi_{P}}=\cos^{-1}\left(\frac{\mathbf{v}\cdot\mathbf{Bv}}{\|\mathbf{v}\|\leavevmode\nobreak\
\|\mathbf{Bv}\|}\right)=5.7981^{\circ}$ (9)

This angle can be used to calculate the rotation $\mathbf{R_{2}}$, making
$\mathbf{v}$ the invariant vector, where $\mathbf{R_{2}Bv}=\mathbf{v}$.

$\begin{split}\mathbf{R_{2}}&=\begin{bmatrix}1&0&0\\\
0&\cos(\psi)&-\sin(\psi)\\\ 0&\sin(\psi)&\cos(\psi)\end{bmatrix}\\\
&=\begin{bmatrix}1&0&0\\\ 0&0.9949&-0.1010\\\ 0&0.1010&0.9949\\\
\end{bmatrix}\end{split}$ (10)

$\mathbf{R}_{3}=\mathbf{R}_{2}\mathbf{R}^{-1}_{1}$ is the overall
misorientation of the $\alpha$ lattice away from the Burgers orientation, with
a rotation angle
${\color[rgb]{1,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,0}\phi_{P}}=\cos^{-1}(\text{R}_{3yy})$,
Figure 1(d).

The final rotation, $\mathbf{R_{3}}$, can now be calculated, where:

$\mathbf{R_{3}}=\mathbf{R_{2}R_{1}^{-1}}=\begin{bmatrix}1&0&0\\\
0&0.99996&-0.0.0093\\\ 0&0.0093&0.99996\\\ \end{bmatrix}$ (11)

This gives the final angle of rotation, the rigid body rotation of the
$\alpha$ lattice away from the Burgers orientation, as 0.5331 °.

###  III.2 Topological theory

Alternatively, the relationship between the phases can be calculated using a
topological model Pond  _et al._ (2003), where the $\beta$ lattice is
stretched and the $\alpha$ lattice is compressed by $\varepsilon_{yy}/2$ along
$\langle 111\rangle_{\beta}\\!\parallel\\!\langle
11\mathrm{bar}{2}0\rangle_{\alpha}$.

Now, the step heights of the $\alpha$ and $\beta$ phases can be calculated
using lattice parameters and angles outlined,

$\begin{split}h_{\alpha}&=a_{\alpha}\sin\left(\chi_{\alpha}\right)=0.2541\;\mathrm{nm}\\\
h_{\beta}&=\frac{\sqrt{3}a_{\beta}}{2}\sin\left(\chi_{\beta}\right)=0.2655\;\mathrm{nm}\end{split}$
(12)

The Burgers vector of the disconnections is given by
$\mathbf{b}=\mathbf{t}_{\beta}-\mathbf{t}_{\alpha}$ with components
$b_{z}=h_{\beta}-h_{\alpha}$ (see Figure 1(c) for definitions) and

$b_{y}=3^{-0.5}a_{\beta}(1+\varepsilon_{yy}/2)-0.5a_{\alpha}(1-\varepsilon_{yy}/2)$
(13)

Pond _et al_. Pond  _et al._ (2003) show that

$\varepsilon_{yy}=\frac{b_{y}\tan\theta+b_{z}\tan^{2}\theta}{h_{\alpha}}=0.0412\;\mathrm{nm}$
(14)

For this system, $b$ = 0.0492 nm, $b_{y}$ = 0.0479 nm and $b_{z}$ = 0.0114 nm.
This quadratic equation can be solved for $\tan\theta$ (Figure 1(a)). Then,
${\color[rgb]{1,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,0}\omega_{T}=\theta_{T}-\phi_{T}}$,
Figure 1(d).
${\color[rgb]{1,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,0}\phi_{T}}$
is then calculated as follows:

$\begin{split}{\color[rgb]{1,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,0}\phi_{T}}&=2\sin^{-1}\left(\frac{(b_{z}\cos\theta-
b_{y}\sin\theta)\sin\theta}{2h_{\alpha}}+\frac{\varepsilon_{yy}\sin\theta\cos\theta}{2}\right)\\\
&=0.5108\mathrm{\SIUnitSymbolDegree}\end{split}$ (15)

Thus, one obtains the disconnection spacing,
${\color[rgb]{1,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,0}L_{T}}$ and
the disconnection spacing along the terrace plane,
${\color[rgb]{1,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,0}\lambda_{T}}$:

$\displaystyle{\color[rgb]{1,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,0}L_{T}}$
$\displaystyle=\frac{h_{\alpha}}{\sin\theta}=1.2473\;\mathrm{nm}$ (16)
$\displaystyle{\color[rgb]{1,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,0}\lambda_{T}}$
$\displaystyle={\color[rgb]{1,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,0}L_{T}}\cos\theta=1.2211\;\mathrm{nm}$
(17)

##  IV Results

###  IV.1 Undeformed $\alpha$/$\beta$ interface

The $\alpha$/$\beta$ interface was imaged using HR-STEM, Figure 2. The top of
the image is viewed along [110]β, and the bottom along [0002]α. Foil
heterogeneity caused deviation from the zone axis in some parts of the image,
reducing the definition of some atomic columns. To address this, the image FFT
was filtered via pattern spot masking, Lucy-Richardson deblurring and Gaussian
point spread function (PSF) re-convolution. The nearest neighbours could then
be calculated for each column enabling analysis of the local crystal symmetry
to estimate interfacial step locations, Figures 2(b) and 4(d). The interfacial
symmetry parameter $M_{i}$ mapped in Figure 2(b) is given by:

$M_{i}=\left(||\textstyle{\sum_{n=1}^{3}}\mathbf{r}_{n}||\\!\cdot\\!||\textstyle{\sum_{n=1}^{2}}\mathbf{r}_{n}||\right)/\,||\mathbf{r}_{1}||^{2}$
(18)

where $\mathbf{r}_{n}$ is the relative position vector of the nearest
neighbour of rank $n$ for a given atomic column taken as the origin, _i.e._
$\mathbf{r}_{1}$ is the first nearest neighbour position. The crystal symmetry
of the $\alpha$ and $\beta$ phases gives non-zero values of $M_{i}$ only at
the interface, due to _e.g._ crystallographic defects and regions with non-
uniform lattice strain. Hence, $M_{i}$ gives a qualitative measure of
deviation from the ideal structure in the vicinity of the interface.

Figure 2: (a) HAADF HR-STEM of the $\alpha$/$\beta$ interface. The top of the
image is viewing the (110) $\beta$ plane, the bottom is the (0002) $\alpha$
plane. The stepped interface can be seen in the middle, (b) colour map of
$M_{i}$, indicating where the positions of steps are along the interface
through deviation from symmetry.

Multiple disconnection spacings were measured from the image giving an average
disconnection spacing $\lambda$ of 4.5 $\pm$1.13 atoms (1.5nm) and an overall
inclination $\omega$ of $12.6\pm 1^{\circ}$, allowing comparison with the PTMC
and topological models. The topological model predicted a disconnection
spacing $\lambda_{T}$ of 4.25 atoms (1.22$\usk$nm), with $\omega_{T}$=12.29∘
and $\phi_{T}$=0.5337∘, whilst the PTMC model predicted
$\omega_{P}$=$12.27^{\circ}$ and $\phi_{P}$=0.5331∘. Good agreement between
both models and the measurements are obtained, similar to the result obtained
by Zheng et al for alloy Ti-5553 Zheng  _et al._ (2018), who only compared to
the topological model.

In addition to disconnection spacing $\lambda$, symmetry analysis highlighted
a distorted region in the $\alpha$ phase adjacent to the interface (light
blue). Its depth can be observed to be $\approx$1nm and within it the $\alpha$
lattice shows a gradual dimensional distortion in the course of deforming into
the $\beta$ lattice. It exhibits elongation and contraction along the
$[\mathrm{bar}{1}2\mathrm{bar}{1}0]_{\alpha}$ and
$[10\mathrm{bar}{1}0]_{\alpha}$ crystallographic directions, respectively.
Given the very thin specimen geometry this effect is unlikely to result from
inclination of the interface relative to the beam direction and can therefore
be assumed to be the accommodation of the new lattice form.

###  IV.2 Interface chemistry

The chemistry of a similar interface was investigated using APT, Figure 3.
When a 2 at.$\%$ Zr isosurface is applied, Figure 3(a), regularly spaced
features can be seen along both of the $\alpha/\beta$ interfaces examined,
labelled interface 1 and 2. To further highlight this, a density plot was
used, figure 3(b-c). These analyses within the plane of the interface make
clear that there are regularly spaced enriched regions of Zr along the
interface, which form linear features.

When composition profiles are taken (along the directions indicated by arrows
in each sub-figure), an increase of Zr of around 1.5 at. $\%$ is measured,
spaced around 5–10 nm apart. These are observed to occur in both interfaces
examined, at multiple locations and in multiple datasets obtained from
different specimens prepared from the same sample, and is hence considered to
be a general feature of such interfaces.

Figure 3: Atom probe tomography of a specimen of fully heated treated Ti-6246.
(a) Two $\alpha$ lamellae were observed within the needle, with the $\beta$
matrix in-between, as labelled. The point cloud represents a volume of $50\usk
nm\times 50\usk nm\times 95\usk nm$. (b-c) 2 at.$\%$ Zr isosurface
highlighting regions of increased Zr concentration at each $\alpha/\beta$
interface. A density contour highlights regularly spaced regions along the
interface with higher concentrations of Zr, which are attributed to possible
defects along the interface. (d-f) Concentration profile across the regions
highlighted in (a-c), respectively.

###  IV.3 Deformed $\alpha$/$\beta$ interface

The Ti-6246 specimen deformed to 10$\%$ strain is shown in Figure 4, where the
image plane is parallel to (111)β//(2$\overline{11}$0)α. The $\alpha$ laths
are less lenticular than in the undeformed alloy, with characteristic ‘bumps’
along the interface. Slip bands can be seen in the $\beta$ matrix, terminating
at large heterogeneous steps along the $\alpha$/$\beta$ interface. Within the
$\beta$ phase, the trace of the slip bands can be seen on $\\{110\\}$ glide
planes, which lie parallel to the beam direction.

Given the symmetry of the $\beta$ and $\alpha$ lattices, a hexagonal centre-
of-symmetry parameter Kelchner  _et al._ (1998) was calculated for each atomic
column, Figure 4(d). The parameter is zero in a perfect $\beta$ crystal, but
takes on high values in the $\alpha$, which possess off-hexagonal symmetry
when viewed along $[2\overline{1}\overline{1}0]_{\alpha}$ . A gross rotation
in the crystal lattice can be seen in the image. Given the numerous
dislocations observed in Figure 4, their pile-up at the interface is the
likely cause of the $\beta$ lattice rotation. The $\beta$ phase is softer than
the $\alpha$ phase, and therefore might be expected to accommodate more of the
applied deformation. Some lattice dislocations in the $\beta$ phase are
evident in Figure 4(d) and are labelled with arrows.

Figure 4: (a) HAADF-STEM overview of the TEM foil (image normal
$[1\mathrm{bar}{2}\mathrm{bar}{1}0]_{\alpha}\|[1\mathrm{bar}{1}\mathrm{bar}{1}]_{\beta}$),
with (b) a magnified image of the $\alpha$ plate structure. The $\alpha$ phase
appears as dark and the $\beta$ phase as a light grey. Large steps can be seen
along the $\alpha$/$\beta$ interface, and evidence of distortion can be seen
in the $\beta$ matrix. (c) shows the increased magnification of the less
coherent step structure and (d) is a colour map of the centre of symmetry
parameter Kelchner  _et al._ (1998), with dislocations highlighted by arrows.

Figure 2 shows clearly that steps in the $\alpha$/$\beta$ interface allow for
the accommodation of lattice mismatch between the two phases in Ti-6246. The
interfacial steps become significantly exaggerated when the alloy is cold
worked, Figure 4. One might hypothesize that these could act as nucleation
sites for the formation of secondary $\alpha$. Conversely, the large shear
bands, seen in Figure 4, might also provide nucleation sites at lower ageing
temperatures.

##  V Discussion

Zherebtsov _et al._ Zherebtsov  _et al._ (2010), investigated the loss of
coherency at stepped $\alpha$/$\beta$ interphase boundaries using the
analytical van der Merwe model. The study showed that the absorption of
lattice dislocations by the interface leads to gradual loss of coherency
substantially increasing the interfacial energy. The minimisation of this
energy is likely to drive the formation of the giant interfacial steps in
deformed Ti-6246 observed in this study.

Formation of interfacial phases has been suggested to accompany the $\beta$ to
$\alpha$ solid state transformation in some Ti alloys, helping to accommodate
the interfacial misfit strain and potentially providing nucleation sites for
the $\alpha$ plates. Kang _et al._ Kang  _et al._ (2000) investigated the
$\alpha$/$\beta$ phase boundary in a near-$\alpha$ titanium alloy, observing a
discontinuous interface phase in a pre-strained material. Nag _et al._ Nag
_et al._ (2009) reported assisted nucleation of $\alpha$ laths from aged-in
nanoscale $\omega$ phase in Ti-5553-0.5Fe, which can be exploited for
microstructural refinement Zheng  _et al._ (2016). Our study has not found any
additional phases associated with the interphase boundary. Instead interfacial
strain relief is provided by a $1\usk$nm thick region where the $\alpha$
lattice shows progressive distortion toward the _bcc_ structure.

Considering the thermodynamics of the interface, deformation has resulted in
an accumulation of defects and associated interfacial defect structures which
are available for energy minimisation by thermally-induced diffusional
rearrangement. Many titanium alloys such as Ti-6246, Ti-6Al-4V and
Ti-6Al-2Sn-4Zr-2Mo are worked in the two phase region, after which a heat
treatment is applied to remodel the primary $\alpha$, termed globularisation.
Jackson _et al._ Jackson  _et al._ (2009) showed in the near-$\beta$ alloy
Ti-10V-2Fe-3Al that deformation causes the $\alpha$ to fragment, a precursor
to globularisation. Dislocation pile-up occurs at the $\alpha$/$\beta$
boundary causing the break up of $\alpha$ plates, leading to penetration of
$\beta$ at the subgrain boundaries. Such dissolution of $\alpha$ by $\beta$
phase must therefore be driven by the diffusion of highly misfitting atoms
such as Mo, which tend to be $\beta$-stabilisers, to highly defective regions
such as those observed here. This provides a mechanistic hypothesis for how
globularisation occurs. Additionally, Poschmann _et al._ Poschmann  _et al._
(2018) found _via_ elasticity theory and molecular dynamics that
$<111>_{\beta}$ screw dislocations can act as heterogeneous nucleation sites
in pure Ti, resulting in an elastically preferential habit plane. This
indicates that after deformation, $\alpha$ nucleation that occurs may be
affected by the Burgers vector of the dislocation, linking dislocation density
to the post transformation microstructure. Therefore the prevalence of
$\\{110\\}_{\beta}$ slip bands observed here could lead to altered nucleation
of secondary $\alpha$. It should also be noted that slip bands through the
$\alpha-\beta$ ensemble provide both a means of work hardening and of flow
localisation, which might be of concern, _e.g._ in fatigue. Such slip transfer
events have been observed and analysed in detail elsewhere Joseph  _et al._
(2018), and therefore will not be discussed further.

Elemental segregation at the $\alpha/\beta$ interface has not yet been seen in
the literature. The Zr peaks observed could be due to defects along the
interface, however the spacing observed (5-10 nm) is inconsistent with the
occurrence of these suggested defects at every interface step. Selective
segregation patterns associated with the local strain state were recently
reported in a detailed analysis by a combination of HR-STEM and APT (Liebscher
_et al._ , 2018). For the APT data presented here, the zone axis along which
these compositional variations were observed is not known. Therefore it cannot
directly be connected crystallographically to the features seen in the HR-STEM
samples. Zr is a relatively neutral stabiliser in Ti alloys, segregating to
neither the $\alpha$ nor the $\beta$ phase. However, the segregation of Zr was
predicted to help lower the stacking fault energy in $\alpha$-Ti Kwasniak  _et
al._ (2016), so segregation hence likely also helps minimise the system’s free
energy. The spacing of the Zr-rich observed features is similar to that of the
slip bands observed after deformation in the $\beta$ matrix, _e.g._ before the
secondary $\alpha$ plates observed here were grown. Ultimately this can not be
known without a fully-correlative, in-situ observation of the slip bands and
then ageing of the $\alpha$, _e.g._ by HR-STEM, followed by correlative APT to
detect the Zr features. It is however reasonable to state, based on similar
observations in the literature (Liebscher  _et al._ , 2018; Araullo-Peters
_et al._ , 2012; Kwiatkowski da Silva  _et al._ , 2018), that the pattern in
the Zr segregation at the interface is due to the presence of line defects.

Titanium plates are held to thicken by ledgewise growth, since the interface
steps will be high energy features with consequent high mobility. This gives
them the ability to ‘sweep up’ solutes and other interfacial defects as they
move. This may provide a rationale for why such segregation might occur. It
may also be assisted by Zr transport along slip bands, provided that there is
a thermodynamic driving force. However, the mobility of such interface defects
would then be kinetically inhibited by the energy required to un-pin from the
solute, and/or the requirement for a waiting time for the solute to diffuse to
the moving defect. This would ultimately limit the ability of secondary
$\alpha$ to coarsen, _e.g._ during elevated temperature service, acting to
stabilise the microstructure in use. It would also limit the ability of such
deformation generated mobile dislocations to interact with existing
interfacial conherency defects, thus strengthening the interface.

##  VI Conclusions

We examined the $\alpha$/$\beta$ interface in Ti-6Al-2Sn-4Zr-6Mo (Ti-6246).
Semi-coherent interface steps are observed at a spacing of 4.5 $\pm$1.13 atoms
and an inclination, $\omega$ of 12.6$\pm 1\mathrm{\SIUnitSymbolDegree}$, in
the as-grown condition which is in good agreement with theory predictions,
where the topological model predicted a disconnection spacing $\lambda_{T}$ of
4.25 atoms (1.22$\usk$nm), with $\omega_{T}$=12.29∘ and $\phi_{T}$=0.5337∘ and
the PTMC model predicted an angle of $\omega_{P}$=$12.27^{\circ}$ and
$\phi_{P}$=0.5331∘. A $\sim 1\usk\mathrm{n}\mathrm{m}$ interface region is
observed where the two lattices are distorted. Deformed $\alpha$ exhibited
larger, less coherent steps with slip bands lying in $\\{110\\}_{\beta}$.
These microstructural steps may provide the sources for the pinch-off process
of globularisation observed during the $\alpha-\beta$ heat treatment of
$\alpha+\beta$ worked titanium alloys. Furthermore, Zr decoration of the
interface was observed, localised within linear features with a spacing of
5-10 nm.

## Acknowledgements

The authors acknowledge funding from EPSRC (grants EP/K034332/1 and
EP/M506345/1 and EP/L025213/1), Rolls-Royce plc and Imperial College. The
assistance of Prof. Robert C. Pond was of great help in improving the
manuscript, particularly the mathematical treatment. Uwe Tezins and Andreas
Sturm for their technical support of the atom probe tomography and focused ion
beam facilities at the Max-Planck-Institut für Eisenforschung. L.T. Stephenson
is acknowledged for his kind help and support for the cryo-FIB and overall
setup of the Laplace project. Dr Yanhong Chang is grateful to the China
Scholarship Council (CSC) for funding of her PhD scholarship. BG acknowledges
financial support from the ERC-CoG-SHINE-771602 and is grateful for the
financial support from the BMBF via the project UGSLIT and the Max-Planck
Gesellschaft via the Laplace project.

## References

  * Lütjering, G. Williams (2003) J. C. Lütjering, G. Williams,  _Titanium_ (Springer, 2003).
  * Lütjering (1999) G. Lütjering, Acta Mater. 39, 4838 (1999), arXiv:0502551 [cond-mat] .
  * Furuhara  _et al._ (1996) T. Furuhara, S. Takagi, H. Watanabe, and T. Maki, Metall. Mater. Trans. A 27, 1635 (1996).
  * Stefansson  _et al._ (2002) N. Stefansson, S. L. Semiatin, and D. Eylon, Metall. Mater. Trans. A 33A (2002).
  * Park  _et al._ (2012) C. H. Park, J. W. Won, J.-W. Park, A. L. Semiatin, and C. S. Lee, Metall. Mater. Trans. A 43A (2012).
  * Furuhara  _et al._ (1991) T. Furuhara, J. Howe, and H. Aaronson, Acta Metall. Mater. 39, 2873 (1991).
  * Tong  _et al._ (2017) V. Tong, S. Joseph, A. K. Ackerman, D. Dye, and T. B. Britton, J. Microsc. 267, 318 (2017), 1701.03014 .
  * Aaronson  _et al._ (1989) H. Aaronson, T. Furuhara, and Y. Mou., (1989).
  * Furuhara and Aaronson (1991) T. Furuhara and H. Aaronson, Acta Metall. Mater. 39, 2857 (1991).
  * Shiflet  _et al._ (1998) G. J. Shiflet, M. a. Mangan, and W. G. Meng, Interface Sci. 6, 133 (1998).
  * Cabibbo  _et al._ (2013) M. Cabibbo, S. Zherebtsov, S. Mironov, and G. Salishchev, J. Mater. Sci. 48, 1100 (2013).
  * Pond  _et al._ (2003) R. Pond, S. Celotto, and J. Hirth, Acta Mater. 51, 5385 (2003).
  * Hirth  _et al._ (2007) J. Hirth, R. Pond, and J. Lothe, Acta Mater. 55, 5428 (2007).
  * Bollmann (1970) W. Bollmann,  _Crystal defects and crystalline interfaces_ (Berlin: Springer-Verlag, 1970).
  * Hirth and Pond (1996) J. Hirth and R. Pond, Acta Mater. 44, 4749 (1996).
  * Zheng  _et al._ (2018) Y. Zheng, R. E. Williams, G. B. Viswanathan, W. A. Clark, and H. L. Fraser, Acta Mater. 150, 25 (2018).
  * Raabe  _et al._ (2014) D. Raabe, M. Herbig, S. Sandl$\"{o}$bes, Y. Li, D. Tytko, M. Kuzmina, D. Ponge, and P.-P. Choi, Curr. Opin. Solid. St. M. 18, 253 (2014).
  * Breen  _et al._ (2016) A. J. Breen, K. Babinsky, A. C. Day, K. Eder, C. J. Oakman, P. W. Trimby, S. Primig, J. M. Cairney, and S. P. Ringer, Microsc. Microanal. 23, 279 (2016).
  * Herbig  _et al._ (2014) M. Herbig, D. Raabe, Y. J. Li, P. Choi, S. Zaefferer, and S. Goto, Phys. Rev. Lett. 112, 126103 (2014).
  * Pandey  _et al._ (2019) P. Pandey, S. K. Makineni, B. Gault, and K. Chattopadhyay, Acta. Mater. 170, 205 (2019).
  * Zhao  _et al._ (2018) G. Zhao, F. D. Geuser, A. da Silva, A. Szczepaniak, B. Gault, D. Ponge, and D. Raabe, Acta. Mater. 156, 318 (2018).
  * Sha  _et al._ (2018) G. Sha, L. Yao, X. Liao, S. P. Ringer, Z. C. Duan, and T. G. Langdon, Ultramicroscopy 111, 500 (2018).
  * Tang  _et al._ (2012) F. Tang, D. S. Gianola, M. P. Moody, K. J. Hemker, and J. M. Cairney, Acta. Mater. 60, 1038 (2012).
  * Miyamoto  _et al._ (2018) G. Miyamoto, A. Goto, N. Takayama, and T. Furuhara, Scr. Mater. 154, 168 (2018).
  * Zhou  _et al._ (2016) X. Zhou, X.-X. Yu, T. Kaub, R. L. Martens, and G. B. Thompson, Sci. Rep. 6, 34642 (2016).
  * Giannuzzi (1999) F. Giannuzzi, L. A. Stevie, Micron 30, 197 (1999).
  * Chang  _et al._ (2019) Y. Chang, W. Lu, J. Gu$\'{e}$nol$\'{e}$, L. T. Stephenson, A. Szczpaniak, P. Kontis, A. K. Ackerman, F. F. Dear, I. Mouton, X. Zhong, S. Zhang, D. Dye, C. H. Liebscher, D. Ponge, S. Korte-Kerzel, D. Raabe, and B. Gault, Nat. Commun. 10, 942 (2019).
  * Stephenson  _et al._ (2018) L. T. Stephenson, A. Szczepaniak, I. Mouton, K. A. K. Rusitzka, A. J. Breen, U. Tezins, A. Sturm, D. Vogel, Y. Chang, P. Kontis, A. Rosenthal, J. D. Shepard, U. Maier, T. F. Kelly, D. Raabe, and B. Gault, PLOS One 13, e0209211 (2018), 1805.10836 .
  * Hellman  _et al._ (2003) O. Hellman, J. Rivage, and D. Seidman, Ultramicroscopy 95, 199 (2003).
  * Bywater and Christian (1972) K. A. Bywater and J. W. Christian, Philosophical Magazine 25, 1249 (1972).
  * Wayman (1964) C. Wayman,  _Introduction to the crystallography of martensitic transformations_ (Macmillan, 1964).
  * Kelchner  _et al._ (1998) C. L. Kelchner, S. J. Plimpton, and J. C. Hamilton, Phys. Rev. B 58, 11085 (1998).
  * Zherebtsov  _et al._ (2010) S. Zherebtsov, G. Salishchev, and S. Lee Semiatin, Philos. Mag. Lett. 90, 903 (2010).
  * Kang  _et al._ (2000) Q. Kang, Z. X. Guo, D. Li, and Y. Y. Liu, Mater. Sci. Eng. A 280, 182 (2000).
  * Nag  _et al._ (2009) S. Nag, R. Banerjee, R. Srinivasan, J. Hwang, M. Harper, and H. Fraser, Acta Mater. 57, 2136 (2009).
  * Zheng  _et al._ (2016) Y. Zheng, R. E. Williams, D. Wang, R. Shi, S. Nag, P. Kami, J. M. Sosa, R. Banerjee, Y. Wang, and H. L. Fraser, Acta Mater. 103, 850 (2016).
  * Jackson  _et al._ (2009) M. Jackson, N. G. Jones, D. Dye, and R. J. Dashwood, Mater. Sci. Eng. A 501, 248 (2009).
  * Poschmann  _et al._ (2018) M. Poschmann, J. Lin, H. Geerlings, I. S. Winter, and D. C. Chrzan, Phys. Rev. Mater. 2, 083606 (2018).
  * Joseph  _et al._ (2018) S. Joseph, I. Bantounas, T. C. Lindley, and D. Dye, Int. J. Plast. 100, 90 (2018).
  * Liebscher  _et al._ (2018) C. H. Liebscher, A. Stoffers, M. Alam, L. Lymperakis, O. Cojocaru-Mirédin, B. Gault, J. Neugebauer, G. Dehm, C. Scheu, and D. Raabe, Phys. Rev. Lett. 121, 15702 (2018).
  * Kwasniak  _et al._ (2016) P. Kwasniak, H. Garbacz, and K. Kurzydlowski, Acta. Mater. 102, 304 (2016).
  * Araullo-Peters  _et al._ (2012) V. J. Araullo-Peters, B. Gault, S. L. Shrestha, L. Yao, M. P. Moody, S. P. Ringer, and J. M. Cairney, Scr. Mater. 66 (2012), 10.1016/j.scriptamat.2012.02.022.
  * Kwiatkowski da Silva  _et al._ (2018) A. Kwiatkowski da Silva, D. Ponge, Z. Peng, G. Inden, Y. Lu, A. Breen, B. Gault, and D. Raabe, Nat. Commun. 9, 1137 (2018).