# Representation of incomplete contact problems by half-planes

H. Andresen${}^{\,\text{a,}}$ , D.A. Hills${}^{\,\text{a}}$, M.R.
Moore${}^{\,\text{b}}$
${}^{\text{a}}$ Department of Engineering Science, University of Oxford, Parks
Road, OX1 3PJ Oxford, United Kingdom
${}^{\text{b}}$ Mathematical Institute, University of Oxford, Andrew Wiles
Building, Radcliffe Observatory Quarter,
Woodstock Road, OX2 6GG Oxford, UK Corresponding author: Tel.: +44 1865
273811;
E-mail address<EMAIL_ADDRESS>(H. Andresen).

###### Abstract

Methods for finding the optimal choices of the applied remote loads – the
applied normal force, moment, shear force and remote bulk stresses – needed to
solve frictional contact problems in partial-slip using half-plane theory are
derived by using data from contacts analysed by the finite element method.
While the normal and shear forces and moment are readily found from
equilibrium considerations, in order to determine the bulk stresses we must
exploit details of the traction ratio and the direct strain within the
contact, both of which are readily extracted from simulations. These contact
loads enable the formulation of an equivalent half-plane problem for the
contact, which can be used to determine much more precise estimates of the
slip-zone sizes than are obtainable from direct use of frictional finite
element analysis, as aggregated data from the finite element output is
employed, and the half-plane analysis will add precision in terms of
satisfaction of the laws of frictional slip and stick.

Keywords: Contact mechanics; Half-plane theory; Incomplete contacts; Prototype
analysis; Partial slip

##  1 Introduction

All real physical contact problems are between three dimensional objects of
finite size. Incomplete (convex) contacts occur very widely in mechanical
engineering, and we would like to be able to study them in great detail,
particularly when the interface is subject to friction. To do this, a powerful
representation is to use a formulation in which each body is idealised as a
half-plane, but, as a precursor, we must first reduce a three-dimensional
problem to one which is geometrically two-dimensional, and then replace the
finite bodies by half-planes which incorporate small ‘bumps’ on the surface
whose profiles match those of the original problem [1].

Figure 1 a) shows a typical two-dimensional contact problem arising in the fan
blade root dovetail of gas turbine engines due to the application of a
centrifugal load, $F_{C}$, vibrational load $F_{V}$, and expansion load $T$.
In its ‘core’ we can identify five quantities which are relevant to the local
contact problem, all of which may be functions of time, and four of which
control the partial slip problem. These are: the normal load, $P$, moment,
$M$, shear force, $Q$, and the differential tension, $\sigma$, developing
between bodies A and B, where $\sigma=\sigma_{\text{A}}-\sigma_{\text{B}}$,
Figure 1.

Figure 1: The fan blade dovetail root contact shown in a) is analysed using
the half-plane representation shown in b).

The normal contact problem is defined by the front face geometry (profile)
together with the applied normal force, $P$, and applied moment, $M$. It is
trivial to deduce these from the finite element output by evaluating

$P=\int_{-a}^{b}p(x)\mathrm{d}x\;\text{,}$ (1)
$M=\int_{-a}^{b}p(x)x\mathrm{d}x\;\text{,}$ (2)

where $p(x)$ is the contact pressure. Shear tractions, which tend to induce
slip, are actually excited by two forms of loading. One is by the application
of an external shear force, $Q$, which again is straightforward to deduce as a
traction resultant by evaluating

$Q=\int_{-a}^{b}q(x)\mathrm{d}x\;\text{,}$ (3)

where $q(x)$ is the shear traction within the contact.

However, there is also a second form of excitation of shear tractions which is
clear in the half-plane model, Figure 1 b), namely the effect of the bulk
tensions exerted remotely parallel with the surface which, if unequal, produce
an (approximately) antisymmetric interfacial shear traction distribution in
the half-plane contact. These remote forms of loading are very easy to
visualise in the half-plane problem, but when we consider finite problems such
as that shown in Figure 1 a), and where the geometry of individual components
looks very different from a half-plane, their ‘equivalent’ is very much harder
to interpret. It is the primary function of this paper to show how we may best
interpret the output of the finite element analysis of a problem such as that
shown in Figure 1 a) to enable finding the optimal choices for the remote
stresses applied to the half-plane contact problem shown in Figure 1 b).

In a structural engineering sense, the prototypical problem shown in Figure 1
a) is redundant, because the two contact flanks mean that the loads carried on
each individual contact cannot be determined by considerations of equilibrium
alone, and we would therefore normally begin our analysis by forming a finite
element model of the problem using a commercial code. This code outputs the
local contact pressure, $p(x)$, and shear tractions, $q(x)$. These outputs,
along with the geometry of the structure together with the coefficient of
friction, provide a connection between the external loads and the local
response acting on the contact interface

$\left\\{\begin{array}[]{c}P\\\ Q\\\ M\\\ \sigma_{\text{A}}\\\
\sigma_{\text{B}}\end{array}\right\\}=\left[\begin{array}[]{ccc}e_{11}\quad
e_{12}\quad e_{13}\\\ e_{21}\quad e_{22}\quad e_{23}\\\ e_{31}\quad
e_{32}\quad e_{33}\\\ e_{41}\quad e_{42}\quad e_{43}\\\ e_{51}\quad
e_{52}\quad e_{53}\end{array}\right]\left\\{\begin{array}[]{c}F_{C}\\\
F_{V}\\\ T\end{array}\right\\}\;\text{.}$ (4)

Since the loads $P$, $Q$, $M$, $\sigma_{A}$ and $\sigma_{B}$ are the necessary
inputs to any half-plane analysis (cf. Figure 1 b)), it is the intention of
this paper is to provide means of determining this connection, here indicated
by the matrix entries $e_{ij}$, for $i=1,2,3,4,5$ and $j=1,2,3$. While the
first nine matrix entries ($e_{1j}$, $e_{2j}$, $e_{3j}$) manifest themselves
in evaluating Eqs. (1, 2, 3), determining the remaining entries ($e_{4j}$,
$e_{5j}$) is far more complicated. But good estimates of $\sigma_{\text{A}}$
and $\sigma_{\text{B}}$ are needed in order to construct, firstly, an
analytical model of the individual contact using a model based on half-plane
domains, where the shape of the contacting surfaces is the same as the
corresponding function for the prototypical contact. The half-plane analysis
becomes advantageous because the computationally justifiable resolution of the
finite element model is too poor to investigate thoroughly the partial slip
problem in full detail [2]. Secondly, by knowing the individual remote
stresses present in each body, a far more accurate representation of the
propagation stresses for potentially initiated cracks in the prototype can be
applied when a simpler laboratory set up is used in an experimental
investigation.

Once the values for all five contact loads, $P$, $Q$, $M$, $\sigma_{\text{A}}$
and $\sigma_{\text{B}}$ are found, the whole of the family of solutions for
partial-slip analysis becomes available, for example the pioneering solutions
of Cattaneo and Mindlin [3], [4] and more recent developments [5], including
dislocation-based formulations [6], [7]. We wish to emphasise that, for
reasons which will become clear, the finite problem is investigated under
fully adhered conditions. This simply allows us to exploit specific properties
of the tractions arising during loading which are relevant in identifying the
tension components. However, this does not imply that any subsequent analysis
must be carried out with this assumption. We will develop the problem in
stages.

##  2 Proportional loading: no moment developed

Figure 2 shows two very common loading sequences for a $P,Q,\sigma$-problem
neglecting the effect of a moment. In red we see a sequential order of load
application and in green we see proportional loading from one load point to
another. The sequence in which the loads are applied greatly affects the
procedure we wish to develop. In real problems the external loads may not be
applied separately, in the sense of forming the contact first, and then
applying the forces which excite shear but, since it is of interest
academically, we discuss details of sequential loading in detail in Appendix
A.

Usually, the external loads applied have the effect of exerting all local
loads proportionally, so that

$\frac{\mathrm{d}P}{P}=\frac{\mathrm{d}M}{M}=\frac{\mathrm{d}Q}{Q}=\frac{\mathrm{d}\sigma_{\text{A}}}{\sigma_{\text{A}}}=\frac{\mathrm{d}\sigma_{\text{B}}}{\sigma_{\text{B}}}\;\text{.}$
(5)

The problem becomes significantly more complicated when there is a moment
present, and it is not possible to treat the problem as comprehensively (see
§3) so here we will begin by looking in some detail at problems where the
moment may be neglected. Consider the problem shown in Figure 1 a) and b), and
suppose that the contacting bodies are elastically similar. In our analysis we
will neglect the shear tractions induced by the application of the normal load
as they will be much smaller in magnitude than the contact pressure when the
bodies are elastically similar and arise only because the contacting bodies
usually have different domain shapes. While the remarks about deducing the
values of normal load and shear force ($P$, $Q$) as traction resultants in §1
still apply, we now need a means of finding the individual remote tensions
($\sigma_{\text{A}},\sigma_{\text{B}}$). To simplify even further, we will
look at the reduced case of a contact in which the indenter is also symmetric,
so that in the half-plane formulation of figure 1 b), $b=a$. Moreover, this
also implies that we neglect the presence of the vibrational external load,
$F_{V}$, which somewhat simplifies the matrix system Eq. (4).

Figure 2: Sequential loading (red) and proportional loading (green) for a
$P,Q,\sigma$-problem.

We shall begin by describing a methodology for finding the bulk tension
difference, $\sigma=\sigma_{A}-\sigma_{B}$. Suppose that we have a contact
whose current half-width is $a$, and we make a small change, $\Delta P$, in
normal load. The corresponding change in contact pressure is given by [8]

$\Delta p(x)=\frac{\Delta P}{\pi\sqrt{a^{2}-x^{2}}}\;\text{.}$ (6)

If at the same time there are small changes in shear force, $\Delta Q,$ and
differential bulk tension, $\Delta\sigma$, the change in shear traction,
$\Delta q(x),$ generated is given by [8]

$\Delta q(x)=\frac{\Delta Q}{\pi\sqrt{a^{2}-x^{2}}}+\frac{\Delta\sigma
x}{4\sqrt{a^{2}-x^{2}}}\;\text{.}$ (7)

If full stick is to be maintained during a positive load increment we require
that [8]

$\frac{|\Delta Q|}{\Delta P}+\frac{\pi}{4}\frac{a\Delta\sigma}{\Delta
P}<f\;\text{.}$ (8)

We wish to find the change in interfacial shear traction moving linearly from
an unloaded state $0$ to state $1$ [$P_{1},Q_{1},\sigma_{1}$]111For
simplicity, we will omit the subscript for the load state from here on., see
Figure 2, under conditions which ensure full stick, i.e. inequality (8) is
satisfied. Suppose we also know, from the contact geometry, the contact law,
$a=g(P)$, and we denote the contact pressure distribution at normal load $P$
by $p(x,P)$. The contact pressure at state 1 is given by integrating Eq. (6),

$p(x)=\frac{1}{\pi}\int_{P_{x}}^{P}\frac{\mathrm{d}\tilde{P}}{\sqrt{a^{2}-x^{2}}}\;\text{,}$
(9)

where $g(P_{x})=x$. We make the change of variable $\tilde{a}=g(\tilde{P})$ in
the integrand, which gives

$p(x)=\frac{1}{\pi}\int_{x}^{a}\frac{1}{g^{{}^{\prime}}(\tilde{P})}\frac{\mathrm{d}\tilde{a}}{\sqrt{\tilde{a}^{2}-x^{2}}}\;\text{,}$
(10)

where a prime indicates differentiation with respect to argument.

Now we consider the developing shear tractions and study first, for
simplicity, the effect of a simultaneously exerted shear force alone. Then,
from Eq. (8), we require the change in normal and shear load to be

$\frac{\Delta Q}{\Delta P}=\frac{Q}{P}=\lambda<f\;\text{,}$ (11)

where the $\Delta$ terms indicate the change in load from state $0$ and $1$.
Hence

$\Delta q(x)=\frac{\lambda\Delta P}{\pi\sqrt{a^{2}-x^{2}}}\;\text{,}$ (12)

so that, if we applied a shear force only,

$q(x,P)=\lambda p(x,P)\;\text{.}$ (13)

If we introduce a differential bulk tension as well, so that both develop
concurrently (satisfying the ‘proportional loading’ property), and set

$\eta=\frac{a\Delta\sigma}{\Delta P}=\frac{a\sigma}{P}\;\text{,}$ (14)

($\Delta\sigma\equiv\sigma$ and $\Delta P\equiv P$ as we start from an
unloaded state), we see that Eq. (12) is now replaced by

$\Delta q(x)=\frac{\lambda\Delta P}{\pi\sqrt{a^{2}-x^{2}}}+\frac{x\eta\Delta
P}{4a\sqrt{a^{2}-x^{2}}}\;\text{.}$ (15)

Thus after integrating Eq. 15 with respect to $P$, we find that

$q(x,P)=\int_{P_{x}}^{P}\left[\frac{\lambda}{\pi\sqrt{a^{2}-x^{2}}}+\frac{x\eta}{4a\sqrt{a^{2}-x^{2}}}\right]\mathrm{d}\tilde{P}=\left[\frac{\lambda}{\pi}+\frac{x\eta}{4a}\right]\int_{x}^{a}\frac{\mathrm{d}\tilde{a}}{g^{{}^{\prime}}(\tilde{P})\sqrt{\tilde{a}^{2}-x^{2}}}\;\text{,}$
(16)

and, by comparison with Eq. (10), we find the traction ratio as

$\frac{q(x,P)}{p(x,P)}=\lambda+\frac{\eta\pi}{4a}x\;\text{,}$ (17)

which clearly reduces to Eq. (13) in the case that the bulk tension difference
is negligible.

As expected, the shear traction is related to a scaled form of the contact
pressure. Thus, given output data for $p(x)$ and $q(x)$ from a finite element
simulation or otherwise, we are able to find the implied values of
$\Delta\sigma$ and $\Delta Q$, or $\sigma$ and $Q$ by considering the traction
ratio across the contact interface, Eq. (17). We shall illustrate this in
detail in the upcoming examples.

We now turn our attention to finding the sum of the bulk stresses,
$\sigma_{\text{A}}+\sigma_{\text{B}}$, and start by writing down the total
surface strains arising along the contact interface in the presence of a
normal load, $P$, a shear force, $Q$, differential tension,
$\sigma=\sigma_{\text{A}}-\sigma_{\text{B}}$, and the average remote bulk
loads, $\nicefrac{{\sigma_{\text{A}}+\sigma_{\text{B}}}}{{2}}$,

$\varepsilon_{xx}(x)=\varepsilon_{xx}^{P}(x)+\varepsilon_{xx}^{Q}(x)+\varepsilon_{xx}^{\sigma}+\varepsilon_{xx}^{\sigma_{\text{A}}+\sigma_{\text{B}}}(x)\;,\;-a<x<a\text{.}$
(18)

The contribution to the total direct strain from a normal force, $P$, can be
written as

$\varepsilon_{xx}^{P}(x)=\frac{\left(1-2\nu\right)}{(1-\nu)\,E^{*}}\,p(x)\;,-a<x<a\;\text{,}$
(19)

where $E^{*}=E/\left(1-\nu^{2}\right)$, $E$ is Young’s modulus and $\nu$ is
Poisson’s ratio. As we have found a solution under no-slip conditions, we can
also find the implied surface strains resulting from the effects of an applied
shear force, $Q$, and the shear tractions arising from differential bulk
tension, $\sigma$, which gives

$\varepsilon_{xx}^{Q}(x)=-\frac{2}{\pi
E^{*}}\,\lambda\,\int_{-a}^{a}\frac{\displaystyle{\,p\left(\xi\right)}}{x-\xi}\mathrm{d}\xi\;,-a<x<a\;\text{,}$
(20)

and

$\varepsilon_{xx}^{\sigma}(x)=-\frac{2}{\pi
E^{*}}\,\frac{\eta\pi}{4a}\,\int_{-a}^{a}\frac{\displaystyle{p\left(\xi\right)\,\xi}}{x-\xi}\,\mathrm{d}\xi\;,-a<x<a\;\text{.}$
(21)

The total strain distribution within the fully stuck contact will therefore
have four contributions, viz. the three just derived
$\varepsilon_{xx}^{P}(x),\varepsilon_{xx}^{Q}(x),\varepsilon_{xx}^{\sigma}(x)$
together with the effect of the average remote bulk loads, i.e.

$\varepsilon_{xx}^{\sigma_{\text{A}}+\sigma_{\text{B}}}\left(x\right)=\frac{\sigma_{\text{A}}+\sigma_{\text{B}}}{2E^{*}}\;\text{.}$
(22)

Hence, given data for the contact pressure, $p(x)$, from finite element
results or otherwise, we can now calculate the individual strain contributions
given by Eqs. (19, 20, 21). A comment is appropriate in relation to the
evaluation of the integral. We will require the surface strains within the
contact ($-a\leq x\leq a$) so that the integral is interpreted in the Cauchy
principal value sense. It will therefore be appropriate to use a numerical
integration scheme to evaluate Eqs. (20, 21), such as Gauss-Chebyshev [9]. In
order to implement this the integrand should be evaluated at the prescribed
integration points. Therefore either the mesh used in the finite element
analysis should be chosen so that the nodes lie at the desired coordinates or
an appropriate interpolation scheme might be used, such as a variant of that
developed by Krenk [10]. As we have seen, the total direct surface strain is a
combination of the strains arising from the tractions and those emanating
directly from the sum of the bulk stresses, and so the latter may be found as

$\sigma_{\text{A}}+\sigma_{\text{B}}=2E^{*}\left[\varepsilon_{xx}(x)-\varepsilon_{xx}^{P}(x)-\varepsilon_{xx}^{Q}(x)-\varepsilon_{xx}^{\sigma}\right]\text{.}$
(23)

In summary, the following steps are needed in order to find the values of $P$,
$Q$, $\sigma_{\text{A}}$ and $\sigma_{\text{B}}$ under proportional loading of
a symmetric indenter in the absence of a moment:

  1. (i)

Find values for the contact tractions $p(x)$ and $q(x)$ from simulations or
otherwise.

  2. (ii)

The contact resultants, $P$ and $Q$, are found from integrating the tractions
using Eqs. (1, 3).

  3. (iii)

The gradient of the linear function
$\nicefrac{{q(x,P)}}{{p(x,P)}}=\lambda+\frac{\eta\pi}{4a}x$ reveals the value
of $\sigma=\sigma_{\text{A}}-\sigma_{\text{B}}$, using Eq. (14).

  4. (iv)

Evaluate the individual contributions to the direct strain,
$\varepsilon_{xx}$, using Eqs. (19, 20, 21) and subtract those from the
numerical output for the total strain to determine the sum of the bulk
stresses using Eq. (23).

  5. (v)

Finally, output
$\sigma_{\text{A}}=\nicefrac{{\sigma}}{{2}}+\nicefrac{{\sigma_{\text{A}}+\sigma_{\text{B}}}}{{2}}$
and
$\sigma_{\text{B}}=\nicefrac{{\sigma_{\text{A}}+\sigma_{\text{B}}}}{{2}}-\nicefrac{{\sigma}}{{2}}$.

We thus have all the required information in the matrix system Eq. (4): given
inputs $F_{C}$ and $T$, we output $p(x)$ and $q(x)$ and are then able to
follow (i)-(v) to recover the loads necessary for an equivalent half-plane
analysis.

One of the advantages of considering the simplified case in which we neglect a
moment and any geometrical asymmetry is that it is often possible to find
closed-form solutions for the equivalent half-plane model, so we are able to
illustrate this methodology for cases in which we already know the
relationships in Eq. (4), which we shall pursue in §2.1. We shall then
conclude our discussion of the symmetric problem by discussing an alternative
to step (iii) in the methodology that will be of great value to our extension
to problems involving a moment or asymmetry in §3.

###  2.1 Demonstration of the methodology for a Hertzian contact

The motivation behind developing this method is to be able to find the remote
stresses applied, which in many real problems such as that shown in Figure 1
a) are difficult to interpret. However, in order to illustrate the methodology
described by the steps (i)-(v), we shall, as a first example, forgo the wider
geometry that leads to the contact as illustrated in Figure 1 a) and consider
the well-known example of a Hertzian contact of two elastically-similar
cylinders. We perform finite element simulations of the configuration
illustrated in Figure 3 to output the contact tractions $p(x)$ and $q(x)$.
Obviously, this is a contrived problem where we actually know the desired
values $P$, $Q$, $\sigma_{A}$ and $\sigma_{B}$, but we will recover the remote
stresses from the finite element output for the tractions, $p(x)$ and $q(x)$,
and the direct surface strain, $\varepsilon_{xx}$, alone, as if
$\sigma_{\text{A}}$ and $\sigma_{\text{B}}$ were unknowns, i.e. we shall
follow steps (iii)-(v) algorithmically to recover the correct values of the
bulk stresses. Generally, the finite element output alone, together with an
appropriate numerical scheme, suffices to solve the problem, and the closed-
form results, if tractable, serve as a point of comparison and simplifies the
calculation.

Figure 3: FE-Model of two half-planes connected by a Hertzian-type geometry
subject to loads $P$, $Q$, $\sigma_{\text{A}}$, and $\sigma_{\text{B}}$.

Figure 3 shows the input configuration for our finite element simulations. We
consider two half-planes connected by a Hertzian-type geometry with radius,
$R$, over a contact patch spanning $[-a,\quad a]$. Body B is constrained in
the vertical direction such that the normal load is supported and shear
loading in the lateral direction is balanced between the two bodies. We
thereby acknowledge that the half-planes are only approximated as each body is
finite in size, where the proportion of contact half-width to width or height
of the two bodies is less than $1/100$. The four loads, $P$, $Q$,
$\sigma_{\text{A}}$, and $\sigma_{\text{B}}$ are applied in proportion from an
unloaded state under a fully adhered conditions until they reach the desired
load state, as shown in Figure 2.

We use the commercial finite element solver Abaqus to solve the contact
problem with quadratic quadrilateral and triangular elements under plane
strain conditions with a mesh refinement toward the contact interface so that
the element size to contact size ratio is less than $1/1000$. The contact
interface is meshed such that nodes are coincidental. The contact is modelled
as a surface-to-surface interaction with default properties for normal and
tangential behaviour suppressing relative lateral motion of the two bodies.

Figure 4: Normal and shear tractions for a Hertzian geometry under
proportional loading in fully adhered conditions.

The contact tractions extracted from the finite element simulation are shown
in Figure 4, where the input values of $P$, $Q$ are such that $\lambda=0.347$.
The input for the remote tensions is and
$\sigma_{\text{A}}=80\nicefrac{{\mathrm{N}}}{{\mathrm{mm^{2}}}}$ and
$\sigma_{\text{B}}=180\nicefrac{{\mathrm{N}}}{{\mathrm{mm^{2}}}}$ giving
$\eta=-0.08$. Thus, our aim is now to reproduce these by following step (iii).

Given $P$, the contact law $a=g(P)$ for a Hertzian geometry is given by [11]

$a=\left(\frac{8PR}{\pi E^{*}}\right)^{\frac{1}{2}}.$ (24)

Thus, for the input value of $P$ and with
$R=100\mathrm{mm},E^{*}=225\;10^{3}\nicefrac{{\mathrm{N}}}{{\mathrm{mm^{2}}}}$,
the contact size at state $1$ as defined in Figure 2 is $a=1.4\mathrm{mm}$. We
now plot the ratio $q(x)/p(x)$ in Figure 5 and find a linear fit, which
referring back to (iii) finds that

$\lambda=0.35\quad\mbox{and}\quad\frac{\eta\pi}{4a}=-0.05\mathrm{mm^{-1}}\Rightarrow\eta=-0.09\Rightarrow\sigma\approx-111\nicefrac{{\mathrm{N}}}{{\mathrm{mm^{2}}}}.$
(25)

As is evident, we are reliably extracting plausible values of $\lambda$ and
$\eta$ from step (iii).

Figure 5: Traction ratio between shear and normal tractions for a Hertzian
geometry under proportional loading in fully adhered conditions.

To recover the sum of the bulk stresses, we follow step (iv) and consider the
direct strain in the contact interface. We can output the total strain
$\varepsilon_{xx}(x)$ directly from the finite element simulations. To
evaluate $\sigma_{A}+\sigma_{B}$, we also need the individual contributions
$\varepsilon_{xx}^{P}(x),\varepsilon_{xx}^{Q}(x),\varepsilon_{xx}^{\sigma}(x)$.
While these can be calculated numerically from the given data for $p(x)$, we
note that, for this simple example, we can in fact evaluate them in closed
form. By Eq. (10), the pressure distribution is given by

$p(x)=\frac{E^{*}}{4R}\int_{x}^{a}\frac{\tilde{a}\,\mathrm{d}\tilde{a}}{\sqrt{\tilde{a}^{2}-x^{2}}}=\frac{E^{*}}{4R}\sqrt{a^{2}-x^{2}}\;\text{,}$
(26)

so that directly evaluating Eq. (19) yields

$\varepsilon_{xx}^{P}(x)=\frac{\left(1-2\nu\right)}{(1-\nu)\,E^{*}}\,p(x)=\frac{\left(1-2\nu\right)}{4R\,(1-\nu)}\,\sqrt{a^{2}-x^{2}}\;\text{,}$
(27)

while Eq. (20) gives

$\varepsilon_{xx}^{Q}(x)=-\frac{2}{\pi
E^{*}}\,\lambda\,\int_{-a}^{a}\frac{\displaystyle{\,p\left(\xi\right)}}{x-\xi}\mathrm{d}\xi=-\frac{1}{2R}\frac{Q}{P}\,x\;\text{,}$
(28)

and Eq. (21) gives

$\varepsilon_{xx}^{\sigma}(x)=-\frac{2}{\pi
E^{*}}\,\frac{\eta\pi}{4a}\,\int_{-a}^{a}\frac{\displaystyle{p\left(\xi\right)}}{x-\xi}\,\xi\,\mathrm{d}\xi=-\frac{1}{16R}\frac{\sigma}{P}\left(a^{2}-2x^{2}\right)\;\text{.}$
(29)

Thus, we find that

$\sigma_{\text{A}}+\sigma_{\text{B}}=2E^{*}\left(\varepsilon_{xx}(x)+\frac{1}{2R}\frac{Q}{P}\,x+\frac{1}{16R}\frac{\sigma}{P}\left(a^{2}-2x^{2}\right)-\frac{\left(1-2\nu\right)}{4R\,(1-\nu)}\,\sqrt{a^{2}-x^{2}}\right)\;\text{,}$
(30)

where $|x|<a$. Figure 6 illustrates that Eq. (30) yields a constant value
using the analytical description and an almost constant value of
$\sigma_{\text{A}}+\sigma_{\text{B}}\approx
260\nicefrac{{\mathrm{N}}}{{\mathrm{mm^{2}}}}$ using the finite element output
for the direct strain. The slight curvature in the numerical result might stem
from the half-plane approximation in the finite element model, see Figure 3.

Figure 6: Sum of the bulk stresses, $\sigma_{\text{A}}+\sigma_{\text{B}}$,
obtained using Eq. (30) and finite element output for the direct strain,
$\varepsilon_{xx}(x)$.

Therefore, combining our values for $\eta$ and
$\sigma_{\text{A}}+\sigma_{\text{B}}$, we finally back out the values

$\sigma_{\text{A}}\approx
74\nicefrac{{\mathrm{N}}}{{\mathrm{mm^{2}}}}\;\mbox{and}\;\sigma_{\text{B}}\approx
186\nicefrac{{\mathrm{N}}}{{\mathrm{mm^{2}}}}$ (31)

from step (v), which again shows good agreement with the (for this example
known) inputs.

With this example, we demonstrated that it is possible to recover all external
load inputs, $P$, $Q$, $\sigma_{\text{A}}$, and $\sigma_{\text{B}}$, from the
tractions and direct strains within the contact interface alone. In §3, we
will show how this methodology may be adapted to find answers to the same
questions in more complicated problems, for instance when a moment is present
or for redundant systems such as the dovetail geometry for gas turbine fan
blade joints.

###  2.2 Using asymptotes in step (iii)

In most real problems, we will not know the contact geometry in a simple form,
but the steps (i)-(v) can still be employed. We conclude by reconsidering step
(iii). Given numerical results for $p(x)$ and $q(x)$, there are basically two
routes which may be followed. The most obvious method is to plot the ratio
$\nicefrac{{q(x)}}{{p(x)}}$ across the length of the contact from the finite
element results directly, as we did in §2.1. This ratio should be linear in
$x$ and have an intercept giving the value of $\lambda$ and the gradient
$\nicefrac{{\eta\pi}}{{4a}}$. This is the preferred method as it applies
averaging of the given numerical results.

An alternative method, whose extension to the asymmetric case will be
exploited in the subsequent section, is to look in detail at the traction-
ratio towards the ends of the contact, where the contact edge values are given
by

$\frac{q(a^{-})}{p(a^{-})}=\lambda+\frac{\eta\pi}{4}=\frac{Q}{P}+\frac{\pi
a\sigma}{4P}\;\text{,}$ (32)

and

$\frac{q(-a^{+})}{p(-a^{+})}=\lambda-\frac{\eta\pi}{4}=\frac{Q}{P}-\frac{\pi
a\sigma}{4P}\;\text{.}$ (33)

If we move the origin to the contact edge and set $s=a+x$, $t=a-x$, we can
express the normal and shear tractions near the contact edges in asymptotic
form as [12]

$p(s)\approx K_{P}^{-}\sqrt{s}\;\text{,}\qquad p(t)\approx
K_{P}^{+}\sqrt{t}\;\text{,}$ (34) $q(s)\approx
K_{Q}^{-}\sqrt{s}\;\text{,}\qquad q(t)\approx K_{Q}^{+}\sqrt{t}\;\text{,}$
(35)

and as $s,t\rightarrow 0$, the ratio is given as

$\frac{q(-a^{+})}{p(-a^{+})}=\frac{K_{Q}^{-}}{K_{P}^{-}}\;\text{,}$ (36)
$\frac{q(a^{-})}{p(a^{-})}=\frac{K_{Q}^{+}}{K_{P}^{+}}\;\text{.}$ (37)

Thus, we intend to find the multipliers
($K_{P}^{-},K_{P}^{+},K_{Q}^{-},K_{Q}^{+}$) from the finite element analysis
output by plotting the ratios
$\nicefrac{{p(s)}}{{\sqrt{s}}},\nicefrac{{q(s)}}{{\sqrt{s}}}$ and taking the
limit $s\rightarrow 0$, and similarly at the other end point. We can then find
($Q,\sigma$) from equations (32) and (33), fulfilling step (iii). The other
steps can be pursued as discussed previously.

##  3 Problems when a moment is developed

We now turn to problems which involve the effects of asymmetry, which may stem
from an externally applied moment or an inherent asymmetry of the contacting
geometry. As we shall see, it is not possible to write down a closed-form
expression for the traction ratio as we did for the symmetric case in Eq. (17)
and we will need to make use of an asymptotic description of the ratio near
the contact edges similar to that outlined in §2.2 in order to find the
differential tension, $\sigma=\sigma_{\text{A}}-\sigma_{\text{B}}$. The other
steps, however, translate directly into the asymmetric regime.

Suppose that we have a contact spanning [-$a\qquad b$], and we make a small
change, $\Delta P$ in normal load together with a small change in applied
moment, $\Delta M$. The corresponding change in contact pressure is given by
[13]

$\Delta p(x)=\frac{\Delta
P}{\pi\sqrt{\left(a+x\right)\left(b-x\right)}}+\frac{4\Delta
M\left(2x+a-b\right)}{\pi\left(a+b\right)^{2}\sqrt{\left(a+x\right)\left(b-x\right)}}\;\text{.}$
(38)

If, at the same time, there are small changes in shear force, $\Delta Q,$ and
differential bulk tension, $\Delta\sigma,$ the change in shear traction,
$\Delta q(x),$ generated is given by

$\Delta q(x)=\frac{\Delta
Q}{\pi\sqrt{\left(a+x\right)\left(b-x\right)}}+\frac{\Delta\sigma\left(2x+a-b\right)}{8\sqrt{\left(a+x\right)\left(b-x\right)}}\;\text{.}$
(39)

The contact law is now given by $a=a(P,M)$ and $b=b(P,M)$. Therefore

$\mathrm{d}a=\frac{\partial a}{\partial P}\mathrm{d}P+\frac{\partial
a}{\partial M}\mathrm{d}M\text{,}\quad\mathrm{d}b=\frac{\partial b}{\partial
P}\mathrm{d}P+\frac{\partial b}{\partial M}\mathrm{d}M\text{.}$ (40)

We demand a proportional load path (Eq. (5)), i.e.
$\mathrm{d}M/M=\mathrm{d}P/P$, so that

$\mathrm{d}a=\left[\frac{\partial a}{\partial P}+\frac{\partial a}{\partial
M}\frac{\mathrm{d}M}{\mathrm{d}P}\right]\mathrm{d}P=\left[\frac{\partial
a}{\partial P}+\frac{\partial a}{\partial
M}\left(\frac{M}{P}\right)\right]\mathrm{d}P\;\text{,}$ (41)
$\mathrm{d}b=\left[\frac{\partial b}{\partial P}+\frac{\partial b}{\partial
M}\frac{\mathrm{d}M}{\mathrm{d}P}\right]\mathrm{d}P=\left[\frac{\partial
b}{\partial P}+\frac{\partial b}{\partial
M}\left(\frac{M}{P}\right)\right]\mathrm{d}P\;\text{,}$ (42)

and hence, in place of equation (9), we now have

$p(x)=\frac{1}{\pi}\int_{P_{x}}^{P}\left[\frac{1}{\sqrt{\left(a+x\right)\left(b-x\right)}}+\frac{4\left(2x+a-b\right)}{\left(a+b\right)^{2}\sqrt{\left(a+x\right)\left(b-x\right)}}\frac{\mathrm{d}M}{\mathrm{d}\tilde{P}}\right]\mathrm{d}\tilde{P}\;\text{,}$
(43)

and proportional loading also requires that
$\mathrm{d}P/P=\mathrm{d}Q/Q=\mathrm{d}\sigma/\sigma$ giving

$q(x)=\int_{P_{x}}^{P}\left[\frac{1}{\pi\sqrt{\left(a+x\right)\left(b-x\right)}}\frac{\mathrm{d}Q}{\mathrm{d}\tilde{P}}+\frac{\left(2x+a-b\right)}{8\sqrt{\left(a+x\right)\left(b-x\right)}}\frac{\mathrm{d}\sigma}{\mathrm{d}\tilde{P}}\right]\mathrm{d}\tilde{P}\;\text{.}$
(44)

Following the symmetric case from §2, we would like to reveal the ratio

$\frac{q(x)}{p(x)}=\frac{\displaystyle{\int_{P_{x}}^{P}\left[\frac{1}{\sqrt{\left(a+x\right)\left(b-x\right)}}\frac{\mathrm{d}Q}{\mathrm{d}\tilde{P}}+\frac{\pi\left(2x+a-b\right)}{8\sqrt{\left(a+x\right)\left(b-x\right)}}\frac{\mathrm{d}\sigma}{\mathrm{d}\tilde{P}}\right]\mathrm{d}\tilde{P}}}{\displaystyle{\int_{P_{x}}^{P}\left[\frac{1}{\sqrt{\left(a+x\right)\left(b-x\right)}}+\frac{4\left(2x+a-b\right)}{\left(a+b\right)^{2}\sqrt{\left(a+x\right)\left(b-x\right)}}\frac{\mathrm{d}M}{\mathrm{d}\tilde{P}}\right]\mathrm{d}\tilde{P}}}\;\text{,}$
(45)

but it is difficult to proceed with the integrals on the right-hand side. In a
real problem we will know both $p(x)$ and $q(x)$ together with

$\frac{Q}{P}=\,\lambda,\quad\frac{M}{aP}=\,\gamma\;\text{,}$ (46)

from the force resultants Eq. (1, 2, 3), where $a$ has been chosen arbitrarily
as means of normalisation. It is the quantity

$\displaystyle\frac{a\,\sigma}{P}=$ $\displaystyle\,\eta\;\text{,}$ (47)

and hence $\sigma$, that we seek. Although we cannot evaluate Eq. (45) for a
general point in closed-form, we are able to find the limits of the traction
ratio at each end of the contact, where $\eta$ is the only remaining unknown.
The results, which are generalisations of those found in §2.2, are

$\frac{q(-a^{+})}{p(-a^{+})}=\frac{1}{8}\frac{\left(1+\left(\frac{b}{a}\right)\right)^{2}\left[8\lambda-\pi\eta\left(1+\frac{b}{a}\right)\right]}{\left(1+\left(\frac{b}{a}\right)\right)^{2}-4\gamma\left(1+\frac{b}{a}\right)}\;\text{,}$
(48)
$\frac{q(b^{-})}{p(b^{-})}=\frac{1}{8}\frac{\left(1+\left(\frac{b}{a}\right)\right)^{2}\left[8\lambda+\pi\eta\left(1+\frac{b}{a}\right)\right]}{\left(1+\left(\frac{b}{a}\right)\right)^{2}+4\gamma\left(1+\frac{b}{a}\right)}\;\text{,}$
(49)

where details of finding the limits can be found in the Appendix B. We note
that we actually only need one of Eq. (48) and Eq. (49) to determine $\eta$
and hence $\sigma$. In practice, however, the second equation serves as a
excellent check on the accuracy of the results.

All that remains is to find the sum of the bulk stresses,
$\sigma_{\text{A}}+\sigma_{\text{B}}$, which can be evaluated by considering
the direct strain in the contact interface, as previously. The only small
change is somewhat notational, since the presence of a moment means that we
replace the term $\varepsilon_{xx}^{P}(x)$ in Eq. (19) by
$\varepsilon_{xx}^{P,M}(x)$.

Therefore, in summary, when there is asymmetry in the problem, the steps
necessary to find the values of $P$, $M$, $Q$, $\sigma_{\text{A}}$ and
$\sigma_{\text{B}}$ in the presence of a moment can be summarised as follows.

  1. (I)

Find values for the contact tractions $p(x)$ and $q(x)$ from simulations or
otherwise.

  2. (II)

The contact resultants for $P$, $M$, and $Q$ are found from integrating the
tractions, using Eqs. (1, 2, 3).

  3. (III)

Obtain the near edge multipliers ($K_{P}^{-},K_{P}^{+},K_{Q}^{-},K_{Q}^{+}$)
from the finite element output by plotting the ratios
$\nicefrac{{p(s)}}{{\sqrt{s}}},\nicefrac{{q(s)}}{{\sqrt{s}}}$ and taking the
limit $s\rightarrow 0$, and similarly at the other end point.

  4. (IV)

Use the ratios of the asymptotic multipliers in order to obtain the left-hand
sides of Eq. (48) and (49), and, hence, a way of determining the value of
$\eta$ and thus $\sigma=\sigma_{\text{A}}-\sigma_{\text{B}}$.

  5. (V)

Evaluate the individual contributions to the direct strain,
$\varepsilon_{xx}$, using Eqs. (19, 20, 21) and subtract those from the
numerical output for the total strain to determine the sum of the bulk
stresses using Eq. (23).

  6. (VI)

Finally, output
$\sigma_{\text{A}}=\nicefrac{{\sigma}}{{2}}+\nicefrac{{\sigma_{\text{A}}+\sigma_{\text{B}}}}{{2}}$
and
$\sigma_{\text{B}}=\nicefrac{{\sigma_{\text{A}}+\sigma_{\text{B}}}}{{2}}-\nicefrac{{\sigma}}{{2}}$.

This algorithm gives the values of the loads necessary to formulate an
equivalent half-plane representation of the full contact problem (cf. Figure
1). We shall now present an example that illustrates the effectiveness of this
methodology.

###  3.1 Dovetail geometry example

We return to the example of a fan blade dovetail root contact displayed in
Figure 1 a), and examine the implementation of a numerical analysis in the
complex scenario of this prototype problem. The joint is commonly used to
connect fan blades to the rotor disk in jet engines. As the engine starts
spinning, centrifugal and expansion forces, $F_{\text{C}}$ and $T$, will act
on the blade and disk, respectively. Note that, here, we consider a
symmetrical configuration for the sake of simplicity (i.e. $F_{V}=0$), while
in reality a vibrational component may act on the blade thereby inducing an
asymmetry about the centre-axis of the blade and a steady state variation of
the local load components we are interested in.

Figure 7: Finite element model of a dovetail prototype subject to external
loads $F_{\text{C}}$ and $T$ and the local response.

The system, as depicted in Figure 7, is statically indeterminate. The
equilibrium equations are insufficient to determine the local forces and
reactions. As explained in the introduction, the finite element analysis of
the structure allows us to connect the external loads to the local response
where a matrix with complicated, yet unknown entries, makes this connection,
see Eq. (4). Our algorithm (I)-(VI) allows us to determine these local forces.

The finite element model consists of a disk and a blade-representation with
symmetry conditions exploited about the vertical centre-axis and the disk
supports forces in the vertical direction. As required by the method
presented, the external loads, $F_{\text{C}}$ and $T$, are applied
proportionally from an unloaded state under fully adhered conditions until
they reach the desired load state. The structure is meshed with a gradual mesh
refinement toward the contact interface and all other properties are the same
as in the model used in §2.1.

For comparison, and in order to verify that the same contact behaviour is
found in a half-plane representation of the dovetail root contact, we use the
local loads found above to initiate an interface between two half-planes
connected by a flat and rounded punch where the contacting shape is the same
as the corresponding function for the dovetail root contact, see Figure 8. In
the half-plane representation, we expect to observe the same contact
behaviour, up to a reasonable degree of accuracy, as in the prototype example.
This means the normal and shear tractions, as well as the direct strain in the
contact interface, ought to match between the two problems.

Figure 8: Finite element model of two half-planes connected by a flat and
rounded geometry subject to loads $P$, $M$, $Q$, $\sigma_{\text{A}}$, and
$\sigma_{\text{B}}$. Figure 9: Comparison between finite element outputs for
prototype, Figure 7, and the half-plane representation, Figure 8, for a) the
normal pressure distribution, $p(x)$, b) the shear traction distribution,
$q(x)$, and c) the direct strain, $\varepsilon_{xx}(x)$, along the interface.

The contact pressure and shear tractions calculated using the two finite
element models are displayed in Figures 9 a) and b) as the black lines. It is
trivial to use these to find the resultants $P$, $M$ and $Q$ using Eqs.
(1)–(3). To determine $\sigma_{\text{A}}$ and $\sigma_{\text{B}}$, since
$M\neq 0$, we follow (III)-(VI) in using the edge asymptotes introduced in §3
to find the difference in bulk tensions, $\sigma$, and then use the direct
strain, $\varepsilon_{xx}$, within the contact interface to reveal the sum of
the bulk stresses, see Figure 9 c).

The prototype and half-plane representation differ from one another in terms
of contact initialisation and mesh refinement. This may cause a slight
difference in the quality of numerical results between the two.
Notwithstanding, the normal tractions and contact extent shown in Figure 9
match up very well between prototype and half-plane. This indicates that the
normal load, $P$, and moment, $M$, have been found accurately from the
prototype.

The shear tractions are affected by the four load components, normal load,
moment, shear load, and differential bulk tension. The comparison of shear
tractions shown in Figure 9 b) and the direct strain in Figure 9 c) gives
reassurance that the values derived for $Q$, $\sigma_{\text{A}}$ and
$\sigma_{\text{B}}$ agree very well indeed between prototype and half-plane
representation.

##  4 Conclusion

We have developed an algorithm which enables practical contact problems that
are analysed, usually, by a finite element procedure using commercial code, to
be matched to half-plane idealisations. We set ourselves the task of finding
the best choices for the contact loads present in a prototype, that is the
normal force $P$, shear force $Q$, moment $M$, and bulk stresses
$\sigma_{\text{A}},\sigma_{\text{B}}$. While exploiting equilibrium conditions
to find the traction resultants $P$, $Q$ and $M$ is trivial, of particular
note is the problem in devising the best-match for the bulk stresses acting
parallel with the surface of the contact, $\sigma_{\text{A}}$ and
$\sigma_{\text{B}}$, whose difference is material in modifying the slip
pattern, but which are separately responsible for propelling cracks. We were
able to extract these values by considering the traction ratio within the
contact as well as the direct strains, which gave us two expressions for the
difference and sum of the bulk stresses respectively. With the extracted
values to hand, we are then able to formulate a half-plane problem to
approximate the full prototype behaviour.

We considered two specific examples to illustrate the methodology. The first
neglected all asymmetry (thus precluding a moment) and looked at the slightly
academic set-up of a Hertzian prototype, in which the results are generally
obtainable in closed form by other means. However, we were able to show that
the algorithm was robust, finding excellent agreement with the expected values
of $P$, $Q$, $\sigma_{\text{A}}$ and $\sigma_{\text{B}}$. The second example
considered a prototype dovetail root geometry, in which there is a moment and
formulating an equivalent half-plane problem is non-trivial. We used the
algorithm to derive the contact loads, and were able to demonstrate that the
methodology is again robust.

Thus, given the results of commercial code, we are able to devise an
equivalent problem using half-plane theory with suitable loads applied that
can be used to analyse finer details of the contact problem that are poorly
resolved by the finite element analysis, for example the regions of slip at
the contact edges. These results can then be used to set up a much simpler
laboratory fretting fatigue test mimicking the prototype contact behaviour
with the highest fidelity possible.

## Acknowledgements

This project has received funding from the European Union’s Horizon 2020
research and innovation programme under the Marie Sklodowska-Curie agreement
No 721865. David Hills thanks Rolls-Royce plc and the EPSRC for the support
under the Prosperity Partnership Grant ’Cornerstone: Mechanical Engineering
Science to Enable Aero Propulsion Futures’, Grant Ref: EP/R004951/1. The
authors would like to thank the comments of the anonymous referees that helped
improve upon a previous version of this manuscript.

## References

  * [1] J. Barber, Contact Mechanics, Springer, 2018.
  * [2] N. Banerjee, D. Hills, D. Dini, The derivation and application of a semi-infinite flat and rounded asymptotic frictionless contact, Int. Jnl. Mech. Sci. 51 (2009) 662–666.
  * [3] C. Cattaneo, Sul Contato di Due Corpo Elastici, Atti Accad. Naz. Lincei, Cl. Sci. Fis., Mat. Nat., Rend. 27 (1938) 342–348, 434–436, 474–478.
  * [4] R. Mindlin, Compliance of elastic bodies in contact, AASME Trans. Jnl. Appl. Mech. 16 (1949) 259–268.
  * [5] H. Andresen, D. Hills, J. Barber, J. Vázquez, Steady state cyclic behaviour of a half-plane contact in partial slip subject to varying normal load, moment, shear load, and moderate differential bulk tension, Int. Jnl. Solids Struct. 182 – 183 (2020) 156 – 161.
  * [6] M. R. Moore, R. Ramesh, D. Hills, J. Barber, Half-plane partial slip contact problems with a constant normal load subject to a shear force and a differential bulk tension, Jnl. Mech. Phys. Solids 118 (2018) 245–253.
  * [7] M. R. Moore, D. Hills, Solution of half-plane contact problems by distributing climb dislocations, Int. Jnl. Solids Struct. 147 (2018) 61–66.
  * [8] D. Hills, M. Davies, J. Barber, An incremental formulation for half-plane contact problems subject to varying normal load, shear, and tension, Jnl. Strain Analysis 46 (2011) 436–443.
  * [9] D. Hills, P. Kelly, D. Pai, A. Korsunsky, Solution of Crack Problems - The Distributed Dislocation Technique, Springer-Science+Media-Business, B. V., 1996\.
  * [10] S. Krenk, On the use of the interpolation polynomial for solutions of singular integral equations, Quarterly of Applied Mathematics 32 (4) (1975) 479–484.
  * [11] H. Hertz, Über die Berührung fester elastischer Körper, Journal für die reine und angewandte Mathematik 92 (1881) 156–171.
  * [12] R. Fleury, D. Hills, R. Ramesh, J. Barber, Incomplete contacts in partial slip subject to varying normal and shear loading, and their representation by asymptotes, Int. Jnl. Mech. Phys. Solids 99 (2017) 178–191.
  * [13] A. Sackfield, C. Truman, D. Hills, The tilted punch under normal and shear load (with application to fretting tests), Int. Jnl. Mech. Sci. 43 (2001) 1881–1892.
  * [14] J. Barber, Elasticity (third edition), Springer, 2010.

##  Appendix A Sequential loading

Figure 2 shows sequential and proportional loading for a $P,Q,\sigma$-problem
neglecting the effect of a moment. The sequence in which the loads are applied
greatly affects the procedure and the solution to a contact problem. For
completeness, we look at the effects of a sequential load application in this
appendix.

Consider the problem shown in Figure 1 b), and suppose that the contacting
bodies are elastically similar. We assume initially, that the coefficient of
friction, $f$, is sufficiently high to inhibit all slip. First, we apply the
normal load, $P$ and potentially a moment, $M$, and establish a contact of
extent [$-a\qquad b$]. In our analysis we will neglect the shear tractions
induced by the application of the normal load as they will be much smaller in
magnitude than the contact pressure when the bodies are elastically similar.
Now we apply the external tangential loads
($Q,\sigma_{\text{A}},\sigma_{\text{B}}$). These will be square root singular
in character at the contact edges - symmetric when induced by a shear force
and antisymmetric when induced by differential bulk tension, as shown in
Figure 10. The full-stick shear traction distribution is given by

$q(x)=\frac{Q}{\pi\sqrt{\left(a+x\right)\left(b-x\right)}}+\frac{\sigma\left(2x+a-b\right)}{8\sqrt{\left(a+x\right)\left(b-x\right)}}\;\text{,}$
(50)

where $\sigma=\sigma_{\text{A}}-\sigma_{\text{B}}$. Suppose we introduce a new
coordinate, $s=a+x$, measured from the left hand contact edge, so that

$q(s)=\frac{Q}{\pi\sqrt{s\left(b+a-s\right)}}+\frac{\sigma\left(2s-a-b\right)}{8\sqrt{s\left(b+a-s\right)}}\;\text{,}$
(51)

and now take the limit $s\rightarrow 0$, so that

$q(s)\sqrt{s}\rightarrow\frac{Q}{\pi\sqrt{\left(a+b\right)}}-\frac{\sigma(a+b)}{8\sqrt{\left(a+b\right)}}\;\text{.}$
(52)

Similarly, if we measure $t=b-x$ from the left hand edge

$q(t)=\frac{Q}{\pi\sqrt{t\left(a+b-t\right)}}+\frac{\sigma(a+b-2t)}{8\sqrt{t\left(a+b-t\right)}}\;\text{,}$
(53)

and, in the limit $t\rightarrow 0$

$q(t)\sqrt{t}\rightarrow\frac{Q}{\pi\sqrt{\left(a+b\right)}}+\frac{\sigma\left(a+b\right)}{8\sqrt{\left(a+b\right)}}\;\text{.}$
(54)

Figure 10: Fully adhered shear tractions due to a) a shear force, $Q$ and b)
differential bulk tension, $\sigma$.

Therefore, by taking the output for $q(s)$ and $q(t)$ from the finite element
analysis and plotting the left hand sides of limits (52) and (54) we can
deduce the value of $Q$ and $\sigma$. Of course, we know the value of the
shear force, $Q$, by another route, equation (3), but this provides an
independent check to our results.

We now turn our attention to a consideration of the absolute bulk tensions
present. The general expression for the strain induced at the surface of a
half-plane, under plane strain, by surface tractions, $p\left(x\right)$ and
$q\left(x\right)$, is [14]

$\varepsilon_{xx}^{p(x),q(x)}(x)=-\frac{2}{\pi
E^{*}}\int_{-a}^{b}\frac{q\left(\xi\right)\mathrm{d}\xi}{x-\xi}+\frac{\left(1-2\nu\right)}{(1-\nu)\,E^{*}}\,p(x)\;\text{,}$
(55)

where $E^{*}=E/\left(1-\nu^{2}\right)$, $E$ is Young’s modulus and $\nu$ is
Poisson’s ratio. We know the contact pressure rigorously, originating from the
normal load, $P$, and moment, $M$. In any practical problem, the total surface
strains, $\varepsilon_{xx}$, everywhere on the surface can be found using the
finite element output. The contribution from the contact pressure (and
therefore $P,M$) is given by

$\varepsilon_{xx}^{P,M}(x)=\frac{\left(1-2\nu\right)}{(1-\nu)\,E^{*}}\,p(x)\;\text{,}$
(56)

and as we have found a solution under no-slip conditions, so we can also find
the implied surface strains resulting from the effects of an applied shear
force, $Q$, and the shear tractions arising from differential bulk tension,
$\sigma$, rigorously, for once and for all, from equation (50), which gives

$\varepsilon_{xx}^{Q}(x)=-\frac{2Q}{\pi^{2}E^{*}}\int_{-a}^{b}\frac{\mathrm{d}\xi}{(x-\xi)\sqrt{\left(a+\xi\right)\left(b-\xi\right)}}=0\;,-a<x<b\;\text{,}$
(57)

and

$\varepsilon_{xx}^{\sigma}(x)=-\frac{\sigma_{\text{A}}-\sigma_{\text{B}}}{4\pi
E^{*}}\int_{-a}^{b}\frac{\left(2\xi+a-b\right)\mathrm{d}\xi}{\left(x-\xi\right)\sqrt{\left(a+\xi\right)\left(b-\xi\right)}}=-\frac{\sigma_{\text{A}}-\sigma_{\text{B}}}{4E^{*}}\;,-a<x<b\;\text{.}$
(58)

The total strain distribution within the fully stuck contact will therefore
have four contributions, viz. the three just derived
$\varepsilon_{xx}^{P,M}(x),\varepsilon_{xx}^{Q}(x),\varepsilon_{xx}^{\sigma}(x)$
together with the effect of the average remote bulk loads, i.e.

$\varepsilon_{xx}^{\sigma_{\text{A}}+\sigma_{\text{B}}}\left(x\right)=\frac{\sigma_{\text{A}}+\sigma_{\text{B}}}{2E^{*}}\;\text{,}$
(59)

giving, therefore, the sum of the bulk stresses. Notice that the results
derived here should be robust in the sense that they are averaged and do not
rely on ‘spot’ values. With sequential loading they are not geometry
dependent: under full stick conditions the shear traction distribution is
fully defined without knowing the relative profile of the contacting bodies.

##  Appendix B Leading-order asymptotic form of (43)–(44)

Consider the expression for the contact pressure for an indenter under both an
applied normal force, $P$, and an applied moment, $M$, as given by (43):

$p(x)=\frac{1}{\pi}\int_{P_{x}}^{P}\left[\frac{1}{\sqrt{\left(a+x\right)\left(b-x\right)}}+\frac{4\left(2x+a-b\right)}{\left(a+b\right)^{2}\sqrt{\left(a+x\right)\left(b-x\right)}}\frac{\mathrm{d}M}{\mathrm{d}\tilde{P}}\right]\mathrm{d}\tilde{P}.$
(60)

where $-a<x<b$. In this expression, the size of the contact set is a function
of both $P$ and $M$ with $a(P,M)=a$ and $b(P,M)=b$. We shall restrict
ourselves to the case in which the loading is proportional, and in particular,
we set

$\frac{\mbox{d}M}{\mbox{d}P}=\gamma a.$ (61)

For notational brevity, we introduce the shorthand notation

$\bar{a}(P)=a(s,\gamma as),\;\bar{b}(s)=b(s,\gamma as).$ (62)

We seek the coefficients of the square-root bounded terms as
$x\rightarrow-a^{+}$, $x\rightarrow b^{-}$. We shall consider the right-hand
contact point in detail to illustrate the methodology, but a similar analysis
follows for the left-hand contact point. Suppose

$x=b-\delta X$ (63)

where $0<\delta\ll 1$ and $X=O(1)$ strictly. Provided that $\bar{b}(P)$ is
sufficiently well-behaved so as to have a Taylor expansion there, we know that

$P_{x}=P-\delta\hat{P}\;\mbox{where}\;\hat{P}=\frac{X}{\bar{b}^{\prime}(P)}+O(\delta).$
(64)

In particular, since we shall only require the leading-order term in the
asymptotic expansion, it will be sufficient for us to assume that
$X-\hat{P}\bar{b}^{\prime}(P)=o(1)$ henceforth. In the integral, we set
$\tilde{P}=P-\delta S$. Then,

$p=\frac{\sqrt{\delta}}{\pi}\int_{0}^{\hat{P}}\left(1+\frac{4\gamma
a(b+a)}{(a+b)^{2}}+O(\delta)\right)\frac{1}{\sqrt{(a+b)+O(\delta)}}\frac{1}{\sqrt{X-S\bar{b}^{\prime}(P)+O(\delta)}}\,\mbox{d}S.$
(65)

Assuming $\delta$ is small, we can expand each term in the integrand in terms
of Taylor series, finding that

$p=\frac{\sqrt{\delta}}{\pi\sqrt{a+b}}\left(1+\frac{4\gamma
a}{(a+b)}\right)\int_{0}^{X/\bar{b}^{\prime}(P)}\frac{1}{\sqrt{X-S\bar{b}^{\prime}(P)}}\,\mbox{d}S+O(\delta^{3/2}),$
(66)

where we have used the approximation for $\hat{P}$ in (64). Thus, after
integrating and returning to the original variables, we see that

$p(x)=\frac{2}{\bar{b}^{\prime}(P)\pi\sqrt{a+b}}\left(1+\frac{4\gamma
a}{(a+b)}\right)\sqrt{b-x}+o\left(\sqrt{b-x}\right)$ (67)

as $x\rightarrow b^{-}$.

By setting $x=-a+\delta X$ where $0<\delta\ll 1$ and $X$ is strictly order
unity, a similar analysis yields

$p(x)=\frac{2}{\bar{a}^{\prime}(P)\pi\sqrt{a+b}}\left(1-\frac{4\gamma
a}{(a+b)}\right)\sqrt{a+x}+o\left(\sqrt{a+x}\right)$ (68)

as $x\rightarrow-a^{+}$.

Naturally, a similar process also holds for the shear tractions, $q(x)$, as
given by (44),

$q(x)=\int_{P_{x}}^{P}\left[\frac{1}{\pi\sqrt{\left(a+x\right)\left(b-x\right)}}\frac{\mbox{d}Q}{\mbox{d}\tilde{P}}+\frac{\left(2x+a-b\right)}{8\sqrt{\left(a+x\right)\left(b-x\right)}}\frac{d\Delta\sigma}{\mbox{d}\tilde{P}}\right]\mbox{d}\tilde{P}.$
(69)

We again assume proportional loading, defining

$\frac{\mbox{d}Q}{\mbox{d}P}=\lambda,\;\frac{\mbox{d}\Delta\sigma}{\mbox{d}P}=\frac{\eta}{a}.$
(70)

Then, an asymptotic analysis yields

$q(x)=\frac{2}{\bar{b}^{\prime}(P)\sqrt{a+b}}\left(\frac{\lambda}{\pi}+\frac{\eta(a+b)}{8a}\right)\sqrt{b-x}+o\left(\sqrt{b-x}\right)$
(71)

as $x\rightarrow b^{-}$, and

$q(x)=\frac{2}{\bar{a}^{\prime}(P)\sqrt{a+b}}\left(\frac{\lambda}{\pi}-\frac{\eta(a+b)}{8a}\right)\sqrt{a+x}+o\left(\sqrt{a+x}\right)$
(72)

as $x\rightarrow-a^{+}$.

Therefore, the desired ratios of the coefficients are given by

$\frac{q(-a^{+})}{p(-a^{+})}=\frac{1}{8}\frac{\left(1+\left(\frac{b}{a}\right)\right)^{2}\left[8\lambda-\pi\eta\left(1+\frac{b}{a}\right)\right]}{\left(1+\left(\frac{b}{a}\right)\right)^{2}-4\gamma\left(1+\frac{b}{a}\right)},$
(73)

as we approach the left-hand contact point, and

$\frac{q(b^{-})}{p(b^{-})}=\frac{1}{8}\frac{\left(1+\left(\frac{b}{a}\right)\right)^{2}\left[8\lambda+\pi\eta\left(1+\frac{b}{a}\right)\right]}{\left(1+\left(\frac{b}{a}\right)\right)^{2}+4\gamma\left(1+\frac{b}{a}\right)},$
(74)

as we approach the right-hand contact point, respectively.


#  Improving the Reusability of Pre-trained Language Models in Real-world
Applications

Somayeh Ghanbarzadeh*, Hamid Palangi$\dagger$, Yan Huang*, Radames Cruz
Moreno$\dagger$, and Hamed Khanpour$\dagger$ *Department of Computer Science
and Engineering
University of North Texas, Denton, TX
Emails<EMAIL_ADDRESS><EMAIL_ADDRESS>$\dagger$ Microsoft
Research, Redmond, WA
Emails:{hpalangi, radames.cruz<EMAIL_ADDRESS>
###### Abstract

The reusability of state-of-the-art Pre-trained Language Models (PLMs) is
often limited by their generalization problem, where their performance
drastically decreases when evaluated on examples that differ from the training
dataset, known as Out-of-Distribution (OOD)/unseen examples. This limitation
arises from PLMs’ reliance on spurious correlations, which work well for
frequent example types but not for general examples. To address this issue, we
propose a training approach called Mask-tuning, which integrates Masked
Language Modeling (MLM) training objectives into the fine-tuning process to
enhance PLMs’ generalization. Comprehensive experiments demonstrate that Mask-
tuning surpasses current state-of-the-art techniques and enhances PLMs’
generalization on OOD datasets while improving their performance on in-
distribution datasets. The findings suggest that Mask-tuning improves the
reusability of PLMs on unseen data, making them more practical and effective
for real-world applications.

###### Index Terms:

NLP applications, Pre-trained language models’ reusability, Transfer learning,
, Integrated training.

##  I Introduction

Fine-tuning large-scale Pre-trained Language Models (PLMs) on a specific-task
dataset has achieved state-of-the-art performance on a variety of Natural
Language Processing (NLP) tasks [1, 2, 3]. However, recent studies [4, 5, 6]
have shown that PLMs trained on the large corpora learn spurious correlations,
which are prediction patterns that work for frequent example types drawn from
the same distribution as the training examples but break down in more
challenging cases such as Out-of-Distribution (OOD)/unseen datasets. If such
patterns often yield correct outputs, the fine-tuning training loss function
that heavily relies on these patterns cannot incentivize the language model to
learn the linguistic patterns from the less frequent example types and
generalize them to OOD examples. The inability to generalize patterns to OOD
datasets is a major obstacle for PLMs’ practical use in real-world
applications.

To improve the PLMs’ reusability on unseen data, several solutions have been
proposed such as: transfer learning through continuing pre-training (e.g., [7,
8, 9]) or performing extra fine-tuning (e.g., [10, 11]) on the domain of
interest. However, besides pre-training being computationally expensive, the
effectiveness of transfer learning is highly limited to the size of the data,
the similarity of the source/target domains, and task complexity [12]. Other
studies [13, 14, 15, 16, 17, 18] proposed different learning strategies
focusing on challenging keywords, called data biases, that lead to forming
spurious correlations. Although these solutions achieved promising results,
their main weakness is the strong assumption of knowing the datasets’ biases
in advance. Also, since some of these studies [13, 17] used data augmentation
for their proposed learning method, determining the number of examples
required for the best performance is challenging [19]. Some of these solutions
also (e.g.,[14, 16]) decrease the PLMs’ performance on in-distribution
datasets.

In this study, we propose a novel approach called Mask-tuning to improve the
reusability of PLMs on unseen data by enhancing the fine-tuning training
process. We draw inspiration from recent data augmentation techniques (e.g.,
[20, 21, 22]) that utilize the Masked Language Model (MLM) to increase the
diversity of examples in the training dataset. Unlike those traditional data
augmentation approaches, Mask-tuning integrates the MLM training objective
into fine-tuning’s training process. This unique integrated training method in
each training batch primarily perturbs the original training examples using
MLM to interrupt the frequent example types (patterns) in the training dataset
and generates perturbed examples. Then Mask-tuning classifies the perturbed
examples through fine-tuning according to the original examples’ ground-truth
labels. An integrated loss from perturbation and classification trains the
Mask-tuning (Figure 1). Our analysis shows that Mask-tuning creates three
times more diversified examples than MLM (SectionIII), demonstrating its
effectiveness in enhancing PLMs’ generalization. In summary, our contributions
are as follows:
I. We study the impact of integrating the MLM training objectives into the
fine-tuning for mitigating PLMs’ generalization problem. Our proposed method,
Mask-tuning, solely uses the downstream task’s training dataset and is a plug-
and-play tool for any PLM that works with original fine-tuning.
II. We conduct comprehensive experiments under a consistent evaluation process
to verify the effectiveness of Mask-tuning using BERT, RoBERTa, and
autoregressive BART language models. We show that Mask-tuning outperforms six
state-of-the-art baselines on three downstream tasks’ in-distribution and
corresponding five OOD datasets.
III. We also conduct three ablation studies to compare Mask-tuning performance
with data augmentation, Mask-tuning without integrated loss, and sequential
training. The results demonstrate the effectiveness of each component of the
Mask-tuning for improving the PLMs’ generalization.

##  II Related Works

Transfer Learning; Transfer learning for improving the PLMs’ generalization
has been implemented through two different approaches, domain- and task-
adaptation. The domain-adaptation continues pre-training on examples with the
same domain as the downstream task dataset. Some studies proposed to pre-train
the language models in their domain of interest from scratch (e.g., [23, 24]).
In contrast, other studies suggested running a second round of pre-training on
the domain of interest (e.g., [25, 26]). Besides, task-adaptation focuses on
sharing knowledge across different tasks [27, 28]. In task-adaptation, the
pre-trained model first fine-tunes on a dataset most related to the target
task and then fine-tunes again on the target task. Both approaches have shown
the benefits of transfer learning for improving PLM’s generalization problem,
especially when target training data is limited. However, re-running the pre-
training phase on a large-scale dataset is computationally expensive.
Furthermore, the effectiveness of transfer learning is highly limited to the
size of the data, the similarity between the source and target tasks and
domains, and task complexity [28]. However, same as original fine-tuning,
Mask-tuning solely uses the target downstream task dataset.
Data Augmentation; Data augmentation is the most straightforward approach for
improving the PLMs’ robustness. In this method, the downstream task dataset is
enriched with examples from the target distribution using various techniques
such as: increasing the size of the training dataset [22, 29], balancing the
existing cues [30], adversarial training [5, 20, 21], augmenting the standard
training dataset with syntactic information [31, 32], creating a partial view
to augment the training data [17], or dropping span of hidden information
[13]. These approaches generate new examples following different methods and
extend the size of the training dataset. Then they use the enlarged dataset
for fine-tuning. However, Yoo and Qi[19] showed that the size of the required
data for the best performance is challenging. The main advantages of Mask-
tuning over traditional data augmentation is that Mask-tuning benefits from
both MLM and fine-tuning objectives. This unique approach allows Mask-tuning
to generate perturbed examples that are more diverse and effective for
improving the model’s generalization. Furthermore, Mask-tuning’s perturbed
examples are double-validated through fine-tuning classification, ensuring
that only high-quality examples are incorporated into the training process.
Learning Process Improvement;  Another area of study to enhance the pre-
trained models’ generalization is introducing new learning techniques that are
robust against spurious patterns in the training dataset [33, 14, 34, 30, 16,
35, 18, 37]. These techniques focus on challenging examples or keywords that
do not allow the pre-trained models to make shortcuts during training. Thus,
they need to design processes to recognize and handle these patterns and
keywords in the training examples and re-training the model on extra text
corpora. These changes often make the model more complicated and
computationally expensive. However, the main weakness of these methods is
their strong assumption of knowing these challenging examples or keywords in
advance [12].

Figure 1: Illustration of the Mask-tuning’s training process. The input of the
fine-tuning is only a perturbed version of the in-distribution training
example.

##  III Proposed Method

We propose Mask-tuning for improving the PLMs’ generalization by enhancing the
training process of the fine-tuning on downstream tasks’ datasets. Mask-tuning
perturbs the sequence of words in a training example to interrupt the frequent
example types in the training dataset and simultaneously validates the
perturbed example according to the original example’s label (ground-truth
label). For this aim, Mask-tuning integrates two training objectives: 1) self-
supervised Masked Language Modeling (MLM) training objective and 2) Fine-
tuning’s classification function. In each training batch, Mask-tuning works as
follows:

Mask-tuning uses self-supervised MLM to randomly mask a certain percentage of
the input tokens in each training example. The MLM training objective is to
predict those masked ones based only on their context with a mean cross-
entropy loss. We denote the first training phase’s loss as MLM-loss
($\mathcal{L}_{MLM}$). The training examples with the predicted token(s),
called _perturbed examples_ , are fed into fine-tuning to be classified based
on the ground-truth label ($y$). So, $p_{\theta}(y^{\prime}=y|\hat{x})$ is the
fine-tuning classification function to predict the perturbed example’s label
($y^{\prime}$) based on the perturbed example ($\hat{x}$) and compute the
fine-tuning training loss ($\mathcal{L}_{Fine-tuning}$), where $\theta$ is the
PLMs’ parameters for the fine-tuning. A weighted aggregation of these two
training processes computes Mask-tuning loss ($\mathcal{L}_{Mask-tuning}$) as
follows:

$\mathcal{L}_{Mask-tuning}=\alpha\
\mathcal{L}_{MLM}+\>(1-\alpha)\mathcal{L}_{Fine-tuning}$ (1)

where ${\alpha}$ is a weighting factor that is employed to adjust the
contribution of each training phase loss for computing the Mask-tuning loss.
The Mask-tuning training objective aims to minimize both self-supervised MLM
loss and supervised fine-tuning classification loss. In the following, we
present how Mask-tuning benefits from the aggregated loss in each training
batch:
I. The MLM’s token-prediction output is correct, so MLM training loss is close
to zero ($L_{MLM}$ $\approx$ $0$); In this case, there would be one of the
following scenarios:
1) The fine-tuning classification’s output is also correct, so fine-tuning
loss is close to zero ($L_{Fine-tuning}$ $\approx$ $0$). In this case, the
Mask-tuning loss is also close to zero ($L_{Mask-tuning}$ $\approx$ $0$). For
instance, we observed the following example in the third training epoch:
$x$: ”take care is nicely performed by a quintet of actresses”, label ($y$): 1
$\Rightarrow$ $\hat{x}$: ”take care is nicely performed by a [group] of
actresses”, (label ($\hat{y}$): 1 AND prediction: correct).
It shows that fine-tuning classification validates that the perturbed example
has the same label as the original example. Thus, there is no need for further
training. This scenario is the primary goal of the Mask-tuning training
objective that differs Mask-tuning from sequential training and data
augmentation, which are investigated in our ablation studies (Section VI).
2) The output of fine-tuning classification is incorrect, so fine-tuning loss
is significant ($L_{Fine-tuning}$ $>$ $0$). For instance, we observed the
following example from the sentiment analysis task:
$x$: ”mumbles his way through the movie.”, label ($y$): $0$ $\Rightarrow$
$\hat{x}$: ”[cheer] his way through the movie .”, (label ($\hat{y}$): 1, AND
prediction: incorrect).
It means fine-tuning classifier did not validate that the perturbed example
has the same label as the original example. In this case, the aggregated loss
(Mask-tuning loss) is large enough for continuing training to minimize the
fine-tuning loss.
II. The MLM’s token-prediction output is incorrect, so MLM training loss is
significant ($L_{MLM}$ $>$ $0$ ); Suppose the predicted token(s) is unrelated
to the original token(s), and the prediction output is incorrect, and MLM-loss
is significant. For instance, we observed the following example in the third
training epoch:
$x$: ”he slaps together his own brand of liberalism”, label ($y$): $0$
$\Rightarrow$ $\hat{x}$: ”he [accept] together his own brand of liberalism”,
(label ($\hat{y}$): 1 AND prediction output: incorrect).
In this case, the Mask-tuning loss is large enough, regardless of the fine-
tuning classification output, to continue training to minimize both MLM and
Fine-tuning losses. As we can see, Mask-tuning is trained based on the
perturbed examples validated by fine-tuning classifier according to the
original examples’ ground-truth label. For this objective, the aggregated loss
incentivizes the MLM and fine-tuning for further training to reach a correct
output from both training objectives and minimize their losses (i.e., MLM
training loss and fine-tuning loss).

##  IV Experimental setup

To evaluate the effectiveness of our proposed method, we first trained Mask-
tuning on the in-distribution downstream tasks’ training dataset. Then, we
evaluated Mask-tuning on the in-distribution and corresponding OOD evaluation
sets. OOD datasets are drawn from a different distribution with more
linguistic variation than the in-distribution training set [6, 5]. PLMs’
performance on OOD datasets indicates their reusability on real-world
applications.
Baselines. We compared the performance of our proposed approach with the
state-of-the-art baselines, which shed light on the existing training
challenges in PLMs. These baselines proposed new learning disciplines to
improve the PLMs’ generalization. For a fair comparison, we followed their
evaluation process (e.g., using the same tasks, OOD datasets, and PLMs).
In-Distribution and OOD Tasks and Datasets. In this study, we conducted
comprehensive experiments on three tasks from GLUE
111https://gluebenchmark.com/tasks benchmark [37] that have available and
wide-used OOD dataset including: Stanford Sentiment Treebank (SST-2), Natural
Language Inference (MNLI [38]), and Paraphrase Identification (QQP [39]).
Their corresponding OOD datasets are IMDB-Cont [40] and IMDB-CAD [4], HANS [5]
and AdvNLI [41], and PAWS [6], respectively.

TABLE I:  Performance comparison of Mask-tuning (Ours) and various baselines
using different PLMs on in-distribution and OOD datasets. All models are
trained on in-distribution dataset and then evaluated on related test set of
in-distribution and OOD dataset.The last row also shows the results of a Large
Language Model (LLM), GPT-3 Few-shot Prompting [44] that is discussed in
Section III. The Fine-tuning has been implemented using the code from
huggingface [42].

|  | Sentiment | NLI | Paraphrase
---|---|---|---|---
|  | In-Dis | OOD | In-Dis | OOD | In-Dis | OOD
Model | Baseline | SST-2 | IMDB-Cont. | IMDB-CAD | MNLI (m/mm) | HANS | AdvNLI | QQP | PAWS
BERTbase | Fine-tuning | 92.43 | 79.08 | 87.00 | 84.30/83.40 | 56.90 | 24.12 | 90.80 | 32.80
| Learned-Mixin+H [14] | - | - | - | 83.97/- | 66.15 | - | - | -
| PoE [16] | - | - | - | 84.19/- | 66.31 | - |  | -
| Regularized-conf [18] | - | - | - | 84.30/84.80 | 69.10 | - | 91.50 | 39.80
| IPT-standard [32] | - | - | - | 84.40/- | 56.70 | - | - | -
| Mask-tuning (ours) |  93.11$\pm$0.1 |  82.75$\pm$0.3 |  88.32$\pm$0.1 |  84.75$\pm$0.2/85.10$\pm$0.1 |  69.52$\pm$0.2 |  26.32$\pm$0.3 |  91.54$\pm$0.1 |  46.74$\pm$0.5
RoBERTabase | Fine-tuning | 94.49 | 84.50 | 88.40 | 87.60/87.50 | 67.80 | 31.20 | 91.50 | 38.45
| Span Cutoff [17] | 95.40 | 85.50 | 89.20 | 88.40/- | 68.40 | 31.10 | 92.00 | 38.80
| HiddenCut [13] | 95.80 | 87.80 | 90.40 | 88.20/- | 71.20 | 32.80 | 92.00 | 41.50
| IPT-standard [32] | - | - | - | 87.70/- | 66.30 | - | - | -
| Mask-Tuning (ours) |  94.60$\pm$0.1 |  88.50$\pm$0.2 |  91.62$\pm$0.1 |  87.72$\pm$0.1/87.83$\pm$0.2 |  75.70$\pm$0.2 |  37.40$\pm$0.6 | 91.62$\pm$0.2 |  44.37$\pm$0.4
BARTbase | Fine-tuning | 93.23 | 82.48 | 86.03 | 84.60/84.80 | 56.30 | 30.51 | 90.50 | 32.27
| Mask-Tuning (ours) |  93.80$\pm$0.2 |  84.00$\pm$0.1 |  87.83$\pm$0.3 |  86.08$\pm$0.2/86.12$\pm$0.3 |  70.48$\pm$0.5 |  35.31$\pm$0.4 |  91.03$\pm$0.1 |  45.71$\pm$0.5
GPT-3 | Few-shots Prompting [44] | - | - | - | 77.60*/- | 75.30 | - | 83.5* | 73.7

TABLE II: Some perturbed examples generated by Mask-tuning.

1 | ”manipulate feminist empowerment tale thinly!”
---|---
| epoch 1: [thinly] $\Rightarrow$ [thinly]
| epoch 2: [thinly] $\Rightarrow$ [lyrically]
| epoch 3: [thinly] $\Rightarrow$ [tenderly]
2 | ”Convey a strong sense of the girls’ environment.”
| epoch 1: [strong] $\Rightarrow$ [strong]
| epoch 2: [sense] $\Rightarrow$ [understanding]
| epoch 3: [girls’] $\Rightarrow$ [UNK]
3 | ”The lead roles are more than competent.”
| epoch 1: [roles] $\Rightarrow$ [roles]
| epoch 2: [competent] $\Rightarrow$ [qualified]
| epoch 3: [roles] $\Rightarrow$ [actors]

Experimental Setup Following the baselines, we performed Mask-tuning using two
PLMs: BERT [1], and RoBERTa [3]. Also, we used BART [2] as an autoregressive
language model with a different architecture from BERT. Since we have a base
and large size of each pre-trained language model, recent study [43]
illustrated that compared with the base size language models, the large size
does not necessarily mitigate the PLMs’ generalization issue. Besides, the
base size models are more efficient and deployable to a broader computational
environment and applications. Thus, we chose the base size of these models to
show the efficacy of the Mask-tuning method.

We followed the original language models’
parameters222https://github.com/huggingface/transformers for the two training
phases of Mask-tuning. However, we empirically changed the learning rate and
batch size for the fine-tuning phase of the Mask-tuning. After several trial
runs, the batch size sets to $32$ for double sentence datasets (i.e., MNLI and
QQP) and $16$ for single sentence datasets (i.e., SST-2). Also, the best
learning rate has been determined through grid search among {2e-5, 3e-5, 4e-5,
5e-5}. We empirically selected the optimal value for $\alpha$ by a grid search
between 0 and 1 with 0.1 increments. The best value of $\alpha$ is 0.6, 0.7,
and 0.8 for SST-2, MNLI, and QQP, respectively. Finally, the masking
percentage is set to 5%, achieving the best performance on in-distribution and
OOD datasets. All experiments were performed with three epochs and an NVIDIA
V100 GPU with five random seeds. The average accuracy value with corresponding
standard deviations is reported on the in-distribution and OOD test sets. We
used huggingface code [42] for performing the original Fine-tuning.

##  V Results and discussion

Table I displays the performance of Mask-tuning and other baselines, with some
studies only reporting results for one model. We also compare the performance
of Mask-tuning and original fine-tuning for the BART model, which has not been
previously reported on OOD datasets. Finally, We present the results for GPT-3
on in-distribution and OOD dataset captured from [44].
Performance on OOD Datasets. The results show that Mask-tuning performance
outperforms all baseline models. To the best of our knowledge, no study has
reported BERT-base model performance on IMDB-Cont and IMDB-CAD. Thus, we
compared the Mask-tuning and original fine-tuning performance. In comparison
with the BERT fine-tuning, Mask-tuning improves the performance on IMDB-Cont
by +3.67%, IMDB-CAD by +1.32%, HANS by +12.62%, AdvNLI by +2.2%, and PAWS by
+13.94%. Moreover, in compare with the RoBERTa fine-tuning, Mask-tuning
improves PLMs’ performance on IMDB-Cont by +4%, IMDB-CAD by +3.22%, HANS by
+7.9%, AdvNLI by +6.2%, and PAWS by +5.92%. Finally, in compare with the BART
fine-tuning, Mask-tuning enhances the BART performance on IMDB-Cont by +0.52%,
IMDB-CAD by +1.8%, HANS by +14.18%, AdvNLI by +4.8%, and PAWS by +13.44%.
Performance on In-Distribution Dataset. Maintaining high performance on in-
distribution datasets while improving performance on OOD datasets is essential
in evaluating an approach. Mask-tuning achieves state-of-the-art performance
on OOD datasets while boosting performance on in-distribution datasets. These
results highlight the effectiveness of Mask-tuning in balancing in-
distribution and OOD performance.

TABLE III:  Performance comparison of Mask-tuning (ours) and our ablation
studies on various fine-tuning process.

| Sentiment | NLI | Paraphrase
---|---|---|---
| In-distribution | OOD | In-distribution. | OOD | In-distribution | OOD
Model | SST-2 | IMDB-Cont. | IMDB-CAD | MNLI (m/mm) | HANS | AdvNLI | QQP | PAWS
BERTbase |  |  |  |  |  |  |  |
Fine-tuning on original train set | 92.43 | 79.08 | 87.00 | 84.30/83.40 | 56.90 | 24.12 | 90.80 | 32.80
Fine-tuning on augmented train set. | 93.11 | 80.27 | 86.49 | 84.06/84.28 | 57.00 | 25.70 | 91.14 | 33.93
Mask-tuning w/o integrated loss | 92.07 | 79.00 | 84.99 | 83.57/84.22 | 50.51 | 23.60 | 89.92 | 38.15
Sequential Training | 91.97 | 80.97 | 86.76 | 83.40/84.00 | 55.60 | 24.20 | 91.00 | 34.82
Mask-tuning (ours) |  93.11$\pm$0.1 |  82.75$\pm$0.3 |  88.32$\pm$0.1 |  84.75$\pm$0.2/85.10$\pm$0.1 |  69.52$\pm$0.2 | 26.32$\pm$0.3 |  91.54$\pm$0.1 |  46.74$\pm$0.5
RoBERTabase |  |  |  |  |  |  |  |
Fine-tuning on original train set | 94.49 | 84.50 | 88.40 | 87.60/87.50 | 67.80 | 31.20 | 91.50 | 38.45
Fine-tuning on augmented train set. | 93.46 | 85.54 | 89.76 | 87.94/87.41 | 72.66 | 32.20 | 91.26 | 42.06
Mask-tuning w/o integrated loss | 93.00 | 81.80 | 87.31 | 86.88/87.24 | 65.16 | 30.00 | 90.09 | 41.00
Sequential Training | 93.26 | 84.86 | 89.00 | 87.50/87.40 | 71.69 | 30.70 | 91.20 | 39.91
Mask-tuning (ours) |  94.60$\pm$0.1 |  88.50$\pm$0.2 |  91.62$\pm$0.1 |  87.72$\pm$0.1/87.83$\pm$0.2 |  75.70$\pm$0.2 |  37.40$\pm$0.6 |  91.62$\pm$0.2 |  44.37$\pm$0.4
BARTbase |  |  |  |  |  |  |  |
Fine-tuning on original train set | 93.23 | 82.48 | 86.03 | 84.60/84.80 | 56.30 | 30.51 | 90.50 | 32.27
Fine-tuning on augmented train set. | 93.11 | 82.99 | 86.63 | 85.90/86.00 | 65.60 | 32.80 | 9.60 | 33.42
Mask-tuning w/o integrated loss | 93.00 | 82.02 | 85.80 | 84.02/84.10 | 55.12 | 32.40 | 89.34 | 35.21
Sequential Training | 92.88 | 82.65 | 86.13 | 84.81/84.25 | 58.00 | 32.50 | 89.95 | 34.62
Mask-tuning (ours) |  93.80$\pm$0.1 |  83.00$\pm$0.1 |  87.83$\pm$0.3 |  86.08$\pm$0.2/86.12$\pm$0.3 |  70.48$\pm$0.5 |  35.31$\pm$0.4 |  91.03$\pm$0.1 |  45.71$\pm$0.5

Comparing Mask-tuning and GPT-3 performance on OOD. Table I illustrates that
GPT-3 (175B parameters) with few-shot prompting [44] achieves better accuracy
on OOD data compared to BERT-based PLMs. However, this performance improvement
was gained at the cost of a significant drop in accuracy on in-distribution
downstream tasks’ datasets. Our experiments show that Mask-tuning improves the
generalization of PLMs by modifying the fine-tuning process. Mask-tuning
achieves comparable or even better performance with GPT-3 on benchmark tasks
like HANS in RoBERTabase (355M parameters), a larger pre-trained model than
BERTbase (110M parameters), without sacrificing performance on in-distribution
data. These findings suggest promising avenues for future research in
enhancing the fine-tuning process of PLMs to boost their generalization.
Analysing the impact of Mask-tuning on training examples’ diversification.
Recent data augmentation studies [20, 22] showed that MLM improved PLM’s
generalization performance due to generating plausible and diversified
examples from the original training examples. In this section, we compare the
capabilities of MLM and Mask-tuning in creating plausible and diversified
examples through three training epochs using BERT. We define a perturbed
example as plausible if there is no syntactic or grammatical error after
predicting a masked token. For example, if a NOUN is predicted for a masked
VERB in an example that consequently makes the example grammatically
incorrect, then we consider it as an implausible example. We randomly selected
200 training examples from each of the three in-distribution training datasets
(SST-2, MNLI, and QQP). For each dataset, we ran two experiments: using 1) MLM
and 2) Mask-tuning for generating perturbed examples. Then we recorded the
masked and predicted tokens for each example through three training epochs. We
presented original and perturbed examples to two annotators to annotate them
as plausible or implausible. Annotators ended up with 93% inter-annotator
agreement. For the remaining 7%, the annotators expressed their views on each
case in the presence of the researchers and finally 100% agreement was
achieved.

We observed that in the first experiment (using MLM), MLM predicts a token
precisely the same as the masked one in 76% of examples. In 18% of examples,
MLM predicts a related plausible token to the masked one at least in one of
the three epochs. In 5% of examples, MLM predicts an implausible token and
finally removes the masked token in 1% of examples ([UNK]). However, the
perturbation results when using Mask-tuning are entirely different from when
using only the MLM. In Mask-tuning, in 60% of examples, the predicted token is
a related plausible token to the masked one versus 18% in the case of using
only MLM. In 37% of examples, mask-tuning predicts a token precisely the same
as the masked one. In 2% of examples, Mask-tuning predicts an implausible
token and finally removes the masked token in 1% of examples ([UNK]). This
analysis shows that mask-tuning improves MLM’s capabilities in generating
plausible and diversified examples by more than three times. Hence, it can be
considered one of the main reasons for Mask-tuning to improve the PLMs’
generalization performance. Table II presents some perturbed examples
generated by Mask-tuning.

##  VI Ablation Study

Three ablation experiments were performed to show the effectiveness of each
component of Mask-tuning. We compare the Mask-tuning performance and ablation
study on various fine-tuning process: data augmentation, Mask-tuning without
integrated loss, and sequential training. Details of these experiments are as
follows:
Mask-tuning Vs. Data augmentation. In this experiment, we aim to compare Mask-
tuning performance and data augmentation. For this aim, we first used MLM to
generate the perturbed examples from the original downstream task’s training
dataset. Then we fine-tuned the PLMs on the augmented training dataset, which
is created by merging original and perturbed downstream training datasets. As
we can see in Table III, results show that directly augmenting the training
data marginally improved the PLM’s performance on both in-distribution and OOD
datasets. However, this improvement is far behind the Mask-tuning performance.
Mask-tuning without integrated-loss. In this experimental setup, we
investigate the effect of an integrated loss on the proposed method’s
performance. We train Mask-tuning on in-distribution downstream datasets based
on only fine-tuning loss function output. Then evaluate the model on the in-
distribution and the OOD evaluation set. As we can see from Table III (third
row of each model’s results), eliminating MLM loss from the mask-tuning loss
hurts the Mask-tuning performance on both in-distribution and OOD datasets.
Hence, the integrated loss has an essential role in Mask-tuning performance
and improving the PLMs’ generalization.
Mask-tuning Vs. Sequential training. This experiment compares the performance
of the Mask-tuning and sequential training on downstream tasks. For this
comparison, we first use MLM to train the PLM on the original downstream task
training dataset. Then we tune the PLM by running original fine-tuning on the
perturbed examples. Finally, we evaluated the PLM on in-distribution and OOD
evaluation sets. The results in Table III (fourth row in each PLM’s results)
show that sequential training noticeably decreased the PLMs’ performance in
most cases.

All in all, the results of these experiments demonstrate the significance of
integrating the three components of Mask-tuning (i.e., input perturbation,
fine-tuning classification, and integrating loss) to improve the PLMs’
generalization.

##  VII Conclusion

In this study, we proposed Mask-tuning, a novel and effective training
approach that addresses the challenges in PLMs’ learning process, which affect
their reusability on unseen data. By integrating the MLM training objective
into fine-tuning, Mask-tuning enhances the PLMs’ generalization. Our
comprehensive experiments on in-distribution and OOD datasets from three
downstream tasks on three PLMs showed that Mask-tuning outperforms six
baselines on OOD datasets while also boosting PLMs’ performance on in-
distribution datasets. These findings demonstrate that Mask-tuning improves
the practicality and effectiveness of PLMs for real-world applications.
Moreover, since Mask-tuning is applicable to any PLM that works with fine-
tuning, it has broad applicability.

## References

  * [1] J. Devlin, M. Chang, K. Lee, K. Toutanova. ”BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding” Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 2019.
  * [2] M. Lewis, Y. Liu, N. Goyal, M. Ghazvininejad, A. Mohamed, O. Levy, V. Stoyanov, L. Zettlemoyer. ”BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension”. Proceedings of the 58th ACL, 2020.
  * [3] Y. Liu, M. Ott, N. Goyal, J. Du, M. Joshi, D. Chen, O. Levy, M. Lewis, L. Zettlemoyer, V. Stoyanov. ”Roberta: A robustly optimized bert pretraining approach”. preprint arXiv, 2019.
  * [4] D Kaushik ,E. Hovy , ZC. Lipton ”Learning the difference that makes a difference with counterfactually-augmented data.” ICLR, 2020.
  * [5] R. T. McCoy, E. Pavlick, T. Linzen. ”Right for the wrong reasons: Diagnosing syntactic heuristics in natural language inference” ACL 2019.
  * [6] Y. Zhang, J. Baldridge, L. He. ”PAWS: Paraphrase Adversaries from Word Scrambling”. NAACL-HLT (1), 2019.
  * [7] E. Alsentzer, J. Murphy, W. Boag, W. Weng, D. Jindi, T. Naumann, M. McDermott. ”Publicly Available Clinical BERT Embeddings”. Proceedings of the 2nd Clinical NLP Workshop, 2019.
  * [8] S. Gururangan, A. Marasović, S. Swayamdipta, K. Lo, I. Beltagy, D. Downey, N. Smith, Noah A. ”Don’t Stop Pretraining: Adapt Language Models to Domains and Tasks”. ACL 2020.
  * [9] J. Lee, W. Yoon, S. Kim, D. Kim, S. Kim, C. H. So, J. Kang. ”BioBERT: a pre-trained biomedical language representation model for biomedical text mining”. Bioinformatics Journal, 2020.
  * [10] Y. Pruksachatkun, J. Phang, H. Liu, P. M. Htut, X. Zhang, R. Y. Pang, C. Vania, K. Kann, S. Bowman. ”Intermediate-Task Transfer Learning with Pretrained Language Models: When and Why Does It Work?”. ACL 2020.
  * [11] T. Vu, T. Wang, T. Munkhdalai, A. Sordoni, A. Trischler, A. Mattarella-Micke, S. Maji, M. Iyyer. ”Exploring and Predicting Transferability across NLP Tasks”. Proceedings of the 2020 Conference on Empirical Methods in NLP (EMNLP), 2020.
  * [12] L. Tu, G. Lalwani, S. Gella, H. He. ”An empirical study on robustness to spurious correlations using pre-trained language models”. Transactions of the ACL 2020.
  * [13] J. Chen, D. Shen, W. Chen, d. Yang. ”Hiddencut: Simple data augmentation for natural language understanding with better generalizability”. ACL-IJCNLP, 2021.
  * [14] C. Clark, M. Yatskar, L. Zettlemoyer. ”Don’t Take the Easy Way Out: Ensemble Based Methods for Avoiding Known Dataset Biases”. EMNLP-IJCNLP, 2019.
  * [15] G. Glavaš, I. Vulić. ”Is Supervised Syntactic Parsing Beneficial for Language Understanding Tasks? An Empirical Investigation”. EACL: Main Volume, 2021.
  * [16] R. K. Mahabadi, Y. Belinkov, J. Henderson. ”End-to-End Bias Mitigation by Modelling Biases in Corpora”. ACL, 2020.
  * [17] D. Shen, M. Zheng, Y. Shen, Y. Qu, W. Chen. ”A simple but tough-to-beat data augmentation approach for natural language understanding and generation”. preprint arXiv 2020.
  * [18] P. A. Utama, N. S. Moosavi, I. Gurevych. ”Mind the Trade-off: Debiasing NLU Models without Degrading the In-distribution Performance”. ACL, 2020.
  * [19] J. Y. Yoo, Y. Qi. ”Towards Improving Adversarial Training of NLP Models”. Findings of the Association for Computational Linguistics: EMNLP 2021.
  * [20] S. Garg, G. Ramakrishnan. ”BAE: BERT-based Adversarial Examples for Text Classification”. EMNLP, 2020.
  * [21] D. Li, Y. Zhang, H. Peng, L. Chen, C. Brockett, M. Sun, W. B. Dolan. ”Contextualized Perturbation for Textual Adversarial Attack”. NAACL, 2021.
  * [22] N. Ng, K. Cho, M. Ghassemi. ”SSMBA: Self-Supervised Manifold Based Data Augmentation for Improving Out-of-Domain Robustness”. EMNLP, 2020.
  * [23] K. Huang, J. Altosaar, R. Ranganath. ”Clinicalbert: Modeling clinical notes and predicting hospital readmission”. CHIL: Proceedings of the ACM Conference on Health, Inference, and Learning 2020.
  * [24] J. Lee, W. Yoon, S. Kim, D. Kim, S. Kim, C. H. So, J. Kang. ”BioBERT: a pre-trained biomedical language representation model for biomedical text mining”. Bioinformatics journal 2020.
  * [25] E. Alsentzer, J. Murphy, W. Boag, W. Weng, D. Jindi, T. Naumann, M. McDermott. ”ublicly Available Clinical BERT Embeddings”. Proceedings of the 2nd Clinical Natural Language Processing Workshop 2019.
  * [26] S. Gururangan, A. Marasović, S. Swayamdipta, K. Lo, I. Beltagy, D. Downey, N. A. Smith. ”Don’t Stop Pretraining: Adapt Language Models to Domains and Tasks”. ACL 2020.
  * [27] Y. ruksachatkun, J. Phang, H. Liu, P. M. Htut, X. Zhang, R. Y. Pang, C. Vania, K. Kann, S. Bowman. ”Intermediate-Task Transfer Learning with Pretrained Language Models: When and Why Does It Work?”. ACL 2020.
  * [28] T. Vu, T. Wang, T. Munkhdalai, A. Sordoni, A. Trischler, A. Mattarella-Micke, S. Maji, M. Iyyer. ”Exploring and Predicting Transferability across NLP Tasks”. Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) 2020.
  * [29] D. Khashabi, T. Khot, A. Sabharwal. ”More bang for your buck: Natural perturbation for robust question answering”. Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) 2020.
  * [30] T. Schuster, D. Shah, Y. J. S. Yeo, D. R. F. Ortiz, E. Santus, R. Barzilay. ”Towards Debiasing Fact Verification Models”. EMNLP-IJCNLP, 2019.
  * [31] J. Min, R. T. McCoy, D. Das, E. Pitler, T. Linzen. ”Syntactic Data Augmentation Increases Robustness to Inference Heuristics”. Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics 2020.
  * [32] G. Glavaš, I. Vulić. ”Is Supervised Syntactic Parsing Beneficial for Language Understanding Tasks? An Empirical Investigation”. Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume 2021.
  * [33] J. Min, R. T. McCoy, D. Das, E. Pitler, T. Linzen. ”Syntactic Data Augmentation Increases Robustness to Inference Heuristics”. Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics 2020.
  * [34] Y. Oren, S. Sagawa, T. Hashimoto, P. Liang. ”Distributionally Robust Language Modeling”. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP).
  * [35] X. Zhou, M. Bansal. ”Towards Robustifying NLI Models Against Lexical Dataset Biases”. Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics 2020.
  * [36] S. J. Moon, S. Mo, K. Lee, J. Lee, J. Shin. ”MASKER: Masked Keyword Regularization for Reliable Text Classification”. Proceedings of the AAAI Conference on Artificial Intelligence 2021.
  * [37] A. Wang, A. Singh, J. Michael, F. Hill, O. Levy, S. R. Bowman. ”Glue: A multi-task benchmark and analysis platform for natural language understanding”. 7th International Conference on Learning Representations, ICLR 2019.
  * [38] A. Williams, N. Nangia, S. R. Bowman. ”A broad-coverage challenge corpus for sentence understanding through inference”. Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL HLT 2018.
  * [39] S. Iyer, N. Dandekar, K. Csernai. ”First Quora dataset release: Question pairs, 2017”. URL https://data. quora. com/First-Quora-Dataset-Release-Question-Pairs
  * [40] M. Gardner, Y. Artzi, V. Basmov, J. Berant, B. Bogin, S. Chen, P. Dasigi, D. Dua, Y. Elazar, A. Gottumukkala, and others. ”Evaluating Models’ Local Decision Boundaries via Contrast Sets”. Proceedings of the 2020 EMNLP: Findings.
  * [41] Y. Nie, A. Williams, E. Dinan, M. Bansal, J. Weston, D. Kiela. ”Adversarial NLI: A New Benchmark for Natural Language Understanding”. Proceedings of the 58th ACL 2020.
  * [42] T. Wolf, L. Debut, V. Sanh, J. Chaumond, C. Delangue, A. Moi, P. Cistac, T. Rault, R. Louf, M. Funtowicz, and others. ”Transformers: State-of-the-art natural language processing”. Proceedings of the 2020 EMNLP: system demonstrations.
  * [43] D. Hendrycks, X. Liu, E. Wallace, A. Dziedzic, R. Krishnan, D. K. Song. ”Pretrained Transformers Improve Out-of-Distribution Robustness”. ACL, 2020.
  * [44] C. Si, Z. Gan, Z. Yang, S. Wang, J. Wang, J. Boyd-Graber, L. Wang. ”Prompting gpt-3 to be reliable”. International Conference on Learning Representations, ICLR 2023.