# Hierarchical Sketch Induction for Paraphrase Generation

Tom Hosking Hao Tang Mirella Lapata
Institute for Language, Cognition and Computation
School of Informatics, University of Edinburgh
10 Crichton Street, Edinburgh EH8 9AB
<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>
###### Abstract

We propose a generative model of paraphrase generation, that encourages
syntactic diversity by conditioning on an explicit syntactic sketch. We
introduce Hierarchical Refinement Quantized Variational Autoencoders (HRQ-
VAE), a method for learning decompositions of dense encodings as a sequence of
discrete latent variables that make iterative refinements of increasing
granularity. This hierarchy of codes is learned through end-to-end training,
and represents fine-to-coarse grained information about the input. We use HRQ-
VAE to encode the syntactic form of an input sentence as a path through the
hierarchy, allowing us to more easily predict syntactic sketches at test time.
Extensive experiments, including a human evaluation, confirm that HRQ-VAE
learns a hierarchical representation of the input space, and generates
paraphrases of higher quality than previous systems.

##  1 Introduction

Humans use natural language to convey information, mapping an abstract idea to
a sentence with a specific surface form. A paraphrase is an alternative
surface form of the same underlying semantic content. The ability to
automatically identify and generate paraphrases is of significant interest,
with applications in data augmentation Iyyer et al. (2018), query rewriting,
Dong et al. (2017) and duplicate question detection Shah et al. (2018).

While autoregressive models of language (including paraphrasing systems)
predict one token at a time, there is evidence that in humans some degree of
planning occurs at a higher level than individual words Levelt (1993); Martin
et al. (2010). Prior work on paraphrase generation has attempted to include
this inductive bias by specifying an alternative surface form as additional
model input, either in the form of target parse trees Iyyer et al. (2018);
Chen et al. (2019a); Kumar et al. (2020), exemplars Meng et al. (2021), or
syntactic codes Shu et al. (2019); Hosking and Lapata (2021). Most of these
approaches suffer from an ‘all or nothing’ problem: the target surface form
must be fully specified during inference. However, predicting the complete
syntactic structure is almost as difficult as predicting the sentence itself,
negating the benefit of the additional planning step.

$\textbf{x}_{syn}$$\textbf{x}_{sem}$$\textbf{z}_{syn}$$\textbf{z}_{sem}$$q_{1}$$q_{2}$$q_{3}$

(a) Posterior (encoder)

y$\textbf{z}_{syn}$$\textbf{z}_{sem}$$q_{1}$$q_{2}$$q_{3}$

(b)  Generative model (decoder)

Figure 1: The generative models underlying our approach. Given some semantic
content $\textbf{z}_{sem}$, we predict a hierarchical set of syntactic codes
$q_{d}$ that describe the output syntactic form at increasing levels
granularity. These are combined to give a syntactic embedding
$\textbf{z}_{syn}$, which is fed to the decoder along with the original
semantic content to generate the output sentence y. During training, the
encoder is driven by a paraphrase $\textbf{x}_{sem}$ and a syntactic exemplar
$\textbf{x}_{syn}$.

In this paper, we propose a generative model for paraphrase generation, that
combines the diversity introduced by an explicit syntactic target with the
tractability of models trained end-to-end. Shown in Figure 1, the model begins
by assuming the existence of some semantic content $\textbf{z}_{sem}$.
Conditioned on this semantic information, the model predicts a syntactic
‘sketch’ in the form of a hierarchical set of discrete codes $q_{1:D}$, that
describe the target syntactic structure with increasing granularity. The
sketch is combined into an embedding $\textbf{z}_{syn}$, and fed along with
the original meaning $\textbf{z}_{sem}$ to a decoder that generates the final
output utterance y. Choosing a discrete representation for the sketch means it
can be predicted from the meaning as a simple classification task, and the
hierarchical nature means that the joint probability over the codes admits an
autoregressive factorisation, making prediction more tractable.

The separation between $\textbf{z}_{sem}$ and $\textbf{z}_{syn}$ is induced by
a training scheme introduced in earlier work Hosking and Lapata (2021); Huang
and Chang (2021) and inspired by prior work on separated latent spaces Chen et
al. (2019b); Bao et al. (2019), whereby the model must reconstruct a target
output from one input with the correct meaning, and another input with the
correct syntactic form. To learn the discretized sketches, we propose a
variant of Vector-Quantized Variational Autoencoders (VQ-VAE, or VQ) that
learns a hierarchy of embeddings within a shared vector space, and represents
an input encoding as a path through this hierarchy. Our approach, which we
call Hierarchical Refinement Quantized Variational Autoencoders or HRQ-VAE,
leads to a decomposition of a dense vector into embeddings of increasing
granularity, representing high-level information at the top level before
gradually refining the encoding over subsequent levels.

Our contributions are summarized as follows:

  * •

We propose a generative model of natural language generation, HRQ-VAE, that
induces a syntactic sketch to account for the diversity exhibited by
paraphrases. We present a parameterization of our generative model that is a
novel method for learning hierarchical discretized embeddings over a single
latent encoding space. These embeddings are trained end-to-end and jointly
with the encoder/decoder.

  * •

We use HRQ-VAE to induce hierarchical sketches for paraphrase generation,
demonstrating that the known factorization over codes makes them easier to
predict at test time, and leads to higher quality paraphrases.

##  2 Latent Syntactic Sketches

###  2.1 Motivation

Let y be a sentence, represented as a sequence of tokens. We assume that y
contains semantic content, that can be represented by a latent variable
$\textbf{z}_{sem}$. Types of semantic content might include the description of
an image, or a question intent. However, the mapping from semantics to surface
form is not unique: in general, there is more than one way to express the
semantic content. Sentences with the same underlying meaning
$\textbf{z}_{sem}$ but different surface form y are paraphrases. Standard
approaches to paraphrasing (e.g., Bowman et al. 2016) map directly from
$\textbf{z}_{sem}$ to y, and do not account for this diversity of syntactic
structure.

Following recent work on syntax-guided paraphrasing Chen et al. (2019a);
Hosking and Lapata (2021), and inspired by evidence that humans plan out
utterances at a higher level than individual words Martin et al. (2010), we
introduce an intermediary sketching step, depicted in Figure 1(b). We assume
that the output sentence y is generated as a function both of the meaning
$\textbf{z}_{sem}$ and of a syntactic encoding $\textbf{z}_{syn}$ that
describes the structure of the output. Moreover, since natural language
displays hierarchical organization in a wide range of ways, including at a
syntactic level (constituents may contain other consituents), we also assume
that the syntactic encoding $\textbf{z}_{syn}$ can be decomposed into a
hierarchical set of discrete latent variables $q_{1:D}$, and that these
$q_{d}$ are conditioned on the meaning $\textbf{z}_{sem}$. This contrasts with
popular model architectures such as VAE Bowman et al. (2015) which use a
_flat_ internal representation in a dense Euclidean vector space.

Intuitively, our generative model corresponds to a process where a person
thinks of a message they wish to convey; then, they decide roughly how to say
it, and incrementally refine this decision; finally, they combine the meaning
with the syntactic sketch to ‘spell out’ the sequence of words making up the
sentence.

###  2.2 Factorization and Objective

The graphical model in Figure 1(b) factorizes as

$p(\textbf{y},\textbf{z}_{sem})=\sum_{q_{1:D},\textbf{z}_{syn}}p(\textbf{y}|\textbf{z}_{sem},\textbf{z}_{syn})\\\
\times p(\textbf{z}_{syn}|q_{1:D})\\\ \times p(\textbf{z}_{sem})\times
p(q_{1}|\textbf{z}_{sem})\prod\limits_{d=2}^{D}\times
p(q_{d}|q_{<d},\textbf{z}_{sem}).$ (1)

Although $q_{1:D}$ are conditionally dependent on $\textbf{z}_{sem}$, we
assume that $\textbf{z}_{sem}$ may be determined from y without needing to
explicitly calculate $q_{1:D}$ or $\textbf{z}_{syn}$. We also assume that the
mapping from discrete codes $q_{1:D}$ to $\textbf{z}_{syn}$ is a deterministic
function $f_{q\rightarrow\textbf{z}}(\cdot)$. The posterior therefore
factorises as

$\phi(\textbf{z}_{sem},\textbf{z}_{syn}|\textbf{y})=\phi(\textbf{z}_{sem}|\textbf{y})\times\phi(\textbf{z}_{syn}|\textbf{y})\\\
\times\phi(q_{1}|\textbf{z}_{syn})\times\prod\limits_{d=2}^{D}\phi(q_{d}|q_{<d},\textbf{z}_{syn}).$
(2)

Figure 2: An illustration of how HRQ-VAE maps an input encoding vector z to a
decomposition of hierarchical discretized encodings. HRQ-VAE compares the
input to a jointly learned codebook of embeddings that become increasingly
granular at lower depths of hierarchy. In this simplified example, with a
depth of 3 and a codebook size of 3, the nearest top-level (colours) embedding
to z is
$\textbf{e}_{{\color[rgb]{1,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,0}red}}$;
then, the residual error
$\boldsymbol{\delta}_{1}=\textbf{z}-\textbf{e}_{{\color[rgb]{1,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,0}red}}$
is compared to the 2nd level of embeddings (shapes), with the nearest being
$\textbf{e}_{\star}$. Finally, the residual error $\boldsymbol{\delta}_{2}$ is
compared to the 3rd level codebook (patterns), where the closest is
$\textbf{e}_{stripes}$. The quantized encoding of z is then
$\textbf{z}\approx\textbf{e}_{{\color[rgb]{1,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,0}red}}+\textbf{e}_{\star}+\textbf{e}_{stripes}$.

The separation between $\textbf{z}_{sem}$ and $q_{1:D}$, such that they
represent the meaning and form of the input respectively, is induced by the
training scheme. During training, the model is trained to reconstruct a target
y using $\textbf{z}_{sem}$ derived from an input with the correct meaning (a
paraphrase) $\textbf{x}_{sem}$, and $q_{1:D}$ from another input with the
correct form (a syntactic exemplar) $\textbf{x}_{syn}$. Hosking and Lapata
(2021) showed that the model therefore learns to encode primarily semantic
information about the input in $\textbf{z}_{sem}$, and primarily syntactic
information in $q_{1:D}$. Exemplars are retrieved from the training data
following to the process described in Hosking and Lapata (2021), with examples
in Appendix C. The setup is shown in Figure 1(a); in summary, during training
we set
$\phi(\textbf{z}_{sem}|\textbf{y})=\phi(\textbf{z}_{sem}|\textbf{x}_{sem})$
and $\phi(q_{d}|\textbf{y},q_{<d})=\phi(q_{d}|\textbf{x}_{syn},q_{<d})$. The
final objective is given by

$\text{ELBO}=\mathbb{E}_{\phi}\big{[}-\log
p(\textbf{y}|\textbf{z}_{sem},q_{1:D}))\\\ -\log
p(q_{1}|\textbf{z}_{sem})-\sum\limits_{d=2}^{D}\log
p(q_{d}|q_{<d},\textbf{z}_{sem})\big{]}\\\
+KL\big{[}\phi(\textbf{z}_{sem}|\textbf{x}_{sem})||p(\textbf{z}_{sem})\big{]},$
(3)

where $q_{d}\sim\phi(q_{d}|\textbf{x}_{syn})$ and
$\textbf{z}_{sem}\sim\phi(\textbf{z}_{sem}|\textbf{x}_{sem})$.

##  3 Neural Parameterisation

We assume a Gaussian distribution for $\textbf{z}_{sem}$, with prior
$p(\textbf{z}_{sem})\sim\mathcal{N}(\textbf{0},\textbf{1})$. The encoders
$\phi(\textbf{z}_{sem}|\textbf{x}_{sem})$ and
$\phi(\textbf{z}_{syn}|\textbf{x}_{syn})$ are Transformers Vaswani et al.
(2017), and we use an autoregressive Transformer decoder for
$p(\textbf{y}|\textbf{z}_{sem},\textbf{z}_{syn})$. The mapping
$f_{q\rightarrow\textbf{z}}(\cdot)$ from $q_{1:D}$ to $\textbf{z}_{syn}$ and
the posterior network $\phi(q_{d}|q_{<d},\textbf{z}_{syn})$ are more complex,
and form a significant part of our contribution.

Our choice of parameterization is learned end-to-end, and ensures that the
sketches learned are hierarchical both in the shared embedding space and in
the information they represent.

###  3.1 Hierarchical Refinement Quantization

Let $\textbf{z}_{syn}\in\mathcal{R}^{\mathbb{D}}$ be the output of the encoder
network $\phi(\textbf{z}_{syn}|\textbf{y})$, that we wish to decompose as a
sequence of discrete hierarchical codes. Recall that $q_{d}\in[1,K]$ are
discrete latent variables corresponding to the codes at different levels in
the hierarchy, $d\in[1,D]$. Each level uses a distinct codebook,
$\textbf{C}_{d}\in\mathbb{R}^{K\times\mathbb{D}}$, which maps each discrete
code to a continuous embedding
$\textbf{C}_{d}(q_{d})\in\mathbb{R}^{\mathbb{D}}$.

The distribution over codes at each level is a softmax distribution, with the
scores $s_{d}$ given by the distance from each of the codebook embeddings to
the residual error between the input and the cumulative embedding from all
previous levels,

$\displaystyle
s_{d}(q)=-\left(\left[\textbf{x}-\sum\limits_{d^{\prime}=1}^{d-1}\textbf{C}_{d^{\prime}}(q_{d^{\prime}})\right]-\textbf{C}_{d}(q)\right)^{2}.$
(4)

Illustrated in Figure 2, these embeddings therefore represent iterative
refinements on the quantization of the input. The posterior network
$\phi(q_{d}|q_{<d},\textbf{z}_{syn})$ iteratively decomposes an encoding
vector into a path through a hierarchy of clusters whose centroids are the
codebook embeddings.

Given a sequence of discrete codes $q_{1:D}$, we deterministically construct
its continuous representation with the composition function
$f_{q\rightarrow\textbf{z}}(\cdot)$,

$\displaystyle\textbf{z}_{syn}=f_{q\rightarrow\textbf{z}}(q_{1:D})=\sum\limits_{d=1}^{D}\textbf{C}_{d}(q_{d}).$
(5)

HRQ-VAE can be viewed as an extension of VQ-VAE van den Oord et al. (2017),
with two significant differences: (1) the codes are hierarchically ordered and
the joint distribution $p(q_{1},\ldots,q_{D})$ admits an autoregressive
factorization; and (2) the HRQ-VAE composition function is a sum, compared to
concatenation in VQ or a complex neural network in VQ-VAE 2 Razavi et al.
(2019). Under HRQ, latent codes describe a path through the learned hierarchy
within a shared encoding space. The form of the posterior
$\phi(q_{d}|q_{<d},\textbf{z}_{syn})$ and the composition function
$f_{q\rightarrow\textbf{z}}(\cdot)$ do not rely on any particular properties
of the paraphrasing task; the technique could be applied to any encoding
space.

#### Initialisation Decay

Smaller perturbations in encoding space should result in more fine grained
changes in the information they encode. Therefore, we encourage ordering
between the levels of hierarchy (such that lower levels encode finer grained
information) by initialising the codebook with a decaying scale, such that
later embeddings have a smaller norm than those higher in the hierarchy.
Specifically, the norm of the embeddings at level $d$ is weighted by a factor
$(\alpha_{init})^{d-1}$.

#### Depth Dropout

To encourage the hierarchy within the encoding space to correspond to
hierarchical properties of the output, we introduce depth dropout, whereby the
hierarchy is truncated at each level during training with some probability
$p_{depth}$. The output of the quantizer is then given by

$\displaystyle\textbf{z}_{syn}=\sum\limits_{d=1}^{D}\left(\textbf{C}_{d}(q_{d})\prod\limits_{d^{\prime}=1}^{d}\gamma_{d^{\prime}}\right),$
(6)

where $\gamma_{h}\sim\text{Bernoulli}(1-p_{depth})$. This means that the model
is sometimes trained to reconstruct the output based only on a partial
encoding of the input, and should learn to cluster similar outputs together at
each level in the hierarchy.

###  3.2 Sketch Prediction Network

During training the decoder is driven using sketches sampled from the encoder,
but at test time exemplars are unavailable and we must predict a distribution
over syntactic sketches $p(q_{1:D}|\textbf{z}_{sem})$. Modelling the sketches
as hierarchical ensures that this distribution admits an autoregressive
factorization.

We use a simple recurrent network to infer valid codes at each level of
hierarchy, using the semantics of the input sentence and the cumulative
embedding of the predicted path so far as input, such that $q_{d}$ is sampled
from
$p(q_{d}|\textbf{z}_{sem},q_{<d})=\texttt{Softmax}(\texttt{MLP}_{d}(\textbf{z}_{sem},\textbf{z}_{<d}))$,
where
$\textbf{z}_{<d}=\sum\limits_{d^{\prime}=1}^{d-1}\textbf{C}_{d^{\prime}}(q_{d^{\prime}})$.
This MLP is trained jointly with the encoder/decoder model, using the outputs
of the posterior network $\phi(q_{d}|\textbf{x}_{syn},q_{<d})$ as targets. To
generate paraphrases as test time, we sample from the sketch prediction model
$p(q_{d}|\textbf{z}_{sem},q_{<d})$ using beam search and condition generation
on these predicted sketches.

Figure 3: t-SNE visualisation of the syntactic encodings $\textbf{z}_{syn}$
for 10k examples from Paralex: colours indicate top-level codes $q_{1}$,
shapes indicate the second level, and patterns are used to label the third
level. Deeper levels in the hierarchy represent finer grained information in
encoding space.

###  3.3 Training Setup

We use the Gumbel reparameterisation trick Jang et al. (2016); Maddison et al.
(2017); Sønderby et al. (2017) for the discrete codes and the standard
Gaussian reparameterisation for the semantic representation. To encourage the
model to use the full codebook, we decayed the Gumbel temperature $\tau$,
according to the schedule given in Appendix A. We approximate the expectation
in Equation 3 by sampling from the training set and updating via
backpropagation Kingma and Welling (2014). The full model was trained jointly
by optimizing the ELBO in Equation 3.

##  4 Experimental Setup

#### Datasets

A paraphrase is ‘an alternative surface form in the same language expressing
the same semantic content as the original form’ Madnani and Dorr (2010), but
it is not always clear what counts as the ‘same semantic content’. Our
approach requires access to reference paraphrases; we evaluate on three
English paraphrasing datasets which have clear grounding for the meaning of
each sentence: Paralex Fader et al. (2013), a dataset of question paraphrase
clusters scraped from WikiAnswers; Quora Question Pairs
(QQP)111https://www.kaggle.com/c/quora-question-pairs sourced from the
community question answering forum Quora; and MSCOCO 2017 Lin et al. (2014), a
set of images that have been captioned by multiple annotators. For the
question datasets, each paraphrase is grounded to the (hypothetical) answer
they share. We use the splits released by Hosking and Lapata (2021). For
MSCOCO, each caption is grounded by the image that it describes. We evaluate
on the public validation set, randomly selecting one caption for each image to
use as input and using the remaining four as references.

#### Model Configuration

Hyperparameters were tuned on the Paralex development set, and reused for the
other evaluations. We set the depth of the hierarchy $D=3$, and the codebook
size $K=16$. The Transformer encoder and decoder consist of 5 layers each, and
we use the vocabulary and token embeddings from BERT-Base Devlin et al.
(2018). We use an initialisation decay factor of $\alpha_{init}=0.5$, and a
depth dropout probability $p_{depth}=0.3$. A full set of hyperparameters is
given in Appendix A, and our code is available at
https://github.com/tomhosking/hrq-vae.

#### Comparison Systems

As baselines, we consider three popular architectures: a vanilla autoencoder
(AE) that learns a single dense vector representation of an input sentence; a
Gaussian Variational AutoEncoder (VAE, Bowman et al., 2015), which learns a
distribution over dense vectors; and a Vector-Quantized Variational
AutoEncoder (VQ-VAE, van den Oord et al., 2017), that represents the full
input sentence as a set of discrete codes. All three models are trained to
generate a sentence from one of its paraphrases in the training data, and are
not trained with an autoencoder objective. We implement a simple tf-idf
baseline Jones (1972), retrieving the question from the training set with the
highest cosine similarity to the input. Finally, we include a basic copy
baseline as a lower bound, that simply uses the input sentences as the output.

We also compare to a range of recent paraphrasing systems. Latent bag-of-words
(BoW, Fu et al., 2019) uses an encoder-decoder model with a discrete bag-of-
words as the latent encoding. SOW/REAP Goyal and Durrett (2020) uses a two
stage approach, deriving a set of feasible syntactic rearrangements that is
used to guide a second encoder-decoder model. BTmPG Lin and Wan (2021) uses
multi-round generation to improve diversity and a reverse paraphrasing model
to preserve semantic fidelity. We use the results after 10 rounds of
paraphrasing. Separator Hosking and Lapata (2021) uses separated, non-
hierarchical encoding spaces for the meaning and form of an input, and an
additional inference model to predict the target syntactic form at test time.
All comparison systems were trained and evaluated on our splits of the
datasets.

As an upper bound, we select a sentence from the evaluation set to use as an
oracle syntactic exemplar, conditioning generation on a sketch that is known
to represent a valid surface form.

| Paralex | QQP | MSCOCO
---|---|---|---
System | BLEU $\uparrow$ | Self-B $\downarrow$ |  iBLEU $\uparrow$ | BLEU $\uparrow$ | Self-B $\downarrow$ |  iBLEU $\uparrow$ | BLEU $\uparrow$ | Self-B $\downarrow$ |  iBLEU $\uparrow$
Copy | 37.10 | 100.00 | 9.68 | 34.52 | 100.00 | 7.61 | 19.85 | 100.00 | -4.12
tf-idf | 25.08 | 25.25 | 15.01 | 24.05 | 62.49 | 6.75 | 18.26 | 38.37 | 6.93
AE | 40.10 | 75.71 | 16.94 | 28.99 | 60.11 | 11.17 | 27.90 | 38.71 | 14.58
VAE | 38.91 | 53.28 | 20.47 | 27.23 | 51.09 | 11.57 | 27.44 | 24.40 | 16.99
VQ-VAE | 40.26 | 65.71 | 19.07 | 16.31 | 21.13 | 8.83 | 25.62 | 22.41 | 16.01
SOW/REAP | 33.09 | 37.07 | 19.06 | 21.27 | 38.01 | 9.41 | 12.51 | 6.47 | 8.71
LBoW | 34.96 | 35.86 | 20.80 | 23.51 | 42.08 | 10.39 | 21.65 | 16.46 | 14.02
BTmPG | 28.40 | 35.99 | 15.52 | 19.83 | 35.11 | 8.84 | 19.76 | 13.04 | 13.20
Separator | 36.36 | 35.37 | 22.01 | 23.68 | 24.20 | 14.10 | 20.59 | 12.76 | 13.92
HRQ-VAE | 39.49 | 33.30 | 24.93 | 33.11 | 40.35 | 18.42 | 27.90 | 16.58 | 19.04
Oracle | 50.58 | 28.09 | 34.85 | 50.47 | 36.84 | 33.01 | 35.80 | 12.85 | 26.07
Table 1: Top-1 paraphrase generation results, without access to oracle
sketches. HRQ-VAE achieves the highest iBLEU scores, indicating the best
tradeoff between quality and diversity. Paired bootstrap resampling Koehn
(2004) indicates that HRQ-VAE significantly improves on all other systems
(p$<0.05$).

##  5 Results

Our experiments were designed to test two primary hypotheses: (1) Does HRQ-VAE
learn hierarchical decompositions of an encoding space? and (2) Does our
choice of generative model enable us to generate high quality and diverse
paraphrases?

###  5.1 Probing the Hierarchy

Figure 3 shows a t-SNE van der Maaten and Hinton (2008) plot of the syntactic
encodings $\textbf{z}_{syn}$ for 10,000 examples from Paralex. The encodings
are labelled by their quantization, so that colours indicate top-level codes
$q_{1}$, shapes denote $q_{2}$, and patterns $q_{3}$. The first plot shows
clear high level structure, with increasingly fine levels of substructure
visible as we zoom into each cluster. This confirms that the discrete codes
are ordered, with lower levels in the hierarchy encoding more fine grained
information.

To confirm that intermediate levels of hierarchy represent valid points in the
encoding space, we generate paraphrases using oracle sketches, but truncate
the sketches at different depths. Masking one level (i.e., using only
$q_{1},q_{2}$) reduces performance by $2.5$ iBLEU points, and two levels by
$5.5$. (iBLEU is an automatic metric for assessing paraphrase quality; see
Section 5.2). Although encodings using the full depth are the most
informative, partial encodings still lead to good quality output, with a
gradual degradation. This implies both that each level in the hierarchy
contains useful information, and that the cluster centroids at each level are
representative of the individual members of those clusters.

###  5.2 Paraphrase Generation

#### Metrics

Our primary metric is iBLEU Sun and Zhou (2012),

$\displaystyle\begin{split}\textrm{iBLEU}=\alpha\textrm{BLEU}(outputs,references)\\\
-(1-\alpha)\textrm{BLEU}(outputs,inputs),\end{split}$ (7)

that measures the fidelity of generated outputs to reference paraphrases as
well as the level of diversity introduced. We use the corpus-level variant.
Following the recommendations of Sun and Zhou (2012), we set $\alpha=0.8$,
with a sensitivity analysis shown in Appendix A. We also report
$\textrm{BLEU}(outputs,references)$ as well as Self-BLEU$(outputs,inputs)$.
The latter allows us to examine the extent to which models generate
paraphrases that differ from the original input.

To evaluate the diversity between multiple candidates generated by the same
system, we report pairwise-BLEU Cao and Wan (2020),

$\displaystyle\begin{split}\textrm{P-BLEU}=\mathbb{E}_{i\neq
j}[\textrm{BLEU}(outputs_{i},outputs_{j})].\end{split}$

This measures the average similarity between the different candidates, with a
lower score indicating more diverse hypotheses.

#### Automatic Evaluation

Shown in Table 1, the results of the automatic evaluation highlight the
importance of measuring both paraphrase quality and similarity to the input:
the Copy baseline is able to achieve high BLEU scores despite simply
duplicating the input. The VAE baseline is competitive but tends to have a hi

Paralex |  Where is the birthplace of woman pro golfer Dottie Pepper?
---|---
VAE |  Where is the birthplace of Pepper pro golfer Dottie?
BTmpG |  What is the birthplace of women pro golfer?
SOW/REAP |  What is the birthplace for golfer?
Latent BoW |  Where did the golfer golfer originate?
Separator |  Where is the birthplace of Dottie?
HRQ-VAE |  Where is Dottie Pepper from?
QQP |  What are the best ways to defrost lobster tails?
VAE |  What are the best ways to defrost lobster tails?
BTmpG |  How can I defrost my tails??
SOW/REAP |  What is defrost?
Latent BoW |  How do you something a something lobster?
Separator |  What are some of the best ways to defrost chicken?
HRQ-VAE |  How do you thaw frozen lobster tails?
MSCOCO |  Set of toy animals sitting in front of a red wooden wagon.
VAE |  Two stuffed animals sitting in front of a toy train.
BTmpG |  A herd of sheep grazing in a field of grass.
SOW/REAP |  A close up of a close up of a street
Latent BoW |  A toy wagon with a toy horse and a toy wagon.
Separator |  A toy model of a toy horse and buggy.
HRQ-VAE |  A group of stuffed animals sitting next to a wooden cart.
Table 2: Examples of generated paraphrases. HRQ-VAE is able to preserve the
original meaning, while introducing significant syntactic variation.

gh Self-BLEU score, indicating that the semantic preservation comes at the
cost of low syntactic diversity. HRQ-VAE achieves both higher BLEU scores and
higher iBLEU scores than the comparison systems, indicating that it is able to
generate higher quality paraphrases without compromising on syntactic
diversity.

The examples in Table 2 demonstrate that HRQ is able to introduce significant
syntactic variation while preserving the original meaning of the input.
However, there is still a gap between generation using predicted sketches and
‘oracle’ sketches (i.e., when the target syntactic form is known in advance),
indicating ample scope for improvement.

$q_{1}$ | $q_{2}$ | $q_{3}$ |  Output
---|---|---|---
Input |  Two types of fats in body ?
0 | 3 | 6 |  What types of fats are in a body?
13 | 7 |  What types of fats are there in body?
2 | 1 | 2 |  How many types of fats are there in the body?
3 | 7 |  How many types of fats are there in a body?
5 | 3 | 6 |  What are the different types of fats in a body?
5 | 7 |  What are the different types of fats in body?
8 | 7 |  Types of fats are different from body fat?
14 |  Two types of fats in body?
13 | 0 | 2 |  What are the different types of fats in the body?
6 |  What are the different types of fats in a body?
3 | 7 |  What are two types of fats in a body?
5 | 7 |  What are the different types of fats in body?
8 |  What are the different types of fats?
14 |  What are the different types of fats in the body?
Table 3: Examples of model output, for a range of different sketches. The left
hand side shows the sketch (i.e., the values of the codes $q_{1:D}$), with the
corresponding model output on the right. $q_{1}$ primarily specifies the wh-
word (e.g., outputs with $q_{1}=13$ are all ‘what’ questions), while
$q_{2},q_{3}$ correspond to more fine grained details, e.g., the outputs with
$q_{3}=6$ all use the article ‘a’ when referring to ‘body’.

#### Worked Example

Since the sketches $q_{1:D}$ are latent variables, interpretation is
difficult. However, a detailed inspection of example output reveals some
structure.

Table 3 shows the model output for a single semantic input drawn from Paralex,
across a range of different syntactic sketches. It shows that $q_{1}$ is
primarily responsible for encoding the question type, with $q_{1}=13$ leading
to ‘what’ questions and $q_{1}=2$ ‘how’ questions. $q_{2}$ and $q_{3}$ encode
more fine grained details; for example, all outputs shown with $q_{3}=6$ use
the indefinite article ‘a’.

We also examine how using increasingly granular sketches refines the syntactic
template of the output. Table 4 shows the model output for a single semantic
input, using varying granularities of sketch extracted from the exemplar. When
no sketch is specified, the model defaults to a canonical phrasing of the
question. When only $q_{1}$ is specified, the output becomes a ‘how many’
question, and when a full sketch is included, the output closely resembles the
exemplar.

Input |  Two types of fat in body?
---|---
Exemplar |  How many states are in the USA?
No sketch |  What are the different types of fats in the body?
$q_{1}$ |  How many types of fats are there in the body?
$q_{1},q_{2}$ |  How many fats does the body have?
$q_{1},q_{2},q_{3}$ |  How many fat are in the body?
Table 4: Model output for varying sketch granularities. When no sketch is
used, the model defaults to the most common phrasing of the question. As more
detail is included, the output converges towards the exemplar.

#### Generating Multiple Paraphrases

We evaluated the ability of our system to generate multiple diverse
paraphrases for a single input, and compared to the other comparison systems
capable of producing more than one output. For both HRQ-VAE and Separator, we
used beam search to sample from the sketch prediction network as in the top-1
case, and condition generation on the top-3 hypotheses predicted. For BTmPG,
we used the paraphrases generated after 3, 6 and 10 rounds. For the VAE, we
conditioned generation on 3 different samples from the encoding space. The
results in Table 5 show that HRQ-VAE is able to generate multiple high quality
paraphrases for a single input, with lower similarity between the candidates
than other systems.

| Paralex | QQP | MSCOCO
---|---|---|---
Model |  iBLEU $\uparrow$ |  P-BLEU $\downarrow$ |  iBLEU $\uparrow$ |  P-BLEU $\downarrow$ |  iBLEU $\uparrow$ |  P-BLEU $\downarrow$
VAE | 20.49 | 67.62 | 11.52 | 64.71 | 17.22 | 55.66
BTmPG | 15.50 | 89.20 | 9.13 | 82.02 | 13.20 | 80.38
Separator | 21.67 | 62.98 | 13.63 | 52.87 | 13.77 | 57.79
HRQ-VAE | 22.75 | 40.48 | 17.49 | 57.29 | 18.39 | 41.29
Table 5: Top-3 generation results. P-BLEU indicates the similarity between the
different candidates, while iBLEU scores reported are the mean across the 3
candidates. HRQ-VAE is able to generate multiple high quality paraphrases with
more diversity between them than comparison systems. Figure 4: Results of our
human evaluation. Although the VAE baseline is the best at preserving sentence
meaning, it is the worst at introducing variation to the output. HRQ-VAE
offers the best balance between dissimilarity and meaning preservation, and is
more fluent than both Separator and Latent BoW.

###  5.3 Human Evaluation

In addition to automatic evaluation we elicited judgements from crowdworkers
on Amazon Mechanical Turk. They were shown a sentence and two paraphrases,
each generated by a different system, and asked to select which one was
preferred along three dimensions: the dissimilarity of the paraphrase compared
to the original sentence; how well the paraphrase reflected the meaning of the
original; and the fluency of the paraphrase (see Appendix B). We evaluated a
total of 300 sentences sampled equally from each of the three evaluation
datasets, and collected 3 ratings for each sample. We assigned each system a
score of $+1$ when it was selected, $-1$ when the other system was selected,
and took the mean over all samples. Negative scores indicate that a system was
selected less often than an alternative. We chose the four best performing
models for our evaluation: HRQ-VAE, Separator, Latent BoW, and VAE.

Figure 4 shows that although the VAE baseline is the best at preserving
question meaning, it is also the worst at introducing variation to the output.
HRQ-VAE better preserves the original question intent compared to the other
systems while introducing more diversity than the VAE, as well as generating
much more fluent output.

###  5.4 Ablations

To confirm that the hierarchical model allows for more expressive sketches, we
performed two ablations. We compared to the full model using oracle sketches,
so that code prediction performance was not a factor. We set the depth $D=1$
and $K=48$, giving equivalent total capacity to the full model ($D=3,K=16$)
but without hierarchy. We also removed the initialisation scaling at lower
depths, instead initialising all codebooks with the same scale. Table 6 shows
that a non-hierarchical model with the same capacity is much less expressive.

We also performed two ablations against the model using predicted sketches; we
removed depth dropout, so that the model is always trained on a full encoding.
We confirm that learning the codebooks jointly with the encoder/decoder leads
to a stronger model, by first training a model with a continuous Gaussian
bottleneck (instead of the HRQ-VAE); then, we recursively apply $k$-means
clustering Lloyd (1982), with the clustering at each level taking place over
the residual error from all levels so far, analogous to HRQ-VAE. The results
of these ablations shown in Table 6 indicate that our approach leads to
improvements over all datasets.

Variant | Paralex | QQP | MSCOCO
---|---|---|---
HRQ-VAE (oracle) | 34.85 | 33.01 | 26.07
No initialisation scaling |  $-$3.06 |  $-$2.48 |  $-$3.02
No hierarchy |  $-$8.84 |  $-$12.72 |  $-$3.10
HRQ-VAE | 24.93 | 18.42 | 19.04
No head dropout |  $-$0.62 |  $-$0.74 |  $-$0.81
Post-hoc k-means |  $-$3.30 |  $-$5.35 |  $-$2.83
Table 6: Changes in iBLEU score for a range of ablations from our full model.
All components lead to an improvement in paraphrase quality across datasets.

##  6 Related Work

#### Hierarchical VAEs

VQ-VAEs were initially proposed in computer vision van den Oord et al. (2017),
and were later extended to be ‘hierarchical’ Razavi et al. (2019). However, in
vision the term refers to a ‘stacked’ version architecture, where the output
of one variational layer is passed through a CNN and then another variational
layer that can be continuous Vahdat and Kautz (2020) or quantized Williams et
al. (2020); Liévin et al. (2019); Willetts et al. (2021). Unlike these
approaches, we induce a _single_ latent space that has hierarchical
properties.

Other work has looked at using the properties of hyperbolic geometry to
encourage autoencoders to learn hierarchical representations. Mathieu et al.
(2019) showed that a model endowed with a Poincaré ball geometry was able to
recover hierarchical structure in datasets, and Surís et al. (2021) used this
property to deal with uncertainty in predicting events in video clips.
However, their work was limited to continuous encoding spaces, and the
hierarchy discovered was known to exist a priori.

#### Syntax-controlled Paraphrase Generation

Prior work on paraphrasing has used retrieval techniques Barzilay and McKeown
(2001), Residual LSTMs Prakash et al. (2016), VAEs Bowman et al. (2016), VQ-
VAEs Roy and Grangier (2019) and pivot languages Mallinson et al. (2017).
Syntax-controlled paraphrase generation has seen significant recent interest,
as a means to explicitly generate diverse surface forms with the same meaning.
However, most previous work has required knowledge of the correct or valid
surface forms to be generated Iyyer et al. (2018); Chen et al. (2019a); Kumar
et al. (2020); Meng et al. (2021). It is generally assumed that the input can
be rewritten without addressing the problem of predicting which template
should be used, which is necessary if the method is to be useful. Hosking and
Lapata (2021) proposed learning a simplified representation of the surface
form using VQ, that could then be predicted at test time. However, the
discrete codes learned by their approach are not independent and do not admit
a known factorization, leading to a mismatch between training and inference.

##  7 Conclusion

We present a generative model of paraphrasing, that uses a hierarchy of
discrete latent variables as a rough syntactic sketch. We introduce HRQ-VAE, a
method for mapping these hierarchical sketches to a continuous encoding space,
and demonstrate that it can indeed learn a hierarchy, with lower levels
representing more fine-grained information. We apply HRQ-VAE to the task of
paraphrase generation, representing the syntactic form of sentences as paths
through a learned hierarchy, that can be predicted during testing. Extensive
experiments across multiple datasets and a human evaluation show that our
method leads to high quality paraphrases. The generative model we introduce
has potential application for any natural language generation task;
$\textbf{z}_{sem}$ could be sourced from a sentence in a different language,
from a different modality (e.g., images or tabular data) or from a task-
specific model (e.g., summarization or machine translation). Furthermore, HRQ-
VAE makes no assumptions about the type of space being represented, and could
in principle be applied to a semantic space, learning a hierarchy over words
or concepts.

## Acknowledgements

We thank our anonymous reviewers for their feedback. This work was supported
in part by the UKRI Centre for Doctoral Training in Natural Language
Processing, funded by the UKRI (grant EP/S022481/1) and the University of
Edinburgh. Lapata acknowledges the support of the European Research Council
(award number 681760, “Translating Multiple Modalities into Text”).

## References

  * Akbik et al. (2018) Alan Akbik, Duncan Blythe, and Roland Vollgraf. 2018.  Contextual string embeddings for sequence labeling.  In _Proceedings of the 27th International Conference on Computational Linguistics_ , pages 1638–1649, Santa Fe, New Mexico, USA. Association for Computational Linguistics.
  * Bao et al. (2019) Yu Bao, Hao Zhou, Shujian Huang, Lei Li, Lili Mou, Olga Vechtomova, Xin-yu Dai, and Jiajun Chen. 2019.  Generating sentences from disentangled syntactic and semantic spaces.  In _Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics_ , pages 6008–6019, Florence, Italy. Association for Computational Linguistics.
  * Barzilay and McKeown (2001) Regina Barzilay and Kathleen R. McKeown. 2001.  Extracting paraphrases from a parallel corpus.  In _Proceedings of the 39th Annual Meeting of the Association for Computational Linguistics_ , pages 50–57, Toulouse, France. Association for Computational Linguistics.
  * Bowman et al. (2016) Samuel R. Bowman, Luke Vilnis, Oriol Vinyals, Andrew Dai, Rafal Jozefowicz, and Samy Bengio. 2016.  Generating sentences from a continuous space.  In _Proceedings of The 20th SIGNLL Conference on Computational Natural Language Learning_ , pages 10–21, Berlin, Germany. Association for Computational Linguistics.
  * Bowman et al. (2015) Samuel R. Bowman, Luke Vilnis, Oriol Vinyals, Andrew M. Dai, Rafal Józefowicz, and Samy Bengio. 2015.  Generating sentences from a continuous space.  _CoRR_ , abs/1511.06349.
  * Cao and Wan (2020) Yue Cao and Xiaojun Wan. 2020.  DivGAN: Towards diverse paraphrase generation via diversified generative adversarial network.  In _Findings of the Association for Computational Linguistics: EMNLP 2020_ , pages 2411–2421, Online. Association for Computational Linguistics.
  * Chen et al. (2019a) Mingda Chen, Qingming Tang, Sam Wiseman, and Kevin Gimpel. 2019a.  Controllable paraphrase generation with a syntactic exemplar.  In _Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics_ , pages 5972–5984, Florence, Italy. Association for Computational Linguistics.
  * Chen et al. (2019b) Mingda Chen, Qingming Tang, Sam Wiseman, and Kevin Gimpel. 2019b.  A multi-task approach for disentangling syntax and semantics in sentence representations.  In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)_ , pages 2453–2464, Minneapolis, Minnesota. Association for Computational Linguistics.
  * Devlin et al. (2018) Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018.  BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding.
  * Dong et al. (2017) Li Dong, Jonathan Mallinson, Siva Reddy, and Mirella Lapata. 2017.  Learning to paraphrase for question answering.  In _Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing_ , pages 875–886, Copenhagen, Denmark. Association for Computational Linguistics.
  * Fader et al. (2013) Anthony Fader, Luke Zettlemoyer, and Oren Etzioni. 2013.  Paraphrase-driven learning for open question answering.  In _Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pages 1608–1618, Sofia, Bulgaria. Association for Computational Linguistics.
  * Fu et al. (2019) Yao Fu, Yansong Feng, and John P Cunningham. 2019.  Paraphrase generation with latent bag of words.  In _Advances in Neural Information Processing Systems_ , volume 32, pages 13645–13656. Curran Associates, Inc.
  * Goyal and Durrett (2020) Tanya Goyal and Greg Durrett. 2020.  Neural syntactic preordering for controlled paraphrase generation.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 238–252, Online. Association for Computational Linguistics.
  * Hosking and Lapata (2021) Tom Hosking and Mirella Lapata. 2021.  Factorising meaning and form for intent-preserving paraphrasing.  In _Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)_ , pages 1405–1418, Online. Association for Computational Linguistics.
  * Huang and Chang (2021) Kuan-Hao Huang and Kai-Wei Chang. 2021.  Generating syntactically controlled paraphrases without using annotated parallel pairs.  In _Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume_ , pages 1022–1033, Online. Association for Computational Linguistics.
  * Iyyer et al. (2018) Mohit Iyyer, John Wieting, Kevin Gimpel, and Luke Zettlemoyer. 2018.  Adversarial example generation with syntactically controlled paraphrase networks.  In _Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers)_ , pages 1875–1885, New Orleans, Louisiana. Association for Computational Linguistics.
  * Jang et al. (2016) Eric Jang, Shixiang Gu, and Ben Poole. 2016.  Categorical reparameterization with gumbel-softmax.  _arXiv preprint arXiv:1611.01144_.
  * Jones (1972) Karen Spärck Jones. 1972.  A statistical interpretation of term specificity and its application in retrieval.  _Journal of Documentation_ , 28:11–21.
  * Kingma and Ba (2015) Diederik P. Kingma and Jimmy Ba. 2015.  Adam: A method for stochastic optimization.  In _3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings_.
  * Kingma and Welling (2014) Diederik P. Kingma and Max Welling. 2014.  Auto-encoding variational bayes.  In _2nd International Conference on Learning Representations, ICLR 2014, Banff, AB, Canada, April 14-16, 2014, Conference Track Proceedings_.
  * Koehn (2004) Philipp Koehn. 2004.  Statistical significance tests for machine translation evaluation.  In _Proceedings of the 2004 Conference on Empirical Methods in Natural Language Processing_ , pages 388–395, Barcelona, Spain. Association for Computational Linguistics.
  * Kumar et al. (2020) Ashutosh Kumar, Kabir Ahuja, Raghuram Vadapalli, and Partha Talukdar. 2020.  Syntax-guided controlled generation of paraphrases.  _Transactions of the Association for Computational Linguistics_ , 8(0):330–345.
  * Levelt (1993) Willem J. M. Levelt. 1993.  _Speaking: From Intention to Articulation_.  The MIT Press.
  * Liévin et al. (2019) Valentin Liévin, Andrea Dittadi, Lars Maaløe, and Ole Winther. 2019.  Towards hierarchical discrete variational autoencoders.
  * Lin et al. (2014) Tsung-Yi Lin, Michael Maire, Serge J. Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C. Lawrence Zitnick. 2014.  Microsoft coco: Common objects in context.  In _ECCV_.
  * Lin and Wan (2021) Zhe Lin and Xiaojun Wan. 2021.  Pushing paraphrase away from original sentence: A multi-round paraphrase generation approach.  In _Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021_ , pages 1548–1557, Online. Association for Computational Linguistics.
  * Lloyd (1982) S. Lloyd. 1982.  Least squares quantization in pcm.  _IEEE Transactions on Information Theory_ , 28(2):129–137.
  * Maddison et al. (2017) C Maddison, A Mnih, and Y Teh. 2017.  The concrete distribution: A continuous relaxation of discrete random variables.  In _Proceedings of the international conference on learning Representations_. International Conference on Learning Representations.
  * Madnani and Dorr (2010) Nitin Madnani and Bonnie J. Dorr. 2010.  Generating phrasal and sentential paraphrases: A survey of data-driven methods.  _Computational Linguistics_ , 36(3):341–387.
  * Mallinson et al. (2017) Jonathan Mallinson, Rico Sennrich, and Mirella Lapata. 2017.  Paraphrasing revisited with neural machine translation.  In _Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 1, Long Papers_ , pages 881–893, Valencia, Spain. Association for Computational Linguistics.
  * Martin et al. (2010) Randi C Martin, Jason E Crowther, Meredith Knight, Franklin P Tamborello II, and Chin-Lung Yang. 2010.  Planning in sentence production: Evidence for the phrase as a default planning scope.  _Cognition_ , 116(2):177–192.
  * Mathieu et al. (2019) Emile Mathieu, Charline Le Lan, Chris J. Maddison, Ryota Tomioka, and Yee Whye Teh. 2019.  Continuous hierarchical representations with poincaré variational auto-encoders.  In _Advances in Neural Information Processing Systems_.
  * Meng et al. (2021) Yuxian Meng, Xiang Ao, Qing He, Xiaofei Sun, Qinghong Han, Fei Wu, Chun fan, and Jiwei Li. 2021.  Conrpg: Paraphrase generation using contexts as regularizer.
  * Post (2018) Matt Post. 2018.  A call for clarity in reporting BLEU scores.  In _Proceedings of the Third Conference on Machine Translation: Research Papers_ , pages 186–191, Brussels, Belgium. Association for Computational Linguistics.
  * Prakash et al. (2016) Aaditya Prakash, Sadid A. Hasan, Kathy Lee, Vivek Datla, Ashequl Qadir, Joey Liu, and Oladimeji Farri. 2016.  Neural paraphrase generation with stacked residual LSTM networks.  In _Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers_ , pages 2923–2934, Osaka, Japan. The COLING 2016 Organizing Committee.
  * Razavi et al. (2019) Ali Razavi, Aäron van den Oord, and Oriol Vinyals. 2019.  _Generating Diverse High-Fidelity Images with VQ-VAE-2_. Curran Associates Inc., Red Hook, NY, USA.
  * Roy and Grangier (2019) Aurko Roy and David Grangier. 2019.  Unsupervised paraphrasing without translation.  In _Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics_ , pages 6033–6039, Florence, Italy. Association for Computational Linguistics.
  * Shah et al. (2018) Darsh Shah, Tao Lei, Alessandro Moschitti, Salvatore Romeo, and Preslav Nakov. 2018\.  Adversarial domain adaptation for duplicate question detection.  In _Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing_ , pages 1056–1063, Brussels, Belgium. Association for Computational Linguistics.
  * Shu et al. (2019) Raphael Shu, Hideki Nakayama, and Kyunghyun Cho. 2019.  Generating diverse translations with sentence codes.  In _Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics_ , pages 1823–1827, Florence, Italy. Association for Computational Linguistics.
  * Sønderby et al. (2017) Casper Kaae Sønderby, Ben Poole, and Andriy Mnih. 2017.  Continuous relaxation training of discrete latent variable image models.  In _Beysian DeepLearning workshop, NIPS_ , volume 201.
  * Sun and Zhou (2012) Hong Sun and Ming Zhou. 2012.  Joint learning of a dual SMT system for paraphrase generation.  In _Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)_ , pages 38–42, Jeju Island, Korea. Association for Computational Linguistics.
  * Surís et al. (2021) Dídac Surís, Ruoshi Liu, and Carl Vondrick. 2021.  Learning the predictability of the future.
  * Vahdat and Kautz (2020) Arash Vahdat and Jan Kautz. 2020.  NVAE: A deep hierarchical variational autoencoder.  In _Neural Information Processing Systems (NeurIPS)_.
  * van den Oord et al. (2017) Aaron van den Oord, Oriol Vinyals, and koray kavukcuoglu. 2017.  Neural discrete representation learning.  In _Advances in Neural Information Processing Systems_ , volume 30. Curran Associates, Inc.
  * van der Maaten and Hinton (2008) Laurens van der Maaten and Geoffrey E. Hinton. 2008.  Visualizing high-dimensional data using t-sne.  _Journal of Machine Learning Research_ , 9:2579–2605.
  * Vaswani et al. (2017) Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Ł ukasz Kaiser, and Illia Polosukhin. 2017.  Attention is all you need.  In _Advances in Neural Information Processing Systems_ , volume 30. Curran Associates, Inc.
  * Willetts et al. (2021) Matthew Willetts, Xenia Miscouridou, Stephen Roberts, and Chris Holmes. 2021.  Relaxed-responsibility hierarchical discrete vaes.
  * Williams et al. (2020) Will Williams, Sam Ringer, Tom Ash, David MacLeod, Jamie Dougherty, and John Hughes. 2020.  Hierarchical quantized autoencoders.  In _Advances in Neural Information Processing Systems_ , volume 33, pages 4524–4535. Curran Associates, Inc.
  * Xie et al. (2017) Ziang Xie, Sida I. Wang, Jiwei Li, Daniel Lévy, Aiming Nie, Dan Jurafsky, and Andrew Y. Ng. 2017.  Data noising as smoothing in neural network language models.  In _5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings_. OpenReview.net.

##  Appendix A Hyperparameters

The hyperparameters given in Table 7 were selected by manual tuning, based on
a combination of: (a) validation iBLEU scores with depth masking, (b)
validation BLEU scores using oracle sketches, and (c) validation iBLEU scores
using predicted syntactic codes.

Encoder/decoder |
---|---
Embedding dimension $D$ |  768
Encoder layers |  5
Decoder layers |  5
Feedforward dimension |  2048
Transformer heads |  8
Semantic/syntactic dim |  192/594
Depth $D$ |  3
Codebook size $K$ |  16
Optimizer |  Adam Kingma and Ba (2015)
Learning rate |  0.01
Batch size |  64
Token dropout |  0.2 Xie et al. (2017)
Decoder |  Beam search
Beam width |  4
Code predictor |
Num. hidden layers |  2
Hidden layer size |  3072
Table 7: Hyperparameter values used for our experiments.

The Gumbel temperature $\tau$ is decayed during training as a function of the
step $t$, according to the following equation:

$\tau(t)=\max(2-\frac{2}{1+e^{t/10000}},0.5).$ (8)

Intuitively, this smoothly decays $\tau$ from an initial value of 2, with a
half-life of 10k steps, to a minimum value of 0.5.

We use $\alpha=0.8$ when calculating iBLEU, but as shown in Figure 5 our
conclusions are not sensitive to this value, and our model outperforms all
comparison systems on all datasets for $0.7\leq\alpha\leq 0.9$.

Figure 5: iBLEU scores for all comparison systems, for a range of values of
$\alpha$.

Models were trained on a single GPU, with training taking between one and
three days depending on the dataset. We use SacreBLEU Post (2018) to calculate
BLEU scores.

##  Appendix B Human Evaluation

Annotators were recruited from the UK and USA via Amazon Mechanical Turk, and
were compensated for their time above a living wage in those countries. A full
Participant Information Sheet was provided, and the study was approved by an
internal ethics committee. Annotators were asked to rate the outputs according
to the following criteria:

  * •

Which system output is the most fluent and grammatical?

  * •

To what extent is the meaning expressed in the original sentence preserved in
the rewritten version, with no additional information added?

  * •

Does the rewritten version use different words or phrasing to the original?
You should choose the system that uses the most different words or word order.

##  Appendix C Exemplar Retrieval Process

Input | How heavy is a moose?
---|---
Chunker output | How [heavy]ADVP is a [moose]NP ?
Template | How ADVP is a NP ?
Exemplar | How much is a surgeon’s income?
Input | What country do parrots live in
Chunker output | What [country]NP do [parrots]NP [live]VP in ?
Template | What NP do NP VP in ?
Exemplar | What religion do Portuguese believe in?
Table 8: Examples of the exemplar retrieval process for training. The input is
tagged by a chunker, ignoring stopwords. An exemplar with the same template is
then retrieved from a different paraphrase cluster. Table reproduced with
permission from Hosking and Lapata (2021).

Our approach requires exemplars during training to induce the separation
between latent spaces. We follow the approach introduced by Hosking and Lapata
(2021). During training, we retrieve exemplars $\textbf{x}_{syn}$ from the
training data following a process which first identifies the underlying syntax
of Y, and finds a question with the same syntactic structure but a different,
arbitrary meaning. We use a shallow approximation of syntax, to ensure the
availability of equivalent exemplars in the training data. An example of the
exemplar retrieval process is shown in Table 8; we first apply a chunker
(FlairNLP, Akbik et al., 2018) to Y, then extract the chunk label for each
tagged span, ignoring stopwords. This gives us the template that Y follows. We
then select a question at random from the training data with the same template
to give $\textbf{x}_{syn}$. If no other questions in the dataset use this
template, we create an exemplar by replacing each chunk with a random sample
of the same type.

##  Appendix D Analysis of Code Properties

(a) Distribution of verbs for each code within level 1.

(b) Distribution of wh- words for each code within level 2.

Figure 6: Plots showing the conditional distributions of two different
sentence features, auxiliary verb and question type, for different values of
the latent codes $q_{d}$. Each column represents the distribution over the
feature for a specific code. The plots show that level 1 is a strong predictor
of verb tense, and level 2 predicts question type, giving some insight into
what syntactic features each level has learned to encode. We have reordered
the columns of the plot to improve readability.

We define two features of sentences: (1) the presence of common auxiliary
verbs that roughly indicate the tense of the sentence (present, future, etc.);
and (2) the presence of different question or ‘wh-’ words222This analysis was
performed for Paralex, which comprises entirely of questions.. We calculate
the distributions of these features for each code $q_{d}$ at different levels,
with the results shown in Figure 6. Each column represents the distribution
over the feature for a specific code. Figure 6(a) shows clear evidence that
the sentences are (at least partly) clustered at the top level based on the
verb used, while Figure 6(b) shows that level 2 encodes the question type.


#  Local spin structure in the layered van der Waals materials MnPSxSe3-x

Raju Baral  baralr@ornl.gov;lead contact Neutron Scattering Division, Oak
Ridge National Laboratory, Oak Ridge, Tennessee 37831, USA.  Amanda V. Haglund
Department of Materials Science and Engineering, University of Tennessee,
Knoxville, TN 37996, USA.  Jue Liu  Neutron Scattering Division, Oak Ridge
National Laboratory, Oak Ridge, Tennessee 37831, USA.  Alexander I. Kolesnikov
Neutron Scattering Division, Oak Ridge National Laboratory, Oak Ridge,
Tennessee 37831, USA.  David Mandrus  Department of Materials Science and
Engineering, University of Tennessee, Knoxville, TN 37996, USA.  Materials
Science and Technology Division, Oak Ridge National Laboratory, Oak Ridge, TN
37831, USA.  Stuart Calder<EMAIL_ADDRESS>Neutron Scattering Division, Oak
Ridge National Laboratory, Oak Ridge, Tennessee 37831, USA.

###### Abstract

Two-dimensional (2D) layered materials, whether in bulk form or reduced to
just a single layer, have potential applications in spintronics and capacity
for advanced quantum phenomena. A prerequisite for harnessing these
opportunities lies in gaining a comprehensive understanding of the spin
behavior in 2D materials. The low dimensionality motivates an understanding of
the spin correlations over a wide length scale, from local to long range
order. In this context, we focus on the magnetism in bulk MnPSe3 and MnPS3, 2D
layered van der Waals antiferromagnetic semiconductors. These materials have
similar honeycomb Mn layers and magnetic ordering temperatures, but distinct
spin orientations and exchange interactions. We utilize neutron scattering to
gain deeper insights into the local magnetic structures and spin correlations
in the paramagnetic and ordered phases by systematically investigating a
MnPSxSe3-x ($x$ = 0, 1, 1.5, 2, 3) series of powder samples using total
neutron scattering measurements. By employing magnetic pair distribution
function (mPDF) analysis, we unraveled the short-range magnetic correlations
in these materials and explored how the non-magnetic anion S/Se mixing impacts
the magnetic correlations. The results reveal that the magnetism can be
gradually tuned through alteration of the non-magnetic S/Se content, which
tunes the atomic structure. The change in magnetic structure is also
accompanied by a control of the magnetic correlation length within the 2D
honeycomb layers. Complimentary inelastic neutron scattering measurements
allowed a quantification of the change in the magnetic exchange interactions
for the series and further highlighted the gradual evolution of spin
interactions in the series MnPSxSe3-x.

##  I Introduction

Figure 1:  The total neutron PDF (including both magnetic and nuclear
scattering) and isolated magnetic PDF of (a) MnPSe3 and (b) MnPS3 at 15 K. The
blue symbols and red curve represent the experimental total PDF data and total
PDF fit, respectively. The gray and black curves represent the isolated
magnetic PDF data and magnetic PDF fit (vertically offset for clarity). The
green curve represents the overall fit residual. (c) Crystal structure of
MnPSe3 with Mn spins pointing in $ab$-plane (space group=R$\bar{3}$). (d)
Crystal structure of MnPS3 with Mn spins pointing out of the $ab$-plane (space
group=C${2}/m$). The hexagonal motif of the Mn ions for (e) MnPSe3 and (f)
MnPS3 . Purple, gray, green, and yellow spheres represent the Mn, P, Se, and S
atoms respectively.

Reducing the dimensionality of a compound down to topologically constrained
layers can uncover new and intriguing behaviors that go beyond well-
established classical understanding. A prime example is graphene, in which a
single layer is extracted from the weakly connected van der Waals (VDW) bonded
layers in graphite [1]. Graphene and other 2D materials have undergone
significant interest stemming from their potential to exhibit novel quantum
effects, such as Dirac semi-metal and quantum anomalous Hall insulating
behavior [2, 3]. Interesting properties can be found in materials ranging from
the ideal two-dimensional (2D) monolayers to qausi-2D materials with more
complex bulk VDW-bonded structures [4]. Developing functional spintronic
materials typically relies on harnessing magnetic and semiconducting
properties, however, graphene lacks these phenomena. As a result, 2D VDW
materials that have intrinsic magnetism and semiconductivity with a honeycomb
arrangement are of interest. A prerequisite for developing functional magnetic
materials and uncovering novel quantum behavior in 2D materials lies in
gaining a comprehensive understanding of the spin behavior over the relevant
length scales.

MnPS3 and MnPSe3 are layered VDW antiferromagnetic materials that posses both
magnetic and semiconducting properties, with the magnetic ions adopting the
same hexagonal motif as graphene. Although each crystallizes in different
space groups, MnPS3 in C$2/m$ and MnPSe3 in R$\bar{3}$, they have nearly
identical Néel temperatures of 78 K for MnPS3 and 74 K for MnPSe3 [5, 6]. Both
of these compositions contain an equivalent fraction of magnetic ions ($n_{s}$
= 1/5) within their unit cells and are S = 5/2 materials. Despite their
similar transition temperatures and magnetic ion content, intriguing
distinctions become apparent in their magnetic behavior. MnPS3 has Heisenberg
antiferromagnetic spins that point out of the hexagonal $ab$-plane [7], while
MnPSe3 displays XY anisotropy in its magnetic behavior with spins lying in the
$ab$-plane [8]. The crystal structures with spin directions are shown in
Fig.1.

By altering the non-magnetic S and Se ion content it is possible to chemically
tune between MnPS3 and MnPSe3 in the series MnPSxSe3-x. This provides a way to
explore and understand the evolution between the two magnetic states.
Stabilization of this series has been shown for a wide concentration of $x$
values for powders and single crystal samples [9, 10, 11, 12]. Based on bulk
characterizations, these studies have found indications of a change in the
magnetic ordering within the series as $x$ is altered. This motivates further
investigation to probe the microscopic spin states and magnetic exchange
interactions.

In this work, we present total neutron scattering and inelastic neutron
scattering measurements to achieve a comprehensive picture of the long-range
and short-range correlations and exchange interactions in the series
MnPSxSe3-x (x = 0, 1, 1.5, 2, 3). Using magnetic pair distribution function
(mPDF) analysis of total neutron scattering data, we present the real-space
visualization of local spin-spin correlations in the paramagnetic state. The
neutron scattering data reveals signatures of short-range magnetic
correlations well above the Néel temperature for MnPS3 and we modeled this
with mPDF analysis to extract the short- and long-range spin states. As the S
content increases towards MnPSe3 the quasi-2D behavior is reduced, as shown by
the decrease in the magnetic correlation length. This behavior is confirmed by
inelastic neutron scattering measurements that show an increase in the
interlayer exchange interactions as the Se content increases. We show that
control of the non-magnetic S and Se content provides a sensitive mechanism to
tune the magnetic spin direction between in-plane and out-of-plane through
alteration of the crystal structure and small modifications in the Mn-Mn
distances. This suggests routes beyond chemical pressure, such as strain,
could provide promising routes to tune the magnetism in layered materials.
Moreover the methodology employed of mPDF, that can access various spin length
scales over a wide temperature range, is shown to be well suited to 2D VDW
materials that can intrinsically contain competing dimensionality of spin
ordering.

##  II Experimental methods

###  II.1 Sample synthesis

Polycrystalline samples of MnPSe3 and MnPS3 were synthesized following the
standard solid-state reaction method as described in Ref. [8]. The samples
MnPSSe2, MnPS1.5Se1.5, and MnPS2Se were similarly synthesized using the
identical method with suitable precursor materials. Confirmation of sample
quality was accomplished through laboratory X-ray diffraction analysis.

###  II.2 Neutron total scattering

Neutron total scattering experiments were conducted on both the Nano Scale-
Ordered Material Diffractometer (NOMAD) and the HB-2A powder diffractometer.
NOMAD is a dedicated total scattering instrument for PDF analysis. The HB-2A
powder diffraction instrument is traditionally utilized for reciprocal space
analysis at low $Q$, however here we use a short wavelength to access higher
$Q$ and perform total scattering measurements and PDF analysis. With the
capabilities to measure samples at millikelvin temperatures and apply magnetic
fields HB-2A can provide unique conditions for magnetic PDF analysis for a
broad range of quantum material research.

For both the HB-2A and NOMAD data the atomic PDF fits were performed using
PDFgui [13] and the magnetic PDF fits were obtained using diffpy.mpdf [14], a
Python package designed for magnetic PDF calculations and fitting.

####  II.2.1 NOMAD

Measurements on MnPSe3 and MnPS3 were conducted at NOMAD, located at the
Spallation Neutron Source (SNS), Oak Ridge National Laboratory (ORNL) [15].
These polycrystalline samples (MnPSe3 and MnPS3) were loaded into 6 mm
vanadium cans in a helium glove box. Total scattering patterns were collected
in a helium cryostat over a range of temperatures from 15 K to 300 K. The
collected data was reduced and Fourier transformed with a Qmax value of 20
$\rm\AA$-1, employing automatic data reduction scripts at NOMAD [16]. The
choice of Qmax value is critical. A high Qmax value can significantly improve
the resolution in real space, but it also introduces statistical noise and
artifacts. Consequently, a Qmax value of 20 $\rm\AA$-1 was strategically
selected to optimize the balance between achieving high resolution and
minimizing noise artifacts.

####  II.2.2 HB-2A

Neutron total scattering patterns of the doped samples (MnPSSe2, MnPS1.5Se1.5,
and MnPS2Se) were collected on the HB-2A powder diffractometer at the High
Flux Isotope Reactor (HFIR) at ORNL[17]. A vertically focusing germanium
monochromator was used to select the wavelength of 1.12 Å from the Ge(117)
reflection. Approximately 5 grams of each sample were loaded into a vanadium
sample holder inside a helium glove box. The total neutron scattering data for
the doped samples were collected at temperatures of 15, 70, 125 and 300 K,
with a collection time of four hours for each temperature point. The collected
data were normalized using a vanadium standard measurement. The detector
position and the wavelength of the neutron beam were calibrated with a Si
measurement. Since HB-2A does not typically serve as a total scattering
instrument, we processed the data further by performing a Fourier-transform
using PDFgetN3 [18] with a Qmax value of 10 Å-1 to produce the PDF data.

Figure 2:  Temperature series of the isolated magnetic PDF data and fits for
MnPSe3 (left) and MnPS3 (right). The colored curves are the magnetic PDF data
and the black curve represents the magnetic PDF fit (vertically offset for
clarity). The long-range ordering transition temperature ($T_{\mathrm{N}}$) is
shown by the blue horizontal dashed line.

###  II.3 Inelastic neutron scattering

Inelastic neutron scattering (INS) measurements were conducted on MnPSSe2,
MnPS1.5Se1.5, and MnPS2Se using the Time-of-Flight Direct Geometry
Spectrometer Sequoia [19] at the SNS, ORNL. An identical set-up was used
compared to previous measurements on MnPSe3 and MnPS3 [8]. The samples were
loaded into a cylindrical aluminum can with a diameter of 1 cm and INS spectra
were measured using the three-sample changer within a closed-cycle
refrigerator. Data were collected at incident energies of $E_{i}$ = 20 meV and
$E_{i}$ = 8 meV, both in high-resolution mode. For $E_{i}$ = 20 meV, the Fermi
chopper operated at a frequency of 240 Hz, with a $T_{0}$ chopper frequency of
60 Hz, resulting in an energy resolution of 0.48 meV at the elastic line.
Similarly, for $E_{i}$ = 8 meV, the Fermi chopper frequency was 120 Hz, and
the $T_{0}$ chopper operated at 30 Hz, providing an energy resolution of 0.18
meV at the elastic line. The energy resolution was determined through DGS
resolution, confirmed by analyzing cuts of the elastic line within the data.
Momentum resolution ($Q$) was derived by fitting the resolution of the Bragg
peaks at the elastic line. Measurements were taken over six hours at
temperatures of 200 K, 100 K, 70 K, and 15 K.

The collected neutron data underwent reduction using Mantid [20]. The data
were adjusted for the Bose thermal population factor. Initial data reduction
and viewing were carried out using the DAVE software [21]. Modeling of the
data was performed using SpinW [22].

##  III Results and discussion

###  III.1 Atomic and magnetic PDF analysis of MnPSe3 and MnPS3

We begin with neutron total scattering measurements on MnPSe3 and MnPS3, with
a focus on probing the spin ordering over a wide temperature range through the
magnetic ordering transtions. This is achieved through atomic and magnetic PDF
analysis of the neutron scattering data, which allows for the same analysis
methodology to be applied in both the long range ordered phase and short range
paramagnetic phase. Fig. 1(a)-(b) shows the results at 15 K for MnPSe3 and
MnPS3. The total scattering data include both atomic and magnetic scattering,
which need to be disentangled. To isolate the magnetic data we employed an
iterative fitting approach. Initially, we used the known atomic structure to
perform the atomic PDF fit [8]. Then, we subtracted the calculated atomic PDF
from the total PDF data to obtain the residual. This residual contains a
magnetic signal which was used for the initial magnetic PDF refinement.
Following this, we refined the atomic PDF again, but this time using the total
PDF data minus the best-fit magnetic PDF signal for the atomic PDF input. The
refined parameters in the first iteration served as starting points for the
second iteration. We repeated this process for a third iteration, starting
with the total PDF data minus the best-fit atomic PDF pattern from the second
iteration, using the refined parameters from the second iteration. Finally, a
third magnetic PDF fit was conducted. The detailed procedure of the iterative
fitting approach of atomic and magnetic PDF is further discussed in the
Supplementary section of Ref.[23].

The analysis was performed on a temperature grid ranging from 15 K to 300 K
and the atomic and magnetic PDF patterns were calculated over a real space
fitting range of 1.5 - 20 Å. For MnPSe3 the low goodness of fit, $R_{w}$ =
0.0742, is evident from the small, featureless fit residual illustrated by the
green curve at the bottom of the panel. When allowed to freely refine, the
lowest $R_{w}$ occurs when the spins are at an angle of 70∘ with the $c$-axis,
however using the previously reported value of 90∘, where the spins are fixed
to the $ab$-plane, produces a nearly identical fit [8]. Fig. 1(b) shows the
atomic and magnetic PDF results for MnPS3, with $R_{w}$ = 0.139. The optimal
magnetic fit occurs when the spins are aligned at an angle of 14.17 $\pm$
2.09∘ with c-axis, which is comparable to previous studies [7].

We performed a series of atomic and magnetic PDF fits for various temperatures
ranging from 15 to 300 K for both compositions. Fig. 2 shows the temperature
series of the magnetic PDF fits for MnPSe3 and MnPS3. For MnPSe3 short range
magnetic order persists above the Néel temperature ($T_{\mathrm{N}}$= 74 K),
up to 100 K and then decreases significantly beyond 100 K becoming almost
unobservable at 200 K. Conversely in the case of MnPS3, we observed a strong
magnetic PDF signal well above the Néel temperature ( $T_{\mathrm{N}}$= 78 K),
with this local order magnetic signal persisting up to 300 K, which is the
maximum temperature that the data were collected.

Figure 3:  (a) Magnetic moments as a function of temperature for MnPSe3 and
MnPS3, evaluated within a fitting range from 1.5 to 20 Å. The red and blue
symbols represent the ordered moment for MnPSe3 and MnPS3, respectively. (b)
Variation of the correlation length with temperature in the paramagnetic
regime for MnPSe3 and MnPS3, based on isotropic fits in the range of 1.5 to 20
Å. (c) Best fit correlation lengths in the $ab$-plane (blue squares) and along
the $c$-axis (orange triangles) as a function of temperature in a paramagnetic
regime for MnPS3using an anisotropic model. The error bars denote the standard
deviation of the best-fit parameter values. Figure 4:  Power law fit to the
integrated intensity of a magnetic Bragg peak in the range $Q$ = 2.68 to 2.73
$\rm A^{-1}$ for (a) MnPSSe2, (b) MnPS1.5Se1.5 and (c) MnPS2Se.

The mPDF analysis allows for a quantitative extraction of the magnetic
correlation length and the ordered moment over a defined real space range. The
ordered moment in the fitting range 1.5 - 20 Å as a function of temperature
for MnPSe3 and MnPS3 are shown in Fig. 3(a). In the long range ordered regime
the ordered moment begins to decrease exponentially as the sample temperature
is increased towards the Néel temperature in both cases. With a further
increase in temperature above the Néel temperature, the ordered moment further
decreases. For MnPSe3 the ordered moment approaches zero for temperatures of
150 K and above. For MnPS3, however, while the ordered moment also decreases
above the Néel temperature it attains a substantial moment of
1.7$\mu_{\mathrm{B}}$ at 300 K, 1/3 of the total moment for Mn. The
correlation length above the transition for both compositions, MnPS3 and
MnPSe3, is illustrated in Fig. 3(b). Below the transition temperature long-
range order is established with an infinite correlation length. But above the
transition with increasing temperature, the correlation length decreases for
both compositions. MnPSe3 has an uncorrelated moment at 150K, whereas the
correlations persist in MnPS3 to 300 K.

We conducted additional magnetic PDF fits for MnPS3 above $T_{\mathrm{N}}$
over a fitting range 1.5 - 20 Å to capture the anisotropic magnetic
correlations to further understand the persistence of local order. We
implemented a model that distinguishes between correlation lengths in the
$ab$-plane and along the $c$-axis. Our analysis reveals that the in-plane
correlation ($\xi_{ab}$) length is approximately 15 times longer than out of
plane correlation length ($\xi_{c}$) for all temperatures above
$T_{\mathrm{N}}$( see Fig. 3 (c)). This highlights that bulk MnPS3 well
approximates a 2D magnet. Undertaking the same anisotropic procedure for
MnPSe3 did not produce a clear distinction between the $ab$-plane and
$c$-axis, within error. This indicates that the magnetism in MnPSe3 is closer
to three dimensional, despite the presence of a van der Waals gap between the
layers.

Magnetic PDF measurements are not yet widely applied to 2D layered materials,
but they can offer detailed insights. The results here show that MnPS3
displays pronounced short-range magnetic correlations within the 2D layers
well above the long range transition temperature, while this behavior is not
found in MnPSe3. This can be observed on a qualitative level from neutron
diffraction data in reciprocal space, however mPDF analysis allows for a
quantitative description in real space that can be applied to various ordering
length scales both above and below the Néel temperature.

###  III.2 Magnetic PDF analysis of MnPSSe2, MnPS1.5Se1.5, and MnPS2Se

Figure 5:  Total neutron and isolated magnetic PDF patterns of (a) MnPSe3 (b) MnPSSe2, (c) MnPS1.5Se1.5, (d) MnPS2Se and (e) MnPS3 at 15 K. The blue, red, gray, black, and green symbols represent the observed PDF data, total PDF fit, isolated magnetic PDF data, magnetic PDF fit, and overall fit residual. (f) Variation of spin angle (theta) with $x$-content for the series MnPSxSe3-x at 15 K temperature. (g) Variation of nearest neighbor Mn-Mn and Mn-S/Se distances ($\text{d}_{\text{NN}}$) with $x$-content at 15 K for MnPSxSe3-x. Table 1: Ordered magnetic moment ($m$), spin-angle ($\theta$) with $c$-axis, and Néel temperatures of MnPSxSe3-x ($x$ = 0, 1, 1.5, 2, 3) at 15 K. The spin-angle for MnPS3 was fixed to 90 degrees, as described in the text. Samples |  $m$ ($\mu_{B}$) |  $\theta$ (degree) |  $T_{\mathrm{N}}$ (K)
---|---|---|---
MnPS3 | 4.884 $\pm$ 0.036 | 14.17 $\pm$ 2.091 | 78 Ref. [5]
MnPS2Se | 4.313 $\pm$ 0.186 | 20.076 $\pm$ 5.382 | 75.31 $\pm$ 0.63
MnPS1.5Se1.5 | 4.370 $\pm$ 0.132 | 55.260 $\pm$ 3.534 | 75.82 $\pm$ 1.35
MnPSSe2 | 4.470 $\pm$ 0.109 | 58.039 $\pm$ 2.529 | 75.42 $\pm$ 0.65
MnPSe3 | 4.666 $\pm$ 0.149 | 90 | 74 Ref. [6]

To gain further insight into the distinct behavior of MnPSe3 and MnPS3, we
conducted atomic and magnetic PDF analysis on samples with controlled S/Se
content of MnPSSe2, MnPS1.5Se1.5, and MnPS2Se to tune and investigate the
behavior between the two materials. The same analysis procedure was applied to
these samples as was used for MnPSe3 and MnPS3, however the measurements were
conducted on different neutron scattering instruments. Data was collected at
15, 70, 125 and 300 K. The atomic and magnetic PDF calculations for MnPSSe2,
MnPS1.5Se1.5 were performed over the same fitting range of 1.5 - 20
$\mathrm{\AA}$ for MnPSSe2 and MnPS1.5Se1.5, while for MnPS2Se, a different
fitting range of 1.5 - 15 $\mathrm{\AA}$ was selected. The variation of the
fitting range on MnPS2Se was due to a notable degradation in the quality of
the fit beyond 15 $\mathrm{\AA}$. There could be several factors for this,
such as increased stacking faults or in-plane disorder from the site mixing
that lead to this behavior.

To obtain the magnetic ordering temperature of MnPSSe2, MnPS1.5Se1.5, and
MnPS2Se we performed neutron diffraction measurements from 25 to 125 K and
obtained the integrated intensity of a magnetic reflection, as shown in Fig.
4. Fitting this to a power law shows that the ordering temperature of MnPSSe2,
MnPS1.5Se1.5, and MnPS2Se all occur within error around the same temperature
of 75 K (see table 1), which is in between the ordering temperatures of 74 K
for MnPSe3 and 78 K for MnPS3.

The same iterative fitting approach was used to refine the atomic and magnetic
PDF patterns as described in subsection III.1. Fig. 5 shows the total PDF and
isolated magnetic PDF analysis patterns for all MnPSxSe3-x ($x$ = 0, 1, 1.5,
2, 3) compositions at 15 K. The best fit atomic and magnetic models for
MnPSSe2, MnPS1.5Se1.5 were obtained using the space group R$\bar{3}$ which is
consistent with the space group of MnPSe3. Similarly, the best atomic and
magnetic fits for MnPS2Se) were observed using the space group C${2}/m$, which
corresponds to the space group for MnPS3. Attempts to use alternative space
groups did not produce improved fits for any compound. On comparing the mPDF
patterns of all the compositions of $x$ = 0, 1, 1.5, 2, 3, it is observed that
the real-space magnetic intensity slowly increases with $x$-content,
suggesting an alteration of the magnetic order. Performing the mPDF fits to
the isolated magnetic data reveals a gradual increase of the spin angle with
alteration of the $x$ content in MnPSxSe3-x. Fig. 5(f) represents the
variation of spin angle with $x$-content at 15 K. Table 1 shows the variation
of calculated order moment and the spin angle of MnPSxSe3-x ($x$ = 0, 1, 1.5,
2, 3) at 15 K. As discussed in section III.1 the spin angle for MnPSe3 was
indistinguishable between 70 degrees and the previously reported value of 90
degrees, as such 90 degrees is presented here.

The results of the mPDF analysis in Fig. 5 show that a gradual change in the
non-magnetic S/Se content is a parameter that can be used to directly control
the magnetic spin direction. The underlying change in this process is to the
crystal structure through chemical pressure as the different sized anions are
altered. Fig. 5(g) shows the variation of nearest neighbor Mn-Mn and Mn-S/Se
distances with $x$-content for MnPSxSe3-x at 15 K. With progressive doping,
both nearest neighbor Mn-Mn and Mn-S/Se distances decreases gradually with
$x$-content. The decrease in the nearest neighbor Mn-Mn distance would be
expected to lead to an increase in magnetic exchange interactions in the
$ab$-plane. In addition, the change in Mn-S/Se distance will alter the crystal
field environment and local anisotropy of the magnetic ion.

##  IV Inelastic Neutron Scattering

Figure 6:  Inelastic neutron scattering data and simulations for MnPSxSe3-x.
Data collected with Ei=20 meV for (a) MnPS3, (b) MnPS2Se, (c) MnPS1.5Se1.5,
(d) MnPSSe2 and (e) MnPSe3. (f)-(j) Best fit models using linear spin wave
theory. Data collected with Ei=8 meV for (k) MnPS3, (l) MnPS2Se, (m)
MnPS1.5Se1.5, (n) MnPSSe2 and (o) MnPSe3. (p)-(t) Best fit models using linear
spin wave theory.

To investigate the influence of altering the non-magnetic S and Se content on
the magnetic exchange interactions in the series MnPSxSe3-x further we
performed INS measurements on the powder samples. Measurements of the undoped
compounds MnPSe3 and MnPS3 have been previously performed to extract the
exchange interactions [8, 24], these values are shown in Table. 2. Comparing
the values shows that MnPS3 has stronger in-plane interactions, however the
interlayer interactions are an order of magnitude smaller than MnPSe3. This
places the bulk MnPS3 compound closer to the ideal 2D limit. This behavior was
observed in the mPDF analysis reported above. In addition MnPS3 has spin
anisotropy, as evidenced by a measurable spin-gap in INS [24], which was
modeled with single-ion anisotropy. No such spin-gap was observed in MnPSe3.
Given these difference in the exchange interactions, it is somewhat surprising
that MnPS3 and MnPSe3 order within a few degrees K of each other. The stronger
(weaker) $ab$-plane interactions and weaker (stronger) interlayer interactions
for MnPS3 (MnPSe3) appear to lead to a balance that results in similar
ordering temperatures. How these interactions change in-between the two parent
compounds is the focus of the INS measurements on MnPSxSe3-x ($x=1,1.5,2$).

The INS data for the full series MnPS3, MnPS2Se, MnPS1.5Se1.5, MnPSSe2, MnPSe3
are shown in Fig. 6. An evolution is observed in the magnetic excitation
spectra as the non-magnetic S and Se ion contents are altered. MnPS3 has the
widest energy bandwidth, indicating stronger magnetic exchange interactions,
and this gradually reduces as S is replaced with Se. In addition, there is an
evolution in the low energy scattering, with the reported spin-gap in MnPS3
reducing and then becoming unobservable. This is replaced with low energy
scattering revealing appreciable interlayer exchange interactions, as observed
most strongly in MnPSe3.

To extract quantitative information on MnPS2Se, MnPS1.5Se1.5 and MnPSSe2 we
follow a similar modeling process as that performed in Ref. [8]. We use linear
spin wave theory with a model based on a 2D Hamiltonian on a honeycomb
lattice,

$\displaystyle\mathcal{H}=\sum_{i\neq
j}J_{ij}\mathbf{S}_{i}\cdot\mathbf{S}_{j}=\sum_{\langle
ij\rangle}2J_{ij}\mathbf{S}_{i}\cdot\mathbf{S}_{j}$

that included $J_{1ab}$, $J_{2ab}$ and $J_{3ab}$ intralayer exchange
interactions, as well as the full spin (S=5/2). Note the exchange interactions
have been reported to be consistent with parameters in the literature and
allow comparisons between MnPSe3 and MnPS3 [24, 25, 26, 27]. Alternative
notation related by $J$/2 is used for MnPS3 in Ref. [28].

The top of the band for MnPS3 and MnPSe3 are sharp and resolution limited for
the Ei=20 meV measurements. There is some broadening for the MnPS2Se,
MnPS1.5Se1.5 and MnPSSe2. This can be attributed to a level of structural
disorder introduced in the S and Se mixed compounds which would damping the
excitations, as often observed in doped samples. This was accounted for in the
modeling by broadening the energy resolution for the $x=1,1.5,2$ analysis. A
series of 1D cuts were taken at constant Q and constant E during the fitting
process. To restrict the number of variables to allow for the extraction of a
minimal model the $J_{1ab}$, $J_{2ab}$ and $J_{3ab}$ interactions for MnPSe3
and MnPS3 where used as starting values. A scale factor was applied to model
the top of the band of magnetic excitations. These values were then fixed and
the interlayer interaction ($J_{c}$) and anisotropy in the form of SIA were
altered in the subsequent modeling. The determined values are shown in Table.
2 and the comparison of experimental data and model are shown in Fig. 6.

The data and modeling shows a consistent trend of reducing the in-plane
interactions and increasing the interlayer interactions as the S content is
diluted with Se. This trend in exchange interactions and magnetic excitation
spectra gradually altering agrees with the gradual switching of spins from in-
plane (MnPSe3) to almost fully along the c-axis in (MnPS3) with altering S/Se
content as found in the mPDF analysis.

Table 2: Exchange interactions from modeling the INS data with linear spin wave theory. All values are in meV. | MnPS3 | MnPS2Se | MnPS1.5Se1.5 | MnPSSe2 | MnPSe3
---|---|---|---|---|---
| Ref. [24] |  |  |  | Ref. [8]
$J_{1ab}$ | 0.77 | 0.716 | 0.529 | 0.501 | 0.45
$J_{2ab}$ | 0.07 | 0.065 | 0.035 | 0.033 | 0.03
$J_{3ab}$ | 0.18 | 0.167 | 0.218 | 0.207 | 0.19
$J_{c}$ | -0.0038 | – | 0.019(4) | 0.025(3) | 0.031
SIA | 0.0086 | 0.0045(15) | – | – | –

##  V Conclusion

In summary, the magnetic behavior of a series of 2D layered van der Waals
materials MnPSxSe3-x ($x$ = 0, 1, 1.5, 2, 3) were investigated through atomic
and magnetic PDF analysis and inelastic neutron scattering. The magnetic
ordering temperature was found to remain within a few Kelvin (74 K - 78 K) for
the whole series, however the spin direction, correlation length and exchange
interactions evolve over a wide range as $x$ is tuned.

The mPDF signal was isolated from the total scattering data to follow the
magnetic behavior over a wide range of temperature and $x$ composition. The
mPDF results revealed short-range antiferromagnetic correlations in the
paramagnetic regime. MnPS3 was found to have a significantly longer
correlation length compared to MnPSe3, with short range order in the 2D layers
existing up to room temperature. A progressive increase in Se-doping leads to
a notable shift in spin orientation from out-of-plane to in-plane. This is
found to occur gradually with $x$ concentration.

The exchange interactions were parameterized by a minimal model
antiferromagnetic Heisenberg Hamiltonian on the honeycomb lattice. These
showed a gradual change in the in-plane interactions, which were largest for
MnPS3 and decreased with increasing $x$. Conversely the interlayer
interactions are largest for MnPSe3 and increase with $x$.

The change in the magnetism in MnPSxSe3-x ($x$ = 0, 1, 1.5, 2, 3) was induced
by altering the non-magnetic S/Se ions which alters the chemical pressure,
therefore structural control is central to the magnetic behavior. This
highlights the ability to design and tune the magnetism in the 2D layers of
MnPSxSe3-x by not only controlling the $x$, but also through other
perturbations including strain and pressure. More generally this investigation
shows how the application of real space analysis of neutron data with mPDF can
provide insights into 2D layered van der Waals materials where understanding
local and long range ordering over a wide range of temperature of structural
composition is often of interest.

## acknowledgments

The authors thank Benjamin Frandsen for the useful discussion. This research
used resources at the High Flux Isotope Reactor and Spallation Neutron Source,
a DOE Office of Science User Facility operated by the Oak Ridge National
Laboratory. D.M. acknowledges support from the Gordon and Betty Moore
Foundation’s EPiQS Initiative, Grant GBMF9069. This manuscript has been
authored by UT-Battelle, LLC under Contract No. DE-AC05-00OR22725 with the
U.S. Department of Energy. The United States Government retains and the
publisher, by accepting the article for publication, acknowledges that the
United States Government retains a non-exclusive, paidup, irrevocable, world-
wide license to publish or reproduce the published form of this manuscript, or
allow others to do so, for United States Government purposes. The Department
of Energy will provide public access to these results of federally sponsored
research in accordance with the DOE Public Access
Plan(http://energy.gov/downloads/doepublic-access-plan).

## References

  * Geim and Grigorieva [2013] A. K. Geim and I. V. Grigorieva, Nature 499, 419 (2013).
  * Zhang  _et al._ [2005] Y. Zhang, Y.-W. Tan, H. L. Stormer, and P. Kim, Nature 438, 201 (2005).
  * Yu  _et al._ [2010] R. Yu, W. Zhang, H.-J. Zhang, S.-C. Zhang, X. Dai, and Z. Fang, Science 329, 61 (2010).
  * Burch  _et al._ [2018] K. S. Burch, D. Mandrus, and J.-G. Park, Nature 563, 47 (2018).
  * Takano  _et al._ [2004] Y. Takano, N. Arai, A. Arai, Y. Takahashi, K. Takase, and K. Sekizawa, Journal of Magnetism and Magnetic Materials 272, E593 (2004).
  * Wiedenmann  _et al._ [1981] A. Wiedenmann, J. Rossat-Mignod, A. Louisy, R. Brec, and J. Rouxel, Solid State Communications 40, 1067 (1981).
  * Ressouche  _et al._ [2010] E. Ressouche, M. Loire, V. Simonet, R. Ballou, A. Stunault, and A. Wildes, Physical Review B 82, 100408 (2010).
  * Calder  _et al._ [2021] S. Calder, A. Haglund, A. I. Kolesnikov, and D. Mandrus, Phys. Rev. B 103, 024414 (2021).
  * Han  _et al._ [2023] H. Han, H. Lin, W. Gan, Y. Liu, R. Xiao, L. Zhang, Y. Li, C. Zhang, and H. Li, Applied Physics Letters 122 (2023).
  * Hillman  _et al._ [2011] B. Hillman, L. Norén, and D. Goossens, in  _Matter and Materials Meeting_ (2011) p. 50.
  * Hou  _et al._ [2020] X. Hou, X. Zhang, Q. Ma, X. Tang, Q. Hao, Y. Cheng, and T. Qiu, Advanced Functional Materials 30, 1910171 (2020).
  * Yan  _et al._ [2011] X. Yan, X. Chen, and J. Qin, Materials Research Bulletin 46, 235 (2011).
  * Farrow  _et al._ [2007] C. L. Farrow, P. Juhás, J. Liu, D. Bryndin, E. S. Božin, J. Bloch, T. Proffen, and S. J. L. Billinge, J. Phys.: Condens. Mat. 19, 335219 (2007).
  * Frandsen  _et al._ [2022] B. A. Frandsen, P. K. Hamilton, J. A. Christensen, E. Stubben, and S. J. L. Billinge, Journal of Applied Crystallography 55, 1377 (2022).
  * Neuefeind  _et al._ [2012] J. Neuefeind, M. Feygenson, J. Carruth, R. Hoffmann, and K. K. Chipley, Nucl. Instrum. Meth. B 287, 68 (2012).
  * McDonnell  _et al._ [2017] M. McDonnell, D. Olds, K. Page, J. Neufeind, M. Tucker, J. Bilheux, W. Zhou, and P. Peterson, Acta Crystallogr. Sect. A 73, a377 (2017).
  * Garlea  _et al._ [2010] V. O. Garlea, B. C. Chakoumakos, S. A. Moore, G. B. Taylor, T. Chae, R. G. Maples, R. A. Riedel, G. W. Lynn, and D. L. Selby, Applied Physics A 99, 531 (2010).
  * Juhás  _et al._ [2018] P. Juhás, J. N. Louwen, L. v. Eijck, E. T. Vogt, and S. J. Billinge, Journal of Applied Crystallography 51, 1492 (2018).
  * Granroth  _et al._ [2010] G. E. Granroth, A. I. Kolesnikov, T. E. Sherline, J. P. Clancy, K. A. Ross, J. P. C. Ruff, B. D. Gaulin, and S. E. Nagler, Journal of Physics: Conference Series 251, 012058 (2010).
  * Arnold  _et al._ [2014] O. Arnold, J.-C. Bilheux, J. Borreguero, A. Buts, S. I. Campbell, L. Chapon, M. Doucet, N. Draper, R. F. Leal, M. Gigg, _et al._ , Nuclear instruments and methods in physics research section a: accelerators, spectrometers, detectors and associated equipment 764, 156 (2014).
  * Azuah  _et al._ [2009] R. T. Azuah, L. R. Kneller, Y. Qiu, P. L. Tregenna-Piggott, C. M. Brown, J. R. Copley, and R. M. Dimeo, Journal of research of the National Institute of Standards and Technology 114, 341 (2009).
  * Toth and Lake [2015] S. Toth and B. Lake, Journal of Physics: Condensed Matter 27, 166002 (2015).
  * Baral  _et al._ [2022] R. Baral, J. A. Christensen, P. K. Hamilton, F. Ye, K. Chesnel, T. D. Sparks, R. Ward, J. Yan, M. A. McGuire, M. E. Manley, _et al._ , Matter 5, 1853 (2022).
  * Wildes  _et al._ [1998] A. Wildes, B. Roessli, B. Lebech, and K. Godfrey, Journal of Physics: Condensed Matter 10, 6417 (1998).
  * Sivadas  _et al._ [2015] N. Sivadas, M. W. Daniels, R. H. Swendsen, S. Okamoto, and D. Xiao, Phys. Rev. B 91, 235425 (2015).
  * Yang  _et al._ [2020] J. Yang, Y. Zhou, Q. Guo, Y. Dedkov, and E. Voloshina, RSC advances 10, 851 (2020).
  * Chittari  _et al._ [2016] B. L. Chittari, Y. Park, D. Lee, M. Han, A. H. MacDonald, E. Hwang, and J. Jung, Phys. Rev. B 94, 184428 (2016).
  * Vaclavkova  _et al._ [2020] D. Vaclavkova, A. Delhomme, C. Faugeras, M. Potemski, A. Bogucki, J. Suffczyński, P. Kossacki, A. R. Wildes, B. Grémaud, and A. Saúl, 2D Materials 7, 035030 (2020).