# Stability of topological descriptors for neuronal morphology

David Beers Email<EMAIL_ADDRESS>Mathematical Institute, University of
Oxford  Heather A. Harrington  Mathematical Institute, University of Oxford
Wellcome Centre for Human Genetics, University of Oxford  Alain Goriely
Mathematical Institute, University of Oxford

###### Abstract

The topological morphology descriptor of a neuron is a multiset of intervals
associated to the shape of the neuron represented as a tree. In practice,
topological morphology descriptors are vectorized using persistence images,
which can help classify and characterize the morphology of broad groups of
neurons. We study the stability of topological morphology descriptors under
small changes to neuronal morphology. We show that the persistence diagram
arising from the topological morphology descriptor of a neuron is stable for
the 1-Wasserstein distance against a range of perturbations to the tree. These
results guarantee that persistence images of topological morphology
descriptors are stable against the same set of perturbations and reliable.

##  1 Introduction

Topological data analysis is a field in applied mathematics concerned with the
shape of data. Arising from this field, the celebrated persistent homology
(PH) algorithm [16] records relevant information regarding geometric data. In
certain settings, PH sidesteps alignment issues of other geometric
descriptors. The persistent homology algorithm takes as input a topological
space $X$ equipped with a function $f$, and returns a multiset of intervals
called a barcode, or equivalently, a multiset of points in the plane called a
persistence diagram. Roughly speaking, the $x$ and $y$ coordinates of each
point in a persistence diagram detail when a particular topological feature
appears and then subsequently disappears in the sequence of shapes
$f(-\infty,t]$ as $t$ varies. The space of persistence diagrams can be
equipped with the bottleneck and 1-Wasserstein distances, defined in Section
2.2. Loosely speaking, the bottleneck distance between two persistence
diagrams is the smallest number $\varepsilon$ that exists allowing a pairing
between points in each diagram moving points no more than $\varepsilon$,
whereas for the 1-Wasserstein distance $\varepsilon$ controls the cumulative
distance the points are moved in a matching. Crucially, persistence diagrams
arising from PH are stable against certain perturbations of $f$ with respect
to these metrics [4, 6, 15]. Persistence diagrams can be vectorized for
machine learning [1, 3]. One such vectorization, persistence images, is stable
under perturbations to the 1-Wasserstein distance [1].

Ideas from topological data analysis have been developed to study neurons.
Neuronal shape plays a key role in cognition [13]; therefore, the morphology
of neurons is often analysed and classified. Mathematicians have proposed
specialized topological descriptors for neuronal morphology. Li et al. [12]
showed that PH can be directly applied to neuronal data to encode
morphologically relevant information in persistence diagrams. Kanari et al.
[8] proposed the topological morphology descriptor (TMD), a barcode derived
from neuronal data using a PH-inspired algorithm. This TMD barcode is derived
from a topological space $X$ equipped with a function $f$, but here $X$ is
required to be a rooted tree $T$. The present authors proved in [2] that when
$f$ increases along paths away from the root, this methodology coincides with
that of [12]. Kanari et al. [8] showed that the TMD is stable against
perturbations of neuronal data for the bottleneck distance. The TMDs of
different neurons can be vectorized using persistence images, which can be
averaged, interpreted and compared. Khalil et al. [10] state the TMD is stable
for the 1-Wasserstein distance against perturbations to an embedding of $T$ in
Euclidean space when $f$ the Euclidean distance of the vertices to the soma;
no quantification or bounds are given measuring the stability of these
perturbations.

Figure 1: The persistence image pipeline applied to neuronal data.

Here, we prove 1-Wasserstein stability of the TMD against perturbations to any
function $f$ and to a class of perturbations to the structure of $T$. We
provide Lipschitz bounds for both kinds of stability. In other words, we show
that the composite of the first two steps in the Figure 1 pipeline of
converting neuronal data to persistence images is stable in greater
generality. As a direct consequence of the work of Adams et al. [1], the
entire pipeline of Figure 1 is stable against the perturbations we consider.
The key to our main stability result is the identification of classes of
perturbation that are stable for the 1-Wasserstein distance. Roughly speaking,
these perturbations correspond to the following perturbations to a tree:

  * •

moving slightly the positions of the neuron’s branches;

  * •

adding a short branch;

  * •

deleting a short branch.

The rigorous definition of these perturbations will be given in Section 3. The
main result of this paper, Theorem 2, is that the TMD is stable against these
perturbations for the 1-Wasserstein distance. Therefore, persistence images
are stable against the same perturbations.

##  2 Background

###  2.1 The topological morphology descriptor (TMD)

Let $T$ denote a finite rooted tree with root $r$. In this paper we say a non-
root vertex $l\in N(T)$ is a leaf if it is a vertex of degree one. The root is
only called a leaf if it has degree zero. A vertex in $T$ is called a branch
point if it has degree three or greater. The root, again an exception, we call
a branch point if it has degree two or greater. We denote by $N(T)$, $E(T)$,
and $L(T)$ the sets of vertices, edges, and leaves of $T$ respectively. We
define the depth of a vertex $v\in N(T)$ to be the number of nodes, minus one,
along the shortest path from $r$ to $v$. The depth of a rooted tree $T$ is the
greatest depth of any of its vertices. Consider a real valued function $f$ on
$N(T)$. The topological morphology descriptor (TMD) of $T$ and $f$, denoted
$\mathrm{TMD}(T,f)$, is a multiset of intervals called a barcode obtained via
the algorithm developed in [8]. If $T$ is a tree representing a neuron with
its root representing the soma and $f$ records a notion of distance of the
vertices of $T$ to the root, then $\mathrm{TMD}(T,f)$ is interpreted as a
decomposition of the underlying neuron into its branches, with each interval
recording how far a branch’s initial and terminal points are from the root.

For any vertex $v$ of $T$ with child $v^{\prime}$ we call the rooted tree
generated by $v^{\prime}$, its children, its children’s children, and so on,
along with the edge $(v,v^{\prime})$, a child branch of $v$. The process of
obtaining a barcode from $T$ and $f$ is as follows:

  1. 1.

Choose any branch point $b$ in $T$;

  2. 2.

Identify one of the children $c$ of $b$ whose child branch maximizes $f$ on
the vertices that are leaves of $T$ descendant from $b$;

  3. 3.

Detach all child branches for children $c^{\prime}\neq c$ of $b$. Replace $T$
with the resulting forest;

  4. 4.

If there are no branch points in the resulting forest, then $T$ is a
collection of intervals, and we are done. Otherwise return to step 1.

What will remain at the end of this procedure is a multiset of intervals with
right endpoints in bijective correspondence with the leaves of $T$. We assign
numbers to the endpoints of these intervals via $f$ and call the resulting
structure $\mathrm{TMD}(T,f)$. We refer to any collection of intervals with
endpoints labelled by real numbers as a barcode. Figure 2 shows an example of
a barcode computed via the TMD. An efficient algorithm for computing the
barcode $\mathrm{TMD}(T,f)$ is given in [8]. Note that in general it may be
the case that the number assigned by $f$ to the left endpoint of an interval
may be greater than that assigned to the right endpoint of an interval.
However, this never happens if $f$ is increasing along paths away from the
root, as a consequence of [2, Theorem 2]. We then define $\mathrm{TMDg}(T,f)$
to be the disjoint union of a copy $(a,b)\in\mathbb{R}^{2}$ for every interval
in $\mathrm{TMD}(T,f)$ with left endpoint $a$ and right endpoint $b$, along
with infinite copies of $(x,x)\in\mathbb{R}^{2}$ for every real number
$x$111In [8], the authors use reversed notation, where an interval with left
endpoint $a$ and right endpoint $b$ gets mapped to $(b,a)$. This is likely
done to suggest a connection between the TMD and extended persistence of the
superlevel set filtration of $f$.. By convention, we refer to the value $a$
the birth of a feature represented by a point $(a,b)$, and we refer to $b$ and
$b-a$ as the death and persistence of the same feature, respectively. Notice
that $\mathrm{TMDg}(T,f)$ will always contain a point $(f(r),L)$, where $L$ is
the maximum of $f$ on the leaves of $T$. For convenience, we denote the
multiset of all diagonal points $(x,x)$, each with infinite multiplicity, as
$\mathrm{diag}$.

Figure 2: Computing the TMD from neuronal data.

###  2.2 Metrics on the space of persistence diagrams

The multiset $\mathrm{TMDg}(T,f)$ is an example of what is called a
persistence diagram, a multiset of $\mathbb{R}^{2}$ containing $\mathrm{diag}$
and only finitely many off-diagonal points. There are many metrics on the
space of persistence diagrams. In this paper we consider two of particular
interest.

######  Definition 1.

A map $\phi$ between persistence diagrams $D_{1}$ and $D_{2}$ is called a
matching if it is bijective. The bottleneck distance between $D_{1}$ and
$D_{2}$ is defined to be

$d_{B}(D_{1},D_{2}):=\inf_{\phi}\sup_{x\in D_{1}}\|x-\phi(x)\|_{\infty},$

where the infimum is taken over all matchings $\phi$ between $D_{1}$ and
$D_{2}$. The 1-Wasserstein distance is defined similarly, as

$d_{1}(D_{1},D_{2}):=\inf_{\phi}\sum_{x\in D_{1}}\|x-\phi(x)\|_{\infty},$

again with the infimum being over matchings $\phi$ between $D_{1}$ and
$D_{2}$.

If for a matching $\phi$ we have

$\sum_{x\in D_{1}}\|x-\phi(x)\|_{\infty}\leq\varepsilon,$

we will say that $\phi$ is an $\varepsilon$-matching222It is more common in
the literature for an $\varepsilon$-matching to refer to a matching such that
$\sup_{x\in D_{1}}\|x-\phi(x)\|_{\infty}\leq\varepsilon$. We use an alternate
definition in this work as the 1-Wasserstein distance is of primary interest
here..

Edelsbrunner and Harer showed [7, Section 8] that $d_{B}$ and $d_{1}$ are
indeed metrics on the space of persistence diagrams. By mapping
$\mathrm{TMD}(T,f)$ to $\mathrm{TMDg}(T,f)$, we attain a pseudometric on the
space of TMDs from either of these metrics on the space of persistence
diagrams. Explicitly, we define

$\begin{split}d_{B}(\mathrm{TMD}(T,f),\mathrm{TMD}(T^{\prime},f^{\prime}))&:=d_{B}(\mathrm{TMDg}(T,f),\mathrm{TMDg}(T^{\prime},f^{\prime})),\\\
d_{1}(\mathrm{TMD}(T,f),\mathrm{TMD}(T^{\prime},f^{\prime}))&:=d_{1}(\mathrm{TMDg}(T,f),\mathrm{TMDg}(T^{\prime},f^{\prime})).\end{split}$

Stability results for the TMD algorithm with respect to the bottleneck
distance were given by Kanari et al. [8, SI, Section 4]. Khalil et al. [10]
state the TMD is stable against perturbations to an embedding of $T$ in
Euclidean space for the 1-Wasserstein distance when $f$ is the Euclidean
distance of the vertices to the soma; however Lipschitz bounds for this kind
of stability are not provided. We prove in Section 3 that the TMD is stable
against perturbations of arbitrary functional information $f$, and against
certain perturbations of the underlying rooted tree $T$. Before we prove these
results we motivate the importance of 1-Wasserstein stability in the next
subsection.

###  2.3 Persistence images

For data science problems, persistence diagrams are often mapped into vector
spaces, which allows them to be analyzed via machine learning [3, 5, 14]. One
popular technique of this style is to transform persistence diagrams into
matrices called persistence images introduced by Adams et al [1]. Persistence
images are of particular interest in the setting of the TMD as they are the
standard type of vectorization used in this context [8, 9, 11] in part because
they provide interpretable visual summaries of averaged persistence diagrams.

Obtaining a persistence image from a persistence diagram is a process
involving three steps. The first step is to transform a persistence diagram
$D$ to another multiset of the plane $\tilde{D}$ by mapping each point $(x,y)$
to $(x,y-x)$. Next we fix a positive number $\sigma$ and define a function
$f_{\sigma}$ to be the sum of 2D Gaussian functions of standard deviation
$\sigma$ centered over each point in $\tilde{D}$, and weighted by their
distance from the $x$-axis. The function $f_{\sigma}$ is called a persistence
surface. A persistence image can then be defined by fixing a grid over a
relevant range of values and generating a matrix where the
$(i,j)^{\mathrm{th}}$ value is the integral of $f_{\sigma}$ over the
$(i,j)^{\mathrm{th}}$ cell. If the dimensions of each cell in the grid are
small relative to $\sigma$, a persistence image can be approximated by
sampling over a grid of points. Persistence images can be viewed as vectors
with a coordinate for each cell in their corresponding grid. Viewing
persistence images as vectors, they are stable in their 1, 2, and $\infty$
norms with respect to the 1-Wasserstein distance of persistence diagrams [1,
Theorem 10]333The observant reader may notice that the Lipschitz bound
provided in the cited theorem is dependent on the $\infty$-norm of our
weighting function. This may cause some alarm since we suggest weighting by
distance to the $x$-axis, an unbounded function. However, going through the
proof one observes that the bound is only dependent on the $\infty$-norm of
the weight function over points in a fixed dataset, which is always bounded,
provided the dataset is finite and each diagram has finitely many nontrivial
points. Furthermore, the weighting the authors themselves use in [1, page 8]
differs from the distance from the $x$-axis function by a constant multiple on
data..

The first transformation step $(x,y)\mapsto(x,y-x)$ can be removed from the
construction of persistence images without disrupting stability guarantees,
provided we instead apply a weight of $|y-x|$ to Gaussians centered at $(x,y)$
when constructing persistence surfaces. Indeed, following the proofs of
Theorems 9 and 10 from [1], it can be observed that the same reasoning holds.
In fact, the bounds we are guaranteed are tighter by a factor of $\sqrt{5}$,
thanks to no longer needing to transform coordinates in the proof of Theorem
9. This can be a slightly convenient alternate version of persistence images
in practice, since in certain settings $\mathrm{TMDg}(T,f)$ can be a
persistence diagram with points in both regions $y<x$ and $y>x$. Similarly, we
can also reflect points in a persistence diagram without the transformation
$(x,y)\mapsto(x,y-x)$ about the diagonal $y=x$, or with the transformation
about the $x$-axis while still ensuring persistence images are 1-Wasserstein
stable. We mention this since Kanari et al. have already used unweighted
analogues of reflected persistence images without the transformation
$(x,y)\mapsto(x,y-x)$ in their previous work on the TMD [8, 9]. In summary,
any transformation of a tree equipped with a function that is stable for the
1-Wassertein distance of its TMD is also stable with respect to popular
metrics on its associated transformed and untransformed, reflected and
unreflected persistence images. This fact motivates the following section.

##  3 1-Wasserstein Stability

In this section we prove the TMD is stable to four types of perturbations. We
build on an inductive argument of [8, SI]. Denote P1, P2, P3, and P4 the four
kinds of perturbations to data from $(T,f)$ to $(T^{\prime},f^{\prime})$:

  * P1

Vertex perturbation. $T^{\prime}=T$ and $f^{\prime}$ satisfies
$|f^{\prime}(v)-f(v)|\leq\varepsilon$ for all vertices $v$ in $N(T)$.

  * P2

Attaching an edge to a vertex. $N(T^{\prime})=N(T)\cup\\{v^{\prime}\\}$,
$E(T^{\prime})=E(T)\cup\\{(v,v^{\prime})\\}$ for some vertex $v\in N(T)$,
$f^{\prime}=f$ on $N(T)$, $|f^{\prime}(v^{\prime})-f(v)|\leq\varepsilon$, and
$f(v)\leq f(l)+\varepsilon$ for at least one leaf $l$ of $T$ that is a
descendant of $v$. Put another way, we attach a short edge to one of the
vertices of $T$, one of whose descendant leaves does not have a much smaller
$f$ value than its own.

  * P3

Attaching an edge to an edge. There is an edge $(n,n^{\prime})$ in $E(T)$,
$N(T^{\prime})=N(T)\cup\\{v,v^{\prime}\\}$,
$E(T^{\prime})=E(T)\cup\\{(n,v),(v,n^{\prime}),(v,v^{\prime})\\}-\\{(n,n^{\prime})\\}$,
$f=f^{\prime}$ on $N(T)$, $|f(v)-f(v^{\prime})|\leq\varepsilon$, and
$f^{\prime}(v)\leq f(l)+\varepsilon$ for at least one leaf $l$ of $T$ that is
a descendant of $v$. In words, we add a vertex interior to an edge of $T$,
ensure that one of the descendant leaves of this vertex does not have a much
smaller $f$ value than its own, and attach a short edge to this vertex.

  * P4

Edge retraction. $(T,f)$ is obtained by a perturbation of type P2 or P3 from
$(T^{\prime},f^{\prime})$.

We leave the root vertex unchanged in all of these classes of perturbation.

We show visualizations of these transformations in Figure 3, along with a type
of transformation we do not claim to be stable.

Figure 3: Tree perturbations we show are stable, and one type of perturbation
that is unstable.

Now we state the main theorem of the paper.

######  Theorem 2.

Let $T$ be a rooted tree and $f:T\to\mathbb{R}$. Let also $T^{\prime}$ be a
rooted tree with $f^{\prime}:T^{\prime}\to\mathbb{R}$.

  1. 1.

If $(T^{\prime},f^{\prime})$ is obtained by a perturbation of type P1 of
$(T,f)$, then

$d_{1}(\mathrm{TMD}(T,f),\mathrm{TMD}(T^{\prime},f^{\prime}))\leq|L(T)|\varepsilon.$

  2. 2.

If $(T^{\prime},f^{\prime})$ is obtained by a perturbation of type P2 or P3 of
$(T,f)$, then

$d_{1}(\mathrm{TMD}(T,f),\mathrm{TMD}(T^{\prime},f^{\prime}))\leq\frac{5}{2}\varepsilon.$

Furthermore, if the vertex $v$ given by the perturbation of type P2 or P3 is
such that $f^{\prime}(v)\leq f(l)$ for some leaf $l$ of $T$ that is a
descendant of $v$, then

$d_{1}(\mathrm{TMD}(T,f),\mathrm{TMD}(T^{\prime},f^{\prime}))\leq\varepsilon.$

  3. 3.

If $(T^{\prime},f^{\prime})$ is obtained by a perturbation of type P4 of
$(T,f)$, then

$d_{1}(\mathrm{TMD}(T,f),\mathrm{TMD}(T^{\prime},f^{\prime}))\leq\frac{5}{2}\varepsilon.$

Furthermore, if the vertex $v$ in $T$ given by the perturbation of type P2 or
P3 from $(T^{\prime},f^{\prime})$ to $(T,f)$ is such that $f(v)\leq
f^{\prime}(l)$ for some leaf $l$ of $T^{\prime}$ that is a descendant of $v$,
then

$d_{1}(\mathrm{TMD}(T,f),\mathrm{TMD}(T^{\prime},f^{\prime}))\leq\varepsilon.$

Note that the 1-Wasserstein stability results are weaker than those obtained
for the bottleneck distance in [8, SI, Theorem 1]. First, the Lipschitz
constant we provide in statement 1 of the theorem is dependent on the number
of leaves in $T$. Second, for statements 2 and 3, we are not allowing for
arbitrarily intervals of a fixed length to be attached to or removed from $T$.
With respect to the 1-Wasserstein distance, it is not hard to see that this
kind of perturbation is unstable. For instance, if $T$ is a single root vertex
equipped with the zero map $f$, and we obtain $(T^{\prime},f^{\prime})$ by
attaching $n$ vertices to this root, each with $f^{\prime}$ value
$\varepsilon$, we can make the 1-Wasserstein distance between the two
associated TMDs arbitrarily large by increasing $n$.

We will first prove statement 1 of this theorem, then statement 2. Statement 3
follows immediately from statement 2, since these statements are identical
with the roles of $(T,f)$ and $(T^{\prime},f^{\prime})$ reversed. Our methods
of proof are based on those of [8, SI, Section 4]. The following proposition
is equivalent to statement 1.

######  Proposition 3.

Let $T$ be a rooted tree with root $r$ and $f,f^{\prime}:N(T)\to\mathbb{R}$
such that $|f(v)-f^{\prime}(v)|\leq\varepsilon$ for all vertices $v$ in
$N(T)$. Then

$d_{1}(\mathrm{TMDg}(T,f),\mathrm{TMDg}(T,f^{\prime}))\leq|L(T)|\varepsilon.$

###### Proof.

If $T$ is a single vertex, then $T$ has one leaf, which is also its root, and
the fact that

$d_{1}(\mathrm{TMDg}(T,f),\mathrm{TMDg}(T,f^{\prime}))\leq\varepsilon$

is clear.

We proceed with the remaining cases by induction on the depth of $T$, each
time proving that every point not in $\mathrm{diag}$ in each persistence
diagram can simultaneously be matched with another point not in
$\mathrm{diag}$ that is $\varepsilon$ away in sup norm, provided $T$ is of
depth $d$. This will imply the desired result since the TMD algorithm produces
exactly one interval for each leaf. We will assume inductively that the
matching we get for trees of depth $d-1$ between functions $f$ and
$f^{\prime}$ sends a copy of $(f(r),L)$ to a point with first coordinate
$f^{\prime}(r)$, and whose inverse sends a copy of
$(f^{\prime}(r),L^{\prime})$ to a point with first coordinate $f(r)$, where
$L$ and $L^{\prime}$ are the maximum values of $f$ and $f^{\prime}$ on the
leaves of $T$ respectively. This additional constraint can clearly be
satisfied for the base case where $T$ has depth zero, and hence is a single
vertex.

Suppose $T$ is a tree of depth $d$ with $f$ and $f^{\prime}$ as in the
statement of the theorem. Let $c_{1},\ldots,c_{n}$ denote the children of the
root of $T$. Each $c_{i}$ is the root of a subtree $T_{i}$ of $T$ whose
vertices are $c_{i}$, its children, its children’s children, and so on. Let
$b_{i}$ be the leaf that takes the greatest value of $f$ subject to the
constraint that $c_{i}$ is an ancestor of $b_{i}$. We define $b_{i}^{\prime}$
similarly for $f^{\prime}$. Thus,

$\begin{split}\mathrm{TMDg}(T,f)&=\mathrm{diag}\sqcup\\{(f(r),f(b_{i})):1\leq
i\leq n\\}\sqcup\bigsqcup_{i=1}^{n}D_{i},\\\
\mathrm{TMDg}(T,f^{\prime})&=\mathrm{diag}\sqcup\\{(f^{\prime}(r),f^{\prime}(b^{\prime}_{i})):1\leq
i\leq n\\}\sqcup\bigsqcup_{i=1}^{n}D^{\prime}_{i},\end{split}$

where

$\begin{split}D_{i}&=\mathrm{TMDg}(T_{i},f)-\\{(f(c_{i}),f(b_{i}))\\},\\\
D^{\prime}_{i}&=\mathrm{TMDg}(T_{i},f^{\prime})-\\{(f^{\prime}(c_{i}),f^{\prime}(b^{\prime}_{i}))\\}.\end{split}$

If we can show that there is a matching of $D_{i}$ and $D^{\prime}_{i}$ that
sends no point more than $\varepsilon$ away with respect to the sup norm, then
the matching that sends each $(f(r),f(b_{i}))$ to
$(f^{\prime}(r),f(b^{\prime}_{i}))$ does the same for $\mathrm{TMDg}(T,f)$ and
$\mathrm{TMDg}(T,f^{\prime})$. Indeed, assuming without loss of generality
that $f(b_{i})\geq f^{\prime}(b^{\prime}_{i})$,

$|f(b_{i})-f^{\prime}(b^{\prime}_{i})|=f(b_{i})-\max_{l\in
L(T_{i})}f^{\prime}(l)\leq f(b_{i})-f^{\prime}(b_{i})\leq\varepsilon$

Further, this matching and its inverse also satisfy our additional inductive
hypothesis regarding points with left coordinate equal to $f(r)$ and
$f^{\prime}(r)$. Since there are exactly $|L(T)|$ points in both persistence
diagrams excluding $\mathrm{diag}$ we will be done once we can find a desired
matching between $D_{i}$ and $D^{\prime}_{i}$.

By inductive hypothesis we already have such a matching between
$\mathrm{TMDg}(T_{i},f)$ and $\mathrm{TMDg}(T_{i},f^{\prime})$, and we will
adjust this matching to produce a matching between $D_{i}$ and
$D^{\prime}_{i}$. If our given matching sends $(f(c_{i}),f(b_{i}))$ to
$(f^{\prime}(c_{i}),f^{\prime}(b^{\prime}_{i}))$ then we immediately get a
desired matching between $D_{i}$ and $D^{\prime}_{i}$.

Otherwise, let $\phi:\mathrm{TMDg}(T_{i},f)\to\mathrm{TMDg}(T_{i},f^{\prime})$
be our matching and let $(x^{\prime},y^{\prime})=\phi((f(c_{i}),f(b_{i})))$,
$(x,y)=\phi^{-1}((f^{\prime}(c_{i}),f^{\prime}(b^{\prime}_{i})))$. Thus we may
define $\tilde{\phi}:D_{i}\to D^{\prime}_{i}$ by sending $(x,y)$ to
$(x^{\prime},y^{\prime})$ and every other point to its image via $\phi$. By
our inductive hypothesis, $x=f(c_{i})$ and $x^{\prime}=f^{\prime}(c_{i})$.
Thus $|x-x^{\prime}|\leq\varepsilon$. Assume without loss of generality that
$f^{\prime}(b^{\prime}_{i})\leq f(b_{i})$. Now assume, with loss of
generality, that $f^{\prime}(b^{\prime}_{i})\geq y$. Thus we have the
equation.

$|y-y^{\prime}|=|y-f^{\prime}(b^{\prime}_{i})+f^{\prime}(b^{\prime}_{i})-y^{\prime}|.$
(1)

Both differences on the right side of the above equation are bounded by
$\varepsilon$, the first due to the matching $\phi$, and the last since by
definition of $b^{\prime}_{i}$, $f^{\prime}(b^{\prime}_{i})\geq y^{\prime}$
and so

$|f^{\prime}(b^{\prime}_{i})-y^{\prime}|=f^{\prime}(b^{\prime}_{i})-y^{\prime}\leq
f(b_{i})-y^{\prime}\leq\varepsilon.$

Notice also that the last difference in Equation 1 is nonnegative. Meanwhile,
$y-f^{\prime}(b^{\prime}_{i})$ is nonpositive since
$f^{\prime}(b^{\prime}_{i})\geq y$. Thus, Equation 1 shows that
$|y-y^{\prime}|$ is a difference of nonnegative numbers each bounded by
$\varepsilon$, and hence itself is less than or equal to $\varepsilon$. Hence
$\tilde{\phi}$ is a matching which alters each coordinate of every point by no
more than $\varepsilon$.

Finally, still assuming that $f^{\prime}(b^{\prime}_{i})\leq f(b_{i})$, we
also assume $f^{\prime}(b^{\prime}_{i})\leq y$. Now the relevant equation is

$|y-y^{\prime}|=|y-f(b_{i})+f(b_{i})-y^{\prime}|.$ (2)

Once again, both differences on the right are bounded by $\varepsilon$, the
second due to the matching $\phi$, and the first since by definition of
$b_{i}$, $y\leq f(b_{i})$, and so

$|y-f(b_{i})|=f(b_{i})-y\leq
f(b_{i})-f^{\prime}(b^{\prime}_{i})\leq\varepsilon.$

As before, we have also just shown the first difference in the right side of
Equation 2 is nonpositive, while the second is nonnegative since $f(b_{i})\geq
f^{\prime}(b^{\prime}_{i})\geq y^{\prime}$. Hence $|y-y^{\prime}|$ is the
difference of two nonnegative numbers bounded by $\varepsilon$, and so is
itself less than $\varepsilon$. Again, this implies that $\tilde{\phi}$ is a
matching which sends no point more than $\varepsilon$ away with respect to the
sup norm and the proof is complete. ∎

######  Remark.

Kanari et al. [8, SI] show perturbations of type P1 are bottleneck stable by
constructing a matching that sends points not in $\mathrm{diag}$ to each
other. Proposition 3 follows from their matching and the fact that the TMD
algorithm produces a diagram with $|L(T)|$ points excluding $\mathrm{diag}$.
The construction of the matching in [8] implicitly assumes that
$b_{i}=b^{\prime}_{i}$. Here, we use a different inductive hypothesis and
therefore include our own full construction for completeness.

In fact, for our above proposition to hold, the values of $f$ and $f^{\prime}$
only need to be close on the root, branch points, and leaves- the following
proposition shows we can remove all other vertices without changing TMDg.

######  Proposition 4.

Let $T$ be a tree with root $r$ and an edge $(a,b)$. Let $T^{\prime}$ be the
tree given by $N(T^{\prime})=N(T)\cup\\{v^{\prime}\\}$ and
$E(T^{\prime})=E(T)\cup\\{(a,v^{\prime}),(v^{\prime},b)\\}-\\{(a,b)\\}$, with
root $r$. Let $f:N(T)\to\mathbb{R}$ and
$f^{\prime}:N(T^{\prime})\to\mathbb{R}$ be such that $f^{\prime}$ restricts to
$f$ on $N(T)$. Then

$\mathrm{TMDg}(T,f)=\mathrm{TMDg}(T^{\prime},f^{\prime}).$

###### Proof.

We prove this by induction on the depth of $T$. If $T$ has depth zero, then
$T$ is a single vertex and the theorem is vacuously true since $T$ has no
edge.

Otherwise, assume the result holds for adding vertices to trees of depth less
than $d$ and $T$ is of depth $d$. Then, first assume neither $a$ nor $b$ is
the root of $T$. Thus, adopting the notation of $c_{i}$, $b_{i}$, $T_{i}$, and
$D_{i}$ from the previous proof, we observe that $a$ and $b$ must in some
$T_{j}$. We define $T^{\prime}_{j}$ by
$N(T^{\prime}_{j})=N(T_{j})\cup\\{v^{\prime}\\}$ and
$E(T^{\prime}_{j})=E(T_{j})\cup\\{(a,v^{\prime}),(v^{\prime},b)\\}-\\{(a,b)\\}$.
We further define
$D^{\prime}_{j}=\mathrm{TMDg}(T^{\prime}_{j},f^{\prime})-\\{(f(r),f(b_{j}))\\}$.
It follows that

$\begin{split}\mathrm{TMDg}(T,f)&=\mathrm{diag}\sqcup\\{(f(r),f(b_{i})):1\leq
i\leq n\\}\sqcup\bigsqcup_{i=1}^{n}D_{i},\\\
\mathrm{TMDg}(T^{\prime},f^{\prime})&=\mathrm{diag}\sqcup\\{(f(r),f(b_{i})):1\leq
i\leq n\\}\sqcup D^{\prime}_{j}\sqcup\bigsqcup_{i\neq j}D_{i}.\end{split}$

But by our inductive hypothesis these $D_{j}$ and $D^{\prime}_{j}$ are
identical, so the above persistence diagrams must be identical as well. It
remains to check the case where either $a$ or $b$ is the root. Without loss of
generality, suppose $a$ is the root of $T$. Then $b=c_{j}$ for some $j$. Let
$T^{\prime}_{j}$ be given by $N(T^{\prime}_{j})=N(T_{j})\cup\\{v^{\prime}\\}$
and $E(T^{\prime}_{j})=E(T_{j})\cup\\{(v^{\prime},c_{j})\\}$. Thus

$\mathrm{TMDg}(T^{\prime},f^{\prime})=\mathrm{diag}\sqcup\\{(f(r),f(b_{i})):1\leq
i\leq n\\}\sqcup D^{\prime}_{j}\sqcup\bigsqcup_{i\neq j}D_{i},$

where

$D^{\prime}_{j}:=\mathrm{TMDg}(T^{\prime}_{j},f^{\prime})-\\{(f^{\prime}(v^{\prime}),f(b_{i}))\\}=\mathrm{TMDg}(T_{j},f)-\\{(f(c_{j}),f(b_{i}))\\}=D_{j}.$

Hence $\mathrm{TMDg}(T,f)=\mathrm{TMDg}(T^{\prime},f^{\prime})$. ∎

The following proposition shows that statement 2 of Theorem 2 holds for
perturbations of type P2. By applying the previous proposition as well, the
statement also holds for perturbations of type P3.

######  Proposition 5.

Let $T$ be a rooted tree with root $r$ and $f:N(T)\to\mathbb{R}$. Let $v$ be
any vertex of $T$ and $T^{\prime}$ be another tree given by
$N(T^{\prime})=N(T)\cup\\{v^{\prime}\\}$ and
$E(T^{\prime})=E(T)\cup\\{(v,v^{\prime})\\}$, with root $r$. Let
$f^{\prime}:N(T^{\prime})\to\mathbb{R}$ be such that $f^{\prime}=f$ on $N(T)$
and $|f^{\prime}(v^{\prime})-f^{\prime}(v)|\leq\varepsilon$. If $f(v)\leq
f(l)+\delta$ for at least one leaf $l$ which is a descendant of $v$ then

$d_{1}(\mathrm{TMDg}(T,f),\mathrm{TMDg}(T^{\prime},f^{\prime}))\leq\varepsilon+\frac{3}{2}\delta.$

###### Proof.

To avoid certain mild irritations during the proof, we will first prove the
proposition when $f^{\prime}$ (and hence also $f$) is injective. We will then
attain the general case by perturbing $f^{\prime}$ via Proposition 3.

For convenience, let $\eta=\varepsilon+\frac{3}{2}\delta$. In this proof we
proceed by induction on the depth $d$ of $v$ instead of the depth of $T$. As
our inductive hypothesis, we assume that when $v$ has depth less than $d$ in a
tree $T$, and $T^{\prime}$ satisfies the hypothesis of the proposition the
following is true. If $l$ maximizes $f$ on the leaves of $T$ and $l^{\prime}$
maximizes $f^{\prime}$ on the leaves of $T^{\prime}$, then we can find a
$\eta$-matching sending $(f(r),f^{\prime}(l^{\prime}))$ to $(f(r),f(l))$.
Hence for our induction step we will need to construct a matching with this
additional property.

For our base case $v=r$. As in the previous proofs, we may write

$\mathrm{TMDg}(T,f)=\mathrm{diag}\sqcup\\{(f(r),f(b_{i})):1\leq i\leq
n\\}\sqcup\bigsqcup_{i=1}^{n}D_{i}$

with $D_{i}$, $T_{i}$, $b_{i}$, and $c_{i}$ defined as in the proof of
Proposition 3. If we attach the vertex $v^{\prime}$ to $r$, then we have

$\mathrm{TMDg}(T^{\prime},f^{\prime})=\mathrm{diag}\sqcup\\{(f(r),f^{\prime}(v^{\prime}))\\}\sqcup\\{(f(r),f(b_{i})):1\leq
i\leq n\\}\sqcup\bigsqcup_{i=1}^{n}D_{i}$

Hence by sending $(f(r),f^{\prime}(v^{\prime}))$ to $(f(r),f(r))$ on the
diagonal, and everything else to itself we obtain a $\eta$-matching between
$\mathrm{TMDg}(T^{\prime},f^{\prime})$ and $\mathrm{TMDg}(T,f)$. However, if
$v^{\prime}$ maximizes $f^{\prime}$ on the leaves of $T^{\prime}$ and the
depth of $T$ is nonzero then this matching does not satisfy the inductive
hypothesis. Whenever this happens, we can instead choose the $\eta$-matching
with the following mappings

$\begin{split}(f(r),f^{\prime}(v^{\prime}))&\longmapsto(f(r),f(b_{k}))\\\
(f(r),f(b_{k}))&\longmapsto((f(r)+f(b_{k}))/2,(f(r)+f(b_{k}))/2)\\\
\end{split}$

that sends everything else to itself, where $b_{k}$ is the leaf maximizing $f$
on the leaves of $T$. Notice that
$|f^{\prime}(v^{\prime})-f(r)|\leq\varepsilon$, and further
$f(r)-f(b_{k})=f(v)-f(b_{k})\leq\delta$, so if $f(b_{k})<f(r)$ then this is a
$\eta$-matching. Otherwise, $f(r)<f(b_{k})<f(v^{\prime})$, and

$\begin{split}|f^{\prime}(v^{\prime})-f(b_{k})|+|f(b_{k})-(f(r)+f(b_{k}))/2|&\leq|f^{\prime}(v^{\prime})-f(b_{k})|+|f(b_{k})-f(r)|\\\
&=f^{\prime}(v^{\prime})-f(b_{k})+f(b_{k})-f(r)\\\
&=f^{\prime}(v^{\prime})-f(r)\\\ &\leq\varepsilon,\end{split}$

so the proposed matching is an $\varepsilon$-matching, and hence a
$\eta$-matching.

Proceeding now by induction, we now assume the inductive hypothesis holds
whenever the $v$ has depth less than $d>0$. Assume now that the depth of $v$
is $d$. Hence $v\neq r$, so then $v$ lies in $T_{j}$ for some $j$. We define
$T^{\prime}_{j}$ to be the tree with
$N(T^{\prime}_{j})=N(T_{j})\cup\\{v^{\prime}\\}$ and
$E(T^{\prime}_{j})=E(T_{j})\cup(v,v^{\prime})$, and $b^{\prime}_{j}$ to be the
leaf in $T^{\prime}_{j}$ that maximizes $f^{\prime}$ over the leaves of
$T^{\prime}_{j}$. Thus

$\mathrm{TMDg}(T^{\prime},f^{\prime})=\mathrm{diag}\sqcup\\{(f(r),f(b_{i})):i\neq
j\\}\sqcup\\{(f(r),f^{\prime}(b^{\prime}_{j}))\\}\sqcup
D^{\prime}_{j}\sqcup\bigsqcup_{i\neq j}D_{i},$

where

$D^{\prime}_{j}=\mathrm{TMDg}(T^{\prime}_{j},f^{\prime})-\\{(f(c_{i}),f^{\prime}(b^{\prime}_{i}))\\}$

For our convenience, we let $\gamma$ denote the number
$\eta-f^{\prime}(b^{\prime}_{j})+f(b_{j})$. Notice that $v$ has depth $d-1$ in
$T_{j}$. Consider the $\eta$-matching $\phi$ between
$\mathrm{TMDg}(T^{\prime}_{j},f^{\prime})$ and $\mathrm{TMDg}(T_{j},f)$
guaranteed to exist by our inductive hypothesis. Since our inductive
hypothesis also demands that
$\phi((f(c_{j}),f(b^{\prime}_{j})))=(f(c_{j}),f(b_{j}))$, $\phi$ restricts to
a $\gamma$-matching between $D^{\prime}_{j}$ and $D_{j}$.

We can use this matching to construct a $\eta$-matching between
$\mathrm{TMDg}(T^{\prime},f^{\prime})$ to $\mathrm{TMDg}(T,f)$ by sending
$(f(r),f^{\prime}(b^{\prime}_{j}))$ to $(f(r),f(b_{j}))$, $D^{\prime}_{j}$ to
$D_{j}$ via $\phi$, and everything else to itself. However, this matching may
not satisfy our additional inductive assumption that points with right
coordinates corresponding to leaves maximizing $f$ are sent to each other. If
this is the case, then either $b^{\prime}_{j}$ is the leaf maximizing
$f^{\prime}$ or $b_{j}$ is the leaf maximizing $f$, but not both. Since
$f^{\prime}(b^{\prime}_{j})\geq f(b_{j})$, it must be the case that
$b^{\prime}_{j}$ maximizes $f^{\prime}$, but $b_{j}$ does not maximize $f$.
Let $b_{k}$ be the leaf that maximizes $f$. In this case, consider the
matching $\psi$:

$\begin{split}(f(r),f(b^{\prime}_{j}))&\longmapsto(f(r),f(b_{k}))\\\
(f(r),f(b_{k}))&\longmapsto(f(r),f(b_{j}))\\\ D^{\prime}_{j}&\longmapsto
D_{j},\end{split}$

that sends everything else to itself. Of course, we define the restriction of
$\psi$ to $D^{\prime}_{j}$ to be $\phi$. Since $f^{\prime}(b^{\prime}_{j})\geq
f(b_{k})\geq f(b_{j})$, we observe

$\gamma+|f^{\prime}(b^{\prime}_{j})-f(b_{k})|+|f(b_{k})-f(b_{j})|=\gamma+f^{\prime}(b^{\prime}_{j})-f(b_{j})=\eta.$

As $\phi$ is a $\gamma$-matching, the fact that $\psi$ is a $\eta$-matching
satisfying our inductive hypothesis is immediate.

Now suppose $f^{\prime}$ on $T^{\prime}$ is not injective. In this case, we
can always find an injective $g^{\prime}$ on $N(T^{\prime})$ satisfying both
that $\|f^{\prime}-g^{\prime}\|_{\infty}\leq\kappa$ for any fixed positive
real $\kappa$ and the hypotheses of the proposition with $\varepsilon$
replaced by $\varepsilon+2\kappa$ and $\delta$ replaced by $\delta+2\kappa$.
In fact, this will always be the case whenever we perturb $f^{\prime}$ at each
vertex by a value no more than $\kappa$ in such a way that the perturbed
function is injective. We let $g$ be the restriction of $g^{\prime}$ to
$N(T)$. Hence by Proposition 3 and the fact that $g^{\prime}$ is injective,

$\begin{split}d_{1}(\mathrm{TMDg}(T,f),\mathrm{TMDg}(T^{\prime},f^{\prime}))&\leq
d_{1}(\mathrm{TMDg}(T,f),\mathrm{TMDg}(T,g))\\\
&+d_{1}(\mathrm{TMDg}(T,g),\mathrm{TMDg}(T^{\prime},g^{\prime}))\\\
&+d_{1}(\mathrm{TMDg}(T^{\prime},g^{\prime}),\mathrm{TMDg}(T^{\prime},f^{\prime})),\end{split}$

and this is less than or equal to

$(|L(T)|+|L(T^{\prime})|)\kappa+5\kappa+\eta.$

Considering a sequence of values for $\kappa$ approaching zero the result is
immediate. ∎

The bound given by this proposition is sharp. We show this with an example in
Figure 4. Indeed the tree on the right of this figure is produced by a
perturbation of the tree on the left of the same type as in the hypothesis of
the proposition. The tree on the left has TMDg equal to
$(0,K-\delta)\sqcup\mathrm{diag}$ while the tree on the right has TMDg equal
to $(0,K+\varepsilon)\sqcup(K,K-\delta)\sqcup\mathrm{diag}$. It is not
difficult to see that for sufficiently large $K$, the optimal matching from
the later diagram to the former is the matching which sends
$(0,K+\varepsilon)$ to $(0,K-\delta)$, $(K,K-\delta)$ to the point
$(K-\delta/2,K-\delta/2)$ on the diagonal, and everything else to itself.

Figure 4: A pair of trees with roots in red showing that the bound in
Proposition 5 is sharp.

##  4 Acknowledgements

AG is grateful for the support by the Engineering and Physical Sciences
Research Council of Great Britain under research grants EP/R020205/1. HAH
gratefully acknowledges EPSRC EP/R005125/1 and EP/T001968/1, the Royal Society
RGF$\backslash$EA$\backslash$201074 and UF150238. DB and HAH are members of
the Centre for Topological Data Analysis, funded in part by EPSRC
EP/R018472/1, and grateful for helpful discussions with centre members. For
the purpose of Open Access, the authors have applied a CC BY public copyright
licence to any Author Accepted Manuscript (AAM) version arising from this
submission.

## References

  * [1] H. Adams, T. Emerson, M. Kirby, R. Neville, C. Peterson, P. Shipman, S. Chepushtanova, E. Hanson, F. Motta, and L. Ziegelmeier.  Persistence images: A stable vector representation of persistent homology.  Journal of Machine Learning Research, 18, 2017.
  * [2] D. Beers, D. Goniotaki, D. P. Hanger, A. Goriely, and H. A. Harrington.  Barcodes distinguish morphology of neuronal tauopathy.  arXiv preprint arXiv:2204.03348, 2022.
  * [3] P. Bubenik et al.  Statistical topological data analysis using persistence landscapes.  J. Mach. Learn. Res., 16(1):77–102, 2015.
  * [4] P. Bubenik and J. A. Scott.  Categorification of persistent homology.  Discrete & Computational Geometry, 51(3):600–627, 2014.
  * [5] M. Carrière, S. Y. Oudot, and M. Ovsjanikov.  Stable topological signatures for points on 3d shapes.  In Computer graphics forum, volume 34, pages 1–12. Wiley Online Library, 2015.
  * [6] D. Cohen-Steiner, H. Edelsbrunner, and J. Harer.  Stability of persistence diagrams.  In Proceedings of the twenty-first annual symposium on Computational geometry, pages 263–271, 2005.
  * [7] H. Edelsbrunner and J. L. Harer.  Computational topology: an introduction.  American Mathematical Society, 2022.
  * [8] L. Kanari, P. Dłotko, M. Scolamiero, R. Levi, J. Shillcock, K. Hess, and H. Markram.  A topological representation of branching neuronal morphologies.  Neuroinformatics, 16(1):3–13, 2018.
  * [9] L. Kanari, S. Ramaswamy, Y. Shi, S. Morand, J. Meystre, R. Perin, M. Abdellah, Y. Wang, K. Hess, and H. Markram.  Objective morphological classification of neocortical pyramidal cells.  Cerebral Cortex, 29(4):1719–1735, 2019.
  * [10] R. Khalil, S. Kallel, A. Farhat, and P. Dlotko.  Topological sholl descriptors for neuronal clustering and classification.  PLOS Computational Biology, 18(6):e1010229, 2022.
  * [11] S. Laturnus, D. Kobak, and P. Berens.  A systematic evaluation of interneuron morphology representations for cell type discrimination.  Neuroinformatics, 18(4):591–609, 2020.
  * [12] Y. Li, D. Wang, G. A. Ascoli, P. Mitra, and Y. Wang.  Metrics for comparing neuronal tree shapes based on persistent homology.  PloS one, 12(8):e0182184, 2017.
  * [13] M. London and M. Häusser.  Dendritic computation.  Annu. Rev. Neurosci., 28:503–532, 2005.
  * [14] V. Robins and K. Turner.  Principal component analysis of persistent homology rank functions with case studies of spatial point patterns, sphere packing and colloids.  Physica D: Nonlinear Phenomena, 334:99–117, 2016.
  * [15] P. Skraba and K. Turner.  Wasserstein stability for persistence diagrams.  arXiv preprint arXiv:2006.16824, 2020.
  * [16] A. Zomorodian and G. Carlsson.  Computing persistent homology.  Discrete & Computational Geometry, 33(2):249–274, 2005.


# Increasing Fairness in Classification of Out of Distribution Data for Facial
Recognition††thanks: Submitted to the editors April, 2024. This work was
funded in part by the US NSF award DMS-2051019.

Gianluca Barone Department of Mathematics, Rowan University, Glassboro, NJ ().
<EMAIL_ADDRESS>Aashrit Cunchala Department of Applied
Mathematics, University of Pittsburgh, Pittsburgh, PA<EMAIL_ADDRESS>Rudy
Nunez Department of Mathematics, Emory University, Atlanta, GA ()
<EMAIL_ADDRESS><EMAIL_ADDRESS>
remarkRemark hypothesisHypothesis claimClaim G. Barone, A. Cunchala, and R.
Nunez Project advisor: Nicole Yang††thanks: Department of Mathematics, Emory
University, Atlanta, GA ()


#  Step-back Profiling: Distilling User History for Personalized Scientific
Writing

Xiangru Tang Equal Contribution. Yale University  Xingyao Zhang∗ Search &
Distribution, Microsoft  Yanjun Shao∗ Yale University  Jie Wu Search &
Distribution, Microsoft  Yilun Zhao Yale University
Arman Cohan Yale University  Ming Gong Search & Distribution, Microsoft
Dongmei Zhang Search & Distribution, Microsoft  Mark Gerstein Yale University

###### Abstract

Large language models (LLMs) excel at a variety of natural language processing
tasks, yet they struggle to generate personalized content for individuals,
particularly in real-world scenarios like scientific writing. Addressing this
challenge, we introduce Step-back Profiling to personalize LLMs by distilling
user history into concise profiles, including essential traits and preferences
of users. Regarding our experiments, we construct a Personalized Scientific
Writing (PSW) dataset to study multi-user personalization. PSW requires the
models to write scientific papers given specialized author groups with diverse
academic backgrounds. As for the results, we demonstrate the effectiveness of
capturing user characteristics via Step-back Profiling for collaborative
writing. Moreover, our approach outperforms the baselines by up to 3.6 points
on the general personalization benchmark (LaMP), including 7 personalization
tasks. Our extensive ablation studies validate the contributions of different
components in our method and provide insights into our task definition. Our
dataset and code are available at https://github.com/gersteinlab/step-back-
profiling.

Step-back Profiling: Distilling User History for Personalized Scientific
Writing

##  1 Introduction

Recently, Large Language Models (LLMs) have made significant progress in
natural language understanding and generation (Wei et al., 2022a; Zhang et
al., 2023b; OpenAI, 2023; Qin et al., 2023). Concurrently, integrating LLMs
with personalization paradigms has paved the way for a vast frontier in
improving user-centric services and applications (Salemi et al., 2023; Chen et
al., 2023; Zhiyuli et al., 2023), as they provide a deeper understanding of
users’ accurate demands and interests than abstract vector-based information
representations. By learning to characterize and emulate user-specific
language patterns, personalized LLMs can enable more engaging and valuable
interactions in domains such as dialogue (Wang et al., 2019; Zhang et al.,
2019b; Character.AI, 2022), recommendation (Zhiyuli et al., 2023; Wang et al.,
2023a), role-playing (Shao et al., 2023; Jiang et al., 2023) and content
creation (Cao et al., 2023; Wei et al., 2022b).

Figure 1: Overview of the Step-back Profiling.

Prior work on personalizing language models (Salemi et al., 2023; Tan and
Jiang, 2023; Zhang et al., 2023a; Chen et al., 2023; Zhiyuli et al., 2023) has
shown promise, but primarily focused on learning user representations in a
single-user context. However, many real-world applications involve multiple
users collaborating on a shared task, such as team-authored scientific papers.
Another practical challenge for LLM personalization is scaling to extensive
user histories while respecting context length limits (Shi et al., 2023; Liu
et al., 2024; Zhang et al., 2024). Directly conditioning on raw personal
histories quickly becomes infeasible as user data grows. Prior methods mostly
use uncompressed history for personalization (Salemi et al., 2023), which
restricts the amount of user-specific information the model can utilize.

Figure 2: Step-back Profiling performance on the LaMP benchmark. Details of
experimental setup can be found in Section 4.2.

As shown in Figure 1, this work proposes a training-free LLM personalization
framework that addresses these challenges through Step-back Profiling, we
distill individual user histories into concise profile representations that
capture high-level concepts and language traits. This enables efficient memory
management and allows the model to focus on salient user characteristics,
grounding personalized generation without excess computation or laborious data
collection (Chen et al., 2023). We show that Step-back Profiling improves
performance over standard personalization methods (retrieval-based) in the
LaMP111https://lamp-benchmark.github.io/, as shown in Figure 2. Moreover, we
introduce a Personalized Scientific Writing (PSW) dataset to study multi-user
personalization. PSW contains research papers collaboratively written by
expert teams, and each author’s background publications are used to construct
profiles. Modeling a group’s collective expertise is crucial for this task, as
different paper sections may reflect knowledge associated with particular
authors. PSW thus poses a challenging and realistic testbed for multi-user
personalization, requiring both abstractions of individual expertise and
dynamic integration of diverse user traits throughout the collaborative
writing process.

##  2 Step-back Profiling

###  2.1 Motivation

Existing methods for personalizing language models struggle to effectively
utilize user histories, particularly in the presence of extraneous details
that can obscure the most pertinent information for a given task (Shi et al.,
2023; Liu et al., 2024). This challenge is magnified in multi-user scenarios,
where models must efficiently extract and integrate knowledge from multiple
users’ histories. While retrieval-augmented methods, such as those employed in
the LaMP benchmark (Salemi et al., 2023), have made progress in scaling to
more extensive user histories, they still operate on raw user data containing
relevant and irrelevant details. To address these limitations, we introduce a
Step-back Profiling approach that distills a user’s raw history into a concise
representation focusing on ’gist’ representations and preferences. Our
approach aims to enable more efficient and effective personalization across
diverse single and multi-user scenarios by reasoning about higher-level traits
instead of verbatim user history.

###  2.2 Procedure

Consider a set of $n$ users denoted by $\mathbb{U}$ = $\\{u_{i}\\}_{i=1}^{n}$,
where each user $u_{i}$ has a preference history
$\mathbb{H}_{i}=\\{(x_{ij},y_{ij})\\}_{j=1}^{m}$ consisting of $m$ input-
output pairs. To effectively generate $P(y|x,\mathbb{H}_{\mathbb{U}})$ based
on users’ preference history, we create a set of user profiles
$\mathbb{P}_{\mathbb{U}}=\\{\mathbb{P}_{u_{i}}|u_{i}\in\mathbb{U}\\}$ using
Step-back Profiling. The complete procedure involves the following steps:

##### User Profile Gisting:

Each user’s history is condensed into a short “gist” representation using an
abstraction function $\operatorname{Gist}(\cdot)$:
$\mathbb{P}_{u_{i}}=\operatorname{Gist}(\mathbb{H}_{i})$. The “gist” captures
the user’s high-level traits and interests.

##### Multi-User Profile Concatenation:

Individual user profiles $\mathbb{P}_{u_{1}}$, $\mathbb{P}_{u_{2}}$, $\cdots$,
$\mathbb{P}_{u_{n}}$ are concatenated to form a unified representation
$\mathbb{P}_{\mathbb{U}}=[\mathbb{P}_{u_{1}};\mathbb{P}_{u_{2}};\cdots;\mathbb{P}_{u_{n}}]$,
where $[\cdot;\cdot]$ is a permutation-sensitive function combining the user
profiles.

##### Retrieval-Augmented Generation (Optional):

Relevant snippets from user histories $\mathbb{H}_{\mathbb{U}}$ may be
retrieved for input $x$ using a retrieval function
$\operatorname{Retrieve}(\cdot)$. We have
$\mathbb{R}_{i,k}=\operatorname{Retrieve}(x,\mathbb{H}_{i},k)$, where
$\mathbb{R}_{i,k}$ is a set of top-$k$ retrieved input-output snippets from
user $u_{i}$’s history $\mathbb{H}_{i}$. The top-$k$ retrieved snippets
$\mathbb{R}_{k}=\\{\mathbb{R}_{i,k}\\}_{i=1}^{N}$ can be concatenated with $x$
to form an augmented input
$\hat{x}=[x;\mathbb{R}_{1,k};\mathbb{R}_{2,k};\cdots;\mathbb{R}_{n,k}]$.

##### Personalized Output Generation:

The personalized language model generates an output
$y=\operatorname{Generate}(\hat{x},\mathbb{P}_{\mathbb{U}})$ by conditioning
on the augmented input $\hat{x}$ (if retrieval is used) or the original input
$x$, along with the concatenated user profile $\mathbb{P}_{\mathbb{U}}$. The
generated output $y$ aligns with the user preferences captured by the Step-
back Profiling while following the input $x$.

##  3 The Personalized Scientific Writing (PSW) Benchmark

###  3.1 Problem Formulation

Personalized language models aim to generate outputs that follow a given input
and align with the users’ styles, preferences, and expertise. In multi-author
collaborative writing, each data entry in the PSW benchmark consists of four
key components: (1) An input sequence $x$ serves as the model’s input; (2) A
target output $y$ that the model is expected to generate; (3) A set of user
histories $\mathbb{H}_{\mathbb{U}}=\\{\mathbb{H}_{u_{i}}\\}_{i=1}^{l}$, where
$l$ is the number of collaborating authors, and each entry
$\mathbb{H}_{u_{i}}$ contains historical input-output pairs for user $u_{i}$;
(4) A set of author roles $\mathbb{C}=\\{c_{i}\\}_{i=1}^{l}$, each
representing the role of the corresponding author $u_{i}$ in the collaborative
writing process.

A personalized language model aims to generate an output $y$ that aligns with
the conditional probability distribution
$P(y|x,\mathbb{H}_{\mathbb{U}},\mathbb{C})$. This means the model should
produce an output that follows the input $x$ and the collaborating authors’
writing styles, preferences, and expertise, as captured by their user
histories $\mathbb{H}_{\mathbb{U}}$ and roles $\mathbb{C}$.

###  3.2 Task Description

##### UP-0: Research Interest Generation:

Before all the PSW tasks, we create a benchmark for user profiling. This
involves compiling a list of research interests that accurately reflect each
author’s expertise and research focus based on their publication history. To
acquire the necessary information, we extract the research interests of each
author from Google Scholar222https://github.com/scholarly-python-
package/scholarly by searching their name.

##### PSW-1: Research Topic Generation:

This task aims to generate a list of research topics that capture the
collaborating authors’ joint expertise and research focus, given their user
profiles. The generated research topics should be relevant to the authors’
past publications and help identify potential research directions for their
collaborative work. We use OpenAI’s GPT-4 model to automatically extract
research topics from selected papers. The extracted topics are then linked to
their respective papers and author profiles.

##### PSW-2: Research Question Generation:

This task focuses on generating a set of research questions that align with
the expertise and interests of the collaborating authors and are relevant to
the target paper. The generated research questions should help guide the
content and structure of the collaborative writing process. We automatically
use OpenAI’s GPT-4 model to extract research questions from the selected
papers for this task. The extracted research questions are then linked to
their papers and author profiles.

##### PSW-3: Paper Abstract Generation:

This task involves generating a paper abstract that summarizes the key points
and contributions of the collaborative research paper, given the user
profiles, research interests, target paper title, and research questions. We
directly retrieve the abstracts from the selected papers using the Semantic
Scholar API 333https://api.semanticscholar.org/. The retrieved abstracts are
then linked to their respective papers and author profiles.

##### PSW-4: Paper Title Generation:

This task aims to generate a suitable title for the collaborative research
paper, considering the user profiles, research interests, research questions,
and paper abstract. The data is collected by Semantic Scholar API as well.

###  3.3 GPT-based Evaluation

LLM-based evaluators, such as G-Eval, have shown high consistency with human
evaluators Liu et al. (2023); Chang et al. (2024), particularly in
personalized text generation Wang et al. (2023b). Therefore, we utilize
GPT-4-turbo with chain-of-thought prompting as a judge to evaluate the
generated outputs on the PSW benchmark in multiple dimensions (Zhang et al.,
2019a), including consistency, fluency, relevance, and novelty. An example of
our evaluation (G-Eval) prompt can be found in Appendix C.

##  4 Experimental Setup

###  4.1 Datasets and Evaluation

LaMP Dataset We follow the established LAMP benchmark Salemi et al. (2023),
encompassing three classification and four text generation tasks.
Specifically, these tasks are Personalized Citation Identification (LaMP-1),
Personalized News Categorization (LaMP-2), Personalized Product Rating
(LaMP-3), Personalized News Headline Generation (LaMP-4), Personalized
Scholarly Title Generation (LaMP-5), Personalized Email Subject Generation
(LaMP-6), and Personalized Tweet Paraphrasing (LaMP-7).

PSW Dataset The dataset includes one individual task, User Profiling (UP-0),
and four collaborative tasks: Research Topics Generation (PSW-1), Research
Question Generation (PSW-2), Paper Abstract Generation (PSW-3), and Paper
Title Generation (PSW-4).

Our evaluation follows the LaMP Salemi et al. (2023) and we employ the metrics
specified in the LaMP for each task. Those include F1, Accuracy, MAE, and RMSE
for classification tasks and ROUGE-1 and ROUGE-L for generation tasks. We
compare several baselines, including non-personalized language models, models
fine-tuned on history data without personalization, and models that use a
simple concatenation of user histories for personalization with retrieval
models.

###  4.2 Main Result

|  | Non-personalized | Personalized§ | Ours
---|---|---|---|---
Dataset | Metric | FlanT5-XXL† | ChatGPT† | FlanT5-XXL† | ChatGPT†
LaMP-1 | Accuracy | 0.522 | 0.510 | 0.675 | 0.701 | 0.624
LaMP-2 | Accuracy | 0.591 | 0.610 | 0.598 | 0.693 | 0.729
F1 | 0.463 | 0.455 | 0.477 | 0.455 | 0.591
LaMP-3 | MAE | 0.357 | 0.699 | 0.282 | 0.658 | 0.274
RMSE | 0.666 | 0.977 | 0.584 | 1.102 | 0.559
LaMP-4 | ROUGE-1 | 0.164 | 0.133 | 0.192 | 0.160 | 0.195
ROUGE-L | 0.149 | 0.118 | 0.178 | 0.142 | 0.180
LaMP-5 | ROUGE-1 | 0.455 | 0.395 | 0.467 | 0.398 | 0.469
ROUGE-L | 0.410 | 0.334 | 0.424 | 0.336 | 0.426
LaMP-6 | ROUGE-1 | 0.332 | - | 0.466 | - | 0.485
ROUGE-L | 0.320 | - | 0.453 | - | 0.464
LaMP-7 | ROUGE-1 | 0.459 | 0.396 | 0.448 | 0.391 | 0.455
ROUGE-L | 0.404 | 0.337 | 0.396 | 0.324 | 0.398

Table 1: Performance comparison of models on the LaMP dataset. †Baseline
results are obtained directly from Salemi et al. (2023). § Personalized means
we use retrieval modules before LLMs.

LaMP Results To ensure a fair comparison, we utilize a user-based separation
from LaMP Salemi et al. (2023). We only grant the model access to the provided
user history and restrict it from accessing any other information.
Additionally, we utilize the same pre-trained retriever in LaMP baselines,
without any additional fine-tuning, to retrieve the top five examples. This
approach is identical to the Non-Personalized setting in Salemi et al. (2023).
Finally, we compare our results with the outcomes reported in the study.

As shown in Table 1, our analysis unveils a notable performance enhancement
through our method’s application, significantly when leveraging the same
backbone language models (GPT-3.5-turbo). It is clear that our “gist”-style
information compression is much more necessary than retrieval methods as the
comparisons in Table 1. In the domain of text generation tasks
(LaMP-4$\sim$7), our method achieves an average improvement of $0.048$ in
Rouge-1 and $0.053$ in Rouge-L, corresponding to gains of $15.2\%$ and
$19.5\%$, respectively. Similarly, for the classification tasks
(LaMP-1$\sim$3), we observe an average +$12.6\%$ accuracy gain of and a
+$42.5\%$ reduction in MAE compared to the Non-Personalized setting. Our
method continues to exhibit better performance across most tasks, even when
compared with FlanT5-XXL, with a fine-tuned retriever as Personalized setting.
The prompt used in this experiment is detailed in Appendix D.

|  | Metrics
---|---|---
Datasets | Method | ROUGE-1 | ROUGE-L | Consistency | Fluency | Relevance | Novelty
UP-0 | Single-Author | 0.267 | 0.233 | 4.32 | 2.01 | 3.59 | /
PSW-1 | Zero-shot | 0.306 | 0.257 | 3.43 | 2.65 | 3.53 | 2.30
Single-Author | 0.325 | 0.266 | 3.44 | 2.47 | 3.61 | 2.59
Multi-Author | 0.337 | 0.280 | 3.59 | 2.58 | 3.67 | 2.63
PSW-2 | Zero-shot | 0.196 | 0.179 | 4.31 | 2.04 | 3.89 | 2.21
Single-Author | 0.190 | 0.171 | 4.20 | 2.23 | 3.67 | 2.01
Multi-Author | 0.201 | 0.186 | 4.60 | 2.39 | 3.91 | 2.38
PSW-3 | Zero-shot | 0.099 | 0.094 | 4.43 | 2.81 | 4.43 | 2.40
Single-Author | 0.131 | 0.124 | 4.94 | 2.94 | 4.70 | 2.40
Multi-Author | 0.145 | 0.131 | 4.92 | 2.94 | 4.71 | 2.45
PSW-4 | Zero-shot | 0.459 | 0.391 | 4.41 | 2.41 | 3.58 | 2.38
Single-Author | 0.472 | 0.409 | 4.59 | 2.49 | 3.78 | 2.60
Multi-Author | 0.505 | 0.444 | 4.64 | 2.59 | 3.79 | 2.64

Table 2: Performance comparison of personalized models on the PSW dataset. We
report additional metrics such as Consistency (1-5), Fluency (1-3), Relevance
(1-5), and Novelty (1-3).

PSW Results: We then evaluate our proposed model using the PSW dataset,
focusing on user profiling (UP-0), personalized idea brainstorming (PSW-1,
PSW-2), and personalized text generation (PSW-3, PSW-4) in three different
settings:

  1. 1.

Zero-shot: Generates outputs based on the input prompt $x$ alone:
$y=\operatorname{Generate}(x)$.

  2. 2.

Single-Author: Personalizes with single user’s profile $P_{u_{i}}$ and
retrieved snippets $R_{i}$: $y=\operatorname{Generate}(\hat{x},P_{u_{i}})$,
where $\hat{x}=[x;\mathbb{R}_{i}]$ and
$\mathbb{R}_{i}=\operatorname{Retrieve}(x,\mathbb{H}_{i},10)$.

  3. 3.

Multi-Author: Personalizes with multiple users’ profiles
$\mathbb{P}_{\mathbb{U}}$ and retrieved snippets $\mathbb{R}$:
$y=\operatorname{Generate}(\hat{x},\mathbb{P}_{\mathbb{U}})$, where
$\hat{x}=[x;\mathbb{R}_{1};\cdots;\mathbb{R}_{n}]$,
$\mathbb{R}_{i}=\operatorname{Retrieve}(x,\mathbb{H}_{i},10)$ for each user
$u_{i}$.

As shown in Table 2, our Multi-Author setting demonstrates superior
performance across all tasks. In PSW-1 and PSW-2, the Multi-Author setting
outperforms both Zero-shot and Single-Author settings, with an average
improvement of +$6.9\%$ in ROUGE-1 and +$7.1\%$ in ROUGE-L. Similarly, for the
PSW-3 and PSW-4, the Multi-Author setting achieves the highest ROUGE scores,
with an average gain of +$28.2\%$ in ROUGE-1 and +$26.6\%$ in ROUGE-L,
compared to the Zero-shot and Single-Author settings. Furthermore, the Multi-
Author setting exhibits the highest scores for additional metrics such as
Consistency, Fluency, Relevance, and Novelty across all tasks, with an average
improvement of +$5.1\%$, +$6.7\%$, +$3.8\%$, and +$6.4\%$, respectively,
compared to the Zero-shot and Single-Author setting. The prompt used in this
experiment is detailed in Appendix E.

###  4.3 Ablation Study

Finally, to evaluate the contribution of each component, we perform an
ablation study when: 1) Switching the order of users and 2) Removing user
profiling.

####  4.3.1 Impact of Author Order

Table 3 shows how changing the author order affects the performance of multi-
user personalized models. We experiment with three variants:

Original: The original author order as provided in the dataset. Swap-Random:
Randomly shuffle the order of authors. Swap-First: Move the first author to
the end of the author list.

|  | Metrics
---|---|---
Datasets | Variants | ROUGE-1 | ROUGE-L | Consistency | Fluency | Relevance | Novelty
PSW-1 | Original | 0.337 | 0.280 | 3.59 | 2.58 | 3.67 | 2.63
Swap-Random | 0.321 | 0.272 | 3.42 | 2.48 | 3.69 | 2.45
Swap-First | 0.314 | 0.260 | 3.35 | 2.42 | 3.48 | 2.37
PSW-2 | Original | 0.201 | 0.186 | 4.60 | 2.39 | 3.91 | 2.38
Swap-Random | 0.193 | 0.178 | 4.53 | 2.30 | 3.85 | 2.42
Swap-First | 0.186 | 0.171 | 4.46 | 2.27 | 3.77 | 2.29
PSW-3 | Original | 0.145 | 0.131 | 4.92 | 2.94 | 4.71 | 2.45
Swap-Random | 0.138 | 0.125 | 4.84 | 2.88 | 4.65 | 2.50
Swap-First | 0.130 | 0.117 | 4.78 | 2.98 | 4.57 | 2.55
PSW-4 | Original | 0.505 | 0.444 | 4.64 | 2.59 | 3.79 | 2.64
Swap-Random | 0.492 | 0.431 | 4.57 | 2.55 | 3.72 | 2.70
Swap-First | 0.483 | 0.421 | 4.50 | 2.50 | 3.64 | 2.76

Table 3: Impact of author order on the performance of multi-user personalized
models We report additional metrics such as Consistency (1-5), Fluency (1-3),
Relevance (1-5), and Novelty (1-3).

The Original order consistently achieves the best performance across all
metrics on all PSW tasks. Randomly swapping authors (Swap-Random) leads to a
slight decline, while moving the first author to the end (Swap-First) results
in a more significant drop. This observation highlights the importance of
preserving the original author order in multi-author collaborative writing
scenarios. The first author, often the lead or corresponding author,
significantly influences the document’s content, structure, and style. As a
result, their writing style and expertise tend to be most prominently
reflected in the document. Disrupting this order introduces noise and hinders
the model’s ability to capture the individual authors’ impact and the logical
progression of ideas, particularly affecting the generation tasks (PSW-3 and
PSW-4), where content and style are heavily influenced by the main author’s
expertise and preferences.

####  4.3.2 Impact of User Profiling

Table 4 reports ablation results on the user profile component:

Original: User profiles constructed using Step-back Profiling. Removed: No
user profiles were used, only retrieving relevant snippets. Random: Replacing
target user profiles with randomly sampled user profiles.

|  | Metrics
---|---|---
Datasets | Profile | ROUGE-1 | ROUGE-L | Consistency | Fluency | Relevance | Novelty
PSW-1 | Original | 0.337 | 0.280 | 3.59 | 2.58 | 3.67 | 2.63
Removed | 0.297 | 0.250 | 3.21 | 2.49 | 3.31 | 2.57
Random | 0.328 | 0.272 | 3.55 | 2.56 | 3.62 | 2.68
PSW-2 | Original | 0.201 | 0.186 | 4.60 | 2.39 | 3.91 | 2.38
Removed | 0.180 | 0.166 | 4.28 | 2.32 | 3.63 | 2.33
Random | 0.195 | 0.182 | 4.57 | 2.42 | 3.89 | 2.45
PSW-3 | Original | 0.145 | 0.131 | 4.92 | 2.94 | 4.71 | 2.45
Removed | 0.128 | 0.115 | 4.70 | 2.87 | 4.50 | 2.41
Random | 0.142 | 0.128 | 4.95 | 2.96 | 4.69 | 2.51
PSW-4 | Original | 0.505 | 0.444 | 4.64 | 2.59 | 3.79 | 2.64
Removed | 0.475 | 0.419 | 4.38 | 2.53 | 3.58 | 2.56
Random | 0.498 | 0.438 | 4.60 | 2.58 | 3.76 | 2.69

Table 4: Impact of the user profile on the performance of multi-user
personalized models. We report additional metrics such as Consistency (1-5),
Fluency (1-3), Relevance (1-5), and Novelty (1-3).

Removing user profiles (Removed) leads to the largest performance decline,
confirming the benefit of Step-back Profiling in multi-user personalization.
Using random profile texts (Random) recovers some of the gaps but still
underperforms the Original profiles. This demonstrates that the distilled user
traits successfully capture useful information for collaborative writing, such
as individual writing styles, expertise, and preferences. The performance gap
between Original and Random profiles highlights the effectiveness of the Step-
back Profiling technique in extracting relevant user characteristics from
their background information. These findings underscore the importance of
incorporating author-specific traits to enable a more personalized and
contextually appropriate generation in multi-user settings.

##  5 Conclusion

In summary, we introduce a training-free technique, Step-back Profiling, for
personalizing large language models by distilling user interactions using gist
into concise profiles. Moreover, we extend the LaMP dataset into the
Personalized Scientific Writing (PSW) dataset to evaluate multi-user scenarios
in collaborative scientific writing. Our experiments show that the proposed
method is effective on the LaMP and PSW datasets. In particular, both single-
user and multi-user settings validate the benefits of profile-guided
personalization. Finally, studying the interpretability and controllability of
profile-guided models can help build user trust and allow for more fine-
grained customization.

## Limitation

Our proposed Step-back Profiling framework has a few limitations that warrant
discussion and could be addressed in future work:

Dataset Specificity The experiments and results presented are primarily based
on the Personalized Scientific Writing (PSW) dataset and the LaMP benchmark.
While these datasets provide a diverse set of tasks, the performance and
applicability of the Step-back Profiling framework may vary with different
datasets or domains not covered by our experiments. Future work should
evaluate the model on more varied datasets to ensure generalizability.

Complexity of Profiles The profile generation process involves distilling user
histories into concise representations. While this method captures essential
traits, it may oversimplify user preferences and neglect nuanced behaviors
present in longer and more complex histories. More sophisticated profiling
techniques that can retain and effectively compress these complexities are
needed.

Scalability and Efficiency Although the Step-back Profiling method improves
memory management, the approach still has scalability concerns, particularly
with very large user histories or an increasing number of collaborators.
Efficiently managing and retrieving relevant user data from extensive
histories without compromising performance remains a challenge.

Dynamic Adaptation The current method creates static profiles based on
available user histories at a given time. However, user preferences and styles
may evolve, especially in dynamic collaborative environments. Developing a
mechanism to update profiles dynamically based on real-time user interactions
and feedback could further enhance the personalization capabilities.

Evaluation Metrics The evaluation relies heavily on established metrics such
as ROUGE and human-aligned scoring via G-Eval, which, while comprehensive, may
not capture all dimensions of personalized content quality. Developing and
employing more specialized evaluation metrics for personalized content
generation, particularly in scientific and collaborative writing, would
provide deeper insights into the effectiveness of the methods.

Human Factors: Although tools like GPT-4 mitigate the involvement of human
evaluation, it is inherently subjective. Future work should consider more
robust and unbiased methods of human evaluation to validate the effectiveness
of personalized outputs objectively.

Ethical and Privacy Concerns Personalizing models using user histories raises
potential ethical and privacy issues. It is crucial to ensure that user data
is handled securely and that privacy concerns are adequately addressed. Future
research should explore more privacy-preserving techniques for
personalization, such as federated learning.

Adapting Step-back Profiling to long histories spanning multiple sessions is
another valuable direction. Future work can explore more advanced profiling
strategies, such as hierarchical representations and dynamic profile updates
based on user feedback.

## Ethical Statement

Dataset Licensing We have constructed the Personalized Scientific Writing
(PSW) dataset, which will be publicly released under the MIT license. This
permissive license allows users to freely use, modify, and distribute the
dataset. By releasing the PSW dataset under the MIT license, we aim to promote
transparency, reproducibility, and wide adoption of our research within the
community.

Artifact Use Consistent With Intended Use Regarding our use of existing
artifacts, we have ensured that our usage is consistent with their intended
purposes, as specified by their creators. For the artifacts we create,
including the PSW dataset, we specify that the intended use is for research
purposes. This is compatible with the original access conditions of any
derivative data we utilized. Derivative data accessed for research purposes
should not be used outside of research contexts.

Personally-Identifying Info We acknowledge that the PSW dataset construction
involved the use of researchers’ real names to accurately reflect their
contributions and expertise. However, to protect individual privacy and
prevent any potential personal information leakage, the publicly released
version of our dataset replaces real names with unique identifiers (IDs). This
anonymization step ensures that no personally identifying information is
disclosed while maintaining the dataset’s utility for research purposes.

We have taken these steps to safeguard the privacy and personal information of
the individuals whose data contributed to our research. Additionally, we have
reviewed the dataset to ensure it does not contain any offensive content.

Documentation Of Artifacts While our dataset does not involve artificial
distributions, we have collected and included gender information in the
metadata. This metadata, along with other relevant descriptive information
about the dataset, will be made publicly available upon the paper’s
acceptance.

## References

  * Cao et al. (2023) Yihan Cao, Siyu Li, Yixin Liu, Zhiling Yan, Yutong Dai, Philip S Yu, and Lichao Sun. 2023.  A comprehensive survey of AI-generated content (AIGC): A history of generative AI from GAN to ChatGPT.  _arXiv preprint arXiv:2303.04226_.
  * Chang et al. (2024) Yupeng Chang, Xu Wang, Jindong Wang, Yuan Wu, Linyi Yang, Kaijie Zhu, Hao Chen, Xiaoyuan Yi, Cunxiang Wang, Yidong Wang, et al. 2024.  A survey on evaluation of large language models.  _ACM Transactions on Intelligent Systems and Technology_ , 15(3):1–45.
  * Character.AI (2022) Character.AI. 2022.  Character.AI.  https://character.ai/.
  * Chen et al. (2023) Jin Chen, Zheng Liu, Xu Huang, Chenwang Wu, Qi Liu, Gangwei Jiang, Yuanhao Pu, Yuxuan Lei, Xiaolong Chen, Xingmei Wang, et al. 2023.  When large language models meet personalization: Perspectives of challenges and opportunities.  _arXiv preprint arXiv:2307.16376_.
  * Fricke (2018) Suzanne Fricke. 2018.  Semantic scholar.  _Journal of the Medical Library Association: JMLA_ , 106(1):145.
  * Jiang et al. (2023) Hang Jiang, Xiajie Zhang, Xubo Cao, Jad Kabbara, and Deb Roy. 2023.  PersonaLLM: Investigating the ability of large language models to express personality traits.  _arXiv preprint arXiv:2305.02547_.
  * Liu et al. (2024) Nelson F Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, and Percy Liang. 2024.  Lost in the middle: How language models use long contexts.  _Transactions of the Association for Computational Linguistics_ , 12:157–173.
  * Liu et al. (2023) Yang Liu, Dan Iter, Yichong Xu, Shuohang Wang, Ruochen Xu, and Chenguang Zhu. 2023.  G-Eval: NLG evaluation using GPT-4 with better human alignment.  _arXiv preprint arXiv:2303.16634_.
  * OpenAI (2023) OpenAI. 2023.  GPT-4 technical report.  _Preprint_ , arXiv:2303.08774.
  * Qin et al. (2023) Yujia Qin, Shihao Liang, Yining Ye, Kunlun Zhu, Lan Yan, Yaxi Lu, Yankai Lin, Xin Cong, Xiangru Tang, Bill Qian, et al. 2023.  ToolLLM: Facilitating large language models to master 16000+ real-world APIs.  _arXiv preprint arXiv:2307.16789_.
  * Salemi et al. (2023) Alireza Salemi, Sheshera Mysore, Michael Bendersky, and Hamed Zamani. 2023.  LaMP: When large language models meet personalization.  _arXiv preprint arXiv:2304.11406_.
  * Shao et al. (2023) Yunfan Shao, Linyang Li, Junqi Dai, and Xipeng Qiu. 2023.  Character-LLM: A trainable agent for role-playing.  _arXiv preprint arXiv:2310.10158_.
  * Shi et al. (2023) Freda Shi, Xinyun Chen, Kanishka Misra, Nathan Scales, David Dohan, Ed H Chi, Nathanael Schärli, and Denny Zhou. 2023.  Large language models can be easily distracted by irrelevant context.  In _International Conference on Machine Learning_ , pages 31210–31227. PMLR.
  * Tan and Jiang (2023) Zhaoxuan Tan and Meng Jiang. 2023.  User modeling in the era of large language models: Current research and future directions.  _arXiv preprint arXiv:2312.11518_.
  * Wang et al. (2019) Xuewei Wang, Weiyan Shi, Richard Kim, Yoojung Oh, Sijia Yang, Jingwen Zhang, and Zhou Yu. 2019.  Persuasion for good: Towards a personalized persuasive dialogue system for social good.  _arXiv preprint arXiv:1906.06725_.
  * Wang et al. (2023a) Yancheng Wang, Ziyan Jiang, Zheng Chen, Fan Yang, Yingxue Zhou, Eunah Cho, Xing Fan, Xiaojiang Huang, Yanbin Lu, and Yingzhen Yang. 2023a.  RecMind: Large language model powered agent for recommendation.  _arXiv preprint arXiv:2308.14296_.
  * Wang et al. (2023b) Yaqing Wang, Jiepu Jiang, Mingyang Zhang, Cheng Li, Yi Liang, Qiaozhu Mei, and Michael Bendersky. 2023b.  Automated evaluation of personalized text generation using large language models.  _arXiv preprint arXiv:2310.11593_.
  * Wei et al. (2022a) Jason Wei, Yi Tay, Rishi Bommasani, Colin Raffel, Barret Zoph, Sebastian Borgeaud, Dani Yogatama, Maarten Bosma, Denny Zhou, Donald Metzler, et al. 2022a.  Emergent abilities of large language models.  _arXiv preprint arXiv:2206.07682_.
  * Wei et al. (2022b) Penghui Wei, Xuanhua Yang, Shaoguo Liu, Liang Wang, and Bo Zheng. 2022b.  CREATER: CTR-driven advertising text generation with controlled pre-training and contrastive fine-tuning.  _arXiv preprint arXiv:2205.08943_.
  * Zhang et al. (2023a) Kai Zhang, Fubang Zhao, Yangyang Kang, and Xiaozhong Liu. 2023a.  Memory-augmented LLM personalization with short-and long-term memory coordination.  _arXiv preprint arXiv:2309.11696_.
  * Zhang et al. (2024) Kaiyan Zhang, Jianyu Wang, Ermo Hua, Biqing Qi, Ning Ding, and Bowen Zhou. 2024.  Cogenesis: A framework collaborating large and small language models for secure context-aware instruction following.  _arXiv preprint arXiv:2403.03129_.
  * Zhang et al. (2019a) Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q Weinberger, and Yoav Artzi. 2019a.  BERTScore: Evaluating text generation with BERT.  _arXiv preprint arXiv:1904.09675_.
  * Zhang et al. (2019b) Yizhe Zhang, Siqi Sun, Michel Galley, Yen-Chun Chen, Chris Brockett, Xiang Gao, Jianfeng Gao, Jingjing Liu, and Bill Dolan. 2019b.  DialoGPT: Large-scale generative pre-training for conversational response generation.  _arXiv preprint arXiv:1911.00536_.
  * Zhang et al. (2023b) Zhuosheng Zhang, Yao Yao, Aston Zhang, Xiangru Tang, Xinbei Ma, Zhiwei He, Yiming Wang, Mark Gerstein, Rui Wang, Gongshen Liu, et al. 2023b.  Igniting language intelligence: The hitchhiker’s guide from chain-of-thought reasoning to language agents.  _arXiv preprint arXiv:2311.11797_.
  * Zhiyuli et al. (2023) Aakas Zhiyuli, Yanfang Chen, Xuan Zhang, and Xun Liang. 2023.  BookGPT: A general framework for book recommendation empowered by large language model.  _arXiv preprint arXiv:2305.15673_.

##  Appendix A The PSW Dataset

Overview. The PSW dataset is constructed using data from the Semantic Scholar
database (Fricke, 2018). We first selected a subset of papers from Software
Engineering published after 2000, considering only papers with at least two
authors to ensure the feasibility of evaluating collaborative writing
scenarios. The collected papers were randomly split into training, validation,
and test subsets.444We only used the test split in this paper since our method
doesn’t require model training. We performed the split at the paper level to
ensure that all tasks within the PSW benchmark had consistent data splits. The
summary of PSW dataset statistics can be found in Table 5.

Statistic | Train | Valid | Test
---|---|---|---
# of Papers | 1,744 | 500 | 500
# of Authors | 6,461 | 1,655 | 1,280
Avg. Authors / Paper | 4.05 | 3.16 | 3.25
Avg. History Papers / Author | 63.47 | 75.34 | 92.21
Avg. Research Interests / Author | 2.84 | 2.77 | 2.79
Avg. Title Length | 97.03 | 95.54 | 96.16
Avg. Abstract Length | 970.92 | 981.36 | 1,037.09
Avg. Research Question Length | 470.57 | 398.22 | 442.31
Avg. References / Paper | 60.24 | 54.85 | 58.93

Table 5: PSW Dataset Statistics with Train / Valid / Test Splits.

##  Appendix B Metrics Visualization on PSW Dataset

Figures 3, 4, and 5 illustrate the results discussed in Section 4.2 and
Appendices 4.3.1 and 4.3.2, respectively.

(a) PSW-1

(b) PSW-2

(c) PSW-3

(d) PSW-4

Figure 3: Performance metrics across three different models: Zero-shot,
Single-Author, and Multi-Author. The Multi-Author model consistently achieves
the highest scores across all datasets.

(a) PSW-1

(b) PSW-2

(c) PSW-3

(d) PSW-4

Figure 4: Impact of author order on the performance across three different
models: Original, Swap-Random, and Swap-First. The Original model consistently
achieves the highest scores across all datasets.

(a) PSW-1

(b) PSW-2

(c) PSW-3

(d) PSW-4

Figure 5: Impact of user profiling on the performance across three different
models: Original, Removed, and Random. The Original model consistently
achieves the highest scores across all datasets.

##  Appendix C Details of G-Eval

Task Description
---
You will be given one result generated for a science paper and several
reference papers. Your task is to rate the result using the following
criteria.
Please make sure you read and understand these instructions carefully. Please
keep this document open while reviewing, and refer to it as needed.
Evaluation Criteria
Consistency (1-5) – the factual alignment between the result and the
corresponding science paper. A factually consistent result contains only
statements entailed by the source document.
Fluency (1-3) – the quality of the result in terms of grammar, spelling,
punctuation, word choice, and sentence structure.
Relevance (1-5) – the selection of important content from the source. The
result should include only important information from the source document.
Novelty (1-3) – the uniqueness and originality of the result in terms of
concept, perspective, and creativity.
Evaluation Task
Now, you are working on evaluating this prediction:
{Prediction Text}
Here are some ground truth results for comparison: [result1, result2, …].
Instruction
Please evaluate the prediction using the above criteria.
Table 6: Prompt template for evaluating the G-Eval metric.

##  Appendix D Prompts for LaMP Tasks

###  D.1 Personalized Citation Identification (LaMP-1)

User Profile
---
Assuming you care a lot about these areas:
Keywords: [keyword1, keyword2, keyword3, …]
Topics: [topics1, topics2, topics3, …]
User History
I give you some titles of papers that you’ve written. Please imitate your
reasons and recommend a paper citation for me. Each example consists of an
abstract, the corresponding title, and a description of the writing style and
keywords for that title.
Example 1
Title: {Title Text}
Abstract: {Abstract Text}
Reason: {Reason}
Citation: [citation1, citation2, …]
Example 2
Title: {Title Text}
Abstract: {Abstract Text}
Reason: {Reason}
Citation: [citation1, citation2, …]
…
Example k
Title: {Title Text}
Abstract: {Abstract Text}
Reason: {Reason}
Citation: [citation1, citation2, …]
Classification Task
Now you have written this title:
Title: {Title Text}
Instruction
Please separately analyze the potential relevant connection of Reference 1 and
Reference 2 to this title. You are citing from one of them. Please decide
which one it would be:
Reference 1: {option1}
Reference 2: {option2}
Just answer with [1] or [2] without explanation.
Table 7: Prompt template for the Personalized Citation Identification (LaMP-1)
task.

###  D.2 Personalized News Categorization (LaMP-2)

User Profile
---
Assuming you care a lot about these areas:
Keywords: [keyword1, keyword2, keyword3, …]
Topics: [topics1, topics2, topics3, …]
User History
I give you some titles and articles that you’ve written with category. Please
imitate your reasons for giving this category. Each example consists of an
abstract, the corresponding title, and a category of it.
Example 1
Article: {Article Text}
Title: {Title Text}
Reason: {Reason}
Category: [category1, category2, …]
Example 2
Article: {Article Text}
Title: {Title Text}
Reason: {Reason}
Category: [category1, category2, …]
…
Example k
Article: {Article Text}
Title: {Title Text}
Reason: {Reason}
Category: [category1, category2, …]
Classification Task
Now you have written this article with the title:
Article: {Article Text}
Title: {Title Text}
Instruction
Which category does this article relate to among the following categories?
Category 1: {option1}
Category 2: {option2}
…
Category K: {optionN}
Just answer with the category name without further explanation.
Table 8: Prompt template for the Personalized News Categorization (LaMP-2)
task.

###  D.3 Personalized Product Rating (LaMP-3)

User Profile
---
Assuming you have written product reviews with the following characteristics:
Most Common Rating: {score${}_{\text{most}}$}
Rating Patterns: [pattern1, pattern2, …]
User History
I provide you with some product reviews you’ve written, along with their
corresponding ratings. Please imitate your reasoning for assigning these
ratings. Each example consists of a product review and its rating.
Example 1
Product Review: {Review Text}
Rating: {Rating}
Example 2
Product Review: {Review Text}
Rating: {Rating}
…
Example k
Product Review: {Review Text}
Rating: {Rating}
Rating Task
Now you have written this new product review:
Product Review: {Review Text}
Based on the review, please analyze its sentiment and how much you like the
product.
Instruction
Follow your previous rating habits and these instructions:
• If you feel satisfied with this product or have concerns but it’s good
overall, it should be rated 5. • If you feel good about this product but
notice some issues, it should be rated as 4. • If you feel OK but have
concerns, it should be rated as 3. • If you feel unsatisfied with this product
but it’s acceptable for some reason, it should be rated as 2. • If you feel
completely disappointed or upset, it should be rated 1.
Your most common rating is {score${}_{\text{most}}$}. You must follow this
rating pattern faithfully and answer with the rating without further
explanation.
Table 9: Prompt template for the Personalized Product Review Rating (LaMP-3)
task.

###  D.4 Personalized News Headline Generation (LaMP-4)

User Profile
---
Assuming you have written headlines with the following characteristics:
Writing Style: [style1, style2, …]
Content Patterns: [patterns1, patterns2, …]
User History
I will provide you with some news articles along with the headlines you’ve
written for them. Please imitate your writing style and content patterns when
generating a new headline. Each example consists of a news article and its
corresponding headline.
Example 1
Article: {Article Text}
Headline: {Headline}
Example 2
Article: {Article Text}
Headline: {Headline}
…
Example k
Article: {Article Text}
Headline: {Headline}
Generation Task
Now that you have been given this news article:
Article: {Article Text}
Instruction
Please write a headline following your previous writing styles and habits. If
you have written headlines with similar content, you could reuse those
headlines and mimic their content.
Table 10: Prompt template for the Personalized News Headline Generation
(LaMP-4) task.

###  D.5 Personalized Scholarly Title Generation (LaMP-5)

User Profile
---
Assuming you have written scholarly titles with the following characteristics:
Writing Style: [style1, style2, …]
Title Patterns: [pattern1, pattern2, …]
User History
I will provide you with some research paper abstracts along with the titles
you’ve written for them. Please imitate your writing style and title patterns
when generating a new title. Each example consists of a paper abstract and its
corresponding title.
Example 1
Abstract: {Abstract Text}
Title: {Title}
Example 2
Abstract: {Abstract Text}
Title: {Title}
…
Example k
Abstract: {Abstract Text}
Title: {Title}
Generation Task
Now that you have been given this paper abstract:
Abstract: {Abstract Text}
Instruction
Please write a title following your previous style and habits, keeping it
clear, accurate, and concise.
Table 11: Prompt template for the Personalized Scholarly Title Generation
(LaMP-5) task.

###  D.6 Personalized Email Subject Generation (LaMP-6)

User Profile
---
Assuming you care a lot about these areas:
Keywords: [keyword1, keyword2, keyword3, …]
Topics: [topics1, topics2, topics3, …]
User History
Let’s say there are some emails you’ve written. Please mimic the style of
these examples. Each example consists of email content, the corresponding
subject, and a description of the writing style for that title.
Example 1
Content: {Email Content}
Writing Style: {Style}
Subject: {Email Subject}
Example 2
Content: {Email Content}
Writing Style: {Style}
Subject: {Email Subject}
…
Example k
Content: {Email Content}
Writing Style: {Style}
Subject: {Email Subject}
Generation Task
Now that you have been given this email content:
Content: {Email Content}
Instruction
Write a title following your previous style and habits. Just answer with the
subject without further explanation.
Table 12: Prompt template for the Personalized Email Subject Generation
(LaMP-6) task.

###  D.7 Personalized Tweet Paraphrasing (LaMP-7)

User Profile
---
Assuming you have written tweets with the following characteristics:
Writing Style: [style1, style2, …]
Tone: [tone1, tone2, …]
Length: [length1, length2, …]
User History
I will provide you with some original tweets along with the paraphrased
versions you’ve written for them. When paraphrasing a new tweet, please
imitate your writing style, tone, and typical length. Each example consists of
an original tweet and its paraphrased version.
Example 1
Original Tweet: {Tweet Text}
Paraphrased Tweet: {Paraphrased Text}
Example 2
Original Tweet: {Tweet Text}
Paraphrased Tweet: {Paraphrased Text}
…
Example k
Original Tweet: {Tweet Text}
Paraphrased Tweet: {Paraphrased Text}
Generation Task
Now that you have been given this tweet:
Original Tweet: {Tweet Text}
Instruction
Please paraphrase it with the following instructions:  • You must use tweet
styles and tones. • You must keep it faithful to the given tweet with similar
keywords and length.
Table 13: Prompt template for the Personalized Tweet Paraphrasing (LaMP-7)
task.

##  Appendix E Prompts for PSW Tasks

###  E.1 Research Interests Generation (UP-0)

User History
---
I will provide you with some research papers you’ve authored. Please summarize
your top research interests based on these papers. Each paper consists of a
title and abstract.
Paper 1
Title: {Title Text}
Abstract: {Abstract Text}
Paper 2
Title: {Title Text}
Abstract: {Abstract Text}
…
Paper k
Title: {Title Text}
Abstract: {Abstract Text}
Instruction
Please summarize your top three research interests based on the provided
papers in the following format:
Research Interests: [interest1, interest2, interest3, …]
Table 14: Prompt template for the Research Interests Generation (UP-0) task.

###  E.2 Personalized Research Paper Title Generation (PSW-1)

User Profile
---
Assuming you are an expert researcher with the following research interests:
Research Interests: [interest1, interest2, interest3, …]
User History
Here are some titles and abstracts from papers you have authored:
Paper 1
Title: {Title}
Abstract: {Abstract}
Paper 2
Title: {Title}
Abstract: {Abstract}
…
Paper k
Title: {Title}
Abstract: {Abstract}
Brainstorm Task
Here are some related papers for reference, each with a title:
Reference 1: {Title}
Reference 2: {Title}
…
Reference N: {Title}
Instruction
Considering your research interests, previous works, and reference papers,
please brainstorm the most promising title for your new research paper.
Table 15: Prompt template for the Personalized Research Paper Title Generation
(PSW-1) task.

###  E.3 Research Question Generation (PSW-2)

User Profile
---
Assuming you are an expert researcher with the following research interests:
Research Interests: [interest1, interest2, interest3, …]
User History
Here are some titles and abstracts from papers you have authored:
Paper 1
Title: {Title}
Abstract: {Abstract}
Paper 2
Title: {Title}
Abstract: {Abstract}
…
Paper k
Title: {Title}
Abstract: {Abstract}
Brainstorm Task
Now you are working on a new paper with the following title:
Title: {Title}
Instruction
Considering the title and research background, please propose the top 3
research questions you aim to address in this new paper.
Table 16: Prompt template for the Research Question Generation (PSW-2) task.

###  E.4 Paper Abstract Generation (PSW-3)

User Profile
---
Assuming you are an expert researcher with the following research interests:
Research Interests: [interest1, interest2, interest3, …]
User History
Here are some titles and abstracts from papers you have authored:
Paper 1
Title: {Title}
Abstract: {Abstract}
Paper 2
Title: {Title}
Abstract: {Abstract}
…
Paper k
Title: {Title}
Abstract: {Abstract}
Generation Task
Now you are working on a new paper with the following title:
Title: {Title}
And you are focusing on solving the following research questions: [question1,
question2, …]
Instruction
Considering the title, research questions, and your writing style in previous
abstracts, please write an abstract for this new paper.
Table 17: Prompt template for the Paper Abstract Generation (PSW-3) task.

###  E.5 Paper Title Generation (PSW-4)

User Profile
---
Assuming you are an expert researcher with the following research interests:
Research Interests: [interest1, interest2, interest3, …]
User History
Here are some titles and abstracts from papers you have authored:
Paper 1
Title: {Title}
Abstract: {Abstract}
Paper 2
Title: {Title}
Abstract: {Abstract}
…
Paper k
Title: {Title}
Abstract: {Abstract}
Generation Task
Now, you are working on a new paper with the following abstract:
Abstract: {Abstract}
And you are focusing on solving the following research questions: [question1,
question2, …]
Instruction
Considering the abstract and your title writing style in previous papers,
please generate a title for this new paper. The title should be clear and
concise and reflect the main topic of the abstract as well as your research
questions.
Table 18: Prompt template for the Paper Title Generation (PSW-4) task.