HOMOLOGY GROUPS OF PIPELINE PETRI NETS 111This work was performed as a part of
the Strategic Development Program at the National Educational Institutions of
the Higher Education, N 2011-PR-054

A.A. Husainov, E.S. Bushmeleva, T.A. Trishina

###### Abstract

We study homology groups of elementary Petri nets for the pipeline systems. We
show that the integral homology groups of these nets equal the group of
integers in dimensions $0$ and $1$, and they are zero in other dimensions. We
prove that directed homology groups of elementary Petri nets are zero in all
dimensions.

2000 Mathematics Subject Classification 18G10, 18G35, 55U10, 68Q10, 68Q85

Key words: homology of small categories, trace monoid, asynchronous transition
system, elementary Petri net, pipeline, semicubical set.

## Introduction

The homology groups of elementary Petri nets were introduced in [1]. In [1],
it was built also an algorithm to compute the first homology group of
asynchronous systems, based on which we calculated the homology of Petri net
of pipeline ${\mathfrak{P}}_{3}$, consisting of three transitions. In [2], an
algorithm for computing the homology groups of elementary Petri net was
developed. An example of calculating the homology groups of elementary Petri
net for pipeline ${\mathfrak{P}}_{4}$

$\textstyle{t_{1}}$ $\scriptstyle{p_{1}}$$\textstyle{t_{2}}$
$\scriptstyle{p_{2}}$$\textstyle{t_{3}}$
$\scriptstyle{p_{3}}$$\textstyle{t_{4}}$

was considered.

We calculated the homology group for ${\mathfrak{P}}_{n}$, with $n=2,3,4,5$ by
the software described in [3]. In these cases, the homology group for the
pipelines are
$H_{0}({\mathfrak{P}}_{n})=H_{1}({\mathfrak{P}}_{n})={\,\mathbb{Z}}$, and
$H_{k}({\mathfrak{P}}_{n})=0$ for $k\geqslant 2$. There was a conjecture that
this is true for all $n\geqslant 2$. In this paper, we prove this conjecture.
In addition, the second co-author of this paper has developed the software
designed to calculate the directed homology groups of the state spaces.
Calculations showed that the directed homology groups of elementary Petri nets
${\mathfrak{P}}_{n}$ for $n=2,3,4,5$ are equal to zero in all dimensions. In
this paper, we show that this is true for all $n\geqslant 2$.

##  1 Preliminaries

For any category ${\mathscr{C}}$ denote by ${\rm Ob\,}{\mathscr{C}}$ the class
of its objects, ${\rm Mor\,}{\mathscr{C}}$ the class of all morphisms, and
${\mathscr{C}}^{op}$ the opposite category. For any $A,B\in{\rm
Ob\,}{\mathscr{C}}$ the set of all morphisms $A\to B$ in a category
${\mathscr{C}}$ is denoted by ${\mathscr{C}}(A,B)$.

###  1.1 Partial maps

For any sets $S$ and $T$ a partial map $f:S\rightharpoonup T$ is a relation
$f\subseteq S\times T$ with the following property:

$(\forall s\in S)(\forall t_{1},t_{2}\in T)~{}(s,t_{1})\in
f~{}\&~{}(s,t_{2})\in f~{}\Rightarrow~{}t_{1}=t_{2}\,.$

We denote $PSet$ category of sets and partial maps.

For any set $S$ denote $S_{*}=S\sqcup\\{*\\}$. Let ${\rm Set}_{*}$ be a
category of sets of the form $S_{*}$ where $S$ are arbitrary sets. For any of
its objects $S_{*}$ and $T_{*}$, let the set of morphisms ${\rm
Set}_{*}(S_{*},T_{*})$ consists of maps $f:S_{*}\to T_{*}$ satisfying the
equality $f(*)=*$. It is easy to see that functor $\Phi:PSet\to{\rm Set}_{*}$,
defined on objects as $\Phi(S)=S_{*}$, and on morphisms

$\Phi(f)(x)=\left\\{\begin{array}[]{cl}f(x),&\mbox{ if }f(x)\mbox{
defined},\\\ ,&\mbox{ if }f(x)\mbox{ is not defined or }x=*,\\\
\end{array}\right.$

will provide an isomorphism of categories $PSet$ and ${\rm Set}_{*}$. This
isomorphism allows us to identify category $PSet$ with ${\rm Set}_{*}$ and we
can consider a partial map as the (total) function preserving point $*$.

###  1.2 Partial action of a monoid

Suppose $M$ be a monoid. The operation in $M$ is denoted by $\cdot_{M}$. Each
monoid $M$ will be considered as a small category that has single object
${\,\rm pt}_{M}$. The set of morphisms of this category is the set $M$. In
particular, the opposite monoid $M^{op}$ is defined. Product of the elements
$\mu_{1},\mu_{2}\in M$ in this monoid is given by
$\mu_{1}\cdot_{M^{op}}\mu_{2}=\mu_{2}\cdot_{M}\mu_{1}$. Partial right action
of a monoid $M$ on the set $S$ is an arbitrary homomorphism $M^{op}\to
PSet(S,S)$. In accordance with the above agreement, a partial action will be
viewed as a homomorphism $M^{op}\to{\rm Set}_{*}(S_{*},S_{*})$. It follows
that it can be regarded as a functor $M^{op}\to{\rm Set}_{*}$.

###  1.3 Trace monoids and state space

Let $E$ be a set, and let $I\subseteq E\times E$ be a irreflective symmetric
relation on $E$. Let $M(E,I)$ be a monoid generated by the set $E$ and given
by the equalities $ab=ba$ for all $(a,b)\in I$. The monoid $M(E,I)$ is called
to be a free partially commutative monoid or trace monoid. $M(E,I)$ is equal
to the quotient monoid $E^{*}/\equiv$ where $E^{*}$ is the monoid of words and
$\equiv$ is the smallest congruence relation containing a pair of $(ab,ba)$
for all pairs $(a,b)\in I$. The space of states is a pair $(S,M(E,I))$, where
$M(E,I)$ is a trace monoid with action on a set $S$.

###  1.4 The state space of elementary Petri nets

For any elementary Petri net ${\,\mathcal{N}}=(P,E,pre,post,s_{0})$, we
consider a trace monoid $M(E,I)$, generated by the set $E$ with the
independence relation

$(a,b)\in I~{}\Leftrightarrow~{}(pre(a)\cup post(a))\cap(pre(b)\cup
post(b))=\emptyset.$

Suppose $a\in E$. For any subset $s\subseteq P$, with properties

  * •

$pre(a)\subseteq s$,

  * •

$(s\setminus pre(a))\cap post(a)=\emptyset$,

we denote $s\cdot a=(s\setminus pre(a))\cup post(a)$. This defines for each
$a\in E$ a partial map $s\mapsto s\cdot a$, of the set $\\{0,1\\}^{P}$ in
$\\{0,1\\}^{P}$. Here the subsets $s\subseteq P$ are considered as their
characteristic functions.

The obtained partial map (corresponding to the relation of the transition from
[4]), can be defined as the set of triples $(s,a,s^{\prime})$ with the
property

$pre(a)\subseteq s~{}\&~{}post(a)\subseteq s^{\prime}~{}\&~{}s\setminus
pre(a)=s^{\prime}\setminus post(a).$

Using the induction on the length of the trace $\mu=a_{1}a_{2}\cdots a_{n}$ by
the formula $s\cdot a_{1}a_{2}\cdots a_{n}=(s\cdot a_{1}a_{2}\cdots
a_{n-1})\cdot a_{n}$, we can define a partial right action of a monoid
$M(E,I)$ on $\\{0,1\\}^{P}$. The resulting state space
$(\\{0,1\\}^{P},M(E,I))$ will be called a state space of elementary Petri
nets.

If we consider only the set of reachable states instead of $\\{0,1\\}^{P}$,
then this partial action will define the space of reachable states of the
elementary Petri net.

##  2 Homology of semicubical sets and state spaces

Recall the definition of semicubical set and its homology groups [5]. We
describe semicubical set corresponding to a state category with the same
homology groups.

###  2.1 Semicubical sets

A semicubical set $X=(X_{n},\partial_{i}^{n,\varepsilon})$ is given as a
sequence sets $X_{n}$, $n\geqslant 0$ with a family of maps
$\partial_{i}^{n,\varepsilon}:X_{n}\to X_{n-1}$, defined by $1\leqslant
i\leqslant n$, $\varepsilon\in\\{0,1\\}$, for which the following diagrams

$\textstyle{X_{n}\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\scriptstyle{\partial_{j}^{n,\beta}}$$\scriptstyle{\partial_{i}^{n,\alpha}}$$\textstyle{X_{n-1}\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\scriptstyle{\partial_{j-1}^{n-1,\beta}}$$\textstyle{X_{n-1}\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\scriptstyle{\partial_{i}^{n-1,\alpha}}$$\textstyle{X_{n-2}}$

are commutative for all $n\geqslant 2,~{}1\leqslant i<j\leqslant n$.

Morphism of semicubical sets $f:X\to\tilde{X}$ is a sequence of maps
$f_{n}:X_{n}\to\tilde{X}_{n}$, $n\geqslant 0$, for which the diagrams

$\textstyle{X_{n}\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\scriptstyle{f_{n}}$$\scriptstyle{\partial_{i}^{n,\varepsilon}}$$\textstyle{\tilde{X}_{n}\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\scriptstyle{\tilde{\partial}_{i}^{n,\varepsilon}}$$\textstyle{X_{n-1}\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\scriptstyle{f_{n-1}}$$\textstyle{\tilde{X}_{n-1}}$

are commutative for all $1\leqslant i\leqslant n$ and
$\varepsilon\in\\{0,1\\}$. If the morphism semicubical sets consist of
inclusions $X_{n}\subseteq\tilde{X}_{n}$ for all $n\geqslant 0$, then $X$ is
called to be a semicubical subset of $\tilde{X}$.

###  2.2 Homology of semicubical sets

Let $X=(X_{n},\partial_{i}^{n,\varepsilon})$ be a semicubical set. Consider
the complex

$0\leftarrow LX_{0}\stackrel{{\scriptstyle
d_{1}}}{{\leftarrow}}LX_{1}\stackrel{{\scriptstyle
d_{2}}}{{\leftarrow}}LX_{2}\stackrel{{\scriptstyle
d_{1}}}{{\leftarrow}}\cdots,$

consisting of free Abelian groups $LX_{n}$, $n\geqslant 0$, generated by the
sets $X_{n}$, and differentials, acting on the basis elements by the formula

$d_{n}(\sigma)=\sum\limits_{i=1}^{n}(-1)^{i}(\partial_{i}^{n,1}(\sigma)-\partial_{i}^{n,0}(\sigma)).$

The homology groups of this complex are called to be homology groups
$H_{n}(X)$ of semicubical set $X$, $n\geqslant 0$.

###### Proposition 2.1

Suppose that $X=X_{1}\cup X_{2}$ is a union of semicubical subsets. Then there
is a long exact sequence of groups

$0\leftarrow H_{0}(X)\leftarrow H_{0}(X_{1})\oplus H_{0}(X_{2})\leftarrow
H_{0}(X_{1}\cap X_{2})\leftarrow\cdots\\\ \cdots\leftarrow H_{n}(X)\leftarrow
H_{n}(X_{1})\oplus H_{n}(X_{2})\leftarrow H_{n}(X_{1}\cap
X_{2})\leftarrow\cdots\\\ $

Proof. Consider the complex $C_{n}(X)=LX_{n}$ with differentials
$d_{n}(\sigma)=\sum\limits_{i=1}^{n}(-1)^{i}(\partial_{i}^{n,1}(\sigma)-\partial_{i}^{n,0}(\sigma))$.
Consider the exact sequence of complexes associated with the homomorphism
$\sigma_{1}\oplus\sigma_{2}\mapsto\sigma_{1}-\sigma_{2}$,

$0\rightarrow C_{n}(X_{1}\cap
X_{2})\stackrel{{\scriptstyle\theta_{n}}}{{\rightarrow}}C_{n}(X_{1})\oplus
C_{n}(X_{2})\stackrel{{\scriptstyle-}}{{\rightarrow}}C_{n}(X_{1}\cup X_{2})\to
0,$

where $\theta_{n}(\sigma)=\sigma\oplus\sigma$ for each $\sigma\in(X_{1}\cap
X_{2})_{n}$. A long exact sequence corresponding to this short exact sequence,
will be required. $\Box$

###  2.3 Homology of state category and semicubical sets

Let $(S,M(E,I))$ be a state space. Consider an arbitrary linear order relation
on $E$. It defines a semicubical set

$Q_{n}(S,E,I)=\\{(s,a_{1},\ldots,a_{n})\in S\times
E^{n}~{}|~{}a_{1}<\cdots<a_{n}~{}\&~{}s\cdot a_{1}\cdots a_{n}\in S\\\
~{}\&~{}(a_{i},a_{j})\in I\mbox{ for all }1\leqslant i<j\leqslant
n\\},~{}~{}n\geqslant 0,$

with boundary operators

$\partial_{i}^{n,\varepsilon}(s,a_{1},\ldots,a_{n})=(s\cdot
a_{i}^{\varepsilon},a_{1},\ldots,a_{i-1},a_{i+1},\ldots,a_{n}),$

for $1\leqslant i\leqslant n$, $\varepsilon\in\\{0,1\\}$. There $a_{i}^{0}=1$
and $a_{i}^{1}=a_{i}$.

Let $(S,M(E,I))$ be a state space. Its state category $K(S)$ is defined as a
small category with objects $s\in S$. Morphisms
$s\stackrel{{\scriptstyle\mu}}{{\to}}t$ are defined as triples of elements
$s,t\in S$ and $\mu\in M(E,I)$ satisfying $s\cdot\mu=t$. Composition is given
by the formula
$(t\stackrel{{\scriptstyle\nu}}{{\to}}u)\circ(s\stackrel{{\scriptstyle\mu}}{{\to}}t)=(s\stackrel{{\scriptstyle\mu\nu}}{{\to}}u)$.

For any elementary Petri net ${\,\mathcal{N}}$, we consider the state space
and its state category. Let $K(S)$ be a full subcategory of the state category
consisting of reachable states. Homology groups $H_{n}({\,\mathcal{N}})$ is
defined as homology groups $H_{n}(K(S))$ of the nerve of the category $K(S)$.
According to [2, Corollary 4], $H_{n}(K(S))\cong H_{n}(Q(S,E,I))$, for all
$n\geqslant 0$. Therefore, the homology groups of elementary Petri nets
$H_{n}({\,\mathcal{N}})$ are isomorphic to the homology of semicubical sets
corresponding to its space of reachable states.

##  3 State category of the pipeline Petri net and its homology groups

In this section, we will explore the state category of pipeline elementary
Petri net and calculate the integral homology groups of this category.

###  3.1 State category of pipeline elementary Petri net

We consider pipeline Petri net ${\mathfrak{P}}_{n}$:

$\textstyle{t_{1}}$ $\scriptstyle{p_{1}}$$\textstyle{t_{2}}$
$\scriptstyle{p_{2}}$$\textstyle{\cdots\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\textstyle{t_{n-1}}$
$\scriptstyle{p_{n-1}}$$\textstyle{t_{n}}$

Represent the state category of this net as union of two partially ordered
sets, each of which has the least and the greatest element.

Let ${\,\mathcal{N}}_{n}$ be the following Petri net

$\scriptstyle{p_{1}}$$\textstyle{t_{2}}$
$\scriptstyle{p_{2}}$$\textstyle{\cdots\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\textstyle{t_{n-1}}$
$\scriptstyle{p_{n-1}}$$\textstyle{t_{n}}$

Denote by ${\mathscr{C}}_{n}$ its the state category. Every partially ordered
set can be considered as a small category ${\mathscr{C}}$ such that for any
objects $x,y\in{\rm Ob\,}{\mathscr{C}}$, the set ${\mathscr{C}}(x,y)$ contains
no more than one element, and if the conjunction of
${\mathscr{C}}(x,y)\not=\emptyset$ and ${\mathscr{C}}(y,x)\not=\emptyset$
implies $x=y$.

Any state of elementary Petri net ${\,\mathcal{N}}_{n}$ is given by an
arbitrary function $f:\\{p_{1},p_{2},\cdots,p_{n-1}\\}\to\\{0,1\\}$. It
follows that the state can be given by a sequence of values
$\varepsilon_{1}\varepsilon_{2}\cdots\varepsilon_{n-1}$, where
$\varepsilon_{i}=f(p_{i})$ of $f$ for $1\leqslant i\leqslant n-1$. Let $S_{1}$
the set of states beginning with $\varepsilon_{1}=1$, and $S_{0}$ the set of
states, beginning with $\varepsilon_{1}=0$. The set of all states will be
equal to $S=S_{1}\sqcup S_{0}$. In general, the elements of $S_{1}$ and
$S_{0}$ are connected by transitions
$10\varepsilon_{3}\cdots\varepsilon_{n-1}\stackrel{{\scriptstyle
t_{2}}}{{\to}}01\varepsilon_{3}\cdots\varepsilon_{n-1}$.

For example, if $n=4$, then $S_{1}$ and $S_{0}$ are linearly ordered sets
corresponding columns of the diagram:

$\textstyle{111\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\scriptstyle{t_{4}}$$\textstyle{011\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\scriptstyle{t_{4}}$$\textstyle{110\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\scriptstyle{t_{3}}$$\textstyle{010\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\scriptstyle{t_{3}}$$\textstyle{101\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\scriptstyle{t_{4}}$$\scriptstyle{t_{2}}$$\textstyle{001\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\scriptstyle{t_{4}}$$\textstyle{100\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\scriptstyle{t_{2}}$$\textstyle{000}$

Transitions between $S_{1}$ and $S_{0}$ is shown by the dashed arrows.

For any state $x=\varepsilon_{1}\cdots\varepsilon_{n-1}$, we introduce a
number $|x|=\varepsilon_{1}\cdot 2^{n-2}+\varepsilon_{2}\cdot
2^{n-3}+\cdots+\varepsilon_{n-1}\cdot 2^{0}$. For example, $|11\cdots
1|=2^{n-1}-1$ and $|00\cdots 0|=0$. If there is a transition
$x\stackrel{{\scriptstyle t_{k}}}{{\to}}y$, then $|x|<|y|$

###### Lemma 3.1

Category ${\mathscr{C}}_{n}$ is a partially ordered set having the greatest
and least elements.

Proof. By induction, we prove that ${\mathscr{C}}_{n}$ is a partially ordered
set with the least element of $11\cdots 1$ and the greatest element $00\cdots
0$. Let it proved for $n-1$. Then ${\mathscr{C}}_{n-1}$ is the partially
ordered set with the least and greatest elements. It is easy to see that the
full subcategory of ${\mathscr{C}}_{n}$ with set of objects $S_{1}$ and
$S_{0}$ are isomorphic to the category ${\mathscr{C}}_{n-1}$. So, they will be
partially ordered sets. We denote these posets by $S_{1}$ and $S_{0}$. In
$S_{1}$, a least element of the induction assumption is $11\cdots 1$, and the
greatest element equals $10\cdots 0$. In $S_{0}$, a least element is $01\cdots
1$, and the greatest equals $00\cdots 0$.

Let $x\mu=y$, $x\in S_{1}$, $y\in S_{0}$. Then $\mu$ contains the letter
$t_{2}$. Therefore, there exist $\mu_{1}$ and $\mu_{2}$, such that
$\mu=\mu_{1}t_{2}\mu_{2}$, and $\mu_{1}$ does not contain $t_{2}$. Now, we
will rearrange the letters contained in $mu_{1}$ with the letter $t_{2}$,
until we do not meet $t_{3}$. For some $\mu_{1}$ and $\mu_{2}$, we obtain the
decomposition $\mu=\mu_{1}t_{3}t_{2}\mu_{2}$.

The following step we will perform until we get the decomposition of the form
$\mu=t_{k}\cdots t_{2}\mu^{\prime}$.

Suppose that $\mu=\mu_{1}tt_{k-1}\cdots t_{2}$, for some $k\geqslant 3$ and
$t\in E$. We describe the action taken by us in each of the following possible
cases:

  1. (i)

$t=t_{k}$ $\Rightarrow$ increase $k$ by $1$,

  2. (ii)

$t=t_{i}$, for some $i>k$ $\Rightarrow$ swap $t$ with $t_{k-1}$, $t_{k-2}$,
$\cdots$, $t_{2}$.

For $i\leqslant k-1$, a decomposition of the form
$\mu=\mu_{1}t_{i}t_{k-1}\cdots t_{2}$ impossible, since an element
$x^{\prime}t_{k-1}\cdots t_{2}$ is defined if and only if $x^{\prime}=11\cdots
10\varepsilon_{k}\cdots\varepsilon_{n-1}$. Such element $x^{\prime}$ can not
be obtained after the action of $t_{i}$ for $2\leqslant i\leqslant k-1$. It
follows that the iteration of the described action will lead to the
decomposition $\mu=t_{k}\cdots t_{2}\mu^{\prime}$.

Now we will prove that $x\mu=x\nu=y\in S$ implies $\mu=\nu$, which implies
that ${\mathscr{C}}_{n}$ is a preordered set.

If $x\mu=x\nu=y\in S_{0}$ and $y\in S_{0}$, then $\mu=\nu$ because of $S_{0}$
is a partially ordered set. The same is true if $x\mu=x\nu=y\in S_{1}$ and
$y\in S_{1}$. There are no $\mu\in M(E,I)$ and $x\in S_{0}$, for which
$x\mu\in S_{1}$.

Now consider the case $x\mu=x\nu=y\in S_{0}$, $x\in S_{1}$. In this case,
there exist decompositions $\mu=t_{k}\cdots t_{2}\mu^{\prime}$ and
$\nu=t_{m}\cdots t_{2}\nu^{\prime}$. Since $xt_{k}t_{k-1}\cdots
t_{2}\mu^{\prime}\in S_{0}$, then $x=11\cdots
10\varepsilon_{k+1}\cdots\varepsilon_{n-1}$, for some $\varepsilon_{k+1}$,
$\cdots,$ $\varepsilon_{n-1}\in\\{0,1\\}$. Hence $m=k$. We get the equality

$xt_{k}\cdots t_{2}\mu^{\prime}=xt_{k}\cdots t_{2}\nu^{\prime}$

Since $xt_{k}\cdots t_{2}\in S_{0}$ and $S_{0}$ – partially ordered set it
follows from this equation $\mu^{\prime}=\nu^{\prime}$. Therefore, $\mu=\nu$
and ${\mathscr{C}}_{n}$ – preordered set. If for some
$x=\varepsilon_{1}\cdots\varepsilon_{n-1}$, $y=\delta_{1}\cdots\delta_{n-1}$,
there exists $\mu\not=1$, which $x\mu=y$, then $|x|>|y|$. Therefore, from
$x\mu=y$ and $y\nu=x$ will follow the $\mu=\nu=1$. Hence otnshenie preorder
will antisymmetric and ${\mathscr{C}}_{n}$ is partially ordered set. $\Box$

###  3.2 The homology groups of pipeline

Let ${\mathfrak{P}}_{n}$ – elementary net pipeline. Delete event $t_{1}$, we
have a network ${\,\mathcal{N}}_{n}$. Remove from ${\mathfrak{P}}_{n}$ event
$t_{2}$, we get the following elementary network:

$\textstyle{t_{1}}$$\scriptstyle{p_{1}}$
$\scriptstyle{p_{2}}$$\textstyle{\cdots\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\textstyle{t_{n-1}}$
$\scriptstyle{p_{n-1}}$$\textstyle{t_{n}}$

We denote it by ${\,\mathcal{N}}^{\prime}_{n}$.

###### Proposition 3.2

Semicubical set $Q({\mathfrak{P}}_{n})$ is the union of subsets semicubical
$Q({\,\mathcal{N}}_{n})\cup Q({\,\mathcal{N}}^{\prime}_{n})$. Intersection of
$Q({\,\mathcal{N}}_{n})\cap Q({\,\mathcal{N}}^{\prime}_{n})$ semicubical be
set with a two components, each of which is isomorphic
$Q({\,\mathcal{N}}_{n-1})$.

Proof. There are $Q_{0}({\mathfrak{P}}_{n})=Q_{0}({\,\mathcal{N}}_{n})\cup
Q_{0}({\,\mathcal{N}}^{\prime}_{n})$, because all of these sets are
$S=\\{0,1\\}^{n-1}$. If $k\geqslant 1$, then since $t_{1}$ and $t_{2}$
dependent, they can not belong a set of mutually independent events
$(e_{1},\cdots,e_{m})$. This means that for every $(s,e_{1},\cdots,e_{m})\in
Q_{m}({\mathfrak{P}}_{n})$ will be a $t_{1}\notin\\{e_{1},\cdots,e_{m}\\}$ or
$t_{2}\notin\\{e_{1},\cdots,e_{m}\\}$, where $(s,e_{1},\cdots,e_{m})\in
Q_{m}({\,\mathcal{N}}_{n})\cup Q_{m}({\,\mathcal{N}}^{\prime}_{n})$. Because
of $(s,e_{1},\cdots,e_{m})\in Q_{m}({\,\mathcal{N}}_{n})$ follows $(s\cdot
e_{i}^{\varepsilon},e_{1},\cdots,e_{i-1},e_{i+1},\cdots,e_{m})\in
Q_{m-1}({\,\mathcal{N}}_{n})$ and from $(s,e_{1},\cdots,e_{m})\in
Q_{m}({\,\mathcal{N}}^{\prime}_{n})$ follows $(s\cdot
e_{i}^{\varepsilon},e_{1},\cdots,e_{i-1},e_{i+1},\cdots,e_{m})\in
Q_{m-1}({\,\mathcal{N}}^{\prime}_{n})$, attachments commute with the boundary
operators. So, $Q({\mathfrak{P}}_{n})$ equals the union its cubic subsets of
$Q({\,\mathcal{N}}_{n})$ and $Q({\,\mathcal{N}}^{\prime}_{n})$. $\Box$

###### Theorem 3.3

$H_{0}({\mathfrak{P}}_{n})=H_{1}({\mathfrak{P}}_{n})={\,\mathbb{Z}}$, and
$H_{k}({\mathfrak{P}}_{n})=0$ at $k>1$.

Proof. By Proposition 3.2, $Q({\mathfrak{P}}_{n})=Q({\,\mathcal{N}}_{n})\cup
Q({\,\mathcal{N}}^{\prime}_{n})$. Apply Proposition 2.1. By Lemma 3.1 category
network states ${\,\mathcal{N}}_{n}$ has initial and terminal object. Let
${\,\mathbb{I}}=\\{0,1\\}$ – partially ordered set consisting of the elements
of $0$ and $1$, with the usual order relation $0<1$. It is easy to see that
the category of network conditions ${\,\mathcal{N}}^{\prime}_{n}$ is
isomorphic to the product of ${\,\mathbb{I}}\times{\mathscr{C}}_{n-2}$.
Therefore it has initial and terminal facilities too. From this follow
isomorphisms
$H_{k}(Q({\,\mathcal{N}}_{n}))=H_{k}(Q({\,\mathcal{N}}^{\prime}_{n}))=0$, for
$k>0$. The exact sequence 2.1 leads to the isomorphisms
$H_{k}(Q({\mathfrak{P}}_{n}))\cong H_{k-1}(Q({\,\mathcal{N}}_{n})\cap
Q({\,\mathcal{N}}^{\prime}_{n}))$, for $k\geqslant 2$. Of Proposition 3.2
follows $Q({\,\mathcal{N}}_{n})\cap Q({\,\mathcal{N}}^{\prime}_{n})\cong
Q({\,\mathcal{N}}_{n-1})\sqcup Q({\,\mathcal{N}}_{n-1})$. We get
$H_{k}(Q({\mathfrak{P}}_{n}))\cong H_{k-1}(Q({\,\mathcal{N}}_{n-1}))\oplus
H_{k-1}(Q({\,\mathcal{N}}_{n-1}))=0$, where $k\geqslant 2$. Consequently,
$H_{k}({\mathfrak{P}}_{n})=0$ for $k\geqslant 2$. Of Proposition 2.1 also
obtain accurate sequence

$0\leftarrow H_{0}(Q({\mathfrak{P}}_{n}))\leftarrow
H_{0}(Q({\,\mathcal{N}}_{n}))\oplus H_{0}(Q({\,\mathcal{N}}^{\prime}_{n}))\\\
\leftarrow H_{0}(Q({\,\mathcal{N}}_{n})\cap
Q({\,\mathcal{N}}^{\prime}_{n}))\leftarrow
H_{1}(Q({\mathfrak{P}}_{n}))\leftarrow 0.$

Group $H_{0}$ freely generated connected components semicubical sets.
Homomorphism

$H_{0}(\theta):H_{0}(Q({\,\mathcal{N}}_{n})\cap
Q({\,\mathcal{N}}^{\prime}_{n}))\to H_{0}(Q({\,\mathcal{N}}_{n}))\oplus
H_{0}(Q({\,\mathcal{N}}^{\prime}_{n}))$

induced chain homomorphism

$\theta_{k}:C_{k}(Q({\,\mathcal{N}}_{n})\cap
Q({\,\mathcal{N}}^{\prime}_{n}))\to C_{k}(Q({\,\mathcal{N}}_{n}))\oplus
C_{k}(Q({\,\mathcal{N}}^{\prime}_{n})),$

defined by $\sigma\in(Q({\,\mathcal{N}}_{n})\cap
Q({\,\mathcal{N}}^{\prime}_{n}))_{k}$ by the formula
$\theta_{k}(\sigma)=\sigma\oplus\sigma$. This implies that $H_{0}(\theta)$
acts on the homology classes of the formula
$H_{0}(\theta)(cls(\sigma))=cls(\sigma)\oplus cls(\sigma)$. Because these
classes homology are connected components semicubical sets, then
$H_{0}(\theta)$ is a homomorphism
${\,\mathbb{Z}}\oplus{\,\mathbb{Z}}\to{\,\mathbb{Z}}\oplus{\,\mathbb{Z}}$,
given by the matrix $\left(\begin{array}[]{cc}1&1\\\ 1&1\end{array}\right)$.
But $H_{1}({\mathfrak{P}}_{n})$ is isomorphic to kernel of the homomorphism
$H_{0}(\theta)$, and $H_{0}({\mathfrak{P}}_{n})$ – cokernel. Hence, for
example, with a cast of this matrix to Smith normal form, we obtain
$H_{0}({\mathfrak{P}}_{n})=H_{1}({\mathfrak{P}}_{n})={\,\mathbb{Z}}$. $\Box$

##  4 Directed homology

We present supporting information on the homology groups Goubault semicubical
sets. We study the properties of directed homology category states and their
interpretation in dimension $0$. Calculate the direction of homology Petri net
pipeline.

###  4.1 Homology groups of Goubault

Let $X=(X_{n},\partial^{n,\varepsilon}_{i})$ semicubical set. For all
$\varepsilon\in\\{0,1\\}$, consider the chain complex of abelian groups
$C_{n}(X)=L(X_{n})$ with the differentials

$0\leftarrow C_{0}(X)\stackrel{{\scriptstyle
d_{1}^{\varepsilon}}}{{\leftarrow}}C_{1}(X)\leftarrow\cdots
C_{n-1}(X)\stackrel{{\scriptstyle
d_{n}^{\varepsilon}}}{{\leftarrow}}C_{n}(X)\leftarrow\cdots,$

where
$d_{n}^{\varepsilon}(\sigma)=\sum\limits_{i=1}^{n}(-1)^{i}\partial_{i}^{n,\varepsilon}(\sigma)$.

Homology groups $H_{n}^{\varepsilon}(X)$ of the complex called homology groups
Goubault semicubical set $X$. They were studied in [6].

Groups $H_{n}^{0}(X)$ are called initial, and $H_{n}^{1}(X)$ – final homology
groups Goubault.

###### Proposition 4.1

Let $X=X_{1}\cup X_{2}$ – union semicubical subsets $X_{1}\subseteq X$ and
$X_{2}\subseteq X$. Then for every $\varepsilon\in\\{0,1\\}$ there is a long
exact sequence

$0\leftarrow H_{0}^{\varepsilon}(X)\leftarrow H_{0}^{\varepsilon}(X_{1})\oplus
H_{0}^{\varepsilon}(X_{2})\leftarrow H_{0}^{\varepsilon}(X_{1}\cap
X_{2})\leftarrow\cdots\\\ \cdots\leftarrow H_{n}^{\varepsilon}(X)\leftarrow
H_{n}^{\varepsilon}(X_{1})\oplus H_{n}^{\varepsilon}(X_{2})\leftarrow
H_{n}^{\varepsilon}(X_{1}\cap X_{2})\leftarrow\cdots\\\ $

Proof repeats almost verbatim the proof of Proposition 2.1.

###  4.2 Directed homology groups of the state category

Homology groups of a small category ${\mathscr{C}}$ with coefficients in the
functor $F:{\mathscr{C}}\rightarrow{\rm Ab}$ is defined as values
$\underrightarrow{\lim}_{n}^{{\mathscr{C}}}F$ of left derived functors of
$\underrightarrow{\lim}^{{\mathscr{C}}}:{\rm Ab}^{{\mathscr{C}}}\to{\rm Ab}$
on $F$.

Let $(S,M(E,I))$ be a state space, and $K(S)$ be its the state category.
Consider the functors $\Delta^{0}{\,\mathbb{Z}}:K(S)\to{\rm Ab}$ and
$\Delta^{1}{\,\mathbb{Z}}:K(S)^{op}\to{\rm Ab}$, taking on the objects the
constant values
$\Delta^{0}{\,\mathbb{Z}}(s)=\Delta^{1}{\,\mathbb{Z}}(s)={\,\mathbb{Z}}$. And
on the morphisms defined by the formulas

$\Delta^{\varepsilon}{\,\mathbb{Z}}(s\stackrel{{\scriptstyle\mu}}{{\to}}s^{\prime})=\left\\{\begin{array}[]{cl}1_{s},&\mbox{
}\mu=1,\\\ 0,&\mbox{ }\mu\not=1,\\\ \end{array}\right.$

###### Definition 4.1

Directed homology groups of the state space defined by the formulas

$H_{n}^{0}(S,M(E,I))=_{def}\underrightarrow{\lim}_{n}^{K(S)}\Delta^{0}{\,\mathbb{Z}},\quad
H_{n}^{1}(S,M(E,I))=_{def}\underrightarrow{\lim}_{n}^{K(S)^{op}}\Delta^{1}{\,\mathbb{Z}}.$

For any elementary Petri nets ${\,\mathcal{N}}$ its homology groups
$H_{n}^{\varepsilon}({\,\mathcal{N}})$ defined as the homology groups of its
state space.

###### Proposition 4.2

For elementary Petri nets ${\,\mathcal{N}}_{n}$ obtained from the pipeline
Petri nets ${\mathfrak{P}}_{n}$ by deleting the transition $t_{1}$, homology
groups equal

$H_{k}^{\varepsilon}({\,\mathcal{N}}_{n})=\left\\{\begin{array}[]{cc}{\,\mathbb{Z}},&\mbox{
if }k=0\\\ 0,&\mbox{ if }k>0.\end{array}\right.$

Proof. By Lemma 3.1, the state category ${\mathscr{C}}_{n}$ of Petri net
${\,\mathcal{N}}_{n}$ has an initial and terminal objects. If a small category
has a terminal object, then the colimit functor on the category equals to the
value of this functor on the terminal object. Hence
$H_{k}^{\varepsilon}({\,\mathcal{N}}_{n})=\underrightarrow{\lim}_{k}\Delta^{\varepsilon}{\,\mathbb{Z}}=0$
for $k>0$ and
$H_{0}^{\varepsilon}({\,\mathcal{N}}_{n})=\underrightarrow{\lim}\Delta^{\varepsilon}{\,\mathbb{Z}}={\,\mathbb{Z}}$.
$\Box$

According to [2, Corollary 5] for any linear order on $E$, there are
isomorphisms

$\underrightarrow{\lim}_{n}^{K(S)}\Delta^{0}{\,\mathbb{Z}}\cong
H_{n}^{0}(Q(S,E,I)),\quad\underrightarrow{\lim}_{n}^{K(S)^{op}}\Delta^{1}{\,\mathbb{Z}}\cong
H_{n}^{1}(Q(S,E,I)).$ (1)

Hence we get the following interpretation of directed homology for a state
space in dimension $0$. State $s$ is called to be a deadlock, if there is no
$a\in E$ satisfying $s\cdot a\in S$. It is called a sender, if there is no
such a pair $s^{\prime}\in S$ and $a\in E$ for which $s^{\prime}\cdot a=s$. In
the state category, deadlock is an object $s$ which has not a morphisms
$\alpha\not=1_{s}$ with $dom\alpha=s$. A sender $s$ has not $\alpha\not=1_{s}$
such that $cod\alpha=s$.

###### Proposition 4.3

The group $H_{0}^{0}(S,M(E,I))$ is isomorphic to the free abelian group
generated by deadlocks, and $H_{0}^{1}(S,M(E,I))$ generated by senders.

Proof. It follows from [2, Corollary 5] that $H_{0}^{0}(S,M(E,I))$ is
isomorphic to the cokernel of $d_{1}^{0}:LQ_{1}(S,E,I)\to LQ_{0}(S,E,I)$
defined as $d_{1}^{0}(s,a)=s$. Cokernel is generated by the set obtained by
the identification with $0$ of all $s\in S$ for which there are $a\in E$
satisfying $s\cdot a\in S$. It is removed from the set $S$ all objects which
are no deadlocks. The remaining set generates the cokernel. It is proved
similarly that $H_{0}^{1}(S,M(E,I))$ generated by senders. $\Box$

###### Example 4.2

State category of pipeline elementary Petri nets has no senders or deadlocks,
hence $H_{0}^{0}({\mathfrak{P}}_{n})=H_{0}^{1}({\mathfrak{P}}_{n})=0$.

###  4.3 Directed homology groups of pipeline

According to the above example, groups $H_{0}^{0}({\mathfrak{P}}_{n})$ and
$H_{0}^{1}({\mathfrak{P}}_{n})$ equal zero. The following statement shows that
$H_{k}^{0}({\mathfrak{P}}_{n})$ and $H_{k}^{1}({\mathfrak{P}}_{n})$ equal $0$
for all $k\geqslant 0$.

###### Theorem 4.4

$H_{k}^{\varepsilon}({\mathfrak{P}}_{n})=0$, for all $n\geqslant 2$,
$k\geqslant 0$ and $\varepsilon\in\\{0,1\\}$.

Proof. Above, we found that the semicubical set $Q({\mathfrak{P}}_{n})$ is
equal to union of semicubical sets $Q({\,\mathcal{N}}_{n})$ and
$Q({\,\mathcal{N}}^{\prime}_{n})$. The state categories of
${\,\mathcal{N}}_{n}$ and ${\,\mathcal{N}}^{\prime}_{n}$ have the greatest and
the least elements. This means that
$H_{k}^{\varepsilon}({\,\mathcal{N}}_{n})=H_{k}^{\varepsilon}({\,\mathcal{N}}^{\prime}_{n})=0$
for $k>0$, and
$H_{0}^{\varepsilon}({\,\mathcal{N}}_{n})=H_{k}^{\varepsilon}({\,\mathcal{N}}^{\prime}_{n})={\,\mathbb{Z}}$.
By the suggestion of 3.2, $Q({\,\mathcal{N}}_{n})\cap
Q({\,\mathcal{N}}^{\prime}_{n})\cong Q({\,\mathcal{N}}_{n-1})\sqcup
Q({\,\mathcal{N}}_{n-1})$. It follows that the exact sequence of the fragment
suggests 4.1

$\leftarrow H_{k-1}^{\varepsilon}(Q({\,\mathcal{N}}_{n})\cap
Q({\,\mathcal{N}}^{\prime}_{n}))\leftarrow
H_{k}^{\varepsilon}(Q({\mathfrak{P}}_{n}))\leftarrow
H_{k}^{\varepsilon}(Q({\,\mathcal{N}}_{n}))\oplus
H_{k}^{\varepsilon}(Q({\,\mathcal{N}}^{\prime}_{n}))\leftarrow$

leads to the equations $H_{k}^{\varepsilon}(Q({\mathfrak{P}}_{n}))=0$ for
$k\geqslant 2$. In addition, there is an exact sequence

$0\leftarrow H_{0}^{\varepsilon}(Q({\mathfrak{P}}_{n}))\leftarrow
H_{0}^{\varepsilon}(Q({\,\mathcal{N}}_{n}))\oplus
H_{0}^{\varepsilon}(Q({\,\mathcal{N}}^{\prime}_{n}))\stackrel{{\scriptstyle
H_{0}^{\varepsilon}(\theta)}}{{\leftarrow}}\\\
H_{0}^{\varepsilon}(Q({\,\mathcal{N}}_{n})\cap
Q({\,\mathcal{N}}^{\prime}_{n}))\leftarrow
H_{1}^{\varepsilon}(Q({\mathfrak{P}}_{n}))\leftarrow 0$

By example 4.2 and isomorphisms (1), we have
$H_{0}^{\varepsilon}(Q({\mathfrak{P}}_{n}))=0$. We obtain an epimorphism
${\,\mathbb{Z}}\oplus{\,\mathbb{Z}}\stackrel{{\scriptstyle
H^{\varepsilon}_{0}(\theta)}}{{\to}}{\,\mathbb{Z}}\oplus{\,\mathbb{Z}}$. But
${\,\mathbb{Z}}\oplus{\,\mathbb{Z}}$ is projective object in the category of
Abelian groups. It follows that there exists a homomorphism $\gamma$ such that
$H_{0}^{\varepsilon}(\theta)\circ\gamma=1_{{\,\mathbb{Z}}\oplus{\,\mathbb{Z}}}$.
This means that the determinant of $H^{\varepsilon}_{0}(\theta)$ is invertible
where the kernel $H_{0}^{\varepsilon}(\theta)$ equally zero. Consequently,
$H_{1}^{\varepsilon}(Q({\mathfrak{P}}_{n}))=0$. We have proved that
$H_{k}^{\varepsilon}(Q({\mathfrak{P}}_{n}))=0$ for all $k\geqslant 0$.
Isomorphisms (1) give $H_{k}^{\varepsilon}({\mathfrak{P}}_{n})\cong
H_{k}^{\varepsilon}(Q({\mathfrak{P}}_{n}))$. $\Box$

## References

  * [1] Husainov A.,  “On the homology of small categories and asynchronous transition systems”,  Homology Homotopy Appl., 6:1 (2004), 439–471; http://www.rmi.acnet.ge/hha
  * [2] Husainov A.A.,  “The Homology of Partial Monoid Actions and Petri Nets”,  Appl. Categor. Struct., 2012. DOI: 10.1007/s10485-012-9280-9
  * [3] Husainov A.A., Bushmeleva E.S.,  “Homology of Asynchronous Systems”,  Actual Problems of Mathematics, Physics, and Computer Science in University and School: Materials International Scientific Conference March 23, 2012, Komsomolsk-on-Amur, Humanities and Education Amur State University, 2012. 24–31.
  * [4] Winskel G., Nielsen M.  Models for Concurrency. Handbook of Logic in Computer Science, Vol.IV, ed. Abramsky, Gabbay and Maibaum. Oxford University Press, 1995\. P.1–148.
  * [5] Husainov A.A.,  “Homology groups of semicubical sets”,  Sib. Math. Journal., 49:1 (2008), 224–237.
  * [6] Goubault E.,  The Geometry of Concurrency,  Thesis Doct. Phylosophy (Mathematics), Ecole Normale Supérieure 1995, 349 p.
  * [7] Gaucher P.,  “About the globular homology of higher dimensional automata”,  Topol. Geom. Differ., 43:2 (2002), 107–156.


# Scaling Synthetic Data Creation with 1,000,000,000 Personas

Xin Chan, Xiaoyang Wang, Dian Yu, Haitao Mi, Dong Yu
Tencent AI Lab Seattle
https://github.com/tencent-ailab/persona-hub

###### Abstract

We propose a novel persona-driven data synthesis methodology that leverages
various perspectives within a large language model (LLM) to create diverse
synthetic data. To fully exploit this methodology at scale, we introduce
Persona Hub – a collection of 1 billion diverse personas automatically curated
from web data. These 1 billion personas ($\sim$13% of the world’s total
population), acting as distributed carriers of world knowledge, can tap into
almost every perspective encapsulated within the LLM, thereby facilitating the
creation of diverse synthetic data at scale for various scenarios. By
showcasing Persona Hub’s use cases in synthesizing high-quality mathematical
and logical reasoning problems, instructions (i.e., user prompts), knowledge-
rich texts, game NPCs and tools (functions) at scale, we demonstrate persona-
driven data synthesis is versatile, scalable, flexible, and easy to use,
potentially driving a paradigm shift in synthetic data creation and
applications in practice, which may have a profound impact on LLM research and
development.

DISCLAIMER: Persona Hub can facilitate synthetic data creation at a billion-
scale to simulate diverse inputs (i.e., use cases) from a wide variety of
real-world users. If this data is used as input to query a target LLM to
obtain its outputs at scale, there is a high risk that the LLM’s knowledge,
intelligence and capabilities will be dumped and easily replicated, thereby
challenging the leading position of the most powerful LLMs (e.g., our approach
allows a 7B LLM to achieve 65% on MATH, matching the performance of
gpt-4-turbo-preview). This tech report is for research purposes only. It is
crucial to avoid misuse and ensure ethical and responsible application. We
discuss its broad impact and potential concerns in detail in Section 5.

Figure 1: Personas can work with a wide range of data synthesis prompts (e.g.,
create a math problem or a user prompt) to guide an LLM to synthesize data
with corresponding perspectives. The 1 billion personas in Persona Hub can
facilitate synthetic data creation for various data synthesis scenarios at a
billion scale.

##  1 Introduction

As synthetic data (Bauer et al., 2024; Liu et al., 2024), typically referring
to data generated by models or algorithms rather than directly by humans,
becomes increasingly valued (Li et al., 2023b) for training large language
models (LLMs), there is a growing interest in data synthesis using LLMs: by
simply specifying a data synthesis prompt, an LLM is expected to produce
desirable synthetic data.

In practice, however, it is non-trivial to create synthetic data at scale:
while we can easily scale up the quantity of synthetic data, it is difficult
to ensure its diversity scales up as well. Without considering
sampling111Sampling is orthogonal to this work. The diversity it introduces
when solely used for data synthesis is usually limited., an LLM can only
produce 1 instance given a data synthesis prompt. Therefore, to create diverse
synthetic data at scale (e.g., 1 billion diverse math problems), a large
number of diverse prompts are needed.

Previous research tends to diversify the data synthesis prompt through the
following two paradigms, but unfortunately, neither can practically achieve
scalable synthetic data creation:

  * •

Instance-driven: This approach diversifies the data synthesis prompt by
leveraging a seed corpus (i.e., creating new instances based on the instances
in the seed corpus). Representative studies include Wang et al. (2022) and Yu
et al. (2023). However, under this paradigm, the diversity of the synthesized
data mainly comes from the seed instances, making it difficult to truly extend
beyond the seed corpus. Given the limited size of a seed corpus in most
practical scenarios, it is challenging for this paradigm to scale up the
creation of synthetic data.

  * •

Key-point-driven: This approach diversifies the data synthesis prompt with a
curated comprehensive list of key points (or concepts) that can be a topic, a
subject, or any knowledge we expect synthetic data to encompass.
Representative studies include Li et al. (2024b) and Huang et al. (2024).
However, this methodology also faces difficulties in scaling synthetic data
creation: it is practically prohibitive to curate a comprehensive list by
enumerating all key points across different levels of granularity, unless
limited to a narrow and specific domain (e.g., mathematics).

To practically achieve diverse synthetic data creation at scale, we propose a
novel persona-driven data synthesis methodology. This is inspired by the
observation that simply adding a persona to a data synthesis prompt can steer
the LLM towards the corresponding perspective to create distinctive synthetic
data, as shown in Figure 1. Since almost any LLM use case can be associated
with a specific persona, we can create all-encompassing synthetic data at
scale as long as we construct a comprehensive persona collection.

Figure 2: From a compression perspective (Delétang et al., 2023; Ge et al.,
2024), Persona Hub ($\sim 10^{10}$ tokens) can be seen as the compressed form
of world knowledge (public web text for training LLMs, $\sim 10^{14}$ tokens)
into distributed carriers. On the other hand, the public web text can be seen
as the decompressed content created by these personas with their knowledge and
experiences.

Fortunately, personas are very easy to scale up. From massive web data, we
automatically construct Persona Hub — a persona collection containing 1
billion diverse personas ($\sim$13% of the world’s total population). As
Figure 2 shows, these 1 billion personas can be regarded as distributed
carriers of world knowledge, and each individual can be associated with their
unique knowledge, experience, interest, personality and profession; thus, they
can tap into almost every perspective encapsulated within the LLM to create
diverse synthetic data at scale, without being limited by the size of a seed
corpus. Moreover, in contrast to key points that typically work with specific
data synthesis prompts, personas can be combined with almost any data
synthesis prompt, benefiting from an LLM’s strong roleplay ability (Shanahan
et al., 2023; Li et al., 2023a; Choi & Li, 2024; Wang et al., 2024), making
them generally applicable to a variety of data synthesis scenarios.

We showcase Persona Hub’s use cases in large-scale creation of math and
logical reasoning problems, instructions (i.e., user prompts), broad-coverage
knowledge-rich texts, game NPCs, and tool (function) development. We
demonstrate that persona-driven data synthesis is versatile, scalable,
flexible, and easy to use, potentially driving a paradigm shift in synthetic
data creation and applications in practice, which may have a profound impact
on LLM research and development.

To facilitate research in persona-driven data synthesis, we initially release
200,000 personas from Persona Hub and following synthetic data samples we
created with various personas, including:

  * •

50,000 math problems

  * •

50,000 instructions

  * •

10,000 game NPCs

  * •

50,000 logical reasoning problems

  * •

10,000 knowledge-rich texts

  * •

5,000 tools (functions)

We are open to releasing more data when we can better assess the potential
risks and concerns, which will be discussed in detail in Section 5.

Note: Our proposed methodology is applicable to almost any popular LLM222We
mainly use publicly available LLMs such as GPT-4 (Achiam et al., 2023),
Llama-3 and Qwen (Team, 2024; qwe, 2024) in our experiments.. The prompts
shown in the figures throughout this paper are not exactly the prompt strings
we used in our experiments; instead, they are simplified to fit the space and
better illustrate the concepts. Interested readers can easily verify our
methodology using the persona samples we have released. It is also worth
noting that the main focus of this work is on creating new synthetic data,
unlike much previous research that focuses on generating synthetic outputs for
specific inputs (e.g., a math problem). Therefore, we use the terms “create”
and “synthesize” interchangeably throughout the paper.

##  2 Persona Hub

We propose two scalable approaches to derive diverse personas to construct
Persona Hub from massive web data: Text-to-Persona and Persona-to-Persona.

Figure 3: The Text-to-Persona approach: it can use any text as input to obtain
corresponding personas just by prompting the LLM “Who is likely to
[read|write|like|dislike|…] the text?”

###  2.1 Text-to-Persona

A person with specific professional experiences and cultural backgrounds will
have unique interests in reading and writing. Therefore, from a specific text,
we can infer a specific persona who is likely to [read|write|like|dislike|…]
the text. Given that text data on the web is virtually unlimited and all-
encompassing, we can obtain a wide-ranging collection of personas simply by
prompting an LLM with these web texts, as shown in Figure 3.

There are many formats (e.g., plain text or structured text) to represent a
persona, which can be controlled within the prompt. The granularity of an
output persona description can also be adjusted through the prompt. For
example, in the first case, a coarse-grained persona might be “a computer
scientist”, whereas the fine-grained persona is “a machine learning researcher
focused on neural network architectures and attention mechanisms”. In our
practice, we ask the LLM (in the prompt) to output persona descriptions as
specifically as possible. Besides specifying the granularity of persona
descriptions in the prompt, input texts can also influence the granularity of
persona descriptions. As shown in Figure 4, if an input text (e.g., from a
mathematical textbook or an academic paper about superconductivity) contains
many detailed elements, the resulting persona description will also be
specific and fine-grained. Therefore, by applying the Text-to-Persona approach
to massive web text data, we can obtain billions (or even trillions) of
diverse personas, encompassing a wide range of aspects across different
granularities.

Figure 4: Persona descriptions will be fine-grained if input texts involve
many detailed elements.

###  2.2 Persona-to-Persona

As discussed above, Text-to-Persona is a highly scalable method that can
synthesize personas covering almost every aspect. However, it may still miss
some personas that have low visibility on the web and thus are less likely to
obtain via Text-to-Persona, such as a child, a beggar, or a behind-the-scenes
crew member of a movie. To supplement the personas that Text-to-Persona might
hardly reach, we propose Persona-to-Persona, which derives personas with
interpersonal relationships from those obtained through Text-to-Persona.

Figure 5: Persona-to-Persona obtains diverse personas via interpersonal
relationships, which can be easily achieved by prompting the LLM “Who is in
close relationship with the given persona?”

As shown in Figure 5, the persona about “a child” can be derived from the
persona of a nurse at a children’s hospital (patient-caregiver relationship).
Similarly, “a beggar” can be derived from the persona of a shelter worker
(assistance relationship), and “a behind-the-scenes movie crew member” can be
derived from the persona of the movie’s lead actor (co-worker relationship).
According to the six degrees of separation theory (Travers & Milgram, 1977),
we perform six iterations of persona relationship expansion for each persona
obtained through Text-to-Persona, thereby enriching our persona collection
even further.

###  2.3 Deduplication

We first run Text-to-Persona on the RedPajama v2 dataset (Computer, 2023) and
then perform Persona-to-Persona, as described in Sections 2.1 and 2.2. After
obtaining billions of personas, it is inevitable that some of the personas
will be identical or extremely similar. To ensure the diversity of Persona
Hub, we deduplicate these personas in two ways:

##### MinHash-based Deduplication

We use MinHash (Broder, 1997) to deduplicate based on the n-gram features of
persona descriptions. Since persona descriptions are usually just 1-2
sentences, much shorter than a document, we simply used 1-gram and a signature
size of 128 for MinHash deduplication. We deduplicate at the similarity
threshold of 0.9.

##### Embedding-based Deduplication

After deduplication based on surface forms (i.e., MinHash with n-gram
features), we also adopt embedding-based deduplication. We use a text
embedding model (e.g., the text-embedding-3-small model from OpenAI) to
compute an embedding for each persona, and then filter out personas with a
cosine semantic similarity greater than 0.9.

Note that although we select 0.9 as the threshold here, we can flexibly adjust
it according to specific needs for further deduplication. For instance, when
the requirement for the number of instances is not high (e.g., only needing 1
million instances) but the demand for diversity is high, we can further apply
a stricter deduplication standard (e.g., discarding personas with a similarity
greater than 0.5).

After deduplication and using simple heuristic methods to filter out low-
quality persona descriptions, we have harvested a total of 1,015,863,523
personas, finally forming our Persona Hub.

##  3 Persona-driven Synthetic Data Creation

Our proposed persona-driven data synthesis approach is straightforward and
effective, which involves integrating a persona into the appropriate position
in a data synthesis prompt. Simple as it appears, it can significantly
influence the LLM to adopt the persona’s perspective to create synthetic data.
Driven by the 1 billion personas in Persona Hub, this approach can easily
create diverse synthetic data at a billion scale.

Figure 6: 0-shot, few-shot and persona-enhanced few-shot prompting methods.

Just as we can use either zero-shot or few-shot methods to prompt an LLM, the
persona-driven methodology is also flexible and compatible with various forms
of prompts to create synthetic data. As shown in Figure 6, we propose three
persona-driven data synthesis prompting methods:

  * •

Zero-shot prompting does not leverage any existing examples (i.e.,
demonstrations), thereby fully exploiting the model’s creativity without being
constrained by specific examples.

  * •

Few-shot prompting can better ensure that the synthesized data meets the
requirements by providing some demonstrations.

  * •

Persona-enhanced few-shot prompting is more effective in enhancing the LLM’s
persona-driven data synthesis capabilities. However, its drawback is that it
requires deriving the corresponding persona for each demonstration in the few-
shot prompt beforehand.

##  4 Use Cases

We demonstrate the use cases of Persona Hub in various data synthesis
scenarios, including the large-scale creation of math and logical reasoning
problems, instructions (i.e., user prompts), knowledge-rich texts, game NPCs,
and tool (function) development.

As mentioned earlier, the persona-driven approach is general and versatile,
making it easily adaptable to different data synthesis scenarios simply by
adjusting the data synthesis prompt. Therefore, we will provide a detailed
technical discussion only for math problem synthesis (Section 4.1) and skip
the detailed discussion for other use cases.

###  4.1 Math Problems

####  4.1.1 Demonstrations

As the initial example (Figure 1) shows, when prompting an LLM to create a
math problem, adding a persona leads the LLM to create math problems related
to that persona. The example in Figure 7(left) further confirms this: when
presented with a linguist persona, the LLM will create a math problem in the
context of computational linguistics. Moreover, adding a persona does not
hinder the flexibility of the prompt – we can still easily specify the focus
(Figure 7(middle)) or difficulty (Figure 7(right)) of our desired math problem
in the prompt.

Figure 7: A linguist persona with different math problem creation prompts that
specify the focus (e.g., geometry) or the difficulty (e.g., Olympiad-level)
Figure 8: Examples of math problems created with personas of professionals
related to the field of mathematics. They tend to be more challenging than
those created with general personas because they usually require a deeper and
more fine-grained understanding of advanced mathematical knowledge and skills.

The examples in Figure 7 demonstrate the use of general personas to create
math problems. We can certainly employ professionals related to mathematics to
create math problems as well. As shown in Figure 8, personas of math
professionals333We can easily harvest a large number of such personas when
running Text-to-Persona (Section 2.1) on public web texts, particularly when
processing texts in the field of mathematics. often mention more advanced and
granular mathematics knowledge and skills (as discussed earlier in Section 2.1
and Figure 4), which in turn allows the created math problems to cover these
mathematical concepts, making them more challenging.

####  4.1.2 Evaluation

##### Data

We select444Technically, if we use the full version of Persona Hub, we can
obtain 1 billion math problems synthesized by an LLM. However, due to the cost
of GPT-4 APIs, we limit our scaling to 1.09M personas for this experiment.
1.09 million personas from Persona Hub and employ the 0-shot prompting method
using GPT-4 to create math problems with these personas, which does not
leverage any instances from benchmarks like MATH (Hendrycks et al., 2021)
during the creation of math problems. This approach allowed us to synthesize
1.09M math problems. Since this work focuses on creating new synthetic data
rather than synthesizing solutions, we simply used gpt-4o
(assistant555Assistant system message in OpenAI API doc: “You are a helpful
assistant.”) to generate solutions to the created problems. Among these 1.09M
math problems, we randomly hold out 20k as a synthetic test set to facilitate
evaluation. The remaining 1.07M problems are used for training.

##### Test sets

We use the following two test sets for evaluation:

  * •

Synthetic Test Set (In-distribution): Since the set of the held-out 20K
problems is produced in the same way as the 1.07M training instances, it can
be considered an in-distribution test set. To ensure the accuracy of the
answers in this test set for increasing the reliability of the evaluation, we
additionally generate solutions using gpt-4o (PoT666Program of thought
prompting (Chen et al., 2022)) and gpt-4-turbo (assistant) in addition to the
solution generated by gpt-4o (assistant). We retain only the test instances
where at least two solutions are consistent. The remaining test set consists
of 11.6K test instances.

  * •

MATH (Out-of-distribution): The most widely recognized benchmark for testing
the mathematical reasoning ability of LLMs. Its test set contains 5,000
competitive-level math problems with reference answers. Since we do not use
any instances from the MATH dataset for data synthesis or training, we regard
the MATH test set as an out-of-distribution test set.

##### Equality Checking

We follow the same evaluation protocol as
OpenAI777https://github.com/openai/simple-evals to check answer equality on
the MATH benchmark. For the synthetic test set, we use a similar method,
except we use Llama-3-70B-Instruct instead of gpt-4-turbo-preview as the
equality checker.

We simply fine-tune the latest open-sourced 7B LLM – Qwen2-7B (qwe, 2024) with
our synthesized 1.07 million math problems and evaluate its greedy decoding
outputs on the above two test sets.

Model | Model Size | Accuracy (%)
---|---|---
Open-sourced LLMs
DeepSeek LLM 67B Chat (Bi et al., 2024) | 67B | 53.2
Phi-3-Mini-4K-Instruct (Abdin et al., 2024) | 3.8B | 68.3
Yi-1.5-34B-Chat (Young et al., 2024) | 34B | 70.4
Qwen1.5-72B-Chat (Team, 2024) | 72B | 60.7
Qwen1.5-110B-Chat (Team, 2024) | 110B | 73.0
Qwen2-7B-Instruct (qwe, 2024) | 7B | 72.1
Qwen2-72B-Instruct (qwe, 2024) | 72B | 77.2
Llama-3-8B-Instruct | 8B | 39.8
Llama-3-70B-Instruct | 70B | 63.5
GPT-4
gpt-4-turbo-2024-04-09 | ? | 88.1
gpt-4o-2024-05-13 | ? | 91.2
This work
Qwen2-7B (fine-tuned w/ the 1.07M synthesized instances) | 7B | 79.4
Table 1: In-distribution evaluation results on the 11.6K synthetic test instances. Model | Model Size | Accuracy (%)
---|---|---
State-of-the-art LLMs
gpt-4o-2024-05-13 | ? | 76.6
gpt-4-turbo-2024-04-09 | ? | $73.4$
gpt-4-turbo-0125-preview | ? | $64.5$
gpt-4-turbo-1106-preview | ? | $64.3$
gpt-4 | ? |  $52.6$∗
Claude 3.5 Sonnet | ? |  $71.1$∗
Claude 3 Opus | ? | $63.8$
Gemini Pro 1.5 (May 2024) | ? |  $67.7$∗
Gemini Ultra | ? |  $53.2$∗
DeepSeek-Coder-V2-Instruct (Zhu et al., 2024) | 236B/21B |  $75.7$∗
Llama-3-70B-Instruct | 70B | $52.8$
Qwen2-72B-Instruct | 72B |  $59.7$∗
Qwen2-7B-Instruct | 7B |  $49.6$∗
This work
Qwen2-7B (fine-tuned w/ the 1.07M synthesized instances) | 7B | 64.9
Table 2: Out-of-distribution evaluation on MATH. Results marked with an
asterisk (*) may not use the OpenAI’s evaluation method. The model fine-tuned
with our synthesized 1.07M math problems achieves 64.9% on MATH, matching the
performance of gpt-4-turbo-preview at only a 7B scale. Figure 9: Accuracy on
MATH with scaling the synthetic instances used for training Qwen2-7B

Table 1 presents the in-distribution (ID) evaluation results on the 11.6K
synthetic test instances. Among the tested open-source LLMs,
Qwen2-72B-Instruct achieves the best result, and the ranking of the other
models is generally consistent with their reported performance on other
mathematical benchmarks. Our model, with the help of the 1.07M synthetic math
problems, achieves nearly 80% accuracy, surpassing all the open-source LLMs.
However, considering that the answers in the synthetic test are not absolutely
reliable and that our model might be the only one using ID training data, this
ID evaluation results should be taken as a reference only.

We present the evaluation results on MATH in Table 2. The 7B model fine-tuned
with the synthetic training data achieved an impressive 64.9% accuracy on MATH
simply using greedy decoding, outperformed only by gpt-4o,
gpt-4-turbo-2024-04-09, Claude 3.5 Sonnet, Gemini Pro 1.5 (May 2024) and
DeepSeek-Coder-V2-Instruct.

Figure 9 presents the performance of the model on MATH when trained with
synthetic math problems at different scales. Its performance trend generally
aligns with the scaling law (Kaplan et al., 2020). Unlike previous research
(Yu et al., 2023; Wang et al., 2023; Li et al., 2024a) that performs scaling
on in-distribution data (e.g., heavily relying on MATH train data to augment
in-distribution data), we did not use any instances from MATH during data
synthesis or training. Therefore, in this out-of-distribution (OOD) evaluation
setting, achieving performance on MATH that surpasses gpt-4-turbo-preview
(1106/0125) is indeed impressive and promising for a 7B model.

Figure 10: Similarities of math problems created by personas with different
similarities: (a) Similarity of math problems when no specific focus is given;
(b) Similarity of math problems when the prompt specifies they must be related
to finance and probability; (c) Similarity of math problems synthesized by
gpt-4o and gpt-35-turbo with persona similarity of 0.9.

We examine the quality of our synthesized math problems: we sample 200
challenging problems (involving high school and university-level math
knowledge points in China), and have two math experts evaluate their validity.
Only 7 out of 200 problems are marked as invalid (e.g., due to insufficient or
conflicting conditions), yielding a reliable validity rate of 96.5%.

Moreover, we specifically examine the impact of differences in personas within
the prompts on the synthesized math problems. We first sample 100 pairs of
personas with semantic similarities888We use OpenAI’s text-embedding-3-small
(dim=512) to obtain the semantic representation and compute cosine similarity
in this experiment. Here, a persona similarity of 0.4 means that a pair of
personas has a semantic similarity within the range of 0.39 to 0.41. of 0.4,
0.6, and 0.8, respectively. For each pair of personas, we use them to create a
pair of math problems using greedy decoding (i.e., temperature=0). Then, we
compute the semantic similarity of these math problem pairs and show the
results in Figure 10.

We can clearly observe that the semantic similarity between synthesized math
problems tends to be correlated with but lower than the similarity between
their corresponding personas. When we add more specific constraints to the
prompts (e.g., math problems about finance and probability), the similarity
between the synthesized math problems tends to become higher (Figure 10(b)).
In Figure 10(c), we also test the similarity of math problems created by
gpt-4o and gpt-35-turbo using highly similar personas (similarity=0.9). The
results indicate that the semantic similarity of the math problems created by
gpt-4o and gpt-35-turbo seems not significantly different: most synthesized
math problems’ similarity falls within the range of 0.6 to 0.75, which is much
lower than the similarity of the personas (0.9). Given these observations, we
believe that using the personas in Persona Hub can ensure the diversity of
synthesized data – even at a billion scale.

###  4.2 Logical Reasoning Problems

Figure 11: Logical reasoning problems created by our proposed persona-driven
methodology Figure 12: Ruozhiba-style logical reasoning problems created with
various personas. Note that Logical Prompt 3 in this figure is a simplified
prompt. In practice, we need to specifically define a Ruozhiba-style logical
reasoning problem in this prompt in order to obtain desired synthetic data.

Similar to math problems, logical reasoning problems can also be easily
synthesized. We present examples of typical logical reasoning problems
synthesized using our proposed persona-driven methodology in Figure 11.

Figure 13: Two typical prompts used for creating instructions (i.e., user
prompts).

Moreover, we also show several Ruozhiba-style999Ruozhiba is a subforum on
Baidu Tieba, featuring numerous intricate and challenging questions posted by
Chinese netizens. These questions combine elements such as puns, polysemy,
causal inversion, and homophones, embedding logical traps that rigorously test
the ability to understand complex Chinese language constructs. Recent research
(Bai et al., 2024) has demonstrated that these data significantly benefit the
improvement of LLMs’ logical reasoning abilities. logical reasoning problems
created with personas in Figure 12. All the examples demonstrate that as long
as we can clearly describe the requirements for the logical reasoning problem
to be created, we can use a large variety of personas to steer the LLM to
generate diverse logical reasoning problems that not only meet the
requirements but are also highly relevant to the personas, even for whimsical
Ruozhiba-style problems.

For more examples, please refer to the 50,000 synthetic reasoning problems we
have released.

###  4.3 Instructions

The end users of LLMs are ultimately humans. We can use Persona Hub to
simulate a variety of users to understand their typical requests for LLM
assistance, resulting in diverse instructions (i.e., user prompts).

Figure 13 shows two typical persona-driven prompts for synthesizing
instructions, corresponding to the zero-shot prompting and persona-enhanced
few-shot prompting methods described in Section 3. The zero-shot method does
not rely on any existing instruction dataset and allows the LLM to generate
various instructions based on different personas. In contrast, the persona-
enhanced few-shot method requires existing instruction datasets (e.g., we use
WildChat (Zhao et al., 2024) in our experiments) to sample some instructions
as demonstrations and involves inferring the associated personas of these
instructions through the Text-to-Persona method described in Section 2.1.
While this approach is more complex, it results in synthesized instructions
that more closely resembles instructions from real users.

With diverse instructions created using Persona Hub, which typically represent
the first turn of a user-LLM conversation, we can easily generate their
subsequent conversational turns using an LLM, resulting in a large number of
simulated user-LLM conversations, which will be valuable for enhancing the
LLM’s instruction-following and conversational abilities. Furthermore, we can
even adopt a similar approach by selecting two personas from Persona Hub and
having LLMs role-play both, thereby simulating conversations (Jandaghi et al.,
2023) between two real people.

As Figure 1 has already shown some instructions created using this
methodology, we skip showing examples here. Readers interested in more
examples can refer to the released 50,000 instructions synthesized through
0-shot and persona-enhanced 2-shot prompting.

###  4.4 Knowledge-rich Texts

Figure 14: Examples of knowledge-rich plain text synthesis with personas

In addition to synthesizing instructions that can enhance the instruction
tuning of LLMs, the persona-driven methodology can be easily adapted to create
knowledge-rich plain text that benefits pre-training and post-training of
LLMs. As illustrated in Figure 14, we can prompt an LLM to write a
Quora101010Quora is a popular question-and-answer website where users can ask
questions and provide answers on a wide range of topics. Articles (i.e.,
posts) on Quora are often written by knowledgeable individuals, including
experts in various fields, ensuring high-quality, well-researched, and
informative content. article111111The methods for synthesizing knowledge-rich
plain text are not limited to having the LLM write Quora articles. For
instance, we can also prompt the LLM to synthesize (educational) reading
material that a persona may be interested in, thereby obtaining a large amount
of knowledge-rich text. using a persona sampled from Persona Hub. This
approach elicits the LLM’s corresponding knowledge and perspective, resulting
in highly informative and knowledge-rich content. By scaling this process with
1 billion personas in Persona Hub, we can easily obtain a vast array of
knowledge-rich texts that cover almost any topic across various levels of
granularity.

###  4.5 Game NPCs

A straightforward and practical application of Persona Hub is creating diverse
NPCs (Non-Player Characters) at scale for games. As long as we can provide a
game’s background and world-building information to the LLM, we can prompt the
LLM to project personas from Persona Hub (which are typically real-world
personas) into characters within the game’s world. In this way, we can
significantly reduce the effort required for brainstorming NPCs during the
game design process.

Figure 15: NPC creation for the game “World of Warcraft” using personas in
Persona Hub

Figure 15 and 16 show concrete examples where we use personas in Persona Hub
to create game NPCs for the game “World of Warcraft121212World of Warcraft,
developed by Blizzard Entertainment, is a highly influential MMORPG with
millions of active players worldwide, spanning over 100 countries since its
release in 2004.” and “Moonlight Blade131313Moonlight Blade (天涯明月刀) is a 3D
martial arts-themed MMORPG developed by Tencent, officially launched in China
on July 1, 2016.”.

Figure 16: NPC creation for the game “Moonlight Blade (天涯明月刀)” using Persona
Hub

###  4.6 Tool (Function) Development

Figure 17: Examples of tool (function) creation with Persona Hub

As Section 4.3 demonstrates, Persona Hub can be used to simulate a wide
variety of real users to anticipate their possible requests (i.e.,
instructions) to an LLM. Similarly, we can use Persona Hub to predict the
tools (Cai et al., 2023; Schick et al., 2024) that users might need, so that
we can pre-build these tools (functions) beforehand. When a real user makes a
similar request, the LLM can directly call these pre-built tools to return
results without having to build tools from scratch each time. This paradigm,
introduced by Persona Hub, is a completely new solution that allows LLMs to
better serve users. We believe it will have great potential in the future as
LLMs become more democratized and multifunctional.

Figure 17 shows examples of tools created with various personas. These tools
provide functionalities that the personas may need (e.g., a cab driver needs
to check traffic conditions) but cannot be accessed by an LLM, greatly
expanding the range of services provided by the LLM. Note that although the
tools in Figure 17 are just interface definitions, these definitions can be
easily converted into code implementations, as shown in Figure 18.

Figure 18: The interface definitions (e.g., the species identification
interface) in Figure 17 can be easily converted into code implementations by
calling an LLM to implement them. The resulting pre-built tools can then be
directly utilized by the LLM in the future, eliminating the need to build them
from scratch each time.

##  5 Broad Impact and Ethical Concerns

###  5.1 Broad Impact

####  5.1.1 Paradigm Shift in Data Creation by Humans and LLMs

Traditionally, it has been widely accepted that while LLMs excel at processing
data (e.g., rewriting, annotation, or generating outputs/solutions to specific
inputs), they are not particularly adept at creating new data. Consequently,
the task of data creation has still largely been the domain of humans, and the
collaboration paradigm between humans and LLMs has always been humans creating
data and LLMs processing it (Maini et al., 2024). However, the introduction of
our proposed persona-driven methodology potentially revolutionizes this
paradigm. With Persona Hub, LLMs are no longer confined to processing existing
data; they can now create various types of new data from a multitude of
perspectives, much like the diverse population of the world.

While the current capabilities of LLMs may not yet fully replace humans in
fulfilling the mission of data creation—whether in terms of data quality or
breadth—the ongoing advancements in LLM capabilities suggest a future where
LLMs will increasingly excel in data creation. As LLMs continue to improve,
both the quality and breadth of the data they can create will also likely
enhance, leading us to a point where LLMs may fully take on the role of data
creation. When this day arrives, we will no longer be constrained (Villalobos
et al., 2024) by the limited high-quality human-produced real-world
data141414https://lilianweng.github.io/posts/2024-02-05-human-data-quality/.
Persona Hub ensures the diversity and coverage of synthetic data,
significantly mitigating concerns about the negative impacts (Shumailov et
al., 2023; Dohmatob et al., 2024) of synthetic data on model training. This
may effectively eliminate the data bottleneck, thereby pushing the scaling law
to its limit.

####  5.1.2 Reality Simulation

In Section 4.3 and 4.6, we have demonstrated that Persona Hub can represent a
vast array of real-world individuals with its 1 billion personas. By employing
these personas to simulate and infer the potential needs and behaviors of real
users, we can not only allow LLMs to autonomously prepare for upcoming use
cases (queries), but also pave the way for LLMs to effectively mimic the real
world, thereby creating many new opportunities. For instance, companies can
use this method to predict how different types of users might react to a new
product launch; governments can foresee the public’s response to new
legislation, considering various population group; in online services that
require user profiling and behavior modeling, Persona Hub can facilitate the
simulation of diverse user behaviors, significantly alleviating the cold start
challenge.

Recent research on LLM roleplay, agent collaboration (Liu et al., 2023; Wang
et al., 2024), strategic reasoning (Gandhi et al., 2023; Zhang et al., 2024),
and related areas can, of course, be facilitated by the vast and diverse
personas in Persona Hub. More ambitiously, the 1 billion personas can even
sustain a well-organized society within a virtual world, such as sandbox
environments (Park et al., 2023), online games, parallel worlds, or the
metaverse, using the method discussed in Section 4.5 to simulate operations
with powerful LLMs. This virtual society can serve as a testing ground for new
policies, radical initiatives, and social dynamics, providing valuable
insights before real-world implementation. By creating a controlled
environment where diverse personas interact, we can observe emergent
behaviors, test hypotheses, and refine strategies in a risk-free setting. This
can not only help deepen our understanding of complex systems but also speed
up innovation by facilitating rapid iteration and experimentation.

####  5.1.3 Full Memory Access of LLMs

When we interact with an LLM in a specific scenario, we can only elicit a
fraction of its memory and capabilities, and we are unable to fully access the
vast knowledge encapsulated within the LLM. However, Persona Hub potentially
offers access to the full memory of an LLM because the 1 billion personas in
Persona Hub can tap into almost every perspective and piece of information
encoded within the LLM.

By leveraging these 1 billion personas, we can create diverse queries and
obtain solutions from a target LLM, thereby transforming the LLM’s
comprehensive memory (parameters) into synthetic data in textual form. If we
consider an LLM as a parameterized compression of world knowledge, then
Persona Hub can be viewed as a distributed carrier-based compression151515As
Figure 2 illustrates, Persona Hub has on the order of $10^{10}$ tokens,
equivalent to a $10,000\times$ compression of the public web text ($10^{14}$
tokens). of world knowledge, as demonstrated in Figure 2. This distributed
carrier-based compression provides us with an opportunity to decompress the
LLM’s parameters back into world knowledge and information it has ever learned
(e.g., using the method discussed in Section 4.4).

However, considering that the current Persona Hub is still in a very
preliminary stage and that today’s LLMs are not yet capable of losslessly
converting their memory into synthetic data due to inevitable hallucination
(Xu et al., 2024), the breadth and quality of the synthetic data generated
through this methodology are still limited. Nevertheless, as Persona Hub
continues to improve and scale, and as LLMs become more powerful (with less
hallucination), we can look forward to a day when it will be possible to
nearly losslessly extract the full memory of an LLM into plain text.

###  5.2 Ethical Concerns

####  5.2.1 Training Data Security and Threats to Current LLM Dominance

As discussed in Section 5.1.3, Persona Hub offers an opportunity to access the
full memory of a target LLM. However, this also introduces a significant
issue: the security of the training data. All data synthesized through the
target LLM essentially represents a form of its seen training data. Therefore,
the process of extensively extracting a target LLM’s memory is essentially
dumping its training data, even though this process is generally lossy.

Moreover, if we employ the method described in Section 4.3 to synthesize
instructions (i.e., user prompts) that nearly covers all use cases to query a
target LLM to obtain its outputs at scale, there is a high risk that the
target LLM’s knowledge, intelligence, and capabilities could be extracted and
replicated. This poses a challenge to the leading position of the most
powerful LLMs, as we have already validated in Section 4.1 through
mathematical reasoning.

Given that current LLMs generally share similar architectures and their
performance advantage primarily lies in their data, this work is likely to
impact the current practices and may potentially serve as a turning point,
accelerating the shift in the competitive landscape of LLMs from one that
heavily depends on data advantage to one that focuses on more advanced
technologies.

####  5.2.2 Miscellaneous

Synthetic data presents a general concern of misinformation and fake news,
which has been frequently discussed in previous research (Pan et al., 2023).
Persona Hub potentially amplifies this issue, as diverse personas bring
diverse writing styles, making machine-generated texts harder to distinguish
from human-generated content (Chakraborty et al., 2023). This increased
difficulty in detection may worsen issues related to data contamination, where
synthetic data is mixed with real data, potentially skewing research results
and public information.

##  6 Conclusion and Future Work

We propose a novel persona-driven data synthesis methodology and present
Persona Hub, a collection of 1 billion diverse personas automatically curated
from web data. We show that this methodology can facilitate the scaling of
synthetic data creation across various scenarios, demonstrating its potential
to revolutionize creation and applications of synthetic data, and its
prospects as a general data synthesis engine for both research and practice.

As the first version of Persona Hub, although it already contains 1 billion
personas, the descriptions of these personas are focused only on major aspects
and lack fine-grained details (e.g., preferences for colors and numbers;
specific family backgrounds, historical contexts, and life experiences). We
plan to refine the personas in subsequent versions of Persona Hub, aiming for
their descriptions to be as detailed as those found in Wikipedia articles
about individuals. These more detailed persona descriptions will make each
persona more unique, thereby scaling up Persona Hub and fostering more
opportunities for synthetic data creation, while also empowering practical
applications such as personalized conversations (e.g., character.ai).

Also, while this work only explores data synthesis with text-based LLMs, the
methodology should also be applicable to multimodal LLMs. Therefore, we will
explore multi-modal synthetic data creation as a future direction. Moreover,
given that specific personas can elicit corresponding perspectives from LLMs,
we are curious about the possibilities of using some super personas to guide
LLMs to explore beyond the scope of existing knowledge. This may provide a new
approach to tapping into the super intelligence of LLMs, which will be studied
in the future.

## References

  * qwe (2024) Qwen2 technical report.  2024\.
  * Abdin et al. (2024) Marah Abdin, Sam Ade Jacobs, Ammar Ahmad Awan, Jyoti Aneja, Ahmed Awadallah, Hany Awadalla, Nguyen Bach, Amit Bahree, Arash Bakhtiari, Harkirat Behl, et al.  Phi-3 technical report: A highly capable language model locally on your phone.  _arXiv preprint arXiv:2404.14219_ , 2024.
  * Achiam et al. (2023) Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al.  Gpt-4 technical report.  _arXiv preprint arXiv:2303.08774_ , 2023.
  * Bai et al. (2024) Yuelin Bai, Xinrun Du, Yiming Liang, Yonggang Jin, Ziqiang Liu, Junting Zhou, Tianyu Zheng, Xincheng Zhang, Nuo Ma, Zekun Wang, et al.  Coig-cqia: Quality is all you need for chinese instruction fine-tuning.  _arXiv preprint arXiv:2403.18058_ , 2024.
  * Bauer et al. (2024) André Bauer, Simon Trapp, Michael Stenger, Robert Leppich, Samuel Kounev, Mark Leznik, Kyle Chard, and Ian Foster.  Comprehensive exploration of synthetic data generation: A survey.  _arXiv preprint arXiv:2401.02524_ , 2024.
  * Bi et al. (2024) Xiao Bi, Deli Chen, Guanting Chen, Shanhuang Chen, Damai Dai, Chengqi Deng, Honghui Ding, Kai Dong, Qiushi Du, Zhe Fu, et al.  Deepseek llm: Scaling open-source language models with longtermism.  _arXiv preprint arXiv:2401.02954_ , 2024.
  * Broder (1997) Andrei Z Broder.  On the resemblance and containment of documents.  In _Proceedings. Compression and Complexity of SEQUENCES 1997 (Cat. No. 97TB100171)_ , pp. 21–29. IEEE, 1997.
  * Cai et al. (2023) Tianle Cai, Xuezhi Wang, Tengyu Ma, Xinyun Chen, and Denny Zhou.  Large language models as tool makers.  _arXiv preprint arXiv:2305.17126_ , 2023.
  * Chakraborty et al. (2023) Souradip Chakraborty, Amrit Singh Bedi, Sicheng Zhu, Bang An, Dinesh Manocha, and Furong Huang.  On the possibilities of ai-generated text detection.  _arXiv preprint arXiv:2304.04736_ , 2023.
  * Chen et al. (2022) Wenhu Chen, Xueguang Ma, Xinyi Wang, and William W Cohen.  Program of thoughts prompting: Disentangling computation from reasoning for numerical reasoning tasks.  _arXiv preprint arXiv:2211.12588_ , 2022.
  * Choi & Li (2024) Hyeong Kyu Choi and Yixuan Li.  Picle: Eliciting diverse behaviors from large language models with persona in-context learning.  In _Forty-first International Conference on Machine Learning_ , 2024.
  * Computer (2023) Together Computer.  Redpajama: an open dataset for training large language models, 2023.  URL https://github.com/togethercomputer/RedPajama-Data.
  * Delétang et al. (2023) Grégoire Delétang, Anian Ruoss, Paul-Ambroise Duquenne, Elliot Catt, Tim Genewein, Christopher Mattern, Jordi Grau-Moya, Li Kevin Wenliang, Matthew Aitchison, Laurent Orseau, et al.  Language modeling is compression.  _arXiv preprint arXiv:2309.10668_ , 2023.
  * Dohmatob et al. (2024) Elvis Dohmatob, Yunzhen Feng, Pu Yang, Francois Charton, and Julia Kempe.  A tale of tails: Model collapse as a change of scaling laws.  _arXiv preprint arXiv:2402.07043_ , 2024.
  * Gandhi et al. (2023) Kanishk Gandhi, Dorsa Sadigh, and Noah D Goodman.  Strategic reasoning with language models.  _arXiv preprint arXiv:2305.19165_ , 2023.
  * Ge et al. (2024) Tao Ge, Hu Jing, Lei Wang, Xun Wang, Si-Qing Chen, and Furu Wei.  In-context autoencoder for context compression in a large language model.  In _The Twelfth International Conference on Learning Representations_ , 2024.  URL https://openreview.net/forum?id=uREj4ZuGJE.
  * Hendrycks et al. (2021) Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt.  Measuring mathematical problem solving with the math dataset.  _arXiv preprint arXiv:2103.03874_ , 2021.
  * Huang et al. (2024) Yiming Huang, Xiao Liu, Yeyun Gong, Zhibin Gou, Yelong Shen, Nan Duan, and Weizhu Chen.  Key-point-driven data synthesis with its enhancement on mathematical reasoning.  _arXiv preprint arXiv:2403.02333_ , 2024.
  * Jandaghi et al. (2023) Pegah Jandaghi, XiangHai Sheng, Xinyi Bai, Jay Pujara, and Hakim Sidahmed.  Faithful persona-based conversational dataset generation with large language models.  _arXiv preprint arXiv:2312.10007_ , 2023.
  * Kaplan et al. (2020) Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei.  Scaling laws for neural language models.  _arXiv preprint arXiv:2001.08361_ , 2020.
  * Li et al. (2024a) Chen Li, Weiqi Wang, Jingcheng Hu, Yixuan Wei, Nanning Zheng, Han Hu, Zheng Zhang, and Houwen Peng.  Common 7b language models already possess strong math capabilities.  _arXiv preprint arXiv:2403.04706_ , 2024a.
  * Li et al. (2024b) Haoran Li, Qingxiu Dong, Zhengyang Tang, Chaojun Wang, Xingxing Zhang, Haoyang Huang, Shaohan Huang, Xiaolong Huang, Zeqiang Huang, Dongdong Zhang, et al.  Synthetic data (almost) from scratch: Generalized instruction tuning for language models.  _arXiv preprint arXiv:2402.13064_ , 2024b.
  * Li et al. (2023a) Junyi Li, Ninareh Mehrabi, Charith Peris, Palash Goyal, Kai-Wei Chang, Aram Galstyan, Richard Zemel, and Rahul Gupta.  On the steerability of large language models toward data-driven personas.  _arXiv preprint arXiv:2311.04978_ , 2023a.
  * Li et al. (2023b) Yuanzhi Li, Sébastien Bubeck, Ronen Eldan, Allie Del Giorno, Suriya Gunasekar, and Yin Tat Lee.  Textbooks are all you need ii: phi-1.5 technical report.  _arXiv preprint arXiv:2309.05463_ , 2023b.
  * Liu et al. (2024) Ruibo Liu, Jerry Wei, Fangyu Liu, Chenglei Si, Yanzhe Zhang, Jinmeng Rao, Steven Zheng, Daiyi Peng, Diyi Yang, Denny Zhou, et al.  Best practices and lessons learned on synthetic data for language models.  _arXiv preprint arXiv:2404.07503_ , 2024.
  * Liu et al. (2023) Zijun Liu, Yanzhe Zhang, Peng Li, Yang Liu, and Diyi Yang.  Dynamic llm-agent network: An llm-agent collaboration framework with agent team optimization.  _arXiv preprint arXiv:2310.02170_ , 2023.
  * Maini et al. (2024) Pratyush Maini, Skyler Seto, He Bai, David Grangier, Yizhe Zhang, and Navdeep Jaitly.  Rephrasing the web: A recipe for compute and data-efficient language modeling.  _arXiv preprint arXiv:2401.16380_ , 2024.
  * Pan et al. (2023) Yikang Pan, Liangming Pan, Wenhu Chen, Preslav Nakov, Min-Yen Kan, and William Yang Wang.  On the risk of misinformation pollution with large language models.  _arXiv preprint arXiv:2305.13661_ , 2023.
  * Park et al. (2023) Joon Sung Park, Joseph O’Brien, Carrie Jun Cai, Meredith Ringel Morris, Percy Liang, and Michael S Bernstein.  Generative agents: Interactive simulacra of human behavior.  In _Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology_ , pp. 1–22, 2023.
  * Schick et al. (2024) Timo Schick, Jane Dwivedi-Yu, Roberto Dessì, Roberta Raileanu, Maria Lomeli, Eric Hambro, Luke Zettlemoyer, Nicola Cancedda, and Thomas Scialom.  Toolformer: Language models can teach themselves to use tools.  _Advances in Neural Information Processing Systems_ , 36, 2024.
  * Shanahan et al. (2023) Murray Shanahan, Kyle McDonell, and Laria Reynolds.  Role play with large language models.  _Nature_ , 623(7987):493–498, 2023.
  * Shumailov et al. (2023) Ilia Shumailov, Zakhar Shumaylov, Yiren Zhao, Yarin Gal, Nicolas Papernot, and Ross Anderson.  The curse of recursion: Training on generated data makes models forget.  _arXiv preprint arXiv:2305.17493_ , 2023.
  * Team (2024) Qwen Team.  Introducing qwen1.5, February 2024.  URL https://qwenlm.github.io/blog/qwen1.5/.
  * Travers & Milgram (1977) Jeffrey Travers and Stanley Milgram.  An experimental study of the small world problem.  In _Social networks_ , pp. 179–197. Elsevier, 1977.
  * Villalobos et al. (2024) Pablo Villalobos, Anson Ho, Jaime Sevilla, Tamay Besiroglu, Lennart Heim, and Marius Hobbhahn.  Position: Will we run out of data? limits of llm scaling based on human-generated data.  In _Forty-first International Conference on Machine Learning_ , 2024.
  * Wang et al. (2023) Ke Wang, Houxing Ren, Aojun Zhou, Zimu Lu, Sichun Luo, Weikang Shi, Renrui Zhang, Linqi Song, Mingjie Zhan, and Hongsheng Li.  Mathcoder: Seamless code integration in llms for enhanced mathematical reasoning.  _arXiv preprint arXiv:2310.03731_ , 2023.
  * Wang et al. (2022) Yizhong Wang, Yeganeh Kordi, Swaroop Mishra, Alisa Liu, Noah A Smith, Daniel Khashabi, and Hannaneh Hajishirzi.  Self-instruct: Aligning language models with self-generated instructions.  _arXiv preprint arXiv:2212.10560_ , 2022.
  * Wang et al. (2024) Zhenhailong Wang, Shaoguang Mao, Wenshan Wu, Tao Ge, Furu Wei, and Heng Ji.  Unleashing the emergent cognitive synergy in large language models: A task-solving agent through multi-persona self-collaboration.  In _Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers)_ , pp. 257–279, 2024.
  * Xu et al. (2024) Ziwei Xu, Sanjay Jain, and Mohan Kankanhalli.  Hallucination is inevitable: An innate limitation of large language models.  _arXiv preprint arXiv:2401.11817_ , 2024.
  * Young et al. (2024) Alex Young, Bei Chen, Chao Li, Chengen Huang, Ge Zhang, Guanwei Zhang, Heng Li, Jiangcheng Zhu, Jianqun Chen, Jing Chang, et al.  Yi: Open foundation models by 01. ai.  _arXiv preprint arXiv:2403.04652_ , 2024.
  * Yu et al. (2023) Longhui Yu, Weisen Jiang, Han Shi, Jincheng Yu, Zhengying Liu, Yu Zhang, James T Kwok, Zhenguo Li, Adrian Weller, and Weiyang Liu.  Metamath: Bootstrap your own mathematical questions for large language models.  _arXiv preprint arXiv:2309.12284_ , 2023.
  * Zhang et al. (2024) Yadong Zhang, Shaoguang Mao, Tao Ge, Xun Wang, Adrian de Wynter, Yan Xia, Wenshan Wu, Ting Song, Man Lan, and Furu Wei.  Llm as a mastermind: A survey of strategic reasoning with large language models.  _arXiv preprint arXiv:2404.01230_ , 2024.
  * Zhao et al. (2024) Wenting Zhao, Xiang Ren, Jack Hessel, Claire Cardie, Yejin Choi, and Yuntian Deng.  Wildchat: 1m chatGPT interaction logs in the wild.  In _The Twelfth International Conference on Learning Representations_ , 2024.  URL https://openreview.net/forum?id=Bl8u7ZRlbM.
  * Zhu et al. (2024) Qihao Zhu, Daya Guo, Zhihong Shao, Dejian Yang, Peiyi Wang, Runxin Xu, Y Wu, Yukun Li, Huazuo Gao, Shirong Ma, et al.  Deepseek-coder-v2: Breaking the barrier of closed-source models in code intelligence.  _arXiv preprint arXiv:2406.11931_ , 2024.