11institutetext: Institute for High Energy Physics, Austrian Academy of
Sciences, Nikolsdorfergasse 18,
A-1050 Vienna, Austria

# Semirelativistic Potential Modelling of Bound States:
Advocating Due Rigour

Wolfgang Lucha 11<EMAIL_ADDRESS>
###### Abstract

The Poincaré-covariant quantum-field-theoretic description of bound states by
the homogeneous Bethe–Salpeter equation usually exhibits an intrinsic
complexity that can be attenuated by allowing this formalism to undergo
various simplifications. The resulting approximate outcome’s reliability can
be assessed by applying several rigorous constraints on the nature of the
bound-state spectra; most prominent here are existence, number and location of
discrete eigenvalues.

##  1 Our Battlefield: Semirelativistic Bound States of Least Complexity

A popular (possibly even analytic) approach to semirelativistic bound states
of scalar particles is known under the name spinless Salpeter equation, which
is just the eigenvalue equation of a Hamiltonian $H$ encompassing both the
relativistic kinetic term of the bound-state constituents and a static
potential $V$ encoding all interactions. For the case of two bound-state
constituents, of masses $m_{1}$ and $m_{2}$, and relative coordinates and
momenta $\bm{x}$ and $\bm{p}$, this Hamiltonian $H$ reads

$H=\sqrt{\bm{p}^{2}+m_{1}^{2}}+\sqrt{\bm{p}^{2}+m_{2}^{2}}+V(\bm{x})\ ,$ (1)

which, for equal masses $m_{1}=m_{2}=m$ of its bound-state constituents,
simplifies (a little bit) to

$\widetilde{H}=2\,\sqrt{\bm{p}^{2}+m^{2}}+V(\bm{x})\ .$

The spinless Salpeter equation may be regarded as some approximation defined
by a sequence of self-evident assumptions that serves to simplify the
homogeneous Bethe–Salpeter equation BS , which constitutes within the
framework of relativistic quantum field theories the adequate approach to
bound states of the fundamental degrees of freedom of such quantum field
theory:

  1. 1.

The most decisive simplification is to disregard any reference to _timelike_
coordinate and momentum variables, thereby defining an _instantaneous_
Bethe–Salpeter formalism WLe .

  2. 2.

Enabling any bound-state constituent to propagate freely entails Salpeter’s
equation SE .

  3. 3.

Skipping impacts of negative-energy contributions gives the _reduced_ Salpeter
equation.

  4. 4.

Ignoring the spins of all bound-state constituents leads to the _spinless_
Salpeter equation.

##  2 General Insights: Elementary Constraints on Bound-State Spectra

Even if being not accessible analytically, the spectrum of the operator
$\widetilde{H}$ is tightly constrained.

1\. Boundedness from below of Hamiltonian operators

is, as kinetic operators are positive, obvious if $V(\bm{x})$ itself is
bounded from below. If not, comparison with the relativistic Coulomb problem
may prove advantageous. The Hamiltonian operator $\widetilde{H}$ with the
Coulomb interaction

$V(\bm{x})\equiv-\frac{\alpha}{|\bm{x}|}\ ,\qquad\alpha>0\ ,$

will be bounded from below IWH1 ; IWH2  if and only if the coupling strength
$\alpha$ satisfies the constraint

$\alpha\leq\frac{4}{\pi}=1.273239\dots\ .$

Within this interval, the spectrum, $\sigma(\widetilde{H})$, of this operator
$\widetilde{H}$ is subject to the lower bound IWH1 ; IWH2

$\sigma(\widetilde{H})\geq
2\,m\,\sqrt{1-\left(\frac{\pi\,\alpha}{4}\right)^{\\!2}}\ .$ (2)

For Coulomb couplings $\alpha\leq 1$, the lower bound (2) may,
straightforwardly, be improved MR  to

$\sigma(\widetilde{H})\geq 2\,m\,\sqrt{\frac{1+\sqrt{1-\alpha^{2}}}{2}}\ .$

2\. Number of discrete eigenstates in the spectrum of $\widetilde{H}$:

For all potentials $V(\bm{x})$ that satisfy

$V(\bm{x})=V(r)\in L^{3/2}({\mathbb{R}}^{3})\cap L^{3}({\mathbb{R}}^{3})\
,\qquad V(\bm{x})\leq 0\ ,\qquad r\equiv|\bm{x}|\ ,$ (3)

the number of bound states of such operator $\widetilde{H}$ may be shown to be
bounded from above by ICD

$N\leq\frac{C}{12\,\pi}\int\limits_{0}^{\infty}{\rm
d}r\,r^{2}\left[|V(r)|\left(|V(r)|+4\,m\right)\right]^{3/2}\ ,$

with the constant $C$ given by $C=14.107590867$ for $m>0$ or $C=6.074898097$
for $m=0$ WL:K .

3\. Upper bounds on the eigenvalues of $\widetilde{H}$

arise as rather trivial consequence of the so-called minimum–maximum theorem
RS : for every self-adjoint operator $H$ bounded from below with (ordered)
eigenvalues $E_{0}\leq E_{1}\leq E_{2}\leq\cdots$, the $d$ (ordered)
eigenvalues
$\widehat{E}_{0}\leq\widehat{E}_{1}\leq\cdots\leq\widehat{E}_{d-1}$ of the
operator $\widehat{H}$ defined by a restriction of $H$ to a $d$-dimensional
subspace of the domain of $H$ provide upper bounds to the first $d$
eigenvalues of $H$ below the onset of the _essential_ spectrum,

$E_{k}\leq\widehat{E}_{k}\qquad\forall\quad k=0,1,2,\dots,d-1\ .$

Its eigenstates can be expanded in terms of some convenient basis WL:T1 ;
WL:T2 , e.g., in products of generalized-Laguerre polynomials,
$L_{k}^{(\gamma)}$, and spherical harmonics ${\cal Y}$ for angular momentum
$\ell$:

$\displaystyle\psi_{k}(\bm{x})\propto
r^{\ell+\beta-1}\exp(-\mu\,r)\,L_{k}^{(2\ell+2\beta)}(2\,\mu\,r)\,{\cal
Y}(\Omega_{\bm{x}})\ ,\qquad k\in{\mathbb{N}}_{0}\ ,$ (4) $\displaystyle
L_{k}^{(\gamma)}(x)\equiv\sum_{t=0}^{k}\,\binom{k+\gamma}{k-t}\,\frac{(-x)^{t}}{t!}\
,\qquad\mu>0\ ,\qquad\beta>-\frac{1}{2}\ .$

4\. Quality of eigenstates

WL:Q1 ; WL:Q2  tentatively localized by whatever technique forms a crucial
aspect that may be quantified by the _relativistic_ generalization WL:RVT1 ;
WL:RVT2  of the virial theorem of nonrelativistic quantum theory, which
relates, for any of the eigenstates $|\chi\rangle$ of the operator (1), the
expectation values of the radial derivatives of both the kinetic terms and the
potential $V(\bm{x})$:

$\left\langle\chi\left|\frac{\bm{p}^{2}}{\sqrt{\bm{p}^{2}+m_{1}^{2}}}+\frac{\bm{p}^{2}}{\sqrt{\bm{p}^{2}+m_{2}^{2}}}\right|\chi\right\rangle=\left\langle\chi\left|\,\bm{x}\cdot\frac{\partial\,V}{\partial\bm{x}}(\bm{x})\right|\chi\right\rangle.$

All of this proved useful in locating the discrete spectra of problems defined
by spherically symmetric (central) potentials $V(\bm{x})=V(r)$ (where
$r\equiv|\bm{x}|$) of Woods–Saxon WL:WS ; WL:Q@W , Hulthén WL:Q@W ; WL:H ,
Yukawa WL:Y , kink-like WL:K  or generalized-Hellmann WL:Hm ; WL:R  type. As
an illustration, an application of all these tools is sketched for the
totality of generalized-Hellmann potentials.

##  3 Application: Class of Potentials Generalizing Hellmann’s Proposal

Let us take the liberty to generalize the form proposed by Hellmann H1 ; H2
to a superposition

$V_{\rm H}(r)\equiv V_{\rm C}(r)+V_{\rm
Y}(r)=-\frac{\kappa}{r}-\upsilon\,\frac{\exp(-b\,r)}{r}$

of a definitely attractive Coulomb potential $V_{\rm C}(r)$ and a Yukawa-
inspired potential $V_{\rm Y}(r)$, with Coulomb coupling $\kappa$, and Yukawa
coupling $\upsilon$ and slope parameter $b$, subject to the constraints

$\kappa\geq 0\ ,\qquad\upsilon\gtreqqless 0\ ,\qquad b>0\ .$

Within the obtained class of _generalized_ Hellmann potentials $V_{\rm H}(r)$,
the _qualitative_ behaviour of any of its members depends on the relative sign
and size of the two coupling strengths $\kappa$ and $\upsilon$ that govern the
Coulomb and Yukawa contributions to such generalized Hellmann potential. These
potentials show a significant diversity: they may be unbounded [Fig. 1(a,b)]
or bounded [Fig. 1(c,d)] from below, and be singular [Fig. 1(a,b,d)] or finite
[Fig. 1(c)] at the origin, $r=0$.

Needless to stress, using the tools sketched in Sect. 2, it is a breeze to
rough out the spectra of the entirety of relativistic Hellmann problems (for
some relevant details, consult Ref. WL:Hm ).

  * •

Evidently, boundedness from below is the question to be answered first and
foremost: Since _any_ of our generalized Hellmann potentials is trivially
bounded from below by conveniently selected Coulomb potentials, suboptimal
bounds on the spectrum $\sigma(\widetilde{H})$ necessarily exist for

$\kappa+\upsilon\leq\frac{4}{\pi}\ .$

For the sum of couplings non-positive, i.e., $\kappa+\upsilon\leq 0$, this
lower bound of Coulomb origin is easily improvable: $V_{\rm H}(r)$ is bounded
from below and hence $\sigma(\widetilde{H})$ by the minimum of $V_{\rm H}(r)$.

Table 1: Binding energies $E_{k}-2\,m\;[m]$ ($k\in{\mathbb{N}}_{0}$) of the spinless relativistic Hellmann problem WL:Hm ; WL:R , for lowest-lying states of radial excitation $n_{\rm r}$ and orbital angular momentum $\ell$, assuming $b=\mu=m$ and $\beta=1$ for the potential-slope and Laguerre-basis parameters, and trial-space dimensions around $d\gtrapprox 29$. |  | Upper bounds
---|---|---
Bound state | $\kappa=\upsilon=\frac{1}{2}$ |  $\kappa=1$, $\upsilon=-1$ |  $\kappa=1$, $\upsilon=-2$
$n_{\rm r}$ | $\ell$ | [case Fig. 1(b)] | [case Fig. 1(c)] | [case Fig. 1(d)]
0 | 0 | $\quad\\!\\!-0.11673$ | $\quad-0.17951$ | $\quad-0.14410$
0 | 1 | $\quad\\!\\!-0.01579$ | $\quad-0.06294$ | $\quad-0.06157$
0 | 2 | $\quad\\!\\!-0.00616$ | $\quad-0.02813$ | $\quad-0.02812$
1 | 0 | $\quad\\!\\!-0.02107$ | $\quad-0.05464$ | $\quad-0.04786$
1 | 1 | $\quad\\!\\!-0.00509$ | $\quad-0.02810$ | $\quad-0.02762$
2 | 0 | $\quad\\!\\!-0.00688$ | $\quad-0.02566$ | $\quad-0.02338$
Lower bound | $\quad\\!\\!-0.58578\dots$ | $\quad-1$ | $\quad-0.37336\dots$
|
---|---
(a) | (b)
|
(c) | (d)

Figure 1: Representatives of four (out of the in total seven) different
categories of generalized Hellmann potentials $V_{\rm H}(r)$, specified by the
(in)equalities of their couplings (a) $\upsilon>\kappa$, (b)
$\upsilon=\kappa$, (c) $\upsilon=-\kappa$, and (d) $\upsilon<-\kappa$,
obtained as sums (solid black) of Coulombic (dashed magenta) and Yukawa
(dotted blue) terms. The systematic classification of all _generalized_
Hellmann potentials can be found in Table 1 of Ref. WL:R .

  * •

The number of discrete eigenstates of _any_ generalized-Hellmann Hamiltonian
may never be bounded from above because any generalized Hellmann potential
becomes Coulomb-like at large distances, in the limit $r\to\infty$. More
technically, any generalized relativistic Hellmann problem fails WL:Hm  to
satisfy the prerequisite (3) that guarantees finiteness ICD  of the number.

  * •

Upper bounds on the eigenvalues corresponding to lowest-lying discrete
eigenstates may be conveniently calculated numerically upon spanning that
$d$-dimensional subspace referred to by the minimum–maximum theorem by a
Laguerre basis (4), as exemplified in Table 1; they can be optimized by
increasing the dimension $d$ and adapting the two parameters $\mu$ and/or
$\beta$.

## References

  * (1) E. E. Salpeter and H. A. Bethe, Phys. Rev. 84 (1951) 1232.
  * (2) W. Lucha and F. F. Schöberl, J. Phys. G: Nucl. Part. Phys. 31 (2005) 1133, arXiv:hep-th/ 0507281.
  * (3) E. E. Salpeter, Phys. Rev. 87 (1952) 328.
  * (4) I. W. Herbst, Commun. Math. Phys. 53 (1977) 285\.
  * (5) I. W. Herbst, Commun. Math. Phys. 55 (1977) 316 (addendum).
  * (6) A. Martin and S. M. Roy, Phys. Lett. B 233 (1989) 407\.
  * (7) I. Daubechies, Commun. Math. Phys. 90 (1983) 511\.
  * (8) W. Lucha and F. F. Schöberl, Int. J. Mod. Phys. A 30 (2015) 1550062, arXiv:1412.4950 [hep-ph].
  * (9) M. Reed and B. Simon, _Methods of Modern Mathematical Physics IV: Analysis of Operators_ (Academic Press, New York, 1978) Sect. XIII.1.
  * (10) W. Lucha and F. F. Schöberl, Int. J. Mod. Phys. A 14 (1999) 2309, arXiv:hep-ph/ 9812368.
  * (11) W. Lucha and F. F. Schöberl, Recent Res. Dev. Phys. 5 (2004) 1423, arXiv:hep-ph/ 0408184.
  * (12) W. Lucha and F. F. Schöberl, Phys. Rev. A 60 (1999) 5091, arXiv:hep-ph/9904391.
  * (13) W. Lucha and F. F. Schöberl, Int. J. Mod. Phys. A 15 (2000) 3221, arXiv:hep-ph/ 9909451.
  * (14) W. Lucha and F. F. Schöberl, Phys. Rev. Lett. 64 (1990) 2733.
  * (15) W. Lucha, Mod. Phys. Lett. A 5 (1990) 2473.
  * (16) W. Lucha and F. F. Schöberl, Int. J. Mod. Phys. A 29 (2014) 1450057, arXiv:1401.5970 [hep-ph].
  * (17) W. Lucha and F. F. Schöberl, EPJ Web Conf. 80 (2014) 00049, arXiv:1407.4624 [hep-ph].
  * (18) W. Lucha and F. F. Schöberl, Int. J. Mod. Phys. A 29 (2014) 1450181, arXiv:1408.4957 [hep-ph].
  * (19) W. Lucha and F. F. Schöberl, Int. J. Mod. Phys. A 29 (2014) 1450195, arXiv:1410.5241 [hep-ph].
  * (20) W. Lucha and F. F. Schöberl, Int. J. Mod. Phys. A 34 (2019) 1950028, arXiv:1812.10756 [hep-ph].
  * (21) W. Lucha, PoS (EPS-HEP2019) (2020) 537, arXiv:1908.08801 [hep-ph].
  * (22) H. Hellmann, J. Chem. Phys. 3 (1935) 61.
  * (23) H. Hellmann and W. Kassatotschkin, J. Chem. Phys. 4 (1936) 324.


GROUP EMBEDDING INTO WREATH PRODUCTS: CARTESIAN DECOMPOSITION APPROACH

Enoch Suleiman

Department of Mathematics, Federal University Gashua, Yobe State, Nigeria.

<EMAIL_ADDRESS>
Muhammed Salihu Audu

Department of Mathematics, University of Jos, Plateau State, Nigeria.

<EMAIL_ADDRESS>
Sunday U. Momoh

Department of Mathematics, University of Jos, Plateau State, Nigeria.

<EMAIL_ADDRESS>
Abstract: In this paper, we showed how a group acting regularly and a diagonal
group are embedded into the wreath products in there product action using the
Cartesian Decomposition.

Keywords: Permutation Group, Embedding, Wreath Products, Cartesian
Decomposition, product action.

2020 Subject Classification: 20B05,20E22, 20B35, 20B30, 20B99

1\. INTRODUCTION

Praeger and Schneider [1] proved that for a given $X$, a subgroup that is
transitive of a given wreath product $Sym\Gamma wrSym\Delta$ on $\Delta$, then
$X$ is shown to be isomorphic to a subgroup of the wreath product of the
permutation group prompted by its stabilizer $X_{\delta}$ on the set $\Gamma$
and also a given group prompted from a set $X$ on $\Delta$. Preager and
Schneider [2] also proved that quasiprimitive permutation groups that is of
simple diagonal type is not in any way isomorphic to a subgroup of wreath
products that is acting on the same point set.

Many other people have made some progress on embedment of Groups into wreath
products [3, 5, 6, 7,10]. In this paper, we proved that a group that is acting
regularly on a given set and a diagonal group acting on a set in product
action are embeddable into wreath products in such actions.

2\. NOTION OF CARTESIAN DECOMPOSITION AND WREATH PRODUCTS

Definition 2.1 [9]: Cartesian decomposition is define given a set $\Omega$
that is finite of partitions of $\Omega,\
\epsilon=\\{\Gamma_{1},\Gamma_{2},\cdots,\Gamma_{k}\\}$, with
$|\Gamma_{i}|\geq 2$, for all $i\geq 1$ and
$|\gamma_{1}\cap\gamma_{2}\cap\cdots\cap\gamma_{k}|=1$ for all
$\gamma_{1}\in\Gamma_{1},\gamma_{2}\in\Gamma_{2},\cdots,\gamma_{k}\in\Gamma_{k}$.

Any Cartesian decomposition is called a trivial Cartesian decomposition if it
comprises of just a single partition, that is partition into singletons.
Cartesian decomposition is called homogeneous if it has the property that all
the $\Gamma_{i}$ have the equal number of elements. If
$\\{\Gamma_{1},\Gamma_{2},\cdots,\Gamma_{k}\\}$, is a particular cartesian
decomposition of a given set $\Omega$, then the defining property yields a
well-defined one-to-one correspondence between $\Omega$ and
$\Gamma_{1}\times\Gamma_{2}\times\cdots\times\Gamma_{k}$ given by
$\omega\mapsto\left(\gamma_{1},\gamma_{2},\cdots,\gamma_{k}\right)$ where, for
a given $i=1,2,\cdots,k$, the block $\gamma_{i}\in\Gamma_{i}$ is the unique
block of $\Gamma_{i}$ which contains $\omega$. Thus the set $\Omega$ can be
obviously recognized with the cartesian product
$\Gamma_{1}\times\Gamma_{2}\times\cdots\times\Gamma_{k}$.

Example 2.2: Let $\Omega=(1,2)^{3}$, we have the subsequent partitions of
$\Omega$ as.

$\Gamma_{1}=\\{\\{(1,1,1),(1,1,2),(1,2,1),(1,2,2)\\},\\{(2,1,1),(2,1,2),(2,2,1),(2,2,2)\\}\\}$
$\Gamma_{2}=\\{\\{(1,1,1),(1,1,2),(2,1,1),(2,1,2)\\},\\{(1,2,1),(1,2,2),(2,2,1),(2,2,2)\\}\\}$
$\Gamma_{3}=\\{\\{(1,1,1),(1,2,1),(2,1,1),(2,2,1)\\},\\{(1,1,2),(1,2,2),(2,1,2),(2,2,2)\\}\\}$

With $|\gamma_{1}\cap\gamma_{2}\cap\gamma_{3}|=1$ for all
$\gamma_{1}\in\Gamma_{1},\gamma_{1}\in\Gamma_{2},\gamma_{3}\in\Gamma_{3}$ and
$|\gamma_{1}|=|\gamma_{2}|=|\gamma_{3}|=2$.

Definition 2.3(Wreath products of groups [1,9] )

Suppose $G$ and $H$ are two given groups with $\phi$ a homomorphism from $H$
to the automorphism group $Aut\left(G\right)$. The semidirect product of $G$
and $H$, denoted by $G\rtimes_{\phi}H$ or simply by $G\rtimes H$, is given as
follows. The underlying set of the group $G\rtimes H$ is the direct product
$G\times H$ of sets and the multiplication of two elements
$\left(g_{1},h_{1}\right)$ and $\left(g_{2},h_{2}\right)$ is defined as

$\left(g_{1},h_{1}\right)\left(g_{2},h_{2}\right):=\left(g_{1}\left(g_{2}\left(h_{1}^{-1}\phi\right)\right),h_{1}h_{2}\right)$

It is routine to check that the semidirect product $G\rtimes H$ is a group. I
can easily be showned that $\overline{G}=\left\\{\left(g,1\right)|g\in
G\right\\}$ is a normal subgroup of $G\rtimes H$ which is isomorphic to $G$,
and $\overline{H}=\left\\{\left(1,h\right)|h\in H\right\\}$ is also a subgroup
of $G\rtimes H$ which is isomorphic to $H$. Further,
$\overline{G}\cap\overline{H}=1$ and $\overline{GH}=G\rtimes H$. Identifying
$\overline{G}$ with $G$ and $H$ with $\overline{H}$, one may view $G$ and $H$
as subgroups of $G\rtimes H$, and we will often write the element
$\left(g,h\right)$ of $G\rtimes H$ as $gh$.

We are going to utilize the ’function notation’ in describing wreath product
with its particular product action. If we have that $G$ is a group, let
$\Delta$ be a set, and let $H$ be the subgroup of $Sym\Delta$. Since our focus
will be on Cartesian decompositions, which, by definition, are finite, we
shall throughout assume that the set $\Delta$ is finite. Take
$B:=Func\left(\Delta,G\right)$, be a collection of all functions defined from
$\Delta$ into $G$. Since $B$ being a given group defined based on the
pointwise multiplication of elements of $B$. It has subgroups $G_{\delta}$,
for $\delta\in\Delta$, defined

$G=\left\\{f\in Func\left(\Delta,G\right)\ |\ \delta^{\prime}f=1\ for\ all\
\delta^{\prime}\in\Delta\backslash\left\\{\delta\right\\}\right\\}$

each $G_{\delta}$ is isomorphic to $G$. Additionally, $B$ is known to be
isomorphic with the direct product of these $|\Delta|$ duplicates of the group
$G$, and the mapping $\sigma_{\delta}:f\mapsto f_{\delta}$ where
$\delta^{\prime}f_{\delta}=\left\\{\begin{array}[]{l}\delta f\ if\ \
\delta^{\prime}=\delta\\\ 1\ \ if\
\delta^{\prime}\neq\delta\end{array}\right.$ is defined as the natural
projection mapping $B\rightarrow G_{\delta}$.

We then give a defininition of a group homomorphism, namely $\tau$, defined
from the group $H$ to $Aut\left(B\right)$: Let $h\in H$ and $f\in B$. We take
$f\left(h\tau\right)$ to be a function that is mapping
$f\left(h\tau\right):\delta\mapsto\delta h^{-1}f$.

It is routine to check that $\tau$ is indeed a homomorphism. The wreath
product $GwrH$ of $G$by $H$ is known in a general sense as semidirect product
of $B$ by $H,$ i.e. $B\rtimes H$ with respect to the given homomorphism
defined as $\tau$, and the subgroup $B$ of $GwrH$ is known as the base group
of the $GwrH$, and group $H$ known as the top group. As the two components of
a semidirect product are considered subgroups of the semidirect product, the
base group $B$ and then the top group $H$ can also be considered subgroups of
the wreath product and, in this way, $B$ becomes a normal subgroup of $GwrH$.
Considering $H$ as a known subgroup of $GwrH$, the conjugation action of $H$
on $B$ will be induced by $\tau$ in $f\left(h\tau\right):\delta\mapsto\delta
h^{-1}f$, and so we obtained:

$\left(\delta h^{-1}\right)f=\delta\left(f^{h}\right)\text{ }for\ all\ h\in
H,\ f\in Func\left(\Delta,G\right),\ for\ all\ \delta\in\Delta.$

The wreath product $GwrH$ has a natural action by conjugation on the set of
subgroups $G_{\delta}$ of its base group.

Remark 2.4[9]: Let us have a closer look at the special case
$\Delta=\\{1,2,\cdots,k\\}$. Then, the wreath product $GwrH$ can be described
using tuples, instead of functions. For $i\in\Delta$, the image if can be
written as $f_{i}$, and $f$ can be given by the $k$-tuple
$\left(f_{1},f_{2},\cdots,f_{k}\right)$, If $h\in H$ and
$\left(g_{1},g_{2},\cdots,g_{k}\right)\in B$ then the conjugate action of $h$
on $\left(g_{1},g_{2},\cdots,g_{k}\right)$ is known as

$\left(g_{1},g_{2},\cdots,g_{k}\right)^{h}=\left(g_{1h^{-1}},g_{2h^{-1}},\cdots,g_{kh^{-1}}\right)$

Hence $H$ permutes the coordinates of the elements of $B$.

Definition 2.5(Product action of Wreath Products [1,9])

Given a product action of a wreath product $GwrH$ defined on the set of
finctions $\Pi=Func(\Delta,\Gamma)$ to be: For $G\leq Sym\Gamma$ and $H\leq
Sym\Delta$, let $f\in Func\left(\Delta,G\right)$ and with $h$ being an elemt
of $H$ and fix

$g=fh$. Let $\phi\in\Pi$, then we define the function $\phi g$ that maps the
element $\delta\in\Delta$ to

$\delta\left(\phi g\right)=\left(\delta h^{-1}\phi\right)\left(\delta
h^{-1}f\right)$

Observe the element $\delta h^{-1}\phi\in\Gamma$, with $\delta h^{-1}f\in
Sym\Gamma$ giving $\left(\delta h^{-1}\phi\right)\left(\delta
h^{-1}f\right)\in\Gamma$, and $\phi g\in Func\left(\Delta,\Gamma\right)=\Pi$,
as expected.

Since $\Delta$ is finite, it is good to express the product action of it’s
given wreath product in the form of a coordinate notation. Suppose that
$\Delta=\\{1,2,\ldots,k\\}$, and view $Func\left(\Delta,G\right)$ and
$Func\left(\Delta,\Gamma\right)$ as $G^{k}$ and $\Gamma^{k}$, respectively, as
in Remark 2.4. Then for
$\left(\gamma_{1},\gamma_{2},\cdots,\gamma_{k}\right)\in\Gamma^{k},\
\left(g_{1},g_{2},\cdots,g_{k}\right)h\in GwrH$ we have

$\left(\gamma_{1},\gamma_{2},\cdots,\gamma_{k}\right)\left(\left(g_{1},g_{2},\cdots,g_{k}\right)h\right)=\left(\gamma_{1h^{-1}}g_{2h^{-1}},\gamma_{2h^{-1}}g_{2h^{-1}},\cdots,\gamma_{1h^{-1}}g_{kh^{-1}}\right)$

To have a deeper understanding of the subgroup(s) of wreath products of groups
we entreat the concept of Cartesian decomposition.

Looking at the set $\Pi=Func\left(\Delta,\Gamma\right)$, and
$\forall\delta\in\Delta$, we give the defininition of a partition
$\Gamma_{\delta}$ of the set $\Pi$ tobe: Take

$\Gamma_{\delta}=\left\\{\gamma_{\delta}|\ \forall\gamma\in\Gamma\right\\},\
where\gamma_{\delta}:=\left\\{\psi\in\Pi\ |\ \delta\psi=\gamma\right\\}$.

We can easily check that $\Gamma_{\delta}$ is certainly a partition of of the
set $\Pi$. The representation shows two significant facts.

First, the mapping $\delta\ \mapsto\ \Gamma_{\delta}$ is defined as a ono-to-
one correspondence between $\Delta$ and
$\\{\Gamma_{\delta}|\delta\in\Delta\\}$.

Second, let $\delta\in\Delta$, be a fixed element, the mapping
$\gamma\mapsto\gamma_{\delta}$ defined is a given one-to-one correspondence
between $\Gamma$ and $\Gamma_{\delta}$. Let $\gamma\in\Gamma$ and
$\delta\in\Delta$, then the element $\gamma_{\delta}\in\Gamma_{\delta}$ is
well thought-out to be ’a copy’ of $\gamma$ in the set $\Gamma_{\delta}$, and
it is known as the $\gamma$-part of $\Gamma_{\delta}$.

The Cartesian product $\displaystyle\prod_{\delta\in\Delta}\Gamma_{\delta}$ is
given to be a one-to-one correspondence with the original set $\Pi$: taking
$\gamma_{\delta}\in\Gamma_{\delta}$, one for all element $\delta\in\Delta,$
the intersection $\displaystyle\bigcap_{\delta\in\Delta}\gamma_{\delta}$
comprises of single element of $\Pi$, namely the map that takes each $\delta$
to the element $\gamma\in\Gamma$ that corresponds to $\gamma_{\delta}$. This
gives a one-to-one corespondence from the Cartesian product
$\displaystyle\prod_{\delta\in\Delta}\Gamma_{\delta}$ to the set of functions
$\Pi$. Then, the set

$\epsilon\ =\ \\{\Gamma_{\delta}\ |\ \forall\ \delta\in\Delta\\}$

is a Cartesian decomposition of the set $\Pi$. Precisely, is a set of
partitions seen as the sets of natural Cartesian decomposition of the set
$\Pi$. Since $Sym\Gamma wrSym\Delta$ is also a group that acts on the set
$\Pi$, and the given action of $Sym\Gamma wrSym\Delta$ is being strectched to
subsets of the set $\Pi$, subsets of subsets, etc. To be specific, we look at
the action of the group $Sym\Gamma wrSym\Delta$ on the particular sets of
partitions of the set $\Pi$. We will observe that
$\\{\Gamma_{\delta}|\delta\in\Delta\\}$ is invariant under the action. The
normal product action of the group $Sym\Gamma wrSym\Delta$ on
$\displaystyle\prod_{\delta\in\Delta}\Gamma_{\delta}$ is known to be
permutationally isomorphic to the action that is defined on the set $\Pi$, and
so the stabiliser in the permutation group $Sym\Pi$ of the Cartesian
decomposition is the permutation group $Sym\Gamma wrSym\Delta$.

If $X$ is a given subgroup of $Sym\Gamma wrSym\Delta$ in its product action on
the set of functions $\Pi=Func(\Delta,\Gamma)$, and $\Delta=\\{1,\cdots,k\\}$,
then we identify $\Pi$ with the set $\Gamma^{k}$ of ordered $k$-tuples of the
elements of the set $\Gamma$, and in this situation, subgroups of $Sym\Gamma
wrSym\Delta$ ascend as automorphism groups of different types of graph
products (see (Praeger and Schneider, 2018b)), as groups of automorphism of
some given codes of length say $k$ on the alphabet $\Gamma$, seen as subsets
of $\Gamma^{k}.$

Then following theorem and its proof can be found in Praeger and Schneider,
2018b, Theorem 5.13.

Theorem 2.6: (Wreath Embedding Theorem [9]) Suppose that $X$ is a given
permutation group on a set $\Omega$ preserving a homogeneous Cartesian
decomposition $\epsilon=\\{\Gamma_{\delta}|\delta\in\Delta\\}$ of $\Omega$,
and let $\Gamma\in\epsilon$. Then there is a permutational isomorphism that
maps $X$ to a subgroup of $Sym\Gamma wrSym\Delta$ with its product action on
$Func(\Delta,\Gamma)$, and maps $\epsilon$ to the natural Cartesian
decomposition of $\epsilon$ defined above.

3\. MAIN RESULTS

Now we are in a better position to prove our results. Embedding a wreath
product in it’s product action is equivalent to proving that a Cartesian
decomposition is preserved.

Definition 3.1[3]: The $G$-action is said to be transitive if $\Omega$ is a
$G$-orbit; that is, for all $\alpha,\beta\in\Omega$ there is a given element
$g\in G$ such that $\alpha g=\beta$. If $G$ is not transitive, then it is
known as intransitive. A permutation group is known as semiregular if all its
point stabilisers are trivial. A permutation group is regular if it is
transitive and semiregular.

Proposition 3.2: Let $T=S^{k}$, for some group $S$, let $S$ act regularly on
$\Gamma$ such that $|\Gamma|=|S|$, then $Sym\Gamma wrS_{k}$ in its natural
action acts on $\Omega=\Gamma^{k}$, and the permutation representation of $T$
is embeddable in $Sym\Gamma wrS_{k}$ acting regularly on $\Omega=\Gamma^{k}$.

Proof: We suppose that $S$ act regularly on $\Gamma$ such that $|\Gamma|=|S|,\
T=S^{k}$, for some group $S$ acts regularly on $\Omega=\Gamma^{k}$. Let
$\Omega=\Gamma^{k}$ and suppose there exist a given subgroup $W$ of
$Sym\Omega$ that is permutationally isomorphic to $Sym\Gamma wrS_{k},$ with
$|\Gamma|\geq 2$ and $k\geq 2$, then $W=Sym\Gamma wrS_{k}$.

The normal subgroup $N=\left(Sym\Gamma\right)^{k}$ is known as the base group
of $W$ and $H\cong S_{k}$ is known as the top group. The product action of $W$
on $\Omega=\Gamma^{k}$ is defined by

$\left(\gamma_{1},\cdots,\gamma_{k}\right)^{xh}=\left(\gamma_{1h^{-1}}^{x_{1h^{-1}}},\cdots,\gamma_{kh^{-1}}^{x_{kh^{-1}}}\right)$

for all
$\left(\gamma_{1},\cdots,\gamma_{k}\right)\in\Omega,x=\left(x_{1},\cdots,x_{k}\right)\in
N$ and $h\in H$, where the image of $\gamma\in\Gamma$ under $y\in Sym\Gamma$
is $\gamma^{y}.\ W$ is obviously transitive on $\Omega=\Gamma^{k}$.

The Cartesian decomposition corresponding to the identity map on $\Omega$ is

$\epsilon=\\{\Gamma_{1},\cdots,\Gamma_{k}\\}$

Where $\Gamma_{i}$ is the partition of $\Omega=\Gamma^{k}$ into disjoint
subsets according to the $i^{th}$ coordinate of a point in
$\Omega=\Gamma^{k}$, that is to say, the parts of $\Gamma_{i}$ are indexed by
$\Gamma$ and the $\gamma$-part is the set of all points
$(\gamma_{1},\cdots,\gamma_{k})$ with $\gamma_{i}=\gamma$. Thus
$|\Gamma_{i}|=|\Gamma|$ for all $i$.

Thus $\epsilon$ is homogenous. Also each element $xh\in W$ maps the partition
$\Gamma_{i}$ to the partition $\Gamma_{ih}$. Thus $W$ preserves the Cartesian
decomposition $\epsilon$.

Also $W$ permutes the partitions $\Gamma_{i}$ transitively. Thus the
permutation representation of $T$ is isomorphic to a subgroup of $Sym\Gamma
wrS_{k}$.

Definition 3.3: (Diagonal group D(T,m) $[3]$) Suppose that $G$ is a group with
order $|G|>1$, and $n\geq 0$ positive integer. Let $\delta\left(G,n+1\right)$
be the diagonal subgroup $\left\\{\left(g,g,\cdots,g\right)|g\in G\right\\}$
of $G^{n+1}$. We select coset representatives for the element $\delta(G,n+1)$
in $G^{n+1}$. A suitable selection is to figure out the direct factors of
$G^{n+1}$ as $G_{0};G_{1};\cdots;G_{n},$ and employ the representatives of the
form $\left(1,g_{1},g_{2},\cdots,g_{n}\right)$ where $g_{i}\in G_{i}$. and let
$\Omega$ denote the collection of all such symbols. Then, $\Omega$ is
bijective by means of $G^{n}$.

We are now going to designate the action of $D(G,n)$ as:

(a) Let $1\leq i\leq n$, the factor $G_{i}$ acts by right multiplication on
symbols in the $i^{th}$ position in the elements of the set $\Omega$.

(b) $G_{0}$ is acting by simultaneous left multiplication of all the
coordinates by the inverse. Since, for $x\in G_{0},\ x$ maps the coset
containing $\left(1,g_{1},g_{2},\cdots,g_{n}\right)$ to the coset containing
$\left(x,g_{1},g_{2},\cdots,g_{n}\right)$, which is equal to the coset
containing $\left(1,x^{-1}g_{1},x^{-1}g_{2},\cdots,x^{-1}g_{n}\right)$
Automorphisms of $G$ also acts simultaneously on each of the coordinates;
nevertheless the inner automorphisms are recognized with the action of
elements in the diagonal subgroup $\delta\left(G,n+1\right)$ (the element
$(\left(x,x,x,\cdots,x\right)$ maps the coset containing
$\left(1,g_{1},g_{2},\cdots,g_{n}\right)$ to the coset containing
$\left(x,g_{1}x,g_{2}x,\cdots,g_{n}x\right)$, which is equal to the coset
containing $\left(1,x^{-1}g_{1}x,x^{-1}g_{2}x,\cdots,x^{-1}g_{n}x\right).$

(c) Elements of the symmetric group $S_{n}$ (fixing coordinate $0$) also acts
by permuting the coordinates in elements of $\Omega$.

(d) Look at the element of $S_{n+1}$ which transposes the coordinates $0$ and
$1$. It is mapping the coset containing
$\left(1,g_{1},g_{2},\cdots,g_{n}\right)$ to the coset containing
$\left(g_{1},1,g_{2},\cdots,g_{n}\right)$ ,that also contains
$\left(1,g_{1}^{-1},g_{1}^{-1}g_{2},\cdots,g_{1}^{-1}g_{n}\right)$. So the
action of the given transposition is

$\left(1,g_{1},g_{2},\cdots,g_{n}\right)\mapsto\left(1,g_{1}^{-1},g_{1}^{-1}g_{2},\cdots,g_{1}^{-1}g_{n}\right).\
$

(e) Now $S_{n}$ and the transposition generates $S_{n+1}.$

Proposition 3.4: Let $D\left(G,k\right)$ be a diagonal group where $G$ is a
group such that $G\leq Sym\Gamma$ and positive integer $k\geq 2$, then
$D\left(G,k\right)$ is embeddable in the wreath product $Sym\Gamma wrS_{k}$
where the wreath product acts on $\Omega=\Gamma^{k}$ and $\Gamma\geq 2$ is a
non-empty set.

Proof: Suppose that $D\left(G,k\right)$ is a diagonal group where $G$ is a
group and there is a positive integer $k\geq 2$.

Now, $Sym\Gamma wrS_{k}$ acts naturally on the set $\Omega=\Gamma^{k}$ in its
product action and is defined by

$\left(\gamma_{1},\cdots,\gamma_{k}\right)^{xh}=\left(\gamma_{1h^{-1}}^{x_{1h^{-1}}},\cdots,\gamma_{kh^{-1}}^{x_{1h^{-1}}}\right)$

for all
$\left(\gamma_{1},\cdots,\gamma_{k}\right)\in\Omega,x=\left(x_{1},\cdots,x_{k}\right)\in\left(Sym\Gamma\right)^{k}$
and $h\in S_{k}$ and the image of $\gamma\in\Gamma$ under $y\in Sym\Gamma$ is
$\gamma^{y}.\ Sym\Gamma wrS_{k}$ is obviously transitive on
$\Omega=\Gamma^{k}$.

Now for each coset of the diagonal group
$\delta(G,k)=\left\\{\left(g,g,\cdots,g\right)|g\in G\right\\}$ of $G^{k}$,
there is a unique representative of the form
$\left(1,g_{1},g_{2},\cdots,g_{k-1}\right)$ and define $\eta$ as

$\eta\left(\left(1,g_{1},g_{2},\cdots,g_{k-1}\right),\left(\left(g,g,...,g\right),\left(g,g,\cdots,g\right)\right)\right)=\left(1,g^{-1}g_{1}g,g^{-1}g_{2}g,\cdots,g^{-1}g_{k}g\right)$

which is the action on $\delta\left(G,k\right)$.

The Cartesian decomposition corresponding to the identity map on $\Omega$ is

$\epsilon=\\{\Gamma_{1},\cdots,\Gamma_{k}\\}$

Where $\Gamma_{i}$ is the partition of $\Omega=\Gamma^{k}$ into disjoint
subsets according to the $i^{th}$ coordinate of a point in
$\Omega=\Gamma^{k}$, that is to say, the parts of $\Gamma_{i}$ are indexed by
$\Gamma$ and the $\gamma$-part is the set of all points
$\left(\gamma_{1},\cdots,\gamma_{k}\right)$ with $\gamma_{1}=\gamma$. Thus
$|\Gamma_{i}|=|\Gamma|$ for all $i$.

Thus $\epsilon$ is homogenous. Also each element of $Sym\Gamma wrS_{k}$ maps
the partition $\Gamma_{i}$ to the partition $\Gamma_{ik}$. Thus $W$ preserves
the Cartesian decomposition $\epsilon$. Thus $D\left(G,k\right)$ is isomorphic
to a subgroup of $Sym\Gamma wrS_{k}.$

Definition 3.5: Suppose that $G$ is a given group and $n$ a positive. Let
$\Omega=G^{n}$ ; this will be the domain of a permutation, and its elements
are written as $\left[x_{1},x_{2},\cdots,x_{n}\right]$, where
$x_{1},x_{2},\cdots,x_{n}\in G$. The diagonal group $D\left(G,n\right)$ is
generated by the following five types of permutations on $\Omega$:

(a) The group $G^{n}$ acting by right multiplication; so the element
$\left(g_{1},g_{2},...,g_{n}\right)$ maps
$\left[x_{1},x_{2},\cdots,x_{n}\right]$ to
$\left[x_{1}g_{1},x_{2}g_{2},\cdots,x_{n}g_{n}\right]$. I will let $G_{i}$ be
the $i^{th}$ factor of $G^{n}$, so that $G_{i}$ acts on the $i^{th}$
coordinate of elements of $\Omega$.

(b) The group $G$, acting by simultaneous left multiplication; so $g$ maps
$\left[x_{1},x_{2},\cdots,x_{n}\right]$ to
$\left[g^{-1}x_{1},g^{-1}x_{2},\cdots,g^{-1}x_{n}\right]$. It will denoted by
$G_{0}$.

(c) The automorphism group of $G$, acting simultaneously on all coordinates.

(d) The symmetric group $S_{n}$, acting by permuting the coordinates.

(e) A permutation $\tau$ , defined by

$\tau:\left[x_{1},x_{2},\cdots,x_{n}\right]\mapsto\left[x_{1}^{-1},x_{1}^{-1}x_{2},\cdots,x_{1}^{-1}x_{n}\right]$.

Proposition 3.6: Let $D\left(G,n\right)$ be a diagonal group, where $G$ is a
finite group and $n$ is a positive integer, and assume that $\Gamma=G$ with
$G$ acting regularly by right multiplication, and identify $G$ with the
corresponding subgroup of $Sym\Gamma$. Suppose also that in
$Aut\left(G\right)$ there is a subgroup $Out\left(G\right)$, which complements
the group of inner automorphisms. Then $D\left(G,n\right)$ is embeddable into
the wreath product $Sym\Gamma wrS_{n}$, acting naturally in product action on
the set $\Omega=\Gamma^{n}$.

Proof: Let $D\left(G,n\right)$ be a diagonal group for a finite group $G$ and
n a positive integer, and let $\Gamma=G$ with $G$ acting by right
multiplication. Now, $W=Sym\Gamma wrS_{n}$ acts naturally on the set
$\Omega=\Gamma^{n}$ in its product action and is defined by

$\left(\gamma_{1},\cdots,\gamma_{n}\right)^{nh}=\left(\gamma_{1h^{-1}}^{x_{1h^{-1}}},\cdots,\gamma_{nh^{-1}}^{x_{nh^{-1}}}\right)$

for all $\left(\gamma_{1},\cdots,\gamma_{n}\right)\in\Omega,\
x=\left(x_{1},\cdots,x_{n}\right)\in\left(Sym\Gamma\right)^{n}$ and $h\in
S_{n}$ and the image of $\gamma\in\Gamma$ under $y\in Sym\Gamma$ is
$\gamma^{y}.\ W=Sym\Gamma wrS_{n}$ is obviously transitive on
$\Omega=\Gamma^{n}.\ \Omega$ is bijective with $G^{n}$.

Now,

$\mathbf{1}\mathbf{.}$We identified $G$ with a subgroup of $Sym\Gamma$. So we
have $M:=G^{n}$ as a subgroup of the base group $\left(Sym\Gamma\right)^{n}$
of $W$.

$\mathbf{2}\mathbf{.}$ From the definition of the action of $W$ it is clear
that the top group $S_{n}$ normalizes $M$.

$\mathbf{3}\mathbf{.}$ Also, $N_{Sym\Gamma}\left(G\right)$ is the holomorph of
$G$, this is known as a semidirect product of $G$ and a group
$A=Aut\left(G\right)$ and $A$ acts on $\Gamma=G$ naturally as automorphisms.

$\mathbf{4}\mathbf{.}$ By assumption $A$ has a subgroup $O\subseteq
Out\left(G\right)$ which complements the inner automorphism group of $G$.

$\mathbf{5}\mathbf{.}$ Hence the normalizer of $M$ in the base group of $W$
contains a semidirect product $MO^{n}$, and we define $D$ as the diagonal
subgroup of $O^{n}$, namely $D=\left\\{\left(x,x,\cdots,x\right)\ |\ x\in
O\right\\}.$

$\mathbf{6}\mathbf{.}$ Finally consider the group generated by $M,\ D$ and
$S_{n}$, this is a copy of $D\left(G,n\right)$ in $W$.

The Cartesian decomposition corresponding to the identity map on
$\Omega=\Gamma^{n}$ is

$\epsilon=\\{\Gamma_{1},\cdots,\Gamma_{n}\\}$

Where $\Gamma_{i}$ is the partition of $\Omega=\Gamma^{n}$ into disjoint
subsets according to the $i^{th}$ coordinate of a point in
$\Omega=\Gamma^{n}$, that is to say, the parts of $\Gamma_{i}$ are indexed by
$\Gamma$ and the $\gamma$-part is the set of all points
$\left(\gamma_{1},\cdots,\gamma_{n}\right)$ with $\gamma_{i}=\gamma$. Thus
$|\Gamma_{i}|=|\Gamma|$ for all $i$.

Thus $\epsilon$ is homogenous. Also each element of $Sym\Gamma wrS_{n}$ maps
the partition $\Gamma_{i}$ to the partition $\Gamma_{ik}$. Thus $W$ preserves
the Cartesian decomposition $\epsilon$. Now since the automorphism group of
$G$ acts simultaneously on all coordinates, thus $W=Sym\Gamma wrS_{n}$
preserves the Cartesian decomposition $\epsilon$ and we conclude that
$D\left(G,n\right)$ is embeddable into the wreath product $Sym\Gamma wrS_{n}.$

4\. CONCLUSION

We proved that a group that is acting regularly on a set and a diagonal group
acting on a set in product action are embeddedable into wreath products in
such actions using the Cartesian decomposition.

5\. ACKNOWLEDGEMENT

We would like to sincerely appreciate Prof. Peter J. Cameron of University of
St Andrews, UK for his suggestion and also Prof. Cheryl E. Praeger of
University of Western Australia, Australia for proof reading, insightful
comments and suggestion. Also our appreciation goes to Prof. Csaba Schneider
of Universidade Federal De Minas Gerais, Brazil for his insightful comment.

6\. REFRENCES

[1] Praeger, C. E., and Schneider, C. Embedding Permustation Groups Into
Wreath Products In Product Action. J. Aust. Math. Soc., (2012) 92, 127-136.
DOI:10.1017/S1446788712000110

[2] Preager, C. E., and Schneider, C. Group Factorisations, Uniform
Automorphisms, and Permutation Groups of Simple Diagonal Type. Israel Journal
of Mathematics, (2018a) 228, 1001-1023. doi:DOI: 10.107/s11856-018-1790-1

[3] Dixon, J. D., and Mortimer, B. Permutation Groups. Berlin, New York:
Springer. (1996).

[4] Sury, B. Wreath Products, Sylow’s Theorem and Fermat’s Little Theorem.
European Journal of Pure and Applied Mathematics, (2010) 3(1), 13-15.

[5] Schneider, C. Wreath products in permutation group theory. Retrieved
September 3, 2019, from www.mat.ufmg.br/ csaba/pdf/marienheide.pdf (2017)

[6] Suleiman, E., and Audu, M. S. Some Embedment of Groups into Wreath
Products. International Journal of Algebra and Statistics, (2020) 9(1-2),
1-10. doi:DOI: 10.20454/ijas.2020.1630

[7] Suleiman, E., and Audu, M. S. On Some Embedment of Groups into Wreath
Products. Advances in Pure Mathematics, (2021) 11, 109-120.
doi:https://doi.org/10.4236/apm.2021.112007

[8] Bailey, R. A., Cameron, P. J., Praeger, C. E., and Schneider, C. (2020).
The geometry of diagonal groups. arXiv:2007.10726v1 [math.GR], (2020)1-61.

[9] Praeger, C. E., and Schneider, C. Permutation Groups and Cartesian
Decompositions. (L. M. 449, Ed.) United Kingdom: Cambridge University Press.
doi:DOI: 10.1017/9781139194006 (2018b)

[10] Praeger, C. E. The Inclusion Problem For Finite Primitive Permutation
Groups. Proc. London Math. Soc., (1990) 60(3), 68-88.


# Particle Transformer for Jet Tagging

Huilin Qu  Congqiao Li  Sitian Qian

###### Abstract

Jet tagging is a critical yet challenging classification task in particle
physics. While deep learning has transformed jet tagging and significantly
improved performance, the lack of a large-scale public dataset impedes further
enhancement. In this work, we present JetClass, a new comprehensive dataset
for jet tagging. The JetClass dataset consists of 100 M jets, about two orders
of magnitude larger than existing public datasets. A total of 10 types of jets
are simulated, including several types unexplored for tagging so far. Based on
the large dataset, we propose a new Transformer-based architecture for jet
tagging, called Particle Transformer (ParT). By incorporating pairwise
particle interactions in the attention mechanism, ParT achieves higher tagging
performance than a plain Transformer and surpasses the previous state-of-the-
art, ParticleNet, by a large margin. The pre-trained ParT models, once fine-
tuned, also substantially enhance the performance on two widely adopted jet
tagging benchmarks. The dataset, code and models are publicly available at
https://github.com/jet-universe/particle_transformer.

Transformer, Jet Tagging, Dataset, ICML

##  1 Introduction

Machine learning has revolutionized how large-scale data samples are analyzed
in particle physics and greatly increased the discovery potential for new
fundamental laws of nature (Radovic et al., 2018). Specifically, deep learning
has transformed how jet tagging, a critical classification task at high-energy
particle colliders such as the CERN LHC, is performed, leading to a drastic
improvement in its performance (Kogler et al., 2019; Larkoski et al., 2020).

Figure 1: Illustration of jet tagging at the CERN LHC. High-energy proton-
proton collisions at the LHC can produce new unstable particles that decay and
yield a collimated spray of outgoing particles. These outgoing particles are
measured by complex particle detector systems, and jets can be built
(“reconstructed”) from these measured particles. The goal of jet tagging is to
classify the jets and identify those arising from particles of high interest,
e.g., the Higgs boson, the $W$ or $Z$ boson, or the top quark.

At the CERN LHC, two beams of protons are accelerated to nearly the speed of
light and made to collide at a frequency of 40 million times per second (40
MHz). Such high-energy collisions can create new unstable particles, which
then decay and produce sprays of outgoing particles. Complex detector systems,
such as the general-purpose ATLAS (ATLAS Collaboration, 2008) and CMS (CMS
Collaboration, 2008) detectors with $\mathcal{O}$(100 M) individual sensors of
various types, are used to measure the positions, trajectories, energies, and
momenta of the outgoing particles. From these measurements, an event is
reconstructed for each collision. The primary goal in the analysis of the
collision data is to identify events involving novel physics processes, an
example of which is the discovery of the Higgs boson (ATLAS Collaboration,
2012; CMS Collaboration, 2012).

Figure 2: Examples of the 10 types of jets in the JetClass dataset, viewed as
particle clouds. Each particle is displayed as a marker, with its coordinates
corresponding to the flying direction of the particle, and its size
proportional to the energy. The circles, triangles (upward- or downward-
directed), and pentagons represent the particle types, which are hadrons,
leptons (electrons or muons), and photons, respectively. The solid (hollow)
markers stand for electrically charged (neutral) particles. The marker color
reflects the displacement of the particle trajectory from the interaction
point of the proton-proton collision, where a larger displacement results in
more blue.

A crucial step in the data analysis process is jet tagging. A jet refers to a
collimated spray of outgoing particles. Jet tagging is the process of
identifying the type of particle that initiates a jet. It is essentially a
classification task that aims to distinguish jets arising from particles of
interest, such as the Higgs boson or the top quark, from other less
interesting types of jets. Jet tagging is a challenging task because the
particle initiating a jet can radiate, and the radiated particles further
produce more particles, leading to a cascade of $\mathcal{O}(10)$ to
$\mathcal{O}(100)$ particles at the end. The radiation also smears the
characteristics of the initial particle and makes the identification very
difficult.

Traditional approaches for jet tagging rely on hand-crafted features motivated
by the principles of quantum chromodynamics (QCD), the theory governing the
evolution of particles inside a jet. The rise of deep learning has led to a
plethora of new approaches (Larkoski et al., 2020). The prevailing ones
represent a jet as a particle cloud, i.e., an unordered and variable-sized set
of the outgoing particles, as illustrated in Figure 1. Based on the particle
cloud representation, ParticleNet (Qu & Gouskos, 2020) adapts the Dynamic
Graph CNN architecture (Wang et al., 2019) and achieves substantial
performance improvement on two representative jet tagging benchmarks. Since
then, several new models (e.g., Mikuni & Canelli (2020, 2021); Shimmin (2021))
have been proposed, but no significant performance improvement has been
reported so far. We deem the lack of a sufficiently large public dataset an
impeding factor.

In this work, we advocate for JetClass, a new large and comprehensive dataset
to advance deep learning for jet tagging. The JetClass dataset (Qu et al.,
2022) consists of 100 M jets for training, about two orders of magnitude
larger than existing public datasets. It also includes more types of jets,
several of which have not been explored for tagging yet but are promising for
future applications at the LHC.

Based on this dataset, we propose Particle Transformer (ParT), a new
Transformer-based architecture for jet tagging. We demonstrate that
Transformer architectures, together with a large dataset, can reach powerful
performance on jet tagging. We introduce a small modification to the attention
mechanism by incorporating a new term characterizing pairwise particle
interactions. The resulting ParT achieves significantly higher performance
than a plain Transformer and surpasses the previous state-of-the-art,
ParticleNet, by a large margin. We also apply the pre-trained ParT models to
two widely adopted jet tagging benchmarks with fine-tuning and observe a
substantial gain on these tasks.

##  2 The JetClass Dataset

We provide an overview of the new JetClass dataset in this section. The
dataset includes a total of 10 types of jets. Representative jets of each type
are visualized as particle clouds in Figure 2. The jets in this dataset
generally fall into two categories. The background jets are initiated by light
quarks or gluons ($q/g$) and are ubiquitously produced at the LHC. The signal
jets are those arising either from the top quarks ($t$), or from the $W$, $Z$
or Higgs ($H$) bosons. For top quarks and Higgs bosons, we further consider
their different decay modes as separate types, because the resulting jets have
rather distinct characteristics and are often tagged individually. The use of
jet tagging typically involves selecting one (or a few) specific type of
signal jets with high confidence, and rejecting background jets as much as
possible, since the background jets usually appear orders of magnitude more
frequently than the targeted signal jets. Note that for several types of
signal jets in this dataset, such as $H\to 4q$, $H\to\ell\nu qq^{\prime}$, and
$t\to b\ell\nu$, no dedicated methods have been developed so far to tag them.
However, as we will demonstrate in Section 5.1, these types of jets can also
be cleanly tagged with deep learning approaches, opening up new possible
territories for jet tagging at the LHC.

Simulation setup. Jets in this dataset are simulated with standard Monte Carlo
event generators used by LHC experiments. The production and decay of the top
quarks and the $W$, $Z$ and Higgs bosons are generated with MadGraph5_amc@nlo
(Alwall et al., 2014). We use pythia (Sjöstrand et al., 2015) to evolve the
produced particles, i.e., performing parton showering and hadronization, and
produce the final outgoing particles111We include multiple parton interactions
but omit pileup interactions in the simulation.. To be close to realistic jets
reconstructed at the ATLAS or CMS experiment, detector effects are simulated
with Delphes (de Favereau et al., 2014) using the CMS detector configuration
provided in Delphes. In addition, the impact parameters of electrically
charged particles are smeared to match the resolution of the CMS tracking
detector (CMS Collaboration, 2014). Jets are clustered from Delphes E-Flow
objects with the anti-$k_{\text{T}}$ algorithm (Cacciari et al., 2008, 2012)
using a distance parameter $R=0.8$. Only jets with transverse momentum in
500–1000 GeV and pseudorapidity $|\eta|<2$ are considered. For signal jets,
only the “high-quality” ones that fully contain the decay products of initial
particles are included222We require all the quarks ($q$) and charged leptons
(electrons or muons, denoted $\ell$) from the decay of the top quark or the
$W$, $Z$ or Higgs boson satisfy $\Delta R(\text{jet},q/\ell)<0.8$, where
$\Delta R(a,b)\equiv\sqrt{(\eta_{a}-\eta_{b})^{2}+(\phi_{a}-\phi_{b})^{2}}$,
in which $\eta$ ($\phi$) is the pseudorapidity (azimuthal angle) of the
momentum of the jet or the particle..

Input features. The dataset provides all constituent particles of each jet as
inputs for jet tagging. Note that the number of particles varies from jet to
jet, typically between 10 and 100, with an average of 30–50 depending on the
jet type. For each particle of a jet, three categories of features are
provided:

  * •

Kinematics. This includes the energy and momentum, described by the 4-vector
$(E,p_{x},p_{y},p_{z})$ in units of GeV, which are the most fundamental
quantities measured by a particle detector. All other kinematic variables can
be computed from the 4-vectors.

  * •

Particle identification. This includes the electric charge, with values of
$\pm 1$ (positively/negatively charged particles) and 0 (neural particles),
and the particle identity determined by the detector systems. For the latter,
a 5-class one-hot encoding is used to be consistent with current LHC
experiments: charged hadron ($\pm 211,\pm 321,\pm 2212$), neutral hadron (0),
electron ($\pm 11$), muon ($\pm 13$), and photon (22). The particle
identification information is especially important for tagging jets involving
a charged lepton, e.g., $H\to\ell\nu qq^{\prime}$ and $t\to b\ell\nu$, as
leptons can be almost unambiguously identified at the LHC.

  * •

Trajectory displacement. This includes the measured values and errors of the
transverse and longitudinal impact parameters of the particle trajectories in
units of mm, in total 4 variables. These measurements are only available for
electrically charged particles, and a value of 0 is used for neutral
particles. The trajectory displacement information is critical for tagging
jets involving a bottom ($b$) or charm ($c$) quark (CMS Collaboration, 2020b),
such as $H\to b\bar{b}$, $H\to c\bar{c}$, $t\to bqq^{\prime}$, etc., but is
missing from most of the existing datasets.

Training, validation and test sets. The training set consists of 100 M jets in
total, equally distributed in the 10 classes. An additional set of 500 k jets
per class (in total 5 M) is intended for model validation. For the evaluation
of performance, a separate test set with 2 M jets in each class (in total 20
M) is provided.

Evaluation metrics. To thoroughly evaluate the performance of deep learning
models on this dataset, we advocate for a series of metrics. Since jet tagging
on this dataset is naturally framed as a multi-class classification task, two
common metrics, i.e., the accuracy and the area under the ROC curve
(AUC)333The AUC can be calculated using roc_auc_score in scikit-learn with
average=’macro’ and multi_class=’ovo’. are adopted to quantify the overall
performance. In addition, we propose the background rejection (i.e., the
inverse of the false positive rate) at a certain signal efficiency (i.e., the
true positive rate, TPR) of $X$%, i.e.,

$\text{Rej}_{X\%}\equiv 1/\text{FPR}\text{ at TPR}=X\%,$ (1)

for each type of signal jets. By default, the $q/g$ jets are considered as the
background, as is the case in most LHC data analyses, and each of the other 9
types of jets can be considered as the signal. The signal efficiency (TPR) for
each signal type is chosen to be representative of actual usages at the LHC
experiments and is typically 50%. It is increased to 99% (99.5%) for
$H\to\ell\nu qq^{\prime}$ ($t\to b\ell\nu$), as these types of jets have more
distinct characteristics and can be more easily separated from $q/g$ jets.
Since the definition of the $\text{Rej}_{X}$ metric involves only two classes,
i.e., the signal class under consideration ($S$) and the background class
($B$), the TPR and FPR are evaluated using a two-class score,

$\text{score}_{S\text{vs}B}\equiv\frac{\text{score}(S)}{\text{score}(S)+\text{score}(B)},$
(2)

where $\text{score}(S)$ and $\text{score}(B)$ are the softmax outputs for
class $S$ and $B$, respectively, to achieve optimal performance for $S$ vs $B$
separation. This is aligned with the convention adopted by the CMS experiment
(CMS Collaboration, 2020b). Note that the background rejection metric,
although rarely used in vision or language tasks, is actually a standard
metric for jet tagging because it is directly related to the discovery
potential at the LHC experiments. A factor of two increase in background
rejection can lead to about 40% increase in the discovery potential, which
would otherwise require a dataset of twice the size, or in other words,
doubling the running time of the LHC.

##  3 Related Work

Jet tagging with deep learning. Deep learning approaches have been proposed
extensively to improve jet tagging. Previous models handle jets with different
representations, e.g., images (de Oliveira et al., 2016), sequences (Guest et
al., 2016), trees (Louppe et al., 2019), graphs (Henrion et al., 2017), with
corresponding deep learning architectures such as 2D CNNs, recurrent or
recursive networks, and graph neural networks. More recently, the particle
cloud representation (Komiske et al., 2019b; Qu & Gouskos, 2020), analogous to
point clouds, which treats a jet as a permutation-invariant set of particles
as visualized in Figure 2, has been proposed. The Deep Sets (Zaheer et al.,
2017) and Dynamic Graph CNN (Wang et al., 2019) architectures are adapted for
jet tagging, resulting in the Energy Flow Network (Komiske et al., 2019b) and
the state-of-the-art, ParticleNet (Qu & Gouskos, 2020), respectively. Since
then, particle clouds have become the prevailing representation of jets and
more architectures based on GAPNet (Chen et al., 2021; Mikuni & Canelli,
2020), the Point Cloud Transformer (Guo et al., 2021; Mikuni & Canelli, 2021)
have been studied, but no significant performance improvement over ParticleNet
has been reported. Lately, researches have been focused more on incorporating
inductive biases from physics principles in the architecture design, such as
the usage of the Lund jet plane (Dreyer et al., 2018; Dreyer & Qu, 2021;
Dreyer et al., 2021, 2022), the Lorentz group symmetry (Bogatskiy et al.,
2020; Gong et al., 2022), and the rotational symmetry (Shimmin, 2021; Dillon
et al., 2021).

Deep-learning-based jet tagging algorithms have been widely adopted in real-
world data analysis at the LHC. For example, the CMS Collaboration develops
the DeepAK8 (CMS Collaboration, 2020b) algorithm to tag jets arising from the
top quark or the Higgs, $W$, or $Z$ boson, using a 1D CNN following the ResNet
(He et al., 2016) architecture, and a significant increase in the discovery
potential for new heavy particles has been achieved (CMS Collaboration, 2021,
2022a). Moreover, using ParticleNet, CMS achieves the first observation of $Z$
boson decay to a pair of charm quarks at a hadron collider and obtains the
most stringent constraint on $H\to c\bar{c}$ decay (CMS Collaboration, 2022c).
ParticleNet is also used by CMS to probe the quartic interaction between the
Higgs and vector bosons, indirectly confirming its existence for the first
time (CMS Collaboration, 2022b). Clearly, advances in jet tagging play a vital
role in accelerating our understanding of elementary particles, the
fundamental building blocks of nature.

Jet tagging datasets. A number of datasets have been published so far to study
jet tagging:

  * •

Top quark tagging dataset (Kasieczka et al., 2019) proposed in Butter et al.
(2019), consisting of 2 M jets in 2 types ($t\to bqq^{\prime}$ and $q/g$) and
providing only the kinematic information.

  * •

Quark-gluon tagging dataset (Komiske et al., 2019a) proposed in Komiske et al.
(2019b), consisting of 2 M jets in 2 types (quark and gluon), and providing
both the kinematic and particle identification information.

  * •

Higgs boson tagging dataset (Duarte, 2019; Chen et al., 2022), containing 3.9
M $H\to b\bar{b}$ jets and 1.9 M $q/g$ jets, with all three categories of
information.

  * •

JetNet dataset (Kansal et al., 2021b) proposed in Kansal et al. (2021a),
containing ${\approx}500\,\mathrm{k}$ jets in 3 types: gluon, light quark, and
top quark, and providing only the kinematic information.

  * •

A multiclass dataset (Pierini et al., 2020) proposed in Moreno et al. (2020),
with 880 k jets in 5 classes: light quark, gluon, $W$ boson, $Z$ boson and top
quark and providing only the kinematic information.

Compared with existing datasets, the JetClass dataset is not only
substantially larger in size, but also more inclusive in terms of the types of
jets contained.

Transformers. Recent years have witnessed the enormous success of Transformer
models. Starting from natural language processing and then spreading to
computer vision, the original Transformer (Vaswani et al., 2017), as well as
its variants, e.g., BERT (Devlin et al., 2019), ViT (Dosovitskiy et al., 2021)
and Swin-Transformer (Liu et al., 2021), have refreshed the performance
records in various tasks, demonstrating the power of Transformer as a
universal architecture. Transformers, and the attention mechanism at its core,
have proved to be powerful for fundamental scientific problems as well. For
example, AlphaFold2 (Jumper et al., 2021), which reaches the state-of-the-art
performance in protein structure prediction, employs the attention mechanism.
In particular, adding a pair bias, derived from pairwise features, to the self
attention helps improve the model explainability.

##  4 Model Architecture

Figure 3: The architecture of (a) Particle Transformer (b) Particle Attention
Block (c) Class Attention Block.

Together with the JetClass dataset, we propose the Particle Transformer (ParT)
as a new baseline for jet tagging. An overview of the ParT architecture is
presented in Figure 3(a). For a jet with $N$ particles, ParT makes use of two
sets of inputs: the particle input includes a list of $C$ features for every
particle and forms an array of a shape $(N,C)$; the interaction444The term
interaction here refers to any feature involving a pair of particles, which
may or may not be related to the physical forces between them. input is a
matrix of $C^{\prime}$ features for every pair of particles, in a shape
$(N,N,C^{\prime})$. The particle and interaction inputs are each followed by
an MLP to project them to a $d$\- and $d^{\prime}$-dimensional embedding,
$\mathbf{x}^{0}\in\mathbb{R}^{N\times d}$ and
$\mathbf{U}\in\mathbb{R}^{N\times N\times d^{\prime}}$, respectively. Unlike
Transformers for NLP and vision, we do not add any ad-hoc positional
encodings, as the particles in a jet are permutation invariant. The spatial
information (i.e., the flying direction of each particle) is directly included
in the particle inputs. We feed the particle embedding $\mathbf{x}^{0}$ into a
stack of $L$ particle attention blocks to produce new embeddings,
$\mathbf{x}^{1},...,\mathbf{x}^{L}$ via multi-head self attention. The
interaction matrix $\mathbf{U}$ is used to augment the scaled dot-product
attention by adding it as a bias to the pre-softmax attention weights. The
same $\mathbf{U}$ is used for all the particle attention blocks. After that,
the last particle embedding $\mathbf{x}^{L}$ is fed into two class attention
blocks, and a global class token $x_{\text{class}}$ is used to extract
information for jet classification via attention to all the particles,
following the CaiT approach (Touvron et al., 2021). The class token is passed
to a single-layer MLP, followed by softmax, to produce the final
classification scores.

Remark. ParT can also be viewed as a graph neural network on a fully-connected
graph, in which each node corresponds to a particle, and the interactions are
the edge features.

Particle interaction features. While the ParT architecture is designed to be
able to process any kinds of pairwise interaction features, for this paper we
only consider a specific scenario in which the interaction features are
derived from the energy-momentum 4-vector, $p=(E,p_{x},p_{y},p_{z})$, of each
particle. This is the most general case for jet tagging, as the particle
4-vectors are available in every jet tagging task. Specifically, for a pair of
particles $a$, $b$ with 4-vectors $p_{a}$, $p_{b}$, we calculate the following
4 features:

$\displaystyle\begin{split}\Delta&=\sqrt{(y_{a}-y_{b})^{2}+(\phi_{a}-\phi_{b})^{2}},\\\
k_{\text{T}}&=\min(p_{\text{T},a},p_{\text{T},b})\Delta,\\\
z&=\min(p_{\text{T},a},p_{\text{T},b})/(p_{\text{T},a}+p_{\text{T},b}),\\\
m^{2}&=(E_{a}+E_{b})^{2}-\|\mathbf{p}_{a}+\mathbf{p}_{b}\|^{2},\end{split}$
(3)

where $y_{i}$ is the rapidity, $\phi_{i}$ is the azimuthal angle,
$p_{\text{T},i}=(p_{x,i}^{2}+p_{y,i}^{2})^{1/2}$ is the transverse momentum,
and $\mathbf{p}_{i}=(p_{x,i},p_{y,i},p_{z,i})$ is the momentum 3-vector and
$\|\cdot\|$ is the norm, for $i=a$, $b$. Since these variables typically have
a long-tail distribution, we take the logarithm and use $(\ln\Delta,\ln
k_{\text{T}},\ln z,\ln m^{2})$ as the interaction features for each particle
pair. The choice of this set of features is motivated by Dreyer & Qu (2021).

Particle attention block. A key component of ParT is the particle attention
block. As illustrated in Figure 3(b), the particle attention block consists of
two stages. The first stage includes a multi-head attention (MHA) module with
a LayerNorm (LN) layer both before and afterwards. The second stage is a
2-layer MLP, with an LN before each linear layer and GELU nonlinearity in
between. Residual connections are added after each stage. The overall block
structure is based on NormFormer (Shleifer et al., 2021), however, we replace
the standard MHA with P-MHA, an augmented version that can also exploit the
pairwise particle interactions directly. The P-MHA is computed as

$\text{P-MHA}(Q,K,V)=\text{SoftMax}(QK^{T}/\sqrt{d_{k}}+\mathbf{U})V,$ (4)

where $Q$, $K$ and $V$ are linear projections of the particle embedding
$\mathbf{x}^{l}$. Essentially, we add the interaction matrix $\mathbf{U}$ to
the pre-softmax attention weights. This allows P-MHA to incorporate particle
interaction features designed from physics principles and modify the dot-
product attention weights, thus increasing the expressiveness of the attention
mechanism.

Class attention block. As illustrated in Figure 3(c), the class attention
block has a similar structure as the particle attention block. However, unlike
in the particle attention block where we compute the self attention between
particles, here we compute the attention between a global class token
$x_{\text{class}}$ and all the particles using the standard MHA. Specifically,
the inputs to the MHA are

$\displaystyle\begin{split}Q&=W_{q}x_{\text{class}}+b_{q},\\\
K&=W_{k}\textbf{z}+b_{k},\\\ V&=W_{v}\textbf{z}+b_{v},\end{split}$ (5)

where $\textbf{z}=[x_{\text{class}},\mathbf{x}^{L}]$ is the concatenation of
the class token and the particle embedding after the last particle attention
block, $\mathbf{x}^{L}$.

Implementation. We implement the ParT model in PyTorch (Paszke et al., 2019).
Specifically, the P-MHA is implemented using the PyTorch’s MultiheadAttention
by providing the interaction matrix $\mathbf{U}$ as the attn_mask input. The
baseline ParT model has a total of $L=8$ particle attention blocks and 2 class
attention blocks. It uses a particle embedding of a dimension $d=128$, encoded
from the input particle features using a 3-layer MLP with (128, 512, 128)
nodes each layer with GELU nonlinearity, and LN is used in between for
normalization. The interaction input features are encoded using a 4-layer
pointwise 1D convolution with (64, 64, 64, 16) channels with GELU nonlinearity
and batch normalization in between to yield a $d^{\prime}=16$ dimensional
interaction matrix. The P-MHA (MHA) in the particle (class) attention blocks
all have 8 heads, with a query dimension $d^{\prime}=16$ for each head, and an
expansion factor of 4 for the MLP. We use a dropout of 0.1 for all particle
attention blocks, and no dropout for the class attention block. The choice of
hyperparameters provides a reasonable baseline but is not extensively
optimized.

##  5 Experiments

We conduct experiments on the new JetClass dataset and show the results in
Section 5.1. The pre-trained ParT models are also applied to two existing
datasets with fine-tuning, and the performance is compared to previous state-
of-the-arts in Section 5.2.

###  5.1 Experiments on JetClass Dataset

Table 1: Jet tagging performance on the JetClass dataset. ParT is compared to
PFN (Komiske et al., 2019b), P-CNN (CMS Collaboration, 2020b) and the state-
of-the-art ParticleNet (Qu & Gouskos, 2020). For all the metrics, a higher
value indicates better performance. The ParT architecture using plain MHAs
instead of P-MHAs, labelled as ParT (plain), is also shown for comparison.

| All classes | $H\to b\bar{b}$ | $H\to c\bar{c}$ | $H\to gg$ | $H\to 4q$ | $H\to\ell\nu qq^{\prime}$ | $t\to bqq^{\prime}$ | $t\to b\ell\nu$ | $W\to qq^{\prime}$ | $Z\to q\bar{q}$
---|---|---|---|---|---|---|---|---|---|---
| Accuracy | AUC | $\text{Rej}_{50\%}$ | $\text{Rej}_{50\%}$ | $\text{Rej}_{50\%}$ | $\text{Rej}_{50\%}$ | $\text{Rej}_{99\%}$ | $\text{Rej}_{50\%}$ | $\text{Rej}_{99.5\%}$ | $\text{Rej}_{50\%}$ | $\text{Rej}_{50\%}$
PFN | 0.772 | 0.9714 | 2924 | 841 | 75 | 198 | 265 | 797 | 721 | 189 | 159
P-CNN | 0.809 | 0.9789 | 4890 | 1276 | 88 | 474 | 947 | 2907 | 2304 | 241 | 204
ParticleNet | 0.844 | 0.9849 | 7634 | 2475 | 104 | 954 | 3339 | 10526 | 11173 | 347 | 283
ParT | 0.861 | 0.9877 | 10638 | 4149 | 123 | 1864 | 5479 | 32787 | 15873 | 543 | 402
ParT (plain) | 0.849 | 0.9859 | 9569 | 2911 | 112 | 1185 | 3868 | 17699 | 12987 | 384 | 311

Setup. For experiments on the JetClass dataset, we use the full set of
particle features, including kinematics, particle identification, and
trajectory displacement, as inputs. The full list of 17 features for each
particle is summarized in Table 2. In addition, the 4 interaction features
introduced in Equation 3 are also used for the ParT model. The training is
performed on the full training set of 100 M jets. We employ the Lookahead
optimizer (Zhang et al., 2019) with $k=6$ and $\alpha=0.5$ to minimize the
cross-entropy loss, and the inner optimizer is RAdam (Liu et al., 2020) with
$\beta_{1}=0.95$, $\beta_{2}=0.999$, and $\epsilon=10^{{-}5}$. A batch size of
512 and an initial learning rate (LR) of 0.001 are used. No weight decay is
applied. We train for a total of 1 M iterations, amounting to around 5 epochs
over the full training set. The LR remains constant for the first 70% of the
iterations, and then decays exponentially, at an interval of every 20 k
iterations, down to 1% of the initial value at the end of the training.
Performance of the model is evaluated every 20 k iterations on the validation
set and a model checkpoint is saved. The checkpoint with the highest accuracy
on the validation set is used to evaluate the final performance on the test
set.

Table 2: Particle input features used for jet tagging on the JetClass, the top
quark tagging (Top) and the quark gluon tagging (QG) datasets. For QG, we
consider two scenarios: QG${}_{\text{exp}}$ is restricted to use only the
5-class experimentally realistic particle identification information, while
QG${}_{\text{full}}$ uses the full set of particle identification information
in the dataset and further distinguish between different types of charged
hadrons and neutral hadrons.

Category | Variable | Definition | JetClass | Top | QG${}_{\text{exp}}$ | QG${}_{\text{full}}$
---|---|---|---|---|---|---
Kinematics | $\Delta\eta$ | difference in pseudorapidity $\eta$ between the particle and the jet axis | ✓ | ✓ | ✓ | ✓
$\Delta\phi$ | difference in azimuthal angle $\phi$ between the particle and the jet axis | ✓ | ✓ | ✓ | ✓
$\log p_{\text{T}}$ | logarithm of the particle’s transverse momentum $p_{\text{T}}$ | ✓ | ✓ | ✓ | ✓
$\log E$ | logarithm of the particle’s energy | ✓ | ✓ | ✓ | ✓
$\log\frac{p_{\text{T}}}{p_{\text{T}}(\text{jet})}$ | logarithm of the particle’s $p_{\text{T}}$ relative to the jet $p_{\text{T}}$ | ✓ | ✓ | ✓ | ✓
$\log\frac{E}{E(\text{jet})}$ | logarithm of the particle’s energy relative to the jet energy | ✓ | ✓ | ✓ | ✓
$\Delta R$ | angular separation between the particle and the jet axis ($\sqrt{(\Delta\eta)^{2}+(\Delta\phi)^{2}}$) | ✓ | ✓ | ✓ | ✓
Particle identification | charge | electric charge of the particle | ✓ | — | ✓ | ✓
Electron | if the particle is an electron (|pid|==11) | ✓ | — | ✓ | ✓
Muon | if the particle is an muon (|pid|==13) | ✓ | — | ✓ | ✓
Photon | if the particle is an photon (pid==22) | ✓ | — | ✓ | ✓
CH | if the particle is an charged hadron (|pid|==211 or 321 or 2212) | ✓ | — | ✓ | ✓555(|pid|==211) + (|pid|==321)*0.5 + (|pid|==2212)*0.2
NH | if the particle is an neutral hadron (|pid|==130 or 2112 or 0) | ✓ | — | ✓ | ✓666(|pid|==130) + (|pid|==2112)*0.2.
Trajectory displacement | $\tanh d_{0}$ | hyperbolic tangent of the transverse impact parameter value | ✓ | — | — | —
$\tanh d_{z}$ | hyperbolic tangent of the longitudinal impact parameter value | ✓ | — | — | —
$\sigma_{d_{0}}$ | error of the measured transverse impact parameter | ✓ | — | — | —
$\sigma_{d_{z}}$ | error of the measured longitudinal impact parameter | ✓ | — | — | —

Baselines. We compare the performance of ParT with 3 baseline models: the PFN
(Komiske et al., 2019b) architecture based on Deep Sets (Zaheer et al., 2017),
the P-CNN architecture used by the DeepAK8 algorithm of the CMS experiment
(CMS Collaboration, 2020b), and the state-of-the-art ParticleNet architecture
(Qu & Gouskos, 2020) adapted from DGCNN (Wang et al., 2019). All the models
are trained end-to-end on the JetClass dataset for the same number of
effective epochs for a direct comparison. For ParticleNet, we directly use the
existing PyTorch implementation. For PFN and P-CNN, we re-implement them in
PyTorch and verify that the published results are reproduced. The optimizer
and LR schedule remain the same as in the training of ParT. The (batch size,
LR) combination is re-optimized and chosen to be (512, 0.01) for ParticleNet
and (4096, 0.02) for PFN and P-CNN.

Results. Performance on the JetClass dataset is evaluated using the metrics
described in Section 2, and the results are summarized in Table 1. The
proposed ParT architecture achieves the best performance on every metric, and
outperforms the existing state-of-the-art, ParticleNet, by a large margin. The
overall accuracy is increased by 1.7% compared to ParticleNet. Moreover, for
the physics-oriented metric, the background rejection, ParT improves over
ParticleNet by a factor of 3 for $t\to bqq^{\prime}$, a factor of 2 for $H\to
4q$, and about 70% for $H\to c\bar{c}$. It is also clear that, the earlier PFN
and P-CNN models lag substantially behind ParticleNet and ParT on this large
dataset, amounting to up to an order of magnitude difference in background
rejection. The large improvement of ParT is likely to lead to a significant
jump in the discovery potential for related physics searches at the LHC.

Another observation is that there is a large variation in tagging performance
between signals of different types. The best separation against the background
$q/g$ jets is achieved for $t\to b\ell\nu$ and $H\to\ell\nu qq^{\prime}$
signals – with the powerful ParT model, these two can be selected almost
perfectly, i.e., at an efficiency of more than 99% with nearly no
contamination from background jets. This opens up new territory for jet
tagging at the LHC, as these types of jets have not been exploited for tagging
so far.

Effectiveness of P-MHA. To quantify the effectiveness of the P-MHA introduced
in ParT, we carry out an ablation study by replacing the P-MHA with a standard
MHA, the resulting architecture is then a plain Transformer and therefore
denoted as ParT (plain). We train ParT (plain) with the same procedure as the
full ParT and the performance is shown in Table 1. A drop of 1.2% in accuracy
is observed compared to the full ParT, and the background rejection is reduced
by 20–30% for most signals. Note that, replacing P-MHA with plain MHA implies
that the particle interaction input is discarded completely, but this does not
lead to any reduction of information content, as the interaction features
defined in Equation 3 are derived purely from the energy-momentum 4-vectors,
which are already used as particle features via the 7 kinematic variables
presented in Table 2. Therefore, the improvement of ParT over a plain
Transformer indeed arise from an efficient exploitation of the particle
kinematic information using the P-MHA.

Table 3: Impacts of the training dataset size. Entries in bold correspond to
the training using the full 100 M training dataset.

| All classes | $H\to b\bar{b}$ | $H\to c\bar{c}$ | $H\to gg$ | $H\to 4q$ | $H\to\ell\nu qq^{\prime}$ | $t\to bqq^{\prime}$ | $t\to b\ell\nu$ | $W\to qq^{\prime}$ | $Z\to q\bar{q}$
---|---|---|---|---|---|---|---|---|---|---
| Accuracy | AUC | $\text{Rej}_{50\%}$ | $\text{Rej}_{50\%}$ | $\text{Rej}_{50\%}$ | $\text{Rej}_{50\%}$ | $\text{Rej}_{99\%}$ | $\text{Rej}_{50\%}$ | $\text{Rej}_{99.5\%}$ | $\text{Rej}_{50\%}$ | $\text{Rej}_{50\%}$
ParticleNet (2 M) | 0.828 | 0.9820 | 5540 | 1681 | 90 | 662 | 1654 | 4049 | 4673 | 260 | 215
ParticleNet (10 M) | 0.837 | 0.9837 | 5848 | 2070 | 96 | 770 | 2350 | 5495 | 6803 | 307 | 253
ParticleNet (100 M) | 0.844 | 0.9849 | 7634 | 2475 | 104 | 954 | 3339 | 10526 | 11173 | 347 | 283
ParT (2 M) | 0.836 | 0.9834 | 5587 | 1982 | 93 | 761 | 1609 | 6061 | 4474 | 307 | 236
ParT (10 M) | 0.850 | 0.9860 | 8734 | 3040 | 110 | 1274 | 3257 | 12579 | 8969 | 431 | 324
ParT (100 M) | 0.861 | 0.9877 | 10638 | 4149 | 123 | 1864 | 5479 | 32787 | 15873 | 543 | 402

Impacts of the training dataset size. To evaluate the impacts of the training
dataset size on the jet tagging performance, we perform additional trainings
using only 2% and 10% of the JetClass dataset. For the former, the training is
performed for only 100 k iterations, as it is already converged by then. For
the latter, the training still lasts for 1 M iterations, although very little
gain is observed compared to the training with only 100 k iterations. No
overfitting is found in either case. The results are summarized in Table 3.
For the ParticleNet model, a drop of 0.7% in accuracy is observed when the
training dataset size is reduced to 10 M, and the drop in accuracy increases
to 1.6% when only 2 M jets are used in the training. For the ParT model, the
impact is even larger, the degradation in accuracy becomes 1.1% and 2.5% when
the training dataset is reduced to 10% and 2%, respectively.

Model complexity. Table 4 compares the model complexity of ParT with the
baselines. While the number of trainable parameters is increased by more than
$5\times$ compared to ParticleNet, the number of floating point operations
(FLOPs) is actually 40% lower. We also observe that the FLOPs of ParT are 30%
higher than ParT (plain), which mostly comes from the encoding of the pairwise
features, because the computational cost there scales quadratically with the
number of particles in a jet.

Table 4: Number of trainable parameters and FLOPs.

| Accuracy | # params | FLOPs
---|---|---|---
PFN | 0.772 | 86.1 k | 4.62 M
P-CNN | 0.809 | 354 k | 15.5 M
ParticleNet | 0.844 | 370 k | 540 M
ParT | 0.861 | 2.14 M | 340 M
ParT (plain) | 0.849 | 2.13 M | 260 M

###  5.2 Fine-Tuning for Other Datasets

Top quark tagging dataset. The top quark tagging benchmark (Butter et al.,
2019) provides a dataset of 2 M (1.2/0.4/0.4 M for train/validation/test) jets
in two classes, $t\to bqq^{\prime}$ (signal) and $q/g$ (background). Only
kinematic features, i.e., the energy-momentum 4-vectors, are provided.
Therefore, we pre-train a ParT model on the JetClass dataset using only the
kinematic features, and then fine-tune it on the top quark tagging dataset.
The particle input features are the 7 kinematic features listed in Table 2,
the same as used by ParticleNet. The JetClass pre-training follows the same
setup as described in Section 5.1. For the fine-tuning, we replace the last
MLP with a new randomly-initialized MLP with 2 output nodes, and then fine-
tune all the weights on the top tagging dataset for 20 epochs. A smaller LR of
0.0001 is used for the pre-trained weights, while a larger LR of 0.005 is used
to update the randomly-initialized weights of the MLP. The LR remains constant
across the full training, with a weight decay of 0.01. We run a total of 9
experiments, starting from the same pre-trained model but different random
initializations of the replaced MLP, and report the performance of the model
with median accuracy and the spread across the 9 trainings, following the
procedure used by ParticleNet. For comparison, we also train ParT from scratch
on this dataset for 20 epochs, using a start LR of 0.001, a schedule that
decays the LR to 1% in the last 30% of the epochs, and a weight decay of 0.01.
Both results are presented in Table 5. The pre-trained ParT achieves a
significant improvement over the existing baselines, increasing
$\text{Rej}_{30\%}$ by 70% compared to ParticleNet, and by 26% compared to the
best-performing model on this dataset, LorentzNet. On the other hand, the ParT
model trained from scratch only reaches similar performance as ParticleNet. We
also investigate a similar pre-training and fine-tuning procedure using the
ParticleNet model, but only a small improvement is observed compared to the
training from scratch, due to the limited capacity of the ParticleNet model.

Table 5: Comparison between ParT and existing models on the top quark tagging
dataset. ParT refers to the model trained from scratch on this dataset.
ParticleNet-f.t. and ParT-f.t. denote the corresponding models pre-trained on
JetClass and fine-tuned on this dataset. Results for other models are quoted
from their published results: P-CNN and ParticleNet (Qu & Gouskos, 2020), PFN
(Komiske et al., 2019b), JEDI-net (Moreno et al., 2020), PCT (Mikuni &
Canelli, 2021), LGN (Bogatskiy et al., 2020), rPCN (Shimmin, 2021), and
LorentzNet (Gong et al., 2022).

| Accuracy | AUC | $\text{Rej}_{50\%}$ | $\text{Rej}_{30\%}$
---|---|---|---|---
P-CNN | 0.930 | 0.9803 | $201\pm 4$ | $759\pm 24$
PFN | — | 0.9819 | $247\pm 3$ | $888\pm 17$
ParticleNet | 0.940 | 0.9858 | $397\pm 7$ | $1615\pm 93$
JEDI-net (w/ $\sum O$) | 0.930 | 0.9807 | — | 774.6
PCT | 0.940 | 0.9855 | $392\pm 7$ | $1533\pm 101$
LGN | 0.929 | 0.964 | — | $435\pm 95$
rPCN | — | 0.9845 | $364\pm 9$ | $1642\pm 93$
LorentzNet | 0.942 | 0.9868 | $498\pm 18$ | $2195\pm 173$
ParT | 0.940 | 0.9858 | $413\pm 16$ | $1602\pm 81$
ParticleNet-f.t. | 0.942 | 0.9866 | $487\pm 9$ | $1771\pm 80$
ParT-f.t. | 0.944 | 0.9877 | $\bf 691\pm 15$ | $\bf 2766\pm 130$

Quark-gluon tagging dataset. We also benchmark ParT on the quark-gluon tagging
dataset (Komiske et al., 2019a) proposed in Komiske et al. (2019b), the target
of which is to separate jets initiated by quarks (signal) from those by gluons
(background). This dataset also consists of 2 M jets, with a recommended
train/validation/test splitting of 1.6/0.2/0.2 M. It provides not only the
kinematic features, but also particle identification information. We consider
two scenarios in the usage of the particle identification information. In the
“exp” scenario, we restrict the information to only 5 classes and do not
attempt to separate electrically charged (and neural) hadrons of different
types, which is the procedure adopted by ParticleNet, and also prescribed by
the JetClass dataset. In the “full” scenario, we consider all particle types
and further distinguish electrically charged (and neural) hadrons into more
types, such as pions, kaons, and protons. We perform the pre-training on
JetClass using only kinematic and particle identification inputs under the
“exp” scenario. For the fine-tuning, we then carry out experiments in both
scenarios. The construction of the input features is described in Table 2. The
pre-training and fine-tuning setup is the same as in the top quark tagging
benchmark, and the fine-tuning also lasts for 20 epochs. Results are
summarized in Table 6. The pre-trained ParT achieves the best performance and
improves existing baselines by a large margin in both scenarios.

Table 6: Comparison between ParT and existing models on the quark-gluon
tagging dataset. ParT refers to the model trained from scratch on this
dataset. ParticleNet-f.t. and ParT-f.t. denote the corresponding models pre-
trained on JetClass and fine-tuned on this dataset. Results for other models
are quoted from their published results: P-CNN and ParticleNet (Qu & Gouskos,
2020), PFN (Komiske et al., 2019b), ABCNet (Mikuni & Canelli, 2020), PCT
(Mikuni & Canelli, 2021), rPCN (Shimmin, 2021), and LorentzNet (Gong et al.,
2022). The subscript “exp” and “full” distinguish models using partial or full
particle identification information.

| Accuracy | AUC | $\text{Rej}_{50\%}$ | $\text{Rej}_{30\%}$
---|---|---|---|---
P-CNN${}_{\text{exp}}$ | 0.827 | 0.9002 | 34.7 | 91.0
PFN${}_{\text{exp}}$ | — | 0.9005 | $34.7\pm 0.4$ | —
ParticleNet${}_{\text{exp}}$ | 0.840 | 0.9116 | $39.8\pm 0.2$ | $98.6\pm 1.3$
rPCN${}_{\text{exp}}$ | — | 0.9081 | $38.6\pm 0.5$ | —
ParT${}_{\text{exp}}$ | 0.840 | 0.9121 | $41.3\pm 0.3$ | $101.2\pm 1.1$
ParticleNet-f.t.${}_{\text{exp}}$ | 0.839 | 0.9115 | $40.1\pm 0.2$ | $100.3\pm 1.0$
ParT-f.t.${}_{\text{exp}}$ | 0.843 | 0.9151 | $\bf 42.4\pm 0.2$ | $\bf 107.9\pm 0.5$
PFN${}_{\text{full}}$ | — | 0.9052 | $37.4\pm 0.7$ | —
ABCNet${}_{\text{full}}$ | 0.840 | 0.9126 | $42.6\pm 0.4$ | $118.4\pm 1.5$
PCT${}_{\text{full}}$ | 0.841 | 0.9140 | $43.2\pm 0.7$ | $118.0\pm 2.2$
LorentzNet${}_{\text{full}}$ | 0.844 | 0.9156 | $42.4\pm 0.4$ | $110.2\pm 1.3$
ParT${}_{\text{full}}$ | 0.849 | 0.9203 | $47.9\pm 0.5$ | $129.5\pm 0.9$
ParT-f.t.${}_{\text{full}}$ | 0.852 | 0.9230 | $\bf 50.6\pm 0.2$ | $\bf 138.7\pm 1.3$

##  6 Discussion and Conclusion

Large-scale datasets have always been a catalyst for new breakthroughs in deep
learning. In this work, we present JetClass, a new large-scale open dataset to
advance deep learning research in particle physics. The dataset consists of
100 M simulated jets, about two orders of magnitude larger than existing
public jet datasets, and covers a broad spectrum of 10 classes of jets in
total, including several novel types that have not been studied with deep
learning so far. While we focus on investigating a classification task, i.e.,
jet tagging, with this dataset, we highlight that this dataset can serve as
the basis for many important deep learning researches in particle physics,
e.g., unsupervised or self-supervised training techniques for particle physics
(e.g., Dillon et al. (2021)), generative models for high-fidelity fast
simulation of particle collisions (e.g., Kansal et al. (2021a)), regression
models to predict jet energy and momentum with higher precision (e.g., CMS
Collaboration (2020a)), and more. We invite the community to explore and
experiment with this dataset and extend the boundary of deep learning and
particle physics even further.

With this large dataset, we introduce Particle Transformer (ParT), a new
architecture that substantially improves jet tagging performance over previous
state-of-the-art. We propose it as a new jet tagging baseline for future
research to improve upon. The effectiveness of ParT arises mainly from the
augmented self-attention, in which we incorporate physics-inspired pairwise
interactions together with the machine-learned dot-product attention. This
approach is likely to be effective for other tasks on similar datasets, such
as point clouds or many-body systems, especially when prior knowledge is
available to describe the interaction or the geometry. On the other hand, one
limitation of using the full pairwise interaction matrix is the increase in
computational time and memory consumption. Novel approaches for particle
(point) embeddings and self-attentions that alleviate the computational cost
(e.g., Zhou et al. (2021); Kitaev et al. (2020)) could be an interesting
direction for future research.

## Acknowledgements

We are grateful to Loukas Gouskos, Qiang Li, and Alexandre De Moor for many
helpful discussions. The work of C. Li and S. Qian is supported by National
Natural Science Foundation of China under Grants No. 12061141002.

## References

  * Alwall et al. (2014) Alwall, J., Frederix, R., Frixione, S., Hirschi, V., Maltoni, F., Mattelaer, O., Shao, H. S., Stelzer, T., Torrielli, P., and Zaro, M.  The automated computation of tree-level and next-to-leading order differential cross sections, and their matching to parton shower simulations.  _JHEP_ , 07:079, 2014.  doi: 10.1007/JHEP07(2014)079.
  * ATLAS Collaboration (2008) ATLAS Collaboration.  The ATLAS Experiment at the CERN Large Hadron Collider.  _JINST_ , 3:S08003, 2008.  doi: 10.1088/1748-0221/3/08/S08003.
  * ATLAS Collaboration (2012) ATLAS Collaboration.  Observation of a new particle in the search for the Standard Model Higgs boson with the ATLAS detector at the LHC.  _Phys. Lett. B_ , 716:1–29, 2012.  doi: 10.1016/j.physletb.2012.08.020.
  * Bogatskiy et al. (2020) Bogatskiy, A., Anderson, B., Offermann, J., Roussi, M., Miller, D., and Kondor, R.  Lorentz group equivariant neural network for particle physics.  In _Proceedings of the 37th International Conference on Machine Learning_ , volume 119 of _Proceedings of Machine Learning Research_ , pp. 992–1002. PMLR, 13–18 Jul 2020.
  * Butter et al. (2019) Butter, A. et al.  The Machine Learning landscape of top taggers.  _SciPost Phys._ , 7:014, 2019.  doi: 10.21468/SciPostPhys.7.1.014.
  * Cacciari et al. (2008) Cacciari, M., Salam, G. P., and Soyez, G.  The anti-$k_{t}$ jet clustering algorithm.  _JHEP_ , 04:063, 2008.  doi: 10.1088/1126-6708/2008/04/063.
  * Cacciari et al. (2012) Cacciari, M., Salam, G. P., and Soyez, G.  FastJet User Manual.  _Eur. Phys. J. C_ , 72:1896, 2012.  doi: 10.1140/epjc/s10052-012-1896-2.
  * Chen et al. (2021) Chen, C., Fragonara, L. Z., and Tsourdos, A.  Gapointnet: Graph attention based point neural network for exploiting local feature of point cloud.  _Neurocomputing_ , 438:122–132, 2021.  doi: https://doi.org/10.1016/j.neucom.2021.01.095.
  * Chen et al. (2022) Chen, Y., Huerta, E. A., Duarte, J., Harris, P., Katz, D. S., Neubauer, M. S., Diaz, D., Mokhtar, F., Kansal, R., Park, S. E., Kindratenko, V. V., Zhao, Z., and Rusack, R.  A FAIR and AI-ready Higgs boson decay dataset.  _Scientific Data_ , 9(1):31, 2022.  doi: 10.1038/s41597-021-01109-0.
  * CMS Collaboration (2008) CMS Collaboration.  The CMS Experiment at the CERN LHC.  _JINST_ , 3:S08004, 2008.  doi: 10.1088/1748-0221/3/08/S08004.
  * CMS Collaboration (2012) CMS Collaboration.  Observation of a New Boson at a Mass of 125 GeV with the CMS Experiment at the LHC.  _Phys. Lett. B_ , 716:30–61, 2012.  doi: 10.1016/j.physletb.2012.08.021.
  * CMS Collaboration (2014) CMS Collaboration.  Description and performance of track and primary-vertex reconstruction with the CMS tracker.  _JINST_ , 9(10):P10009, 2014.  doi: 10.1088/1748-0221/9/10/P10009.
  * CMS Collaboration (2020a) CMS Collaboration.  A deep neural network for simultaneous estimation of b jet energy and resolution.  _Comput. Softw. Big Sci._ , 4(1):10, 2020a.  doi: 10.1007/s41781-020-00041-z.
  * CMS Collaboration (2020b) CMS Collaboration.  Identification of heavy, energetic, hadronically decaying particles using machine-learning techniques.  _JINST_ , 15(06):P06005, 2020b.  doi: 10.1088/1748-0221/15/06/P06005.
  * CMS Collaboration (2021) CMS Collaboration.  Search for top squark production in fully-hadronic final states in proton-proton collisions at $\sqrt{s}=$ 13 TeV.  _Phys. Rev. D_ , 104(5):052001, 2021.  doi: 10.1103/PhysRevD.104.052001.
  * CMS Collaboration (2022a) CMS Collaboration.  Search for resonances decaying to three W bosons in proton-proton collisions at $\sqrt{s}$ = 13 TeV.  1 2022a.  Accepted for publication in Phys. Rev. Lett.
  * CMS Collaboration (2022b) CMS Collaboration.  Search for nonresonant pair production of highly energetic Higgs bosons decaying to bottom quarks.  2022b.  Submitted to Phys. Rev. Lett.
  * CMS Collaboration (2022c) CMS Collaboration.  Search for Higgs boson decay to a charm quark-antiquark pair in proton-proton collisions at $\sqrt{s}$ = 13 TeV.  2022c.  Submitted to Phys. Rev. Lett.
  * de Favereau et al. (2014) de Favereau, J., Delaere, C., Demin, P., Giammanco, A., Lemaître, V., Mertens, A., and Selvaggi, M.  DELPHES 3, A modular framework for fast simulation of a generic collider experiment.  _JHEP_ , 02:057, 2014.  doi: 10.1007/JHEP02(2014)057.
  * de Oliveira et al. (2016) de Oliveira, L., Kagan, M., Mackey, L., Nachman, B., and Schwartzman, A.  Jet-images — deep learning edition.  _JHEP_ , 07:069, 2016.  doi: 10.1007/JHEP07(2016)069.
  * Devlin et al. (2019) Devlin, J., Chang, M.-W., Lee, K., and Toutanova, K.  BERT: Pre-training of deep bidirectional transformers for language understanding.  In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)_ , pp. 4171–4186, Minneapolis, Minnesota, June 2019.  doi: 10.18653/v1/N19-1423.
  * Dillon et al. (2021) Dillon, B. M., Kasieczka, G., Olischlager, H., Plehn, T., Sorrenson, P., and Vogel, L.  Symmetries, Safety, and Self-Supervision.  8 2021.
  * Dosovitskiy et al. (2021) Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., Uszkoreit, J., and Houlsby, N.  An image is worth 16x16 words: Transformers for image recognition at scale.  In _International Conference on Learning Representations_ , 2021.
  * Dreyer et al. (2021) Dreyer, F., Soyez, G., and Takacs, A.  Quarks and gluons in the Lund plane.  _arXiv preprint arXiv:2112.09140_ , 12 2021.
  * Dreyer & Qu (2021) Dreyer, F. A. and Qu, H.  Jet tagging in the Lund plane with graph networks.  _JHEP_ , 03:052, 2021.  doi: 10.1007/JHEP03(2021)052.
  * Dreyer et al. (2018) Dreyer, F. A., Salam, G. P., and Soyez, G.  The Lund Jet Plane.  _JHEP_ , 12:064, 2018.  doi: 10.1007/JHEP12(2018)064.
  * Dreyer et al. (2022) Dreyer, F. A., Grabarczyk, R., and Monni, P. F.  Leveraging universality of jet taggers through transfer learning.  _arXiv preprint arXiv:2203.06210_ , 3 2022.
  * Duarte (2019) Duarte, J.  Sample with jet, track and secondary vertex properties for Hbb tagging ML studies HiggsToBBNTuple_HiggsToBB_QCD_RunII_13TeV_MC, 2019\.  URL http://opendata.cern.ch/record/12102.
  * Gong et al. (2022) Gong, S., Meng, Q., Zhang, J., Qu, H., Li, C., Qian, S., Du, W., Ma, Z.-M., and Liu, T.-Y.  An Efficient Lorentz Equivariant Graph Neural Network for Jet Tagging.  _arXiv preprint arXiv:2201.08187_ , 1 2022.
  * Guest et al. (2016) Guest, D., Collado, J., Baldi, P., Hsu, S.-C., Urban, G., and Whiteson, D.  Jet Flavor Classification in High-Energy Physics with Deep Neural Networks.  _Phys. Rev. D_ , 94(11):112002, 2016.  doi: 10.1103/PhysRevD.94.112002.
  * Guo et al. (2021) Guo, M.-H., Cai, J.-X., Liu, Z.-N., Mu, T.-J., Martin, R. R., and Hu, S.-M.  PCT: Point cloud transformer.  _Computational Visual Media_ , 7(2):187–199, June 2021.  doi: 10.1007/s41095-021-0229-5.
  * He et al. (2016) He, K., Zhang, X., Ren, S., and Sun, J.  Deep residual learning for image recognition.  In _2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)_ , pp. 770–778, 2016.  doi: 10.1109/CVPR.2016.90.
  * Henrion et al. (2017) Henrion, I., Brehmer, J., Bruna, J., Cho, K., Cranmer, K., Louppe, G., and Rochette, G.  Neural Message Passing for Jet Physics.  In _Deep Learning for Physical Sciences Workshop at the 31st Conference on Neural Information Processing Systems (NeurIPS)_ , 2017.
  * Jumper et al. (2021) Jumper, J., Evans, R., Pritzel, A., Green, T., Figurnov, M., Ronneberger, O., Tunyasuvunakool, K., Bates, R., Žídek, A., Potapenko, A., Bridgland, A., Meyer, C., Kohl, S. A. A., Ballard, A. J., Cowie, A., Romera-Paredes, B., Nikolov, S., Jain, R., Adler, J., Back, T., Petersen, S., Reiman, D., Clancy, E., Zielinski, M., Steinegger, M., Pacholska, M., Berghammer, T., Bodenstein, S., Silver, D., Vinyals, O., Senior, A. W., Kavukcuoglu, K., Kohli, P., and Hassabis, D.  Highly accurate protein structure prediction with AlphaFold.  _Nature_ , 596(7873):583–589, 2021.  doi: 10.1038/s41586-021-03819-2.
  * Kansal et al. (2021a) Kansal, R., Duarte, J., Su, H., Orzari, B., Tomei, T., Pierini, M., Touranakou, M., Vlimant, J.-R., and Gunopulos, D.  Particle cloud generation with message passing generative adversarial networks.  In _Advances in Neural Information Processing Systems_ , volume 34, pp. 23858–23871, 2021a.
  * Kansal et al. (2021b) Kansal, R., Duarte, J., Su, H., Orzari, B., Tomei, T., Pierini, M., Touranakou, M., Vlimant, J.-R., and Gunopulos, D.  Jetnet, May 2021b.  URL https://doi.org/10.5281/zenodo.5502543.
  * Kasieczka et al. (2019) Kasieczka, G., Plehn, T., Thompson, J., and Russel, M.  Top Quark Tagging Reference Dataset, March 2019.
  * Kitaev et al. (2020) Kitaev, N., Kaiser, L., and Levskaya, A.  Reformer: The efficient transformer.  In _International Conference on Learning Representations_ , 2020.
  * Kogler et al. (2019) Kogler, R. et al.  Jet Substructure at the Large Hadron Collider: Experimental Review.  _Rev. Mod. Phys._ , 91(4):045003, 2019.  doi: 10.1103/RevModPhys.91.045003.
  * Komiske et al. (2019a) Komiske, P., Metodiev, E., and Thaler, J.  Pythia8 Quark and Gluon Jets for Energy Flow, may 2019a.
  * Komiske et al. (2019b) Komiske, P. T., Metodiev, E. M., and Thaler, J.  Energy Flow Networks: Deep Sets for Particle Jets.  _JHEP_ , 01:121, 2019b.  doi: 10.1007/JHEP01(2019)121.
  * Larkoski et al. (2020) Larkoski, A. J., Moult, I., and Nachman, B.  Jet Substructure at the Large Hadron Collider: A Review of Recent Advances in Theory and Machine Learning.  _Phys. Rept._ , 841:1–63, 2020.  doi: 10.1016/j.physrep.2019.11.001.
  * Liu et al. (2020) Liu, L., Jiang, H., He, P., Chen, W., Liu, X., Gao, J., and Han, J.  On the variance of the adaptive learning rate and beyond.  In _International Conference on Learning Representations_ , 2020.
  * Liu et al. (2021) Liu, Z., Lin, Y., Cao, Y., Hu, H., Wei, Y., Zhang, Z., Lin, S., and Guo, B.  Swin transformer: Hierarchical vision transformer using shifted windows.  In _Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)_ , pp. 10012–10022, October 2021.
  * Louppe et al. (2019) Louppe, G., Cho, K., Becot, C., and Cranmer, K.  QCD-Aware Recursive Neural Networks for Jet Physics.  _JHEP_ , 01:057, 2019.  doi: 10.1007/JHEP01(2019)057.
  * Mikuni & Canelli (2020) Mikuni, V. and Canelli, F.  ABCNet: An attention-based method for particle tagging.  _Eur. Phys. J. Plus_ , 135(6):463, 2020.  doi: 10.1140/epjp/s13360-020-00497-3.
  * Mikuni & Canelli (2021) Mikuni, V. and Canelli, F.  Point cloud transformers applied to collider physics.  _Mach. Learn. Sci. Tech._ , 2(3):035027, 2021\.  doi: 10.1088/2632-2153/ac07f6.
  * Moreno et al. (2020) Moreno, E. A., Cerri, O., Duarte, J. M., Newman, H. B., Nguyen, T. Q., Periwal, A. a., Pierini, M., Serikova, A., Spiropulu, M., and Vlimant, J.-R.  JEDI-net: a jet identification algorithm based on interaction networks.  _Eur. Phys. J. C_ , 80(1):58, 2020.  doi: 10.1140/epjc/s10052-020-7608-4.
  * Paszke et al. (2019) Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., Antiga, L., Desmaison, A., Kopf, A., Yang, E., DeVito, Z., Raison, M., Tejani, A., Chilamkurthy, S., Steiner, B., Fang, L., Bai, J., and Chintala, S.  Pytorch: An imperative style, high-performance deep learning library.  In _Advances in Neural Information Processing Systems 32_ , pp. 8024–8035. 2019.
  * Pierini et al. (2020) Pierini, M., Duarte, J. M., Tran, N., and Freytsis, M.  Hls4ml lhc jet dataset (150 particles), January 2020.  URL https://doi.org/10.5281/zenodo.3602260.
  * Qu & Gouskos (2020) Qu, H. and Gouskos, L.  ParticleNet: Jet Tagging via Particle Clouds.  _Phys. Rev. D_ , 101(5):056019, 2020.  doi: 10.1103/PhysRevD.101.056019.
  * Qu et al. (2022) Qu, H., Li, C., and Qian, S.  JetClass: A large-scale dataset for deep learning in jet physics, June 2022.  URL https://doi.org/10.5281/zenodo.6619768.
  * Radovic et al. (2018) Radovic, A., Williams, M., Rousseau, D., Kagan, M., Bonacorsi, D., Himmel, A., Aurisano, A., Terao, K., and Wongjirad, T.  Machine learning at the energy and intensity frontiers of particle physics.  _Nature_ , 560(7716):41–48, 2018.  doi: 10.1038/s41586-018-0361-2.
  * Shimmin (2021) Shimmin, C.  Particle Convolution for High Energy Physics.  _arXiv preprint arXiv:2107.02908_ , 7 2021.
  * Shleifer et al. (2021) Shleifer, S., Weston, J., and Ott, M.  Normformer: Improved transformer pretraining with extra normalization.  _arXiv preprint arXiv:2110.09456_ , 2021.
  * Sjöstrand et al. (2015) Sjöstrand, T., Ask, S., Christiansen, J. R., Corke, R., Desai, N., Ilten, P., Mrenna, S., Prestel, S., Rasmussen, C. O., and Skands, P. Z.  An introduction to PYTHIA 8.2.  _Comput. Phys. Commun._ , 191:159–177, 2015.  doi: 10.1016/j.cpc.2015.01.024.
  * Touvron et al. (2021) Touvron, H., Cord, M., Sablayrolles, A., Synnaeve, G., and Jégou, H.  Going deeper with image transformers.  In _Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)_ , pp. 32–42, October 2021.
  * Vaswani et al. (2017) Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L. u., and Polosukhin, I.  Attention is all you need.  In _Advances in Neural Information Processing Systems_ , volume 30, 2017.
  * Wang et al. (2019) Wang, Y., Sun, Y., Liu, Z., Sarma, S. E., Bronstein, M. M., and Solomon, J. M.  Dynamic graph cnn for learning on point clouds.  _ACM Trans. Graph._ , 38(5), oct 2019.  doi: 10.1145/3326362.
  * Zaheer et al. (2017) Zaheer, M., Kottur, S., Ravanbakhsh, S., Poczos, B., Salakhutdinov, R. R., and Smola, A. J.  Deep sets.  In _Advances in Neural Information Processing Systems_ , volume 30, 2017.
  * Zhang et al. (2019) Zhang, M., Lucas, J., Ba, J., and Hinton, G. E.  Lookahead optimizer: k steps forward, 1 step back.  In _Advances in Neural Information Processing Systems_ , volume 32, 2019.
  * Zhou et al. (2021) Zhou, H., Zhang, S., Peng, J., Zhang, S., Li, J., Xiong, H., and Zhang, W.  Informer: Beyond efficient transformer for long sequence time-series forecasting.  In _AAAI Conference on Artificial Intelligence_ , volume 35, pp. 11106–11115, May 2021.