11institutetext:  INAF, Osservatorio di Astrofisica e Scienza dello Spazio,
via Piero Gobetti 93/3, 40129 Bologna, Italy 11email<EMAIL_ADDRESS>22institutetext: INFN, Sezione di Bologna, viale Berti Pichat 6/2, I-40127
Bologna, Italy 33institutetext: Department of Astronomy, University of Geneva,
Ch. d’Ecogia 16, CH-1290 Versoix, Switzerland

# Tracing the non-thermal pressure and hydrostatic bias
in galaxy clusters

S. Ettori 1122 D. Eckert 33

We present a modelization of the non-thermal pressure, $P_{NT}$, and we apply
it to the X-ray (and Sunayev-Zel’dovich) derived radial profiles of the X-COP
galaxy clusters. We relate the amount of non-thermal pressure support to the
hydrostatic bias, $b$, and speculate on how we can interpret this $P_{NT}$ in
terms of the expected levels of turbulent velocity and magnetic fields.
Current upper limits on the turbulent velocity in the intracluster plasma are
used to build a distribution $\mathcal{N}(<b)-b$, from which we infer that 50
per cent of local galaxy clusters should have $b<0.2$ ($b<0.33$ in 80 per cent
of the population). The measured bias in the X-COP sample that includes
relaxed massive nearby systems is 0.03 in 50% of the objects and 0.17 in 80%
of them. All these values are below the amount of bias required to reconcile
the observed cluster number count in the cosmological framework set from
Planck.

###### Key Words.:

Galaxies: clusters: intracluster medium - X-rays: galaxies: clusters -
galaxies: clusters: general - cosmology: dark matter

##  1 Introduction

Plasma in galaxy clusters and groups is an almost completely ionized gas, is a
generator of magnetic fields, and is subjected to turbulent motions. It is
expected to thermalize after the accretion into the potential well on a
timescale of the order of few gigayears, with a residual kinetic component
whose distribution and total amplitude are still unknown.

In this Letter, we propose a modelization of this residual non-thermal
pressure, which supports the gas in equilibrium within an assumed dark-matter
gravitation field, and apply the suggested technique to the observed
thermodynamic profiles recovered for the objects in our XMM-Newton Cluster
Outskirts Project (X-COP) sample (Eckert et al., 2017). We build the relations
between this non-thermal pressure and any magnetic fields and/or turbulent
velocity in the plasma and determine how the non-thermal pressure translates
into a hydrostatic bias on the mass measurement.

##  2 The hydrostatic equation, the non-thermal pressure, and the hydrostatic
bias

Euler’s equation for an ideal fluid (i.e. a fluid in which thermal
conductivity and viscosity do not play a relevant role) in a gravitational
potential $\phi$ and with a velocity $\@vec{v}$, pressure $P_{\rm gas}$, and
density $\rho_{\rm gas}$ is (e.g. Landau & Lifshitz, 1959; Suto et al., 2013)

$\frac{\partial\@vec{v}}{\partial
t}+(\@vec{v}\cdot\nabla)\@vec{v}=-\frac{1}{\rho_{\rm gas}}\nabla P_{\rm
gas}-\nabla\phi.$ (1)

Lau et al. (2013), Nelson et al. (2014), and Biffi et al. (2016) discuss the
relative role of the different contributors to the total potential, allowing
the total mass, $M_{\rm tot}$, to be written as the quantity that satisfies
the following differential equation:

$\frac{1}{\rho_{\rm gas}}\frac{dP_{\rm
tot}}{dr}=-\frac{d\phi}{dr}=-\frac{GM_{\rm tot}}{r^{2}},$ (2)

where $G$ is the gravitational constant and $P_{\rm tot}=P_{T}+P_{NT}$ is the
total pressure that we can express as the sum of the thermal component
($P_{T}=P_{\rm gas}=\rho_{\rm gas}\,kT_{\rm gas}/(\mu m_{\rm u})=n_{\rm
gas}\,kT_{\rm gas}$ 111Throughout this Letter, $n$ is the electron density
that relates to the gas density via the equation $\mu n_{\rm gas}=\mu_{e}n$,
with $\mu\approx 0.6$ and $\mu_{e}\approx 1.16$) and a non-thermal (not better
defined) quantity, $P_{NT}$, that includes components generated from the
velocity field, $\@vec{v}$, in Eq. 1 and accounts for: the rotational support
due to mean tangential motions of gas; spatial variations in the mean radial
streaming gas velocities; eventual correlations between the radial and
tangential components of the random motions; and temporal variations in the
mean radial gas velocities at a fixed radius. These temporal variations, also
known as the acceleration bias, have been shown in Nelson et al. (2014) to be
less than few per cent in relaxed systems but comparable to the effect induced
from turbulent and bulk motions in disturbed ones, with a contribution that
spans values between 40 and 20 per cent of the non-thermal pressure support
moving outwards (see e.g. Biffi et al., 2016; Angelinelli et al., 2020).

If we write the total mass as $M_{\rm tot}\sim r^{2}/n\,dP_{\rm tot}/dr$ and
the mass component due to the thermal-only pressure as $M_{T}\sim
r^{2}/n\,dP_{T}/dr$, the following relation holds between the two

$M_{\rm tot}=M_{T}\frac{1+P_{\rm
tot}(d\alpha/dr)/(dP_{T}/dr)}{1-\alpha}\,=\,M_{T}\frac{1+A}{1-\alpha},$ (3)

where

$\displaystyle\alpha$ $\displaystyle=$ $\displaystyle\frac{P_{NT}}{P_{\rm
tot}},$ $\displaystyle A$ $\displaystyle=$ $\displaystyle P_{\rm
tot}\frac{d\alpha/dr}{dP_{T}/dr}.$ (4)

We can now link this expression to the hydrostatic bias, $b$, the correction
needed to reconcile hydrostatic mass measurements with the expected ‘true’
value (see e.g. the reviews in Ettori et al., 2013; Pratt et al., 2019, for an
extensive discussion on the measurements of $b$ and their cosmological
implications):

$M_{\Delta,T}=(1-b)M_{\Delta,\rm tot},$ (5)

where we consider the estimates of the mass at a given overdensity
$\Delta=M_{\Delta}/(4/3\pi\rho_{cz}R_{\Delta}^{3}$) with respect to the
critical mass density at redshift $z$, $\rho_{cz}=3H_{z}^{2}/(8\pi G)$, with
$H_{z}=H_{0}\left[\Omega_{\rm m}(1+z)^{3}+1-\Omega_{\rm m}\right]^{1/2}$ being
the Hubble constant at that redshift in a flat Universe with $\Omega_{\rm
m}=1-\Lambda$. All the measurements quoted in the present analysis have been
made assuming $H_{0}=70$ km s-1 Mpc-1 and $\Omega_{\rm m}=0.3$.

In general, the computation of Eq. 5 relies on the availability of
$M_{\Delta,T}$ from X-ray measurements and $M_{\Delta,\rm tot}$ from what is
assumed to be a less biased true mass proxy, for example the one based on
weak-lensing signals (e.g. Meneghetti et al., 2010). We note, however, that
some assumptions have to be made on where (i.e. at which radius) the
comparison is done to estimate $b$. If the mass is estimated at a given
overdensity, $\Delta$, or at the same physical radius, $r$, by combining Eqs.
3 and 5, we can write

$b=b(r)=\frac{\alpha(r)+A(r)}{1+A(r)},$ (6)

which becomes $b=\alpha$ when $\alpha$ is a constant (i.e. $d\alpha/dr=0$, and
thus $A=0$). Hereafter, we assume implicitly that $b$ depends on radius. When
needed, we will indicate to which radius (typically $R_{500}$) we are
referring for some specific value of $b$. In the case under consideration, the
mass values are estimated at the same physical radius (e.g. at the
overdensity, $\Delta$, for the total mass profile, $M_{\rm tot}$, i.e. the
radius $R_{\Delta,\rm tot}$), but only $M_{T}(R_{\Delta,T})$ is known, and
$R_{\Delta,\rm tot}$ has to be inferred. Then, by definition, the following
relation holds between $M_{\Delta,\rm tot}=M_{\rm tot}(R_{\Delta,\rm tot})$
and $M_{\Delta,T}=M_{T}(R_{\Delta,T})$, $M_{\Delta,\rm tot}/R_{\Delta,\rm
tot}^{3}=M_{\Delta,T}/R_{\Delta,T}^{3}$, allowing us to write

$\displaystyle M_{\rm tot}(R_{\Delta,\rm tot})$ $\displaystyle=$
$\displaystyle M_{T}(R_{\Delta,T})\frac{R_{\Delta,tot}^{3}}{R_{\Delta,T}^{3}}$
(7) $\displaystyle=$ $\displaystyle
M_{T}(R_{\Delta,T})\,C_{M}(R_{\Delta,T},R_{\Delta,tot})\frac{1+A}{1-\alpha}\Big{\rvert}_{R_{\Delta,tot}}.$

Under the assumption that we measure $M_{T}(R_{\Delta,T})$, the hydrostatic
bias is estimated as $(1-b)=R_{\Delta,T}^{3}/R_{\Delta,tot}^{3}$ or
$(1-b)=(1-\alpha)/(1+A)|_{R_{\Delta,tot}}/C_{M}(R_{\Delta,T},R_{\Delta,tot})$,
where $C_{M}(R_{\Delta,T},R_{\Delta,tot})$ is the ratio between the ‘thermal’
mass evaluated at $R_{\Delta,tot}$ and at $R_{\Delta,T}$, which can be
evaluated for a given mass profile of $f(r)=M_{T}(r)/M_{T0}$ as
$C_{M}(R_{\Delta,T},R_{\Delta,tot})=f(R_{\Delta,tot})/f(R_{\Delta,T})$.

##  3 Modeling the non-thermal pressure

In this section we investigate how these calculations can be done by adopting
some functional forms for $P_{T}$ and $P_{NT}$.

As we have shown in Ghirardini et al. (2019b), a polytropic function, with an
effective polytropic index $\gamma$, is a very good representation for the
thermal gas distribution: $P_{T}=P_{0T}\,(n/n_{0})^{\gamma}$. Given our
ignorance on the true, and probably not univocal, origin of the non-thermal
pressure (apart from speculations based on hydrodynamical simulations in e.g.
Shi & Komatsu, 2014; Nelson et al., 2014; Angelinelli et al., 2020), we
continue to adopt the thermal electronically charged gas distribution as a
proxy for its distribution: $P_{NT}=P_{0NT}\,(n/n_{0})^{\beta}$. Then,

$\alpha=\frac{P_{NT}}{P_{\rm
tot}}=\left[\frac{P_{0T}}{P_{0NT}}\left(\frac{n}{n_{0}}\right)^{\gamma-\beta}+1\right]^{-1}$
(8) $A=P_{\rm
tot}\frac{d\alpha/dr}{dP_{T}/dr}=\left(\frac{\beta}{\gamma}-1\right)\alpha,$
(9)

where we have made use of the following relations: $dP_{T}/dr=P_{0T}\gamma
n^{\gamma-1}dn/dr$ and $dP_{NT}/dr=P_{0NT}\beta n^{\beta-1}dn/dr$.

As final step, we can now also rewrite the hydrostatic bias in Eq. 6 as

$b=\frac{\alpha+A}{1+A}=\frac{\alpha\beta}{(1-\alpha)\gamma+\alpha\beta}=\frac{\beta
P_{NT}}{\gamma P_{T}+\beta P_{NT}}.$ (10)

Again, when $d\alpha/dr=0$, then $\gamma=\beta$, and
$b=P_{NT}/(P_{T}+P_{NT})=\alpha$. For physical reasons, $\alpha\geq 0$ and
$\gamma>0$. Then, $b\geq 0$ only when $\beta\geq 0$. If $d\alpha/dr\neq 0$,
then we have a complete modelization of the radial behaviour of the
hydrostatic bias, $b$.

To constrain the non-thermal component, Shi et al. (2016) propose adopting an
analytic model (described in Shi & Komatsu, 2014) that is able to predict the
amplitude of non-thermal pressure and its radial, mass, and redshift
dependences, but only once the mass accretion history of the cluster is known.
Alternatively, and under the common condition that thermodynamic profiles
based on only X-rays and Sunayev-Zel’dovich (SZ; Sunyaev & Zel’dovich, 1972)
effect are available, some external information on the total gravitational
potential is needed. For instance, the total mass profiles can be derived from
gravitational lensing data (see e.g. Sayers et al., 2021) or from stellar
kinematics (as in early-type galaxies; see e.g. Churazov et al., 2008;
Humphrey et al., 2013). Fusco-Femiano & Lapi (2013) fix the gas fraction at
the viral radius equal to the cosmic baryon fraction, with the contribution
from stars subtracted, to model any extra contribution in pressure given X-ray
derived profiles. To account for the complex physical processes that can
affect the relative distributions of the baryons in a cosmological contest, we
(Ghirardini et al., 2018; Eckert et al., 2019) propose using a ‘universal’ gas
fraction, $f_{g,U}$, derived from a mean baryon fraction measured at some
given overdensities (typically $\Delta=$500 and 200) in massive haloes
extracted from hydrodynamical simulations and hence corrected for a
statistical contribution of the stellar fraction (i.e. $f_{g,U}=f_{\rm
b,sims}-f_{\rm star,stat}$). In the following analysis, we adopt the
measurements of this universal gas fraction presented and discussed in our
previous work (see e.g. Sect. 3.1 in Eckert et al., 2019):
$f_{g,U}(R_{500})=0.131\pm 0.009$ and $f_{g,U}(R_{200})=0.134\pm 0.007$. We
discuss below the impact of this assumption on our constraints of the non-
thermal pressure.

In Eckert et al. (2019), a model for $\alpha$ (instead of $P_{NT}$) –
presented in Nelson et al. (2014) as a simple, not physically motivated,
universal functional form of the radial behaviour observed in cosmological
hydrodynamical simulations – is adopted and solved iteratively in its two free
parameters (a third scale parameter is maintained fixed). In the present work,
we suggest adopting a modelization of the non-thermal pressure as directly
proportional to the electron density – which is more physically plausible
since the electron density is the natural reservoir of the magnetic fields and
particles to be accelerated – and with model parameters that are more easily
interpretable as physical quantities.

The total true mass is then recovered by comparing $f_{g,U}$ to the observed
$f_{g,T}$, which is based on the ‘thermal-only’ gravitational mass profile
(described in Ettori et al., 2019) and on the gas mass profile previously
corrected for some gas clumping that might bias the gas density reconstruction
(see the discussion on the recovered thermodynamic profiles in Ghirardini et
al. 2019a and Sect. 5.1 in Eckert et al. 2019 on the systematic uncertainties
affecting $f_{g,T}$). By propagating the hydrostatic bias in Eqs. 5 and 6, we
can write

$f_{g}=\frac{M_{g}}{M_{\rm
tot}}\Big{\rvert}_{R_{\Delta,tot}}=\frac{M_{g,T}}{M_{T}}\Big{\rvert}_{R_{\Delta,tot}}\,\frac{1-\alpha}{1+A}\Big{\rvert}_{R_{\Delta,tot}}.$
(11)

We note that the gas fraction will be evaluated at two different radii,
$R_{\Delta,tot}$ and $R_{\Delta,T}$, which correspond to the true and thermal-
only mass profile, respectively. With only $R_{\Delta,T}$ known from direct
observations ($R_{\Delta,tot}$ depends on the mass profile corrected for the
hydrostatic bias), we compute all these quantities at $R_{\Delta,T}$ and write

$\frac{M_{g}}{M_{\rm
tot}}\Big{\rvert}_{R_{\Delta,T}}=f_{g}\Big{\rvert}_{R_{\Delta,tot}}\,C_{f}(R_{\Delta,tot},R_{\Delta,T})=f_{g,T}\Big{\rvert}_{R_{\Delta,T}}\,\frac{1-\alpha}{1-\alpha(\gamma-\beta)/\gamma}\Big{\rvert}_{R_{\Delta,T}}.$
(12)

In this equation: $f_{g}\rvert_{R_{\Delta,tot}}$ is extracted from
hydrodynamical simulations; $f_{g,T}\rvert_{R_{\Delta,T}}$ and $\gamma$ are
measured from the observations; and $\alpha$ and $\beta$ are the unknowns.

The correction factor
$C_{f}(R_{\Delta,tot},R_{\Delta,T})=f_{g}(R_{\Delta,T})/f_{g}(R_{\Delta,tot})$
used to convert the gas fraction from numerical simulations at different radii
can be estimated by assuming that $n\sim r^{-\epsilon}$ in the region of
interest (around $R_{500}$ and $R_{200}$). Then,
$M_{g}(R_{\Delta,T})/M_{g}(R_{\Delta,tot})=(R_{\Delta,T}/R_{\Delta,tot})^{3-\epsilon}$,
and, using Eq. 7,
$C_{f}=f_{g}(R_{\Delta,T})/f_{g}(R_{\Delta,tot})\approx(R_{\Delta,T}/R_{\Delta,tot})^{3-\epsilon}\times
C_{M}(R_{\Delta,T},R_{\Delta,tot})$. Moreover, by assuming the power-law
behaviour of $n$ and $P$, and applying the HE to $M_{T}$ and $M_{\rm tot}$ as
in Eq. 3, we obtain that

$\displaystyle M_{T}=r^{1+\epsilon-\epsilon\gamma}$ $\displaystyle
M_{tot}=r^{1+\epsilon-\epsilon\gamma}$
$\displaystyle\left(1+\frac{\beta}{\gamma}\frac{P_{0NT}}{P_{0T}}r^{\gamma\epsilon-\beta\epsilon}\right),$
(13)

which can be used to estimate
$C_{M}(R_{\Delta,T},R_{\Delta,tot})=M_{tot}(R_{\Delta,tot})/M_{tot}(R_{\Delta,T})$.
Using the relation between $R_{\Delta,tot}$ and $R_{\Delta,T}$ (see Eq. 7)
with Eq. 13, we can then write

$R_{\Delta,T}^{\epsilon-\gamma\epsilon-2}=R_{\Delta,tot}^{\epsilon-\gamma\epsilon-2}\left(1+\frac{\beta}{\gamma}\frac{P_{0NT}}{P_{0T}}R_{\Delta,tot}^{\gamma\epsilon-\beta\epsilon}\right)$
(14)

and, finally,

$C_{f}=\frac{f_{g}(R_{\Delta,T})}{f_{g}(R_{\Delta,tot})}=\left(\frac{R_{\Delta,T}}{R_{\Delta,tot}}\right)^{-\epsilon}\left(1+\frac{\beta}{\gamma}\frac{P_{0NT}}{P_{0T}}R_{\Delta,T}^{\gamma\epsilon-\beta\epsilon}\right)^{-1}.$
(15)

In the present analysis, because we do not use the profiles in numerical
simulations, we assume $C_{f}=1$.

Considering that we know (i) the gas fraction, both the true and universal
one, $f_{g,U}$, and the observed thermal one, $f_{g,T}$, at (at least) two
given points ($R_{500}$ and $R_{200}$, from the thermal mass profile,
$M_{T}$), (ii) the gas density, $n$, and (iii) a (mean) polytropic index,
$\gamma$, in that radial range, we can estimate $\alpha$ and $\beta$,
measuring both the normalization $P_{0NT}$ and the radial dependence of the
modelled $P_{NT}$. In details, if we define $F_{1}=f_{g,U}/f_{g,T}|_{r_{1}}$,
$F_{2}=f_{g,U}/f_{g,T}|_{r_{2}}$, and $\gamma=\hat{\gamma}$ as estimated from
the recovered density and pressure profiles, and if we assume that the
unknowns $p_{0}=P_{0T}/P_{0NT}$ and $\beta$ do not vary with radius (though
this assumption can also be relaxed once more measurements of $F_{i}$ are
available), we can write $\alpha_{i}=P_{NT}/P_{\rm
tot}|_{r_{i}}=\left[p_{0}\,(n_{i}/n_{0})^{\hat{\gamma}-\beta}+1\right]^{-1}$
and the following system of two equations with the two unknowns:

$\cases{F}_{1}=\frac{1-\alpha_{1}}{1+\alpha_{1}(\beta-\hat{\gamma})/\hat{\gamma}}\\\
F_{2}=\frac{1-\alpha_{2}}{1+\alpha_{2}(\beta-\hat{\gamma})/\hat{\gamma}}{}\,.$
(16)

We note that the impact of the assumption on the value of $f_{g,U}$ at a given
radius can be easily quantified from Eq. 16. We represent this function by
plotting in Fig. 1 how $\alpha$ varies as a function of $\beta$ for different
measurements of the effective polytropic index, $\gamma$, and of the ratios
between the expected and the observed gas mass fraction, $F$. As expected,
$\alpha$ increases as $F$ decreases, for example by assuming lower values of
the universal gas fraction, $f_{g,U}$, for a given measured $f_{g,T}$.
Estimates of $\alpha$ between 0 and 1 are obtainable for any $\beta$ only when
$F<1$, with $\alpha\rightarrow 0$ when $F\rightarrow 1$. For values of $F>1$,
$\alpha$ will be larger than zero only when $\beta<\gamma(F-1)/F$.

Figure 1: Representation of Eq. 16 in the form
$\alpha=(1-F)/\left[1+F(\beta-\gamma)/\gamma\right]$ for different values of
$F=f_{g,U}/f_{g,T}$ and of the effective polytropic index, $\gamma$.  Table 1:
Parameters of the non-thermal pressure model for the X-COP clusters.

Cluster |  $\alpha_{500}/\alpha_{200}$ (E19) | $F_{500}/F_{200}$ | $\gamma$ | $P_{0NT}/P_{0T}$ | $\beta$ |  $\alpha_{500}/\alpha_{200}$ (this work)
---|---|---|---|---|---|---
A1644 | $10.5_{-10.5}^{+0.0}/14.8_{-14.8}^{+0.0}$ | $1.023\pm 0.095/1.063\pm 0.108$ | $1.117\pm 0.066$ | - | - | $0.0^{+0.0}_{+0.0}/0.0^{+0.0}_{+0.0}$
A1795 | $2.2_{-2.2}^{+5.6}/6.7_{-4.5}^{+6.0}$ | $0.942\pm 0.073/0.931\pm 0.058$ | $1.090\pm 0.096$ | 1.79e-02 | 0.937 | $6.6^{+3.8}_{-2.0}/8.0^{+12.6}_{-5.6}$
A2029 | $6.0_{-5.7}^{+5.8}/10.4_{-10.4}^{+9.0}$ | $0.929\pm 0.072/0.882\pm 0.058$ | $1.249\pm 0.068$ | 1.51e-03 | 0.749 | $11.3^{+5.0}_{-3.5}/18.3^{+11.0}_{-10.1}$
A2142 | $15.8_{-4.8}^{+4.5}/18.6_{-8.8}^{+7.1}$ | $0.829\pm 0.063/0.798\pm 0.050$ | $1.252\pm 0.060$ | 3.80e-02 | 1.038 | $19.9^{+2.9}_{-2.3}/23.4^{+7.6}_{-5.6}$
A2255 | $5.6_{-5.6}^{+6.8}/6.1_{-6.1}^{+6.3}$ | $0.856\pm 0.085/0.918\pm 0.123$ | $1.173\pm 0.040$ | 4.68e+00 | 1.597 | $11.0^{+8.0}_{-2.3}/6.2^{+17.1}_{-3.3}$
A2319 | $43.6_{-3.6}^{+3.5}/52.3_{-4.6}^{+3.4}$ | $0.693\pm 0.056/0.565\pm 0.041$ | $1.304\pm 0.043$ | 1.20e-03 | 0.484 | $54.4^{+11.8}_{-7.3}/67.4^{+12.5}_{-10.7}$
A3158 | $8.5_{-5.8}^{+5.7}/12.5_{-11.6}^{+8.9}$ | $0.903\pm 0.076/0.865\pm 0.072$ | $1.194\pm 0.055$ | 6.59e-03 | 0.838 | $13.2^{+5.8}_{-6.0}/18.3^{+14.0}_{-13.3}$
A3266 | $11.2_{-11.2}^{+0.0}/15.9_{-15.9}^{+0.0}$ | $0.992\pm 0.096/1.241\pm 0.217$ | $0.899\pm 0.031$ | - | - | $0.0^{+0.0}_{+0.0}/0.0^{+0.0}_{+0.0}$
A644 | $3.2_{-3.2}^{+6.4}/5.6_{-5.6}^{+6.4}$ | $0.992\pm 0.113/0.964\pm 0.116$ | $1.356\pm 0.118$ | - | - | $0.0^{+14.6}_{+3.9}/0.0^{+26.6}_{+1.1}$
A85 | $10.2_{-5.6}^{+4.9}/11.5_{-9.5}^{+8.9}$ | $0.873\pm 0.067/0.843\pm 0.058$ | $0.871\pm 0.045$ | 5.20e-02 | 0.735 | $14.7^{+3.4}_{-2.7}/18.1^{+8.7}_{-6.3}$
RXC1825 | $5.1_{-5.1}^{+5.1}/15.2_{-7.8}^{+6.4}$ | $0.985\pm 0.077/0.865\pm 0.060$ | $0.984\pm 0.050$ | - | - | $0.0^{+28.9}_{+9.8}/0.0^{+57.8}_{+14.1}$
ZW1215 | $11.9_{-11.9}^{+0.0}/15.7_{-15.7}^{+0.0}$ | $1.236\pm 0.126/1.457\pm 0.162$ | $0.999\pm 0.053$ | - | - | $0.0^{+0.0}_{+0.0}/0.0^{+0.0}_{+0.0}$

222For each object, we quote: the constraints on $\alpha$ at $R_{500}$ and
$R_{200}$ both in Eckert et al. (2019) (second column) and in the present work
(last column); the estimated $F_{i}$; the effective polytropic index,
$\gamma$, measured in the radial range $[0.3,1.6]R_{500}$; the parameters
$1/p_{0}=P_{0NT}/P_{0T}$ and $\beta$, from which $\alpha$ is estimated (see
Eq. 8). The quoted errors are at $1\sigma$ c.l., and the quoted errors on
$\alpha$ are obtained from 100 Monte Carlo runs by propagating the statistical
errors on the observed gas density and pressure profiles, and the observed gas
fractions.

Figure 2:  Example of the analysis applied to A1795. Upper left: Distribution
of the ratio of the expected gas fraction (the dashed orange line represents
$R_{500}$ and dashed cyan $R_{200}$) and observed gas fraction, $F_{i}$ (solid
horizontal lines, with errors represented by dotted lines; red represents
$R_{500}$ and blue $R_{200}$) as a function of the slope, $\beta$ (see Eq.
16); the best-fit value is indicated by the yellow dot. Upper right:
Distribution of $\alpha$ as a function of radius with, overplotted in purple,
the prediction from the model of $\alpha_{\rm turb}$ in Angelinelli et al.
(2020) and the expectations on the level of turbulence assuming $\eta=1/3$
(green dashed line) and magnetic energy $C_{E}=0.05$ (green dotted line; see
Sect. 4 for details). Lower left: Distribution of the turbulent velocity as a
function of radius (see Eq. 22). Lower right: Distribution of the relative
energy (Eq. 23) as a function of radius. In red, we show the region used to
constrain $\gamma$; $R_{500}$ and $R_{200}$ are indicated with vertical dashed
and dotted lines.

By dividing $F_{1}$ by $F_{2}$, we can solve for $p_{0}$, included in
$\alpha_{i}$, as a function of $\beta$ and after some math obtain

$p_{0}=\frac{\beta}{\hat{\gamma}}\left[\left(\frac{n_{1}}{n_{2}}\right)^{\hat{\gamma}-\beta}-\frac{F_{1}}{F_{2}}\right]\left(\frac{n_{1}}{n_{0}}\right)^{\beta-\hat{\gamma}}\left(\frac{F_{1}}{F_{2}}-1\right)^{-1}.$
(17)

Then, we solve numerically using an array of values for $\beta$ and by
inserting Eq. 17 into the second term of one of the two equations in Eq. 16,
as represented in Fig. 2. We present the results for the X-COP objects in
Table 2. All our estimates are consistent with the results presented in Eckert
et al. (2019). For five clusters, the constraints on $\alpha$ are consistent
with zero: three of them have an estimated $F_{i}>1$ at $R_{500}$ and/or
$R_{200}$, implying a negative hydrostatic bias.

We note that our algorithm allows the radial behaviour of $\alpha$ to be
constrained without any assumptions on the underlying functional form (such as
those suggested from recent numerical simulations; see e.g. Shaw et al. 2010;
Nelson et al. 2014; Angelinelli et al. 2020) and on a single object basis,
although a universal mean gas fraction has to be imposed on all the systems.

##  4 Discussion on the interpretation of $P_{NT}$

The properties of the intracluster plasma are regulated through the action of
cosmological supersonic flows, which accrete mass onto the cluster halo and
inject energy that is then dissipated over different scales and under
different forms, including non-thermal plasma components such as relativistic
particles and magnetic fields (see e.g. Brunetti & Jones, 2014). Miniati &
Beresnyak (2015) present evidence from numerical simulations that the energy
components follow a hierarchy that can be represented through some general
relations between thermal energy, $E_{T}$, energy in turbulence, $E_{\rm
turb}$, and energy in magnetic field $E_{B}=B^{2}/(8\pi)$:

$\frac{E_{T}}{E_{\rm turb}}=3\left(\frac{\eta}{1/3}\right)^{-1},\hskip
14.22636pt\frac{E_{T}}{E_{B}}=60\,(\gamma-1)\,\left(\frac{\eta}{1/3}\right)^{-1}\left(\frac{C_{E}}{0.05}\right)^{-1},$
(18)

where $\eta$ is the fraction of thermal energy originating from turbulent
dissipation and $C_{E}$ is the fraction of turbulent (kinetic) energy that
converts into magnetic energy; $\eta$ and $C_{E}$ have expected values of
about 1/3 and 0.05, respectively.

From the above equations, we can write that the magnetic field, $B$, is
related to the thermal pressure, $P_{T}$, as

$\frac{B^{2}}{8\pi}=\frac{\eta\,C_{E}}{\gamma-1}\,P_{T}.$ (19)

Using the polytropic law for $P_{T}=P_{0T}(n/n_{0})^{\gamma}$, we can then
write
$B=(8\pi\,\eta\,C_{E}\,P_{0T})^{0.5}(\gamma-1)^{-0.5}(n/n_{0})^{\gamma/2}=B_{0}(n/n_{0})^{\gamma_{B}}$.
With this modelization, we estimate a median $B_{0}$ of about 15 $\mu$G at 100
kpc. Furthermore, we can write the magnetic pressure as a fraction of the
total pressure as

$\frac{P_{B}}{P_{tot}}=\frac{\eta
C_{E}/(\gamma-1)\,P_{T}}{P_{T}+P_{NT}}=\frac{\eta
C_{E}/(\gamma-1)\,P_{T}}{P_{T}+\eta P_{T}}=\frac{\eta
C_{E}}{(\gamma-1)\,(1+\eta)}.$ (20)

In the extreme case that the entire non-thermal pressure support is in the
form of turbulence, then $P_{NT}=E_{\rm turb}$ and
$\eta=P_{NT}/P_{T}=\alpha/(1-\alpha)$, or $\alpha=\eta/(\eta+1)$ , which we
overplot as a dashed green line in Fig. 2 for $\eta=1/3$. We indicate the
fraction of magnetic energy for $C_{E}=0.05$ with a dotted green line. From
our dataset of measurements of $\alpha(R_{500})$, by requiring that
$P_{B}\equiv P_{NT}$, we obtain median (mean) values of $\eta$ of about 0.15
(0.3) and $C_{E}$, from the inversion of Eq. 20, $\sim$ 0.19 (0.16), with the
former being more than half of, and the latter three to four times larger
than, the estimates from numerical simulations.

Finally, if we convert $P_{NT}$ into the velocity dispersion associated with
the turbulence of the intracluster medium (ICM), we can write

$P_{NT}=a\,\rho\,\sigma_{\rm turb}^{2}=a\,\mu_{e}m_{p}n\,\sigma_{\rm
turb}^{2},$ (21)

which implies

$\sigma_{\rm
turb}^{2}=\frac{P_{0NT}\,(n/n_{0})^{\beta-1}}{a\,\mu_{e}m_{p}}=\frac{\alpha\,P_{T}}{1-\alpha}\frac{1}{a\,\mu_{e}m_{p}n},$
(22)

where $a=$1/3 and 1 for the isotropic value and the radial component only,
respectively, and $\rho$ is the gas mass density equal to $\mu_{e}m_{p}n$
(e.g. Vazza et al., 2018; Angelinelli et al., 2020). Similar treatment for the
determination of the non-thermal pressure support is discussed in Humphrey et
al. (2013), where the constraints from stellar dynamics are used to provide an
unbiased estimate of the true gravitational potential in elliptical galaxies.

With the thermal velocity dispersion defined as $\sigma_{\rm
th}^{2}=k_{B}T/(a\mu_{e}m_{p})=P_{T}/(a\mu_{e}m_{p}n)$, we can write the ratio
between the turbulent velocity dispersion and the velocity dispersion produced
by thermal motions as

$\frac{\sigma_{\rm turb}^{2}}{\sigma_{\rm th}^{2}}=\frac{\alpha}{1-\alpha}.$
(23)

Inverting this equation, we can write the hydrostatic bias as a function of
the turbulent velocity,

$b=\left(\frac{\sigma_{\rm th}^{2}}{\sigma_{\rm turb}^{2}}+1\right)^{-1}.$
(24)

We used published values of upper limits on $\sigma_{\rm turb}$ to reconstruct
the expected upper limits on $b$ (from Eq. 24, $b$ decreases as $\sigma_{\rm
turb}$ decreases). In Fig. 3, we show the $\mathcal{N}(<b)-b$ (or hydrostatic
bias density) distribution based on the constraints presented in Sanders et
al. (2011) and Pinto et al. (2015) and obtained from the width of emission
lines in XMM-Newton Reflection Grating Spectrometer spectra extracted from the
inner regions of a sample of 62 galaxy clusters, groups, and elliptical
galaxies and the 44 nearby, bright objects in the CHEERS sample, respectively.
We note that the upper limits on $b$ are larger in cooler systems. Limiting
our analysis to the clusters with $kT>2$ keV, and interpolating the
distribution measured in the hotter systems of the CHEERS sample, we estimate
upper limits on $b<(0.33,0.19,0.09)$ in 80, 50, and 20 per cent of the
population. Considering that these measurements are limited to the cluster’s
cores, we estimated a median correction on the bias $b$, estimated at
different radii, in the X-COP sample (see inset in Fig. 3) and obtain
$b(0.1R_{500})\approx 0.5b(R_{500})$, with a general increase in the bias
moving outwards. Applying this correction to the upper limits in Pinto et al.
(2015), we evaluate $b<(0.64,0.41,0.20)$ in 80, 50, and 20 per cent of the
population, respectively.

We can now compare these upper limits with the results we obtain for the X-COP
sample (see Fig. 3). The distribution of the bias in our objects covers lower
values of $b$ (with 80% of the objects with $b<0.17$ and 50% with $b<0.03$ at
$R_{500}$), confirming that our bright local systems are, on average, relaxed
(i.e. closer to the assumption of a spherically symmetric gas distribution in
hydrostatic equilibrium with the underlying gravitational potential), as they
were selected at origin. Three objects studied in our work are also present in
Pinto et al. (2015). From the 1 $\sigma$ upper limits quoted there, we infer
that $b=\alpha$ from turbulence of about $<$11%, $<$51%, and $<$29% for A85,
A1795, and A2029, respectively, Only for A85 does this value appear to be
similar to the constraints presented in Table 2; for A2029 and A1795, the
levels of bias induced from the current upper limits are more than two and
seven times the constraints we obtain.

Overall, these distributions of the hydrostatic bias, $b$, at $R_{500}$ are in
good agreement with the mean values of about 10-20 per cent obtained in
hydrodynamical simulations of massive haloes in a cosmological context (see
e.g. Biffi et al., 2016; Pearce et al., 2020).

Figure 3:  Density distribution of the hydrostatic bias, $b$. The values of
$b$ are inferred from Eq. 10, using the values tabulated in Table 2, for the
X-COP objects at $R_{500}$ (solid line) and at $R_{200}$ (dotted line), and
from Eq. 24 for the upper limits for the clusters in Pinto et al. (2015) (16
out of 44 with $T>2$ keV; upper limits estimated within 0.8 arcmin) and
Sanders et al. (2011) (36 out of 62). The purple line shows the constraints
that include the correction to extrapolate the estimates from the core to
$R_{500}$ (see text and the inset, where we plot median distribution, with
first and third quartiles, of the hydrostatic bias in the X-COP sample as a
function of radius). The shaded region shows the constraints obtained from the
Planck collaboration by combining the cosmic microwave background anisotropies
and the cluster number count (Planck Collaboration et al., 2020).

##  5 Conclusions

We have presented a new model of the non-thermal pressure as
$P_{NT}=P_{0NT}\,(n/n_{0})^{\beta}$ and have applied it to the X-COP results
on the radial profiles of the gas density, temperature, and pressure to
constrain the parameters $P_{0NT}$ and $\beta$. We converted this non-thermal
pressure support to the bias $b$ that affects the reconstruction of the mass
profile through the hydrostatic equilibrium equation. We also translated
$P_{NT}$ into expected contributions in the form of magnetic fields and
turbulent velocity in the ICM. Regarding turbulent velocity, we put
interesting constraints on the expected distribution of the hydrostatic bias
using upper limits on the turbulent velocities available in the literature. We
propose the hydrostatic bias density distribution $\mathcal{N}(<b)-b$ as a way
to assess statistically interesting and useful quantities that have an impact
on the use of galaxy clusters for astrophysical and cosmological purposes. We
show that current constraints (both upper limits on the turbulent velocity in
the ICM and measurements from the gas mass fraction in X-COP; see Fig. 3)
suggest that most of the studied galaxy clusters have a $b$ lower than (or
marginally consistent with, when the upper limits extrapolated up to $R_{500}$
are considered) the value of $b=0.38(\pm 0.03)$ required to reconcile the
measurements of the cosmic microwave background anisotropies with the SZ-based
cluster number count from Planck data (Planck Collaboration et al., 2020). We
note that to accommodate for this value of $b$ with the median estimates of
$\beta$ (0.84) and $\gamma$ (1.19) recovered in the X-COP sample, by using
Eqs. 10 and 16 and the measured $f_{g,T}$, we require $f_{g,U}(R_{500})$ to be
62% of the adopted value of 0.13 (i.e. a universal gas fraction of about
0.08). This value is more typical for $10^{14}M_{\odot}$ systems, where the
action of feedback has a larger impact than in more massive, X-COP-like,
objects on the expected distribution of baryons, at least as it is resolved in
state-of-the-art cosmological hydrodynamical simulations (see e.g. Fig. 15 in
Eckert et al., 2021).

Future X-ray instruments with improved spectral resolution (e.g.
XRISM333https://xrism.isas.jaxa.jp/en/ and X-IFU444http://x-ifu.irap.omp.eu/
on board Athena555https://www.the-athena-x-ray-observatory.eu/) will tighten
these limits, providing significant constraints on the turbulent velocity and
allowing the picture of the energy and mass budget of galaxy clusters to be
completed. In the meantime, projects such as CHEX-MATE666http://xmm-
heritage.oas.inaf.it/ will provide a direct measurement of $b$ by combining
different mass proxies in an SZ-selected representative sample of galaxy
clusters.

###### Acknowledgements.

We thank the referee for some valuable comments. SE acknowledges financial
contribution from the contracts ASI-INAF Athena 2019-27-HH.0, “Attività di
Studio per la comunità scientifica di Astrofisica delle Alte Energie e Fisica
Astroparticellare” (Accordo Attuativo ASI-INAF n. 2017-14-H.0), INAF
mainstream project 1.05.01.86.10, and from the European Union’s Horizon 2020
Programme under the AHEAD2020 project (grant agreement n. 871158).

## References

  * Angelinelli et al. (2020) Angelinelli, M., Vazza, F., Giocoli, C., et al. 2020, MNRAS, 495, 864
  * Biffi et al. (2016) Biffi, V., Borgani, S., Murante, G., et al. 2016, ApJ, 827, 112
  * Brunetti & Jones (2014) Brunetti, G. & Jones, T. W. 2014, International Journal of Modern Physics D, 23, 1430007
  * Churazov et al. (2008) Churazov, E., Forman, W., Vikhlinin, A., et al. 2008, MNRAS, 388, 1062
  * Eckert et al. (2017) Eckert, D., Ettori, S., Pointecouteau, E., et al. 2017, Astronomische Nachrichten, 338, 293
  * Eckert et al. (2021) Eckert, D., Gaspari, M., Gastaldello, F., Le Brun, A. M. C., & O’Sullivan, E. 2021, Universe, 7, 142
  * Eckert et al. (2019) Eckert, D., Ghirardini, V., Ettori, S., et al. 2019, A&A, 621, A40
  * Ettori et al. (2013) Ettori, S., Donnarumma, A., Pointecouteau, E., et al. 2013, Space Sci. Rev., 177, 119
  * Ettori et al. (2019) Ettori, S., Ghirardini, V., Eckert, D., et al. 2019, A&A, 621, A39
  * Fusco-Femiano & Lapi (2013) Fusco-Femiano, R. & Lapi, A. 2013, ApJ, 771, 102
  * Ghirardini et al. (2019a) Ghirardini, V., Eckert, D., Ettori, S., et al. 2019a, A&A, 621, A41
  * Ghirardini et al. (2019b) Ghirardini, V., Ettori, S., Eckert, D., & Molendi, S. 2019b, A&A, 627, A19
  * Ghirardini et al. (2018) Ghirardini, V., Ettori, S., Eckert, D., et al. 2018, A&A, 614, A7
  * Humphrey et al. (2013) Humphrey, P. J., Buote, D. A., Brighenti, F., Gebhardt, K., & Mathews, W. G. 2013, MNRAS, 430, 1516
  * Landau & Lifshitz (1959) Landau, L. D. & Lifshitz, E. M. 1959, Fluid mechanics (Pergamon Press)
  * Lau et al. (2013) Lau, E. T., Nagai, D., & Nelson, K. 2013, ApJ, 777, 151
  * Meneghetti et al. (2010) Meneghetti, M., Rasia, E., Merten, J., et al. 2010, A&A, 514, A93
  * Miniati & Beresnyak (2015) Miniati, F. & Beresnyak, A. 2015, Nature, 523, 59
  * Nelson et al. (2014) Nelson, K., Lau, E. T., & Nagai, D. 2014, ApJ, 792, 25
  * Pearce et al. (2020) Pearce, F. A., Kay, S. T., Barnes, D. J., Bower, R. G., & Schaller, M. 2020, MNRAS, 491, 1622
  * Pinto et al. (2015) Pinto, C., Sanders, J. S., Werner, N., et al. 2015, A&A, 575, A38
  * Planck Collaboration et al. (2020) Planck Collaboration, Aghanim, N., Akrami, Y., et al. 2020, A&A, 641, A6
  * Pratt et al. (2019) Pratt, G. W., Arnaud, M., Biviano, A., et al. 2019, Space Sci. Rev., 215, 25
  * Sanders et al. (2011) Sanders, J. S., Fabian, A. C., & Smith, R. K. 2011, MNRAS, 410, 1797
  * Sayers et al. (2021) Sayers, J., Sereno, M., Ettori, S., et al. 2021, MNRAS, 505, 4338
  * Shaw et al. (2010) Shaw, L. D., Nagai, D., Bhattacharya, S., & Lau, E. T. 2010, ApJ, 725, 1452
  * Shi & Komatsu (2014) Shi, X. & Komatsu, E. 2014, MNRAS, 442, 521
  * Shi et al. (2016) Shi, X., Komatsu, E., Nagai, D., & Lau, E. T. 2016, MNRAS, 455, 2936
  * Sunyaev & Zel’dovich (1972) Sunyaev, R. A. & Zel’dovich, Y. B. 1972, Comments on Astrophysics and Space Physics, 4, 173
  * Suto et al. (2013) Suto, D., Kawahara, H., Kitayama, T., et al. 2013, ApJ, 767, 79
  * Vazza et al. (2018) Vazza, F., Angelinelli, M., Jones, T. W., et al. 2018, MNRAS, 481, L120


11institutetext<EMAIL_ADDRESS>
# A Comprehensive Comparison of Pre-training Language Models

Tong Guo

###### Abstract

Recently, the development of pre-trained language models has brought natural
language processing (NLP) tasks to the new state-of-the-art. In this paper we
explore the efficiency of various pre-trained language models. We pre-train a
list of transformer-based models with the same amount of text and the same
training steps. The experimental results shows that the most improvement upon
the origin BERT is adding the RNN-layer to capture more contextual information
for short text understanding. But the conclusion is: There are no remarkable
improvement for short text understanding for similar BERT structures. Data-
centric method[12] can achieve better performance.

###### Keywords:

Deep Learning Pre-trained Language Model

##  1 Introduction

In recent years, deep learning [2] and BERT [1] have shown significant
improvement on almost all the NLP tasks. However, it lacks of a fair
comparison on the transformer-based models, due to the pre-training datasets
are different and the pre-training computing resources are different. In
industry NLP applications, we need to find the most efficient BERT-like model,
as the computing resources is limited.

In the paper, we pre-train a list of transformer-based models on the same
datasets and pre-training steps. Then we evaluate the pre-trained performance
by fine-tuning on our large text classification downstream task.

##  2 Relate work

BERT [1], or Bidirectional Encoder Representations from Transformers, is a
multi-layer transformer-encoder based [7] deep model, which produces
contextual token representations that have been pre-trained from unlabeled
text and fine-tuned for the supervised downstream tasks. BERT obtains state-
of-the-art results on a wide array of Natural Language Processing (NLP) tasks,
which include the GLUE[3] benchmark and CLUE[6] benchmark. There are two steps
in BERT’s framework: pre-training and fine-tuning. During pre-training, the
model is trained on unlabeled data by using masked language model task and
next sentence prediction task. Apart from output layers, the same
architectures are used in both pre-training and fine-tuning. The same pre-
trained model parameters are used to initialize models for different down-
stream tasks.

##  3 The Models

In this section we describe the BERT-like models we use. First we describe the
notations we use. We use $L$ to indicate the input sequence length. We use $N$
to indicate the hidden size.

The code is released at https://github.com/guotong1988/bert_compare

###  3.1 The Origin BERT

We refer the origin BERT to the code released by
Google111https://github.com/google-research/bert.

###  3.2 TextCNN-BERT

The intuition we try this architecture is that we think convolution layer can
extract features that is different from self-attention layer. We learn the
model architecture from TextCNN [10]. We use TextCNN to extract feature from
sequence input. Then the output of TextCNN concat the sequence input for the
self-attention layer. In detail, we use a convolution kernel $R^{1\times N}$
which in channel is $1$ and out channel is $N$ to output a tensor $R^{L\times
N}$. Then we concat it to the embedding layer to get a tensor $R^{2L\times N}$
for the next self-attention-layer.

###  3.3 Ngram-BERT

The intuition we try this architecture is that we think the N-gram info can be
the supplement for the one-token-level sequence. We add the N-gram info for
the Origin BERT. In detail, we concat 2-gram of token embeddings to get a
tensor which shape is $R^{L\times 2N}$. Then we use a matrix $R^{2N\times N}$
to transform it to a tensor $R^{L\times N}$ . Then we concat $R^{L\times N}$
to the embedding layer for the next self-attention layer.

###  3.4 Dense-BERT

The intuition we try this architecture is that we think the residual
connection of transformer layers can be improved by dense connection. We learn
the model architecture from DenseNet[11]. We add the dense connections in all
the transformer-encoder layers. In detail, each transformer layer’s input is
the output of all previous layers. Although the experiment results below shows
that Dense-BERT is not better than the origin BERT under the almost same
parameter size, we found Dense-BERT improve the accuracy performance more as
the layer number go larger.

###  3.5 ConvBERT

ConvBERT [9] is using span-based dynamic convolution to improve BERT. The code
is from here222https://github.com/yitu-opensource/ConvBert.

###  3.6 BORT

BORT [8] is an optimal subarchitecture extraction for BERT by neural
architecture search. We follow the final parameter setting of BORT. We only
use the final parameter setting of BORT and do not use other methods proposed
by the paper[8].

###  3.7 Relative Position Embedding BERT (RTE-BERT)

We replace the embedding layer of origin BERT by the relative position
embedding. The code is from
here333https://github.com/tensorflow/tensor2tensor. We extract a easy-to-use
relative position embedding code from tensor2tensor and put them to
here444https://github.com/guotong1988/transformer_relative_position_embedding.

###  3.8 RNN-BERT

We use RNN layer to capture more position info for the transformer-encoder
layer. In detail, the embedding layer is followed by the LSTM layer. Then the
output of LSTM layer and the embedding layer are added for the next self-
attention layer. We found that concating the output of LSTM layer and the
embedding layer do not get better result.

###  3.9 RNN-IN-BERT

We use RNN layer to capture more position info for each transformer-encoder
layer. In detail, the embedding layer is followed by the LSTM layer. Then the
output of LSTM layer and the embedding layer are added for the next self-
attention layer. The LSTM layers are also inserted between the self-attention
layers.

Table 1: The downstream text classification evaluation results of our own dataset, under the same computing resources. Model | layer setting | Accuracy
---|---|---
Origin BERT | 3 layer, 768 hidden size | 91.41%
TextCNN-BERT | 3 layer, 768 hidden size | 91.42%
Ngram-BERT | 3 layer, 768 hidden size | 91.32%
Dense-BERT | 4 layer, 1024 hidden size | 91.36%
ConvBERT[9] | 3 layer, 768 hidden size | 90.68%
BORT[8] | 4 layer, 1024 hidden size | 91.30%
RTE-BERT | 3 layer, 768 hidden size | 91.36%
RNN-BERT | 3 layer, 768 hidden size | 91.49%
RNN-IN-BERT | 3 layer, 768 hidden size | 91.51%
Table 2: The downstream text classification evaluation results of CLUE TNEWS dataset, under the same computing resources. Model | layer setting | Accuracy
---|---|---
Origin BERT | 3 layer, 768 hidden size | 54.38%
RNN-BERT | 3 layer, 768 hidden size | 54.41%
RNN-IN-BERT | 3 layer, 768 hidden size | 54.81%

##  4 Experiments

In this section we describe detail of experiment parameters and show the
experiment results. The pre-training dataset size is 600,000,000 Chinese
sentences and the downstream fine-tuning text classification dataset size is
2,000,000 Chinese sentences.

In pre-training, we use 400 batch size, 64 sequence length. We pre-train each
kind of BERT-like model for 1,000,000 steps in the same pre-training dataset.

In fine-tuning, we use 100 batch size, 64 sequence length. We use Adam [4]
with learning rate of 1e-5 and use a dropout [5] probability of 0.1 on all
layers.

For all kinds of BERT-like models, the total parameter will be no difference
of 20%.

##  5 Analysis and Conclusion

As it is shown in Table 1, we get the conclusion that the only lack of origin
BERT is that the position embedding of transformer can not capture all the
position or contextual info of the input sequence.

We will do more experiments on CLUE[6] in the future.

## References

  * [1] Devlin J, Chang M W, Lee K, et al. Bert: Pre-training of deep bidirectional transformers for language understanding[J]. arXiv preprint arXiv:1810.04805, 2018.
  * [2] Krizhevsky A, Sutskever I, Hinton G E. Imagenet classification with deep convolutional neural networks[J]. Advances in neural information processing systems, 2012, 25: 1097-1105.
  * [3] Wang A, Singh A, Michael J, et al. GLUE: A multi-task benchmark and analysis platform for natural language understanding[J]. arXiv preprint arXiv:1804.07461, 2018.
  * [4] Kingma D P, Ba J. Adam: A method for stochastic optimization[J]. arXiv preprint arXiv:1412.6980, 2014.
  * [5] Srivastava N, Hinton G, Krizhevsky A, et al. Dropout: a simple way to prevent neural networks from overfitting[J]. The journal of machine learning research, 2014, 15(1): 1929-1958.
  * [6] Xu L, Hu H, Zhang X, et al. Clue: A chinese language understanding evaluation benchmark[J]. arXiv preprint arXiv:2004.05986, 2020.
  * [7] Vaswani A, Shazeer N, Parmar N, et al. Attention is all you need[C]//Advances in neural information processing systems. 2017: 5998-6008.
  * [8] de Wynter, Adrian, and Daniel J. Perry. ”Optimal Subarchitecture Extraction For BERT.” arXiv preprint arXiv:2010.10499 (2020).
  * [9] Jiang, Zihang, et al. ”Convbert: Improving bert with span-based dynamic convolution.” arXiv preprint arXiv:2008.02496 (2020).
  * [10] Zhang, Ye, and Byron Wallace. ”A sensitivity analysis of (and practitioners’ guide to) convolutional neural networks for sentence classification.” arXiv preprint arXiv:1510.03820 (2015).
  * [11] Huang, Gao, et al. ”Densely connected convolutional networks.” Proceedings of the IEEE conference on computer vision and pattern recognition. 2017.
  * [12] Guo, Tong (2021): Re-Label Is All You Need. TechRxiv. Preprint. https://doi.org/10.36227/techrxiv.17128475.v11


# Classification of contact seaweeds

Vincent E. Coll  Dept. of Mathematics, Lehigh University, Bethlehem, PA 18015:
<EMAIL_ADDRESS>Jr  Dept. of Mathematics, Lehigh University, Bethlehem, PA
18015<EMAIL_ADDRESS>Nicholas Russoniello  Dept. of Mathematics, College
of William & Mary, Williamsburg, VA 23185<EMAIL_ADDRESS>
###### Abstract

A celebrated result of Gromov ensures the existence of a contact structure on
any connected, non-compact, odd dimensional Lie group. In general, such
structures are not invariant under left translation. The problem of finding
which Lie groups admit a left-invariant contact structure resolves to the
question of determining when a Lie algebra $\mathfrak{g}$ is contact; that is,
admits a one-form $\varphi\in\mathfrak{g}^{*}$ such that
$\varphi\wedge(d\varphi)^{k}\neq 0.$

In full generality, this remains an open question; however we settle it for
the important category of the evocatively named seaweed algebras by showing
that an index-one seaweed is contact precisely when it is quasi-reductive.
Seaweeds were introduced by Dergachev and Kirillov who initiated the
development of their index theory – since completed by Joseph, Panyushev,
Yakimova, and Coll, among others. Recall that a contact Lie algebra has index
one – but not characteristically so. Leveraging recent work of Panyushev,
Baur, Moreau, Duflo, Khalgui, Torasso, Yakimova, and Ammari, who collectively
classified quasi-reductive seaweeds, our equivalence yields a full
classification of contact seaweeds. We remark that since type-A and type-C
seaweeds are de facto quasi-reductive (by a result of Panyushev), in these
types index one alone suffices to ensure the existence of a contact form.

Mathematics Subject Classification 2020: 17Bxx, 53D10

Key Words and Phrases: contact Lie algebra, contact structure, seaweeds,
quasi-reductive, stable

††Vincent E. Coll, Jr<EMAIL_ADDRESS>is the corresponding author.

##  1 Introduction

This short note, the coda of a sequence of three papers, provides, per the
following theorem, a complete classification of contact seaweed (or
biparabolic) Lie algebras.

######  Theorem 5.

An index-one seaweed is contact if and only if it is quasi-reductive.

The first two papers of this sequence establish Theorem 5 for type-A (Coll et
al. [12], 2022) and type-C (Coll and Russoniello [13], 2022) seaweeds by
showing, in these classical cases, that index one is sufficient to identify
that the seaweed is contact. This is unexpected, as a contact Lie algebra has
index one – but not characteristically so. These prefatory papers use
constructive combinatorial methods to develop explicit contact forms for
index-one seaweeds. Panyushev’s result ([25], 2005) that seaweeds in type-A
and type-C are quasi-reductive provided a clue to the final form of the above
theorem. And while the comprehensive Theorem 5 subsumes the results of the
first two papers, the methods employed here to establish the classification
are algebraic and existential, rather than constructive. More specifically, we
leverage recent results presented in a series of articles by Ammari ([3],
2014; [4], 2014; and [2], 2022), which establish that the “stability” of a
seaweed algebra implies its quasi-reductivity. This yields the proof heuristic
for Theorem 5 in the following diagrammatic form.

CONTACTQRSTABLE(Ammari)

Remark. At first blush, Theorem 5 appears to be a bare equivalence of the
notions of contact and quasi-reductivity; however, the collective works of
Panyushev [25], Baur and Moreau ([6], 2011), Duflo et al. ([18], 2012),
Panyushev and Yakimova ([26], 2018), and Ammari [2] have yielded a complete,
explicit classification of quasi-reductive seaweeds. It follows that Theorem 5
provides a concrete classification in the standard meaning of the term.

The structure of this note is as follows. In Section 2 we acknowledge some
preliminary technical results before moving on to the proof of Theorem 5 in
Section 3. Section 4 is a brief Epilogue which provides a broader context for
Theorem 5. We are thus able to posit several interesting follow-up questions
which merit ongoing investigation.

##  2 Preliminaries

Convention: All Lie algebras ($\mathfrak{g},[-,-])$ are over $\mathbb{C}$.

A $(2k+1)-$dimensional Lie algebra $\mathfrak{g}$ is called contact if it
admits a contact form; that is, a one-form $\varphi\in\mathfrak{g}^{*}$ such
that $\varphi\wedge(d\varphi)^{k}\neq 0.$ If $G$ is an underlying Lie group of
$\mathfrak{g}$ then each such $\varphi$ gives rise to a left-invariant contact
structure on $G,$ and $\varphi\wedge(d\varphi)^{k}$ is a volume form on $G$. A
now-classical result of Gromov ([21], 1969) ensures the existence of a contact
structure on any connected, non-compact,111The groups underlying the Lie
algebras here considered are non-compact, since all compact complex Lie groups
are necessarily abelian. odd-dimensional Lie group. But, in general, such
structures are not invariant under left translations of the Lie group. The
problem of finding which Lie groups admit a left-invariant contact structure
resolves to cataloging contact Lie algebras and remains a problem of ongoing
and topical interest (see [1], [10], [15], [20], [23], and [27]). Here, we
offer a complete classification for seaweed algebras.

Recall that a seaweed algebra, or simply “seaweed,” is a subalgebra of a
reductive Lie algebra $\mathfrak{r}$ defined by the intersection of two
parabolic subalgebras of $\mathfrak{r}$ whose sum is $\mathfrak{r}$.222This
basis-free definition of seaweed has prompted others to refer to these
algebras as biparabolic. The evocative “seaweed” derives from the wavy shape
displayed when these algebras are represented in their matrix form in the
classical types. Seaweeds were first introduced by Dergachev and Kirillov
([14], 2000) where the authors studied the “index” of such algebras in the
case where $\mathfrak{r}=\mathfrak{gl}(n).$ Formally, the index of a Lie
algebra $\mathfrak{g}$ is an algebraic invariant first introduced by Dixmier
([16], 1974) and defined as follows:

${\rm ind\hskip
2.84544pt}\mathfrak{g}=\min_{\varphi\in\mathfrak{g}^{*}}\dim(\ker(B_{\varphi})),$

where $B_{\varphi}$ is the skew-symmetric Kirillov form given by
$B_{\varphi}(x,y)=-d\varphi(x,y)=\varphi([x,y]),$ for all
$x,y\in\mathfrak{g}.$ Contact algebras necessarily have index one.

To round out the needed notions for our study, we require the concepts of
“quasi-reductivity” and “stability” of a Lie algebra. If $\mathfrak{g}$ is an
algebraic Lie algebra (in particular, a seaweed) with center
$Z(\mathfrak{g}),$ then $\mathfrak{g}$ is quasi-reductive if there exists a
one-form $\varphi\in\mathfrak{g}^{*}$ such that
$\faktor{\ker(B_{\varphi})}{Z(\mathfrak{g})}$ is a reductive Lie algebra whose
center consists of semisimple elements of $\mathfrak{g}.$ Such a one-form
$\varphi$ is called a form of reductive type. Quasi-reductive Lie groups were
first introduced in (Duflo [17], 1982), where it is shown, in particular, that
the coadjoint orbits of forms of reductive type can be associated to square
integrable unitary representations of the group (see [5], [18], and [24]).
Recent work has led to the classification of quasi-reductive seaweeds (see
[2],[6],[18],[25], and [26]). Of interest here is a result of Duflo et al.
([18], Théorème 3.4.1) which establishes, in particular, that all quasi-
reductive seaweeds admit a stable one-form, i.e., a one-form
$\varphi\in\mathfrak{g}^{*}$ for which there is a Zariski open neighborhood
$V\ni\varphi$ such that, for all $\psi\in V,$ the kernels $\ker(B_{\varphi})$
and $\ker(B_{\psi})$ are conjugate under the algebraic adjoint group of
$\mathfrak{g}.$ As we will consider stable one-forms in the proof of Theorem
5, we provide the following useful equivalence.

######  Lemma 1 (Tauvel and Yu [28], Théorème 1.7, 2004; cf. Ammari [4],
2014).

Let $\mathfrak{g}$ be algebraic. A one-form $\varphi$ is stable if and only if

$[\ker(B_{\varphi}),\mathfrak{g}]\cap\ker(B_{\varphi})=\\{0\\}.$

We call a Lie algebra that admits a stable one-form a stable Lie algebra, and
so by Duflo et al., each quasi-reductive seaweed is stable. It remained an
open question as to whether all stable seaweeds were quasi-reductive until
Ammari showed – in a series of articles [2], [3], and [4] – that, for
seaweeds, the two notions are commensurate.

######  Theorem 2 (Ammari [2], Théorème 5.4; [3], Propositions 4.22, 4.24,
5.27, and 5.28; [4], Théorème 4.2.2).

A seaweed is quasi-reductive if and only if it is stable.

With the assemblage of Lemma 1 and Theorem 2, we have the groundwork in place
to proceed to the algebraic classification of contact seaweeds in the next
section.

##  3 Proof of the Main Result

In this section, we provide a proof of Theorem 5. We first establish some
terminology and two technical lemmas, the first of which is a classical
characterization of contact Lie algebras.

######  Lemma 3.

Let $\mathfrak{g}$ be an arbitrary Lie algebra. If ${\rm ind\hskip
2.84544pt}\mathfrak{g}=1,$ then a regular one-form
$\varphi\in\mathfrak{g}^{*}$ with $\ker(B_{\varphi})=\mathbb{C}x$ is contact
if and only if $\varphi(x)\neq 0.$

###### Proof.

See Lemma 23 in [12]. ∎

Lemma 3 establishes that the existence of a contact form on an index-one Lie
algebra $\mathfrak{g}$ is equivalent to the existence of a Reeb vector
associated to a regular one-form $\varphi$; that is, a regular
$\varphi\in\mathfrak{g}^{*}$ is contact precisely when there is some
$x_{\varphi}\in\mathfrak{g}$ such that
$\ker(B_{\varphi})=\mathbb{C}x_{\varphi}$ and $\varphi(x_{\varphi})=1.$ The
identification of the Reeb vector yields a useful vector space decomposition
of a contact Lie algebra
$\mathfrak{g}=\mathbb{C}x_{\varphi}\oplus\ker(\varphi).$ The subspace
$\ker(\varphi)\subset\mathfrak{g}$ defines the contact structure on an
underlying Lie group $G$ of $\mathfrak{g}$ (see [19]), and such a
decomposition gives rise to a particular contact basis of $\mathfrak{g}$ with
respect to $\varphi,$ which is explicitly described by the following lemma.

######  Lemma 4 (Goze and Remm [20], 2014; cf. Alvarez et al. [1], 2021).

Let $\mathfrak{g}$ be a $(2k+1)-$dimensional contact Lie algebra with contact
form $\varphi\in\mathfrak{g}^{*}.$ Then there exists a basis
$\\{E_{1}^{*},E_{2}^{*},\dots,E_{2k+1}^{*}\\}$ of $\mathfrak{g}^{*}$ such that
$E_{1}^{*}=\varphi$ and

$B_{\varphi}=E_{2}^{*}\wedge E_{3}^{*}+\dots+E_{2k}^{*}\wedge E_{2k+1}^{*}.$

Moreover, if $\\{E_{1},E_{2},\dots,E_{2k+1}\\}$ is the dual basis of
$\\{E_{1}^{*},E_{2}^{*},\dots,E_{2k+1}^{*}\\},$ we have that

$\displaystyle[E_{2\ell},E_{2\ell+1}]$
$\displaystyle=E_{1}+\sum_{r=2}^{2k+1}c_{2\ell,2\ell+1}^{r}E_{r},~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}\ell=1,\dots,k,\text{
and }$ $\displaystyle[E_{i},E_{j}]$
$\displaystyle=\sum_{r=2}^{2k+1}c_{i,j}^{r}E_{r},~{}~{}~{}~{}~{}~{}~{}~{}1\leq
i,j\leq 2k+1\text{ such that }(i,j)\neq(2\ell,2\ell+1),$

where $c_{i,j}^{r}\in\mathbb{R}$ for all $i,j,r=2,\dots,2k+1.$

We are now in a position to prove the main result of this article.

######  Theorem 5.

An index-one seaweed is contact if and only if is quasi-reductive.

###### Proof.

We begin with the reverse implication (QR$\implies$contact). Let
$\mathfrak{s}$ be an index-one, quasi-reductive seaweed. If $\mathfrak{s}$ has
nonzero center, then clearly $\mathfrak{s}$ is contact, so assume
$Z(\mathfrak{s})=0.$ Let $\varphi\in\mathfrak{s}^{*}$ be a regular form of
reductive type, and let $h$ be a semisimple element of $\mathfrak{s}$ that
spans $\ker(B_{\varphi}).$ If $\varphi(h)\neq 0,$ then $\mathfrak{s}$ is
contact by Lemma 3.

If $\varphi(h)=0,$ then construct a Cartan subalgebra
$\mathfrak{h}\subset\mathfrak{s}$ that contains $h,$ and let $\mathscr{B}_{h}$
be a Cartan-Weyl basis for $\mathfrak{s}$ generated from $h$ (see Coll et al.
[11], Lemma 55). Since, for all $x\in\mathfrak{s},$ $\dim(\ker(B_{\varphi}))$
is upper semicontinuous with respect to $\varphi(x),$ there exists
$\varepsilon>0$ and a one-form $\psi\in\mathfrak{s}^{*}$ such that if
$\psi(x)=\varphi(x)$ for all $x\in\mathscr{B}_{\mathfrak{h}}\setminus\\{h\\}$
and $|\psi(h)|<2\varepsilon,$ then

${\rm ind\hskip
2.84544pt}\mathfrak{g}=1\leq\dim\ker(B_{\psi})\leq\dim\ker(B_{\varphi})=1;$

that is, if $\mathscr{B}_{h}^{*}$ is the basis of $\mathfrak{s}^{*}$ dual to
$\mathscr{B}_{h},$ then we may choose an $\varepsilon>0$ such that
$\psi=\varphi+\varepsilon h^{*}$ is regular on $\mathfrak{s}.$ Fix such a
$\psi,$ and note that

$\ker(B_{\psi})=\ker(B_{\varphi})=\mathbb{C}h$

since $\mathscr{B}_{h}$ is a Cartan-Weyl basis. Therefore, $\psi$ is a regular
one-form on $\mathfrak{s}$ such that $\ker(B_{\psi})=\mathbb{C}h$ and
$\psi(h)=\varepsilon\neq 0,$ so $\mathfrak{s}$ is contact by Lemma 3.

For the forward implication (contact$\implies$QR), assume that $\mathfrak{s}$
is a contact seaweed with contact form $\varphi.$ We claim that $\varphi$ is
also a stable one-form. To establish the claim, let

$\mathscr{B}_{\varphi}=\\{E_{1},E_{2},\dots,E_{\dim(\mathfrak{s})}\\}$

be a contact basis of $\mathfrak{s}$ with respect to $\varphi,$ and let
$\mathscr{B}^{*}_{\varphi}$ be the basis of $\mathfrak{s}^{*}$ dual to
$\mathscr{B}_{\varphi}.$ Then $\varphi=E_{1}^{*}$ and

$\varphi([E_{1},E_{i}])=E_{1}^{*}\left(\sum_{r=2}^{\dim\mathfrak{s}}c_{1,i}^{r}E_{r}\right)=0,$
(1)

where $c_{i,j}^{r}\in\mathbb{R}$ for all $i=1,\dots,\dim(\mathfrak{s});$ that
is, $\ker(B_{\varphi})=\mathbb{C}E_{1}.$ Therefore, by Equation (1),

$[\ker(B_{\varphi}),\mathfrak{s}]\cap\ker(B_{\varphi})=\\{0\\},$

so $\varphi$ is stable by Lemma 1. An application of Theorem 2 completes the
proof. ∎

##  4 Epilogue

A Lie proset algebra $\mathfrak{P}$ is a subalgebra of a reductive Lie algebra
$\mathfrak{r}$ that contains a Cartan subalgebra of $\mathfrak{r}.$333The
moniker “proset” derives from the fact that, in the classical types, the
matrix forms of these algebras naturally correspond to pre-ordered sets (see
[23]). The category of Lie proset algebras contains seaweed algebras and Lie
poset algebras444If $\mathfrak{r}$ is a reductive Lie algebra with a Cartan
subalgebra $\mathfrak{h}$ and a corresponding Borel subalgebra $\mathfrak{b}$,
then a Lie poset algebra $\mathfrak{g}$ is any algebra such that
$\mathfrak{h}\subset\mathfrak{g}\subset\mathfrak{b}$. These algebras are Lie
semidirect products and are naturally defined by a partial ordering on a set,
where the relations are encoded in a Hasse diagram (see [8]). as
subcategories. In fact, the proof of Theorem 5 shows that if $\mathfrak{P}$ is
an index-one, quasi-reductive Lie proset algebra, then $\mathfrak{P}$ is
contact – and so stable. However, we do not yet know whether the notions of
quasi-reductivity and stability are equivalent for Lie proset algebras, and so
we are left with the question of whether or not every contact Lie proset
algebra is quasi-reductive. One approach to addressing this question is to
show that a contact Lie proset algebra admits a contact form with a semisimple
Reeb vector. This suffices since for a centerless, contact Lie proset algebra
$\mathfrak{P}$ with contact form $\varphi$,
$\faktor{\ker(B_{\varphi})}{Z(\mathfrak{P})}=\ker(B_{\varphi})=\mathbb{C}x_{\varphi},$
where $x_{\varphi}$ is the Reeb vector corresponding to $\varphi.$ Therefore,
if $x_{\varphi}$ is a semisimple element of $\mathfrak{P},$ then $\varphi$ is
a form of reductive type. Such an approach is reliant on a bevy of contact Lie
proset algebras; however, such examples are difficult to come by, mainly due
to lack of a comprehensive computational index theory which specializes to the
(rather complete) index theory of seaweeds (see [7], [14], and [22]) and the
nascent (though substantive) index theory of Lie poset algebras (see [9] and
[11]).

## References

  * [1] M. A. Alvarez, M. C. Rodríguez-Vallarte, and G. Salgado. “Deformation theory of contact Lie algebras as double extensions.” Proc. Am. Math. Soc., 149(5): 1827-1836, 2021.
  * [2] K. Ammari. “Stabilité des sous-algèbres biparaboliques des algèbres de Lie simples.” J. Lie Theory, 32(1): 239-260, 2022.
  * [3] K. Ammari. “Stabilité des sous-algèbres paraboliques des algèbres de Lie simples exceptionnelles.” Bull. des Sci. Math., 138(5): 614-625, 2014.
  * [4] K. Ammari. “Stabilité des sous-algèbres paraboliques de $\mathfrak{so}(n)$.” J. Lie Theory, 24(1): 97-122, 2014.
  * [5] N. H. Anh. “Algebraic groups with square-integrable representations.” Bull. Amer. Math. Soc., 80(3): 539-542, 1974.
  * [6] K. Baur and A. Moreau. “Quasi-reductive (bi)parabolic subalgebras in reductive Lie algebras.” Ann. Inst. Fourier, 61(2): 417-451, 2011.
  * [7] A. Cameron, V. E. Coll, Jr., and M. Hyatt. “Combinatorial index formulas for Lie algebras of seaweed type.” Commun. Algebra, 48(12): 5430-5454, 2020.
  * [8] V. E. Coll, Jr. and M. Gerstenhaber. “Cohomology of Lie semidirect products and poset algebras.” J. Lie Theory, 26(1): 79–95, 2016.
  * [9] V. E. Coll, Jr. and N. Mayers. “The index of Lie poset algebras.” J. Comb. Theory Ser. A, 177: 105331, 2021.
  * [10] V. E. Coll, Jr., N. Mayers, and N. Russoniello. “Contact Lie poset algebras.” Electron. J. Comb., 29(3): #P3.35, 2022.
  * [11] V. E. Coll, Jr., N. Mayers, and N. Russoniello. “The index and spectrum of Lie poset algebras of types B, C, and D.” Electron. J. Comb., 28(3): #P3.47, 2021.
  * [12] V. E. Coll, Jr., N. Mayers, N. Russoniello, and G. Salgado. “Contact seaweeds.” Pac. J. Math., 320(1): 45-60, 2022.
  * [13] V. E. Coll, Jr. and N. Russoniello. “Contact seaweeds II: type-C.” arXiv:2211.00095, 2022.
  * [14] V. Dergachev and A. Kirillov. “Index of Lie algebras of seaweed type.” J. Lie Theory, 10(2): 331-343, 2000.
  * [15] A. Diatta. “Left invariant contact structures on Lie groups.” Diff. Geom. Appl., 26(5): 544-552, 2008.
  * [16] J. Dixmier. Enveloping Algebras. Vol. 14. Newnes, 1977.
  * [17] M. Duflo. “Théorie de Mackey pour les groupes de Lie algébriques.” Acta. Math., 149: 153-213, 1982.
  * [18] M. Duflo, M. S. Khalgui, and P. Torasso. “Algèbres de Lie quasi-réductives.” Transform. Groups, 17: 417-470, 2012.
  * [19] H. Geiges. “Contact Geometry,” in F. J. E. Dillen and L. C. A. Verstraelen, editors, Handbook of Differential Geometry. Vol. 2, chapter 5: 315-382. Elsevier, 2006.
  * [20] M. Goze and E. Remm. “Contact and Frobeniusian forms on Lie groups.” Diff. Geom. Appl., 35: 74-94, 2014.
  * [21] M. Gromov. “Stable mappings of foliations into manifolds.” Izv. Akad. Nauk SSSR Ser. Mat., 33(4): 707-734, 1969; Math. USSR-Izv, 3(4): 671-694, 1969.
  * [22] A. Joseph. “On semi-invariants and index for biparabolic (seaweed) algebras, I.” J. Algebra, 305(1): 487-515, 2006.
  * [23] N. Mayers and N. Russoniello. “On toral posets and contact Lie algebras.” J. Geom. Phys., 190: 104861, 2023.
  * [24] A. Moreau and O. Yakimova. “Coadjoint orbits of reductive type of parabolic and seaweed Lie subalgebras.” Int. Math. Res. Not., 2012(19): 4475-4519, 2012.
  * [25] D. Panyushev. “An extension of Raïs’ theorem and seaweed subalgebras of simple Lie algebras.” Ann. de l’Institut Fourier, 55(3): 693-715, 2005.
  * [26] D. Panyushev and O. Yakimova. “On seaweed subalgebras and meander graphs in type D.” J. Pure Appl. Algebra, 222(11): 3414-3431, 2018.
  * [27] G. Salgado-González. “Invariants of contact Lie algebras.” J. Geom. Phys., 144: 388-396, 2019.
  * [28] P. Tauvel and R. Yu. “Indice et formes linéaires stables dans les algèbres de Lie.” J. Algebra, 273(2): 507-516, 2004.