# Model-independent Reconstruction of UV Luminosity Function and Reionization
Epoch

Debabrata Adak  Dhiraj Kumar Hazra  Sourav Mitra  Aditi Krishak

###### Abstract

We conduct a first comprehensive study of the Luminosity Function (LF) using a
non-parametric approach. We use Gaussian Process to fit available luminosity
data between redshifts $z\sim 2-8$. Our free-form LF in the non-parametric
approach rules out the conventional Schechter function model to describe the
abundance-magnitude relation at redshifts $z=3$ and $4$. Hints of deviation
from the Schechter function are also noticed at redshifts 2, 7 and 8 at lower
statistical significance. Significant deviation starts for brighter ionizing
sources at $M_{\rm UV}\lesssim-21$. The UV luminosity density data at
different redshifts are then derived by integrating the LFs obtained from both
methods with a truncation magnitude of $-17$. In our analysis, we also include
the first 90 arcmin2 JWST/NIRCam data at $z\sim 9-12$. Since at larger
magnitudes, we do not find major deviations from the Schechter function, the
integrated luminosity density differs marginally between the two methods.
Finally, we obtain the history of reionization from a joint analysis of UV
luminosity density data along with the ionization fraction data and Planck
observation of Cosmic Microwave Background. The history of reionization is not
affected by the deviation of LFs from Schechter function at lower magnitudes.
We derive reionization optical depth to be $\tau_{\rm
re}=0.0494^{+0.0007}_{-0.0006}$ and the duration between 10% and 90%
completion of ionization process is found to be $\Delta z\sim
1.627^{+0.059}_{-0.071}$.

##  1 Introduction

The process of cosmic reionization is an outstanding problem in extragalactic
astronomy. During the redshifts $z\sim 6-20$ almost all of the hydrogen in the
universe became ionized [1, 2, 3, 4, 5]. A similar process was followed for
helium at a later time and helium reionization was completed at around $z\sim
2.5-3.5$ [6]. For this process, the high-redshift ($z\gtrsim 6$) star-forming
galaxies are often considered to be the dominant contributors of ionizing
photons since the abundance of quasars dramatically declines at $z\sim 6$ [7,
8, 9]. A variety of theoretical and observational studies have shown that the
contribution of Active Galactic Nuclei has a very minimal ($\lesssim$ 1%)
contribution to the total ionisation budget at $z\geq$ 6 [10, 11]. Most of the
studies suggest that star-forming galaxies inside the low mass halo
($M_{h}\lesssim 10^{9.5}M_{\odot}$) are sufficient enough to complete the
reionization process [12, 13]. Therefore, their time-dependent abundance, and
the redshift evolution of UV luminosity density ($\rho_{\rm UV}$) derived from
rest-frame UV luminosity function (LF) of galaxies are of significant interest
for understanding reionization history (see, [14]).

Significant advancement has been made in determining the behaviour of LFs at
redshifts $z\sim 4-10$ using Hubble Frontier Fields (HFF, [15]) survey data up
to magnitude $\simeq-15$ [16, 17, 18]. More recently, the same has been
studied in [19, 20] and [21] with galaxy candidates down to magnitude
$\simeq-22$. However, along with studying the faint end of the UV LF, it is
also important to investigate the bright-end shape of LF. While the
conventional Schechter function [22] seems to be a good description of the LFs
for ionizing candidates at the fainter end of the magnitude, an exponential
decline has been reported in their number densities at the bright-end [23]. It
is thought to be caused by heating from an active galactic nucleus (AGN,
[24]), inefficient cooling of gas inside high-mass dark matter halos at low
redshifts [25] etc. Recent works of [26] and [27] found that both double
power-law (DPL, [28]) and lensed Schechter function [29] are better fitted to
UV luminosity data including brightest galaxy candidates observed in the early
data from James Webb Space Telescope (JWST, [30]) and Great Optically Luminous
Dropout Research data of Subaru HSC (GOLDRUSH) (corrected for AGN
contribution) respectively. Publications [31] and [32] independently
established that LFs are more consistent with DPL at $z\sim 9-10$. More recent
studies in [26, 33] and [34] also reported an excess of number density
compared to that described by the Schechter best fit at $z\sim 4-7$.

In this paper, we address the constraints on reionization history in a two-
step process. First, we reconstruct the profile of the LFs using Gaussian
process regression (GP), a nonparametric regression method at redshifts $z\sim
2-8$, and Schechter function model at redshifts $z\sim 2-12$ using the HFF
data, early JWST data and HSC data. We compare these model-independent LFs
with the conventional Schechter function model of LFs. We derive the UV
luminosity densities by integrating LFs, and investigate the redshift
evolution of UV luminosity densities derived from both profiles. Finally, we
use these luminosity densities to constrain the history of reionization along
with jointly fitting two other observational data: the CMB power spectrum data
of temperature and polarization anisotropy from Planck observation [4] and
neutral hydrogen fraction data from galaxy, quasar and gamma-ray burst
observations.

The paper is organised as follows. In section 2 we briefly discuss the
reionization process, different functions used and details of Gaussian process
regression. In section 3 we describe the ancillary data sets used in this
work. In section 4 we present our results for the UV luminosity functions
obtained using two different methods, their comparison, derived UV luminosity
densities and the corresponding constraints on reionization history. Finally
in section 5 we summarise the work.

##  2 Methodology

###  2.1 Cosmic Reionization

The process of reionization of the intergalactic medium (IGM) is a balance
between ionization of hydrogen and helium atoms by cosmic photons and
recombination of free electrons and protons to form neutral hydrogen and
helium. Analytical and numerical modelling of this process traces a long
history [35, 13, 36] (see the references therein). The process is studied by
the redshift evolution of volume filling factor $Q{HII}$ which is governed by
the ionization histories of both hydrogen and helium. The time evolution of
$Q{HII}$ is obtained by solving the ordinary differential equation (e.g. [37])

$\dot{Q}{HII}=\frac{\dot{n}_{ion}}{\langle
n_{H}\rangle}-\frac{Q{HII}}{t_{rec}},$ (2.1)

where $\langle n_{H}\rangle=\frac{X_{p}\Omega_{b}\rho_{c}}{m_{H}}$ is the mean
comoving number density of hydrogen and depends on the primordial mass-
fraction of hydrogen $X_{p}$, critical density $\rho_{c}$, baryon density
$\Omega_{b}$ and $m_{H}$ is the mass of atomic hydrogen. $t_{rec}$ denotes the
average recombination time in the IGM,

$t_{rec}=\frac{1}{C_{\rm
HII}\alpha_{B}(T)\left(1+\frac{Y_{p}}{4X_{p}}\right)\langle
n_{H}\rangle(1+z)^{3}},$ (2.2)

where $\alpha_{B}(T)$ is the recombination coefficient for hydrogen (we assume
the IGM temperature T to be 20,000 K) and $Y_{p}=1-X_{p}$ is the primordial
helium abundance. The clumping factor $C_{\rm HII}$ accounts for the
inhomogeneity of the IGM, and is not very well constrained from observations.
Recent simulations suggest a possible range of $C_{\rm HII}$ value from 1 to 6
[38, 39, 40, 41]. In this work we use the fixed value of $C_{\rm HII}=3$ [42]
for simplicity. The comoving production rate of available ionizing photons in
the IGM at some redshift is

$\displaystyle\dot{n}_{ion}$ $\displaystyle=\int_{-\infty}^{M_{\rm
trunc}}f_{esc}(M)\xi_{ion}(M)\Phi(M)L(M)dM$ $\displaystyle=\langle
f_{esc}\xi_{ion}\rangle\rho_{\rm UV}.$ (2.3)

This depends on the intrinsic production rate of Lyman continuum (LyC) photons
supplied by stellar populations of galaxies, which is parameterized by a
numerical factor $\xi_{ion}$ to count the ionizing photons per unit UV
luminosity, the escape fraction $f_{esc}$ and total luminosity density
$\rho_{\rm UV}$ from star-forming galaxies with a truncation absolute
magnitude $M_{\rm trunc}$. $f_{esc}$ is a crucial parameter and is not well-
constrained from direct observations of LyC photons [43], which is mainly
limited to $z\sim$ 2 – 4.5 due to a dramatic increase in the opacity of IGM at
high redshifts making direct observation of LyC photons difficult [44].
Moreover, the trend of $f_{esc}$ with halo mass is not well understood from
theoretical modelling [45, 46, 47]. All pre-JWST literatures suggest that a
low ionisation efficiency of around $\log_{10}\xi_{ion}=25.2$ Hz $erg^{-1}$
and hence $f_{esc}$ of 0.2 [37] that are sufficient to give reionization
history consistent with CMB data. However new JWST data suggests a higher
$\xi_{ion}$ [48] that increases at redshifts higher than $z\sim$ 9 [49, 50].
It is apparent that $\xi_{ion}$ and $f_{esc}$ are completely degenerate
parameters. A recent data-driven model-independent study [51] found that a
constant value of $f_{esc}$ for $z\geq$ 6 is permitted. Therefore, we take
$f_{esc}$ as a constant factor and, instead of considering $\xi_{ion}$ and
$f_{esc}$ to be independent parameters, we consider the magnitude-averaged
value of $\langle f_{esc}\xi_{ion}\rangle$ as a single parameter in this work.

Luminosity density $\rho_{\rm UV}$ is obtained from the luminosity function
$\phi(M_{\rm UV})$ via integration,

$\rho_{\rm UV}=\int_{-\infty}^{M_{\rm trunc}}\Phi(M)L(M)dM,$ (2.4)

where $L(M)$ is the luminosity. One can set the truncation magnitude at
$M_{\rm trunc}=-10$ corresponding to the predicted range of minimum halo-mass
that can host star-forming galaxies [52]. However, due to insufficient data at
some redshifts at larger magnitudes ($M_{\rm UV}>-17$), our free-form LFs
obtained using GP will be biased to the mean function beyond the range of
training data. Therefore, we restrict ourselves to $M_{\rm trunc}=-17$.
Accurate estimation of $\rho_{\rm UV}$ from UV galaxies requires a careful
analysis of the LF profile that can well-describe the number densities of
star-forming galaxy samples down to observed limits. The best model of LF in
literature is often assumed to be the Schechter function [22],

$\Phi(M)=0.4\,\ln
10\,\phi^{*}\,[10^{0.4(M^{*}-M)}]^{1+\alpha}\,\exp[-10^{0.4(M^{*}-M)}]$ (2.5)

parameterized by $\phi^{*}$ (Mpc-3mag-1), $M^{*}$ and $\alpha$. We find the
posterior distributions of Schechter function parameters in redshifts $z\sim
2-12$ using the UV luminosity data sets described in section 3. We also fit
same data sets using the non-parametric method discussed in subsection 2.2. We
then obtain $\rho_{\rm UV}$ corresponding to each of these best-fit LFs using
Equation 2.5.

In order to constrain the reionization process, we need to adopt a parametric
form [53, 19, 20] or a free-form [54, 55, 56, 57] of $\rho_{\rm UV}$ evolution
to constrain $\dot{n}_{ion}$. The model-independent reconstruction of
reionization process in [56] rules out the single power-law [53] form which
that is unable to replicate the decline at $z\sim 8$. This results in an
incorrect value of the Thomson scattering optical depth. Therefore, in our
analysis we use the logarithmic double power law [19, 20] to describe
$\rho_{\rm UV}$,

$\rho_{\rm UV}=\frac{2\rho_{{\rm UV},z=z_{\rm tilt}}}{10^{a(z-z_{\rm
tilt})}+10^{b(z-z_{\rm tilt})}},$ (2.6)

where $\rho_{{\rm UV},z=z_{\rm tilt}}$ is the normalization factor at $z_{\rm
tilt}\sim 8$, and $a$ and $b$ are the slopes.

Once the evolution of $Q{HII}$ from Eq. 2.1 is determined, we compute the
reionization optical depth at redshift $z$ using

$\tau_{\rm
re}=\int_{0}^{z}\frac{c(1+z^{{}^{\prime}})^{2}}{H(z^{{}^{\prime}})}Q{HII}(z^{{}^{\prime}})\langle
n_{H}\rangle\sigma_{T}(1+\eta\frac{Y_{p}}{4X_{p}})dz^{{}^{\prime}},$ (2.7)

where $c$ is the speed of light, $H(z)$ is the Hubble parameter, and
$\sigma_{T}$ is the Thomson scattering cross section. Here we assume that
helium is singly ionized ($\eta$ = 1) at $z>4$ and doubly ionized ($\eta$ = 2)
at $z\leq 4$ [36].

###  2.2 Gaussian Process Regression

Gaussian process regression is a non-parametric Bayesian regression method.
This method has been extensively used in cosmological data analysis [58, 59,
60]. A Gaussian process is a collection of random variables such that any
finite subset of these random variables has a multivariate Gaussian
distribution [61]. A GP is described by its mean and covariance functions,
defined as $\mu(\mathrm{x})=\mathbb{E}[f(\mathrm{x})]$, and
$k(\mathrm{x},\mathrm{x^{\prime}})=\mathbb{E}[(f(\mathrm{x})-\mu(\mathrm{x}))(f(\mathrm{x^{\prime}})-\mu(\mathrm{x^{\prime}}))]$,
respectively, for a real process $f(\mathrm{x})$. In particular, here
$f(\mathrm{x})$ defines the luminosity-magnitude relation guided by data for a
given mean function. The covariance function gives the covariance between two
random variables and characterizes the covariance matrix having elements
$C_{i,j}=k(x_{i},x_{j})$. Given a finite set of training points
$\mathrm{x}=\\{x_{i}\\}$, a function $f(\mathrm{x})$ evaluated at each $x_{i}$
is a Gaussian random variable and the vector $\mathrm{f}=\\{f_{i}\\}$ has a
multivariate Gaussian distribution given as
$\mathrm{f}\sim\mathcal{N}\left(\mu(\mathrm{x}),C(\mathrm{x},\mathrm{x})\right)$.
The choice of the covariance functions is important. We choose the Radial
Basis Function (RBF) kernel as the covariance function for our analysis,
defined as
$k(x_{i},x_{j})=\sigma_{\ell}\text{exp}\left(-\frac{(x_{i}-x_{j})^{2}}{2\ell^{2}}\right)$,
where $\sigma_{\ell}$ and $\ell$ are the kernel hyperparameters. The
$\sigma_{\ell}$ is the amplitude parameter that can be thought as an offset
that decides the tilt of reconstructed function $f(\mathrm{x})$ from given
mean function and $\ell$ describes the characteristic length of correlation.
Data points act as training points to optimize the hyperparameters and provide
posterior prediction along with uncertainty on the predictions for the given
test points.

Although GP is a formalism to be used here to study actual trends of data to
define the luminosity-magnitude relation, there are practical technicalities
regarding the choice of mean function that require special care. In principle,
one can choose a zero mean function or any random function to start with. In
this study, however, we are particularly interested in checking the validity
of the Schechter function to be a correct description of the luminosity-
magnitude relation. To test that, we allow the three Schechter function
parameters to vary along with the GP hyperparameters, $\ell$ and
$\sigma_{\ell}$. In this way, the hyperparamter posteriors, marginalized over
the Schechter function parameters can indicate, in a conservative way, whether
the Schechter function is a correct model to address the observational data.

##  3 Data sets

We use rest frame UV luminosity function data for redshifts $z\sim
2,~{}3,~{}4,~{}5,~{}6,~{}7$ derived in [62] using HUDF, HFF, and CANDELS
fields and HFF data compiled by [20]. For $z\sim 4-7$ we add data from Hyper
Suprime-Cam (HSC) Subaru Strategic Program (SSP) survey [63]. These data sets
are corrected for active galactic nucleus (AGN) contribution and mostly at
brighter end of luminosity. For redshifts 8, 9 and 10 we use luminosity
function data from [62], [32], [64] and [65]. We also use the data redshift 9
and 12 from JWST [66].

We use neutral hydrogen fraction data to constrain reionization from
observations of Ly$\alpha$-emitting galaxies [67, 68, 69, 46], high-redshift
quasar spectra [41, 70, 71], gamma ray bursts [72, 73], dark fraction in the
spectra of bright quasars [2] and ionized near-zones around high-redshift
quasars [74, 75].

From Planck CMB, we use Planck binned $\tt Plik$ TTTEEE likelihood with low
multipole temperature and polarization likelihoods and the lensing likelihood
as discussed in Planck baseline [4].

##  4 Results

###  4.1 Relation between magnitude and Galaxy UV LF

To characterise the galaxy UV LF, we use both Schechter function (Equation
2.5) and GP to fit the luminosity data. We use data for the largest possible
range of magnitude, $-25<M_{\rm UV}<-14$ wherever available. We use CosmoMc
[76] as a generic sampler to explore the parameter space.

For fitting the Schechter function we keep all three parameters
$M^{*},\phi^{*}$ and $\alpha$ free. However, we notice that for $z>8$ the data
is not able to constrain all the parameters. Therefore, at $z\geq 9$, we keep
the slope $\alpha$ fixed at $-2.35$ following [27] and determine the other two
Schechter parameters at $z\sim$ 9–12. We find the estimated parameters are
consistent with previous results of [32, 62, 20]. For $z\sim 10$ and $12$, we
fix the prior on $M^{\ast}_{UV}$ as the 68% range obtained in $z\sim 9$
analysis. In Figure 1 we present posterior distributions of the Schechter
function parameters. We fit the free-form of LF using GP with the same data
sets keeping GP hyperparameters ($\ell$ and $\sigma_{\ell}$) free along with
three Schechter parameters. We avoid GP fitting at $z\sim$ 9 – 12 due to the
unavailability of low magnitude high signal-to-noise data where deviation is
expected and at these redshifts, even all Schechter parameters cannot be
optimised. The analysis of ten redshifts is divided into three plots.

Figure 1: Constraints on the Schechter function parameters using galaxy
luminosity data at redshifts $z\sim 2-12$.

Random samples of luminosity function from the chains for Schechter function
are plotted in blue lines in Figure 2 and 3. Modified functions from the
Gaussian process are plotted in orange lines in the same figures. Deviations
from Schechter function below magnitude $M_{\rm UV}=-21$ can be noticed. A
similar deviation is also reported in [26]. It is important to note that most
of the previous studies are based on HFF and HUDF data [77, 21, 20, 62] and
others ground-based observations [32] that are limited by the magnitude larger
than $-23$. According to those studies, Schechter function is the best
description of the magnitude-luminosity relation. In our study, we combine
those previous data sets with ancillary luminosity data sets of brighter
galaxies ($M_{\rm UV}\lesssim-23$) from HSC data [27]. Above this magnitude,
the AGN contamination to the galaxy luminosity is negligible [78, 79, 80, 81,
82] and therefore the previous results are consistent with our results up to
certain magnitudes. The added HSC data of brighter galaxies are corrected for
contribution from AGNs [27]. We find maximum excess in the bright end shape of
galaxy UV LF instead of expected exponential drop of Schechter function at all
redshifts where low magnitude ($M_{\rm UV}\lesssim-21$) data are available. In
certain redshifts, due to high signal-to-noise ratio, the deviation is
detected at high statistical significance while for other redshifts we obtain
a similar deviation with lower significance. Importantly, at $z=3,4$ the
hyperparameters posteriors plotted in the upper left panel of Figure 4
indicate that the Schechter function (used as a mean function in this
analysis) is ruled out by the data at high significance. Data from $z=2$ and
$7$ also prefer modification over Schechter function at around $\sim$95%
confidence level (C.L). In order to understand the source for this deviation,
we reanalyse $z=4$ data with two different data cuts (with $M_{\rm UV}>-23$
and $M_{\rm UV}>-21$). The results are presented in the lower right panel of
Figure 4. When the brightest part of the observation $-24<M_{\rm UV}<-23$ is
not used, we notice a decrease in the significance in the GP hyperparameters
ruling out Schechter function at 3$\sigma$. When we use more conservative cuts
by masking data between $-24<M_{\rm UV}<-21$, we find that further drop in
significance. These tests reveal that the modifications to Schechter function
are needed by the luminosity observations at the lowest magnitude (the
brightest) objects.

Figure 2: Comparison of luminosity functions from GP and Schechter function
model derived from posterior samples of corresponding parameters at redshifts
$z\sim 2-7$.

Figure 3: Same as Figure 2 for $z\sim 8,9,10$ and $12$. Model | Schechter fit |  Gaussian process $\log_{10}\rho_{\rm UV}$
---|---|---
$z$ | $\log_{10}\phi^{\ast}$ | $M^{\ast}_{\rm UV}$ | $\alpha$ | $\log_{10}\rho_{\rm UV}$
2 | $-2.238\pm 0.026$ | $-19.940\pm 0.061$ | $-1.357\pm 0.012$ | $26.426\pm 0.010$ | $26.436\pm 0.012$
3 | $-2.297\pm 0.019$ | $-20.216\pm 0.027$ | $-1.301\pm 0.015$ | $26.4741\pm 0.0063$ | $26.489^{+0.014}_{-0.012}$
4 | $-2.689\pm 0.047$ | $-20.711\pm 0.057$ | $-1.673\pm 0.040$ | $26.470\pm 0.014$ | $26.470\pm 0.013$
5 | $-3.095\pm 0.062$ | $-21.039\pm 0.077$ | $-1.755^{+0.047}_{-0.055}$ | $26.264\pm 0.019$ | $26.275\pm 0.016$
6 | $-3.28\pm 0.12$ | $-20.90\pm 0.13$ | $-1.962^{+0.076}_{-0.091}$ | $26.165^{+0.033}_{-0.028}$ | $26.188^{+0.022}_{-0.026}$
7 | $-3.17\pm 0.11$ | $-20.53\pm 0.14$ | $-1.890^{+0.062}_{-0.074}$ | $26.039\pm 0.022$ | $26.071^{+0.034}_{-0.023}$
8 | $-3.88^{+0.51}_{-0.23}$ | $-20.68^{+0.46}_{-0.24}$ | $-2.13\pm 0.27$ | $25.616^{+0.099}_{-0.066}$ | $25.705\pm 0.084$
9 | $-4.94^{+0.46}_{-0.31}$ | $-21.36^{+0.54}_{-0.38}$ | $-2.35$ | $25.12^{+0.16}_{-0.11}$ | –
10 | $-5.79^{+1.1}_{-0.84}$ | $-21.8^{+1.6}_{-1.2}$ | $-2.35$ | $24.51^{+0.19}_{-0.15}$ | –
12 | $-6.19^{+0.67}_{-1.3}$ | $<-19.1$ | $-2.35$ | $24.40^{+0.23}_{-0.12}$ | –
Table 1: Constraints on luminosity density obtained from Schechter fit and the
GP fit to the data at different redshifts between $2<z<8$. For
$z=9,~{}10,~{}12$ we just provide only the Schechter fit as even the three
parameters can not be constrained with two tailed distribution from the data,
due to low signal-to-noise ratio.

Figure 4: Constraints on the GP hyperparameters at different redshifts.
Redshifts between $z=2-12$ are divided into 3 plots. At lower redshifts,
mainly at $z=3$ and $4$, the data prefers significant deviation from the
Schechter function. At higher redshifts, although there are some hints of
deviations at $z=7,~{}8$, they are not statistically significant. For $z=4$,
where we find most significant deviation, we reanalyze the data with two cuts
$M_{\rm UV}>-23$ and $M_{\rm UV}>-21$ (bottom right plot). We notice that when
we include the data from lower magnitudes, the Schechter function becomes
increasingly inconsistent with the data. Figure 5: Luminosity density
evolution across redshift $z\sim 4-12$. Black circles and blue diamonds are
$\rho_{\rm UV}(z)$s obtained from integrating the luminosity functions from GP
and Schechter respectively down to the magnitude $M_{\rm UV}$ = –17. The
orange line is best-fit logarithmic double power-law. The shaded regions are
1$\sigma$ and 2$\sigma$ confidence level to the logarithmic double power-law
fit to $\rho_{\rm UV}$ obtained from Schechter function fit. Figure 6:
Redshift evolution of $Q{HII}$ across $z\sim$4 –15. The dashed blue and solid
magenta lines are best-fit $Q{HII}$ estimated using $\rho_{\rm UV}$s from GP
and Schechter functions respectively. The gray (cyan) curves are random
samples of $Q{HII}$ for these two respective cases. The $Q{HII}$ data sets
used for joint fitting are also shown.

###  4.2 Evolution of UV Luminosity Density

We compute the UV luminosity density $\rho_{\rm UV}$ following Equation 2.4
integrating down to $M_{\rm UV}=-17$. The samples of LFs from Schechter and GP
analyses are used to estimate the posterior distribution of the derived
parameter $\rho_{\rm UV}$. The mean and the 68% bounds are provided in Table
1. The changes in luminosity density compared to the Schechter function model
are not noticeable in the GP results. Though Schechter function is ruled out
by the data at certain redshifts, the major required modifications are noticed
at the brightest end of the function and the brightest end does not contribute
significantly to the integral of the UV luminosity density as the function
drops logarithmically with the increase in brightness. In Figure 5, we plot
the redshift evolution of the luminosity density obtained from the GP (and
Schechter fit) and a logarithmic double power law fit to the data (for $z\geq
6$). We demonstrate only 1$\sigma$ and 2$\sigma$ spread for logarithmic double
power-law fit to luminosity density obtained from the Schechter fit. We notice
outliers in the data w.r.t. the model around redshifts 9 and 10. Recent JWST
data suggests an enhanced population of star-forming Galaxies above redshifts
$z\sim 9$ [83, 84]. Compared to [56] new data may seem to hint towards a
modification to the luminosity density evolution model. However, note that the
data from JWST can not constrain all of the Schechter function parameters due
to low signal-to-noise ratio detection, in particular, the slope remains
unconstrained (see, Figure 1). This may bias the estimation of the luminosity
density and therefore, with this data we avoid the exploration for the
possible deviation from the double power law model. We expect to revisit this
issue in the future when new data from JWST are made available.

###  4.3 Constraints on Reionization

We obtain the constraint on redshift evolution of IGM neutral hydrogen
fraction from joint fitting of Planck data, $Q{HII}$ data, and UV luminosity
density data derived in subsection 4.2. We treat the four parameters in
logarithmic double power-law form of $\rho_{UV}$ and $\log_{10}\langle
f_{esc}\xi_{ion}\rangle$ as free parameters. For $C_{\rm HII}$ we use a fixed
value of 3.

In Figure 6 we compare the $Q{HII}$ evolution across redshift $z\sim 4-15$
solving Equation 2.1 using $\rho_{\rm UV}$ data sets obtained from both
parametric and non-parametric methods. The blue dashed line and magenta solid
line are the best fit $Q{HII}$ obtained with $\rho_{\rm UV}$ from GP and
Schechter fits respectively. The grey and cyan lines indicate random samples
of $Q{HII}$ obtained from these fits respectively. We also overplot the
$Q{HII}$ data set used in our analysis. Both the $\rho_{\rm UV}$ data sets
provide similar constraints on reionization history. This is not surprising as
we discussed that the obtained luminosity density follows similar redshift
evolution owing to the similar nature of luminosity functions at the faint
source end. This result suggests that the reionization process is mainly
driven by fainter galaxies. The excess dropout galaxies at the brighter-end of
magnitude have limited contribution to the reionization process due to lower
number density or possibly low escape fraction. Earlier publications in [85,
86, 87, 11, 48] reached to a similar conclusions using model-based
reionization study. As shown in Figure 6 we find much tighter bound of
$Q{HII}$ as compared to previous works in [85, 86, 54, 56, 51]. We quote the
midpoint (50%) redshift of reionization $z_{\rm re}$ in Table 2 which is less
than the Planck reported value [4]. Our constraints on the duration of
reionization (redshift difference between 10% to 90% reionization) are $\Delta
z\sim 1.627^{+0.059}_{-0.071}$ and $1.627^{+0.060}_{-0.070}$ with 68%
confidence interval using Schechter function and GP based $\rho_{\rm UV}$ data
sets respectively. This is consistent with a upper bound of $\Delta$z$<$2.8 as
reported in [88] from the Kinetic Sunyaev–Zel’dovich effect. This implies our
values suggest a sharper reionization history. Finally our constrain on
reionization optical depth is $\tau_{\rm re}=0.0492^{+0.0008}_{-0.0006}$ and
$0.0494^{+0.0007}_{-0.0006}$ from Schechter and GP analyses. These values are
also listed in Table 2. However, we would like to highlight here that the
uncertainties in the reionization histories quoted in this subsection are
underestimated as we have fixed both the clumping factor and the escape
fraction in our analyses. Therefore, these summary statistics on optical depth
and the duration of reionization should be used with caution in any subsequent
analysis. The exercise in this subsection has been performed to indicate that
the constraints on the cosmology remain unaltered even though Schechter
function is ruled out by the observational data. The flattening/increase in
the luminosity density data at high reshifts as indicated by the JWST data in
this analysis does not show an impact in the optical depth constraints as –
(1) we have not used the very high redshift observations ($z\sim 14,~{}16$)
from JWST, and (2) the double power-law form used for the luminosity density
does not capture its increasing trend at high redshifts.

| using $\log_{10}[\rho_{\rm UV}]$ from Schechter | using $\log_{10}[\rho_{\rm UV}]$ from GP
---|---|---
$\tau_{\rm re}$ | 0.0492${}^{+0.0008}_{-0.0006}$ | 0.0494${}^{+0.0007}_{-0.0006}$
$z_{\rm re}$ | $6.778^{+0.054}_{-0.029}$ | $6.777^{+0.055}_{-0.028}$
$\Delta z$ | $1.627^{+0.059}_{-0.071}$ | $1.627^{+0.060}_{-0.070}$
Table 2: Summary of constraints on reionization history using $\rho_{\rm UV}$
data sets obtained from both parametric and non-parametric methods.

##  5 Summary

In this paper, we investigate the UV luminosity functions between redshifts
$z\sim 2-12$ over a wide range of magnitude $-25\lesssim M_{\rm
UV}\lesssim-14$ based on HST, HSC and JWST data sets. We test whether the
conventional Schechter function is a valid theoretical model that can address
the data for such a wide range of magnitudes and redshifts and what are the
implications of any modified function to the reionization history.

We fit both commonly used Schechter function model at all redshifts $z\sim
2-12$ and perform a free-form reconstruction at redshifts $z\sim 2-8$ using
Gaussian process to the same data sets. We find although Schechter function is
a very good description of the dropout galaxies for the fainter end of
magnitude ($M_{\rm UV}\gtrsim-21$), its exponential tail is inconsistent with
brighter dropout galaxies at almost all redshifts where low magnitude data are
available. The Gaussian process regression allows a free-form reconstruction
of UV LFs and therefore can well describe the excess LF at the bright-end
which is not possible to address with Schechter function. We obtain the UV
luminosity densities integrating over both LF forms down to the magnitude
$M_{\rm UV}=-17$. Since at the fainter end of the magnitude, Schechter
function is consistent with the data and the fainter end contributes
dominantly to the luminosity density integral due to the maximum availability
of Galaxies, we find similar luminosity densities in both the methods.
Therefore, reionization history is found to be similar in both cases. This
implies that brighter dropout galaxies have insignificant contributions in
reionization process (supported by earlier publications as well). However,
integrating to more fainter sources $M_{\rm UV}>-15$ can indicate certain
differences which we do not explore by considering $M_{\rm UV}=-17$ as the
conservative choice.

The data at redshifts 9, 10 and 12 do not have a high signal-to-noise ratio
and therefore can not constrain all the Schechter function parameters.
Therefore testing a modification to Schechter function is beyond the scope of
this paper with the available data. More observational data from JWST NIRSpec
will be required for further studies at higher redshifts. Furthermore, we have
not incorporated redshift dependencies of clumping factor and escape fraction
into our simplistic reionization model and also neglect the contributions from
quasars as reionization sources which could potentially enhance the accuracy
of our findings. It would be intriguing to incorporate all these modifications
into our model and reassess the current analysis when new data from JWST and
other sources with significant improvement in the signal-to-noise ratio are
made available. We defer these extensions for future investigations.

##  6 Acknowledgement

All the computations in this paper are done using the HPC Nandadevi and Kamet
(https://hpc.imsc.res.in) at the Institute of Mathematical Sciences, Chennai,
India. The authors would like to thank Yuichi Harikane for providing the UV
luminosity data sets used in this work. DKH would like to thank Daniela
Paoletti for certain important discussions. DKH would like to acknowledge the
support from the Indo-French Centre for the Promotion of Advanced Research –
CEFIPRA grant no. 6704-4 and the support through the India-Italy “RELIC -
Reconstructing Early and Late events In Cosmology” mobility program.

## References

  * [1] X. Fan, M.A. Strauss, R.H. Becker, R.L. White, J.E. Gunn, G.R. Knapp et al., _Constraining the Evolution of the Ionizing Background and the Epoch of Reionization with z ~6 Quasars. II. A Sample of 19 Quasars_, _AJ_ 132 (2006) 117 [astro-ph/0512082].
  * [2] I.D. McGreer, A. Mesinger and V. D’Odorico, _Model-independent evidence in favour of an end to reionization by z $\approx$ 6_, _MNRAS_ 447 (2015) 499 [1411.5375].
  * [3] D.K. Hazra and G.F. Smoot, _Witnessing the reionization history using cosmic microwave background observation from planck_ , _Journal of Cosmology and Astroparticle Physics_ 2017 (2017) 028.
  * [4] Planck Collaboration, N. Aghanim, Y. Akrami, M. Ashdown, J. Aumont, C. Baccigalupi et al., _Planck 2018 results. VI. Cosmological parameters_ , _A &A_ 641 (2020) A6 [1807.06209].
  * [5] D. Paoletti, D.K. Hazra, F. Finelli and G.F. Smoot, _The asymmetry of dawn: evidence for asymmetric reionization histories from a joint analysis of cosmic microwave background and astrophysical data_ , 2405.09506.
  * [6] G. Worseck, J.X. Prochaska, J.F. Hennawi and M. McQuinn, _Early and Extended Helium Reionization over More Than 600 Million Years of Cosmic Time_ , _ApJ_ 825 (2016) 144 [1405.7405].
  * [7] A. Fontana, E. Vanzella, L. Pentericci, M. Castellano, M. Giavalisco, A. Grazian et al., _The Lack of Intense Ly $\alpha$ in Ultradeep Spectra of z = 7 Candidates in GOODS-S: Imprint of Reionization?_, _ApJ_ 725 (2010) L205 [1010.2754].
  * [8] C.J. Willott, P. Delorme, C. Reylé, L. Albert, J. Bergeron, D. Crampton et al., _The Canada-France High-z Quasar Survey: Nine New Quasars and the Luminosity Function at Redshift 6_ , _AJ_ 139 (2010) 906 [0912.0281].
  * [9] M. Onoue, N. Kashikawa, C.J. Willott, P. Hibon, M. Im, H. Furusawa et al., _Minor Contribution of Quasars to Ionizing Photon Budget at z $\sim$ 6: Update on Quasar Luminosity Function at the Faint End with Subaru/Suprime-Cam_, _ApJ_ 847 (2017) L15 [1709.04413].
  * [10] S. Hassan, R. Davé, S. Mitra, K. Finlator, B. Ciardi and M.G. Santos, _Constraining the contribution of active galactic nuclei to reionization_ , _Monthly Notices of the Royal Astronomical Society_ 473 (2017) 227 [https://academic.oup.com/mnras/article-pdf/473/1/227/21308514/stx2194.pdf].
  * [11] P. Dayal, M. Volonteri, T.R. Choudhury, R. Schneider, M. Trebitsch, N.Y. Gnedin et al., _Reionization with galaxies and active galactic nuclei_ , _MNRAS_ 495 (2020) 3065 [2001.06021].
  * [12] C. Liu, S.J. Mutch, P.W. Angel, A.R. Duffy, P.M. Geil, G.B. Poole et al., _Dark-ages reionization and galaxy formation simulation – IV. UV luminosity functions of high-redshift galaxies_ , _Monthly Notices of the Royal Astronomical Society_ 462 (2016) 235 [https://academic.oup.com/mnras/article-pdf/462/1/235/18468542/stw1015.pdf].
  * [13] T.R. Choudhury and A. Ferrara, _Experimental constraints on self-consistent reionization models_ , _Monthly Notices of the Royal Astronomical Society_ 361 (2005) 577 [https://academic.oup.com/mnras/article-pdf/361/2/577/18660784/361-2-577.pdf].
  * [14] B.E. Robertson, _Estimating Luminosity Function Constraints from High-Redshift Galaxy Surveys_ , _ApJ_ 713 (2010) 1266 [1001.1008].
  * [15] J.M. Lotz, A. Koekemoer, D. Coe, N. Grogin, P. Capak, J. Mack et al., _The Frontier Fields: Survey Design and Initial Results_ , _ApJ_ 837 (2017) 97 [1605.06567].
  * [16] M.A. Schenker, B.E. Robertson, R.S. Ellis, Y. Ono, R.J. McLure, J.S. Dunlop et al., _The uv luminosity function of star-forming galaxies via dropout selection at redshifts $z\sim 7$ and 8 from the 2012 ultra deep field campaign_, _The Astrophysical Journal_ 768 (2013) 196.
  * [17] R.S. Ellis, R.J. McLure, J.S. Dunlop, B.E. Robertson, Y. Ono, M.A. Schenker et al., _The Abundance of Star-forming Galaxies in the Redshift Range 8.5-12: New Results from the 2012 Hubble Ultra Deep Field Campaign_ , _ApJ_ 763 (2013) L7 [1211.6804].
  * [18] R.J. McLure, J.S. Dunlop, R.A.A. Bowler, E. Curtis-Lake, M. Schenker, R.S. Ellis et al., _A new multifield determination of the galaxy luminosity function at z = 7–9 incorporating the 2012 Hubble Ultra-Deep Field imaging_ , _Monthly Notices of the Royal Astronomical Society_ 432 (2013) 2696 [https://academic.oup.com/mnras/article-pdf/432/4/2696/9501883/stt627.pdf].
  * [19] M. Ishigaki, R. Kawamata, M. Ouchi, M. Oguri, K. Shimasaku and Y. Ono, _Hubble Frontier Fields First Complete Cluster Data: Faint Galaxies at $z\sim 5-10$ for UV Luminosity Functions and Cosmic Reionization_, _Astrophys. J._ 799 (2015) 12 [1408.6903].
  * [20] M. Ishigaki, R. Kawamata, M. Ouchi, M. Oguri, K. Shimasaku and Y. Ono, _Full-data Results of Hubble Frontier Fields: UV Luminosity Functions at $z\sim 6-10$ and a Consistent Picture of Cosmic Reionization_, _ApJ_ 854 (2018) 73 [1702.04867].
  * [21] R.J. Bouwens et al., _UV Luminosity Functions at redshifts $z\sim$4 to $z\sim$10: 10000 Galaxies from HST Legacy Fields_, _Astrophys. J._ 803 (2015) 34 [1403.4295].
  * [22] P. Schechter, _An analytic expression for the luminosity function for galaxies._ , _ApJ_ 203 (1976) 297.
  * [23] J. Loveday, P. Norberg, I.K. Baldry, S.P. Driver, A.M. Hopkins, J.A. Peacock et al., _Galaxy and Mass Assembly (GAMA): ugriz galaxy luminosity functions_ , _Monthly Notices of the Royal Astronomical Society_ 420 (2012) 1239 [https://academic.oup.com/mnras/article-pdf/420/2/1239/3065107/mnras0420-1239.pdf].
  * [24] R.G. Bower, A.J. Benson, R. Malbon, J.C. Helly, C.S. Frenk, C.M. Baugh et al., _Breaking the hierarchy of galaxy formation_ , _Monthly Notices of the Royal Astronomical Society_ 370 (2006) 645 [https://academic.oup.com/mnras/article-pdf/370/2/645/2898993/mnras0370-0645.pdf].
  * [25] A.J. Benson, R.G. Bower, C.S. Frenk, C.G. Lacey, C.M. Baugh and S. Cole, _What Shapes the Luminosity Function of Galaxies?_ , _ApJ_ 599 (2003) 38 [astro-ph/0302450].
  * [26] Y. Ono, M. Ouchi, Y. Harikane, J. Toshikawa, M. Rauch, S. Yuma et al., _Great Optically Luminous Dropout Research Using Subaru HSC (GOLDRUSH). I. UV luminosity functions at z $\sim$ 4-7 derived with the half-million dropouts on the 100 deg2 sky_, _PASJ_ 70 (2018) S10 [1704.06004].
  * [27] Y. Harikane, A.K. Inoue, K. Mawatari, T. Hashimoto, S. Yamanaka, Y. Fudamoto et al., _A search for h-dropout lyman break galaxies at $z\sim 12-16$_, _The Astrophysical Journal_ 929 (2022) 1.
  * [28] R.A.A. Bowler, J.S. Dunlop, R.J. McLure, H.J. McCracken, B. Milvang-Jensen, H. Furusawa et al., _Discovery of bright $z\simeq 7$ galaxies in the UltraVISTA survey_, _Monthly Notices of the Royal Astronomical Society_ 426 (2012) 2772 [https://academic.oup.com/mnras/article-pdf/426/4/2772/3299565/426-4-2772.pdf].
  * [29] J.S.B. Wyithe, H. Yan, R.A. Windhorst and S. Mao, _A distortion of very-high-redshift galaxy number counts by gravitational lensing_ , _Nature_ 469 (2011) 181 [1101.2291].
  * [30] J. Rigby, M. Perrin, M. McElwain, R. Kimble, S. Friedman, M. Lallo et al., _The Science Performance of JWST as Characterized in Commissioning_ , _PASP_ 135 (2023) 048001 [2207.05632].
  * [31] M. Stefanon, I. Labbé, R.J. Bouwens, P. Oesch, M.L.N. Ashby, K.I. Caputi et al., _The Brightest z $\gtrsim$ 8 Galaxies over the COSMOS UltraVISTA Field_, _ApJ_ 883 (2019) 99 [1902.10713].
  * [32] R.A.A. Bowler, M.J. Jarvis, J.S. Dunlop, R.J. McLure, D.J. McLeod, N.J. Adams et al., _A lack of evolution in the very bright end of the galaxy luminosity function from $z\simeq 8$ to 10_, _MNRAS_ 493 (2020) 2059 [1911.12832].
  * [33] M.L. Stevans, S.L. Finkelstein, I. Wold, L. Kawinwanichakij, C. Papovich, S. Sherman et al., _Bridging Star-forming Galaxy and AGN Ultraviolet Luminosity Functions at z = 4 with the SHELA Wide-field Survey_ , _ApJ_ 863 (2018) 63 [1806.05187].
  * [34] N.J. Adams, R.A.A. Bowler, M.J. Jarvis, B. Häußler, R.J. McLure, A. Bunker et al., _The rest-frame UV luminosity function at $z\simeq 4$: a significant contribution of AGNs to the bright end of the galaxy population_, _MNRAS_ 494 (2020) 1771 [1912.01626].
  * [35] R. Barkana and A. Loeb, _In the beginning: the first sources of light and the reionization of the universe_ , _Phys. Rep._ 349 (2001) 125 [astro-ph/0010468].
  * [36] M. Kuhlen and C.-A. Faucher-GiguÃšre, _Concordance models of reionization: implications for faint galaxies and escape fraction evolution_ , _Monthly Notices of the Royal Astronomical Society_ 423 (2012) 862 [https://academic.oup.com/mnras/article-pdf/423/1/862/18612580/mnras0423-0862.pdf].
  * [37] B.E. Robertson, S.R. Furlanetto, E. Schneider, S. Charlot, R.S. Ellis, D.P. Stark et al., _New Constraints on Cosmic Reionization from the 2012 Hubble Ultra Deep Field Campaign_ , _ApJ_ 768 (2013) 71 [1301.1228].
  * [38] I.T. Iliev, G. Mellema, U.L. Pen, H. Merz, P.R. Shapiro and M.A. Alvarez, _Simulating cosmic reionization at large scales - I. The geometry of reionization_ , _MNRAS_ 369 (2006) 1625 [astro-ph/0512187].
  * [39] A.H. Pawlik, J. Schaye and E. van Scherpenzeel, _Keeping the Universe ionized: photoheating and the clumping factor of the high-redshift intergalactic medium_ , _MNRAS_ 394 (2009) 1812 [0807.3963].
  * [40] K. Finlator, S.P. Oh, F. Özel and R. Davé, _Gas clumping in self-consistent reionization models_ , _Monthly Notices of the Royal Astronomical Society_ 427 (2012) 2464 [https://academic.oup.com/mnras/article-pdf/427/3/2464/3857876/427-3-2464.pdf].
  * [41] J. Schroeder, A. Mesinger and Z. Haiman, _Evidence of Gunn-Peterson damping wings in high-z quasar spectra: strengthening the case for incomplete reionization at z $\sim$ 6-7_, _MNRAS_ 428 (2013) 3058 [1204.2838].
  * [42] J.M. Shull, A. Harness, M. Trenti and B.D. Smith, _Critical star formation rates for reionization: Full reionization occurs at redshift z= 7_ , _The Astrophysical Journal_ 747 (2012) 100.
  * [43] B.E. Robertson, _Galaxy formation and reionization: Key unknowns and expected breakthroughs by the james webb space telescope_ , _Annual Review of Astronomy and Astrophysics_ 60 (2022) 121 [https://doi.org/10.1146/annurev-astro-120221-044656].
  * [44] A.K. Inoue, I. Shimizu, I. Iwata and M. Tanaka, _An updated analytic model for attenuation by the intergalactic medium_ , _MNRAS_ 442 (2014) 1805 [1402.0677].
  * [45] E.R. Fernandez and J.M. Shull, _The effect of galactic properties on the escape fraction of ionizing photons_ , _The Astrophysical Journal_ 731 (2011) 20.
  * [46] C.A. Mason, T. Treu, M. Dijkstra, A. Mesinger, M. Trenti, L. Pentericci et al., _The Universe Is Reionizing at z $\sim$ 7: Bayesian Inference of the IGM Neutral Fraction Using Ly$\alpha$ Emission from Galaxies_, _Astrophys. J._ 856 (2018) 2 [1709.05356].
  * [47] X. Ma, D. Kasen, P.F. Hopkins, C.-A. Faucher-GiguÃšre, E. Quataert, D. KereÅ¡ et al., _The difficulty of getting high escape fractions of ionizing photons from high-redshift galaxies: a view from the FIRE cosmological simulations_ , _Monthly Notices of the Royal Astronomical Society_ 453 (2015) 960 [https://academic.oup.com/mnras/article-pdf/453/1/960/4935561/stv1679.pdf].
  * [48] H. Atek, I. Labbé, L.J. Furtak, I. Chemerynska, S. Fujimoto, D.J. Setton et al., _Most of the photons that reionized the Universe came from dwarf galaxies_ , _Nature_ 626 (2024) 975 [2308.08540].
  * [49] C. Simmonds, S. Tacchella, K. Hainline, B.D. Johnson, W. McClymont, B. Robertson et al., _Low-mass bursty galaxies in JADES efficiently produce ionizing photons and could represent the main drivers of reionization_ , _MNRAS_ 527 (2024) 6139 [2310.01112].
  * [50] J.B. Muñoz, J. Mirocha, J. Chisholm, S.R. Furlanetto and C. Mason, _Reionization after JWST: a photon budget crisis?_ , _arXiv e-prints_ (2024) arXiv:2404.07250 [2404.07250].
  * [51] S. Mitra and A. Chatterjee, _Non-parametric reconstruction of photon escape fraction from reionization_ , _MNRAS_ 523 (2023) L35 [2303.02704].
  * [52] C.-A. Faucher-GiguÃšre, D. KereÅ¡ and C.-P. Ma, _The baryonic assembly of dark matter haloes_ , _Monthly Notices of the Royal Astronomical Society_ 417 (2011) 2982 [https://academic.oup.com/mnras/article-pdf/417/4/2982/3842290/mnras0417-2982.pdf].
  * [53] Y.-W. Yu, K. Cheng, M. Chu and S. Yeung, _Cosmic histories of star formation and reionization: an analysis with a power-law approximation_ , _Journal of Cosmology and Astroparticle Physics_ 2012 (2012) 023.
  * [54] D.K. Hazra, D. Paoletti, F. Finelli and G.F. Smoot, _Joining bits and pieces of reionization history_ , _Phys. Rev. Lett._ 125 (2020) 071301.
  * [55] D. Paoletti, D.K. Hazra, F. Finelli and G.F. Smoot, _Extended reionization in models beyond $\Lambda$CDM with Planck 2018 data_, _JCAP_ 09 (2020) 005 [2005.12222].
  * [56] A. Krishak and D.K. Hazra, _Gaussian process reconstruction of reionization history_ , _The Astrophysical Journal_ 922 (2021) 95.
  * [57] D. Paoletti, D.K. Hazra, F. Finelli and G.F. Smoot, _Dark twilight joined with the light of dawn to unveil the reionization history_ , _Phys. Rev. D_ 104 (2021) 123549 [2107.10693].
  * [58] M. Seikel, C. Clarkson and M. Smith, _Reconstruction of dark energy and expansion dynamics using gaussian processes_ , _Journal of Cosmology and Astroparticle Physics_ 2012 (2012) 036.
  * [59] A. Shafieloo, A.G. Kim and E.V. Linder, _Model independent tests of cosmic growth versus expansion_ , _Phys. Rev. D_ 87 (2013) 023520.
  * [60] R. Calderon, A. Shafieloo, D. Kumar Hazra and W. Sohn, _On the consistency of $\Lambda$CDM with CMB measurements in light of the latest Planck, ACT and SPT data_, _J. Cosmology Astropart. Phys_ 2023 (2023) 059 [2302.14300].
  * [61] C.E. Rasmussen and C.K.I. Williams, _Gaussian processes for machine learning._ , Adaptive computation and machine learning, MIT Press (2006).
  * [62] R.J. Bouwens, P.A. Oesch, M. Stefanon, G. Illingworth, I. Labbé, N. Reddy et al., _New Determinations of the UV Luminosity Functions from z 9 to 2 Show a Remarkable Consistency with Halo Growth and a Constant Star Formation Efficiency_ , _AJ_ 162 (2021) 47 [2102.07775].
  * [63] Y. Harikane, Y. Ono, M. Ouchi, C. Liu, M. Sawicki, T. Shibuya et al., _GOLDRUSH. IV. Luminosity Functions and Clustering Revealed with 4,000,000 Galaxies at z 2-7: Galaxy-AGN Transition, Star Formation Efficiency, and Implication for Evolution at z ¿ 10_ , _ApJS_ 259 (2022) 20 [2108.01090].
  * [64] P.A. Oesch, R.J. Bouwens, G.D. Illingworth, I. Labbé and M. Stefanon, _The dearth of $z\sim 10$ galaxies in all hst legacy fields—the rapid evolution of the galaxy population in the first 500 myr_, _The Astrophysical Journal_ 855 (2018) 105.
  * [65] D.J. McLeod, R.J. McLure and J.S. Dunlop, _The z = 9-10 galaxy population in the Hubble Frontier Fields and CLASH surveys: the z = 9 luminosity function and further evidence for a smooth decline in ultraviolet luminosity density at z $\geq$ 8_, _MNRAS_ 459 (2016) 3812 [1602.05199].
  * [66] Y. Harikane, M. Ouchi, M. Oguri, Y. Ono, K. Nakajima, Y. Isobe et al., _A Comprehensive Study of Galaxies at z 9-16 Found in the Early JWST Data: Ultraviolet Luminosity Functions and Cosmic Star Formation History at the Pre-reionization Epoch_ , _ApJS_ 265 (2023) 5 [2208.01612].
  * [67] Y. Ono et al., _Spectroscopic Confirmation of Three z-Dropout Galaxies at z = 6.844 - 7.213: Demographics of Lyman-Alpha Emission in z ~ 7 Galaxies_ , _Astrophys. J._ 744 (2012) 83 [1107.3159].
  * [68] V. Tilvi, C. Papovich, S.L. Finkelstein, J. Long, M. Song, M. Dickinson et al., _Rapid Decline of L $y\alpha$ Emission Toward the Reionization Era_, _Astrophys. J._ 794 (2014) 5 [1405.4869].
  * [69] M.A. Schenker, R.S. Ellis, N.P. Konidaris and D.P. Stark, _Line Emitting Galaxies Beyond a Redshift of 7: An Improved Method for Estimating the Evolving Neutrality of the Intergalactic Medium_ , _Astrophys. J._ 795 (2014) 20 [1404.4632].
  * [70] B. Greig, A. Mesinger, Z. Haiman and R.A. Simcoe, _Are we witnessing the epoch of reionization at z=7.1 from the spectrum of J1120+0641?_ , _Mon. Not. Roy. Astron. Soc._ 466 (2017) 4239 [1606.00441].
  * [71] F.B. Davies et al., _Quantitative Constraints on the Reionization History from the IGM Damping Wing Signature in Two Quasars at z ¿ 7_ , _Astrophys. J._ 864 (2018) 142 [1802.06066].
  * [72] T. Totani, N. Kawai, G. Kosugi, K. Aoki, T. Yamada, M. Iye et al., _Implications for the cosmic reionization from the optical afterglow spectrum of the gamma-ray burst 050904 at z = 6.3_ , _Publ. Astron. Soc. Jap._ 58 (2006) 485 [astro-ph/0512154].
  * [73] M. McQuinn, A. Lidz, M. Zaldarriaga, L. Hernquist and S. Dutta, _Probing the neutral fraction of the IGM with GRBs during the epoch of reionization_ , _MNRAS_ 388 (2008) 1101 [0710.1018].
  * [74] J.S. Bolton, M.G. Haehnelt, S.J. Warren, P.C. Hewett, D.J. Mortlock, B.P. Venemans et al., _How neutral is the intergalactic medium surrounding the redshift z = 7.085 quasar ULAS J1120+0641?_ , _MNRAS_ 416 (2011) L70 [1106.6089].
  * [75] D.J. Mortlock, S.J. Warren, B.P. Venemans, M. Patel, P.C. Hewett, R.G. McMahon et al., _A luminous quasar at a redshift of z = 7.085_ , _Nature_ 474 (2011) 616 [1106.6088].
  * [76] A. Lewis and S. Bridle, _Cosmological parameters from CMB and other data: A Monte Carlo approach_ , _Phys. Rev._ D66 (2002) 103511 [astro-ph/0205436].
  * [77] S.L. Finkelstein, R.E. Ryan, C. Papovich, M. Dickinson, M. Song, R.S. Somerville et al., _The evolution of the galaxy rest-frame ultraviolet luminosity function over the first two billion years_ , _The Astrophysical Journal_ 810 (2015) 71.
  * [78] E. Glikman, S.G. Djorgovski, D. Stern, A. Dey, B.T. Jannuzi and K.-S. Lee, _The faint end of the quasar luminosity function at $z\sim 4$: Implications for ionization of the intergalactic medium and cosmic downsizing*_, _The Astrophysical Journal Letters_ 728 (2011) L26.
  * [79] E. Giallongo, A. Grazian, F. Fiore, A. Fontana, L. Pentericci, E. Vanzella et al., _Faint AGNs at z ¿ 4 in the CANDELS GOODS-S field: looking for contributors to the reionization of the Universe_ , _A &A_ 578 (2015) A83 [1502.02562].
  * [80] M. Niida, T. Nagao, H. Ikeda, K. Matsuoka, M.A.R. Kobayashi, Y. Toba et al., _Revisiting the completeness and luminosity function in high-redshift low-luminosity quasar surveys_ , _The Astrophysical Journal_ 832 (2016) 208.
  * [81] M. Akiyama, W. He, H. Ikeda, M. Niida, T. Nagao, J. Bosch et al., _The quasar luminosity function at redshift 4 with the Hyper Suprime-Cam Wide Survey_ , _Publications of the Astronomical Society of Japan_ 70 (2017)  [https://academic.oup.com/pasj/article-pdf/70/SP1/S34/23692427/psx091.pdf].
  * [82] S. Parsa, J.S. Dunlop and R.J. McLure, _No evidence for a significant AGN contribution to cosmic hydrogen reionization_ , _Monthly Notices of the Royal Astronomical Society_ 474 (2017) 2904 [https://academic.oup.com/mnras/article-pdf/474/3/2904/22843311/stx2887.pdf].
  * [83] S.L. Finkelstein, M.B. Bagley, H.C. Ferguson, S.M. Wilkins, J.S. Kartaltepe, C. Papovich et al., _CEERS Key Paper. I. An Early Look into the First 500 Myr of Galaxy Formation with JWST_ , _ApJ_ 946 (2023) L13 [2211.05792].
  * [84] D.J. Eisenstein, C. Willott, S. Alberts, S. Arribas, N. Bonaventura, A.J. Bunker et al., _Overview of the JWST Advanced Deep Extragalactic Survey (JADES)_ , _arXiv e-prints_ (2023) arXiv:2306.02465 [2306.02465].
  * [85] S. Mitra, T.R. Choudhury and A. Ferrara, _Cosmic reionization after Planck._ , _MNRAS_ 454 (2015) L76 [1505.05507].
  * [86] S. Mitra, T.R. Choudhury and B. Ratra, _First study of reionization in the Planck 2015 normalized closed $\Lambda$CDM inflation model_, _MNRAS_ 479 (2018) 4566 [1712.00018].
  * [87] S.L. Finkelstein, A. D’Aloisio, J.-P. Paardekooper, J. Ryan, Russell, P. Behroozi, K. Finlator et al., _Conditions for Reionizing the Universe with a Low Galaxy Ionizing Photon Escape Fraction_ , _ApJ_ 879 (2019) 36 [1902.02792].
  * [88] Planck Collaboration, R. Adam, N. Aghanim, M. Ashdown, J. Aumont, C. Baccigalupi et al., _Planck intermediate results. XLVII. Planck constraints on reionization history_ , _A &A_ 596 (2016) A108 [1605.03507].


# New self-consistent effective one-body theory for spinless binaries based on
the post-Minkowskian approximation

Jiliang Jing111<EMAIL_ADDRESS>Department of Physics, Key Laboratory of
Low Dimensional Quantum Structures and Quantum Control of Ministry of
Education, and Synergetic Innovation Center for Quantum Effects and
Applications, Hunan Normal University, Changsha, Hunan 410081, P. R. China
Center for Gravitation and Cosmology, College of Physical Science and
Technology, Yangzhou University, Yangzhou 225009, P. R. China  Sheng Long
Department of Physics, Key Laboratory of Low Dimensional Quantum Structures
and Quantum Control of Ministry of Education, and Synergetic Innovation Center
for Quantum Effects and Applications, Hunan Normal University, Changsha, Hunan
410081, P. R. China  Weike Deng  Department of Physics, Key Laboratory of Low
Dimensional Quantum Structures and Quantum Control of Ministry of Education,
and Synergetic Innovation Center for Quantum Effects and Applications, Hunan
Normal University, Changsha, Hunan 410081, P. R. China  Mengjie
<EMAIL_ADDRESS>Department of Physics, Key Laboratory of Low
Dimensional Quantum Structures and Quantum Control of Ministry of Education,
and Synergetic Innovation Center for Quantum Effects and Applications, Hunan
Normal University, Changsha, Hunan 410081, P. R. China  Jieci Wang333
<EMAIL_ADDRESS>Department of Physics, Key Laboratory of Low Dimensional
Quantum Structures and Quantum Control of Ministry of Education, and
Synergetic Innovation Center for Quantum Effects and Applications, Hunan
Normal University, Changsha, Hunan 410081, P. R. China

###### Abstract

The effective one-body theories, introduced by Buonanno and Damour, are novel
approaches to constructing a gravitational waveform template. By taking a
gauge in which $\psi_{1}^{B}$ and $\psi_{3}^{B}$ vanish, we find a decoupled
equation with separable variables for $\psi^{B}_{4}$ for gravitational
perturbation in the effective metric obtained in the post-Minkowskian
approximation. Furthermore, we set up a new self-consistent effective one-body
theory for spinless binaries, which can be applicable to any post-Minkowskian
orders. This theory not only releases the assumption that $v/c$ should be a
small quantity but also resolves the contradiction that the Hamiltonian,
radiation-reaction force, and waveform are constructed from different physical
models in the effective one-body theory with the post-Newtonian approximation.
Compared with our previous theory (Science China, 65, 260411, (2022)), the
computational effort for the radiation-reaction force and waveform in this new
theory will be tremendously reduced.

Hamilton equations, coalescing compact object binary system, self-consistent
effective one-body theory

###### pacs:

04.25.Nx, 04.30.Db, 04.20.Cv

##  I Introduction

The investigation on gravitational waves (GWs) has attracted much attention
since 1918 einstein18 ; BonVanMet62 ; Sac62 ; deAdeBoc00 ; KomSmiDun11 ;
PerAldDel98 ; Jing2021 ; Jing2019 , and many direct detections of GWs have
been recently announced by LIGO, Virgo, and KAGRA Abbott2016 ; Abbott2016(2) ;
Abbott2017 ; Abbott2017(2) ; Abbott2017(3) ; Abbott2019 ; Abbott20211 ;
Abbott20212 ; Abbott20213 ; Nitz2021 . The success of direct detections is
based on the development of technology and theoretical research. In
theoretical studies, a gravitational waveform template (GWT) plays a central
role. The basis of the GWT is to study the late dynamical evolution of a
coalescing compact object binary system.

The effective one-body (EOB) theory based on the post-Newtonian (PN)
approximation introduced by Buonanno and Damour Damour1999  is a novel
approach to studying the late dynamical evolution of a coalescing binary
system. The EOB theory can provide an estimate of the gravitational waveform
emitted throughout the inspiral, plunge, and coalescence phases for spinless
and spin binaries Damour2000 ; Damour2000(2) ; Damour2001 ; Damour2006 . By
calibrating the EOB model to numerical relativity simulations, EOB waveforms
were improved, which are applied to the GW data analysis Cook ; Pan ; Pan1 ;
Damour2008 ; Damour20082 ; Boyle ; Damour20083 ; Pan2 ; Pan3 ; Damour2007 ;
Damour2009 ; Pan4 ; Bar ; Pan5 ; Pan2014 ; Bohe ; Damour2015 ; Cao2017 .

After the great success of the EOB theory based on PN approximation, in 2016,
Damour Damour2016  developed the EOB theory with post-Minkowskian (PM)
approximation, in which the assumption that $v/c$ should be a small quantity
was released. Since then, the correlational research based on the EOB theory
with PM approximation has attracted great attention Damour2017 ; Damour2018 ;
Damour2018new ; Antonelli2019 ; Damour2019 ; Damour2020 ; HeLin2016 ;
Blanchet2018 ; Cheung2018 ; Vines2019 ; Cristofoli2019 ; Collado2019 ;
Bern2019 ; Bern20192 ; Plefka2019 ; BiniDamour2020 ; Cheung2020 .

The Hamilton equations Damour2000  of an EOB system based on the PN/PM
approximation show that, for a self-consistent effective one-body (SCEOB)
theory, all quantities appearing in equations should be constructed from the
same physical model. Moreover, as we will show in this paper, the waveform
should also be based on the same physical model. That is, the Hamiltonian,
radiation-reaction force (RRF), and waveform should be constructed in terms of
the same effective spacetime.

To determine the expressions of the RRF and waveform for the “plus” and
“cross” modes of GWs, we should first find the decoupled and variable
separable equation for $\psi^{B}_{4}$ in the effective spacetime. Recently, we
Jing  have successfully derived the decoupled equations of $\psi^{B}_{4}$ for
even and odd parities in the Regge–Wheeler gauge Thompson  by dividing the
perturbation part of the metric into odd and even parities. The decoupled
equations can be used to study the RRF and waveform in the effective spacetime
and to set up an SCEOB theory. The explicit calculations for this model,
however, are arduous tasks because we have to simultaneously solve two
equations for the odd and even parities. Moreover, the equation for the odd
parity is a third-order differential equation.

In this study, we derive another new decoupled and variable separable equation
for $\psi^{B}_{4}$ in the effective spacetime and obtain the corresponding
formal solution. To do so, we first take a gauge in which $\psi_{1}^{B}$ and
$\psi_{3}^{B}$ vanish. This task can be done because, in a linear perturbation
theory, $\psi_{0}^{B}$ and $\psi_{4}^{B}$ are gauge invariant, whereas
$\psi_{1}^{B}$ and $\psi_{3}^{B}$ are not Chandrasekhar . In this gauge, the
decoupled equation for $\psi^{B}_{4}$ can be obtained. Then, we separate
variables for the decoupled equation of $\psi_{4}^{B}$ and obtain a formal
solution. Based on this solution, we present the formulas for the RRF and
waveform. Then, we set up an SCEOB theory for the spinless binaries, which is
appropriate for any PM order. The theory not only releases the assumption that
$v/c$ should be a small quantity but also resolves the contradiction that the
Hamiltonian, RRF, and waveform are constructed from different physical models
in the EOB theory with PN approximation. Compared with our previous SCEOB
theory Jing , the computational effort for the RRF and waveform in the new
SCEOB theory was tremendously reduced.

The rest of the paper is organized as follows: In Sec. II, we derive a
decoupled and variable separable equation for $\psi^{B}_{4}$ and determine the
corresponding formal solution in the effective spacetime. In Sec. III, we set
up the SCEOB theory for spinless binaries based on PM approximation. In Sec.
IV, the final conclusions are presented.

##  II Formal solution for $\psi^{B}_{4}$

In an SCEOB theory, $\psi^{B}_{4}$ plays a central role in determining the RRF
and waveform for the “plus” and “cross” modes of GWs. Therefore, we will first
derive the decoupled and variables separable equation of motion and then
present the corresponding formal solution for $\psi^{B}_{4}$ in the effective
spacetime, which is described by Jing :

$\displaystyle ds_{\rm
eff}^{2}=g^{\text{eff}}_{\mu\nu}dx^{\mu}dx^{\nu}=\frac{\Delta_{r}}{r^{2}}dt^{2}-\frac{r^{2}}{\Delta_{r}}dr^{2}-r^{2}(d\theta^{2}+\sin^{2}\theta
d\varphi^{2}),$ (1)

with

$\displaystyle\Delta_{r}=r^{2}-2GM_{0}r+\sum_{i=2}^{\infty}a_{i}\frac{(GM_{0}\big{)}^{i}}{r^{i-2}},$
(2)

where the definition for all parameters can be found in Ref. Jing . In
particular, the effective spacetime (1) is type $D$ Carmeli .

###  II.1 Decoupled equation for $\psi^{B}_{4}$

To decouple $\psi^{B}_{4}$ in the effective spacetime, we take the null
tetrads for the spacetime (1) as

$\displaystyle l^{\mu}$
$\displaystyle=\Big{\\{}\frac{r^{2}}{\Delta_{r}},1,0,0\Big{\\}},$
$\displaystyle n^{\mu}$
$\displaystyle=\Big{\\{}\frac{1}{2},-\frac{\Delta_{r}}{2r^{2}},0,0\Big{\\}},$
$\displaystyle m^{\mu}$
$\displaystyle=\frac{1}{\sqrt{2}r}\Big{\\{}0,0,1,\frac{{i}}{\sin\theta}\Big{\\}}.$

Then, we have

$\displaystyle\rho=-\dfrac{1}{r},\quad\mu=-\dfrac{\Delta_{r}}{2r^{3}},\quad\gamma=\dfrac{\Delta_{r}^{\prime}}{4r^{2}}+\mu,\quad\alpha=-\dfrac{\cot\theta}{2\sqrt{2}r},\quad\beta=\dfrac{\cot\theta}{2\sqrt{2}r},$
(3)

here and hereafter, ′ represents a derivative with respect to $r$, and all
other spin coefficients are equal to zero.

Ref. Carmeli  shows that there are three equations related to $\psi_{4}$:

$\displaystyle\Delta\lambda-\overline{\delta}\nu=-(\mu+\overline{\mu})\lambda-(3\gamma-\overline{\gamma})\lambda+(3\alpha+\overline{\beta})\nu-\psi_{4},$
(4)
$\displaystyle\overline{\delta}\psi_{3}-D\psi_{4}+\overline{\delta}\phi_{21}-\Delta\phi_{20}=3\lambda\psi_{2}-2\alpha\psi_{3}-\rho\psi_{4}-2\nu\phi_{10}+2\lambda\phi_{11}$
$\displaystyle\ \ \ \
+(2\gamma-2\overline{\gamma}+\overline{\mu})\phi_{20}-2\alpha\phi_{21}-\overline{\sigma}\phi_{22},$
(5)
$\displaystyle\Delta\psi_{3}-\delta\psi_{4}+\overline{\delta}\phi_{22}-\Delta\phi_{21}=3\nu\psi_{2}-2(\gamma+2\mu)\psi_{3}+4\beta\psi_{4}-2\nu\phi_{11}-\overline{\nu}\phi_{20}$
$\displaystyle\ \ \ \
+2\lambda\phi_{12}+2(\gamma+\overline{\mu})\phi_{21}-2(\overline{\beta}+\alpha)\phi_{22}.$
(6)

We Jing  have shown that, in the effective background spacetime, the
gravitational perturbation described by

$\displaystyle g_{\mu\nu}=g^{\text{eff}}_{\mu\nu}+\varepsilon
h_{\mu\nu}^{B}\;,$ (7)

can be achieved by perturbing all null tetrad quantities. Then, from Eqs. (4),
(5) and (6), we derive the following perturbation equations:

$\displaystyle\psi_{4}^{B}+(\Delta+3\gamma-\overline{\gamma}+\mu+\overline{\mu})\lambda^{B}-(\overline{\delta}+3\alpha+\overline{\beta})\nu^{B}=0,$
(8)
$\displaystyle\nu^{B}(3\psi_{2}-2\phi_{11})-(\Delta+2\gamma+4\mu)\psi_{3}^{B}+(\delta+4\beta)\psi_{4}^{B}$
$\displaystyle\ \ \ \
+(\Delta+2\gamma+2\overline{\mu})\phi_{21}^{B}-(\overline{\delta}+2\alpha+2\overline{\beta})\phi_{22}^{B}=0,$
(9)
$\displaystyle\lambda^{B}(3\psi_{2}+2\phi_{11})-(\delta+2\alpha)\psi_{3}^{B}+(D-\rho)\psi_{4}^{B}$
$\displaystyle\ \ \ \
-(\overline{\delta}+2\alpha)\phi_{21}^{B}+(\Delta+2\gamma-2\overline{\gamma}+\overline{\mu})\phi_{20}^{B}=0,$
(10)

where all quantities without and with the superscript $B$ represent the
background and perturbation quantities, respectively.

Chandrasekhar pointed out that, for a linear perturbation given by (7),
$\psi_{0}^{B}$ and $\psi_{4}^{B}$ are gauge invariant, whereas $\psi_{1}^{B}$
and $\psi_{3}^{B}$ are not Chandrasekhar . Thus, we can choose a gauge in
which $\psi_{1}^{B}$ and $\psi_{3}^{B}$ vanish without affecting
$\psi_{0}^{B}$ and $\psi_{4}^{B}$. In this gauge, from Eqs. (9) and (10), we
get $\nu^{B}$ and $\lambda^{B}$:

$\displaystyle\nu^{B}=\frac{-(\Delta+2\gamma+2\overline{\mu})\phi_{21}^{B}-(\delta+4\beta)\psi_{4}^{B}+(\overline{\delta}+2\alpha+2\overline{\beta})\phi_{22}^{B}}{3\psi_{2}-2\phi_{11}},$
(11)
$\displaystyle\lambda^{B}=\frac{(\overline{\delta}+2\alpha)\phi_{21}^{B}-(D-\rho)\psi_{4}^{B}-(\Delta+2\gamma-2\overline{\gamma}+\overline{\mu})\phi_{20}^{B}}{3\psi_{2}+2\phi_{11}}.$
(12)

Substituting Eqs. (11) and (12) into Eq. (8), we obtain the decoupled equation
for $\psi_{4}^{B}$:

$\displaystyle\Big{[}(\Delta+3\gamma-\overline{\gamma}+\mu+\overline{\mu}-F_{1})(D-\rho)-F_{2}(\overline{\delta}+3\alpha+\overline{\beta}-F_{3})(\delta+4\beta)-F_{4}\Big{]}\psi_{4}^{B}=T_{4}^{B},$
(13)

with

$\displaystyle T_{4}^{B}$
$\displaystyle=\Big{(}(\Delta+3\gamma+\mu-\overline{\gamma}+\overline{\mu})-F_{1}\Big{)}\Big{[}(\overline{\delta}+2\alpha-2\overline{\tau})\phi_{21}^{B}-(\Delta+2\gamma-2\overline{\gamma}+\overline{\mu})\phi_{20}^{B}\Big{]}$
$\displaystyle-
F_{2}\Big{(}(\overline{\delta}+3\alpha+\overline{\beta})-F_{3}\Big{)}\Big{[}(\overline{\delta}+2\alpha+2\overline{\beta})\phi_{22}^{B}-(\Delta+2\gamma+2\overline{\mu})\phi_{21}^{B}\Big{]},$
(14)

where the functions $F_{i}$ ($i=1,2,3,4$) are defined as

$\displaystyle
F_{1}=-\dfrac{\Delta_{r}}{2r^{2}(3\psi_{2}+2\phi_{11})}\dfrac{\partial}{\partial
r}\big{(}3\psi_{2}+2\phi_{11}\big{)},\;\;\;\;\;\;F_{2}=\dfrac{3\psi_{2}+2\phi_{11}}{3\psi_{2}-2\phi_{11}},$
$\displaystyle
F_{3}=\dfrac{1}{\sqrt{2}r(3\psi_{2}-2\phi_{11})}\dfrac{\partial}{\partial\theta}\big{(}3\psi_{2}-2\phi_{11}\big{)},\;\;\;\;\;\;\;\,F_{4}=3\psi_{2}+2\phi_{11}.$
(15)

We aim to study a general perturbation as a superposition of waves with
different frequencies ($\sigma^{\dagger}$) and periods ($2m\pi$,
$m=0,1,2,....$) in $\varphi$, so the perturbation in the $t$ and $\varphi$
directions may be written as $e^{i(\sigma^{\dagger}t+m\varphi)}$.

To solve Eq. (13), we first introduce new derivative operators:

$\displaystyle\mathscr{D}_{n}=\partial_{r}+\frac{ir^{2}\sigma^{\dagger}}{\Delta_{r}}+n\frac{\Delta^{\prime}_{r}}{\Delta_{r}},\qquad\mathscr{D}_{n}^{\dagger}=\partial_{r}-\frac{ir^{2}\sigma^{\dagger}}{\Delta_{r}}+n\frac{\Delta^{\prime}_{r}}{\Delta_{r}},$
$\displaystyle\mathscr{L}_{n}=\partial_{\theta}+\frac{m}{\sin\theta}+n\cot\theta\qquad\mathscr{L}_{n}^{\dagger}=\partial_{\theta}-\frac{m}{\sin\theta}+n\cot\theta.$
(16)

Then, the intrinsic derivatives, appearing in Eqs. (13) and (II.1), may be
rewritten as

$\displaystyle D=\mathscr{D}_{0},\qquad$
$\displaystyle\Delta=-\frac{\Delta_{r}}{2r^{2}}\mathscr{D}_{0}^{\dagger},\qquad\delta=\frac{1}{\sqrt{2}r}\mathscr{L}_{0}^{\dagger},\qquad\overline{\delta}=\frac{1}{\sqrt{2}r}\mathscr{L}_{0}.$
(17)

From Eqs. (3) and (17) and by taking $\psi_{4}^{B}=r^{-4}\phi_{4}^{B}$, the
decoupled GW equation (13) becomes

$\displaystyle\Big{[}\Delta_{r}\Big{(}\mathscr{D}_{-1}^{\dagger}+\frac{2r^{2}}{\Delta_{r}}F_{1}\Big{)}\Big{(}\mathscr{D}_{0}-\frac{3}{r}\Big{)}+F_{2}\Big{(}\mathscr{L}_{-1}-\sqrt{2}rF_{3}\Big{)}\mathscr{L}_{2}^{\dagger}+2r^{2}F_{4}\Big{]}\phi_{4}^{B}=\mathscr{T}_{4},$
(18)

where

$\displaystyle\mathscr{T}_{4}$ $\displaystyle=-2r^{6}T_{4}^{B}$
$\displaystyle=F_{2}r^{4}\Big{(}\mathscr{L}_{-1}-\sqrt{2}rF_{3}\Big{)}\mathscr{L}_{0}\phi_{22}^{B}+\frac{\Delta_{r}^{2}r^{2}}{2}\Big{(}\mathscr{D}_{0}^{\dagger}+\frac{2}{r}+\frac{2r^{2}}{\Delta_{r}}F_{1}\Big{)}\Big{(}\mathscr{D}_{0}^{\dagger}+\frac{1}{r}\Big{)}\phi_{20}^{B}$
$\displaystyle+\frac{\Delta_{r}r^{3}}{\sqrt{2}}\Big{[}\Big{(}\mathscr{D}_{-1}^{\dagger}+\frac{3}{r}+\frac{2r^{2}}{\Delta_{r}}F_{1}\Big{)}\mathscr{L}_{-1}+F_{2}\Big{(}\mathscr{L}_{-1}-\sqrt{2}rF_{3}\Big{)}\Big{(}\mathscr{D}_{-1}^{\dagger}+\frac{4}{r}\Big{)}\Big{]}\phi_{21}^{B}.$
(19)

Of note, although we decouple $\psi_{4}^{B}$ by taking the gauge in which
$\psi_{1}^{B}$ and $\psi_{3}^{B}$ vanish, Eq. (18) with $a_{i}=0$ ($i\geq 2$)
is the same as that of the Schwarzschild spacetime obtained in Ref. Teukolsky
.

###  II.2 Separation of the variables for $\psi^{B}_{4}$

For the spinless case, the non-vanishing trace-free Ricci and Weyl tensors in
the null tetrads (II.1) are

$\displaystyle\phi_{11}=\frac{1}{8r^{4}}\Big{[}4\Delta_{r}-4r\Delta_{r}^{\prime}+r^{2}\Delta_{r}^{\prime\prime}+2r^{2}\Big{]},$
$\displaystyle\psi_{2}=\frac{1}{12r^{4}}\Big{[}12\Delta_{r}-6r\Delta_{r}^{\prime}+r^{2}\Delta_{r}^{\prime\prime}-2r^{2}\Big{]},$
(20)

which indicate that, from Eq. (15), $F_{3}=0$ and all other functions
$F_{j}\,(j=1,2,4)$ are only $r$ dependent, so the variables of $\phi_{4}^{B}$
may be separated. To achieve this goal, we expand $\phi_{4}^{B}$ in terms of
${}_{-2}Y_{lm}(\theta)$:

$\phi_{4}^{B}=\sum_{lm}\frac{1}{\sqrt{2\pi}}\int d\omega e^{-i(\omega
t-m\varphi)}\,_{-2}Y_{lm}(\theta)R_{lm\omega}(r),$ (21)

where we have taken $\sigma^{\dagger}=-\omega$, and the functions
${}_{-2}Y_{lm}(\theta)$ may be normalized as

$\int_{0}^{\pi}\,{}_{-2}Y_{\ell m}^{*}(\theta)\,_{-2}Y_{\ell
m}(\theta)\sin\theta\mathrm{d}\theta=1.$ (22)

To compare our results with those obtained by Sasaki and Tagoshi Sasaki , we
take the normalized (22) instead of the usually normalized $\int\,_{-2}Y_{\ell
m}^{u*}(\theta)\,_{-2}\,Y_{\ell m}^{u}(\theta){d}\Omega=1$, which shows that
$\,{}_{-2}Y_{\ell m}^{u}(\theta)=\,_{-2}\,Y_{\ell m}(\theta)/\sqrt{2\pi}$.
Then, we obtain, from Eqs. (18) and (19), the separated radial equation:

$\displaystyle\left[r^{3}F_{4}\Delta_{r}^{2}\frac{\mathrm{d}}{\mathrm{d}r}\Big{(}\frac{1}{r^{3}F_{4}\Delta_{r}}\frac{\mathrm{d}}{\mathrm{d}r}\Big{)}+V(r)\right]R_{lm\omega}(r)=T_{\ell
m\omega}(r),$ (23)

with

$\displaystyle
V(r)=\frac{r^{2}\omega(r^{2}\omega+2i\Delta_{r}^{\prime})}{\Delta_{r}}-ir\omega\Big{(}5+\frac{2r^{3}F_{1}}{\Delta_{r}}\Big{)}+\frac{3(\Delta_{r}+r\Delta_{r}^{\prime})}{r^{2}}-6rF_{1}+2r^{2}F_{4}-\lambda
F_{2},$ (24) $\displaystyle
T_{lm\omega}(r)=\frac{1}{2\pi}\int_{-\infty}^{+\infty}dt\int d\Omega\
\mathscr{T}_{4}\ e^{i(\omega t-m\varphi)}\frac{\
{}_{-2}Y^{*}_{lm}(\theta)}{\sqrt{2\pi}},$ (25)

where $\lambda$ is the eigenvalue of$\ {}_{-2}Y_{lm}(\theta)$, and

$\displaystyle\mathscr{T}_{4}$ $\displaystyle=4\pi\Bigg{\\{}r^{4}F_{2}\
\mathscr{L}_{-1}\Big{[}\mathscr{L}_{0}\Big{(}T_{nn}\Big{)}\Big{]}+\frac{\Delta_{r}^{2}}{2}F_{4}\
\mathscr{D}_{0}^{\dagger}\Big{[}\frac{r}{F_{4}}\mathscr{D}_{0}^{\dagger}\Big{(}rT_{\overline{m}\overline{m}}\Big{)}\Big{]}$
$\displaystyle+\frac{\Delta_{r}^{2}}{\sqrt{2}}\ F_{4}\
\mathscr{D}_{0}^{\dagger}\Big{[}\frac{r^{3}}{\Delta_{r}F_{4}}\mathscr{L}_{-1}\Big{(}T_{\overline{m}n}\Big{)}\Big{]}+\frac{\Delta_{r}^{2}}{\sqrt{2}}\
\frac{F_{2}}{r}\mathscr{L}_{-1}\Big{[}\mathscr{D}_{0}^{\dagger}\Big{(}\frac{r^{4}}{\Delta_{r}}T_{\overline{m}n}\Big{)}\Big{]}\Bigg{\\}}.$
(26)

In Eq. (26), we introduce $T_{nn}$, $T_{\overline{m}\overline{m}}$ and
$T_{\overline{m}n}$, which are

$T_{nn}=\dfrac{1}{4\pi}\phi^{B}_{22},\;\;\;\;\;\;T_{\overline{m}\overline{m}}=\dfrac{1}{4\pi}\phi^{B}_{20},\;\;\;\;\;\;T_{\overline{m}n}=\dfrac{1}{4\pi}\phi^{B}_{21}.$

###  II.3 Formal solution for $\psi_{4}^{B}$ in the effective spacetime

In this subsection, we solve Eq. (23) using the Green function method. The
homogeneous solutions for Eq. (23) can be expressed as

$\displaystyle R^{\rm in}_{\ell m\omega}\to\left\\{\begin{array}[]{cc}B^{\rm
trans}_{\ell m\omega}\Delta_{r}^{2}e^{-i\omega r^{*}},&\text{for}\ \ r\to
r_{+},\\\ r^{3}B^{\rm ref}_{\ell m\omega}e^{i\omega r^{*}}+r^{-1}B^{\rm
inc}_{\ell m\omega}e^{-i\omega r^{*}},&\text{for}\ \
r\to+\infty,\end{array}\right.$ (29)

$\displaystyle R^{\rm up}_{\ell m\omega}\to\left\\{\begin{array}[]{cc}C^{\rm
up}_{\ell m\omega}e^{i\omega r^{*}}+\Delta_{r}^{2}C^{\rm ref}_{\ell
m\omega}e^{-i\omega r^{*}},&\text{for}\ \ r\to r_{+},\\\ C^{\rm trans}_{\ell
m\omega}r^{3}e^{i\omega r^{*}},&\text{for}\ \ r\to+\infty,\end{array}\right.,$
(32)

where $r^{*}$ is the standard tortoise coordinate defined by
$r^{*}=\int\frac{r^{2}}{\Delta_{r}}dr$. Then, the inhomogeneous solution of
the radial equation (23), based on the causality property of waves generated
by a source, may be constructed as

$R_{\ell m\omega}=\frac{1}{2i\omega C^{\rm trans}_{\ell m\omega}B^{\rm
inc}_{\ell m\omega}}\left\\{R^{\rm up}_{\ell
m\omega}\int^{r}_{r_{+}}d\tilde{r}\frac{R^{\rm in}_{\ell
m\omega}(\tilde{r})T_{\ell
m\omega}(\tilde{r})}{\tilde{r}^{3}F_{4}(\tilde{r})\Delta_{r}^{2}(\tilde{r})}+R^{\rm
in}_{\ell m\omega}\int^{\infty}_{r}d\tilde{r}\frac{R^{\rm up}_{\ell
m\omega}(\tilde{r})T_{\ell
m\omega}(\tilde{r})}{\tilde{r}^{3}F_{4}(\tilde{r})\Delta_{r}^{2}(\tilde{r})}\right\\}.$
(33)

Therefore, the solution for the radial equation (23) at the horizon is

$R_{\ell m\omega}(r\to r_{+})\to\frac{{B^{\rm trans}_{\ell
m\omega}\Delta_{r}^{2}e^{-i\omega r^{*}}}}{2i\omega C^{\rm trans}_{\ell
m\omega}B^{\rm inc}_{\ell m\omega}}\int^{\infty}_{r_{+}}d\tilde{r}\frac{R^{\rm
up}_{\ell m\omega}(\tilde{r})T_{\ell
m\omega}(\tilde{r})}{\tilde{r}^{3}F_{4}(\tilde{r})\Delta_{r}^{2}(\tilde{r})}\equiv\tilde{Z}^{\rm
H}_{\ell m\omega}\Delta_{r}^{2}e^{-i\omega r^{*}}\,,$ (34)

whereas the counterpart at the infinity is

$R_{\ell m\omega}(r\to\infty)\to\frac{r^{3}e^{i\omega r^{*}}}{2i\omega B^{\rm
inc}_{\ell m\omega}}\int^{\infty}_{r_{+}}d\tilde{r}\frac{T_{\ell
m\omega}(\tilde{r})R^{\rm in}_{\ell
m\omega}(\tilde{r})}{\tilde{r}^{3}F_{4}(\tilde{r})\Delta_{r}^{2}(\tilde{r})}\equiv\tilde{Z}_{\ell
m\omega}r^{3}e^{i\omega r^{*}}\,.$ (35)

The energy momentum tensor for the EOB theory, i.e., a particle with the mass
$m_{0}$ orbits around a massive black hole described by the effective metric,
takes the form Sasaki81

$T^{\mu\nu}=\frac{m_{0}}{r^{2}\sin\theta
dt/d\tau}\frac{dx^{\mu}}{d\tau}\frac{dx^{\nu}}{d\tau}\delta(r-r(t))\delta(\theta-\theta(t))\delta(\varphi-\varphi(t)),$
(36)

where $x^{\mu}=\bigl{(}t,\,r(t),\,\theta(t),\,\varphi(t)\bigr{)}$ is a
geodesic trajectory and $\tau=\tau(t)$ is the proper time along the geodesic.
By means of Eqs. (26), (36), and (25) and performing the integration by parts,
we obtain

$\displaystyle
T_{lm\omega}(r)=-\frac{4m_{0}}{\sqrt{2\pi}}\int^{\infty}_{-\infty}dt\int
d\theta e^{i\omega t-im\varphi(t)}$ $\displaystyle\ \ \ \
\times\Bigg{\\{}-\frac{1}{2}\mathscr{L}_{1}^{{\dagger}}\Big{(}\mathscr{L}_{2}^{{\dagger}}\
{}_{-2}Y_{\ell
m}(\theta)\Big{)}C_{nn}r^{4}F_{2}\,\delta(r-r(t))\delta(\theta-\theta(t))$
$\displaystyle\ \ \ \
+\frac{\Delta_{r}^{2}}{\sqrt{2}r}\Big{(}(1+F_{2})\mathscr{L}_{2}^{{\dagger}}\
{}_{-2}Y_{\ell
m}(\theta)\Big{)}\mathscr{D}_{0}^{\dagger}\Big{[}\frac{C_{{\overline{m}}n}r^{4}}{\Delta_{r}}\delta(r-r(t))\delta(\theta-\theta(t))\Big{]}$
$\displaystyle\ \ \ \ +\frac{1}{2\sqrt{2}}\Big{[}F_{4}\frac{\partial}{\partial
r}\Big{(}\frac{1}{rF_{4}}\Big{)}\Big{]}\Big{(}\mathscr{L}_{2}^{{\dagger}}\
{}_{-2}Y_{\ell
m}(\theta)\Big{)}C_{{\overline{m}}n}\Delta_{r}\,r^{4}\,\delta(r-r(t))\delta(\theta-\theta(t))$
$\displaystyle\ \ \ \ -\frac{1}{4}\Delta_{r}^{2}\ {}_{-2}Y_{\ell
m}(\theta)F_{4}\mathscr{D}_{0}^{\dagger}\Big{[}\frac{r}{F_{4}}\mathscr{D}_{0}^{\dagger}\Big{(}rC_{{\overline{m}}{\overline{m}}}\delta(r-r(t))\delta(\theta-\theta(t))\Big{)}\Big{]}\Bigg{\\}}.$
(37)

with

$\displaystyle C_{nn}$ $\displaystyle=$
$\displaystyle\frac{\Delta_{r}}{4r^{4}E}\left(E+\frac{dr}{d\tau}\right)^{2},$
$\displaystyle C_{{\overline{m}}n}$ $\displaystyle=$
$\displaystyle\frac{\Delta_{r}}{2\sqrt{2}r^{5}E}\left(E+\frac{dr}{d\tau}\right)\left(\frac{iL}{\sin\theta}-r^{2}\frac{d\theta}{d\tau}\right),$
$\displaystyle C_{{\overline{m}}{\overline{m}}}$ $\displaystyle=$
$\displaystyle\frac{\Delta_{r}}{2r^{6}E}\left(r^{2}\frac{d\theta}{d\tau}-\frac{iL}{\sin\theta}\right)^{2}.$
(38)

Eq. (37) can be rewritten as

$\displaystyle T_{\ell m\omega}(r)$ $\displaystyle=$ $\displaystyle-
m_{0}\int^{\infty}_{-\infty}dte^{i\omega
t-im\varphi(t)}\Delta_{r}^{2}\Big{\\{}A_{0}\delta(r-r(t))+\Big{[}A_{1}\delta(r-r(t))\Big{]}^{\prime}$
(39) $\displaystyle+$
$\displaystyle\Big{[}A_{2}\delta(r-r(t))\Big{]}^{\prime\prime}\Big{\\}},$

with

$\displaystyle A_{0}$ $\displaystyle=$ $\displaystyle
A_{nn\,0}+A_{{\overline{m}}n\,0}+A_{{\overline{m}}{\overline{m}}\,0},$
$\displaystyle A_{1}$ $\displaystyle=$ $\displaystyle
A_{{\overline{m}}n\,1}+A_{{\overline{m}}{\overline{m}}\,1},$ $\displaystyle
A_{2}$ $\displaystyle=$ $\displaystyle A_{{\overline{m}}{\overline{m}}\,2},$
(40)

where

$\displaystyle A_{nn\,0}$ $\displaystyle=$
$\displaystyle-\frac{2\,r^{4}}{\sqrt{2\pi}\,\Delta_{r}^{2}}\,C_{nn}\,F_{2}\,\mathscr{L}_{1}^{+}\Big{[}\mathscr{L}_{2}^{+}\Big{(}\
_{-2}Y_{\ell m}(\theta)\Big{)}\Big{]},$ $\displaystyle A_{{\overline{m}}n\,0}$
$\displaystyle=$
$\displaystyle\frac{r^{3}}{\sqrt{\pi}\Delta_{r}}\,C_{{\overline{m}}n}\Big{[}\big{(}1+F_{2}\big{)}\frac{ir^{2}\omega}{\Delta_{r}}+\frac{F_{2}}{r}-F_{2}^{\prime}-\frac{F_{4}^{\prime}}{F_{4}}\Big{]}\mathscr{L}_{2}^{{\dagger}}\Big{(}\
_{-2}Y_{\ell m}(\theta)\Big{)},$ $\displaystyle
A_{{\overline{m}}{\overline{m}}\,0}$ $\displaystyle=$
$\displaystyle\frac{r^{2}}{\sqrt{2\pi}}\,C_{{\overline{m}}{\overline{m}}}\
{}_{-2}Y_{\ell
m}(\theta)\Bigl{[}i\Bigl{(}\frac{r^{2}\omega}{\Delta_{r}}\Bigr{)}^{\prime}+\frac{r^{4}\omega^{2}}{\Delta_{r}^{2}}+\frac{ir^{2}\omega}{\Delta_{r}}\Big{(}\frac{1}{r}+\frac{F_{4}^{\prime}}{F_{4}}\Big{)}+\frac{F_{4}^{\prime}}{rF_{4}}+\Big{(}\frac{F_{4}^{\prime}}{F_{4}}\Big{)}^{\prime}\Bigr{]},$
$\displaystyle A_{{\overline{m}}n\,1}$ $\displaystyle=$
$\displaystyle\frac{r^{3}}{\sqrt{\pi}\Delta_{r}}\,C_{{\overline{m}}n}\big{(}1+F_{2}\big{)}\mathscr{L}_{2}^{{\dagger}}\Big{(}\
_{-2}Y_{\ell m}(\theta)\Big{)},$ $\displaystyle
A_{{\overline{m}}{\overline{m}}\,1}$ $\displaystyle=$
$\displaystyle\frac{r^{2}}{\sqrt{2\pi}}\,C_{{\overline{m}}{\overline{m}}}\
{}_{-2}Y_{\ell
m}(\theta)\Bigl{(}-2i\frac{r^{2}\omega}{\Delta_{r}}+\frac{1}{r}+\frac{F_{4}^{\prime}}{F_{4}}\Bigr{)},$
$\displaystyle A_{{\overline{m}}{\overline{m}}\,2}$ $\displaystyle=$
$\displaystyle-\frac{r^{2}}{\sqrt{2\pi}}\,C_{{\overline{m}}{\overline{m}}}\
{}_{-2}Y_{\ell m}(\theta),$ (41)

Inserting Eq. (39) into Eq. (35), we obtain

$\displaystyle\tilde{Z}_{\ell m\omega}=\frac{m_{0}}{2i\omega B^{\rm inc}_{\ell
m\omega}}\int^{\infty}_{-\infty}dte^{i\omega
t-im\varphi(t)}\Bigl{[}A_{0}\frac{R^{\rm in}_{\ell
m\omega}}{r^{3}F_{4}}-A_{1}\Big{(}\frac{R^{\rm in}_{\ell
m\omega}}{r^{3}F_{4}}\Big{)}^{\prime}+A_{2}\Big{(}\frac{R^{\rm in}_{\ell
m\omega}}{r^{3}F_{4}}\Big{)}^{\prime\prime}\Bigr{]}_{r=r(t),\theta=\theta(t)}.$
(42)

Because we focus on circular orbits, $r(t)$ in Eq. (42) is not related to the
time, so we can take $r(t)=r_{0}$. On the geodesic trajectory, we also have
$\theta(t)=\theta_{0}$ and $\varphi(t)=\Omega\,t$, where $\Omega$ is the
angular velocity. Then, carrying out the integration for Eq. (42), we find

$\displaystyle\tilde{Z}_{\ell m\omega}$ $\displaystyle=$
$\displaystyle\frac{\pi m_{0}}{i\omega B^{\rm inc}_{\ell
m\omega}}\Bigl{[}A_{0}\frac{R^{\rm in}_{\ell
m\omega}}{r^{3}F_{4}}-A_{1}\Big{(}\frac{R^{\rm in}_{\ell
m\omega}}{r^{3}F_{4}}\Big{)}^{\prime}+A_{2}\Big{(}\frac{R^{\rm in}_{\ell
m\omega}}{r^{3}F_{4}}\Big{)}^{\prime\prime}\Bigr{]}_{r_{0},\theta_{0}}\delta(\omega-\omega_{n})\,,$
(43)

where $\omega_{n}=m\,\Omega$. Then, noting the definition
$\psi_{4}^{B}=\phi^{B}_{4}/r^{4}$ and using Eqs. (21), (35) and (43), we find
that the formal solution for $\psi_{4}^{B}$ is given by

$\displaystyle\psi^{B}_{4}$ $\displaystyle=$
$\displaystyle\frac{1}{r}\sum_{\ell mn}\frac{\pi m_{0}}{i\omega_{n}B^{\rm
inc}_{\ell m\omega_{n}}}\Bigl{[}A_{0}\frac{R^{\rm in}_{\ell
m\omega_{n}}}{r^{3}F_{4}}-A_{1}\Big{(}\frac{R^{\rm in}_{\ell
m\omega_{n}}}{r^{3}F_{4}}\Big{)}^{\prime}+A_{2}\Big{(}\frac{R^{\rm in}_{\ell
m\omega_{n}}}{r^{3}F_{4}}\Big{)}^{\prime\prime}\Bigr{]}_{r_{0},\theta_{0}}\frac{{}_{-2}Y_{\ell
m}}{\sqrt{2\pi}}e^{i\omega_{n}(r^{*}-t)+im\varphi}.$

##  III Framework of the self-consistent EOB theory for spinless binaries

The basis of GWT is the late dynamical evolution of a coalescing compact
object binary system. The dynamical evolution of the system is described by
Damour2000 ; Jing

$\displaystyle\frac{dr}{dt}-\frac{\partial H}{\partial P_{r}}=0,\ \ \ \ \ \ \
\frac{d\varphi}{dt}-\frac{\partial H}{\partial P_{\varphi}}=0,$
$\displaystyle\frac{dP_{r}}{dt}+\frac{\partial H}{\partial r}={\cal F}_{r},\ \
\ \ \frac{dP_{\varphi}}{dt}={\cal F}_{\varphi}.$ (45)

Eq. (45) shows that, for an SCEOB theory, $H$, ${\cal F}_{r}$, and ${\cal
F}_{\varphi}$ and the waveform should be based on the same physical model.

With the effective line element (1), energy map, and solution for
$\psi_{4}^{B}$ (LABEL:psi41) at hand, we now set up an SCEOB theory for a
spinless coalescing compact object binary system. That is, we present the
expressions of the Hamiltonian, RRF, and waveform for the system in the
following.

First, using Buonanno and Damour’s proposals Damour1999 ; Damour2000  and the
energy map Eq. (7) in Ref. Jing , we know that the Hamiltonian in Eq. (45) is
still described by Eq. (10) in Ref. Jing , which shows that the Hamiltonian is
related to the effective spacetime described by Eq. (1).

Second, the general relation between the flux of the GW energy emitted to
infinity and the RRF is described by $\frac{dE}{dt}=\dot{R}\,{\cal
F}_{R}+\dot{\varphi}\,{\cal F}_{\varphi}\,.$ Buonanno and Damour Damour2000
showed that, for quasi-circular orbits, ${\cal F}_{R}$ turns into zero, so we
have

${\cal F}_{\varphi}=\frac{1}{\dot{\varphi}}\,\frac{dE}{dt}.$ (46)

Furthermore, the flux can be shown by Ref:poisson ; TagoshiSasaki745 :

$\displaystyle\frac{dE}{dt}=\frac{1}{16\pi
G}\int(\dot{h}_{+}^{2}+\dot{h}_{\times}^{2})r^{2}d\Omega.$ (47)

Noting $\psi^{B}_{4}=\frac{1}{2}(\ddot{h}_{+}-i\ddot{h}_{\times})$ at infinity
and using Eq. (LABEL:psi41), we can get the gravitational waveform
$h_{+}-ih_{\times}$ at infinity. By substituting $h_{+}-ih_{\times}$ into Eq.
(47), we can obtain the expression for the flux of the GW energy emitted to
infinity. Then, using Eq. (46), we find that the RRF appearing in Eq. (45) is
given by

$\displaystyle{\cal
F}_{\varphi}=\frac{1}{\dot{\varphi}}\sum_{\scriptstyle\ell=2}^{\infty}\sum_{m=1}^{\ell}\dfrac{2\pi
m_{0}^{2}}{G\omega_{n}^{4}}\Bigg{|}\frac{1}{B^{\rm inc}_{\ell
m\omega_{n}}}\Bigl{[}A_{0}\frac{R^{\rm in}_{\ell
m\omega_{n}}}{r^{3}F_{4}}-A_{2}\Big{(}\frac{R^{\rm in}_{\ell
m\omega_{n}}}{r^{3}F_{4}}\Big{)}^{\prime}+A_{2}\Big{(}\frac{R^{\rm in}_{\ell
m\omega_{n}}}{r^{3}F_{4}}\Big{)}^{\prime\prime}\Bigr{]}_{r_{0},\theta_{0}}\Bigg{|}^{2},$
(48)

which indicates that the RRF is constructed in terms of the effective
spacetime.

Finally, by comparing $\psi^{B}_{4}$ with the waveform described by Kidder

$\displaystyle
h_{+}-ih_{\times}=\sum_{l=2}^{\infty}\sum_{m=-l}^{l}h^{lm}\frac{\
{}_{-2}Y^{lm}(\theta,\varphi)}{\sqrt{2\pi}},$ (49)

we find

$\displaystyle h^{lm}=\frac{1}{r}\frac{2\pi m_{0}}{i\omega_{n}^{3}B^{\rm
inc}_{\ell m\omega_{n}}}\Bigl{[}A_{0}\frac{R^{\rm in}_{\ell
m\omega_{n}}}{r^{3}F_{4}}-A_{1}\Big{(}\frac{R^{\rm in}_{\ell
m\omega_{n}}}{r^{3}F_{4}}\Big{)}^{\prime}+A_{2}\Big{(}\frac{R^{\rm in}_{\ell
m\omega_{n}}}{r^{3}F_{4}}\Big{)}^{\prime\prime}\Bigr{]}_{r_{0},\theta_{0}}e^{i\omega_{n}(r^{*}-t)}.$
(50)

Clearly, the calculation for the waveform $h^{lm}$ is also based on the
effective spacetime.

To improve the waveform, Damour and Nagar proposed [32] that, based on the
waveforms obtained using the PN perturbation, multipolar waveforms should be
built as

$h_{\ell m}=h_{\ell m}^{(N,\epsilon_{p})}\,\hat{S}_{\rm
eff}^{(\epsilon_{p})}\,T_{\ell m}\,e^{i\delta_{\ell m}}\,f_{\ell m}\,,$ (51)

where $h_{\ell m}^{(N,\epsilon_{p})}$ is the leading Newtonian order term,
$\hat{S}_{\rm eff}^{(\epsilon_{p})}$ is the relativistic conserved energy or
angular momentum of the effective moving source, $T_{\ell m}$ resumes an
infinite number of leading logarithms entering the tail effect,
$e^{i\delta_{\ell m}}$ is an additional phase correction, and $f_{\ell m}$ is
the remaining (essentially nonlinear) PN effects. Furthermore, to reproduce
effects in the numerical simulations that go beyond the quasi-circular motion
assumption, Pan et al Pan3  introduced a non-quasicircular (NQC) effect in
$h_{lm}$.

We believe that the waveform Eq. (50) can be improved using Damour–Nagar–Pan’s
proposals Damour2007 ; Damour2009 ; Pan2 ; Pan3 .

##  IV Conclusions and discussion

The basis of GWT is the late dynamical evolution of a coalescing compact
object binary system. To study the dynamical evolution, Buonanno et al.
presented, based on the PN/PM approximation, a novel approach to map the two-
body problem onto an EOB problem. Based on the Hamilton equations describing
the dynamical evolution, in an SCEOB theory, the Hamiltonian $H$, RRF ${\cal
F}_{\varphi}$, and waveform should be constructed from the same physical
model.

To build such an SCEOB theory, the key step is to find the decoupled equation
with separable variables for $\psi^{B}_{4}$ in the effective background
spacetime. Chandrasekhar pointed out Chandrasekhar  that, for a linear
gravitational perturbation described by Eq. (7), $\psi_{0}^{B}$ and
$\psi_{4}^{B}$ are gauge invariant, whereas $\psi_{1}^{B}$ and $\psi_{3}^{B}$
are not. Therefore, we can take a gauge in which $\psi_{1}^{B}$ and
$\psi_{3}^{B}$ vanish without affecting $\psi_{0}^{B}$ and $\psi_{4}^{B}$. In
this gauge, we obtained the decoupled equation for $\psi^{B}_{4}$ in the
effective spacetime. Then, we separated variables for the decoupled equation
and obtained a formal solution of $\psi^{B}_{4}$. Based on the solution, we
presented the formulas for the RRF and waveform.

With the Hamiltonian Eq. (10) in Ref. Jing , RRF (48) and waveform (50), which
are based on the same effective spacetime, at hand, we set up an SCEOB theory
for spinless binaries, which can be applicable to any PM orders. The theory
not only releases the assumption that $v/c$ should be a small quantity but
also resolves the contradiction that the Hamiltonian, RRF, and waveform are
constructed from different physical models in the EOB theory based on PN
approximation. Compared with our previous SCEOB theory Jing , the
computational effort for the RRF and waveform in the new SCEOB theory will be
tremendously reduced.

Generally, the usefulness of an EOB model critically depends on the accuracy
of the waveform template. Next, we will construct the GWT based on the SCEOB
theory for the spinless binaries and compare it with that of the numerical
relativity. We expect that the advantages of the SCEOB theory can improve the
accuracy of the GWT. Furthermore, we will extend the study to spin binaries.

###### Acknowledgements.

We would like to thank professors S. Chen and Q. Pan for useful discussions on
the manuscript. This work was supported by the Grant of NSFC Nos. 12035005,
12122504 and 11875025, and National Key Research and Development Program of
China No. 2020YFC2201400.

## References

  * (1) A. Einstein, ${\rm\ddot{U}}$ber gravitationswellen, Sitzungsberichte der K${\rm\ddot{o}}$niglich Preussischen Akademie der Wissenschaften Berlin. part 1: 154-167, (1918).
  * (2) H. Bondi, M. G. J. van der Burg and A. W. K. Metzner, Gravitational waves in general relativity VII. Waves from axi-symmetric isolated systems, Proc. R. Soc. London A269, 21 (1962).
  * (3) R. K. Sachs, Gravitational waves in general relativity VIII. Waves in asymptotically flat space-time, Proc. R. Soc. London A270, 103 (1962).
  * (4) P. de Bernardis, P. A. R. Ade, J. J. Bock, J. R. Bond, J. Borrill, A. Boscaleri, K. Coble, B. P. Crill, G. De Gasperis, P. C. Farese et al., A flat universe from high-resolution maps of the cosmic microwave background radiation, Nature 404, 955 (2000).
  * (5) E. Komatsu, K. M. Smith, J. Dunkley, C. L. Bennett, B. Gold, G. Hinshaw, N. Jarosik, D. Larson, M. R. Nolta, L. Page et al., Seven-year wilkinson microwave anisotropy probe observations: Cosmological interpretation, Astrophys. J. Suppl. S 192, 18 (2011).
  * (6) S. Perlmutter, G. Aldering, M. Della Valle, S. Deustua, R. S. Ellis, S. Fabbro, A. Fruchter, G. Goldhaber, A. Goobar, D. E. Groom, I. M. Hook et al., Discovery of a supernova explosion at half the age of the universe and its cosmological implications, Nature 391, 51 (1998).
  * (7) Y. H. Zou, M. J. Wang and J. L Jing, Test of a model coupling of electromagnetic and gravitational fields by using high-frequency gravitational waves, Sci. China, Phys. Mech. Astron. 64, 250411 (2021).
  * (8) X. K. He, J. L. Jing and Z. J. Cao, Generalized gravitomagnetic field and gravitational waves, Sci. China, Phys. Mech. Astron. 62, 110422 (2019).
  * (9) B. P. Abbott et al. (LIGO Scientific and Virgo Collaborations), Phys. Rev. Lett. 116, 061102 (2016).
  * (10) B. P. Abbott et al. (LIGO Scientific and Virgo Collaborations), Phys. Rev. Lett. 116, 241103 (2016).
  * (11) B. P. Abbott et al. (LIGO Scientific and Virgo Collaborations), Phys. Rev. Lett. 118, 221101 (2017).
  * (12) B. P. Abbott et al. (LIGO Scientific and Virgo Collaborations), Phys. Rev. Lett. 119, 141101 (2017).
  * (13) B. P. Abbott et al. (LIGO Scientific and Virgo Collaborations), Phys. Rev. Lett. 119, 161101 (2017).
  * (14) B. P. Abbott et al. (LIGO Scientific and Virgo Collaborations), Phys. Rev. X 9, 031040 (2019), doi.org/10.1103/PhysRevX.9.031040.
  * (15) R. Abbott et al. (LIGO Scientific Collaboration and Virgo Collaboration), Phys. Rev. X 11, 021053 (2021), doi.org/10.1103/PhysRevX.11.021053.
  * (16) R. Abbott et al. (LIGO Scientific Collaboration, Virgo Collaboration and KAGRA Collaboration), arXiv e-prints (2021), arXiv: 2105.15120 [astro-ph].
  * (17) R. Abbott et al. (LIGO Scientific Collaboration and Virgo Collaboration), arXiv e-prints (2021), arXiv: 2110.09834.
  * (18) A. H. Nitz, C. D. Capano, S. Kumar, Y. F. Wang, S. Kastha, M. Schafer, R. Dhurkunde, and M. Cabero, Catalog of gravitational waves from compact-binary mergers,  arXiv e-prints (2021), arXiv:2105.09151.
  * (19) A. Buonanno and T. Damour, Effective one-body approach to general relativistic two-body dynamics, Phys. Rev. D 59, 084006 (1999).
  * (20) A. Buonanno and T. Damour, Transition from inspiral to plunge in binary black hole coalescences, Phys. Rev. D 62, 064015 (2000).
  * (21) T. Damour, P. Jaranowski and G. Schafer, On the determination of the last stable orbit for circular general relativistic binaries at the third post-Newtonian approximation, Phys. Rev. D 62, 084011 (2000).
  * (22) T. Damour, Coalescence of Two Spinning Black Holes: An Effective One-Body Approach, Phys. Rev. D 64, 124013 (2001).
  * (23) A. Buonanno, Y. Chen and T. Damour, Transition from inspiral to plunge in precessing binaries of spinning black holes, Phys. Rev. D 74, 104005 (2006).
  * (24) T. Damour, P. Jaranowski and G. Schafer, Fourth post-Newtonian effective one-body dynamics, Phys. Rev. D 91, 084024 (2015).
  * (25) A. Buonanno, G. B. Cook, and F. Pretorius, Inspiral, merger, and ring-down of equal-mass black-hole binaries, Phys. Rev. D 75, 124018 (2007).
  * (26) Y. Pan, A. Buonanno, J. G. Baker, J. Centrella, B. J. Kelly et al., Data-analysis driven comparison of analytic and numerical coalescing binary waveforms: Nonspinning case, Phys. Rev. D 77, 024014 (2008).
  * (27) A. Buonanno, Y. Pan, J. G. Baker, J. Centrella, B. J. Kelly, S. T. McWilliams, and J. R. van Meter, Approaching faithful templates for nonspinning binary black holes using the effective-one-body approach,  Phys. Rev. D 76, 104049 (2007).
  * (28) T. Damour and A. Nagar, Comparing effective-one-body gravitational waveforms to accurate numerical data, Phys. Rev. D 77, 024043 (2008).
  * (29) T. Damour, A. Nagar, E. N. Dorband, D. Pollney, and L. Rezzolla, Faithful effective-one-body waveforms of equal-mass coalescing black-hole binaries, Phys. Rev. D 77, 084017 (2008).
  * (30) M. Boyle, A. Buonanno, L. E. Kidder, A. H. Mroue, Y. Pan et al., High-accuracy numerical simulation of black-hole binaries: Computation of the gravitational-wave energy flux and comparisons with post-Newtonian approximates, Phys. Rev. D 78, 104020 (2008).
  * (31) T. Damour, A. Nagar, M. Hannam, S. Husa, and B. Brugmann, Accurate effective-one-body waveforms of inspiralling and coalescing black-hole binaries,  Phys. Rev. D 78, 044039 (2008).
  * (32) T. Damour and A. Nagar, Faithful effective-one-body waveforms of small-mass-ratio coalescing black hole binaries, Phys. Rev. D 76, 064028 (2007).
  * (33) A. Buonanno, Y. Pan, H. P. Pfeiffer, M. A. Scheel, L. T. Buchman, L. E. Kidder, Effective-one-body waveforms calibrated to numerical relativity simulations: Coalescence of nonspinning, equal-mass black holes, Phys. Rev. D 79, 124028 (2009).
  * (34) Y. Pan, A. Buonanno, L. T. Buchman, T. Chu, L. E. Kidder, H. P. Pfeiffer, M. A. Scheel, Effective-one-body waveforms calibrated to numerical relativity simulations: Coalescence of nonprecessing, spinning, equal-mass black holes,  Phys. Rev. D 81, 084041 (2010).
  * (35) T. Damour and A. Nagar, Improved analytical description of inspiralling and coalescing black-hole binaries, Phys. Rev. D 79, 081503(R) (2009).
  * (36) Y. Pan, A. Buonanno, M. Boyle, L. T. Buchman, L. E. Kidder et al., Inspiral-merger-ringdown multipolar waveforms of nonspinning black-hole binaries using the effective-one-body formalism, Phys. Rev. D 84, 124052 (2011).
  * (37) E. Barausse, A. Buonanno, S. A. Hughes, G. Khanna, S. Sullivan et al., Modeling multipolar gravitational-wave emission from small mass-ratio mergers,  Phys. Rev. D 85, 024046 (2012).
  * (38) A. Taracchini, Y. Pan, A. Buonanno, E. Barausse, M. Boyle, T. Chu, G. Lovelace, H. P. Pfeiffer, and M. A. Scheel, Prototype effective-one-body model for nonprecessing spinning inspiral-merger-ringdown waveforms, Phys. Rev. D 86, 024011 (2012).
  * (39) A. Taracchini, A. Buonanno, Y. Pan, T. Hinderer, M. Boyle, D. A. Hemberger, L. E. Kidder, G. Lovelace, A. H. Mroue, H. P. Pfeiffer, et al., Effective-one-body model for black-hole binaries with generic mass ratios and spins,  Phys. Rev. D 89, 061502 (2014).
  * (40) A. Bohé, L. Shao, A. Taracchini, A. Buonanno, S. Babak, et al, Improved effective-one-body model of spinning, nonprecessing binary black holes for the era of gravitational-wave astrophysics with advanced detectors, Phys. Rev. D 95, 044028 (2017).
  * (41) Z. Cao and B. Han, Waveform model for an eccentric binary black hole based on the effective-one-body-numerical-relativity formalism, Phys. Rev. D 96, 044028 (2017).
  * (42) T. Damour, Gravitational scattering, post-Minkowskian approximation and Effective One-Body theory, Phys. Rev. D 94, 104015 (2016).
  * (43) D. Bini and T. Damour, Gravitational spin-orbit coupling in binary systems, post -Minkowskian approximation, and effective-one-body theory, Phys. Rev. D 96, 104038 (2017).
  * (44) D. Bini and T. Damour, Gravitational spin-orbit coupling in binary systems at the second post-Minkowskian approximation, Phys. Rev. D 98, 044036 (2018).
  * (45) T. Damour, High-energy gravitational scattering and the general relativistic two-body problem, Phys. Rev. D 97, 044038 (2018).
  * (46) A. Antonelli, A. Buonanno, J. Steinhoff, M. van de Meent and J. Vines, Energetics of two-body Hamiltonians in post-Minkowskian gravity, Phys. Rev. D 99, 104004 (2019)
  * (47) T. Damour, Classical and quantum scattering in post-Minkowskian gravity, Phys. Rev. D 102, 024060 (2020)
  * (48) D. Bini, T. Damour and A. Geralico, Scattering of tidally interacting bodies in post-Minkowskian gravity, Phys. Rev. D 101, 044039 (2020)
  * (49) G. He and W. Lin, Second-order time delay by a radially moving Kerr-Newman black hole, Phys. Rev. D 94, 063011(2016)
  * (50) L. Blanchet and A. Fokas, Equations of motion of self-gravitating N-body systems in the first post-Minkowskian approximation, Phys. Rev. D 98, 084005 (2018)
  * (51) C. Cheung, I. Rothstein and M. Solon, From Scattering Amplitudes to Classical Potentials in the Post-Minkowskian Expansion, Phys. Rev. Lett. 121 , 251101 (2018)
  * (52) J. Vines, J. Steinhoff and A. Buonnanno, Spinning-black-hole scattering and the test-black-hole limit at second post-Minkowskian order, Phys. Rev. D 99, 064054 (2019)
  * (53) A. Cristofoli, N. Bjerrum-Bohr, P. Damgaard and P. Vanhove, Post-Minkowskian Hamiltonians in general relativity, Phys. Rev. D 100, 084040 (2019)
  * (54) A. Collado, P. Vecchia and R. Russo, Revisiting the second post-Minkowskian eikonal and the dynamics of binary black holes, Phys. Rev. D 100, 066028 (2019)
  * (55) J. Plefka, C. Shi, J.Steinhoff and T. Wang, Breakdown of the classical double copy for the effective action of dilaton-gravity at NNLO, Phys. Rev. D 100, 086006 (2019)
  * (56) D. Bini, T. Damour and A. Geralico, Scattering of tidally interacting bodies in post-Minkowskian gravity, Phys. Rev. D 101, 044039 (2020)
  * (57) C. Cheung and M. Solon, Tidal Effects in the Post-Minkowskian Expansion, Phys. Rev. Lett. 125 , 191601 (2020)
  * (58) Z. Bern, C. Cheung, R. Roiban, C. H. Shen, M. P. Solon and M. Zeng, Scattering Amplitudes and the Conservative Hamiltonian for Binary Systems at Third Post-Minkowskian Order,  Phys.Rev. Lett. 122, 201603 (2019).
  * (59) Z. Bern, C. Cheung, R. Robin, C. H. Shen, M. P. Solon and M. Zeng, Black Hole Binary Dynamics from the Double Copy and Effective Theory, JHEP,10,206(2019).
  * (60) Jiliang Jing, Shuai Chen, Manman Sun, Xiaokai He, and Mengjie Wang, Jieci Wang, Self-consistent Effective-one-body theory for spinless binaries based on post-Minkowskian approximation I: Hamiltonian and decoupled equation for $\psi^{B}_{4}$. Sci. China, Phys. Mech. Astron. 65, 260411 (2022), ArXiv: 2112.09838.
  * (61) J. E. Thompson, H. Chen and B. F. Whiting, Gauge invariant perturbations of the Schwarzschild spacetime,  Class. Quantum Grav. 34 (2017) 174001.
  * (62) S. Chandrasekhar, The mathematical theory of black hole, Oxford University Press, New York, Oxford, 1993.
  * (63) M. Carmeli, Classical fields: general relativity and gauge theory, A Wiley-intersclence Publication, New York, 1982.
  * (64) S. A. Teukolsky, Perturbations of a Rotating Black Hole. I. Fundamental Equations for Gravitational, Electromagnetic, and Neutrino-Field Perturbations, Astrophys. J. 185, 635 (1973).
  * (65) M. Sasaki and H. Tagoshi, Analytic black hole perturbation approach to gravitational radiation, Living Rev. Relativity, 6, 6 (2003).
  * (66) M. Sasaki and T. Nakamura, The Regge-Wheeler equation with sources for both even and odd parity perturbations of the Schwarzschild geometry, Phys. Lett. A 87, 85 (1981).
  * (67) E. Poisson, Gravitational radiation from a particle in circular orbit around a black hole. I. Analytical results for the nonrotating case, Phys. Rev. D 47, 1497 (1993).
  * (68) H. Tagoshi and M. Sasaki, Post-Newtonian expansion of gravitational waves from a particle in circular orbit around a Schwarzschild black hole, Prog. Theor. Phys. 92, 745 (1994).
  * (69) L. E. Kidder, Using full information computing modes of post-Newtonian waveforms from inspiralling compact binaries in circular orbit,  Phys. Rev. D 77, 044016 (2008).