# Electrostatic potential shape of gate defined quantum point contacts

M. Geier  Dahlem Center for Complex Quantum Systems and Physics Department,
Freie Universität Berlin, Arnimallee 14, 14195 Berlin, Germany  J. Freudenfeld
Paul-Drude-Institut für Festkörperelektronik, Leibniz-Institut im
Forschungsverbund Berlin e.V., Hausvogteiplatz 5-7, 10117 Berlin, Germany  J.
T. Silva  Paul-Drude-Institut für Festkörperelektronik, Leibniz-Institut im
Forschungsverbund Berlin e.V., Hausvogteiplatz 5-7, 10117 Berlin, Germany  V.
Umansky  Weizmann Institute of Science, Rehovot 76100, Israel  D.
Reuter111Present address: Department of Physics, Paderborn University,
Warburger Straße 100, 33098 Paderborn Angewandte Festkörperphysik, Ruhr-
Universität Bochum, Universitätsstrasse 150, 44780 Bochum, Germany  A. D.
Wieck  Angewandte Festkörperphysik, Ruhr-Universität Bochum,
Universitätsstrasse 150, 44780 Bochum, Germany  P. W. Brouwer  Dahlem Center
for Complex Quantum Systems and Physics Department, Freie Universität Berlin,
Arnimallee 14, 14195 Berlin, Germany  S. Ludwig  Paul-Drude-Institut für
Festkörperelektronik, Leibniz-Institut im Forschungsverbund Berlin e.V.,
Hausvogteiplatz 5-7, 10117 Berlin, Germany

###### Abstract

Quantum point contacts (QPC) are fundamental building blocks of nanoelectronic
circuits. For their emission dynamics as well as for interaction effects such
as the 0.7-anomaly the details of the electrostatic potential are important,
but the precise potential shapes are usually unknown. Here, we measure the
one-dimensional subband spacings of various QPCs as a function of their
conductance and compare our findings with models of lateral parabolic versus
hard wall confinement. We find that a gate-defined QPC near pinch-off is
compatible with the parabolic saddle point scenario. However, as the number of
populated subbands is increased Coulomb screening flattens the potential
bottom and a description in terms of a finite hard wall potential becomes more
realistic.

###### pacs:

##  I introduction

Given the importance of quantum point contacts (QPC) as fundamental building
blocks of nanoelectronic circuits and the vast amount of literature about them
Landauer (1981); van Wees et al. (1988a); Wharam et al. (1988); Berggren and
Pepper (2010); Micolich (2011), surprisingly little is known about the shape
of their electrostatic potential as a function of gate voltages. However,
knowledge of the precise confinement potential is crucial for understanding
interaction effects in QPCs Rejec and Meir (2006); Koop et al. (2007); Bauer
et al. (2013); Heyder et al. (2015) as well as their carrier emission dynamics
Topinka et al. (2000); Freudenfeld et al. (2020), which is central for
optimizing a quantum electronic circuit. The lateral confinement defines the
mode structure of the one-dimensional (1D) channel while the longitudinal
potential shape governs the coupling of the 1D modes into the surrounding
2DES. Populating the 1D channel with electrons by increasing the voltage
applied to the split gates enhances Coulomb screening inside the constriction.
As a consequence, the lateral confinement potential undergoes a transition
from an unscreened approximately parabolic shape near pinch-off towards a
screened potential for many occupied 1D subbands. Such a transition had been
theoretically predicted Laux et al. (1988). Here, we experimentally
demonstrate it using transport spectroscopy at finite source-drain voltage.

Details of the confinement vary between individual devices produced by various
layouts based on different methods, which include the field effect Wharam et
al. (1988); van Wees et al. (1988a), etching Martin et al. (2008) or oxidation
Senz et al. (2001) techniques and more Fricke et al. (2006); Rössler et al.
(2010). The manifestation of 1D conductance quantization, $G=NG_{\text{Q}}$
with $G_{\text{Q}}=2e^{2}/h$ and $N=1,2,3,\dots$, at cryogenic temperatures is
often seen as a quality feature of QPCs. An “optimally” designed QPC has
several conductance steps that are approximately equidistant in gate voltage
as the QPC is opened up starting from pinch-off at $G=0$. It is tempting to
interpret the presence of equidistant conductance steps van Wees et al.
(1988a); Taboryski et al. (1995); Thomas et al. (1998); Hew et al. (2008);
Rössler et al. (2011); Burke et al. (2012) as a signature of a parabolic
transverse confinement potential as introduced in Ref. Büttiker (1990), since
such a potential has transverse modes at equally spaced energies. However,
this interpretation is questionable as the distance of the conductance steps
as a function of gate voltage is not one-to-one related with the energy
spacing of the 1D modes Rössler et al. (2011); Micolich and Zülicke (2011).

Figure 1: (a) Scanning electron microscope (SEM) picture of Ti/Au gates (light
gray) on the wafer surface (dark) of QPC1 and sketch of the electric field
effect device. Negative voltage $V_{\text{g}}$ applied to the gates (yellow)
is used to locally deplete the 2DES (blue, where conducting, gray where
depleted) 107 nm beneath the surface. (b) Pinch-off curve
$G(V_{\text{g}})/G_{\text{Q}}$ of QPC1 using the source-drain voltage $V=-0.5$
mV; solid line: raw data; dots: corrected for lead resistance
$R_{\text{lead}}=4.62$ k$\Omega$ which includes $4.4$ k$\Omega$ resistance of
external RC filters; inset: simplified circuit diagram of the measurement. (c)
Finite bias spectroscopy of QPC1,
$\text{d}g/\text{d}V_{\text{g}}(V_{\text{QPC}},V_{\text{g}})$, accounting for
the lead resistance (see main text). Local maxima of
$\text{d}g/\text{d}V_{\text{g}}$ (white lines) indicate transitions between
adjacent conductance plateaus. (d) SEM picture of a screen gate equivalent to
that of QPC2. As shown in the sketch, the actual device is covered with a 130
nm thick layer of cross-linked PMMA which carries a global top gate. (e)
Pinch-off curves of QPC2 corrected for a gate voltage dependent lead
resistance, including a constant $4.4$ k$\Omega$ resistance of external RC
filters, cf. Fig. 2: $G(V_{\text{t}})/G_{\text{Q}}$ for $V_{\text{s}}=0.5\,$V
and $G(V_{\text{s}})/G_{\text{Q}}$ for $V_{\text{t}}=-3.4\,$V at $V=-0.1$ mV.
(f) $\text{d}g/\text{d}V_{\text{g}}(V_{\text{QPC}},V_{\text{t}})$ of QPC2,
accounting for the lead resistance. Additional lines and symbols in panels (c)
and (f) are explained in the main text.

We study QPCs of two different designs, but both defined using gate voltages
by means of the electric field effect. In agreement with previous publications
Büttiker (1990); Berggren and Pepper (2010); Burke et al. (2012); Heyder et
al. (2015) our findings are consistent with a parabolic confinement potential
near the pinch-off point of the QPCs. However, as the conductance of a QPC is
increased, more and more carriers populate the 1D subbands and thereby arrange
themselves to partially screen the electric field induced by the applied gate
voltages. The resulting effective potential is then a function of the position
of all charges, which also includes the usually not well-known distribution of
surface states and charged bulk defects. A precise theoretical description of
this screening effect requires a three-dimensional self-consistent calculation
solving the classical Poisson equations together with the quantum mechanical
Schrödinger equations Laux et al. (1988); Yakimenko et al. (2013); Birner et
al. (2007). A self consistent Poisson-Schrödinger calculation performed for a
set of fictitious boundary conditions and for the case of a standard split-
gate defined QPC suggests a transition from a parabolic lateral confinement
for $N=1$ towards a truncated parabola and, eventually, a hard wall
confinement as $N$ is increased Laux et al. (1988). To test this scenario we
measure non-linear response transport through our QPC from which we identify
the energy spacings between its highest occupied 1D modes. We compare our
results to the two extreme scenarios for the lateral electrostatic
confinement: parabolic confinement as, e.g., in Refs. Weisz and Berggren
(1989); Hew et al. (2008); Song et al. (2009); Rössler et al. (2011) and a
hard-wall confinement as, e.g., in Refs. van Wees et al. (1988b); Gloos et al.
(2006). Our results are inconsistent with parabolic confinement for $N\geq 4$
but are consistent with a transition from a parabolic lateral confinement at
$N=1$ towards a hard wall potential as the QPCs are opened up.

##  II Transport spectroscopy of quantum point contacts

Our QPCs are formed using the electric field effect in a 2D electron system
(2DES) embedded 107 nm beneath the surface of a (Al,Ga)As/GaAs
heterostructure. The 2DESs Fermi energy and mobility measured at cryogenic
temperatures are $E_{\text{F}}\simeq 10.9$ meV and $\mu_{e}\simeq 2.6\times
10^{6}\,\text{cm}^{2}/$Vs for QPC1 and similar for QPC2. We performed all
measurements in a helium-3 evaporation cryostat at temperatures near $T=250$
mK. In Fig. 1(a) and (d) we present scanning electron microscope images of the
two QPC samples and sketches of the gate layouts. For QPC1 shown in panel (a)
we use a standard split-gate layout and define the 1D constriction of the 2DES
by applying a negative voltage $V_{\text{g}}$ to both gates while the 2DES and
a back gate approximately 500 $\mu$m below the surface are at ground
potential. The resulting linear response pinch-off curve,
$G(V_{\text{g}})/G_{\text{Q}}$, is presented in Fig. 1(b). It features clear
and, for $N<6$, nearly equidistant steps of quantized conductance. To create
the second QPC2, see Fig. 1 (d), we use a global top gate to globally deplete
the 2DES. Only below a screen gate placed in between the top gate and the 2DES
we induce a finite density of free electrons Bachsoliani et al. (2017). The
screen gate shapes a narrow constriction, i.e., a QPC between 2D leads. Both,
the QPC conductance and the carrier density in the leads are controlled by the
combination of the voltages $V_{\text{t}}$ and $V_{\text{s}}$ applied to the
top gate and screen gate, respectively. We present example pinch-off curves
$G(V_{\text{t}})$ for fixed $V_{\text{s}}=0.5\,$V and $G(V_{\text{s}})$ for
constant $V_{\text{t}}=-3.4\,$V in Fig. 1(e). Note, that the screen gate
voltage is restricted to $V_{\text{s}}\lesssim 0.5\,$V as larger
$V_{\text{s}}$ causes a leakage current from the gate into the 2DES (as
expected for a Schottky barrier).

All our pinch-off curves feature smooth transitions between quantized
conductance plateaus. They indicate that the potential varies slowly and
smoothly in current direction, reminiscent of a parabolic potential profile in
the longitudinal direction, which results in reflectionless contacts between
constriction and leads.

Quantized conductance is a consequence of the energy quantization in a 1D
channel caused by the lateral confinement of the constriction. To
experimentally determine the energies of the 1D modes we need a known energy
scale to compare with. For this reason we measure the differential conductance
$g=\text{d}I/\text{d}V$ (e.g., using a lock-in amplifier) as a function of
source-drain voltage $V$ along the pinch-off curves. In panels (c) and (f) of
Fig. 1 we plot the differential transconductances
$\text{d}g/\text{d}V_{\text{g}}$ ($\text{d}g/\text{d}V_{\text{t}}$) for the
two QPCs as a function of the gate voltage and the bias voltage
$V_{\text{QPC}}$ (defined below) dropping across the QPC. In these plots steps
of the conductance $G(V_{\text{g}},V_{\text{QPC}})$
[$G(V_{\text{t}},V_{\text{QPC}})$] appear as lines of positive differential
transconductance (white). Red lines are a guide for the eye, indicating
resonances between the 1D modes and the chemical potentials of the source and
drain leads. Along the $N$th line of positive (negative) slope counted from
the bottom of the plot, the $N$th 1D subband bottom energy is equal to the
chemical potential in the source (drain) lead,
$\varepsilon_{N}=\mu_{\text{S}}$ ($\varepsilon_{N}=\mu_{\text{D}}$). The lines
frame diamond shaped regions around $V_{\text{QPC}}=0$. Within these regions
the conductance takes the quantized values $G=NG_{\text{Q}}$. Intersection
points at $V_{\text{QPC}}=0$ indicate steps of the linear response pinch-off
curves, i.e., $G=(N-0.5)G_{\text{Q}}$. At intersection points at finite
$V_{\text{QPC}}\neq 0$ the chemical potential drop across a QPC equals the
energy spacing between the corresponding 1D modes,
$|\mu_{\text{S}}-\mu_{\text{D}}|=eV_{\text{QPC}}=\varepsilon_{N}-\varepsilon_{M}$.
The additional curved lines of enhanced differential transconductance within
the $N=1$ diamond indicate the 0.7-anomaly Rejec and Meir (2006); Koop et al.
(2007); Micolich (2011); Bauer et al. (2013); Heyder et al. (2015) which is
not a topic of this article.

Since the source-drain voltage $V$ is applied across the QPC and its leads
(which is always the case, because of the finite contact sizes even for a
four-terminal measurement), the voltage drop across a QPC is
$V_{\text{QPC}}=V-V_{\text{lead}}=V-R_{\text{lead}}I$, cf. sketch in Fig.
1(b). The lead resistance can be directly determined from the linear response
pinch-off curves by forcing the conductance plateaus to their quantized
values, $R_{\text{lead}}=V/I-(NG_{\text{Q}})^{-1}$. Our pinch-off curves in
panels (b) and (e) of Fig. 1 are already corrected for the lead resistances,
while for QPC1 we additionally plot the uncorrected curve, i.e., the raw data
as a solid line. For completeness we present the lead resistances for all
three pinch-off curves in Fig. 2.

Figure 2: Resistances $R_{\text{lead}}$ of the leads to the QPCs, cf. sketch
in Fig. 1(b). For the split gate design of QPC1 $R_{\text{lead}}$ is constant
while it is a function of gate voltages for QPC2.

From these we determine the voltage drop across the QPC,
$V_{\text{QPC}}=V/(R_{\text{lead}}G+1)$, which is the x-axis in panels (c) and
(f) of Fig. 1. The tapered shape of the region of plotted data is a result of
correcting for the lead resistances (we measured between $-8\text{mV}\leq
V\leq 8\,$mV).

At the intersection points marked by red squares in panels (c) and (f) of Fig.
1 the bias $V_{\text{QPC}}$ is precisely equal to the energy spacings between
subsequent subbands,

$\displaystyle\delta\varepsilon(N)=$
$\displaystyle\,\varepsilon_{N+1}-\varepsilon_{N}$ $\displaystyle=$
$\displaystyle\,eV_{\text{QPC}}.$ (1)

We plot $\delta\varepsilon(N)$ in Fig. 3

Figure 3: Subband spacings $\delta\epsilon(N)$ of both QPCs for the three
pinch-off curves presented in Fig. 1. Lines are guides for the eyes. At the
intersection of the two lines of QPC2 the gate voltages $V_{\text{s}}$ and
$V_{\text{t}}$ would be identical for both measurements. Error bars reflect
the uncertainties of the red lines in Figs. 1(c) and (d).

for all three pinch-off curves. Related to the variations in geometry the
three implementations of QPCs have different subband spacings. However, as a
general feature we observe a strong decrease of $\delta\varepsilon(N)$ as the
QPCs are opened and $N$ is increased.

##  III Hard wall versus parabolic lateral confinement

Given reflectionless contacts the conductance of a QPC is limited by its
strongest lateral confinement in the center of the constriction. The measured
subband spacings are uniquely related with this lateral confinement. In the
following, we compare the two most common models describing the lateral
confinement, namely a hard-wall versus a parabolic potential. These two models
may be considered the extreme limits of a “continuum” of realistic scenarios
for the transverse confinement.

###  III.1 Lateral hard-wall potential

Figure 4: Comparison between hard-wall (left column) and parabolic (right
column) potential models of the lateral confinement. (a) Width of hard-wall
potential $W(N)$. (b) Offset of hard-wall potential $\Phi_{0}(N)$. (c) Shape
of hard-wall potential for $1\leq N\leq 9$, only for QPC1. (d) Curvature of
parabolic potential $\omega_{y}(N)$. (e) Offset of parabolic potential
$\Phi_{0}(N)$. (f) Shape of parabolic potential for $1\leq N\leq 9$, only for
QPC1. Error bars in panels (a), (b), (d) and (e) are calculated by error
propagation from the error of $\delta\varepsilon(N)$, cf. Fig. 3.

For the lateral hard-wall potential we model the transverse confinement
potential $\Phi(y)$ as

$\Phi(y)=\begin{cases}\Phi_{0},&|y|\leq{W}/{2}\\\
\infty,&|y|>{W}/{2}\,,\end{cases}$ (2)

where the two parameters $W$ and $\Phi_{0}$ are the width and offset of the
hard-wall potential well. An offset can be caused by a partial depletion of
the constriction related to the incomplete screening in a semi conductor with
a small carrier density. The threshold energies for the transverse modes are

$E_{n}=\frac{\pi^{2}\hbar^{2}n^{2}}{2m^{\star}W^{2}}+\Phi_{0}\,$ (3)

where $m^{\star}=0.067m_{0}$ is the effective mass of the electrons in GaAs,
$m_{0}$ being the free electron mass. Using Eq. (II) to relate the bias
voltage at the intersection points marked by the red squares in Fig. 1(c) and
(f) to the subband spacing $\delta\varepsilon(N)=E_{N+1}-E_{N}$, we calculate
the widths

$W(N)=\pi\hbar\sqrt{\frac{2N+1}{2m\delta\varepsilon(N)}}.$ (4)

Neglecting additional screening effects from the applied bias voltage, these
values of $W(N)$ apply everywhere along the (almost horizontal) lines
connecting pairs of red squares, see the yellow lines for $N=2$ in Fig. 1(c)
and (f). In particular, this allows us to extend our estimate of the width
$W(N)$ to $V_{\text{QPC}}=0$, indicated for $N=2$ by the yellow dot in Fig.
1(c) and (f). Substituting $W$ in Eq. (3) with $W(N)$ we then find the
potential offset $\Phi_{0}$ using the relation $E_{\text{F}}\simeq
E_{N}+0.5\delta\epsilon(N)$, which gives

$\Phi_{0}(N)\simeq
E_{\text{F}}-\delta\varepsilon(N)\,\left(\frac{N^{2}}{2N+1}+\frac{1}{2}\right)\,.$
(5)

The potential shift by $0.5\delta\epsilon(N)$ accounts for the difference
between the $N$th subband bottom $E_{N}$ and the Fermi level $E_{\text{F}}$ in
the center of each diamond at $V_{\text{QPC}}=0$, assuming symmetric coupling
between the 1D constriction and both leads. (The assumption of symmetric
coupling is confirmed by the fact that the lines connecting pairs of red
squares in Fig. 1(c) and (f) are almost horizontal.)

###  III.2 Lateral parabolic potential

To model a lateral parabolic potential we use

$\Phi(y)=\Phi_{0}+\frac{m\omega_{y}^{2}y^{2}}{2},$ (6)

where $\omega_{y}$ and $\Phi_{0}$ are the characteristic frequency and offset
of the parabolic potential well. In analogy to the analysis assuming hard-wall
potentials we determine the two parameters from the measured subband spacings.
At the intersection points indicated with red squares in Fig. 1(c) and (f) we
find

$\hbar\omega_{y}(N)=eV_{\text{QPC}}=\delta\varepsilon(N)$ (7)

and in the centers of the diamonds at $V_{\text{QPC}}=0$ in addition

$\Phi_{0}(N)\simeq E_{F}-N\hbar\omega_{y}\,.$ (8)

###  III.3 Comparison of the two potential shapes

In Fig. 4 we directly compare our results for the hard-wall potential shown in
the left column and for assuming parabolic confinement plotted on the right
hand side. We present the parameters $W$ and $\Phi_{0}$ as a function of the
subband number $N$ for all three QPC implementations for the hard-wall
potential in panels (a) and (b) and $\omega_{y}$ and $\Phi_{0}$ for the
parabolic potential in panels (d) and (e). The results are qualitatively
similar for the various implementations of QPCs; the variations in $W$ or
$\omega_{y}$ between QPCs indicate that the lateral confinement potential of
QPC2 is slightly wider compared to QPC1. In panels (c) and (f) showing the
actual potentials for QPC1, for comparison we indicate the lithographic
distance of 250 nm between the gates seen in the inset of Fig. 1(a). It
corresponds to the white area between regions of gray background. The width of
the hard-wall potential slightly exceeds the lithographic width for $N=9$.
QPC1 does not show further plateaus for $N>9$.

Comparing the two models a substantial difference is visible in $\Phi_{0}(N)$.
While for $N=1$ the potential offset is similar for both models with
$\Phi_{0}/E_{\text{F}}\simeq 0.6$, in case of the hard-wall potential it
slowly decreases to $\Phi_{0}/E_{\text{F}}\simeq 0.4$ at $N=4$ and stays
approximately constant at that level as the QPC is opened further. In
contrast, the decrease of the offset $\Phi_{0}(N)$ of the parabolic potential
with $N$ is much steeper, such that for $N\gtrsim 4$ it moves below the bottom
of the conduction band in the 2D leads, indicated as a dashed line at
$\Phi=0$. We are not aware of a realistic mechanism that could lead to such an
over-screening of the negative voltages applied to the control gates
($V_{\text{g}}$ for QPC1 or $V_{\text{t}}$ for QPC2).

##  IV Discussion and summary

The main result of our simple analysis starting from the measured subband
spacings $\delta\varepsilon(N)$ is, that for $N\geq 4$ we can exclude a
parabolic lateral confinement potential for our QPCs. Based on a self-
consistent calculation it has been suggested that the increasing population of
the 1D constriction with $N$ as a QPC is opened up leads to an increased
screening of the electric field originating from the charged control gates.
For a gate defined QPC this process can cause a transition from a parabolic
confinement for the case of little screening, i.e., $N=1$ towards a truncated
potential with a flat bottom at larger $N$ where many carriers populate the
constriction Laux et al. (1988). Our findings are in favor of such a scenario.
The hard-wall potential presents a somewhat unrealistic extreme case of strong
screening. Nevertheless, for $N\geq 4$ it seems more realistic than the other
extreme, namely the parabolic potential. The true shape of the lateral
confinement potential of a QPC for $N\geq 4$ likely lies between these two
extremes, maybe close to a truncated parabola Laux et al. (1988); Wharam et
al. (1989), i.e., a parabola with a flat bottom identical to that of a hard-
wall potential but with smoothly increasing side walls of constant curvature
as the case for a parabola.

In summary, a parabolic saddle point potential is likely a realistic
description of a QPC near pinch-off, although our measurement can also be
explained with a hard-wall confinement in this regime. However, as the QPC is
opened up beyond $N\simeq 4$, the parabolic lateral confinement turns out to
be a bad approximation. In this regime of enhanced screening a hard-wall
potential is the better approximation.

##  V appendix

###  V.1 Coupling between control gates and the QPC

The electrostatic potential shaping the QPCs is generated and controlled via
the field effect by applying voltages to nearby metal gates. The size of the
plateaus of quantized conductance in the pinch-off curves as a function of
gate voltage, cf. Fig. 1(b) and (e), is proportional to the capacitive
coupling between the control gates and the QPC, which we approximate as a
conducting 1D-channel with the carrier density $n_{\text{1D}}$. We determine
the approximate capacitance per unit length between gate and QPC as

$c_{\text{1D}}=e\delta n_{\text{1D}}/\delta V_{\text{gate}}\,,$ (9)

where $\delta n_{\text{1D}}$ is the carrier density increase as the voltage on
the control gate is increased by $\delta V_{\text{gate}}$. If we take for
$\delta V_{\text{gate}}$ the voltage difference between two subsequent
intersection points of the source- and drain-resonances at $V_{\text{QPC}}=0$
in Fig. 1(c) and (f), $\delta n_{\text{1D}}$ corresponds to the difference of
the values of $n_{\text{1D}}$ at these points with $N$ versus $N+1$ subbands
being populated. The 1D carrier density is

$n_{\text{1D}}(N)=\int_{0}^{\infty}D_{\text{1D}}(E)f(E)\text{d}E\,,$ (10)

where $D_{\text{1D}}=\frac{1}{\pi\hbar}\sqrt{\frac{2m^{\star}}{E}}$ is the 1D
electron density of states and $f(E)$ the Fermi-Dirac distribution. Given
$k_{\text{B}}T\ll E_{\text{F}}$ we approximate $f(E)=1$ for $E<E_{\text{F}}$
and $f(E)=0$ for $E>E_{\text{F}}$. Summing up all 1D modes which are actually
populated for the QPC tuned to the conductance $G=NG_{\text{Q}}$ we find

$\displaystyle n_{\text{1D}}(N)$
$\displaystyle=\frac{\sqrt{2m^{\star}}}{\pi\hbar}\sum_{n=1}^{N}\int_{E_{n}}^{E_{\text{F}}}\frac{1}{\sqrt{E-E_{n}}}dE$
$\displaystyle=\frac{\sqrt{8m^{\star}}}{\pi\hbar}\sum_{n=1}^{N}\sqrt{E_{\text{F}}-E_{n}}\,.$
(11)

Inserting $\delta n_{\text{1D}}(N)=n_{\text{1D}}(N+1)-n_{\text{1D}}(N)$ from
Eq. (V.1) in Eq. (9) we finally determine the 1D capacitance density as

$c_{\text{1D}}(N)=\frac{\sqrt{8m^{\star}e^{2}}}{\pi\hbar}\,\frac{\sqrt{E_{\text{F}}-E_{N+1}}}{\delta
V_{\text{gate}}(N)}\,,$ (12)

where $\delta V_{\text{gate}}(N)$ is the width of the $Nth$ plateau of the
pinch-off curve, cf. Fig. 1, measured between the conductance
$(N+0.5)G_{\text{Q}}$ and $(N-0.5)G_{\text{Q}}$. Substituting $E_{N+1}$ with
the according eigen-energy of the hard-wall potential using Eq. (3) we can now
determine $c_{\text{1D}}(N)$. In Fig. 5

Figure 5: 1D carrier density $n_{\text{1D}}(N)$ assuming an infinitely long
hard-wall 1D channel of width $W(N)$ and depth $E_{\text{F}}-\Phi_{0}(N)$ of
QPC1 (red squares, rhs axis) and corresponding 1D capacitance density
$c_{\text{1D}}(N)$ (blue triangles, lhs axis).

we present the 1D capacitance density $c_{\text{1D}}(N)$, which is the slope
of the also shown 1D carrier density $n_{\text{1D}}(N)$. The strong decrease
of the capacitance with $N$ for $N\leq 4$ is a direct signature of the
increase of the screening of the electric field of the gates with growing
carrier density.

In addition, the variations in capacitance as a function of $N$ explain the
counter-intuitive result that the subband spacings $\delta\epsilon(N)$
strongly vary in a region of almost equal widths of the plateaus of quantized
conductance of the pinch-off curve, cf. Figs. 1(b),(e) and Fig. 3.

###  V.2 Width of the 1D constriction as a function of gate voltage

In Fig. 4(a) we have presented the width of the hard-wall potential $W(N)$. In
Fig. 6

Figure 6: Width of the hard-wall potential $W(V_{\text{g}})$ for QPC1 [same
data as the $W(N)$ in Fig. 4(a)]. The slope of the red line is
$\text{d}W/\text{d}V_{\text{g}}=300\,\text{nm}/$V, cf. main text.

we plot $W(V_{\text{g}})$ for QPC1. Next, we compare this result with the
dependence of the depletion region of a gate voltage using a different sample
on the same wafer material. The sample shown in Fig. 7(a)

Figure 7: (a) QPC nominally identical to QPC1 for $N=4$ coupled to a
hemispherical mirror defined by a negative gate voltage $V_{\text{m}}$. (b)
Conductance of the QPC as a function of $V_{\text{m}}$. (c) Fourier transform
of the conductance. From the peak value, we determine the period $\delta
V_{\text{m}}\simeq 150\,$mV of the oscillations in panel (b).

contains a QPC nominally identical to QPC1 and a hemispherical mirror gate.
The two samples have been prepared in parallel and on the same wafer. In Fig.
7(b) we present the conductance of the QPC as a function of the voltage
applied to the mirror gate. The bare conductance (without mirror) is
$G=4G_{\text{Q}}$. However, with the 2DES below the mirror gate depleted it is
reduced roughly by a factor 2, because of enhanced back scattering through the
QPC. At the same time $G(V_{\text{m}})$ oscillates with a visibility of 40 %.
Both, the conductance reduction and oscillation are related to the formation
of localized modes inside the hemispherical resonator. The oscillation can be
interpreted in analogy to the oscillations of the standing wave in a Fabry-
Pérot resonator, while here, the coherent electrons generate the standing
wave. By increasing the gate voltage $V_{\text{m}}$ we decrease the area of
2DES depleted next to the mirror gate and thereby increase the length of the
resonator (the distance between the QPC and the mirror). Per period of the
conductance oscillation the length of the resonator is reduced by half of the
Fermi wavelength
$\text{d}L_{\text{res}}/\text{d}V_{\text{m}}=0.5\lambda_{\text{F}}/\delta
V_{\text{m}}$ with the resonator length $L_{\text{res}}$. We determine the
averaged period from the fast Fourier transform of the oscillation, cf. Fig.
7(c), and find $\delta V_{\text{m}}\simeq 150\,$mV. With
$\lambda_{\text{F}}=45\,$nm we finally estimate the rate of the depletion
length reduction as
$\text{d}L_{\text{res}}/\text{d}V_{\text{m}}=150\,\text{nm}/$V. Changing the
voltage applied to the QPC gates instead of the mirror gates results in the
same conclusion while in this case the interference pattern appears on top of
the QPC pinch-off curve.

To estimate the depletion of the electron system between the QPC gates we have
to add up the electric fields of both gates. Based on the fact that the same
voltage is applied to both gates and the distance between gates is more than
two times larger than the distance between each gate and the 2DES we neglect
any influences that a gate has on the depletion caused by the other gate. From
the slope of the red line in Fig. 6 we find the dependence of the width of the
hard-wall potential $W(V_{\text{g}})$ as a function of gate voltage to be
$\text{d}W/\text{d}V_{\text{g}}\simeq 300\,\text{nm}/$V, twice as large as the
effect of a single mirror gate. This finding supports the applicability of the
hard-wall model for QPCs with $N\geq 4$.

##  VI Acknowledgments

We thank Philipp Altpeter for technical support and are grateful for financial
support from the DFG via Grant No. LU 819/11-1. M.G. acknowledges support by
project A03 of the CRC-TR 183. A.D.W. acknowledges gratefully support of DFG-
TRR160, BMBF - Q.Link.X 16KIS0867 and the DFH/UFA CDFA-05-06.

M. Geier and J. Freudenfeld contributed equally to this work.

## References

  * Landauer (1981) R. Landauer, Physics Letters A 85, 91 (1981), ISSN 0375-9601, URL http://www.sciencedirect.com/science/article/pii/0375960181902309.
  * van Wees et al. (1988a) B. J. van Wees, H. van Houten, C. W. J. Beenakker, J. G. Williamson, L. P. Kouwenhoven, D. van der Marel, and C. T. Foxon, Phys. Rev. Lett. 60, 848 (1988a), URL https://link.aps.org/doi/10.1103/PhysRevLett.60.848.
  * Wharam et al. (1988) D. A. Wharam, T. J. Thornton, R. Newbury, M. Pepper, H. Ahmed, J. E. F. Frost, D. G. Hasko, D. C. Peacock, D. A. Ritchie, and G. A. C. Jones, Journal of Physics C: Solid State Physics 21, L209 (1988), URL http://stacks.iop.org/0022-3719/21/i=8/a=002.
  * Berggren and Pepper (2010) K.-F. Berggren and M. Pepper, Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences 368, 1141 (2010), URL https://royalsocietypublishing.org/doi/abs/10.1098/rsta.2009.0226.
  * Micolich (2011) A. P. Micolich, Journal of Physics: Condensed Matter 23, 443201 (2011), URL http://stacks.iop.org/0953-8984/23/i=44/a=443201.
  * Rejec and Meir (2006) T. Rejec and Y. Meir, Nature 442, 900–903 (2006), ISSN 1476-4687, URL https://www.nature.com/articles/nature05054.
  * Koop et al. (2007) E. J. Koop, A. I. Lerescu, J. Liu, B. J. van Wees, D. Reuter, A. D. Wieck, and C. H. van der Wal, Journal of Superconductivity and Novel Magnetism 20, 433–441 (2007), ISSN 1557-1947, URL https://link.springer.com/article/10.1007/s10948-007-0289-5.
  * Bauer et al. (2013) F. Bauer, J. Heyder, E. Schubert, D. Borowsky, D. Taubert, B. Bruognolo, D. Schuh, W. Wegscheider, J. von Delft, and S. Ludwig, Nature 501, 73–78 (2013), URL https://www.nature.com/articles/nature12421.
  * Heyder et al. (2015) J. Heyder, F. Bauer, E. Schubert, D. Borowsky, D. Schuh, W. Wegscheider, J. von Delft, and S. Ludwig, Phys. Rev. B 92, 195401 (2015), URL http://link.aps.org/doi/10.1103/PhysRevB.92.195401.
  * Topinka et al. (2000) M. A. Topinka, B. J. LeRoy, S. E. J. Shaw, E. J. Heller, R. M. Westervelt, K. D. Maranowski, and A. C. Gossard, Science 289, 2323 (2000), ISSN 0036-8075, URL http://science.sciencemag.org/content/289/5488/2323.
  * Freudenfeld et al. (2020) J. Freudenfeld, M. Geier, V. Umansky, P. Brouwer, and S. Ludwig, arXiv p. arXiv:2002.12340 (2020), URL https://arxiv.org/abs/2002.12340.
  * Laux et al. (1988) S. Laux, D. Frank, and F. Stern, Surface Science 196, 101 (1988), ISSN 0039-6028, URL http://www.sciencedirect.com/science/article/pii/0039602888906711.
  * Martin et al. (2008) T. P. Martin, C. A. Marlow, L. Samuelson, A. R. Hamilton, H. Linke, and R. P. Taylor, Phys. Rev. B 77, 155309 (2008), URL https://link.aps.org/doi/10.1103/PhysRevB.77.155309.
  * Senz et al. (2001) V. Senz, T. Heinzel, T. Ihn, S. Lindemann, R. Held, K. Ensslin, W. Wegscheider, and M. Bichler, Journal of Physics: Condensed Matter 13, 3831 (2001), URL https://iopscience.iop.org/article/10.1088/0953-8984/13/17/303.
  * Fricke et al. (2006) C. Fricke, J. Regul, F. Hohls, D. Reuter, A. Wieck, and R. Haug, Physica E: Low-dimensional Systems and Nanostructures 34, 519–521 (2006), ISSN 1386-9477, URL http://www.sciencedirect.com/science/article/pii/S1386947706001500.
  * Rössler et al. (2010) C. Rössler, M. Herz, M. Bichler, and S. Ludwig, Solid State Communications 150, 861 (2010), ISSN 0038-1098, URL http://www.sciencedirect.com/science/article/pii/S0038109810000906.
  * Taboryski et al. (1995) R. Taboryski, A. Kristensen, C. B. Sørensen, and P. E. Lindelof, Phys. Rev. B 51, 2282 (1995), URL https://link.aps.org/doi/10.1103/PhysRevB.51.2282.
  * Thomas et al. (1998) K. J. Thomas, J. T. Nicholls, N. J. Appleyard, M. Y. Simmons, M. Pepper, D. R. Mace, W. R. Tribe, and D. A. Ritchie, Phys. Rev. B 58, 4846 (1998), URL https://link.aps.org/doi/10.1103/PhysRevB.58.4846.
  * Hew et al. (2008) W. Hew, K. Thomas, I. Farrer, D. Anderson, D. Ritchie, and M. Pepper, Physica E: Low-dimensional Systems and Nanostructures 40, 1645 (2008), ISSN 1386-9477, URL http://www.sciencedirect.com/science/article/pii/S1386947707006364.
  * Rössler et al. (2011) C. Rössler, S. Baer, E. de Wiljes, P.-L. Ardelt, T. Ihn, K. Ensslin, C. Reichl, and W. Wegscheider, New Journal of Physics 13, 113006 (2011), URL https://iopscience.iop.org/article/10.1088/1367-2630/13/11/113006.
  * Burke et al. (2012) A. M. Burke, O. Klochan, I. Farrer, D. A. Ritchie, A. R. Hamilton, and A. P. Micolich, Nano Lett. 12, 4495 (2012), ISSN 1530-6984, URL https://pubs.acs.org/doi/abs/10.1021/nl301566d.
  * Büttiker (1990) M. Büttiker, Phys. Rev. B 41, 7906 (1990), URL https://link.aps.org/doi/10.1103/PhysRevB.41.7906.
  * Micolich and Zülicke (2011) A. P. Micolich and U. Zülicke, Journal of Physics: Condensed Matter 23, 362201 (2011), URL https://iopscience.iop.org/article/10.1088/0953-8984/23/36/362201.
  * Yakimenko et al. (2013) I. I. Yakimenko, V. S. Tsykunov, and K.-F. Berggren, Journal of Physics: Condensed Matter 25, 072201 (2013), URL https://iopscience.iop.org/article/10.1088/0953-8984/25/7/072201.
  * Birner et al. (2007) S. Birner, T. Zibold, T. Andlauer, T. Kubis, M. Sabathil, A. Trellakis, and P. Vogl, IEEE Transactions on Electron Devices 54, 2137 (2007), ISSN 1557-9646, URL https://ieeexplore.ieee.org/document/4294186.
  * Weisz and Berggren (1989) J. F. Weisz and K.-F. Berggren, Phys. Rev. B 40, 1325 (1989), URL https://link.aps.org/doi/10.1103/PhysRevB.40.1325.
  * Song et al. (2009) J. Song, Y. Kawano, K. Ishibashi, J. Mikalopas, G. R. Aizin, N. Aoki, J. L. Reno, Y. Ochiai, and J. P. Bird, Applied Physics Letters 95, 233115 (2009), URL https://aip.scitation.org/doi/10.1063/1.3272677.
  * van Wees et al. (1988b) B. J. van Wees, L. P. Kouwenhoven, H. van Houten, C. W. J. Beenakker, J. E. Mooij, C. T. Foxon, and J. J. Harris, Phys. Rev. B 38, 3625 (1988b), URL https://link.aps.org/doi/10.1103/PhysRevB.38.3625.
  * Gloos et al. (2006) K. Gloos, P. Utko, M. Aagesen, C. B. Sørensen, J. B. Hansen, and P. E. Lindelof, Phys. Rev. B 73, 125326 (2006), URL https://link.aps.org/doi/10.1103/PhysRevB.73.125326.
  * Bachsoliani et al. (2017) N. Bachsoliani, S. Platonov, A. D. Wieck, and S. Ludwig, Phys. Rev. Applied 8, 064015 (2017), URL https://link.aps.org/doi/10.1103/PhysRevApplied.8.064015.
  * Wharam et al. (1989) D. A. Wharam, U. Ekenberg, M. Pepper, D. G. Hasko, H. Ahmed, J. E. F. Frost, D. A. Ritchie, D. C. Peacock, and G. A. C. Jones, Phys. Rev. B 39, 6283 (1989), URL https://link.aps.org/doi/10.1103/PhysRevB.39.6283.


# Compact Binary Systems Waveform Generation with Generative Pre-trained
Transformer

Ruijun Shi  Equal contribution  Department of Astronomy, Beijing Normal
University, Beijing 100875, China  Yue Zhou  Equal contribution  Peng Cheng
Laboratory, Shenzhen 518055, China  Tianyu Zhao  Department of Astronomy,
Beijing Normal University, Beijing 100875, China  Peng Cheng Laboratory,
Shenzhen 518055, China  Zhoujian Cao  Department of Astronomy, Beijing Normal
University, Beijing 100875, China  Zhixiang Ren  Corresponding author
<EMAIL_ADDRESS>Peng Cheng Laboratory, Shenzhen 518055, China

(August 31, 2024)

###### Abstract

Space-based gravitational wave detection is one of the most anticipated
gravitational wave (GW) detection projects in the next decade, which will
detect abundant compact binary systems. However, the precise prediction of
space GW waveforms remains unexplored. To solve the data processing difficulty
in the increasing waveform complexity caused by detectors’ response and
second-generation time-delay interferometry (TDI 2.0), an interpretable pre-
trained large model named CBS-GPT (Compact Binary Systems Waveform Generation
with Generative Pre-trained Transformer) is proposed. For compact binary
system waveforms, three models were trained to predict the waveforms of
massive black hole binary (MBHB), extreme mass-ratio inspirals (EMRIs), and
galactic binary (GB), achieving prediction accuracies of 98%, 91%, and 99%,
respectively. The CBS-GPT model exhibits notable interpretability, with its
hidden parameters effectively capturing the intricate information of
waveforms, even with complex instrument response and a wide parameter range.
Our research demonstrates the potential of large pre-trained models in
gravitational wave data processing, opening up new opportunities for future
tasks such as gap completion, GW signal detection, and signal noise reduction.

††preprint: APS/123-QED

##  I Introduction

Figure 1: Overview of CBS-GPT pipeline. Inputs contain three kinds of GW
sources (MBHB, EMRIs, and gb), after feeding into CBS-GPT, the successive
waveforms are extrapolated.

The first direct detection of binary black hole merger event (GW150914)[1, 2]
by Laser Interferometer Gravitational-Wave Observatory (LIGO) has opened an
innovative window to understand the universe, which provides direct evidence
for the validity of Einstein’s General Relativity. Gravitational wave (GW)
observations will clarify many questions in astrophysics, cosmology, and
fundamental physics[3, 4, 5, 6, 7, 8, 9, 10]. So far, the ground-based
gravitational wave detector has reported over a hundred compact binary
coalesces (CBC) events[11], and recently pulsar timing array (PTA) has also
successfully detected sound evidence of the existence of Stochastic
Gravitational Wave Background (SGWB)[12, 13, 14, 15]. To gain a deeper
understanding and an overall picture of gravitational wave cosmology[16], the
field of low-frequency gravitational waves needs to be widely covered. Hence,
space-based gravitational wave detector projects have been launched. For
example, Laser Interferometer Space Antenna (LISA)[17], Taiji[18, 19] and
Tianqin[20] are all scheduled for the 2030s. The space-based gravitational
wave detection could avoid terrestrial noise[21] and make the detection of
low-frequency ($10^{-4}-0.1\mathrm{Hz}$) gravitational wave signals more
promising. In particular, future space-based gravitational wave detections are
expected to detect a richer variety of gravitational wave sources including
massive black hold binary (MBHB), extreme mass-ratio inspirals (EMRIs), and
galactic binary (GB)[17].

In space-based and ground-based detections, GW signals are both extremely weak
and usually buried in instrumental noise. Traditional methods will face
challenges in meeting the demands of gravitational wave data analysis in the
future. Artificial Intelligence (AI)-driven frameworks have shed some new
light on this issue. Specifically, AI techniques have been successfully
applied in various subjects such as gravitational wave signal detection[22,
23, 24, 25, 26, 27], parameter estimation[28, 29, 30], signal extraction[31]
and noise reduction[32, 33, 34, 35]with promising results. Additionally, the
target of space gravitational wave detectors is also one type of complex and
multi-scale waveforms (such as MBHB, EMRIs, and GB). Therefore, AI is expected
to be a valuable tool for understanding and predicting waveform, as well as
providing guidance and enlightenment on data-driven methods for GW framework
design in the near future. Some previous studies focused on generating black
hold binaries (BHB) waveforms. Lee et al.[36] employed a Recurrent Neural
Network (RNN) that is capable of generating BHB waveforms during the merging
and ringdown phases of non-spinning binary black hole coalescence. Khan et
al.[37] demonstrated that vanilla transformer can learn quasi-circular,
spinning, non-precessing binary black hole mergers waveform. Similarly, Chua
et al.[38] used a greedy algorithm to build a reduced basis, allowing for the
rapid generation of BHB waveforms. Recently, large-scale language models (LLM)
based on attention mechanism have shown their tremendous power[39, 40, 41,
42]. Some studies indicate that similar architectures can be applied to the GW
data analysis[31, 35]. Developing a deep learning data processing framework
for space-based GW detection is an impending necessity.

However, the above investigations did not consider the space-based GW
detectors’ response, and the second-generation time-delay interferometry (TDI
2.0) combinations. Moreover, prior investigations have limited their analysis
to a restricted range of waveform parameters. In our paper, we are dedicated
to further investigation on more complex waveforms and training a pre-trained
model to facilitate solving downstream problems. Hence, we proposed CBS-GPT
(Compact Binary Systems Waveform Generation with Generative Pre-trained
Transformer) model, which is an interpretable, transformer-based, and self-
supervised large model for prediction of compact binary sources (MBHB, EMRIs,
and GB). In CBS-GPT, patching and hybrid embedding mechanisms are proposed for
full extraction of waveform features. By utilizing the self-attention
mechanism and mean square error loss, CBS-GPT is trained for each GW waveform
source. The experiment results illustrate that CBS-GPT can accurately predict
the subsequent waveform based on the input waveform. The average overlap
between the predicted and target waveforms of MBHB, EMRIs, and GB reaches
0.981, 0.912, and 0.991, respectively. We have also discovered that waveform
complexity can significantly influence the model’s prediction performance and
CBS-GPT can match the key frequencies effectively. Finally, through attention
map visualization and calculating the correlation between the waveform and
hidden parameters of the model, we discover that CBS-GPT can pay attention to
the area with the same phase, confirming that CBS-GPT can adapt itself to
learn waveform features even under complex instrument response and wide
parameter range.

The rest of this paper is organized as follows. Section II describes data
generation and the CBS-GPT model architecture. In Section III, we present our
overlap and attention map results and discuss the overlap and CBS-GPT
interpretability outcomes. Finally, Section IV highlights our findings and
proposes potential future research based on the results.

##  II Methodology

###  II.1 Data

(a) MBHB

(b) EMRIs

(c) GB

Figure 2: TDI2.0 response complicates waveforms. To simplify waveform
comparison here, all waveforms were normalized to have a maximum amplitude of
1. The $\Delta t$ in the figure represents the sampling rate. The effects of
different parameters on time and frequency domain are shown on the left and
right panels, respectively. (a) MBHB waveforms at different $M_{tot}$. At high
frequencies, the TDI response function has a greater impact. The gray line
represents the TDI 2.0 transfer function in the frequency domain. (b) EMRIs
waveforms at different $e_{0}$. As the eccentricity increases, the EMRIs
waveform becomes more and more complex in the frequency domain. (c) GB
waveforms at different $f$. The GB signal is relatively simple and is a
single-frequency signal.

Space-based GW detectors’ targets are GW signals at frequencies of
$[10^{-4},0.1]\text{Hz}$. We focus on three compact binary sources that are of
major interest to LISA: MBHB, EMRIs, and GB. Figure 2 displays data examples.
Detailed information of the data generation process is given below.

####  II.1.1 MBHB

MBHB is one of LISA’s main detection targets[17]. The mass spectrum,
formation, and evolution of MBHB, as well as the interplay between MBHB and
their host galaxies, can all be better understood through observations. In
this paper, SEOBNRv4_opt[43] with including $l=m=2$ mode is used to generate
MBHB waveform. The parameter space of the MBHB dataset is shown in Table 2(a).
In Figure 2(a), the TDI 2.0 transfer function significantly affects high-
frequency transmissions due to the lower total mass of MBHB. Firstly, we
generate MBHB time-series waveforms with a length of 20,000 points with a
sampling rate of 5 seconds. During the training phase, each sample of training
waveforms is truncated to 4,200 points, with 4,000 before and 200 points after
the location of merge time. During the inference phase, the 4,000 valid points
before the merge time are fed into CBS-GPT to predict the succeeding 200
points, hence achieving a 20:1 extrapolation prediction of MBHB waveform.

Table 1: Parameters distribution of training set and test set

Parameter |  Description |  Parameter distribution
---|---|---
$M_{tot}$ |  Total mass of massive black hole binaries $m_{1}+m_{2}$ |  log-Uniform $[5.5,7]M_{\odot}$
$q$ |  Mass ratio $\frac{m_{2}}{m_{1}}$ |  Uniform $[0.1,1]$
$S^{z}_{1},S^{z}_{2}$ |  Spin parameters of two black holes |  Uniform $[-0.99,0.99]$
$\iota$, $\psi$ |  The inclination angle and polarization angle |  Uniform $[0,\pi]$
$\phi_{c}$ |  Coalescence phase. |  Uniform $[0,2\pi]$
$\lambda$ |  Ecliptic longitude |  Uniform $[0,2\pi]$
$\beta$ |  Ecliptic latitude |  Uniform $[0,\pi]$
(a) Parameters space of MBHB dataset

Parameter |  Description |  Parameter distribution
---|---|---
$M$ |  The mass of massive black hole (MBH) |  Uniform $[10^{5}-10^{7}]M_{\odot}$
$m$ |  The mass of stellar-mass compact |  Fix $[10M_{\odot}]$
$a$ |  Spin parameter of MBH |  Uniform $[10^{-3},0.8]$
$p_{0}$ |  Semi-latus rectum |  Uniform $[10,16]$
$e_{0}$ |  Eccentricity |  Uniform $[10^{-3},0.4]$
$\iota_{0}$ |  The orbit’s inclination angle from the equatorial plane |  Uniform $[-0.98,0.98]$
$\theta_{S}$, $\phi_{S}$ |  The polar and azimuthal sky location angle |  Uniform $[10^{-3},\pi]$
$\theta_{K}$, $\phi_{K}$ |  The azimuthal and polar angles describing the orientation of the spin angular momentum vector of the MBH |  Uniform $[10^{-3},\pi]$
$\Phi_{\varphi,0},\Phi_{\theta,0},\Phi_{r,0}$ |  The phase of azimuthal, polar, and radial modes |  Fix $[0]$
(b) Parameters space of EMRIs dataset

Parameter |  Description |  Parameter distribution
---|---|---
$f$ |  Frequency |  log-Uniform $[-4,-2]$ Hz
$\dot{f}$ |  The derivative of $f$ |  Fix $[10^{-14}]$
$A$ |  Amplitude |  Uniform $[10^{-23},10^{-21}]$
$\iota_{0}$, $\psi$, $\phi_{0}$ |  The inclination angle, polarization angle and initial phase |  Uniform $[0,\pi]$
$\lambda$ |  Ecliptic longitude |  Uniform $[0,2\pi]$
$\beta$ |  Ecliptic latitude |  Uniform $[0,\pi]$
(c) Parameters space of GB dataset

####  II.1.2 EMRIs

EMRIs are a kind of black hole binary system with a mass ratio of $m/M\simeq
10^{-4}-10^{-7}$ and massive black holes have a mass range of $M\simeq
10^{5}-10^{7}M_{\odot}$. EMRIs waveforms are able to encapsulate the
properties of space-time near a massive black hole. EMRIs are among the
primary detection targets for the space-based GW detectors, possessing the
potential to unveil new physical phenomena[44, 45, 46]. We employ
FastEMRIsWaveforms (FEW) package[47] to generate EMRIs waveforms with a
sampling rate of 5s. A random slice of 4,200 points is selected to form a
waveform for training and inference. Training dataset with randomly sampled
parameters is repeatedly regenerated to guarantee complete parameter space
coverage. Before feeding the waveform into the model, we standardized it to
help the model capture the information from EMRIs waveforms. The parameter
space of the EMRIs dataset is shown in Table 2(b). The complexity of the EMRIs
waveform is visible in Figure 2(b). As the eccentricity increases, the
complexity of the waveform also increases, which is more obvious in the
frequency domain.

####  II.1.3 Galactic binary

Within the Milky Way galaxy, a substantial population of binary white dwarf
systems exists, posing foreground noise challenges for space-based
gravitational wave detectors. We use the following GB model (Equation 1) to
generate GB waveform[48]:

$\displaystyle h_{+}^{\text{src}}{(t)}$
$\displaystyle=\mathcal{A}(1+\cos^{2}\iota)\cos\Phi(t),$ (1) $\displaystyle
h_{\times}^{\text{sr}\mathrm{c}}(t)$
$\displaystyle=2\mathcal{A}\sin\iota\sin\Phi(t),$ $\displaystyle\Phi(t)$
$\displaystyle=\phi_{0}+2\pi
f_{0}t+\pi\dot{f}_{0}t^{2}+\frac{\pi}{3}\ddot{f}_{0}t^{3},$
$\displaystyle\ddot{f}_{0}$
$\displaystyle=\frac{11}{3}\frac{\dot{f}_{0}^{2}}{f_{0}}.$

Similar to EMRIs, GB waveforms are generated with a duration of 1 year and a
sampling rate of 1/15 Hz. A slice of 4,200 points is randomly truncated for
training and inference. The parameter space of the GB dataset is shown in
Table 2(c).

####  II.1.4 Detector response and TDI 2.0

After generating the waveform, we project it into the LISA detector[49]:

$\displaystyle h_{I,II}(t)=$
$\displaystyle\frac{\sqrt{3}}{2}h_{+}(t)F_{I,II}^{+}(\theta_{S},\phi_{S},\psi_{S})+$
(2)
$\displaystyle\frac{\sqrt{3}}{2}h_{\times}(t)F_{I,II}^{\times}(\theta_{S},\phi_{S},\psi_{S}),$

where $F_{I,II}^{+}$ and $F_{I,II}^{\times}$ are the antenna pattern
functions, $\phi_{S}$ is the polarisation angle, $\theta_{S},\phi_{S}$ are the
polar and azimuthal sky location angles, respectively. Space-based
gravitational wave detectors have unequal arm lengths, which result in
significant laser frequency noise. To mitigate this issue, TDI techniques are
commonly employed to suppress laser frequency noise[50, 51]. As a result, the
detector response and the second-generation Michelson combinations (TDI 2.0)
response of GW are calculated using Fastlisaresponse[52] in this case. TDI 2.0
generates three channels X, Y, and Z. By combining X, Y, and Z, three
independent channels A, E, and T are obtained,

$\displaystyle A$ $\displaystyle=(Z-X)/\sqrt{2},$ (3) $\displaystyle E$
$\displaystyle=(X-2Y+Z)/\sqrt{6},$ $\displaystyle T$
$\displaystyle=(X+Y+Z)/\sqrt{3}.$

In contrast to the waveform template, the incorporation of response functions
and TDI 2.0 combination introduces increased complexity to the waveform,
especially in the high-frequency part. As depicted in Figure 2, MBHB waveforms
exhibit significant differences at various parameter values. The increase in
complexity presents more challenges in our work while also making our method
more practical in future space-based GW research.

###  II.2 CBS-GPT Model

Transformer[53] is a class of deep learning models that has exhibited
excellent performance in various tasks, such as natural language processing
(NLP)[40, 41, 42] and computer vision[54]. We incorporate the masked self-
attention mechanism and feed-forward neural network of transformer to build
our CBS-GPT model.

Patching Firstly, each input waveform $x^{i}$ is divided into non-overlapping
patches, and we refer to each patch as a "token" here. The number of tokens
depends on the length of each patch and the total length of the waveform. For
all three types of signal sources, we have an input waveform with 4,200
sampling points, which is segmented into 1,050 tokens, and each token has a
length of 4. Prior to patching, the input waveform is preprocessed by Z-score
normalization, which facilitates the model in capturing waveform information
more effectively:

$\text{Normal}(s)=\frac{s-\text{Mean}(s)}{\text{Std}(s)}$ (4)

where $s$ represents the input waveform, $\text{Mean}(\cdot)$ and
$\text{Std}(\cdot)$ represent finding the mean and standard variance of the
waveform respectively.

Hybrid Embedding Different from NLP applications, in our scenario, each token
contains richer physical information, and can not be tokenized by common
tokenizers as in NLP. Hence, the hybrid embedding module is utilized in our
model. As Figure 1 shows, it includes a dense layer and positional embedding
layer. The dense layer performs linear projection, which achieves dimension
mapping and meanwhile preserves the entire information of the input sequence.
The dense layer projects the token into a high-dimensional space of dimension
$d_{model}=1280$. The positional layer is also a linear layer with learnable
parameters that encodes positional relationships between tokens, which is
rather important in improving prediction accuracy[53].

Encoder block The encoder contains $36$ blocks. Each block mainly consists of
an attention module and a feed-forward neural network. As for the attention
module, masked multi-head self-attention (MMHA) is adopted in our work, which
enables information to be projected into matrices in different ways, thereby
enhancing the expressive capacity of the model. The computation process of the
attention module is as follows:

$\displaystyle
head_{i}(Q_{i},K_{i},V_{i})=\text{softmax}\left(\frac{Q_{i}K_{i}^{T}\cdot\mathrm{mask}}{\sqrt{d}}\right)V_{i},$
(5)

where $Q_{i}=W^{Q}x_{i},K_{i}=W^{K}x_{i},V_{i}=W^{V}x_{i}$ represent queries,
keys and values of the $i$-th attention head respectively. Then, concatenate
the outputs of all the heads and apply a linear transformation to obtain the
final multi-head attention output:

$\mathrm{MMHA}(Q,K,V)=\mathrm{Concat}(head_{1},...,head_{H})W^{H}\>,$ (6)
$\mathrm{H^{\prime}}=\mathrm{LayerNorm}(\mathrm{MMHA}(Q,K,V))+x_{i}\>,$ (7)

where $x_{i}$ represents the hybrid embedding output or the output of the
previous encoder block, and $H^{\prime}$ is the output of the attention
module. In MMHA block, $W^{Q}$, $W^{K}$, $W^{V}$ and $W^{H}$ are learnable
parameters.

The feed-forward network (FFN) that composed of two dense layers is connected
to each attention module. In order to get better results, we employ the
residual connection, which helps reduce the vanishing gradient problem.

$\mathrm{Inter}(H^{\prime})=\mathrm{GeLU}(H^{\prime}W_{1}+b_{1})W_{2}+b_{2}$
(8)
$\mathrm{FFN}(H^{\prime})=\mathrm{LayerNorm}(\mathrm{Inter}(H^{\prime}))+H^{\prime}$
(9)

Loss Function Mean Squared Error (MSE) loss is used to measure the difference
between the predictions and the true values as Equation 10 illustrates.

$\mathcal{L}=\mathbb{E}_{\bm{y}}\frac{1}{M}\sum_{i=1}^{M}\|\hat{\bm{y}}^{(i)}-\bm{y}^{(i)}\|_{2}^{2}\text{.}$
(10)

Where $\bm{y}^{(i)}$ and $\hat{\bm{y}}^{(i)}$ represent prediction and true
value of each token. $M$ represents the total input sampling points which
equals 4,200.

###  II.3 Training and Inference

CBS-GPT contains a total of 36 encoder blocks, with each block having 20
attention heads. The hidden size of each attention head is 64. And the hidden
size of two dense layers of FFN is 5,120 and 1,280, respectively. During
training, Adam[55] optimizer with $\beta_{1}=0.9$, $\beta_{2}=0.999$ is used,
and the initial learning rate is 2e-4. To make sure that the training dataset
covers the entire parameter space, the training dataset is continuously
updated, and the parameters of each waveform are randomly selected from the
parameter space of the sources during training. After passing through the LISA
response, each waveform is divided into three channels (A, E, and T). In this
study, the E channel is selected to train the model. The model was trained on
two NVIDIA V100 GPU for approximately 30 hours.

During inference, the initial input sequence contains 1,000 valid tokens and
50 masked tokens that are masked with zero, whose corresponding value in the
mask matrix equals 1. In the first step, the 1,001-st token is predicted and
replaces the previous 1,001-st token, and so forth, 50 successive tokens are
predicted based on 1,000 valid input tokens.

##  III Results and discussion

(a) MBHB

(b) EMRIs

(c) GB

Figure 3: The overlap distributions of MBHB, EMRIs, and GB are shown in the
left panel. The parameters that have a greater impact on overlap are displayed
in the middle and the right panels. MBHB, EMRIs, and GB have a mean overlap of
0.981, 0.912, and 0.991, respectively. (a) For MBHB waveforms, the biggest
impact on the prediction results is $M_{tot}$ and spin parameters
$\chi_{\text{eff}}$. The middle sub-figure showcases differences in overlap
distribution associated with $M_{tot}$, and the right sub-figure portrays the
overlap heat map of $M_{tot}$ and $\chi_{\text{eff}}$. (b) For EMRIs
waveforms, $e_{0}$ and $M$ have the greatest impact on overlap. As $e_{0}$
increases, the EMRIs waveform becomes increasingly complex. The middle sub-
figure showcases differences in overlap distribution associated with $e_{0}$,
and the right sub-figure portrays the overlap heat map of $e_{0}$ and $M$. (c)
For GB waveforms, the parameter with the greatest impact is the frequency
parameter. The middle sub-figure showcases differences in overlap distribution
associated with $f$, and the right sub-figure portrays the overlap heat map of
$f$ and overlap.

###  III.1 Overlap Result

During inference, for each signal source, 10,000 waveforms with random uniform
parameter distribution are generated to test GP-GPT’s performance. By
providing the former part of each signal (4,000 sampling points), the
following 200 sampling points are extrapolated through CBS-GPT. Overlap is
defined to evaluate the extrapolation accuracy of the predicted waveform.

Overlap is calculated between the target waveform and predicted waveform
generated by CBS-GPT as stated in Equation 11. The overlap $\mathcal{O}$
ranges between $[0,1]$, with values closer to 1 indicating that the predicted
waveform is more similar to the target waveform.

$\mathcal{O}({h}_{t},{h}_{p})=\max_{t_{c}}\left(\hat{h}_{t}|\hat{h}_{p}[t_{c}]\right)^{1/2},$
(11)

with

$\displaystyle(h|s)$
$\displaystyle=2\int_{f_{\min}}^{f_{\max}}\frac{\tilde{h}^{*}(f)\tilde{s}(f)+\tilde{h}(f)\tilde{s}^{*}(f)}{S_{n}(f)}df,$
(12) $\displaystyle\hat{h}$ $\displaystyle=\frac{h}{\sqrt{(h|h)}}$

where $t_{c}$ represents time-shifted and we set $S_{n}(f)=1$. Overall, in the
context of extrapolation tasks targeting MBHB, GB, and EMRIs signals, CBS-GPT
has demonstrated remarkable efficacy, with over 50% of the overlaps exceeding
0.99. These results illustrate CBS-GPT’s capability to learn from
instrumentally responsive gravitational waveforms. Figure 3 demonstrates the
prediction performance of each waveform under varying parameter conditions.
This demonstrates that the CBS-GPT model can learn waveform characteristics
with a wide variety of parameters.

####  III.1.1 Results of MBHB

During inference, as described in Section II.3 ,a consistent input-to-
prediction length ratio of 20:1 is applied. The distribution of overlap and
examples of predicted waveforms are shown in Figure 3(a) and Figure 4(a), with
mean and median overlap of 0.981 and 0.992, respectively. In particular,
because of the short merge-ringdown phase, a smaller time range is shown on
the left sub-figure in Figure 4(a). The overlap results reveal that CBS-GPT
can forecast the waveform of the merge-ringdown phase based on the inspiral
phase characteristics. CBS-GPT exhibits optimal inference performance when
considering a total mass of approximately $10^{6.5}M_{\odot}$ as shown in
Figure 3(a). Our observations reveal a notable pattern: CBS-GPT is similar to
the gravitational wave detector that has its sensitive frequency band, as the
MBHB waveform frequency is directly linked to the total mass of MBHB. This
phenomenon has also been observed in other signal sources.

In particular, the CBS-GPT model is sensitive to total mass, mass ratio, and
spin parameters. Here we use the effective spin parameter $\chi_{\text{eff}}$
to represent the spin parameter [56]:

$\chi_{\text{eff}}=\frac{S_{1}^{z}}{1+q}+\frac{qS_{2}^{z}}{1+q}.$ (13)

The overlap is higher in low total mass and high effective spin
$\chi_{\text{eff}}$ area as shown in Figure 3(a). As the total mass decreases,
the frequency of its waveform gradually increases. Since the TDI 2.0 transfer
functions in the high-frequency part is more complex[57, 58], the waveform is
also more complex. Consequently, the model’s performance experiences a slight
decrease. Nevertheless, as depicted in the left panel of Figure 4(a), CBS-GPT
demonstrates the ability to learn the predominant features of the waveform,
even in the presence of a complex waveform. While at lower frequencies (or
higher total mass), the accumulation period of the input waveform is
constrained. Comparing low and high-mass situations to those involving
intermediate masses, the performance of mid-frequency band prediction is the
best. Moreover, even in these less ideal circumstances, CBS-GPT still can
successfully recover a significant portion of the signals.

(a) MBHB waveforms

(b) EMRIs waveforms

(c) GB waveforms

Figure 4: CBS-GPT prediction results of MBHB, EMRIs, and GB. We set the
predicted starting point at time zero. To clarify, the blue line represents
the conjunction of the last part of the input waveform and target label, the
orange line is the predicted waveform, and the gray line is the difference
between the predicted and target waveform. The subfigure on the upper-left
represents the corresponding spectrum and difference in the frequency domain.

####  III.1.2 Results of Continous Waveform: EMRIs and GB

Similarly, CBS-GPT tests are performed on 10,000 EMRIs and GB waveforms with a
20:1 extrapolation rate. Overlap distribution of the EMRIs and GB is shown in
Figure 3(b) and Figure 3(c), respectively. Regarding GB, its mean and median
overlap both exceed 0.99. And mean and median overlap of EMRIs are equal to
0.912 and 0.997, respectively. While the mean overlap of EMRIs is slightly
lower, its median overlap aligns with that observed in the MBHB and GB
waveforms. Examples of predicted EMRIs and GB waveform are shown in Figure
4(b) and Figure 4(c).

Specifically, the overlap distribution of EMRIs significantly influenced by
the mass parameters and eccentricity parameters. As depicted in Figure 3(b),
when $e_{0}$ is less than $0.1$, the majority of overlaps remain below 0.9. As
the eccentricity increases, the waveform features become more complex in
waveform amplitude. Therefore, when the eccentricity is higher, the
corresponding overlap tends to decrease.

In contrast to MBHB and EMRIs signals, the GB signal presents a comparatively
straightforward, single-frequency waveform. In the GB dataset, the frequency
parameter has the greatest impact on the waveform. When the frequency is >
$10^{-3.5}$Hz, the overlap is basically higher than 0.9. The overlap analysis
for GB signals demonstrates the model’s frequency sensitivity, with a distinct
preference for learning the characteristics associated with intermediate
frequency signals. Meanwhile, as the extrapolation length increases, the
prediction accuracy for the three sources decreases.

###  III.2 Interpretability

From a probabilistic perspective, CBS-GPT’s prediction of the waveform can be
expressed by the conditional probability of predicting the next token as
$x_{i+1}$ by given the previous $i$ tokens[41]:

$L({x_{i+1}})=\sum_{k=1}^{i-1}\log P(x_{i}|x_{i-k},\ldots,x_{1};\Theta)\>,$
(14)

where $\Theta$ represents the model parameters. The probability can be
represented by the attention mechanism, which can be effectively visualized by
an attention map. The attention map allows us to visualize the reasoning
process and attention mechanism when predicting waveform, making it easier to
get knowledge to interpret gravitational wave data. Here, the inference
process is visualized using the attention map:

$A=\frac{1}{H}\sum_{i=1}^{H}\text{softmax}(\frac{Q_{i}K_{i}^{T}\cdot\mathrm{mask}}{\sqrt{d}})\>,$
(15)

where $H$ represents all attention heads of the last encoder block. In Figure
5, the vertical axis represents the model input waveform, and the horizontal
axis represents the predicted waveform.

When predicting continuous gravitational waveforms (EMRIs and GB), CBS-GPT
tends to pay attention to the input points of the same phase, and the
periodicity is shown in each row and column of the attention map (Figure 5(g)
\- 5(f)). The attention map exhibits a grid-like pattern that is closely
related to the phase of the waveform, with the scale of the grid expanding as
the frequency decreases. Hence, we perform a correlation calculation between
the waveform and the hidden parameters, with detailed information available in
Appendix A. In our sample, the correlation coefficient for continuous
waveforms exceeds 0.8. This demonstrates that the model can accurately match
the waveform’s frequency and phase information. This mode assists CBS-GPT in
successfully extrapolating input data, demonstrating its capacity to learn
information across various scales.

During the prediction of the merge-ringdown phase of MBHB waveforms, attention
primarily focuses on near-diagonal elements as showcased in Figure5(a) \-
5(c). In contrast to the continuous gravitational wave signals, the emphasis
is more centralized around the information proximate to the merger phase. The
amplitude reaches zero after the merge-ringdown, and the main focus of the
attention mechanism lies on the merging stage and the stage after the merge,
with relatively less attention directed towards information in the inspiral
phase. The focus of the attention map also changes under different total
masses, which demonstrates that CBS-GPT predicts waveforms by matching
waveform phases and frequency. Moreover, as Figure 4 demonstrates that CBS-GPT
accurately estimates the main frequency, the attention map also effectively
captures the phase information of the waveform.

(a) MBHB: $M_{tot}=10^{5.5}M_{\odot}$

(b) MBHB: $M_{tot}=10^{6.25}M_{\odot}$

(c) MBHB: $M_{tot}=10^{7}M_{\odot}$

(d) EMRIs: $e_{0}=0.01$

(e) EMRIs: $e_{0}=0.30$

(f) EMRIs: $e_{0}=0.40$

(g) GB: $f=10^{-2}$Hz

(h) GB: $f=10^{-3}$Hz

(i) GB: $f=10^{-3.5}$Hz

Figure 5: Attention maps of the last encoder layer. For clear presentation,
only part of the attention map is displayed. The blue lines on the left and
bottom panels represent the input waveforms, whose 1,001-st to 1,050-th tokens
are padded with zero-value during inference, and the orange line represents
the waveform predicted by CBS-GPT. The term Similarity in the title of each
figure denotes the correlation coefficient between the waveform and the hidden
parameters, which is significantly high.

##  IV Conclusion

In this paper, we introduce the CBS-GPT model, which consists of hybrid
embedding and encoder blocks. The CBS-GPT is applied to the prediction of GW
waveform after the response of the space-based gravitational wave detector and
TDI 2.0 combination. Three models are trained separately for MBHB, EMRIs, and
GB wave sources to predict the next 200-point waveform by feeding the previous
4,000-point waveform. In the case of complex instrument response, wide
parameter range, and large frequency band span, the average overlaps between
the predicted waveform and the target waveform of MBHB, EMRIs, and GB reach
0.981, 0.912, and 0.991, respectively. Intermediate-frequency waveforms
exhibit superior prediction performance compared to both high-frequency and
low-frequency counterparts.

At the same time, the visualization of the attention map illustrates that CBS-
GPT adjusts the attention map based on the varying frequencies of the
waveform. We discovered that the correlation coefficient between the hidden
parameters of the CBS-GPT model and the waveforms is relatively high,
indicating that the model can match the waveform’s phase information extremely
well. This allows our model to accurately match phase and frequency
information. Our results show that CBS-GPT has the ability to comprehend
detailed waveform properties, make predictions based on the input waveform
data, and exhibit remarkable learning performance over waveforms of varied
frequencies. We are confident that large models can be further used in
gravitational wave data processing in the future. For instance, the powerful
learning capabilities of CBS-GPT hold the potential for future applications in
tasks such as the gap completion of data from space-based GW detectors, GW
signal detection, and signal noise reduction.

###### Acknowledgements.

This research was supported by the Peng Cheng Laboratory and Peng Cheng Cloud-
Brain. This work was also supported in part by the National Key Research and
Development Program of China Grant No. 2021YFC2203001 and in part by the NSFC
(No. 11920101003 and No. 12021003). Z.C. was supported by the
“Interdisciplinary Research Funds of Beijing Normal University" and CAS
Project for Young Scientists in Basic Research YSBR-006.

##  Appendix A Correlation coefficient between waveform and hidden parameters

We introduce the correlation coefficient between the waveform and hidden
parameters to assess the level of correlation and demonstrate that the
attention map can capture the phase information of the waveform. Firstly, we
compute the mean value of each token of the patched input to get the sequence
$M$. Subsequently, the outer product of $M$ is computed, resulting in the
auto-correlation matrix. As the attention map is processed by masking and
normalized by the softmax function, we do a similar adjustment to the auto-
correlation matrix as Equation 16 shows.

$\displaystyle R_{\text{mask}}=\text{Mask}(M\otimes M-\text{min}(M\otimes M))$
(16) $\displaystyle
R_{\text{Norm}}=\text{RowNorm}\left(R_{\text{mask}}\right),$

where $\text{RowNorm}(\cdot)$ denotes the normalization of each row of the
matrix. To assess the correlation between the two matrices, we calculate the
Pearson correlation coefficient between the flattened attention map and
flattened $R_{\text{Norm}}$ (Equation 17):

$\displaystyle\rho_{A,R_{\text{Norm}}}$
$\displaystyle=\rho\left\\{\text{Flatten}(A),\text{Flatten}(R_{\text{Norm}})\right\\}$
$\displaystyle=\frac{n\sum_{1}^{n}{A_{F}}_{i}{R_{F}}_{i}-\sum_{1}^{n}{A_{F}}_{i}\sum_{1}^{n}{R_{F}}_{i}}{\sqrt{n\sum_{1}^{n}{A_{F}}_{i}^{2}-(\sum_{1}^{n}{A_{F}}_{i})^{2}}\sqrt{n\sum_{1}^{n}{R_{F}}_{i}^{2}-(\sum_{1}^{n}{R_{F}}_{i})^{2}}}$
(17)

where $\text{Flatten}(\cdot)$ denotes flattening the matrix into one
dimension, and $n$ represents the length after flattening. Finally,
$\rho_{A,R_{\text{Norm}}}$ is defined as the correlation coefficient between
waveform and hidden parameters.

## References

  * Abbott  _et al._ [2016a] B. P. Abbott, R. Abbott, T. D. Abbott, M. R. Abernathy, F. Acernese, K. Ackley, C. Adams, T. Adams, P. Addesso, R. X. Adhikari, et al., Physical Review Letters 116, 061102 (2016a).
  * Abbott  _et al._ [2016b] B. P. Abbott, R. Abbott, T. D. Abbott, M. R. Abernathy, F. Acernese, K. Ackley, C. Adams, T. Adams, P. Addesso, R. X. Adhikari, et al., Physical Review D 93, 122003 (2016b).
  * Schutz [1986] B. F. Schutz, Nature 323, 310 (1986).
  * Sathyaprakash and Schutz [2009] B. S. Sathyaprakash and B. F. Schutz, Living Reviews in Relativity 12, 2 (2009).
  * Klein  _et al._ [2016] A. Klein, E. Barausse, A. Sesana, A. Petiteau, E. Berti, S. Babak, J. Gair, S. Aoudia, I. Hinder, F. Ohme, et al., Physical Review D 93, 024003 (2016), arxiv:1511.05581 [astro-ph, physics:gr-qc].
  * Tamanini  _et al._ [2016] N. Tamanini, C. Caprini, E. Barausse, A. Sesana, A. Klein, and A. Petiteau, Journal of Cosmology and Astroparticle Physics 2016, 002 (2016), arxiv:1601.07112 [astro-ph, physics:gr-qc].
  * Bartolo  _et al._ [2016] N. Bartolo, C. Caprini, V. Domcke, D. G. Figueroa, J. Garcia-Bellido, M. C. Guzzetti, M. Liguori, S. Matarrese, M. Peloso, A. Petiteau, et al., Journal of Cosmology and Astroparticle Physics 2016, 026 (2016), arxiv:1610.06481 [astro-ph, physics:gr-qc, physics:hep-ph].
  * Caprini  _et al._ [2016] C. Caprini, M. Hindmarsh, S. Huber, T. Konstandin, J. Kozaczuk, G. Nardini, J. M. No, A. Petiteau, P. Schwaller, G. Servant, et al., Journal of Cosmology and Astroparticle Physics 2016, 001 (2016), arxiv:1512.06239 [astro-ph, physics:gr-qc, physics:hep-ph].
  * Babak  _et al._ [2017a] S. Babak, J. Gair, A. Sesana, E. Barausse, C. F. Sopuerta, C. P. L. Berry, E. Berti, P. Amaro-Seoane, A. Petiteau, and A. Klein, Physical Review D 95, 103012 (2017a), arxiv:1703.09722 [astro-ph, physics:gr-qc].
  * Arun  _et al._ [2022] K. G. Arun, E. Belgacem, R. Benkel, L. Bernard, E. Berti, G. Bertone, M. Besancon, D. Blas, C. G. Böhmer, R. Brito, et al., Living Reviews in Relativity 25, 4 (2022), arxiv:2205.01597 [gr-qc].
  * Collaboration  _et al._ [2021] T. L. S. Collaboration, the Virgo Collaboration, the KAGRA Collaboration, R. Abbott, T. D. Abbott, F. Acernese, K. Ackley, C. Adams, N. Adhikari, R. X. Adhikari, et al., “GWTC-3: Compact Binary Coalescences Observed by LIGO and Virgo During the Second Part of the Third Observing Run,” (2021), arxiv:2111.03606 [astro-ph, physics:gr-qc].
  * Agazie  _et al._ [2023] G. Agazie, A. Anumarlapudi, A. M. Archibald, Z. Arzoumanian, P. T. Baker, B. Bécsy, L. Blecha, A. Brazier, P. R. Brook, S. Burke-Spolaor, et al., The Astrophysical Journal Letters 951, L8 (2023).
  * Xu  _et al._ [2023] H. Xu, S. Chen, Y. Guo, J. Jiang, B. Wang, J. Xu, Z. Xue, R. N. Caballero, J. Yuan, Y. Xu, et al., Research in Astronomy and Astrophysics 23, 075024 (2023), arxiv:2306.16216 [astro-ph, physics:gr-qc].
  * EPTA Collaboration and InPTA Collaboration:  _et al._ [2023] EPTA Collaboration and InPTA Collaboration:, J. Antoniadis, P. Arumugam, S. Arumugam, S. Babak, M. Bagchi, A.-S. Bak Nielsen, C. G. Bassa, A. Bathula, A. Berthereau, et al., Astronomy & Astrophysics 678, A50 (2023).
  * Reardon  _et al._ [2023] D. J. Reardon, A. Zic, R. M. Shannon, G. B. Hobbs, M. Bailes, V. Di Marco, A. Kapur, A. F. Rogers, E. Thrane, J. Askew, et al., The Astrophysical Journal Letters 951, L6 (2023).
  * Bailes  _et al._ [2021] M. Bailes, B. K. Berger, P. R. Brady, M. Branchesi, K. Danzmann, M. Evans, K. Holley-Bockelmann, B. R. Iyer, T. Kajita, S. Katsanevas, et al., Nature Reviews Physics 3, 344 (2021).
  * Amaro-Seoane  _et al._ [2017] P. Amaro-Seoane, H. Audley, S. Babak, J. Baker, E. Barausse, P. Bender, E. Berti, P. Binetruy, M. Born, D. Bortoluzzi, et al., “Laser Interferometer Space Antenna,” (2017), arxiv:1702.00786 [astro-ph].
  * Hu and Wu [2017] W.-R. Hu and Y.-L. Wu, National Science Review 4, 685 (2017).
  * Ren  _et al._ [2023] Z. Ren, T. Zhao, Z. Cao, Z.-K. Guo, W.-B. Han, H.-B. Jin, and Y.-L. Wu, Frontiers of Physics 18, 64302 (2023).
  * Luo  _et al._ [2016] J. Luo, L.-S. Chen, H.-Z. Duan, Y.-G. Gong, S. Hu, J. Ji, Q. Liu, J. Mei, V. Milyukov, M. Sazhin, et al., Classical and Quantum Gravity 33, 035010 (2016), arxiv:1512.02076 [astro-ph, physics:gr-qc].
  * Matichard  _et al._ [2015] F. Matichard, B. Lantz, R. Mittleman, K. Mason, J. Kissel, B. Abbott, S. Biscans, J. McIver, R. Abbott, S. Abbott, et al., Classical and Quantum Gravity 32, 185003 (2015).
  * Wang  _et al._ [2020] H. Wang, S. Wu, Z. Cao, X. Liu, and J.-Y. Zhu, Physical Review D 101, 104003 (2020).
  * George and Huerta [2018a] D. George and E. Huerta, Physics Letters B 778, 64 (2018a).
  * George and Huerta [2018b] D. George and E. A. Huerta, Physical Review D 97, 044039 (2018b).
  * Gabbard  _et al._ [2018] H. Gabbard, M. Williams, F. Hayes, and C. Messenger, Physical Review Letters 120, 141103 (2018).
  * Badger  _et al._ [2023] C. Badger, K. Martinovic, A. Torres-Forné, M. Sakellariadou, and J. A. Font, Physical Review Letters 130, 091401 (2023).
  * Zhang  _et al._ [2022a] X.-T. Zhang, C. Messenger, N. Korsakova, M. L. Chan, Y.-M. Hu, and J.-d. Zhang, Physical Review D 105, 123027 (2022a), arxiv:2202.07158 [astro-ph, physics:gr-qc].
  * Dax  _et al._ [2021] M. Dax, S. R. Green, J. Gair, J. H. Macke, A. Buonanno, and B. Schölkopf, Physical Review Letters 127, 241103 (2021).
  * Gabbard  _et al._ [2022] H. Gabbard, C. Messenger, I. S. Heng, F. Tonolini, and R. Murray-Smith, Nature Physics 18, 112 (2022).
  * Wang  _et al._ [2022] H. Wang, Z. Cao, Y. Zhou, Z.-K. Guo, and Z. Ren, Big Data Mining and Analytics 5, 53 (2022).
  * Zhao  _et al._ [2023] T. Zhao, R. Lyu, H. Wang, Z. Cao, and Z. Ren, Communications Physics 6, 212 (2023).
  * Ormiston  _et al._ [2020] R. Ormiston, T. Nguyen, M. Coughlin, R. X. Adhikari, and E. Katsavounidis, Physical Review Research 2, 033066 (2020).
  * Wei and Huerta [2020] W. Wei and E. Huerta, Physics Letters B 800, 135081 (2020).
  * Colgan  _et al._ [2020] R. E. Colgan, K. R. Corley, Y. Lau, I. Bartos, J. N. Wright, Z. Márka, and S. Márka, Physical Review D 101, 102003 (2020).
  * Ren  _et al._ [2022] Z. Ren, H. Wang, Y. Zhou, Z.-K. Guo, and Z. Cao, “Intelligent noise suppression for gravitational wave observational data,” (2022), arxiv:2212.14283 [astro-ph, physics:gr-qc].
  * Lee  _et al._ [2021] J. Lee, S. H. Oh, K. Kim, G. Cho, J. J. Oh, E. J. Son, and H. M. Lee, Physical Review D 103, 123023 (2021), arxiv:2101.05685 [astro-ph, physics:gr-qc].
  * Khan  _et al._ [2022] A. Khan, E. A. Huerta, and H. Zheng, Physical Review D 105, 024024 (2022), arxiv:2110.06968 [astro-ph, physics:gr-qc].
  * Chua  _et al._ [2019] A. J. K. Chua, C. R. Galley, and M. Vallisneri, Physical Review Letters 122, 211101 (2019), arxiv:1811.05491 [astro-ph, physics:gr-qc, stat].
  * Devlin  _et al._ [2019] J. Devlin, M. Chang, K. Lee, and K. Toutanova, “BERT: pre-training of deep bidirectional transformers for language understanding,”  (2019).
  * Radford and Narasimhan [2018] A. Radford and K. Narasimhan,  (2018).
  * Radford  _et al._ [2019] A. Radford, J. Wu, R. Child, D. Luan, D. Amodei, and I. Sutskever,  (2019).
  * Brown  _et al._ [2020] T. B. Brown, B. Mann, N. Ryder, M. Subbiah, J. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell, et al.,  _Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020, virtual_ ,  (2020).
  * Bohé  _et al._ [2017] A. Bohé, L. Shao, A. Taracchini, A. Buonanno, S. Babak, I. W. Harry, I. Hinder, S. Ossokine, M. Pürrer, V. Raymond, et al., Physical Review D 95, 044028 (2017), arxiv:1611.03703 [gr-qc].
  * Babak  _et al._ [2017b] S. Babak, J. Gair, A. Sesana, E. Barausse, C. F. Sopuerta, C. P. L. Berry, E. Berti, P. Amaro-Seoane, A. Petiteau, and A. Klein, Physical Review D 95, 103012 (2017b).
  * Maselli  _et al._ [2022] A. Maselli, N. Franchini, L. Gualtieri, T. P. Sotiriou, S. Barsanti, and P. Pani, Nature Astronomy 6, 464 (2022).
  * Amaro-Seoane  _et al._ [2023] P. Amaro-Seoane, J. Andrews, M. Arca Sedda, A. Askar, Q. Baghi, R. Balasov, I. Bartos, S. S. Bavera, J. Bellovary, C. P. L. Berry, et al., Living Reviews in Relativity 26, 2 (2023).
  * Katz  _et al._ [2021] M. L. Katz, A. J. K. Chua, L. Speri, N. Warburton, and S. A. Hughes, Physical Review D 104, 064047 (2021).
  * Zhang  _et al._ [2022b] X.-H. Zhang, S.-D. Zhao, S. D. Mohanty, and Y.-X. Liu, Physical Review D 106, 102004 (2022b), arxiv:2206.12083 [gr-qc].
  * Robson  _et al._ [2019] T. Robson, N. Cornish, and C. Liu, Classical and Quantum Gravity 36, 105011 (2019), arxiv:1803.01944 [astro-ph, physics:gr-qc].
  * Tinto  _et al._ [2004] M. Tinto, F. B. Estabrook, and J. W. Armstrong, Physical Review D 69, 082001 (2004).
  * Tinto and Dhurandhar [2014] M. Tinto and S. V. Dhurandhar, Living Reviews in Relativity 17, 6 (2014).
  * Katz  _et al._ [2022] M. L. Katz, J.-B. Bayle, A. J. K. Chua, and M. Vallisneri, Physical Review D 106, 103001 (2022), arxiv:2204.06633 [astro-ph, physics:gr-qc].
  * Vaswani  _et al._ [2017] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, and I. Polosukhin, “Attention is all you need,”  (2017).
  * Liu  _et al._ [2021] Z. Liu, Y. Lin, Y. Cao, H. Hu, Y. Wei, Z. Zhang, S. Lin, and B. Guo, “Swin transformer: Hierarchical vision transformer using shifted windows,”  (2021).
  * Kingma and Ba [2017] D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” (2017), arxiv:1412.6980 [cs].
  * Barack  _et al._ [2019] L. Barack, V. Cardoso, S. Nissanke, T. P. Sotiriou, A. Askar, C. Belczynski, G. Bertone, E. Bon, D. Blas, R. Brito, et al., Classical and Quantum Gravity 36, 143001 (2019).
  * Larson  _et al._ [2000] S. L. Larson, W. A. Hiscock, and R. W. Hellings, Physical Review D 62, 062001 (2000), arxiv:gr-qc/9909080.
  * Babak  _et al._ [2021] S. Babak, M. Hewitson, and A. Petiteau, “LISA Sensitivity and SNR Calculations,” (2021), arxiv:2108.01167 [astro-ph, physics:gr-qc].