# Constraining primordial non-Gaussianity by combining next-generation galaxy
and 21 cm intensity mapping surveys

Sheean Jolicoeur1, Roy Maartens1,2,3, Simthembile Dlamini1

###### Abstract

Surveys of the matter distribution contain ‘fossil’ information on possible
non-Gaussianity that is generated in the primordial Universe. This primordial
signal survives only on the largest scales where cosmic variance is strongest.
By combining different surveys in a multi-tracer approach, we can suppress the
cosmic variance and significantly improve the precision on the level of
primordial non-Gaussianity. We consider a combination of an optical galaxy
survey, like the recently initiated DESI survey, together with a new and very
different type of survey, a 21 cm intensity mapping survey, like the upcoming
SKAO survey. A Fisher forecast of the precision on the local primordial non-
Gaussianity parameter $f_{\mathrm{NL}}$, shows that this multi-tracer
combination, together with non-overlap single-tracer information, can deliver
precision comparable to that from the CMB. Taking account of the largest
systematic, i.e. foreground contamination in intensity mapping, we find that
$\sigma(f_{\mathrm{NL}})\sim 3$.

##  1 Introduction

Constraining the local primordial non-Gaussianity (PNG) parameter
$f_{\mathrm{NL}}$ presents an important challenge for next-generation large-
scale structure surveys. Local-type PNG affects the power spectrum of dark
matter tracers by inducing a scale-dependent correction to the tracer bias [1,
2]. This correction is strongly suppressed on small to medium scales, but on
very large scales, it is $\propto f_{\mathrm{NL}}(H_{0}^{2}/k^{2})$. This
means that ultra-large survey volumes are required for high-precision
constraints.

The Planck survey of the cosmic microwave background (CMB) gives the current
state-of-the-art constraint $\sigma(f_{\mathrm{NL}})=5.1$ from the bispectrum
[3]. Future CMB surveys will lead to an improvement on the Planck precision,
but not by much and not sufficient to access $\sigma(f_{\mathrm{NL}})\lesssim
1$. This level of precision would enable us to rule out many single-field and
multi-field inflationary scenarios [4, 5, 6]. The hope is that this goal can
be achieved by using the extra modes in 3-dimensional surveys of the large-
scale structure (see e.g. [7, 8, 9, 10, 11, 12]). However, extracting
$f_{\mathrm{NL}}$ from these surveys faces formidable difficulties:

  * •

Observational systematics on very large scales [13, 14, 15, 16, 17, 18, 19].
For our simplified Fisher analysis, we neglect these systematics, except for
the foreground contamination of 21 cm intensity mapping (see below).

  * •

Astrophysical complications entailed in the modelling of tracer clustering, in
particular, the uncertainties in modelling the scale-dependent bias [20, 21].
We use a simplified model since a full treatment requires simulations, beyond
the scope of this paper.

  * •

A theoretical systematic arising from the neglect of lensing magnification and
other relativistic lightcone effects on the power spectrum, which can bias the
best-fit $f_{\mathrm{NL}}$ [22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33,
34, 35]. Although the bias on best-fit can be large, the error
$\sigma(f_{\mathrm{NL}})$ is not significantly affected and we therefore omit
these lightcone effects.

  * •

Cosmic (or sample) variance that dominates ultra-large scale measurements. We
deal with this problem via a multi-tracer approach – which also helps to
mitigate uncorrelated systematics.

Recent constraints on $f_{\mathrm{NL}}$ have used the BOSS galaxy [36] and
eBOSS quasar [37] samples, with the tightest constraint to date of
$\sigma(f_{\mathrm{NL}})=21$ from eBOSS [38]. The much larger volumes of
upcoming surveys should facilitate significant improvement on this precision.
But if we rely on individual surveys using the power spectrum of single
tracers, it turns out that $\sigma(f_{\mathrm{NL}})\lesssim 1$ is not
achievable – even if we neglect systematics and complexities in scale-
dependent bias [11]. The fundamental problem is cosmic variance.

A way to evade cosmic variance is the multi-tracer technique [39, 40, 41, 42,
43, 44, 45, 46, 47, 48]. Forecasts indicate that multi-tracing next-generation
surveys can surpass the CMB precision, and in some cases can achieve
$\sigma(f_{\mathrm{NL}})\lesssim 1$ [49, 50, 51, 47, 48, 52, 53, 54, 55, 56,
57, 58, 59, 60]. The performance of the multi-tracer improves when the
difference in tracer properties, especially the Gaussian clustering biases, is
significant. It also helps to suppress uncorrelated systematics.

In this paper, our aim is not to identify survey combinations that can achieve
$\sigma(f_{\mathrm{NL}})\lesssim 1$ – for this purpose, one typically requires
the very high number densities of photometric surveys [51, 47]. Instead, our
aim is to investigate the improvements over single-tracer constraints when
using a new type of spectroscopic large-scale structure survey – neutral
hydrogen (HI) intensity mapping of the 21 cm emission line – in combination
with spectroscopic galaxy surveys. Such a pair of spectroscopic samples has
very different clustering biases and systematics, which accentuates the
advantages of a multi-tracer analysis.

We use a simple Fisher forecast on mock surveys, which are based on next-
generation surveys that are starting to observe or close to starting. The two
surveys we have in mind are: the Dark Energy Spectroscopic Instrument (DESI),
with the Bright Galaxy Sample (BGS) and Emission Line Galaxy (ELG) samples
[61, 62], and the Square Kilometre Array Observatory (SKAO), with intensity
mapping surveys in Bands 1 and 2 [56, 63]. Galaxy number count surveys are
well established, but a cosmological HI intensity mapping survey has not yet
been implemented. However, pilot surveys on the SKAO precursor telescope
MeerKAT have already been used to:
(a) measure the cross-power between MeerKAT and WiggleZ galaxy surveys [64],
and
(b) start developing a pipeline for the planned SKAO surveys [65, 66].

We combine pairs of these surveys at low and high redshifts, using the Fisher
single-tracer constraints in the non-overlap volume and the multi-tracer
constraints in the overlap volume. The results show a reasonable improvement
for the low-redshift pair, and a significant improvement for the high-redshift
pair, compared to the standard single-tracer forecasts for
$\sigma(f_{\mathrm{NL}})$. The combination of all low- and high-redshift
surveys improve on Planck, with $\sigma(f_{\mathrm{NL}})\sim 3$. This
constraint is based on avoiding foreground contamination of HI intensity
mapping on very large scales. Foreground contamination of HI intensity mapping
has a strong effect on the precision: if we neglect this foreground
contamination, we find that $\sigma(f_{\mathrm{NL}})\sim 1$.

##  2 Multi-tracer power spectra

At first order in perturbations, the density (or temperature) contrast of a
tracer $A$ is related to the matter density contrast $\delta$ by

$\Delta_{A}(z,\bm{k})=\Big{[}b_{A}(z)+f(z)\mu^{2}\Big{]}\delta(z,\bm{k})\,,$
(2.1)

where $b_{A}$ is the (Gaussian) clustering bias,
$\mu=\bm{\hat{k}}\cdot\bm{{n}}$ is the projection along the line-of-sight
direction $\bm{n}$, and $f$ is the linear growth rate. We assume $b_{A}$ has a
known redshift evolution $\beta_{A}$, following [67], so that

$b_{A}(z)=b_{A0}\,\beta_{A}(z)\,,$ (2.2)

where the amplitude $b_{A0}$ is a free parameter. We highlight the point that
a multi-tracer approach reduces the impact of this assumption on
$\sigma(f_{\mathrm{NL}})$ [35].

The Fourier power spectra at tree-level are given by

$\big{\langle}\Delta_{A}(z,\bm{k})\Delta_{B}(z,\bm{k}^{\prime})\big{\rangle}=(2\pi)^{3}P_{AB}(z,\bm{k})\delta^{\mathrm{D}}(\bm{k}+\bm{k}^{\prime})\,,$
(2.3)

where the linear matter power spectrum (computed using CLASS [68]) is

$P_{AB}=\big{(}b_{A}+f\mu^{2}\big{)}\big{(}b_{B}+f\mu^{2}\big{)}P.$ (2.4)

In the presence of local PNG, the bias acquires a scale-dependent correction:

$\hat{b}_{A}(z,k)=b_{A}(z)+b_{A\phi}(z)\frac{f_{\rm
NL}}{\mathcal{M}(z,k)}\,,\quad\mathcal{M}(z,k)=\frac{2}{3\Omega_{m0}H_{0}^{2}}\frac{D(z)}{g_{\rm
in}}T(k)\,k^{2}\,.$ (2.5)

Here $T(k)$ is the matter transfer function (normalized to 1 on very large
scales), $D$ is the growth factor (normalized to 1 today) and $g_{\rm in}$ is
the initial growth suppression function, defined deep in the matter era. For
$\Lambda$CDM

$g_{\rm
in}=\frac{3}{5}\big{(}1+z\big{)}D\bigg{(}1+\frac{2f}{3\Omega_{m}}\bigg{)}\,.$
(2.6)

The non-Gaussian bias factor $b_{A\phi}$ is halo-dependent and should be
determined with the aid of simulations [69, 70, 20]. A simplified model
reduces it to

$b_{A\phi}(z)=2\delta_{\rm c}\big{[}b_{A}(z)-p_{A}\big{]}\,,$ (2.7)

where the critical collapse density is given by $\delta_{\rm c}=1.686\,$ and
$p_{A}$ are halo-dependent constants. In the simplest (universal) halo model,
$p_{A}=1$ for all tracers $A$. But this model is not consistent with
simulations [69, 70, 20]. An improved (but still over-simplified) model allows
the constant $p_{A}$ to vary with tracer. For galaxies chosen by stellar mass,
simulations indicate that the rough approximation [69]

$p_{g}\approx 0.55\,,$ (2.8)

is an improvement, and we use this for the DESI-like samples. For HI intensity
mapping, we use the approximation [70]

$p_{H}\approx 1.25\,.$ (2.9)

The $f_{\mathrm{NL}}$ terms in the power spectra are of order
$H_{0}^{2}/k^{2}$ on ultra-large scales, $k\lesssim k_{\rm eq}$, where
$T\approx 1$. Local PNG therefore dominates the power on ultra-large scales –
but these are also the scales where cosmic variance is largest. We deal with
the cosmic variance via a multi-tracer analysis, which includes the
information from all auto- and cross-power spectra of the two (or more)
tracers (see section 4).

##  3 Modelling the surveys

We consider two mock spectroscopic surveys:

$A=g$: galaxy survey, similar to DESI surveys [61, 62];

$A=H$: 21 cm HI intensity mapping (IM) survey, similar to SKAO surveys [56,
71, 63].

In both cases, we have a low- and a high-redshift survey. The sky and redshift
coverage of the individual and overlapping surveys are given in Table 1.

Table 1: Sky area and redshift range of surveys. Survey | Sample |  $\Omega_{\mathrm{sky}}$ |  $t_{\mathrm{tot}}$ |  redshift
---|---|---|---|---
|  | $\big{[}10^{3}\,\mathrm{deg}^{2}\big{]}$ | $\big{[}10^{3}\,\mathrm{hr}\big{]}$ |  range
$g$ (DESI-like) | BGS | 14 | - | 0.00–0.50
| ELG | 14 | - | 0.60–1.70
$H$ (SKAO-like) | Band 2 | 20 | 10 | 0.10–0.58
| Band 1 | 20 | 10 | 0.35–3.05
$g\times H$ (low $z$) | BGS $\times$ Band 2 | 10 | 5 | 0.10–0.50
$g\times H$ (high $z$) | ELG $\times$ Band 1 | 10 | 5 | 0.60–1.70
Figure 1: Background comoving galaxy number densities (red, left-hand
$y$-axis) and HI IM temperature (blue, right-hand $y$-axis).

The one-parameter model (2.2) for the Gaussian clustering bias of the galaxy
surveys follows [29]

$b_{g}(z)=\frac{b_{g0}}{D(z)}\,.$ (3.1)

Fiducial values are $b_{g0}=1.34$ for the Bright Galaxy Sample (BGS) and
$b_{g0}=0.84$ for the Emission Line Galaxy (ELG) sample. For the HI IM surveys
we use a fit based on [72, 73]:

$b_{H}(z)={b_{H0}}{\big{(}1+0.823\,z-0.0546\,z^{2}\big{)}}\quad\mbox{with
fiducial value}~{}b_{H0}=0.842\,.$ (3.2)

The background comoving number density $\bar{n}_{g}$ for the BGS and ELG
samples is modelled following [29], which leads to the smoothed curves shown
in Figure 1. For HI IM, the background brightness temperature is modelled via
the fit given in [65, 74]:

$\bar{T}_{H}(z)=0.0559+0.2324\,z-0.0241\,z^{2}~{}~{}\mathrm{mK}\,,$ (3.3)

which is also shown in Figure 1.

###  3.1 Noise

In galaxy surveys, the shot noise (assumed to be Poissonian) is

$P^{\rm shot}_{gg}(z)=\frac{1}{\bar{n}_{g}(z)}\,.$ (3.4)

Then the total signal that we measure for the galaxy auto-power spectrum is

$\tilde{P}_{gg}(z,k,\mu)=P_{gg}(z,k,\mu)+P^{\rm shot}_{gg}(z)\,,$ (3.5)

where $P_{gg}$ is given by (2.4).

In IM surveys, there is shot noise, but also thermal (or instrumental) noise.
The Poissonian shot noise (see [75] for non-Poissonian corrections) is derived
in a halo-model framework and given by [76, 72],

$P_{H\\!H}^{\rm
shot}(z)=\frac{1}{\bar{n}_{H}(z)}=\frac{1}{\bar{\rho}_{H}^{\,2}(z)}\int\mathrm{d}M\,{\cal
N}_{h}(M,z)\,M_{H}^{2}(M,z)\,,$ (3.6)

where $\bar{n}_{H}$ is the effective average comoving number density of HI,
$\bar{\rho}_{H}$ is the average comoving HI density, ${\cal N}_{h}$ is the
halo mass function (average comoving halo number density per mass) and $M_{H}$
is the average HI mass in a halo of mass $M$. However, on the linear scales
that we consider, the shot noise is much smaller than the thermal noise and
can be safely neglected [76, 72].

The thermal noise in HI IM depends on the sky temperature in the radio band,
the survey specifications and the array configuration (single-dish or
interferometer). For the single-dish mode of SKAO-like surveys, the thermal
noise power spectrum is [77, 78, 79]:

$P_{H\\!H}^{\mathrm{therm}}(z)=\frac{\Omega_{\mathrm{sky}}}{2\nu_{21}t_{\mathrm{tot}}}\,\frac{(1+z)r(z)^{2}}{\mathcal{H}(z)}\,\left[\frac{T_{\mathrm{sys}}(z)}{\bar{T}_{H}(z)}\right]^{2}\,\frac{1}{N_{\rm
d}}\,,$ (3.7)

where $\nu_{21}=1420\,\mathrm{MHz}$ is the rest-frame frequency of the 21 cm
emission, $t_{\mathrm{tot}}$ is the total observing time, and the number of
dishes is $N_{\mathrm{d}}=197$ (with dish diameter $D_{\mathrm{d}}=15\,$m).
The system temperature is modelled as [80]:

$T_{\mathrm{sys}}(z)=T_{\rm d}(z)+T_{\rm sky}(z)=T_{\rm
d}(z)+2.7+25\bigg{[}\frac{400\,\mathrm{MHz}}{\nu_{21}}(1+z)\bigg{]}^{2.75}~{}\mathrm{K},$
(3.8)

where $T_{\rm d}$ is the dish receiver temperature (see [71]). The total
signal is then

$\tilde{P}_{H\\!H}(z,k,\mu)=P_{H\\!H}(z,k,\mu)+P^{\rm
shot}_{H\\!H}(z)+P_{H\\!H}^{\mathrm{therm}}(z)\approx
P_{H\\!H}(z,k,\mu)+P_{H\\!H}^{\mathrm{therm}}(z)\,.$ (3.9)

The noise for all surveys is shown in Figure 2.

In the case of the cross-power spectrum $P_{gH}$, cross-shot noise arises if
the halos that host galaxies and HI overlap. Following the arguments given in
[71, 81], we assume that the cross-shot noise may be neglected. The total
cross-power signal is then

$\tilde{P}_{gH}=P_{gH}~{}~{}\mbox{with}~{}~{}P_{gH}^{\rm shot}\approx 0\,.$
(3.10)

Figure 2:  Shot noise for galaxy surveys (_left_) and thermal noise for IM
surveys (_right_).

###  3.2 Intensity mapping beam and foregrounds

HI IM surveys like the SKAO surveys have poor angular resolution, which
results in power loss on small transverse scales, i.e. for large
$k_{\perp}=(1-\mu^{2})^{1/2}k$. This effect is typically modelled by a
Gaussian beam factor [77]:

${\cal D}_{\rm b}(z,k,\mu)=\exp\left[-{(1-\mu^{2})k^{2}r(z)^{2}\theta_{\rm
b}(z)^{2}\over 16\ln 2}\right]\quad\mbox{with}\quad\theta_{\rm
b}(z)=1.22\,{\lambda_{21}(1+z)\over D_{\rm d}}\,.$ (3.11)

HI IM surveys are also contaminated by foregrounds that are much larger than
the HI signal. Since these foregrounds are spectrally smooth, they can be
separated from the non-smooth signal on small to medium scales. However, on
very large radial scales, i.e. for small $k_{\|}=\mu k$, the signal becomes
smoother and therefore the separation fails. A comprehensive treatment
includes simulations of foreground cleaning of the HI signal (e.g. [15, 18]).
For a simplified Fisher forecast we can instead use a foreground avoidance
approach, by excising the regions of Fourier space where the foregrounds are
significant. This means removing large radial scales, which can be modelled
via an exponential suppression factor [82, 73]:

${\cal D}_{\rm fg}(k,\mu)={1-\exp\bigg{[}-\Big{(}\frac{\mu k}{k_{\rm
fg}}\Big{)}^{2}\bigg{]}\,.}$ (3.12)

We choose a typically used value:

$k_{\rm fg}=0.01\,h\,{\rm Mpc}^{-1}\,.$ (3.13)

Figure 3 illustrates the consequences of foreground avoidance for the
monopoles of the HI power spectra, $P_{AH,0}$. It is clear that large-scale
(small $k$) power in $P_{AH,0}$ is lost and that the effect of local PNG on
these large scales is suppressed by foreground avoidance. Note that $f_{\rm
NL}$ reduces the HI power on very large scales at $z=0.3$, since $b_{H}<1.25$,
which leads to $b_{H\phi}<0$ in (2.7). Also note that the beam effect on
$P_{AH,0}$, which tends to suppress small scale (large $k$) modes, is
negligible at the low redshift, but becomes apparent at higher redshift.

In summary, the effects of the radio telescope beam and radio foreground
avoidance lead to the following modifications of the power spectra $P_{AH}$:

$\displaystyle P_{H\\!H}(z,{k},\mu)$ $\displaystyle\rightarrow$
$\displaystyle{\cal D}_{\rm fg}(k,\mu)\,{{\cal D}_{\rm
b}(z,k,\mu)}^{2}\,P_{H\\!H}(z,{k},\mu)\,,$ (3.14) $\displaystyle
P_{gH}(z,{k},\mu)$ $\displaystyle\rightarrow$ $\displaystyle{\cal D}_{\rm
fg}(k,\mu)\,{\cal D}_{\rm b}(z,k,\mu)\,P_{gH}(z,{k},\mu)\,.$ (3.15)

Figure 3: The monopole power spectra at $z=0.3$ (_left_) and $z=1.0$
(_right_).

##  4 Fisher forecast

For a multi-tracer combination of two dark matter tracers $g$ and $H$, the
data vector of power spectra is

$\bm{P}=\big{(}P_{gg}\,,\,P_{gH}\,,\,P_{H\\!H}\big{)}\,.$ (4.1)

We exclude the noise from the data vector, since the noise is independent of
the cosmological and nuisance parameters. (The noise appears in the
covariance, as given below.) The standard cosmological parameters are measured
on medium to small scales and are effectively independent of the ultra-large
scale parameter $f_{\mathrm{NL}}$. Fixing their values could bias the best-fit
value of $f_{\mathrm{NL}}$ but is unlikely to have any significant impact on
$\sigma(f_{\mathrm{NL}})$. We include in the Fisher analysis the cosmological
parameters that directly affect the large-scale amplitude ($\sigma_{8,0}$) and
shape ($n_{s}$) of the power spectrum. We therefore consider the following
cosmological and nuisance parameters:

$\vartheta_{\alpha}=\big{(}\sigma_{8,0},\,n_{s},\,f_{\mathrm{NL}};\,b_{g0},\,b_{H0}\big{)}\,.$
(4.2)

Here we assume that the degeneracies between $b_{A}$ and $\sigma_{8,0}$, and
between $\bar{T}_{H}$ and $\sigma_{8,0}$, have been broken by other surveys
that focus on medium to small scales.

The covariance for the multi-tracer power spectra is given by [83, 84]:

$\mathrm{Cov}(\bm{P},\bm{P})=\frac{k_{\rm f}^{3}}{4\pi k^{2}\Delta
k}\,\frac{2}{\Delta\mu}\,\begin{pmatrix}\tilde{P}_{gg}^{2}&&\tilde{P}_{gg}\tilde{P}_{gH}&&\tilde{P}_{gH}^{2}\\\
&&&&\\\
\tilde{P}_{gg}\tilde{P}_{gH}&&\frac{1}{2}\big{[}\tilde{P}_{gg}\tilde{P}_{H\\!H}+\tilde{P}_{gH}^{2}\big{]}&&\tilde{P}_{H\\!H}\tilde{P}_{gH}\\\
&&&&\\\
\tilde{P}_{gH}^{2}&&\tilde{P}_{H\\!H}\tilde{P}_{gH}&&\tilde{P}_{H\\!H}^{2}\end{pmatrix}\,.$
(4.3)

Here $\Delta k$ and $\Delta\mu$ are the bin-widths for $k$ and $\mu$ and
$k_{\mathrm{f}}$ is the fundamental mode, corresponding to the longest
wavelength, which is determined by the comoving survey volume of the redshift
bin centred at $z$:

$V(z)=\frac{\Omega_{\mathrm{sky}}}{3}\bigg{[}r\Big{(}z+\frac{\Delta
z}{2}\Big{)}^{3}-r\Big{(}z-\frac{\Delta
z}{2}\Big{)}^{3}\bigg{]}=\bigg{[}\frac{2\pi}{k_{\mathrm{f}}(z)}\bigg{]}^{3}\,.$
(4.4)

Then the multi-tracer Fisher matrix in a redshift bin is

$F_{\alpha\beta}^{\bm{P}}=\sum_{\mu=-1}^{+1}\,\sum_{k=k_{\mathrm{min}}}^{k_{\mathrm{max}}}\,\partial_{\alpha}\,\bm{P}\cdot\mathrm{Cov}(\bm{P},\bm{P})^{-1}\cdot\partial_{\beta}\,\bm{P}^{\mathrm{T}}\,,$
(4.5)

where $\partial_{\alpha}=\partial\,/\,\partial\vartheta_{\alpha}$. We choose
the bin-widths and $k_{\mathrm{min}}$ following [85, 86, 87]:

$\Delta\mu=0.04\,,~{}~{}\Delta
k=k_{\mathrm{f}}\,,~{}~{}k_{\mathrm{min}}=k_{\mathrm{f}}\,.$ (4.6)

The smallest scale (largest $k$) is chosen to ensure that linear perturbations
remain accurate, since we do not require information from nonlinear scales:

$k_{\mathrm{max}}=0.08(1+z)^{2/(2+n_{s})}\,h\,\text{Mpc}^{-1}\,.$ (4.7)

The galaxy and HI IM surveys do not have the same sky area and redshift ranges
(see Table 1). The overlap redshift ranges are obvious; for the sky areas, we
assume a nominal overlap area of 10,000 deg2. The multi-tracer applies in the
overlap redshift range and overlap sky area. However, we can add the
independent Fisher matrices obtained in the non-overlapping regions of the two
individual surveys to the multi-tracer Fisher matrix in the overlapping region
[71]. In detail, the non-overlap region is

  * •

the non-overlap sky area of each survey, across the full redshift range of
each survey;

  * •

the overlap sky area, across the non-overlap parts of the redshift ranges of
each survey.

Then the full Fisher matrix (denoted by $g\otimes H$) is

$F_{\alpha\beta}^{g\otimes
H}=F_{\alpha\beta}^{\bm{P}}(\text{overlap})+F_{\alpha\beta}^{g}(\text{non-
overlap})+F_{\alpha\beta}^{H}(\text{non-overlap}).$ (4.8)

Figure 4 shows the 1$\sigma$ error contours computed from this Fisher matrix
after marginalising over the bias parameters in (4.2). We use the fiducial
values $\sigma_{8,0}=0.8102$, $n_{s}=0.9665$ and $f_{\mathrm{NL}}=0$, keeping
other cosmological parameters fixed to their Planck 2018 best-fit values [88].
Table 2 lists the marginalised $\sigma(f_{\mathrm{NL}})$ constraints,
including the improvements (in parentheses) that follow when we ignore the HI
IM foregrounds, i.e. when $k_{\rm fg}=0$.

Figure 4: 1$\sigma$ contours, including foreground avoidance in HI intensity
mapping and marginalising over bias parameters. Table 2: Cumulative
marginalised error $\sigma(f_{\mathrm{NL}})$ for: the single-tracer surveys;
the low-$z$ and high-$z$ multi-tracer pairs – including single-tracer
information from the non-overlap volumes as in (4.8); and the sum of the
multi-tracer pairs.
Values in parenthesis correspond to ignoring foregrounds, i.e., with $k_{\mathrm{fg}}=0$. Survey | $\sigma(f_{\mathrm{NL}})$
---|---
BGS | 20.8
ELG | 3.5
Band 2 | 96.3 (57.7)
Band 1 | 4.3 (1.3)
BGS $\otimes$ Band 2 | 9.2 (2.3)
ELG $\otimes$ Band 1 | 3.2 (1.3)
BGS $\otimes$ Band 2 + ELG $\otimes$ Band 1 | 2.9 (1.1)

##  5 Conclusion

We applied a simplified Fisher forecast analysis in order to gain insights
into the improvements on $\sigma(f_{\mathrm{NL}})$ from a new multi-tracer
combination of large-scale structure surveys that will be made possible by the
upcoming HI intensity mapping surveys with the SKAO. For the spectroscopic
galaxy samples, we used mock surveys based on DESI. This allowed for multi-
tracer pairs at low and high redshifts, in order to see the level of improved
precision from the higher volume at high redshifts.

We included all available Fisher information from these mock surveys, using
the multi-tracer Fisher matrix on the overlap volume (sky area and redshift
range) and the single-tracer Fisher matrices on the non-overlap volumes, as in
(4.8). Our results for $\sigma(f_{\mathrm{NL}})$ are summarised in Table 2,
which gives the single-tracer results for the 4 samples, the full multi-tracer
results from (4.8) for the low- and high-redshift pairs, and finally the
results from the Fisher sum of these pairs. The impact of foreground avoidance
can be seen from the results (in parenthesis) when foregrounds are ignored,
corresponding to $k_{\rm fg}=0$ in (3.12).

Figure 4 shows the $1\sigma$ contours for the 3 cosmological parameters
($\sigma_{8,\,0}$, $n_{s}$, $f_{\mathrm{NL}}$), marginalising over the 2 bias
nuisance parameters ($b_{g0}$, $b_{H0}$).

As expected, the foreground avoidance severely degrades the constraining power
of the HI intensity surveys. Without foregrounds, the high-redshift Band 1
survey would perform as well as the multi-tracer combination with the ELG
survey: $\sigma(f_{\mathrm{NL}})=1.3$. The reduced precision on $f_{\rm NL}$
for single-tracer HI intensity constraints is consistent with the findings of
[15], where foreground cleaning is simulated. However, using the multi-tracer
with a spectroscopic galaxy survey helps to reduce the effect of foregrounds
on $\sigma(f_{\mathrm{NL}})$, as shown by Table 2.

The multi-tracer analysis is also expected to mitigate other systematics in
the galaxy and HI intensity samples. For example, HI intensity mapping is also
affected by radio frequency interference, receiver noise, calibration errors
and polarisation leakage. Incorporating these and the galaxy systematics
requires extensive simulations, beyond the scope of our paper.

Acknowledgements
We thank Dionysis Karagiannis for very helpful comments. We are supported by
the South African Radio Astronomy Observatory (SARAO) and the National
Research Foundation (Grant No. 75415).

## References

  * [1] S. Matarrese and L. Verde, The effect of primordial non-Gaussianity on halo bias, Astrophys. J. Lett. 677 (2008) L77–L80, [arXiv:0801.4826].
  * [2] N. Dalal, O. Dore, D. Huterer, and A. Shirokov, The imprints of primordial non-gaussianities on large-scale structure: scale dependent bias and abundance of virialized objects, Phys. Rev. D77 (2008) 123514, [arXiv:0710.4560].
  * [3] Planck Collaboration, Y. Akrami et al., Planck 2018 results. IX. Constraints on primordial non-Gaussianity, Astron. Astrophys. 641 (2020) A9, [arXiv:1905.05697].
  * [4] M. Alvarez et al., Testing Inflation with Large Scale Structure: Connecting Hopes with Reality, arXiv:1412.4671.
  * [5] R. de Putter, J. Gleyzes, and O. Doré, Next non-Gaussianity frontier: What can a measurement with $\sigma$(fNL)$\lesssim$1 tell us about multifield inflation?, Phys. Rev. D 95 (2017), no. 12 123507, [arXiv:1612.05248].
  * [6] P. D. Meerburg et al., Primordial Non-Gaussianity, arXiv:1903.04409.
  * [7] T. Giannantonio, C. Porciani, J. Carron, A. Amara, and A. Pillepich, Constraining primordial non-Gaussianity with future galaxy surveys, Mon. Not. Roy. Astron. Soc. 422 (2012) 2854–2877, [arXiv:1109.0958].
  * [8] S. Camera, M. G. Santos, P. G. Ferreira, and L. Ferramacho, Cosmology on Ultra-Large Scales with HI Intensity Mapping: Limits on Primordial non-Gaussianity, Phys. Rev. Lett. 111 (2013) 171302, [arXiv:1305.6928].
  * [9] A. Font-Ribera, P. McDonald, N. Mostek, B. A. Reid, H.-J. Seo, and A. Slosar, DESI and other dark energy experiments in the era of neutrino mass measurements, JCAP 05 (2014) 023, [arXiv:1308.4164].
  * [10] S. Camera, M. G. Santos, and R. Maartens, Probing primordial non-Gaussianity with SKA galaxy redshift surveys: a fully relativistic analysis, Mon. Not. Roy. Astron. Soc. 448 (2015), no. 2 1035–1043, [arXiv:1409.8286].
  * [11] D. Alonso, P. Bull, P. G. Ferreira, R. Maartens, and M. Santos, Ultra large-scale cosmology in next-generation experiments with single tracers, Astrophys. J. 814 (2015), no. 2 145, [arXiv:1505.07596].
  * [12] A. Raccanelli, F. Montanari, D. Bertacca, O. Doré, and R. Durrer, Cosmological Measurements with General Relativistic Galaxy Correlations, JCAP 05 (2016) 009, [arXiv:1505.06179].
  * [13] B. Leistedt, H. V. Peiris, and N. Roth, Constraints on Primordial Non-Gaussianity from 800 000 Photometric Quasars, Phys. Rev. Lett. 113 (2014), no. 22 221301, [arXiv:1405.4315].
  * [14] J. Fonseca and M. Liguori, Measuring ultralarge scale effects in the presence of 21 cm intensity mapping foregrounds, Mon. Not. Roy. Astron. Soc. 504 (2021), no. 1 267–279, [arXiv:2011.11510].
  * [15] S. Cunnington, S. Camera, and A. Pourtsidou, The degeneracy between primordial non-Gaussianity and foregrounds in 21 cm intensity mapping experiments, Mon. Not. Roy. Astron. Soc. 499 (2020), no. 3 4054–4067, [arXiv:2007.12126].
  * [16] M. Rezaie et al., Primordial non-Gaussianity from the completed SDSS-IV extended Baryon Oscillation Spectroscopic Survey – I: Catalogue preparation and systematic mitigation, Mon. Not. Roy. Astron. Soc. 506 (2021), no. 3 3439–3454, [arXiv:2106.13724].
  * [17] R. H. Liu and P. C. Breysse, Coupling parsec and gigaparsec scales: Primordial non-Gaussianity with multitracer intensity mapping, Phys. Rev. D 103 (2021), no. 6 063520, [arXiv:2002.10483].
  * [18] M. Spinelli, I. P. Carucci, S. Cunnington, S. E. Harper, M. O. Irfan, J. Fonseca, A. Pourtsidou, and L. Wolz, SKAO HI intensity mapping: blind foreground subtraction challenge, Mon. Not. Roy. Astron. Soc. 509 (2021), no. 2 2048–2074, [arXiv:2107.10814].
  * [19] W. Riquelme et al., Primordial non-Gaussianity with Angular correlation function: Integral constraint and validation for DES, arXiv:2209.07187.
  * [20] A. Barreira, Can we actually constrain fNL using the scale-dependent bias effect? An illustration of the impact of galaxy bias uncertainties using the BOSS DR12 galaxy power spectrum, JCAP 11 (2022) 013, [arXiv:2205.05673].
  * [21] T. Lazeyras, A. Barreira, F. Schmidt, and V. Desjacques, Assembly bias in the local PNG halo bias and its implication for $f_{\rm NL}$ constraints, arXiv:2209.07251.
  * [22] T. Namikawa, T. Okamura, and A. Taruya, Magnification effect on the detection of primordial non-Gaussianity from photometric surveys, Phys. Rev. D 83 (2011) 123514, [arXiv:1103.1118].
  * [23] M. Bruni, R. Crittenden, K. Koyama, R. Maartens, C. Pitrou, and D. Wands, Disentangling non-Gaussianity, bias and GR effects in the galaxy distribution, Phys. Rev. D 85 (2012) 041301, [arXiv:1106.3999].
  * [24] D. Jeong, F. Schmidt, and C. M. Hirata, Large-scale clustering of galaxies in general relativity, Phys. Rev. D85 (2012) 023504, [arXiv:1107.5427].
  * [25] S. Camera, R. Maartens, and M. G. Santos, Einstein’s legacy in galaxy surveys, Mon. Not. Roy. Astron. Soc. 451 (2015), no. 1 L80–L84, [arXiv:1412.4781].
  * [26] A. Kehagias, A. M. Dizgah, J. Norena, H. Perrier, and A. Riotto, A Consistency Relation for the Observed Galaxy Bispectrum and the Local non-Gaussianity from Relativistic Corrections, JCAP 1508 (2015), no. 08 018, [arXiv:1503.04467].
  * [27] C. S. Lorenz, D. Alonso, and P. G. Ferreira, Impact of relativistic effects on cosmological parameter estimation, Phys. Rev. D 97 (2018), no. 2 023537, [arXiv:1710.02477].
  * [28] D. Contreras, M. C. Johnson, and J. B. Mertens, Towards detection of relativistic effects in galaxy number counts using kSZ Tomography, JCAP 10 (2019) 024, [arXiv:1904.10033].
  * [29] G. Jelic-Cizmek, F. Lepori, C. Bonvin, and R. Durrer, On the importance of lensing for galaxy clustering in photometric and spectroscopic surveys, JCAP 04 (2021) 055, [arXiv:2004.12981].
  * [30] J. L. Bernal, N. Bellomo, A. Raccanelli, and L. Verde, Beware of commonly used approximations. Part II. Estimating systematic biases in the best-fit parameters, JCAP 10 (2020) 017, [arXiv:2005.09666].
  * [31] M. S. Wang, F. Beutler, and D. Bacon, Impact of Relativistic Effects on the Primordial Non-Gaussianity Signature in the Large-Scale Clustering of Quasars, Mon. Not. Roy. Astron. Soc. 499 (2020), no. 2 2598–2607, [arXiv:2007.01802].
  * [32] R. Maartens, S. Jolicoeur, O. Umeh, E. M. De Weerd, and C. Clarkson, Local primordial non-Gaussianity in the relativistic galaxy bispectrum, JCAP 04 (2021) 013, [arXiv:2011.13660].
  * [33] E. Castorina and E. di Dio, The observed galaxy power spectrum in General Relativity, JCAP 01 (2022), no. 01 061, [arXiv:2106.08857].
  * [34] M. Martinelli, R. Dalal, F. Majidi, Y. Akrami, S. Camera, and E. Sellentin, Ultralarge-scale approximations and galaxy clustering: Debiasing constraints on cosmological parameters, Mon. Not. Roy. Astron. Soc. 510 (2022), no. 2 1964–1977, [arXiv:2106.15604].
  * [35] J.-A. Viljoen, J. Fonseca, and R. Maartens, Multi-wavelength spectroscopic probes: biases from neglecting light-cone effects, JCAP 12 (2021), no. 12 004, [arXiv:2108.05746].
  * [36] G. Cabass, M. M. Ivanov, O. H. E. Philcox, M. Simonović, and M. Zaldarriaga, Constraints on multifield inflation from the BOSS galaxy survey, Phys. Rev. D 106 (2022), no. 4 043506, [arXiv:2204.01781].
  * [37] E. Castorina et al., Redshift-weighted constraints on primordial non-Gaussianity from the clustering of the eBOSS DR14 quasars in Fourier space, JCAP 09 (2019) 010, [arXiv:1904.08859].
  * [38] E.-M. Mueller et al., The clustering of galaxies in the completed SDSS-IV extended Baryon Oscillation Spectroscopic Survey: Primordial non-Gaussianity in Fourier Space, arXiv:2106.13725.
  * [39] U. Seljak, Extracting primordial non-gaussianity without cosmic variance, Phys. Rev. Lett. 102 (2009) 021302, [arXiv:0807.1770].
  * [40] P. McDonald and U. Seljak, How to measure redshift-space distortions without sample variance, JCAP 10 (2009) 007, [arXiv:0810.0323].
  * [41] G. M. Bernstein and Y.-C. Cai, Cosmology without cosmic variance, Mon. Not. Roy. Astron. Soc. 416 (2011) 3009, [arXiv:1104.3862].
  * [42] N. Hamaus, U. Seljak, and V. Desjacques, Optimal Constraints on Local Primordial Non-Gaussianity from the Two-Point Statistics of Large-Scale Structure, Phys. Rev. D 84 (2011) 083509, [arXiv:1104.2321].
  * [43] L. R. Abramo and K. E. Leonard, Why multi-tracer surveys beat cosmic variance, Mon. Not. Roy. Astron. Soc. 432 (2013) 318, [arXiv:1302.5444].
  * [44] L. R. Abramo, L. F. Secco, and A. Loureiro, Fourier analysis of multitracer cosmological surveys, Mon. Not. Roy. Astron. Soc. 455 (2016), no. 4 3871–3889, [arXiv:1505.04106].
  * [45] A. Witzemann, D. Alonso, J. Fonseca, and M. G. Santos, Simulated multitracer analyses with HI intensity mapping, Mon. Not. Roy. Astron. Soc. 485 (2019), no. 4 5519–5531, [arXiv:1808.03093].
  * [46] L. R. Abramo, J. a. V. Dinarte Ferri, I. L. Tashiro, and A. Loureiro, Fisher matrix for the angular power spectrum of multi-tracer galaxy surveys, JCAP 08 (2022) 073, [arXiv:2204.05057].
  * [47] D. Alonso and P. G. Ferreira, Constraining ultralarge-scale cosmology with multiple tracers in optical and radio surveys, Phys. Rev. D 92 (2015), no. 6 063525, [arXiv:1507.03550].
  * [48] J. Fonseca, S. Camera, M. Santos, and R. Maartens, Hunting down horizon-scale effects with multi-wavelength surveys, Astrophys. J. 812 (2015), no. 2 L22, [arXiv:1507.04605].
  * [49] L. D. Ferramacho, M. G. Santos, M. J. Jarvis, and S. Camera, Radio galaxy populations and the multitracer technique: pushing the limits on primordial non-Gaussianity, Mon. Not. Roy. Astron. Soc. 442 (2014), no. 3 2511–2518, [arXiv:1402.2290].
  * [50] D. Yamauchi, K. Takahashi, and M. Oguri, Constraining primordial non-Gaussianity via a multitracer technique with surveys by Euclid and the Square Kilometre Array, Phys. Rev. D 90 (2014), no. 8 083520, [arXiv:1407.5453].
  * [51] R. de Putter and O. Doré, Designing an Inflation Galaxy Survey: how to measure $\sigma(f_{\rm NL})\sim 1$ using scale-dependent galaxy bias, Phys. Rev. D 95 (2017), no. 12 123513, [arXiv:1412.3854].
  * [52] J. Fonseca, R. Maartens, and M. G. Santos, Probing the primordial Universe with MeerKAT and DES, Mon. Not. Roy. Astron. Soc. 466 (2017), no. 3 2780–2786, [arXiv:1611.01322].
  * [53] M. Schmittfull and U. Seljak, Parameter constraints from cross-correlation of CMB lensing with galaxy clustering, Phys. Rev. D 97 (2018), no. 12 123540, [arXiv:1710.09465].
  * [54] M. Münchmeyer, M. S. Madhavacheril, S. Ferraro, M. C. Johnson, and K. M. Smith, Constraining local non-Gaussianities with kinetic Sunyaev-Zeldovich tomography, Phys. Rev. D 100 (2019), no. 8 083508, [arXiv:1810.13424].
  * [55] J. Fonseca, R. Maartens, and M. G. Santos, Synergies between intensity maps of hydrogen lines, Mon. Not. Roy. Astron. Soc. 479 (2018), no. 3 3490–3497, [arXiv:1803.07077].
  * [56] SKA Collaboration, D. J. Bacon et al., Cosmology with Phase 1 of the Square Kilometre Array: Red Book 2018: Technical specifications and performance forecasts, Publ. Astron. Soc. Austral. 37 (2020) e007, [arXiv:1811.02743].
  * [57] Z. Gomes, S. Camera, M. J. Jarvis, C. Hale, and J. Fonseca, Non-Gaussianity constraints using future radio continuum surveys and the multitracer technique, Mon. Not. Roy. Astron. Soc. 492 (2020), no. 1 1513–1522, [arXiv:1912.08362].
  * [58] M. Ballardini, W. L. Matthewson, and R. Maartens, Constraining primordial non-Gaussianity using two galaxy surveys and CMB lensing, Mon. Not. Roy. Astron. Soc. 489 (2019), no. 2 1950–1956, [arXiv:1906.04730].
  * [59] J. R. Bermejo-Climent, M. Ballardini, F. Finelli, D. Paoletti, R. Maartens, J. A. Rubiño Martín, and L. Valenziano, Cosmological parameter forecasts by a joint 2D tomographic approach to CMB and galaxy clustering, Phys. Rev. D 103 (2021), no. 10 103502, [arXiv:2106.05267].
  * [60] J.-A. Viljoen, J. Fonseca, and R. Maartens, Multi-wavelength spectroscopic probes: prospects for primordial non-Gaussianity and relativistic effects, JCAP 11 (2021) 010, [arXiv:2107.14057].
  * [61] DESI Collaboration, A. Aghamousa et al., The DESI Experiment Part I: Science,Targeting, and Survey Design, arXiv:1611.00036.
  * [62] S. Yahia-Cherif, A. Blanchard, S. Camera, S. Casas, S. Ilić, K. Markovic, A. Pourtsidou, Z. Sakr, D. Sapone, and I. Tutusaus, Validating the Fisher approach for stage IV spectroscopic surveys, Astron. Astrophys. 649 (2021) A52, [arXiv:2007.01812].
  * [63] M. Berti, M. Spinelli, and M. Viel, Multipole expansion for 21cm Intensity Mapping power spectrum: forecasted cosmological parameters estimation for the SKA Observatory, arXiv:2209.07595.
  * [64] S. Cunnington et al., HI intensity mapping with MeerKAT: power spectrum detection in cross-correlation with WiggleZ galaxies, arXiv:2206.01579.
  * [65] MeerKLASS Collaboration, M. G. Santos et al., MeerKLASS: MeerKAT Large Area Synoptic Survey, in MeerKAT Science: On the Pathway to the SKA, 9, 2017.  arXiv:1709.06099.
  * [66] J. Wang et al., HI intensity mapping with MeerKAT: calibration pipeline for multidish autocorrelation observations, Mon. Not. Roy. Astron. Soc. 505 (2021), no. 3 3698–3721, [arXiv:2011.13789].
  * [67] N. Agarwal, V. Desjacques, D. Jeong, and F. Schmidt, Information content in the redshift-space galaxy power spectrum and bispectrum, JCAP 03 (2021) 021, [arXiv:2007.04340].
  * [68] D. Blas, J. Lesgourgues, and T. Tram, The Cosmic Linear Anisotropy Solving System (CLASS) II: Approximation schemes, JCAP 1107 (2011) 034, [arXiv:1104.2933].
  * [69] A. Barreira, G. Cabass, F. Schmidt, A. Pillepich, and D. Nelson, Galaxy bias and primordial non-Gaussianity: insights from galaxy formation simulations with IllustrisTNG, JCAP 12 (2020) 013, [arXiv:2006.09368].
  * [70] A. Barreira, The local PNG bias of neutral Hydrogen, H${I}$, JCAP 04 (2022), no. 04 057, [arXiv:2112.03253].
  * [71] J.-A. Viljoen, J. Fonseca, and R. Maartens, Constraining the growth rate by combining multiple future surveys, JCAP 09 (2020) 054, [arXiv:2007.04656].
  * [72] F. Villaescusa-Navarro et al., Ingredients for 21 cm Intensity Mapping, Astrophys. J. 866 (2018), no. 2 135, [arXiv:1804.09180].
  * [73] S. Cunnington, Detecting the power spectrum turnover with HI intensity mapping, Mon. Not. Roy. Astron. Soc. 512 (2022), no. 2 2408–2425, [arXiv:2202.13828].
  * [74] J. Fonseca, J.-A. Viljoen, and R. Maartens, Constraints on the growth rate using the observed galaxy power spectrum, JCAP 1912 (2019), no. 12 028, [arXiv:1907.02975].
  * [75] O. Umeh, R. Maartens, H. Padmanabhan, and S. Camera, The effect of finite halo size on the clustering of neutral hydrogen, arXiv:2102.06116.
  * [76] E. Castorina and F. Villaescusa-Navarro, On the spatial distribution of neutral hydrogen in the Universe: bias and shot-noise of the HI power spectrum, Mon. Not. Roy. Astron. Soc. 471 (2017), no. 2 1788–1796, [arXiv:1609.05157].
  * [77] P. Bull, P. G. Ferreira, P. Patel, and M. G. Santos, Late-time cosmology with 21cm intensity mapping experiments, Astrophys. J. 803 (2015), no. 1 21, [arXiv:1405.1452].
  * [78] D. Alonso, P. G. Ferreira, M. J. Jarvis, and K. Moodley, Calibrating photometric redshifts with intensity mapping observations, Phys. Rev. D 96 (2017), no. 4 043515, [arXiv:1704.01941].
  * [79] S. Jolicoeur, R. Maartens, E. M. De Weerd, O. Umeh, C. Clarkson, and S. Camera, Detecting the relativistic bispectrum in 21cm intensity maps, JCAP 06 (2021) 039, [arXiv:2009.06197].
  * [80] Cosmic Visions 21 cm Collaboration, R. Ansari et al., Inflation and Early Dark Energy with a Stage II Hydrogen Intensity Mapping experiment, arXiv:1810.09572.
  * [81] S. Casas, I. P. Carucci, V. Pettorino, S. Camera, and M. Martinelli, Constraining gravity with synergies between radio and optical cosmological surveys, arXiv:2210.05705.
  * [82] J. L. Bernal, P. C. Breysse, H. Gil-Marin, and E. D. Kovetz, User’s guide to extracting cosmological information from line-intensity maps, Phys. Rev. D 100 (2019), no. 12 123522, [arXiv:1907.10067].
  * [83] A. Barreira, On the impact of galaxy bias uncertainties on primordial non-Gaussianity constraints, JCAP 12 (2020) 031, [arXiv:2009.06622].
  * [84] D. Karagiannis, R. Maartens, J. Fonseca, S. Camera, and C. Clarkson, Multi-tracer power spectra and bispectra I: Formalism, arXiv:2301.nnnnn.
  * [85] D. Karagiannis, A. Lazanu, M. Liguori, A. Raccanelli, N. Bartolo, and L. Verde, Constraining primordial non-Gaussianity with bispectrum and power spectrum from upcoming optical and radio surveys, Mon. Not. Roy. Astron. Soc. 478 (2018), no. 1 1341–1376, [arXiv:1801.09280].
  * [86] V. Yankelevich and C. Porciani, Cosmological information in the redshift-space bispectrum, Mon. Not. Roy. Astron. Soc. 483 (2019), no. 2 2078–2099, [arXiv:1807.07076].
  * [87] R. Maartens, S. Jolicoeur, O. Umeh, E. M. De Weerd, C. Clarkson, and S. Camera, Detecting the relativistic galaxy bispectrum, JCAP 03 (2020), no. 03 065, [arXiv:1911.02398].
  * [88] Planck Collaboration, N. Aghanim et al., Planck 2018 results. VI. Cosmological parameters, Astron. Astrophys. 641 (2020) A6, [arXiv:1807.06209]. [Erratum: Astron.Astrophys. 652, C4 (2021)].


# A cost effective eye movement tracker based wheel chair control algorithm
for people with paraplegia

Skanda Upadhyaya
Department of Electrical and Electronics Engineering
National Institute of Technology Karnataka
Surathkal, India  Siddhanth P. Rao
Department of Electrical and Electronics Engineering
National Institute of Technology Karnataka
Surathkal, India  Shravan Bhat
Department of Electrical and Electronics Engineering
National Institute of Technology Karnataka
Surathkal, India  V Ashwin
Department of Electrical and Electronics Engineering
National Institute of Technology Karnataka
Surathkal, India  Krishnan Chemmangat Assistant Professor Department of
Electrical and Electronics Engineering
National Institute of Technology Karnataka
Surathkal, India
<EMAIL_ADDRESS>
###### Abstract

Spinal cord injuries can often lead to quadriplegia in patients limiting their
mobility. Wheelchairs could be a good proposition for patients, but most of
them operate either manually or with the help of electric motors operated with
a joystick. This, however, requires the use of hands, making it unsuitable for
quadriplegic patients. Controlling the eye movement, on the other hand, is
retained even by the people who undergo brain injury. Monitoring the movements
in the eye can be a helpful tool in generating control signals for the
wheelchair. This paper is an approach on converting obtained signals from the
eye into meaningful signals by trying to control a bot which imitates a
wheelchair. The overall system is cost-effective and uses simple image
processing and pattern recognition to control the bot. An android application
is developed, which could be used by the patients’ aid for more refined
control of the wheelchair in the actual scenario.

###### Index Terms:

Eye Ball Tracking, Raspberry Pi, Convolutional Neural Network, iDroid, TCP/UDP
Protocol, Dense Layer, Maxpool Layer, ESP8266, Arduino UNO, Motor Driver,
Firebase

##  I Introduction

According to a study conducted by American Spinal Injury Association in 2016
it was found that a total of 12500 people are affected by the spinal cord
injuries in varying degree of spinal cord injuries [1]. According to their
estimates, the global yearly average is anywhere between 1.3 to 2.6 hundred
thousand cases. Many of these injuries lead to quadriplegia which severely
affects the mobility of a person. Apart from trauma, quadriplegia can be
inherited [2]. A motorised wheelchair appropriately controlled by signal
generated by the patients body could be an effective solution for this
problem. Control signals such as electromyogram (EMG) [3],
electroencephalogram (EEG) [4], or electrooculogram (EOG) [5], generated from
the quadriplegic patients eyes, face or head could be used to control this
device. However, capturing and processing minute signals is often difficult
and the sensing and signal conditioning circuits are highly sophisticated and
expensive making the affordability a big concern. In a country like India
where the total number of physically disabled people was found to be close to
5.36 million in the year 2016, such eye tracking devices can give them a feel
of self reliability [6].

Another alternative to using physiological signals for the control of
wheelchair is to track the eye movement of the patient. The history of eye
tracking dates back to mid 19th century. Some of the earlier applications of
eye tracking were in text detection, text reading, to measure the
effectiveness of advertisements, to control devices using the eye [7].
Nowadays, eye tracking is used in scientific research, augmented reality,
medical field and smart devices[8, 9, 10]. The system requires simple cameras
clubbed with image processing and pattern recognition algorithms and can be
implemented in commonly available processors. This also brings down the cost
of the overall control system in comparison to those which uses physiological
signals[11, 12]. This project aims to make a reliable eye tracking model and
implement the hardware. This model should be able to accurately and
efficiently track the eyeball movement and assign commands to each unique
movement, to move a bot/ wheelchair [13, 14, 15].

This model can be helpful in different fields.
The following flowchart shown in Fig. 1 gives our approach on the
implementation of the model.

Figure 1: Flowchart

##  II Eye Tracking

say generally about the eye tracking algorithm with the overall flow chart
here.

###  II-A Hardware Setup

  * •

Raspberry Pi Camera: It is a 5 megapixel camera module which was set to record
video at 60 frames per second (fps).It has been highly optimized to work with
the Raspberry Pi Processor. It has extremely fast transmission speeds due to
its Parallel-In-Parallel-Out (PIPO) set of ribbon cables. The RPi camera is
mounted, at a distance of 9cm to 12cm, in front of the eye for the best focus.
This RPi camera then captures the movements of the pupil of the eye and sends
it to the RaspberryPi for processing.

  * •

RaspberryPi: It is a very powerful and low-cost micro-processor. It has a 1GB
RAM capacity and supports many of the common python libraries. The RaspberryPi
is used to process the incoming video frames from the RPi Camera. The incoming
frame from the RPi camera is a three-dimension colour image This coloured
image was first made into a greyscaled image and then resized to a 128px by
128px image.This was done to reduce the overall computational time.Then this
resized gray scaled image is passed onto the image processing model sitting on
the RP

###  II-B Image Processing and control command generation

The image processing model we used is a convolutional neural network trained
on a self created dataset. It receives the image from rpi local host,
processes it and predicts the orientation of the eye. The predicted
orientation is passed on to the firebase.
The dataset is self made using pictures clicked of around 10 persons. The
dataset distribution is as follows:
Down: 2045 images
Up: 2475 images
Left: 2858 images
Right: 2797 images
Straight: 3058 images
The images are of various sizes in the RGB(color) format.

Figure 2: CNN Flowchart

####  II-B1 Data Preprocessing

In the data preprocessing phase, all the images are cropped to focus only on
the eye and the images are converted to gray scale. The images are further
resized to 128 by 128 pixels to have a uniform size throughout the dataset.

####  II-B2 Neural Network architecture

As the flow chart depicts, the neural network consists of four sets of
convolutional layer, dropout layer and max pooling layer followed by
flattening layer and full connection using dense layer. The convolutional
layers have 32, 64, 128 and 128 filters in successive layers. The kernel size
of these filters are fixed to 3 by 3. Dropout layers are added to reduce over-
fitting with a dropout rate of 0.4. The kernel size in max pooling layers are
fixed to 2 by 2. A flatten layer flattens out all the inputs received by it
which is then fed to fully connected dense layer with 256 neurons. The output
of this layer is given to the final output layer with 5 neurons which predict
the orientation. The convolutional layers and deep dense layer uses rectified
linear units as activation function. The output layer uses softmax to predict
the probabilities that the input belongs to a certain class.

Figure 3: Rectified Linear Unit Figure 4: Softmax Function

####  II-B3  Training Phase

In the training phase, the processed images are first loaded. The entire
dataset is further divided into training and validation sets randomly. The
validation set is approximately 30% of the total dataset. The model is trained
with adam optimizer and categorical cross-entropy as loss function for 25
epochs. The trained model is saved.

##  III Mobile Application

An Android Application is made to enable manual override of the bot during
certain situations. It has options to update the direction and even the speed
of the bot to the firebase. It also allows the user to give voice commands.
The android application acts as a safety precaution in case the RaspberryPi is
unable to send controls and communicate with firebase.

##  IV iDroid

iDroid is the hardware prototype of a wheelchair. The prototype of the
wheelchair is composed mainly Arduino UNO, NodeMCU, Ultrasonic Sensor (HC-
SR04) and L293D Motor Driver Shield. The Arduino and Motor Driver Shield are
interfaced and made into a single unit. The Motors and the Ultrasonic Sensor
are interfaced to the Motor Driver Shield. A power bank with dual port is used
to power the Arduino and the Motor Driver Shield for better performance. The
firebase has a tag named Signals which is updated according to the eyeball
movement of the user. The following table indicates the relation between the
movement of the eye and the movement of the bot. These signals will be
considered valid and get updated to the firebase only if the same signal is
given by the user for more than 30 frames. This information is then received
from the firebase using the NodeMCU. This received information is then
communicated to the Arduino using the SoftwareSerial library. The Arduino now
receives the information and signals the Motor Driver Shield to appropriately
move the bot. All this happens with a delay of about 500ms. When the bot is in
motion the Ultrasonic Sensor continuously checks for any obstacles. If any
obstacle is detected within the threshold distance, the bot stops immediately.

###  IV-A NodeMCU

NodeMCU was chosen for this project because of its ability to connect easily
to a nearby Wi-Fi Router or a Mobile Hotspot. It is small in size, inexpensive
and has low power requirements. It also has a plenty of online learning
resources and help from the online community.

Moreover it can be programmed using the Arduino IDE by installing some
necessary board packages and thereby reduces the effort of getting used to a
new programming environment. The GPIO pins are used for Serial Communication
with the help of the SoftwareSerial Library.

The NodeMCU connects to a nearby Wi-Fi Router or a Mobile Hotspot and fetches
the information from the Firebase. After this it communicates the information
to the Arduino using the SoftwareSerial library.

Movement of the eye Movement of the bot Down Stop Left Left Right Right Up
Start Straight Forward

TABLE I: Control Signal

###  IV-B Arduino UNO

Arduino was chosen for this project because it can easily connect with
different sensors and has got many shields which provide it with additional
capabilities. It also has got a large support from the Arduino community which
is very active. Arduino UNO was chosen because it is inexpensive and small in
size. It also is the most popular and the most used board among the other
microcontrollers of the Arduino Family and a result has very good
documentation too.

The Arduino is programmed using the Arduino IDE. The TX and RX pins on the
board are the pins meant for Serial Communication. The other pins can be used
for Serial Communication by using the SoftwareSerial Library.

The information that the NodeMCU receives from the Firebase is communicated to
the Arduino using the SoftwareSerial library. The Arduino receives the
information and signals the Motor Driver Shield to appropriately the move to
the bot according to the command received.

###  IV-C L293D Motor Driver Shield

L293D Motor Driver Shield was chosen because of its ability to power 4 bi-
directional DC Motors. It also allows easy interface of additional sensors
thereby eliminating the need of a breadboard.

The Motor Driver Shield offers an 8 bit speed control (0-255) for varying the
speed of the bot. The speed of the bot is gradually increased to the required
value to prevent sudden jerks to the user and also sudden loading of the power
source. The use of the AFMotor Library simplifies the coding part as it has
direct inbuilt functions for controlling the motors.

###  IV-D Ultrasonic Sensor (HC-SR04)

Ultrasonic Sensor was chosen for this project because it is inexpensive, easy
to use, small in size, available readily, highly accurate and detects most
type of obstacles. When the bot is in motion the Ultrasonic Sensor
continuously checks for any obstacles. If any obstacle is detected within the
threshold distance, the bot stops immediately.

Figure 5: Ultrasonic Timing Diagram

The Ultrasonic Sonic Sensor consists of 2 Ultrasonic Transducers which act as
Transmitter and Receiver. The Trigger Pin of the Ultrasonic Sensor is given a
pulse of a minimum duration of 10 µS, resulting in the transmission a sonic
burst of 8 pulses with a frequency of 40 KHz. Once this is transmitted the
Echo pin goes high and then goes low only if the pulses reflect from the
surface of an obstacle and reach the sensor. If there are no obstacles
encountered then the echo signal goes low by itself after 38 mS.

Hence once we know the time and speed, we estimate the distance of the
obstacle from the bot and its accuracy is further increased by using the
NewPing library.

##  V Results and Discussions

###  V-A Performance of the eye tracking algorithm

The confusion matrix is shown in Tab

Actual Predicted class class Down Left Right Up Straight Down 628 0 0 0 0 Left
0 851 0 0 0 Right 0 1 840 0 0 Up 0 1 0 714 3 Straight 0 1 0 0 931

TABLE II: Confusion Matrix.

The classification report is as follows:

Classes Precision Recall F1 Score Support Down 1 1 1 628 Left 1 1 1 851 Right
1 1 1 841 Up 1 0.99 1 718 Straight 1 1 1 932

TABLE III: Classification Report

###  V-B Overall system performance

##  VI Conclusions

  * •

The RPi camera needs sufficient illumination to capture the images correctly.
The prediction on the RPi camera is slow as it requires high computation power
to process the convolutional neural network. Also, RPi faced compatibility
issues with TensorFlow versions. So RPi just resizes the captured images,
grayscale’s the image and feeds this to the computer to do the predictions.
The resizing of images and conversion from color to grayscale is done to
reduce latency issues. It should be ensured that the RPi camera focuses only
on the eye and doesn’t move much.

  * •

The predictions are fast enough to detect even the quick blinks. To avoid the
detection of unintentional blinks, the firebase is not updated until the
predictions are the same for a minimum of 20 frames. This also reduces latency
issues in firebase.

  * •

The four-wheeler iDroid being used requires a lot of power. So the iDroid must
be powered by well charged Power Bank.

  * •

The training accuracy is found to be 99.99848%. This indicates that there is a
chance that the model is overfitting. Attempts has been made to make the model
more robust and error-free by adding dropout layer and making the data set
skew free.

  * •

The image processing model(CNN) is able to process and predict at a rate of
15-16 FPS.

## References

  * [1] T. T. Roberts, G. R. Leonard, and D. J. Cepela, “Classifications in brief: American spinal injury association (asia) impairment scale,” 2017.
  * [2] C. Blackstone, “Hereditary spastic paraplegia,” in _Handbook of clinical neurology_. Elsevier, 2018, vol. 148, pp. 633–652.
  * [3] G. Jang, J. Kim, S. Lee, and Y. Choi, “Emg-based continuous control scheme with simple classifier for electric-powered wheelchair,” _IEEE Transactions on Industrial Electronics_ , vol. 63, no. 6, pp. 3695–3705, 2016\.
  * [4] Z. Al-Qaysi, B. Zaidan, A. Zaidan, and M. Suzani, “A review of disability eeg based wheelchair control system: Coherent taxonomy, open challenges and recommendations,” _Computer methods and programs in biomedicine_ , vol. 164, pp. 221–237, 2018.
  * [5] Q. Huang, S. He, Q. Wang, Z. Gu, N. Peng, K. Li, Y. Zhang, M. Shao, and Y. Li, “An eog-based human–machine interface for wheelchair control,” _IEEE Transactions on Biomedical Engineering_ , vol. 65, no. 9, pp. 2023–2032, 2017\.
  * [6] D. Verma, P. Dash, S. Bhaskar, R. P. Pal, K. Jain, R.P.Srivastava, Hansraj, Namit, and H.P.Kesan, “Disabled persons in india. a statistical profile 2016,” Social Statistics Division. Ministry of Statistics and Programme Implementation. Government of India, Tech. Rep., 2016.
  * [7] H. Singh, “Human eye tracking and related issues: A review,” _International Journal of Scientific and Research Publications_ , vol. 2, pp. 1–9, 09 2012.
  * [8] S. Naspetti, R. Pierdicca, S. Mandolesi, M. Paolanti, E. Frontoni, and R. Zanoli, “Automatic analysis of eye-tracking data for augmented reality applications: A prospective outlook,” in _Augmented Reality, Virtual Reality, and Computer Graphics_ , L. T. De Paolis and A. Mongelli, Eds. Cham: Springer International Publishing, 2016, pp. 217–230.
  * [9] K. Harezlak and P. Kasprowski, “Application of eye tracking in medicine: A survey, research issues and challenges,” _Computerized Medical Imaging and Graphics_ , vol. 65, 05 2017.
  * [10] Y. Chen, C. Chiang, C. Yu, W. Sun, and S. Yuan, “Real-time eye tracking and event identification techniques for smart tv applications,” in _2014 IEEE International Conference on Consumer Electronics - Taiwan_ , 2014, pp. 63–64.
  * [11] R. Meimban, A. Monsura, J. Ranada, and A. R. Fernando, “Eye-controlled wheelchair using infrared detection,” _International Journal of Simulation: Systems, Science and Technology_ , vol. 19, pp. 12.1–12.6, 06 2018\.
  * [12] A. Guin and B. Baishya, “Brain controlled wheelchair using labview,” Ph.D. dissertation, 05 2013.
  * [13] P. S. Gajwani and S. A. Chhabria, “Eye motion tracking for wheelchair control,” _International Journal of Information Technology_ , vol. 2, no. 2, pp. 185–187, 2010.
  * [14] G. Gautam, G. Sumanth, K. Karthikeyan, S. Sundar, and D. Venkataraman, “Eye movement based electronic wheel chair for physically challenged persons,” _International journal of scientific & technology research_, vol. 3, no. 2, pp. 206–212, 2014.
  * [15] S. Plesnick, D. Repice, and P. Loughnane, “Eye-controlled wheelchair,” in _2014 IEEE Canada International Humanitarian Technology Conference-(IHTC)_. IEEE, 2014, pp. 1–4.