# CEERS Key Paper VIII: Emission Line Ratios from NIRSpec and NIRCam Wide-
Field Slitless Spectroscopy at $z>2$

Bren E. Backhaus Department of Physics, 196A Auditorium Road, Unit 3046, University of Connecticut, Storrs, CT 06269, USA  Jonathan R. Trump Department of Physics, 196A Auditorium Road, Unit 3046, University of Connecticut, Storrs, CT 06269, USA  Nor Pirzkal ESA/AURA Space Telescope Science Institute  Guillermo Barro Department of Physics, University of the Pacific, Stockton, CA 90340 USA  Steven L. Finkelstein Department of Astronomy, The University of Texas at Austin, Austin, TX, USA  Pablo Arrabal Haro NSF’s National Optical-Infrared Astronomy Research Laboratory, 950 N. Cherry Ave., Tucson, AZ 85719, USA  Raymond C. Simons Department of Physics, 196A Auditorium Road, Unit 3046, University of Connecticut, Storrs, CT 06269, USA  Jessica Wessner Department of Physics, 196A Auditorium Road, Unit 3046, University of Connecticut, Storrs, CT 06269, USA  Nikko J. Cleri Department of Physics and Astronomy, Texas A&M University, College Station, TX, 77843-4242 USA  George P. and Cynthia Woods Mitchell Institute for Fundamental Physics and Astronomy, Texas A&M University, College Station, TX, 77843-4242 USA  Micaela B. Bagley Department of Astronomy, The University of Texas at Austin, Austin, TX, USA  Michaela Hirschmann Institute of Physics, Laboratory of Galaxy Evolution, Ecole Polytechnique Fédérale de Lausanne (EPFL), Observatoire de Sauverny, 1290 Versoix, Switzerland  David C. Nicholls Research School of Astronomy and Astrophysics, Australian National University, Canberra, ACT 2600, Australia  Mark Dickinson NSF’s National Optical-Infrared Astronomy Research Laboratory, 950 N. Cherry Ave., Tucson, AZ 85719, USA  Jeyhan S. Kartaltepe Laboratory for Multiwavelength Astrophysics, School of Physics and Astronomy, Rochester Institute of Technology, 84 Lomb Memorial Drive, Rochester, NY 14623, USA  Casey Papovich Department of Physics and Astronomy, Texas A&M University, College Station, TX, 77843-4242 USA  George P. and Cynthia Woods Mitchell Institute for Fundamental Physics and Astronomy, Texas A&M University, College Station, TX, 77843-4242 USA  Dale D. Kocevski Department of Physics and Astronomy, Colby College, Waterville, ME 04901, USA  Anton M. Koekemoer Space Telescope Science Institute, 3700 San Martin Dr., Baltimore, MD 21218, USA  Laura Bisigello Dipartimento di Fisica e Astronomia "G.Galilei", Universitá di Padova, Via Marzolo 8, I-35131 Padova, Italy  INAF–Osservatorio Astronomico di Padova, Vicolo dell’Osservatorio 5, I-35122, Padova, Italy  Anne E. Jaskot Department of Astronomy, Williams College, Williamstown, MA, 01267, USA  Ray A. Lucas Space Telescope Science Institute, 3700 San Martin Drive, Baltimore, MD 21218, USA  Intae Jung Space Telescope Science Institute, 3700 San Martin Drive Baltimore, MD 21218, United States  Stephen M. Wilkins Astronomy Centre, University of Sussex, Falmer, Brighton BN1 9QH, UK  Institute of Space Sciences and Astronomy, University of Malta, Msida MSD 2080, Malta  L. Y. Aaron Yung NASA Postdoctoral Fellow  Astrophysics Science Division, NASA Goddard Space Flight Center, 8800 Greenbelt Rd, Greenbelt, MD 20771, USA  Henry C. Ferguson Space Telescope Science Institute, Baltimore, MD, USA  Adriano Fontana INAF - Osservatorio Astronomico di Roma, via di Frascati 33, 00078 Monte Porzio Catone, Italy  Andrea Grazian INAF–Osservatorio Astronomico di Padova, Vicolo dell’Osservatorio 5, I-35122, Padova, Italy  Norman A. Grogin Space Telescope Science Institute, Baltimore, MD, USA  Lisa J. Kewley Center for Astrophysics | Harvard & Smithsonian, 60 Garden Street, Cambridge, MA 02138, USA  Allison Kirkpatrick Department of Physics and Astronomy, University of Kansas, Lawrence, KS 66045, USA  Jennifer M. Lotz Gemini Observatory/NSF’s National Optical-Infrared Astronomy Research Laboratory, 950 N. Cherry Ave., Tucson, AZ 85719, USA  Laura Pentericci INAF - Osservatorio Astronomico di Roma, via di Frascati 33, 00078 Monte Porzio Catone, Italy  Pablo G. Pérez-González Centro de Astrobiología (CAB), CSIC-INTA, Ctra. de Ajalvir km 4, Torrejón de Ardoz, E-28850, Madrid, Spain  Swara Ravindranath Space Telescope Science Institute, 3700 San Martin Drive, Baltimore, MD 21218, USA  Rachel S. Somerville Center for Computational Astrophysics, Flatiron Institute, 162 5th Avenue, New York, NY, 10010, USA  Guang Yang Kapteyn Astronomical Institute, University of Groningen, P.O. Box 800, 9700 AV Groningen, The Netherlands  Benne W. Holwerda Physics & Astronomy Department, University of Louisville, 40292 KY, Louisville, USA  Peter Kurczynski Astrophysics Science Division, NASA Goddard Space Flight Center, 8800 Greenbelt Rd, Greenbelt, MD 20771, USA  Nimish P. Hathi Space Telescope Science Institute, Baltimore, MD, USA  Caitlin Rose Laboratory for Multiwavelength Astrophysics, School of Physics and Astronomy, Rochester Institute of Technology, 84 Lomb Memorial Drive, Rochester, NY 14623, USA  Kelcey Davis Department of Physics, 196A Auditorium Road, Unit 3046, University of Connecticut, Storrs, CT 06269, USA

###### Abstract

We use James Webb Space Telescope Near-Infrared Camera Wide Field Slitless
Spectroscopy (NIRCam WFSS) and Near-Infrared spectrograph (NIRSpec) in the
Cosmic Evolution Early Release survey (CEERS) to measure rest-frame optical
emission-line of 155 galaxies at $z>2$. The blind NIRCam grism observations
include a sample of galaxies with bright emission lines that were not observed
on the NIRSpec masks. We study the changes of the H$\alpha$,
$\hbox{[{O~{}III}]}/\hbox{{\rm H}$\beta$}$, and
$\hbox{[{Ne~{}III}]}/\hbox{[{O~{}II}]}$ emission lines in terms of redshift by
comparing to lower redshift SDSS and CLEAR samples. We find a significant
($>3\sigma$) correlation between $\hbox{[{O~{}III}]}/\hbox{{\rm H}$\beta$}$
with redshift, while $\hbox{[{Ne~{}III}]}/\hbox{[{O~{}II}]}$ has a marginal
(2$\sigma$) correlation with redshift. We compare
$\hbox{[{O~{}III}]}/\hbox{{\rm H}$\beta$}$ and
$\hbox{[{Ne~{}III}]}/\hbox{[{O~{}II}]}$ to stellar mass and H$\beta$ SFR. We
find that both emission-line ratios have a correlation with H$\beta$ SFR and
an anti-correlation with stellar mass across the redshifts $0<z<9$. Comparison
with MAPPINGS V models indicates that these trends are consistent with lower
metallicity and higher ionization in low-mass and high-SFR galaxies. We
additionally compare to IllustrisTNG predictions and find that they
effectively describe the highest $\hbox{[{O~{}III}]}/\hbox{{\rm H}$\beta$}$
ratios observed in our sample, without the need to invoke MAPPINGS models with
significant shock ionizionation components.

Active galaxies – emission line galaxies – Galaxy evolution – Galaxies

##  1 Introduction

Galaxy emission lines provide a wealth of information about galaxy formation
and physical properties. Emission lines can be used to determine a galaxy’s
interstellar medium (ISM) conditions such as the metallicity, ionization, and
density, as well as the physical properties such as star formation rate
(Kennicutt & Evans 2012; Brinchmann et al. 2004) and dust attenuation
(Cardelli et al. 1989; Calzetti & Heckman 1999; Reddy et al. 2016; Shapley et
al. 2023b). One way to analyze emission lines is by comparing ratios of lines
at similar wavelengths to gain information on galaxy ISM conditions. Picking
emission-line ratios with similar wavelength makes the ratio less sensitive to
dust attenuation. The most well-known emission-line ratio diagrams, BPT
(Baldwin et al. 1981) and VO87 (Veilleux & Osterbrock 1987), compare
$\hbox{[{O~{}III}]}\lambda 5007/\hbox{{\rm H}$\beta$}$ with
$\hbox{[{N~{}II}]}\lambda 6583/\hbox{{\rm H}$\alpha$}$ or
$\hbox{[{S~{}II}]}\lambda 6583/\hbox{{\rm H}$\alpha$}$ to identify high
ionization galaxies. These diagrams make use of the strongest emission lines
in rest-frame optical spectra (Baldwin et al. 1981; Veilleux & Osterbrock
1987; Kauffmann et al. 2003a; Kewley et al. 2006) to identify the dominant
ionizing sources in galaxies.

Studying emission lines at different redshifts enables an understanding of how
the physical conditions of galaxies change over cosmic time. At higher
redshifts the strong rest-optical lines move to the near-IR. The cosmic star
formation rate density is much higher at $z\sim 2$ than in the local Universe
Madau & Dickinson (2014). Studies have found that galaxies at $z\sim 2$ have
lower metallicities (Steidel et al. 2014; Henry et al. 2013; Papovich et al.
2022), have higher ionization in the ISM, and exhibit $\alpha$-enhancement
(Steidel et al. 2016; Shapley et al. 2019) than the local universe (Liu et al.
2008; Shapley et al. 2015; Strom et al. 2018), which is consistent with cosmic
noon galaxies having higher star formation and more AGN. Due to the higher
ionization, lower metallicity, and $\alpha$-element enhancement of star-
forming galaxies at $z\sim 2$, the BPT and VO87 diagrams are not effective at
distinguishing star-forming galaxies from AGN (Coil et al. 2015; Backhaus et
al. 2022; Cleri et al. 2023a). At redshift $z\sim 2$ new emission-line
diagrams were established such as the OHNO diagram, comparing
$\hbox{[{Ne~{}III}]}\lambda 3869/\hbox{[{O~{}II}]}\lambda 3726+3729$ to
$\hbox{[{O~{}III}]}/\hbox{{\rm H}$\beta$}$, to find high ionization sources
instead (Backhaus et al. 2022; Zeimann et al. 2015).

Figure 1: Distribution of redshifts for each emission line sample: $Ha$(top),
[O III](middle), [Ne III] and [O II](bottom). These are named by there
emission line which has a $SNR>3$, in the case of [Ne III] and [O II] either
line may reach this requirement. The samples come from the SDSS(left), CLEAR
(middle), and CEERS(right). In the right panel the gray histograms are the
NIRSpec observations and blue are NIRCam WFSS.

JWST now gives us access to the early universe, where the first galaxies and
black holes are being formed (e.g., Larson et al. 2023; Finkelstein et al.
2023). Which gives us the opportunity to study and understand the ISM
conditions of galaxies at this early period of the universe. JWST observations
will show how these galaxies assembled and evolved into the universe we see
today. JWST near-IR spectroscopy allows us to view the rest-frame optical
emission lines of galaxies at $2<z<9$.

Previous work has been done with NIRSpec spectroscopy from the JWST Early
Release Observations (ERO) (Pontoppidan et al. 2022) of the lensing cluster
SMACS J0723.3-7327 to investigate emission-lines of $z>5$ galaxies, however
this uses a small sample of 3-6 galaxies. Schaerer et al. (2022), Curti et al.
(2023), Trump et al. (2022) studied the metallicities of these galaxies. Trump
et al. (2022), Cleri et al. (2023b), and Brinchmann (2022) went further and
compared emission lines at $z>5$ to local samples, finding higher
($\hbox{[{Ne~{}III}]}/\hbox{[{O~{}II}]}$) in the early galaxies compared to
the local ($z\sim 0$) galaxies indicating higher ionization. Additionally,
analysis has been done with the Cosmic Evolution Early Release Science (CEERS)
survey program. The NIRSpec spectroscopy included in the CEERS survey has been
used to gain a wealth of information such as: spectroscopically confirmed
high-redshift galaxies (Arrabal Haro et al. 2023; Curtis-Lake et al. 2023;
Fujimoto et al. 2023), identifying and characterizing high-redshift AGN
(Kocevski et al. 2022; Larson et al. 2023), study Ly$\alpha$ emission (Jung et
al. 2023; Tang et al. 2023), and studying ISM conditions as a function of
galaxy properties such as stellar mass and star formation rate (Shapley et al.
2023a, b). Shapley et al. (2023b) found there was not significant evolution
between stellar mass and $\hbox{[{O~{}III}]}/\hbox{{\rm H}$\beta$}$ for
galaxies above $z>3$, but their stellar masses alone suggest subsolar
metallicity. Shapley et al. (2023b) also showed that the $z>5$ galaxies
preference higher H$\alpha$ SFR when compared to Speagle et al. (2014)
predictions. The ISM conditions of galaxies at $z>5$ were shown to have high
ionization and low metallicity (>0.1 $Z_{\odot}$)(Sanders et al. 2023a).
Sanders et al. (2023b) makes use of $T_{e}$ based metallicity to calibrate
strong-line metallicity estimators for $z>5$ galaxies (Reddy et al. 2023).

In this work, we use NIRCam WFSS and NIRSpec MSA spectroscopy taken as part of
CEERS to investigate the rest-frame optical emission-line evolution and galaxy
properties of $\sim$155 galaxies at $z>2$. In Section 2 we describe our data
reductions and sample selection. Section 2 also establishes our comparison
samples of galaxies at $z\sim 0$ and $z\sim 2$. In Section 3 we compare
galaxies in our NIRSpec and NIRCam WFSS samples. In Section 4 we use our three
subsamples covering the epoch of reionization ($z>6$), cosmic noon ($z\sim
2$), and the local universe ($z\sim 0$) to study how each emission-line ratio
evolves with redshift. Section 5 describes the connections between galaxy
properties and emission-line ratios at different redshifts. Section 6 presents
the ISM conditions inferred by the emission-line ratios. We summarize our
results in Section 7. In this work, we assume a $\Lambda$ cold dark matter
cosmology with $\Omega_{M}=0.3$, $\Omega_{\Lambda}=0.7$, and $H_{0}=70$
$kms^{-1}$ $Mpc^{-1}$ (Planck Collaboration et al. 2016).

##  2 Observational Data and Sample

###  2.1 JWST WFSS and MSA Spectroscopy

Our parent galaxy sample comes from JWST observations taken by the CEERS
program, ERS-1345 (PI: Steven Finkelstein). CEERS uses NIRCam WFSS and NIRSpec
multi-object spectroscopy to cover $\sim$100 arcmin2 of the Extended Groth
Strip Hubble Space Telescope (HST) legacy field (EGS, Davis et al. 2007) which
is covered by the CANDELS HST survey (Grogin et al. 2011; Koekemoer et al.
2011). Our paper focuses on the four pointings with NIRCam WFSS and six
pointings with NIRSpec. CEERS has four NIRCam WFSS pointings that partially
overlap with the CEERS NIRSpec observations allowing the emission line
measurements of both instruments to be compared, allowing NIRSpec to be
calibrated for slit losses.

Figure 2: Left: Distribution of H$\beta$ SFR and stellar mass for our $\hbox{[{O~{}III}]}/\hbox{{\rm H}$\beta$}$ sample. Right: Distribution of H$\alpha$ SFR and stellar mass for our H$\alpha$ sample. In both the panels galaxies observed by NIRCam WFSS are represented by black points and arrows to represent lower limits,and galaxies observed by NIRSpec are represented by pink points. The red and gray contours represent the CLEAR and SDSS samples, respectively, with contour level indicating relative galaxy density of each sample.  Figure 3: Example of NIRCam WFSS direct image (left) with 1D/2D Column and Row WFSS spectra for an example galaxy, $016\\_B\\_13569$, that is in our $\hbox{[{O~{}III}]}/\hbox{{\rm H}$\beta$}$ sample. Vertical lines in the 1D spectra indicate the emission lines of interest. We also include the direct image and 1D/2D NIRSpec spectra of 717 from the G395M filter (bottom).  Table 1: $\hbox{[{O~{}III}]}/\hbox{{\rm H}$\beta$}$ NIRCam Sample  ID | R.A. | Dec. | z |  [O III] Column |  [O III] Row |  H$\beta$ Column |  H$\beta$ Row | $\log(M_{\star})$ | log(SFR) | $A_{V}$
---|---|---|---|---|---|---|---|---|---|---
| deg | deg |  | $10^{-18}erg/s/cm^{2}$ | $10^{-18}erg/s/cm^{2}$ | $10^{-18}erg/s/cm^{2}$ | $10^{-18}erg/s/cm^{2}$ | [$M_{\odot}$] | [$M_{\odot}$]/yr | mag
012_B_22258 | 214.929820 | 52.862205 | 5.635 | 49.37$\pm$ 6.33 | 41.37 $\pm$5.92 | 6.00$\pm$4.91 | 5.07 $\pm$4.91 | 10.05 | 2.16 | 0.8
013_A_8436 | 214.958411 | 52.843681 | 5.999 | 17.26 $\pm$2.02 | 17.06$\pm$2.03 | 3.65$\pm$ 2.07 | 2.33$\pm$1.87 | 9.37 | 0.45 | 0.0
015_B_13732 | 214.987915 | 52.879438 | 6.177 | 44.09 $\pm$4.89 | 48.33$\pm$3.98 | 6.03$\pm$5.04 | 7.84$\pm$3.30 | 9.95 | 1.18 | 0.0
015_B_19309 | 214.970312 | 52.881717 | 5.640 | 8.63$\pm$3.05 | 4.14$\pm$2.18 | 5.47$\pm$3.73 | 2.80$\pm$2.17 | 8.87 | 0.22 | 0.0
012_B_30258 | 214.918284 | 52.879359 | 5.645 | 10.95$\pm$3.65 | 14.16 $\pm$3.47 | 2.85 $\pm$5.02 | 1.21$\pm$3.91 | 9.20 | 0.65 | 0.0
013_A_15609 | 214.948684 | 52.856466 | 6.530 | 3.47$\pm$1.29 | 7.49 $\pm$2.71 | 0.01 $\pm$1.19 | 1.52 $\pm$1.88 | 9.12 | 0.18 | 0.0
015_B_19805 | 214.958370 | 52.875115 | 6.170 | 34.25$\pm$12.26 | 33.86$\pm$5.80 | 2.43 $\pm$10.46 | 4.76$\pm$6.41 | 9.37 | 0.19 | 0.1
016_A_13158 | 215.122563 | 52.973201 | 6.380 | 42.02$\pm$5.94 | 49.80 $\pm$5.33 | 2.27 $\pm$4.11 | 4.32$\pm$ 5.29 | 9.89 | 1.31 | 0.0
016_A_16127 | 215.127984 | 52.984951 | 6.664 | 37.96$\pm$5.68 | 33.45$\pm$ 4.37 | 0.77 $\pm$4.58 | 1.70 $\pm$2.64 | 9.98 | 0.81 | 0.0
016_A_18444 | 215.106518 | 52.975820 | 6.174 | 12.27 $\pm$4.72 | 14.37$\pm$ 2.93 | 3.09$\pm$4.14 | 0.27$\pm$2.07 | 9.48 | 0.54 | 0.0
012_A_27683 | 214.989007 | 52.919644 | 6.670 | 11.81 $\pm$2.79 | 11.91$\pm$3.97 | 2.01$\pm$1.92 | 1.37$\pm$ 2.13 | 9.74 | 0.81 | 0.0
013_A_10524 | 214.949129 | 52.843185 | 6.716 | 18.91$\pm$ 4.62 | 16.83 $\pm$3.23 | 10.57 $\pm$4.10 | 0.52$\pm$3.69 | 9.83 | 1.82 | 0.8
013_B_17930 | 214.869985 | 52.807034 | 6.749 | 31.76 $\pm$6.47 | 32.30 $\pm$4.05 | 6.41$\pm$2.28 | 2.21$\pm$2.68 | 10.19 | 0.73 | 0.0
015_B_5666 | 215.000956 | 52.865869 | 6.670 | 11.18$\pm$2.92 | 6.66$\pm$3.25 | 2.36$\pm$2.04 | 1.78$\pm$2.25 | 9.23 | 0.50 | 0.0
012_B_28896 | 214.921870 | 52.876193 | 5.630 | 14.94$\pm$6.96 | 12.38$\pm$3.83 | 2.35$\pm$3.91 | 4.41$\pm$ 1.50 | 9.24 | 0.14 | 0.0
015_A_16826 | 215.032039 | 52.918960 | 6.170 | 15.34 $\pm$ 3.71 | 14.03$\pm$3.33 | 4.51$\pm$4.90 | 2.12$\pm$1.99 | 9.70 | 0.76 | 0.1
015_A_17952 | 215.023039 | 52.915309 | 6.165 | 9.08$\pm$ 2.35 | 7.55 $\pm$3.45 | 0.84$\pm$3.02 | 2.67$\pm$1.51 | 9.25 | 0.31 | 0.0
015_B_10107 | 214.987324 | 52.868911 | 5.665 | 17.88$\pm$5.82 | 18.44$\pm$4.50 | 3.56$\pm$7.44 | 7.92$\pm$ 4.12 | 9.38 | 0.82 | 0.0
016_B_13823 | 215.090344 | 52.951601 | 5.501 | 18.82$\pm$8.36 | 29.20$\pm$4.27 | 5.69$\pm$13.39 | 4.50 $\pm$4.36 | 9.21 | 0.36 | 0.0
Table 1: The emission line flux measurements for the [O III] NIRCam WFSS sample in units of $10^{-18}erg/s/cm^{2}$. Other columns show the spectroscopic redshift, the stellar mass, SED SFR and dust attenuation from the FAST SED fitting. The ID indicates the field$\\_$panel$\\_$WFSS ID number. Table 3: H$\alpha$ SFR NIRCam Sample  ID | R.A. | Dec. | z |  H$\alpha$ Column |  H$\alpha$ Row | $\log(M_{\star})$ | log(SFR) | $A_{V}$
---|---|---|---|---|---|---|---|---
| deg | deg |  | $10^{-18}erg/s/cm^{2}$ | $10^{-18}erg/s/cm^{2}$ | [$M_{\odot}$] | [$M_{\odot}$]/yr | mag
012_A_25344 | 214.977646 | 52.903536 | 4.770 | 8.17 $\pm$1.79 | 21.61 $\pm$ 7.47 | 9.84 | 0.66 | 0.0
012_A_25431 | 214.977566 | 52.903450 | 4.550 | 9.36 $\pm$0.85 | 3.35 $\pm$1.88 | 7.46 | -0.07 | 1.3
012_A_31041 | 214.953995 | 52.907856 | 4.550 | 8.52 $\pm$3.03 | 6.49$\pm$1.44 | 7.04 | -0.53 | 0.8
012_A_35810 | 214.942294 | 52.919362 | 3.938 | 38.27 $\pm$8.28 | 50.02$\pm$6.51 | 7.73 | 0.20 | 1.3
012_A_36545 | 214.937471 | 52.918283 | 3.938 | 29.55$\pm$9.62 | 24.45 $\pm$10.11 | 10.25 | 0.24 | 0.1
012_B_26391 | 214.927867 | 52.871022 | 4.805 | 12.42$\pm$1.67 | 10.74$\pm$3.26 | 8.24 | -1.49 | 0.6
012_B_29360 | 214.917888 | 52.875555 | 4.717 | 12.09$\pm$ 2.36 | 12.83$\pm$3.73 | 8.06 | -1.95 | 0.3
013_A_19410 | 214.943410 | 52.864098 | 4.675 | 20.50 $\pm$5.27 | 12.32$\pm$5.67 | 10.32 | 1.38 | 0.8
013_A_20628 | 214.941356 | 52.864855 | 4.678 | 8.29$\pm$2.49 | 7.33 $\pm$2.36 | 7.86 | -1.62 | 0.0
013_B_13408 | 214.894682 | 52.812130 | 4.876 | 14.12$\pm$3.88 | 18.39 $\pm$4.52 | 8.08 | -2.73 | 0.3
015_A_20196 | 215.022027 | 52.920785 | 4.540 | 14.11$\pm$3.47 | 6.73 $\pm$2.95 | 9.03 | -9.90 | 2.9
015_B_16325 | 214.978092 | 52.879514 | 4.545 | 16.27$\pm$4.19 | 21.61 $\pm$3.85 | 7.48 | -0.09 | 1.2
015_B_17048 | 214.985862 | 52.886907 | 4.546 | 9.39$\pm$3.17 | 10.73$\pm$ 3.36 | 7.20 | -0.33 | 1.3
016_A_5887 | 215.151987 | 52.974048 | 4.480 | 29.21$\pm$ 4.90 | 18.21 $\pm$ 2.09 | 8.42 | 0.61 | 1.9
016_A_8530 | 215.149544 | 52.978974 | 4.525 | 1.93$\pm$1.59 | 6.29 $\pm$ 0.15 | 8.07 | -8.80 | 0.7
016_B_15982 | 215.059038 | 52.936442 | 4.280 | 65.93$\pm$16.13 | 89.81$\pm$2.93 | 9.53 | 0.97 | 0.0
016_B_18194 | 215.079996 | 52.956800 | 4.745 | 30.26$\pm$7.73 | 17.87$\pm$5.60 | 8.50 | -0.46 | 0.5
016_B_8414 | 215.083373 | 52.931987 | 4.117 | 11.18 $\pm$ 3.61 | 20.14 $\pm$ 3.73 | 8.05 | -1.68 | 0.7
Table 2: The H$\alpha$ flux measurements from NIRCam WFSS, spectroscopic redshift, the stellar mass, SED SFR and dust attenuation from the FAST SED fitting. The ID indicates the field$\\_$panel$\\_$WFSS ID number. Table 5: NIRSpec Sample  R.A. | Dec. | z | [O III] | H$\beta$ | H$\alpha$ | [Ne III] | [O II] | $\log(M_{\star})$ | log(SFR) | $A_{V}$
---|---|---|---|---|---|---|---|---|---|---
deg | deg |  | $10^{-18}ergs/s/cm^{2}$ | $10^{-18}ergs/s/cm^{2}$ | $10^{-18}ergs/s/cm^{2}$ | $10^{-18}ergs/s/cm^{2}$ | $10^{-18}ergs/s/cm^{2}$ | [$M_{\odot}$] |  $[M_{\odot}]$/yr | mag
214.957160 | 52.872372 | 3.228 | 2.57 $\pm$0.31 | 0.54 $\pm$0.43 | 1.57$\pm$0.21 | 1.82 $\pm$ 0.52 | 1.22$\pm$0.47 | 8.91 | 0.18 | 0.0
214.959997 | 52.831169 | 4.900 | 3.09 $\pm$0.19 | 0.25$\pm$0.12 | 1.20 $\pm$0.08 | -99.00 $\pm$0.00 | -99.00 $\pm$0.00 | 9.11 | 0.67 | 0.0
214.893181 | 52.882484 | 3.000 | 1.83$\pm$0.22 | 0.78$\pm$ 0.35 | 1.49 $\pm$ 0.27 | 2.05$\pm$0.54 | 2.10$\pm$ 0.59 | 8.80 | 0.06 | 0.0
214.943900 | 52.850052 | 5.001 | 65.25$\pm$19.57 | 8.19$\pm$0.88 | 30.26$\pm$ 0.84 | 3.60 $\pm$ 0.45 | 11.17$\pm$0.84 | 9.48 | 0.74 | 0.0
214.941496 | 52.850565 | 2.540 | 10.00 $\pm$4.94 | 5.29 $\pm$0.52 | 18.24$\pm$ 1.76 | 0.56$\pm$ 1.13 | 10.69$\pm$0.80 | 9.03 | 0.48 | 0.0
214.907360 | 52.844535 | 2.010 | 15.23 $\pm$1.67 | 1.83$\pm$1.01 | 5.80$\pm$0.25 | 1.05$\pm$ 0.77 | 2.53$\pm$1.02 | 7.54 | -0.86 | 0.0
214.898480 | 52.861709 | 1.922 | 1.35$\pm$0.64 | 0.79 $\pm$ 0.58 | 4.22$\pm$0.53 | 0.52$\pm$ 229.54 | 4.63$\pm$0.60 | 9.76 | 0.84 | 0.3
214.909604 | 52.880284 | 2.144 | 4.79 $\pm$0.37 | 1.19 $\pm$0.42 | 2.85$\pm$ 0.18 | 0.36$\pm$ 0.46 | 2.59$\pm$0.43 | 8.85 | 0.41 | 0.0
214.966546 | 52.846672 | 2.136 | 19.14 $\pm$3.83 | 3.99$\pm$0.54 | 14.20$\pm$ 0.76 | 1.07 $\pm$.92 | 11.29$\pm$1.42 | 9.18 | 0.25 | 0.0
214.940182 | 52.836026 | 1.699 | 9.83$\pm$0.97 | 6.28$\pm$1.29 | 25.11$\pm$1.09 | 2.14 $\pm$1.29 | 16.69$\pm$2.59 | 10.18 | 1.19 | 0.8
Table 3: The emission-line fluxes from the NIRSpec sample. The galaxy IDs are
formatted as pointing_ID and redshift. Emission-line fluxes are reported in
units of $10^{-18}ergs/s/cm^{2}$. The SFR and dust attenuation measurements
are produced from SED fitting. A machine-readable version of the full table is
available.

The four NIRCam WFSS pointings use the F356W filter to cover $3.14-3.98$ µm
including a suite of rest-frame optical lines for $0<z<9$ galaxies with a
total exposure time of 2490 sec between the two orthogonal gratings in the
WFSS. The WFSS spectra have spectroscopic resolving power R$\sim$1,600 at
$\sim 4\micron$. The details of the NIRCam wide-field slitless spectrograph
(WFSS) are described by Greene et al. (2017).

The six NIRSpec spectroscopy pointings use the G140M/F100LP, G235M/F170LP and
G395M/F290LP filters, spanning 1-5 $\micron$. Four of these pointings were
observed in medium resolution, R$\sim$1000, and in prism R$\sim$100\. Two
pointings had light-leak failures in the prism observations and we use only
the medium-resolution grating observations (the failed prism observations were
rescheduled and observed later but are not used in this work). The details of
the NIRSpec instrument are described by Jakobsen et al. (2022). Further
information on the programs used in the CEERS extractions and fits can be read
in Bagley et al. (2023) and Arrabal Haro et al. (2023).

####  2.1.1 Data Reduction and Sample Selection

Information on the reductions of the NIRSpec data can be found in Arrabal Haro
et al., (in prep.). The reduction’s main steps are as follows. The NIRSpec
data is processed with the STScI Calibration Pipeline version 1.8.5
111https://jwst-pipeline.readthedocs.io/en/latest/index.html and the
Calibration Reference Data System (CRDS). The calwebb_detector1 pipeline
module was used to reduce the uncalibrated images by applying the correction
for “snowballs” events caused by cosmic rays, the 1/f noise correction, and
doing a saturation check. An improved correction of the “snowball”
events222https://jwst-docs.stsci.edu/data-artifacts-and-features/snowballs-
and-shower-artifacts are applied to the jump step. This step also creates the
count-rate maps (CRMs).

The generated CRMs were then passed through the calwebb_spec2 pipeline to
create two-dimensional (2D) cutouts of the slitlets. The calwebb_spec2
pipeline performs the background subtraction by using a 3-nod pattern,
corrects the flat-fields, implements the wavelength and photometric
calibrations and resamples the 2D spectra to correct the distortion of the
spectral trace.

The calwebb_spec3 pipeline stage creates the final 2D spectra, by combining
the images of the three nods. Our data uses the default slit loss correction
settings to create this spectra. The 1D spectra are created by extracting from
the 2D spectra using customized apertures. These custom apertures are visually
defined for targets to maximize the signal-to-noise ratios (SNRs).

The NIRSpec flux uncertainties are underestimated by a factor of $\sim 2$,
determined by comparing he normalized median absolute deviation (NMAD) of the
flux to the median of the flux uncertainty for each source. We correct for
this by increasing the flux uncertainty of each spectrum by the ratio of the
NMAD of the flux to the median flux uncertainty,
$\mathrm{NMAD}(f)/\mathrm{median}(\sigma_{f})$.

The NIRCAM WFSS data were first processed with the Stage 1 STScI pipeline to
apply bias and dark corrections and perform on-the-ramp fitting to detect and
remove cosmic ray impacts. A broad filter F356W flat-field was applied to the
data as it was shown to reduce pixel-to-pixel variation. Finally, proper world
coordinate information was added using the AssignWCS() Pipeline task. Spectra
were extracted from these processed datasets following the Simulation Based
Extraction Pirzkal et al. (2017). We used an extraction catalog derived from
the CEERS F356W mosaic. Spectral contamination was modeled and subtracted
using pixel-level SEDs for each source computed from the CEERS F277W, F356W,
and F444W mosaics. Finally, the dispersed background was subtracted as a
combination of pre-launch synthetic dispersed F356W background and of an
additional row (column) model of the residuals in the GRISMR (GRISMC)
observations. This results in the modeling and extractions of 24,000 spectra.
Continuum is detected in sources down to approximately $\sim 25.75$ AB
magnitude.

The NIRCam WFSS flux uncertainties are underestimated by a factor of $\sim 3$
in the row dispersion and $\sim 4$ in the column dispersion. This was
determined and corrected using the same method with the ratio of NMAD of the
flux to the median of the uncertainty used for the NIRSpec sample.

Our photometric redshifts used for sample selection are obtained using the
same method outlined in Finkelstein et al. (2022). Briefly, the multi-band
SEDs were fit using the EAZYpy (Brammer et al. 2008) software package.
Probability distribution functions (PDFs) are created by fitting non-negative
linear combinations of templates to the observed data. The templates used are
a set of 12 FSPS “$tweak\\_fsps\\_QSF\\_12\\_v3$” Conroy & Gunn (2010), and
six additional templates to cover bluer colors. These bluer color templates
are shown by Larson et al. (2023) to improve the photometric redshift fits for
z$>9$ galaxies. These templates use stellar population models created with
BPASS Eldridge & Stanway (2009) with 5$\%$ solar metallicities and young
stellar populations, $\log$(age/yr)=6,6.5,7.

####  2.1.2 Sample Selection

To extract the emission line flux from both NIRCam WFSS, we first use
photometric redshifts to constrain our sample to find galaxies of interest.
Our constraints were determined by which redshifts would have our desired
emission lines land in the F356W filter. This would give redshift ranges of
$3.7<z<5.1$ for H$\alpha$, a $5.4<z<7$ range for [O III], and $7.4<z<9.3$ for
[O II]. We increase the range of each redshift bin to account for galaxies
that may have an inaccurate photometric redshift, using ranges of $3.2<z<5.5$
for H$\alpha$, a $4.9<z<7.5$ range for [O III], and $6.9<z<9.8$ for [O II] to
be further inspected. The photometric redshift overlap between [O III] and [O
II] is due to adding $\Delta z=0.5$ to select galaxies. Meanwhile for NIRSpec
we use the photometric redshift constraint of $1.6<z<6.8$ as this covers all
emission lines spanning the G140M, G235M, and G395M filters. We determined the
spectroscopic redshift for NIRCam and NIRSpec for each source using the best-
fit line center of the brightest emission line in each spectrum, usually [O
III]$\lambda$5008 or H$\alpha$ as they are the brightest lines in the spectra.
To measure emission line fluxes we find the best-fit Gaussian function (and
associated uncertainties) using a Levenberg-Marquardt least-squares method
implemented by the scipy curvefit python code.

Our NIRSpec sample has a total of 118 galaxies, 93 galaxies in the H$\alpha$
sample, 96 galaxies in the [O III] sample, and 59 in the [O II] sample. These
galaxies follow the same sample selection process as NIRCam WFSS, requiring a
$S/N>3$ for at least one emission line. The last sample includes [Ne
III]$\lambda$3869 and the blended [O II]$\lambda$3726+3729 doublet, which is
accessible up to z$\sim$8\. Figure 3 shows examples of NIRSpec 1D and 2D
spectra on the bottom right panels.

We do not dust correct the emission lines of our samples because both
H$\alpha$ and H$\beta$ are not available for all galaxies in the sample. The
emission-line ratios used in this work are not affected by dust attenuation
due to the fact that the emission-line pairs are close in wavelength and are
nearly equally affected by dust. We note that 62$\%$ of our [O III]/H$\beta$
galaxies between NIRSpec MSA and NIRCam WFSS at $z>4$ have an SED $A_{V}<0.2$,
so we expect the H$\beta$ SFR estimates to be minimally effected by dust.
Meanwhile only 48$\%$ of our H$\alpha$ galaxies at $z>4$ have an SED
$A_{V}<0.2$, and only three galaxies in the NIRCam WFSS H$\alpha$ sample have
SED $A_{V}<0.2$. Galaxies affected by dust will have a lower limit for
H$\beta$ or H$\alpha$ SFR.

SFR is calculated from either the H$\beta$ or H$\alpha$ emission line,
depending on redshift, by following the Kennicutt & Evans (2012) SFR relation
for H$\alpha$ and $\hbox{{\rm H}$\alpha$}/\hbox{{\rm H}$\beta$}=2.86$
(assuming Case B recombination, $T=10^{4}$ K, and $n_{e}=10^{4}$ cm-3;
Osterbrock 1989):

$\log({\rm SFR})[M_{\odot}/\mathrm{yr}]=\log[L(\hbox{{\rm H}$\beta$})]-40.82\
$ (1)

$\log({\rm SFR})[M_{\odot}/\mathrm{yr}]=\log[L(\hbox{{\rm H}$\alpha$})]-41.27$
(2)

Due to the H$\beta$ and H$\alpha$ lines not being dust corrected these SFR are
a lower limit.

We create two WFSS subsamples of galaxies based on a signal-to-noise ratio of
$\mathrm{S/N}>3$ detection for at least one line of interest when the two
orients are coadded. The first includes galaxies with $\mathrm{S/N}>3$ in
H$\alpha$ at z$\sim$4\. The second includes galaxies with $\mathrm{S/N}>3$ [O
III]$\lambda$5007 emission lines at z$\sim$6\. We note there are no galaxies
with [Ne III]$\lambda$3868 or [O II]$\lambda$3728 detected at S/N$>$3 in the
CEERS NIRCam WFSS observations due to the low observation depth. We visually
inspect the filter image, 1D, and 2D spectra of galaxies selected for these
samples to ensure the emission line is detected in both orients. An example of
one galaxy inspected and included in the samples is shown in Figure 3.

The sample selection and inspection results in 18 galaxies with H$\alpha$ in
the redshift range $4<z<5$ and 19 galaxies with [O III] in the redshift range
$5.5<z<7$ from the NIRCam WFSS.

Tables 1 and 2 present the source IDs, spectroscopic redshifts, emission line
flux, stellar mass, SED SFR, and dust attenuation measurements of the NIRCam
WFSS [O III] and H$\alpha$ galaxy samples. Our NIRSpec sample is shown in
Table 3 showing the ID, redshift, emission line fluxes. Figures 1 and 2 show
the redshift distribution, stellar mass and SFR of both samples.

Figure 4: Left: Comparison of the NIRSpec and NIRCam WFSS measurements of
F277W continuum flux and [O III] flux. Comparison between emission line flux
from NIRSpec (gray histogram) and NIRCam WFSS (blue histograms) for Middle:
H$\alpha$ emission and Right: [O III] emission. Both these emission lines were
seen in both instruments. A blind survey like NIRCam WFSS includes galaxies
that have been missed in the targeted NIRSpec observations: the NIRCam WFSS
galaxies tend to have stronger emission lines.

####  2.1.3 Stellar Mass

Stellar masses and dust attenuations in our NIRCam WFSS and NIRSpec sample
come from fitting the optical and NIR SEDs from the long-wavelength NIRCam
filters using FAST Kriek et al. (2009). These models assume stellar population
synthesis models used by Bruzual & Charlot (2003), following the initial mass
function (IMF) defined by Chabrier (2003), a Calzetti et al. (2000) dust
attenuation law, and a delayed exponential star formation history.

###  2.2 Lower Redshift Comparison Samples

We established two $z\sim 2$ comparison samples to cover the peak of cosmic
star formation (Madau & Dickinson 2014) and supermassive black hole growth
(Aird et al. 2010). A SDSS $z\sim 0$ comparison sample was also established to
cover the local universe (Brinchmann et al. 2004; Kauffmann et al. 2003b, c).

####  2.2.1 SDSS $z\sim 0$ Sample

The $z\sim 0$ comparison sample is created from the Sloan Digital Sky Survey
(SDSS; York et al. 2000) Data Release 10 (Ahn et al. 2014). The SDSS data set
used a 2.5 m telescope at Apache Point Observatory to cover 14,555 deg2 in the
sky with $R\sim 2000$ over $3800<\lambda<9200$ Å (Smee et al. 2013).

Emission-line measurements and redshifts for the SDSS data set are computed by
Bolton et al. (2012), using a stellar template to correct the continuum for
stellar absorption. Stellar masses are estimated by Montero-Dorta et al.
(2016) from the broadband $ugriz$ SDSS photometry using a grid of templates
made from the FSPS stellar population synthesis code (Conroy et al. 2009).
These templates assume a Kroupa (2001) IMF and fit for the dust attenuation
following Charlot & Fall (2000) and Calzetti et al. (2000).

The low-redshift, z$\sim$0, comparison sample was selected using the same
$\mathrm{S/N}>3$ line detection thresholds as for the CEERS and CLEAR samples.
These selection criteria results in 284,523 galaxies with H$\alpha$, 231,999
galaxies with $\hbox{[{O~{}III}]}/\hbox{{\rm H}$\beta$}$, and 27,847 galaxies
with $\hbox{[{Ne~{}III}]}/\hbox{[{O~{}II}]}$. We note this SDSS sample has no
cuts on whether a galaxy is an AGN or SF.

####  2.2.2 CLEAR z$\sim$1.5 Sample

Our $z\sim 2$ comparison sample comes from the CANDELS Ly$\alpha$ Emission at
Reionization (CLEAR) survey (Simons et al. 2023) and HST near-IR spectroscopy
with the G102 and G141 grisms taken as part of the 3D-HST program (Momcheva et
al. 2016; Brammer et al. 2012; van Dokkum et al. 2011).

We select the CLEAR comparison sample using the same $\mathrm{S/N}>3$ line
detection thresholds as for the CEERS sample, visually inspecting the direct
image, 1D and 2D spectra to remove galaxies with contaminated spectra. This
gives us 2890 galaxies with H$\alpha$, 1534 galaxies with [O III], and 505 in
our [O II] sample. Due to this sample’s low spectral resolution ($R\sim 100$),
H$\alpha$ is blended with [N II]$\lambda$6583+6548. This blending causes the
H$\alpha$ fluxes of CLEAR galaxies to effectively be lower limits, although
most $z\sim 2$ galaxies have $\hbox{[{N~{}II}]}/\hbox{{\rm H}$\alpha$}\ll 1$
(e.g., Shapley et al. 2015).

####  2.2.3 MOSDEF z$\sim 3$ Sample

We also compare to the stacked line-ratio measurements from the MOSFIRE Deep
Evolution Field (MOSDEF) observations used in Sanders et al. (2021) at
redshift $z\sim 3.3$. This sample was observed by the Multi-Object
Spectrometer For Infrared Exploration (MOSFIRE; McLean et al. (2012)) on the
10 m Keck I telescope, observing 48.5 nights over a four year period to obtain
rest-frame optical spectra of $1.4<z<3.8$ galaxies (Kriek et al. 2015). This
survey covers the AEGIS, COSMOS, and GOODS-N fields. In order to cover
multiple emission lines, two to three filters are used in the survey to
observe H-band selected galaxies. MOSDEF adopts a slit width of 0.7
arcseconds, which results in a spectral resolution of R = 3400, 3000, 3650 and
3600 for Y, J, H, and K, respectively.

We use the stacked line-ratio measurements from Sanders et al. (2021), which
were calculated from $\sim$750 galaxies at z $\sim$ 2.3 and $\sim$375 galaxies
at z$\sim$ 3.3. For a full description of the MOSDEF survey design and data
reduction, see Kriek et al. (2015).

Figure 5: The $\log{\hbox{[{O~{}III}]}/\hbox{{\rm H}$\beta$}}$ emission-line
ratio versus redshift. The black point is the median redshift and
$\log{\hbox{[{O~{}III}]}/\hbox{{\rm H}$\beta$}}$ of the SDSS sample York et
al. (2000), with the error bars representing the standard deviation of the
sample. The gray points and pink squares are from the CLEAR and MOSDEF samples
respectively (Simons et al. 2023; Sanders et al. 2021). The SMACS observations
are marked as crimson squares. NIRSpec AGN from CEERS are purple stars. The
NIRSpec and WFSS samples are represented by pink points and red points and
arrows respectively. The yellow line is the linear fit to the NIRSpec, and
NIRCam WFSS data. The cyan lines are the rolling median detection limit (MDL)
to the best-fit line for the NIRSpec galaxies which indicates the range of
galaxies we could see based on our S/N cutoffs. This increase of
$\hbox{[{O~{}III}]}/\hbox{{\rm H}$\beta$}$ with redshift is also shown in the
linear fit line, with a slope of (0.06$\pm$0.02).

##  3 NIRCam WFSS and NIRSpec MSA Comparison

The NIRCam WFSS is a blind survey that allows us to gain access to a sample of
galaxies that may be missed by targeted NIRSpec observations, which require
pre-selection of galaxies based on redshift and brightness. In the right panel
of Figure 4 the NIRSpec $z>5$ galaxies are represented by gray circles, while
the blue circles represent the combined column and row measurements of NIRCam
WFSS. The NIRCam WFSS galaxies occupy a similar region of continuum
measurements, however these galaxies tend to have higher [O III] emission.

Due to this difference in sample selection, the NIRCam WFSS provides an
opportunity to view different types of galaxies. These galaxies also have
differences in the emission line measurements. Figure 4 also shows a
comparison of the flux measurements of [O III] and H$\alpha$ between NIRCam
WFSS and NIRSpec. These emission lines were chosen as they have S/N$>3$.
NIRSpec is represented by the gray histogram and the combined measurements
from both the Row and Column dispersions for NIRCam WFSS are shown in the blue
histograms. This comparison shows that NIRCam WFSS tends to include galaxies
with brighter emission line fluxes. This indicates that the CEERS NIRSpec
observations are missing galaxies with more extreme emission lines. This
should be kept in mind in our following sections when comparing NIRSpec and
NIRCam WFSS galaxies.

##  4 Redshift Evolution of Emission Line Galaxies

We measure the galaxy emission-line ratios with redshift by comparing CEERS
galaxies with $z\sim 2$ galaxies from CLEAR and MOSDEF and with $z\sim 0$
galaxies from SDSS.

Figure 6: The $\log{\hbox{[{Ne~{}III}]}/\hbox{[{O~{}II}]}}$ emission-line
ratio versus redshift. The black point is the median redshift and
$\log{\hbox{[{Ne~{}III}]}/\hbox{[{O~{}II}]}}$ of the SDSS sample York et al.
(2000). The gray point is a median value from CLEAR Simons et al. (2023). The
pink points and red squares are from the CEERS NIRSpec and SMACS samples
respectively. The NIRSpec $\hbox{[{Ne~{}III}]}/\hbox{[{O~{}II}]}$ line ratio
has a 2.5$\sigma$ slope with redshift, (0.05$\pm$0.02), as shown by the yellow
best fit line. However, if you remove the SDSS and CLEAR samples we find the
rest of the sample has a slope of 0.05 with a 2$\sigma$ certainty.

In Figure 5 we plot the [O III]/H$\beta$ emission-line ratios against
redshift. The black point is the median value from the SDSS sample, the gray
points median values from two redshift bins from the CLEAR sample, and the
Sanders et al. (2021) redshift 3.3 binned data are shown as pink squares. The
pink circles are our NIRSpec sample and the red points are the CEERS sample.
Arrows in our CEERS WFSS sample represent lower limits in the emission-line
ratio due to undetected H$\beta$. Other high redshift galaxies from SMACS ERO
NIRSpec observations (Trump et al. 2022) and broad-line AGN from Kocevski et
al. (2022) and Larson et al. (2023) are marked as crimson squares, purple
stars, and gold diamonds respectively. There is a 0.49 dex increase in
$\hbox{[{O~{}III}]}/\hbox{{\rm H}$\beta$}$ between the SDSS sample and CLEAR,
see Backhaus et al. (2022) which carefully matched the two samples to avoid
differences in sample selection for details and discussion. We also find that
the $z>5$ CEERS sample has 0.33 dex higher $\hbox{[{O~{}III}]}/\hbox{{\rm
H}$\beta$}$ than the CLEAR sample, but we note this may be due to the
difference in the luminosity selection. A linmix linear fit to the line ratios
measured from CEERS NIRSpec and NIRCam WFSS observations indicates a shallow
but significant (3$\sigma$) increase of [O III]/H$\beta$ with redshift, this
fit includes the lower limits. Many of the NIRCam WFSS line ratios are lower
limits due to undetected H$\beta$ lines: these limits are generally consistent
with the measured NIRSpec line ratios but is also consistent with a steeper
increase of [O III]/H$\beta$ with redshift.

The median detection limits (MDL) are created by taking the upper and lower
limit of each galaxy. The resulting plotted lines are the difference between
the detection limits and the fit. This line represents the lowest signal that
can be observed with a 1$\sigma$ detection of the line flux. We note that our
NIRSpec MSA data at high redshifts are well separated from the rolling MDL,
indicating there is no selection bias for our high redshift sample. Thus the
small increase of [O III]/H$\beta$ ratio with redshift is not likely to be a
simple result of a changing detection limit with redshift.

Figure 6 shows the [Ne III]/[O II] ratio with redshift, using the same
notation as Figure 5. The $\hbox{[{Ne~{}III}]}/\hbox{[{O~{}II}]}$ ratio also
increases with redshift. There is a 0.34 dex increase between the SDSS and
CLEAR samples. This is different from the results from Backhaus et al. (2022)
which found $\hbox{[{Ne~{}III}]}/\hbox{[{O~{}II}]}$ had a 0.2 dex difference
between SDSS and CLEAR. This may be because Backhaus et al. (2022) required a
S/N>1 in both [O II] and [Ne III] where we only require S/N>3 for [O II] for
both SDSS, CLEAR, and CEERS samples. There is also a 0.37 dex increase between
CLEAR and galaxies with z>5\. This is smaller than the 0.5 dex increase
between CLEAR and SMACS reported by Trump et al. (2022), however Trump et al.
(2022) only had 5 galaxies above $z>5$. There is only a marginal correlation
between the $\hbox{[{Ne~{}III}]}/\hbox{[{O~{}II}]}$ ratio and redshift in the
CEERS samples with a slope of $0.05\pm 0.02$. The rolling MDL indicates the
CEERS $z>2$ sample may be affected by the detection limit caused by the
S/N$>1$ detection threshold for [Ne III], such that there might exist $z>1.5$
galaxies with lower $\hbox{[{Ne~{}III}]}/\hbox{[{O~{}II}]}$ that are
undetected. On the other hand, the measured line ratios are well-separated
from the upper detection limit and there appears to be a genuine lack of
high-$\hbox{[{Ne~{}III}]}/\hbox{[{O~{}II}]}$ galaxies at $z>2$.

Figure 7: The H$\alpha$ SFR versus redshift. The black point is the median
value of the SDSS sample York et al. (2000). The gray point is a median value
from CLEAR Simons et al. (2023). The pink points and red circles are from the
CEERS NIRSpec and NIRCam WFSS samples respectively. The cyan and green lines
represent the lower detection of the NIRSpec and NIRCam WFSS data
respectively. There is a 1.5 dex increase in the median H$\alpha$ SFR
measurements from SDSS to CLEAR and a 0.5 dex increase in the median H$\alpha$
SFR measurements between CLEAR and $z>5$ galaxies. The yellow best fit line to
the NIRSpec and NIRCam H$\alpha$ SFR a 8$\sigma$ slope with redshift,
(0.24$\pm$0.03).

Finally, we show the relationship between H$\alpha$ SFR with redshift in
Figure 7. This sample is not dust corrected as we do not have a pair of
hydrogen lines in all observations and the SED-based $A_{V}$ may be unreliable
for estimating the nebular attenuation. We note that the H$\alpha$ NIRCam WFSS
galaxies have higher $A_{V}$ measured by their SED fitting than the other
samples. This would push the galaxies to higher H$\alpha$ SFR. We again see a
correlation of H$\alpha$ SFR with redshift, With a 1.5 dex increase between
SDSS and CLEAR, and a 0.5 dex increase between CLEAR and CEERS galaxies with
$z>5$. We see a 0.7 dex difference between the NIRCam WFSS and NIRSpec
galaxies between $3.8<z<5$ due to the different detection limits of each
instrument.

The relationships between emission lines and galaxy properties and ISM
conditions are further explored in Section 5 and 6 of the paper.

##  5 Emission Line Properties with Galaxy Stellar Mass and SFR

We will now investigate how the emission-line ratios correlate to galaxy
propterties such as stellar mass and SFR.

Figure 8: Left: The relationships between the $\hbox{[{O~{}III}]}/\hbox{{\rm
H}$\beta$}$ emission-line ratio and the galaxy H$\beta$ luminosity and
H$\beta$ SFR. Right: The relationships between the
$\hbox{[{O~{}III}]}/\hbox{{\rm H}$\beta$}$ emission-line ratio and stellar
mass. In both panels the gray and red contours are the SDSS and CLEAR sample
respectively. The black points and arrows are $\hbox{[{O~{}III}]}/\hbox{{\rm
H}$\beta$}$ measurements and limits from NIRCam WFSS observations. The yellow
fit line is to the CEERS $z>5$ sample and randomly selected SDSS and CLEAR
galaxies matching the size of the CEERS sample. All the IllustrisTNG
simulations are shown as the purple region.  Figure 9: Left: The relationships
between the $\hbox{[{Ne~{}III}]}/\hbox{[{O~{}II}]}$ emission-line ratio and
the galaxy H$\beta$ luminosity and H$\beta$ SFR. Right: The relationships
between the $\hbox{[{Ne~{}III}]}/\hbox{[{O~{}II}]}$ emission-line ratio and
stellar mass. In both panels the gray and red contours are the SDSS and CLEAR
sample respectively. The black points and arrows are
$\hbox{[{O~{}III}]}/\hbox{{\rm H}$\beta$}$ measurements and limits from NIRCam
WFSS observations. The yellow fit line is to the CEERS $z>5$ sample and
randomly selected SDSS and CLEAR galaxies matching the size of the CEERS
sample. There is a significant ($>3\sigma$) relationship between
$\hbox{[{Ne~{}III}]}/\hbox{[{O~{}II}]}$ and H$\beta$ SFR and a marginal
relationship ($2.8\sigma$) with stellar mass.  Figure 10: Top Left: Comparison
of $\hbox{[{O~{}III}]}/\hbox{{\rm H}$\beta$}$ measurements of our sample to
MAPPINGS V models of ISM ionization and metallicities. The gray histogram
represents the distribution of the NIRSpec measurements of
$\hbox{[{O~{}III}]}/\hbox{{\rm H}$\beta$}$ while the blue histogram represents
the NIRCam WFSS sample, both CEERS samples do not have a measured metallicity.
We also include the four SMACS galaxies which have metallicity measurements
using the coronal [O III]$\lambda$4364\. Top Right: Comparison of
$\hbox{[{Ne~{}III}]}/\hbox{[{O~{}II}]}$ measurements of our sample to MAPPINGS
V models of the ISM’s ionization and metallicity. The gray histogram
represents the distribution of the NIRSpec measurements of
$\hbox{[{Ne~{}III}]}/\hbox{[{O~{}II}]}$. Bottom: Distribution of the CEERS
$z>5$ sample where the metallicities are derived using the strong-line
calibrations provided in Sanders et al. (2023b).

Figure 8 relates the $\hbox{[{O~{}III}]}/\hbox{{\rm H}$\beta$}$ ratio to
H$\beta$ SFR and stellar mass. Figure 9 similarly compares
$\hbox{[{Ne~{}III}]}/\hbox{[{O~{}II}]}$ to H$\beta$ SFR and stellar mass.
Equation 1 was used to derive H$\beta$ SFR, this is a proxy for the unobscured
galaxy SFR. Here our $z\sim 0$ SDSS and $z\sim 2$ CLEAR samples are
represented by the gray and red contours, respectively. Our NIRCam WFSS
galaxies are represented by black points for well measured galaxies and black
arrow for lower limits. NIRSpec galaxies are shown as pink points. In Figure 8
we include the median value of the Illustris similations as the purple line.
We calculate a best-fit line from the python linmix linear regression package,
using subsamples of SDSS galaxies and CLEAR galaxies the same size of the
CEERS sample.

We additionally compare our observed [O III]/H$\beta$ to the simulated ratios
of Hirschmann et al. (2023). These model line ratios are built on the
IllustrisTNG simulations (reference), applying both Cloudy (Ferland et al.
2013) and MAPPINGS V (Sutherland et al. 2018) photoionization models to the
simulated galaxies. These simulated galaxies include a mix of nebular emission
contributions from star-forming HII regions, post-AGB stars, shocks, and AGN
narrow-line regions. The details of these model line ratios are described by
Hirschmann et al. (2017) and Hirschmann et al. (2022). We compare to model
line ratios from simulated $z\sim 6$ galaxies with
$\log(M_{\star}/M_{\sun})\sim 9$ and
$\log(\mathrm{SFR}/[M_{\sun}/\mathrm{yr}])\sim 0.5$, matching the median
stellar mass and SFR of the observed galaxies. The range of line ratios from
these simulated galaxies are shown as purple shading in Figure 8.

The $\hbox{[{O~{}III}]}/\hbox{{\rm H}$\beta$}$ emission-line ratio has a
significant ($>$3$\sigma$) correlation with H$\beta$ SFR with a slope of
$0.26\pm 0.03$, and a significant anti-correlation with stellar mass with a
slope of $-0.33\pm 0.05$. Our $z>5$ NIRCam WFSS and NIRSpec sample are on
average 0.5 dex higher in H$\beta$ SFR than the CLEAR galaxies. The
IllustrisTNG simulated line ratios are effective at reproducing the highest [O
III]/H$\beta$ ratios observed in our sample, but most of our observed galaxies
have lower [O III]/H$\beta$ ratios than the simulated galaxies of similar
galaxy mass and SFR. We attempted to do multiple linear regression on our
sample to detangle the relationship between the $\hbox{[{O~{}III}]}/\hbox{{\rm
H}$\beta$}$ emission line with redshift, H$\beta$ SFR, and stellar mass
however due to the sample size and the differences between the NIRSpec and
NIRCam WFSS samples produced unreliable results. A larger sample size would be
needed for further analysis.

Figure 11: Theoretical OHNO line ratio predictions from the MAPPINGS V models
(colored lines) compared to the low-redshift SDSS galaxies (gray contours),
cosmic noon CLEAR galaxies (black points), SMACS galaxies (red stars), CEERS
AGN (purple stars) and CEERS NIRSpec galaxies (pink points). Due to the
constraint of the filter only $\hbox{[{O~{}III}]}/\hbox{{\rm H}$\beta$}$ can
be observed for the NIRCam WFSS observations. These galaxies are represented
by the histogram in the right hand panel, the gray histogram represents the
NIRCam WFSS measurements with a lower limit and red histogram are galaxies
with a S/N measurement$>3$ for both emission lines. The arrow on this
histogram indicates that the true [O III]/H$\beta$ values may be higher. The
red histogram in the panel represents the well measured galaxies from the
NIRCam WFSS, this sample is also represented by the pink bar across the panel.
Model ionization is indicated by the color of the line, and metallicity
decreases from left to right for each set of connected model points. Inset
vectors indicate the direction and amplitude of 1 dex increases in metallicity
and pressure. The black dashed line is the empirical AGN/SF dividing line
defined for $z\sim 2$ galaxies in Backhaus et al. (2022). The CEERS data set
is best described by moderate/high ionization and a moderate metallicity,
while CLEAR is described by lower ionization and more moderate metallicity.

The $\hbox{[{Ne~{}III}]}/\hbox{[{O~{}II}]}$ ratio has a significant
correlation with H$\beta$ SFR with a slope of $0.23\pm 0.04$, which is similar
to $\hbox{[{O~{}III}]}/\hbox{{\rm H}$\beta$}$. Similar to
$\hbox{[{O~{}III}]}/\hbox{{\rm H}$\beta$}$,
$\hbox{[{Ne~{}III}]}/\hbox{[{O~{}II}]}$ has a marginal (2$\sigma$)
anticorrelation to stellar mass with a slope of $-0.2\pm 0.07$, but is about
three times less significant what was found between stellar mass and
$\hbox{[{O~{}III}]}/\hbox{{\rm H}$\beta$}$.

The anti-correlation between $\log(\hbox{[{O~{}III}]}/\hbox{{\rm H}$\beta$})$
and stellar mass has been shown in Dickey et al. (2016) and Kashino et al.
(2019), and is due to lower metallicity and higher ionization in galaxies with
higher specific star formation rates (sSFR). This was also shown in
simulations done in Hirschmann et al. (2017) and Hirschmann et al. (2022).

##  6 ISM conditions at $z>5$

We compare our measurements of $\hbox{[{O~{}III}]}/\hbox{{\rm H}$\beta$}$ and
$\hbox{[{Ne~{}III}]}/\hbox{[{O~{}II}]}$ to theoretical models to infer the
physical conditions of the ISM. The models we compare to come from Kewley et
al. (2019) which uses both the Starburst99 (Leitherer et al. 1999) models of
stellar ionizing spectra and the MAPPINGS V ionization code (Sutherland et al.
2018). The Starburst99 model spectra includes mass loss and uses a Salpeter
(1955) IMF. The atomic data used in MAPPINGS V comes from the CHIANTI 8
database (Dere et al. 1997; Del Zanna et al. 2015) which includes the effects
of excitation, dust depletion, recombination and photoionization in the model
HII regions. The “Pressure Models” of Kewley et al. (2019) describe synthetic
emission-line spectra created from different combinations of pressure
$\log(P/k)$, ionization $\log(q)$, and metallicity $Z/Z_{\odot}$. These models
interpolate between Starburst99 models and CHIANTI 8 data in order to match
the grid. We use the following values of ionization and metallicity:

  * •

Ionization $\log(q)=[7,8,9]$, units of cm s-1

  * •

Metallicity $Z/Z_{\odot}=[0.05,0.2,0.4,1.0]$

Due to neither emission line ratio varying significantly with pressure, we
choose $\log{P}=8$ for the models. We also note the $Z/Z_{\odot}=0.05$ bin is
extrapolated in the Starburst99 input spectra which causes those synthetic
spectra to be the least certain of the theoretical predictions.

In Figure 10 we compare our high redshift sample to the MAPPINGS V models. The
gray and blue histograms in both the [O III]/H$\beta$ and [Ne III]/[O II]
panel represent the NIRSpec and NIRCam WFSS emission-line ratio distribution,
respectively. The four SMACS galaxies that have metallicity measurements are
also included as red squares. These are compared to the MAPPINGS V models that
are represented by the three colored lines, where each color is a different
ionization and each point is a different metallicity. The rest of the [O
III]/H$\beta$ ratios cover a range of moderate to high ionization. The [Ne
III]/[O II] ratios are similarly well-matched to MAPPINGS V models for
moderate to high ionization.

Due to the degeneracy between ionization and metallicity in the MAPPINGS V
model prediction, it is difficult to infer ISM metallicity from the [O
III]/H$\beta$ ratio in Figure 10. To combat this we also make use of the OHNO
diagram in Figure 11, to give us a more narrow range of ISM conditions and
highlight the evolution of ISM conditions with redshift. The gray contours
represent the SDSS sample, the galaxies which are in the AGN region of the
diagram do have higher ionization. The black points are the CLEAR sample from
CLEAR. We include the measurements from the SMACS galaxies from Trump et al.
(2022) and the two NIRSpec AGN from Kocevski et al. (2022) as red squares and
purple stars respectively. The $z=8.7$ AGN observed by Larson et al. (2023) is
shown as a yellow star. Our sample is shown as pink circles representing the
NIRSpec sample above $z>5$ and a pink bar for NIRCam WFSS. This bar represents
the range of [O III]/H$\beta$ covered by the WFSS sample as no [Ne III]/[O II]
measurements were detected for these galaxies. The MAPPINGS V models are
represented the same way as in Figure 10. The inset arrow shows the typical
scale and direction of $\sim$1 dex changes in metallicity. From this we can
see our $z>5$ sample prefers a moderate to high ionization, $\log(q)=8,9$,
with a moderate metallicity, $Z/Z_{\odot}=0.2,0.4$. The MAPPINGS V models
indicate that ionization increases from the $z\sim 0$ SDSS sample to the
$z\sim 2$ CLEAR sample to the $z>5$ CEERS sample. We note that the three AGN
sources have a higher [Ne III]/[O II] ratio and are best described by MAPPINGS
V models with higher ionization.

##  7 Summary

In this work, we studied optical emission line ratios from $z\sim 0$ to $z\sim
9$ using SDSS, CLEAR, and CEERS data sets. We have used NIRCam WFSS to define
two samples measuring H$\alpha$ of 18 galaxies at $3.9<z<4.9$ and
$\hbox{[{O~{}III}]}/\hbox{{\rm H}$\beta$}$ of 19 galaxies at $5.5<z<6.7$. This
sample was found by first setting a constraint on the photometric redshifts,
before visually inspecting for the emission lines. We also have three samples
using NIRSpec observations to give additional measurements of H$\alpha$ in 93
galaxies, $\hbox{[{O~{}III}]}/\hbox{{\rm H}$\beta$}$ of 96 galaxies, and a
$\hbox{[{Ne~{}III}]}/\hbox{[{O~{}II}]}$ for 59 gal. The NIRSpec sample is
selected with a $\mathrm{S/N}>2$ in the emission lines of interest, before
being visually inspected.

We studied these emission lines and summarize our results as follows:

  * •

Our CEERS samples cover a redshift range of $2<z<9$ and show a significant
($3\sigma$) correlation between $\hbox{[{O~{}III}]}/\hbox{{\rm H}$\beta$}$
with redshift, $0.06\pm 0.02$. There was also a marginal $2\sigma$ correlation
with $\hbox{[{Ne~{}III}]}/\hbox{[{O~{}II}]}$ with redshift, $0.05\pm 0.02$.
When looking at the H$\alpha$ SFR a very strong and significant correlation
with a slope of $0.24\pm 0.03$, with redshift was found.

  * •

We see a 0.33 dex increase in $\hbox{[{O~{}III}]}/\hbox{{\rm H}$\beta$}$, a
0.37 dex increase in $\hbox{[{Ne~{}III}]}/\hbox{[{O~{}II}]}$, and a 0.5 dex
increase with H$\alpha$ SFR when comparing our CEERS $z>5$ sample to our
$z\sim 2$ sample from CLEAR .

  * •

We found [O III]/H$\beta$ and [Ne III]/[O II] both have a significant
correlations with H$\beta$ SFR, with slopes of $0.26\pm 0.03$ and $0.23\pm
0.04$ respectively. [O III]/H$\beta$ is also shown to have a significant
anticorrelation with stellar mass with a slope of $-0.36\pm 0.05$, while [Ne
III]/[O II] has a marginal anticorrelation with a slope of $-0.2\pm 0.07$.

  * •

The IllustrisTNG simulations match the measurements of our highest [O
III]/H$\beta$ values when we compare them to our observed galaxies at similar
redshift, SFR, and stellar masses.

  * •

When comparing our high redshift $z>5$ sample to MAPPINGS V model spectra we
found they are best described by high ionization with moderate metallicity.
Comparing the CEERS line ratios with $z\sim 0$ and $z\sim 2$ samples indicates
that the ISM ionization increases with increasing redshift.

These results indicate how emission-line ratios are evolving with redshift. We
found that the galaxies’ physical conditions at $z>5$ have higher SFRs and
lower stellar mass. These galaxy properties are explained by higher ionization
and lower metallicities.

Larger samples of $z>5$ galaxies with JWST spectroscopy are needed to better
disentangle the relationship between ISM conditions and galaxy properties at
cosmic dawn. The NIRCam WFSS observations are very useful as a blind survey
that includes galaxies missed by targeted NIRSpec observations, but the
single-filter NIRCam WFSS observations in CEERS results in a limited
wavelength range that includes only a single emission-line ratio for these
galaxies. Multi-filter NIRCam WFSS observations would provide broader
wavelength coverage and more effectively probe the ISM conditions for blind
and representative samples of emission-line galaxies in the early Universe.

##  8 Acknowledgements

We acknowledge the work of our colleagues in the CEERS collaboration and
everyone involved in the JWST mission. BEB and JRT acknowledge support from
NASA grants JWST-ERS-01345, JWST-AR-01721, and NSF grant CAREER-1945546.

## References

  * Ahn et al. (2014) Ahn, C. P., Alexandroff, R., Allende Prieto, C., et al. 2014, ApJS, 211, 17
  * Aird et al. (2010) Aird, J., Nandra, K., Laird, E. S., et al. 2010, MNRAS, 401, 2531
  * Arrabal Haro et al. (2023) Arrabal Haro, P., Dickinson, M., Finkelstein, S. L., et al. 2023, arXiv e-prints, arXiv:2303.15431
  * Backhaus et al. (2022) Backhaus, B. E., Trump, J. R., Cleri, N. J., et al. 2022, ApJ, 926, 161
  * Bagley et al. (2023) Bagley, M. B., Finkelstein, S. L., Koekemoer, A. M., et al. 2023, ApJ, 946, L12
  * Baldwin et al. (1981) Baldwin, J. A., Phillips, M. M., & Terlevich, R. 1981, PASP, 93, 5
  * Bolton et al. (2012) Bolton, A. S., Schlegel, D. J., Aubourg, É., et al. 2012, AJ, 144, 144
  * Brammer et al. (2008) Brammer, G. B., van Dokkum, P. G., & Coppi, P. 2008, ApJ, 686, 1503
  * Brammer et al. (2012) Brammer, G. B., van Dokkum, P. G., Franx, M., et al. 2012, ApJS, 200, 13
  * Brinchmann (2022) Brinchmann, J. 2022, arXiv e-prints, arXiv:2208.07467
  * Brinchmann et al. (2004) Brinchmann, J., Charlot, S., White, S. D. M., et al. 2004, MNRAS, 351, 1151
  * Bruzual & Charlot (2003) Bruzual, G., & Charlot, S. 2003, MNRAS, 344, 1000
  * Calzetti et al. (2000) Calzetti, D., Armus, L., Bohlin, R. C., et al. 2000, ApJ, 533, 682
  * Calzetti & Heckman (1999) Calzetti, D., & Heckman, T. M. 1999, ApJ, 519, 27
  * Cardelli et al. (1989) Cardelli, J. A., Clayton, G. C., & Mathis, J. S. 1989, ApJ, 345, 245
  * Chabrier (2003) Chabrier, G. 2003, Publications of the Astronomical Society of the Pacific, 115, 763
  * Charlot & Fall (2000) Charlot, S., & Fall, S. M. 2000, ApJ, 539, 718
  * Cleri et al. (2023a) Cleri, N. J., Yang, G., Papovich, C., et al. 2023a, ApJ, 948, 112
  * Cleri et al. (2023b) Cleri, N. J., Olivier, G. M., Hutchison, T. A., et al. 2023b, arXiv e-prints, arXiv:2301.07745
  * Coil et al. (2015) Coil, A. L., Aird, J., Reddy, N., et al. 2015, ApJ, 801, 35
  * Conroy & Gunn (2010) Conroy, C., & Gunn, J. E. 2010, ApJ, 712, 833
  * Conroy et al. (2009) Conroy, C., Gunn, J. E., & White, M. 2009, ApJ, 699, 486
  * Curti et al. (2023) Curti, M., D’Eugenio, F., Carniani, S., et al. 2023, MNRAS, 518, 425
  * Curtis-Lake et al. (2023) Curtis-Lake, E., Carniani, S., Cameron, A., et al. 2023, Nature Astronomy, arXiv:2212.04568
  * Davis et al. (2007) Davis, M., Guhathakurta, P., Konidaris, N. P., et al. 2007, ApJ, 660, L1
  * Del Zanna et al. (2015) Del Zanna, G., Dere, K. P., Young, P. R., Landi, E., & Mason, H. E. 2015, A&A, 582, A56
  * Dere et al. (1997) Dere, K. P., Landi, E., Mason, H. E., Monsignori Fossi, B. C., & Young, P. R. 1997, A&AS, 125, 149
  * Dickey et al. (2016) Dickey, C. M., van Dokkum, P. G., Oesch, P. A., et al. 2016, ApJ, 828, L11
  * Eldridge & Stanway (2009) Eldridge, J. J., & Stanway, E. R. 2009, MNRAS, 400, 1019
  * Ferland et al. (2013) Ferland, G. J., Porter, R. L., van Hoof, P. A. M., et al. 2013, Rev. Mexicana Astron. Astrofis., 49, 137
  * Finkelstein et al. (2022) Finkelstein, S. L., Bagley, M., Song, M., et al. 2022, ApJ, 928, 52
  * Finkelstein et al. (2023) Finkelstein, S. L., Bagley, M. B., Ferguson, H. C., et al. 2023, ApJ, 946, L13
  * Fujimoto et al. (2023) Fujimoto, S., Kohno, K., Ouchi, M., et al. 2023, arXiv e-prints, arXiv:2303.01658
  * Greene et al. (2017) Greene, T. P., Kelly, D. M., Stansberry, J., et al. 2017, Journal of Astronomical Telescopes, Instruments, and Systems, 3, 035001
  * Grogin et al. (2011) Grogin, N. A., Kocevski, D. D., Faber, S. M., et al. 2011, ApJS, 197, 35
  * Henry et al. (2013) Henry, A., Scarlata, C., Domínguez, A., et al. 2013, ApJ, 776, L27
  * Hirschmann et al. (2017) Hirschmann, M., Charlot, S., Feltre, A., et al. 2017, MNRAS, 472, 2468
  * Hirschmann et al. (2023) Hirschmann, M., Charlot, S., & Somerville, R. S. 2023, arXiv e-prints, arXiv:2305.03753
  * Hirschmann et al. (2022) Hirschmann, M., Charlot, S., Feltre, A., et al. 2022, arXiv e-prints, arXiv:2212.02522
  * Jakobsen et al. (2022) Jakobsen, P., Ferruit, P., Alves de Oliveira, C., et al. 2022, A&A, 661, A80
  * Jung et al. (2023) Jung, I., Finkelstein, S. L., Arrabal Haro, P., et al. 2023, arXiv e-prints, arXiv:2304.05385
  * Kashino et al. (2019) Kashino, D., Silverman, J. D., Sanders, D., et al. 2019, ApJS, 241, 10
  * Kauffmann et al. (2003a) Kauffmann, G., Heckman, T. M., Tremonti, C., et al. 2003a, MNRAS, 346, 1055
  * Kauffmann et al. (2003b) Kauffmann, G., Heckman, T. M., White, S. D. M., et al. 2003b, MNRAS, 341, 33
  * Kauffmann et al. (2003c) —. 2003c, MNRAS, 341, 54
  * Kennicutt & Evans (2012) Kennicutt, R. C., & Evans, N. J. 2012, ARA&A, 50, 531
  * Kewley et al. (2006) Kewley, L. J., Groves, B., Kauffmann, G., & Heckman, T. 2006, MNRAS, 372, 961
  * Kewley et al. (2019) Kewley, L. J., Nicholls, D. C., Sutherland, R., et al. 2019, ApJ, 880, 16
  * Kocevski et al. (2022) Kocevski, D. D., Barro, G., McGrath, E. J., et al. 2022, arXiv e-prints, arXiv:2208.14480
  * Koekemoer et al. (2011) Koekemoer, A. M., Faber, S. M., Ferguson, H. C., et al. 2011, ApJS, 197, 36
  * Kriek et al. (2009) Kriek, M., van Dokkum, P. G., Labbé, I., et al. 2009, ApJ, 700, 221
  * Kriek et al. (2015) Kriek, M., Shapley, A. E., Reddy, N. A., et al. 2015, The Astrophysical Journal Supplement Series, 218, 15
  * Kroupa (2001) Kroupa, P. 2001, MNRAS, 322, 231
  * Larson et al. (2023) Larson, R. L., Finkelstein, S. L., Kocevski, D. D., et al. 2023, arXiv e-prints, arXiv:2303.08918
  * Leitherer et al. (1999) Leitherer, C., Schaerer, D., Goldader, J. D., et al. 1999, ApJS, 123, 3
  * Liu et al. (2008) Liu, X., Shapley, A. E., Coil, A. L., Brinchmann, J., & Ma, C.-P. 2008, ApJ, 678, 758
  * Madau & Dickinson (2014) Madau, P., & Dickinson, M. 2014, ARA&A, 52, 415
  * McLean et al. (2012) McLean, I. S., Steidel, C. C., Epps, H. W., et al. 2012, in Society of Photo-Optical Instrumentation Engineers (SPIE) Conference Series, Vol. 8446, Ground-based and Airborne Instrumentation for Astronomy IV, ed. I. S. McLean, S. K. Ramsay, & H. Takami, 84460J
  * Momcheva et al. (2016) Momcheva, I. G., Brammer, G. B., van Dokkum, P. G., et al. 2016, ApJS, 225, 27
  * Montero-Dorta et al. (2016) Montero-Dorta, A. D., Bolton, A. S., Brownstein, J. R., et al. 2016, MNRAS, 461, 1131
  * Osterbrock (1989) Osterbrock, D. E. 1989, Astrophysics of gaseous nebulae and active galactic nuclei (University Science Books)
  * Papovich et al. (2022) Papovich, C., Simons, R. C., Estrada-Carpenter, V., et al. 2022, ApJ, 937, 22
  * Pirzkal et al. (2017) Pirzkal, N., Malhotra, S., Ryan, R. E., et al. 2017, ApJ, 846, 84
  * Planck Collaboration et al. (2016) Planck Collaboration, Ade, P. A. R., Aghanim, N., et al. 2016, A&A, 594, A13
  * Pontoppidan et al. (2022) Pontoppidan, K. M., Barrientes, J., Blome, C., et al. 2022, ApJ, 936, L14
  * Reddy et al. (2016) Reddy, N. A., Steidel, C. C., Pettini, M., & Bogosavljević, M. 2016, ApJ, 828, 107
  * Reddy et al. (2023) Reddy, N. A., Topping, M. W., Sanders, R. L., Shapley, A. E., & Brammer, G. 2023, arXiv e-prints, arXiv:2303.11397
  * Salpeter (1955) Salpeter, E. E. 1955, ApJ, 121, 161
  * Sanders et al. (2023a) Sanders, R. L., Shapley, A. E., Topping, M. W., Reddy, N. A., & Brammer, G. B. 2023a, arXiv e-prints, arXiv:2301.06696
  * Sanders et al. (2023b) —. 2023b, arXiv e-prints, arXiv:2303.08149
  * Sanders et al. (2021) Sanders, R. L., Shapley, A. E., Jones, T., et al. 2021, ApJ, 914, 19
  * Schaerer et al. (2022) Schaerer, D., Marques-Chaves, R., Barrufet, L., et al. 2022, arXiv e-prints, arXiv:2207.10034
  * Shapley et al. (2023a) Shapley, A. E., Reddy, N. A., Sanders, R. L., Topping, M. W., & Brammer, G. B. 2023a, arXiv e-prints, arXiv:2303.00410
  * Shapley et al. (2023b) Shapley, A. E., Sanders, R. L., Reddy, N. A., Topping, M. W., & Brammer, G. B. 2023b, arXiv e-prints, arXiv:2301.03241
  * Shapley et al. (2015) Shapley, A. E., Reddy, N. A., Kriek, M., et al. 2015, ApJ, 801, 88
  * Shapley et al. (2019) Shapley, A. E., Sanders, R. L., Shao, P., et al. 2019, ApJ, 881, L35
  * Simons et al. (2023) Simons, R. C., Papovich, C., Momcheva, I. G., et al. 2023, ApJS, 266, 13
  * Smee et al. (2013) Smee, S. A., Gunn, J. E., Uomoto, A., et al. 2013, AJ, 146, 32
  * Speagle et al. (2014) Speagle, J. S., Steinhardt, C. L., Capak, P. L., & Silverman, J. D. 2014, ApJS, 214, 15
  * Steidel et al. (2016) Steidel, C. C., Strom, A. L., Pettini, M., et al. 2016, ApJ, 826, 159
  * Steidel et al. (2014) Steidel, C. C., Rudie, G. C., Strom, A. L., et al. 2014, ApJ, 795, 165
  * Strom et al. (2018) Strom, A. L., Steidel, C. C., Rudie, G. C., Trainor, R. F., & Pettini, M. 2018, ApJ, 868, 117
  * Sutherland et al. (2018) Sutherland, R., Dopita, M., Binette, L., & Groves, B. 2018, MAPPINGS V: Astrophysical plasma modeling code, , , ascl:1807.005
  * Tang et al. (2023) Tang, M., Stark, D. P., Chen, Z., et al. 2023, arXiv e-prints, arXiv:2301.07072
  * Trump et al. (2022) Trump, J. R., Arrabal Haro, P., Simons, R. C., et al. 2022, arXiv e-prints, arXiv:2207.12388
  * van Dokkum et al. (2011) van Dokkum, P. G., Brammer, G., Fumagalli, M., et al. 2011, ApJ, 743, L15
  * Veilleux & Osterbrock (1987) Veilleux, S., & Osterbrock, D. E. 1987, ApJS, 63, 295
  * York et al. (2000) York, D. G., Adelman, J., Anderson, John E., J., et al. 2000, AJ, 120, 1579
  * Zeimann et al. (2015) Zeimann, G. R., Ciardullo, R., Gebhardt, H., et al. 2015, ApJ, 798, 29


# Universal Knowledge Graph Embeddings

N’Dah Jean Kouagou  Caglar Demir  Hamada M. Zahera  Adrian Wilke
Stefan Heindorf  Jiayi Li  Axel-Cyrille Ngonga Ngomo
Department of Computer Science, Paderborn University, Germany
{ndah.jean.kouagou, caglar.demir, hamada.zahera, adrian.wilke, heindorf,
jiayili<EMAIL_ADDRESS>
###### Abstract

A variety of knowledge graph embedding approaches have been developed. Most of
them obtain embeddings by learning the structure of the knowledge graph within
a link prediction setting. As a result, the embeddings reflect only the
semantics of a single knowledge graph, and embeddings for different knowledge
graphs are not aligned, e.g., they cannot be used to find similar entities
across knowledge graphs via nearest neighbor search. However, knowledge graph
embedding applications such as entity disambiguation require a more global
representation, i.e., a representation that is valid across multiple sources.
We propose to learn universal knowledge graph embeddings from large-scale
interlinked knowledge sources. To this end, we fuse large knowledge graphs
based on the owl:sameAs relation such that every entity is represented by a
unique identity. We instantiate our idea by computing universal embeddings
based on DBpedia and Wikidata yielding embeddings for about 180 million
entities, 15 thousand relations, and 1.2 billion triples. Moreover, we develop
a convenient API to provide embeddings as a service. Experiments on link
prediction show that universal knowledge graph embeddings encode better
semantics compared to embeddings computed on a single knowledge graph. For
reproducibility purposes, we provide our source code and datasets open access
at https://github.com/dice-group/Universal_Embeddings

##  1 Introduction

Knowledge graph embedding (KGE) models are used for a multitude of
applications (?; ?), including link prediction, triple classification, entity
classification, and entity resolution. Several approaches to compute KGEs have
been proposed in the literature, e.g., TransE (?), RDF2Vec (?), and ComplEx
(?).

While pretrained models for few knowledge graphs (KGs) are available, their
embedding spaces are not aligned, i.e., same entities have different
representations across different knowledge graphs. As a result, the usability
of such embeddings is often limited to downstream tasks on the KG they were
trained on (?). However, a growing number of real-world applications of KG
embeddings (e.g., entity resolution) require entities to have a consistent
representation across different sources.

The need for these unified representations for entities recently motivated
several works (?; ?; ?; ?; ?; ?). Some of these approaches are tailored
towards multi-lingual KG embeddings, i.e., the task of computing aligned
embeddings between multiple language versions of the same KG by relying on the
available owl:sameAs links (?; ?; ?). Other approaches employ a bootstrapping
strategy based on the matching scores between entities during training (?) or
use additional information on entities such as attribute embeddings (?; ?).
Although current alignment approaches for KGs have shown promising results on
benchmark datasets, they inherently suffer from scalability issues. This is
corroborated by the lack of pretrained KG embeddings for large datasets such
as Wikidata (?) and DBpedia (?) using the aforementioned approaches. Moreover,
most entity alignment approaches can only handle two KGs at a time, and do not
assign the same embedding vector to matching entities111Two entities are
matching if they correspond to the same real world entity.

In this paper, we merge a given set of KGs into a single KG to compute
embeddings that capture comprehensive information about each entity. We call
these embeddings _universal knowledge graph embeddings_. By assigning a unique
ID to all matching entities, we not only reduce memory consumption and
computation costs but also tackle KG incompleteness—a well-known issue in the
research community (?; ?; ?). For instance, Wikidata and DBpedia contain 360
and 138 triples about Iraq, respectively. Consequently, a traditional KGE
model trained on either KG can only capture incomplete information about their
shared entities. Our approach mitigates this limitation by integrating
information from both KGs into the embeddings.

To quantify the quality of our embeddings, we apply our approach to 4 KGE
models, i.e., DistMult(?), ComplEx(?), QMult (?), and ConEx (?), and evaluate
their performance on link prediction. Overall, our results suggest that the
benefits of using additional information derived from sameAs links become
particularly noticeable in ConEx. We use the latter to compute and provide
high-quality unified embeddings for the most populous KGs of the Linked Open
Data (LOD) cloud222https://lod-cloud.net, i.e., DBpedia and Wikidata. The
merged graph encompasses about 180 million entities, 15 thousand relations,
and 1.2 billion triples. Moreover, we develop an API333https://embeddings.cc/
with convenient methods to make the computed embeddings easily accessible.

##  2 Related Work

A knowledge graph embedding (KGE) model denoted by $\phi$ maps a knowledge
graph (KG) into a continuous vector space—commonly by solving an optimization
problem. This optimization aims to preserve the structural information of the
input KG. For example, transitional distance models such as TransE (?) compute
embeddings by modelling each triple as a translation between its head and tail
entities. Specifically, TransE represents both entities and relations as
vectors in the same semantic space and learns embeddings by minimizing the
distance between $h+r$ and $t$ for every triple $(h,r,t)$ in the considered
KG. DistMult (?) adopts the scoring technique of TransE but uses
multiplications to model entiy and relation interactions. Other types of KG
embedding models include ComplEx (?), ConEx (?) and RESCAL (?). ComplEx and
ConEx model entities and relations as complex vectors (i.e. with real and
imaginary parts) to handle both symmetric and antisymmetric relations. As
ComplEx cannot handle transitive relations (see ? ?), ConEx further improves
on ComplEx by applying a 2D convolution operation on complex-valued embeddings
of head entities and relations. By associating each relation with a matrix,
RESCAL captures pairwise interactions between entities and is regarded as one
of the most expressive models (?). As KGs grow in size, computation-efficient
algorithms are required to train KGs consisting of millions of entities and
billions of triples. ? ? developed DGL-KE, an open-source package that employs
several optimization techniques to accelerate training on large KGs. For
example, they partition a large KG to perform gradient updates on each
partition and regularly fetch embeddings from other partitions which involves
a significant communication overhead.

Although KG embeddings can benefit downstream tasks such as link prediction
and KG completion, their successful application is often limited to the KGs
they were trained on. As a result, applications to other tasks, such as
_entity resolution on two or multiple KGs_ , require aligned KG embeddings.

Algorithm 1 Function FuseKGs

0: $\mathcal{G}_{1}$, $\mathcal{G}_{2}$, $\dots$, $\mathcal{G}_{N}$

0: $\mathcal{G}^{*}$ # merged knowledge graphs

1: $\mathcal{G}^{*}\leftarrow\mathcal{G}_{1}$ # Initialization

2: for $i=2,\dots,N$ do

3: for ($e_{1}$, $r$, $e_{2}$) $\in$ $\mathcal{G}_{i}$ do

4: if $\exists\ e^{*}_{1}\in\mathcal{E}_{\mathcal{G}^{*}}$ such that
$e_{1}\texttt{:sameAs:}e^{*}_{1}$ then

5: $e_{1}\leftarrow e^{*}_{1}$ # Rename entity $e_{1}$

6: end if

7: if $\exists\ e^{*}_{2}\in\mathcal{E}_{\mathcal{G}^{*}}$ such that
$e_{2}\texttt{:sameAs:}e^{*}_{2}$ then

8: $e_{2}\leftarrow e^{*}_{2}$ # Rename entity $e_{2}$

9: end if

10: Add ($e_{1}$, $r$, $e_{2}$) to $\mathcal{G}^{*}$

11: end for

12: end for

13: return $\mathcal{G}^{*}$

##  3 Universal Knowledge Graph Embeddings

###  3.1 Preliminaries

A KG $\mathcal{G}$ can be regarded as a set of triples
$\mathcal{G}=\\{(e_{1}^{(i)},r^{(i)},e_{2}^{(i)})\\}_{i=1}^{n}\subseteq\mathcal{E}_{\mathcal{G}}\times\mathcal{R}_{\mathcal{G}}\times\mathcal{E}_{\mathcal{G}}$,
where $\mathcal{E}_{\mathcal{G}}$ and $\mathcal{R}_{\mathcal{G}}$ represent
its sets of entities and relations, respectively. When there is no ambiguity,
we simply write $\mathcal{E}$ and $\mathcal{R}$. Let
$\mathcal{G}_{1},\dots,\mathcal{G}_{N}$ denote $N$ KGs (in an arbitrary
order), e.g., $\text{DBpedia},\text{Wikidata},\text{Freebase}$. Alignments
between $\mathcal{G}_{i}$ and $\mathcal{G}_{j}$ are given by sameAs links. We
use these links to fuse the given KGs as described in the next section.

###  3.2 Graph Fusion and Embedding Computation

In this work, we fuse all KGs $\mathcal{G}_{i}$ for $i=1,\dots,N$ into a
single KG $\mathcal{G}^{*}$ where all aligned entities are represented by a
unique ID. Algorithm 1 describes how the fusion is carried out. First, the
algorithm chooses a reference KG444It does not matter which KG is chosen as we
end up with the same number of triples in any case. (in this work, we select
$\mathcal{G}_{1}$) as the initial set of triples for $\mathcal{G}^{*}$ (line
1). Then, it iterates over the rest of the KGs (line 2) and adds their triples
(line 10). In this process, entities that are already present in
$\mathcal{G}^{*}$ via sameAs links are renamed accordingly (lines 3–9). Once
$\mathcal{G}^{*}$ is constructed, a KGE model (e.g. ConEx) can be applied to
learn universal embeddings.

Table 1: Statistics of the full datasets for _universal knowledge graph embeddings_. Deg. denotes the average degree of entities. Dataset | $|\mathcal{E}|$ | $|\mathcal{R}|$ | $|\mathcal{G}|$ | $|\texttt{sameAs}|$ | Deg.
---|---|---|---|---|---
DBpedia | 91,684,304 | 13,783 |  0616,564,603 | 33,860,047 | 13.45
Wikidata | 94,468,182 |  01,436 |  0667,666,110 | 33,860,047 | 14.14
MERGE | 179,706,494 | 15,219 | 1,284,230,713 | 33,860,047 | 14.29

###  3.3 Knowledge Graphs

We downloaded and preprocessed the September 2022 version of DBpedia (?) and
the March 2022 version of Wikidata (?). In this work, we only consider the
English version of DBpedia, and its external links to Wikidata, i.e., sameAs
links. The preprocessing step is concerned with the removal of non triplet-
formatted files and literals.

#### DBpedia ($\mathcal{G}_{1}$).

DBpedia555https://databus.dbpedia.org/dbpedia/collections/dbpedia-
snapshot-2022-09/ is the most popular and prominent KG in the LOD. It is
automatically created based on Wikipedia information, such as infobox tables,
categorizations, and links to external websites. Since DBpedia serves as the
hub for LOD, it contains many links to other LOD datasets such as Freebase,
Caligraph, and Wikidata.

#### Wikidata ($\mathcal{G}_{2}$).

Wikidata666https://dumps.wikimedia.org/wikidatawiki/entities/ is a community-
created knowledge base providing factual information to Wikipedia and other
projects by the Wikimedia Foundation. As of April 2022, Wikidata contains over
$97$ million items and $1.37$ billion statements. Each item page contains
labels, short descriptions, aliases, statements, and site links. Each
statement consists of a claim and an optional reference, and each claim
consists of a property-value pair and optional qualifiers.

#### Statistics of Knowledge Graphs.

Table 1 presents the statistics of DBpedia and Wikidata after our
preprocessing step. MERGE is obtained by applying Algorithm 1 to {DBpedia,
Wikidata}. In a normal scenario, the sum of the numbers of entities of DBpedia
and Wikidata should be equal to that of MERGE and sameAs links. However, some
entities in DBpedia were matched with multiple entities in Wikidata via sameAs
links, and vice versa. This caused the equality not to hold as can be seen in
the table.

##  4 Experiments

We conduct our experiments to answer one fundamental question: ”How do our
universal knowledge embeddings compare to embeddings from traditional KGE
approaches?” To this end, we set up a link prediction task where we compare
two independently trained instances of the same embedding model (see the next
sections for more details).

Table 2: Statistics of evaluation datasets. Deg. is the average degree of entities.  Dataset | $|\mathcal{E}|$ | $|\mathcal{R}|$ | $|\mathcal{G}|$ | $|\texttt{sameAs}|$ | Deg.
---|---|---|---|---|---
DBpediatrain | 31,116 |  0 392 |  069,667 | 22,102 | 4.48
DBpediatest | 15,602 |  0 279 |  015,374 | 10,471 | 1.97
Wikidatatrain | 72,058 |  0 707 | 235,814 | 22,102 | 6.55
Wikidatatest | 41,137 |  0 465 |  053,761 | 10,471 | 2.61
MERGEtrain | 81,836 | 1,099 | 305,481 | 22,102 | 7.47

###  4.1 Evaluation Setup

#### Evaluation Datasets.

We conduct our experiments on subsets of DBpedia and Wikidata, due to the
computational complexity of our evaluation metrics. Specifically, we randomly
select 1% of entities in DBpedia that share sameAs links to Wikidata, then we
obtain their 1-hop neighborhood together with the corresponding relation
types. We then analogously compute the corresponding subset of Wikidata by
using entities identified by the 1% initially selected in DBpedia. The samples
we obtain are then randomly split into training and test datasets. Overall, we
obtain five datasets for our experiments: DBpediatrain and DBpediatest (split
of the DBpedia sample), Wikidatatrain and Wikidatatest (split of the Wikidata
sample), and MERGEtrain (merge of DBpediatrain and Wikidatatrain using
Algorithm 1). For the sake of clarity, we use the notations DBpedia+ and
Wikidata+ to refer to MERGEtrain depending on whether we are evaluating
embedding models on DBpedia or Wikidata. Note that the splits are performed in
a way that all entities and relation types in training datasets also appear in
the test datasets. This allows us to not encounter out-of-vocabulary entities
and relations at inference time. The sizes of the splits are specified in
Table 2. The average degree abbreviated as Deg. represents the average number
of edges connected to an entity.

#### Metrics.

We use two standard metrics to evaluate KGEs: hits@$k$ (H@$k$) and mean
reciprocal rank (MRR). Formally, let $\mathcal{G}$ be a knowledge graph, i.e.,
a set of triples. We denote by $\mathit{rank}[e_{1}|r,e_{2}]$ the rank of the
score of $e_{1}$ given the relation $r$ and the tail entity $e_{2}$ among the
set of all scores $\\{\mathit{score}(e|r,e_{2})\text{ s. t.
}e\in\mathcal{E}_{\mathcal{G}}\\}$. Similarly, $\mathit{rank}[e_{2}|e_{1},r]$
denotes the rank of the score of $e_{2}$ given the head entity $e_{1}$ and the
relation $r$ among $\\{\mathit{score}(e|e_{1},r)\text{ s. t.
}e\in\mathcal{E}_{\mathcal{G}}\\}$. We define the metrics Hits@$k$ and MRR as

$\displaystyle\text{Hits@}k=$
$\displaystyle\frac{1}{2|\mathcal{G}|}\sum_{(e_{1},r,e_{2})\in\mathcal{G}}\mathbbm{1}(\mathit{rank}[e_{1}|r,e_{2}]\leq
k)$ $\displaystyle+$
$\displaystyle\mathbbm{1}(\mathit{rank}[e_{2}|e_{1},r]\leq k),$ (1)
$\displaystyle\textsc{MRR}=$
$\displaystyle\frac{1}{2|\mathcal{G}|}\sum_{(e_{1},r,e_{2})\in\mathcal{G}}\frac{1}{\mathit{rank}[e_{1}|r,e_{2}]}$
$\displaystyle+$ $\displaystyle\frac{1}{\mathit{rank}[e_{2}|e_{1},r]}.$ (2)

#### Hardware.

The entire DBpedia and Wikidata datasets for which we provide embeddings as a
service were processed on a virtual machine (VM) with 128 CPUs (AMD EPYC 7742
64-Core Processor) and 1TB RAM. The computation of universal knowledge graph
embeddings was carried out using the DICE embedding framework (?) on a 1TB RAM
VM with 2 NVIDIA RTX A5000 GPUs of 24GB memory each.

Table 3: Link prediction results in terms of mean reciprocal rank (MRR) and hits@k (H@k). We compare the performance of each embedding model on the two types of datasets: single KG (DBpedia, Wikidata) and enriched KG (DBpedia+, Wikidata+). Hence, the bold values correspond to the best performance achieved horizontal-wise. All models use 32 embedding dimensions and have approximately the same number of parameters. | DBpedia | DBpedia+
---|---|---
| MRR | H@1 | H@3 | H@10 | MRR | H@1 | H@3 | H@10
ConEx-train | 0.371 | 0.298 | 0.396 | 0.502 | 0.898 | 0.863 | 0.924 | 0.962
ConEx-test | 0.181 | 0.134 | 0.191 | 0.269 | 0.349 | 0.280 | 0.378 | 0.478
ComplEx-train | 0.181 | 0.122 | 0.205 | 0.292 | 0.278 | 0.206 | 0.306 | 0.408
ComplEx-test | 0.136 | 0.103 | 0.143 | 0.194 | 0.163 | 0.123 | 0.170 | 0.237
QMult-train | 0.230 | 0.168 | 0.250 | 0.348 | 0.129 | 0.099 | 0.135 | 0.183
QMult-test | 0.148 | 0.112 | 0.155 | 0.213 | 0.064 | 0.041 | 0.070 | 0.113
DistMult-train | 0.145 | 0.100 | 0.152 | 0.243 | 0.125 | 0.094 | 0.133 | 0.183
DistMult-test | 0.118 | 0.084 | 0.125 | 0.177 | 0.057 | 0.032 | 0.060 | 0.109
| Wikidata | Wikidata+
| MRR | H@1 | H@3 | H@10 | MRR | H@1 | H@3 | H@10
ConEx-train | 0.795 | 0.730 | 0.842 | 0.911 | 0.855 | 0.806 | 0.889 | 0.941
ConEx-test | 0.280 | 0.227 | 0.302 | 0.376 | 0.299 | 0.247 | 0.324 | 0.392
ComplEx-train | 0.232 | 0.188 | 0.244 | 0.303 | 0.278 | 0.206 | 0.306 | 0.408
ComplEx-test | 0.126 | 0.088 | 0.140 | 0.197 | 0.166 | 0.123 | 0.181 | 0.247
QMult-train | 0.138 | 0.105 | 0.147 | 0.199 | 0.129 | 0.099 | 0.135 | 0.183
QMult-test | 0.077 | 0.053 | 0.085 | 0.122 | 0.092 | 0.069 | 0.100 | 0.135
DistMult-train | 0.190 | 0.145 | 0.207 | 0.263 | 0.125 | 0.094 | 0.133 | 0.183
DistMult-test | 0.107 | 0.075 | 0.118 | 0.165 | 0.086 | 0.060 | 0.096 | 0.136

###  4.2 Results and Discussion

In Table 3, we present the results of our experiments comparing the
performance of embedding models trained on a single KG against those trained
on merged KGs (by leveraging sameAs links as described in Algorithm 1). Four
embedding models are considered: ConEx (?) , ComplEx (?), QMult (?) and
DistMult (?). From the table, we can observe that ConEx achieves the highest
performance w.r.t. all metrics on all datasets. Moreover, its performance on
the merged KGs (DBpedia+ and Wikidata+) is notably higher as compared to that
on DBpedia and Wikidata. The ComplEx model also performs better on DBpedia+
and Wikidata+ than on DBpedia and Wikidata, respectively. On the other side,
DistMult and QMult perform poorly on both the training and test datasets.

One would expect that additional information added about entities in DBpedia+
and Wikidata+ improves the performance of embedding models on downstream tasks
such as link prediction. Although this is clearly the case for ConEx and
ComplEx (with up to 2$\times$ improvement for ConEx), we observed the opposite
on DistMult and QMult. Interestingly, the poor performing models correspond to
the extreme cases of model complexity, i.e., DistMult is the simplest and
QMult is the most expressive among the four models we considered. This
suggests that with 32 embedding dimensions, DistMult cannot learn meaningful
representations for entities and relations in our evaluation data due to its
simplicity. Likewise, QMult fails to find optimal representations of entities
and relations because it cannot encode its inherent high degree of freedom in
32 dimensions. The ConEx architecture appears to balance well between
expressiveness and the chosen number of embedding dimensions. In fact, our
preliminary experiments with 300 embedding dimensions ranked DistMult the top
best performing model ahead of ConEx and ComplEx at the cost of longer
training times and memory consumption. In view of this observation, we use
ConEx with 32 embedding dimensions to compute our universal embeddings for
large KGs and provide them on a platform (see next section). The answer to the
fundamental question behind our work is hence that _we can learn rich
embeddings on a KG that integrates information (about entities) from different
external sources, in particular other KGs_. A precondition to achieve this
goal is intrinsic to common challenges in representation learning: finding
fitting hyper-parameters.

##  5 Implementation of a Service Platform

We offer computed data as an open service, following the FAIR principles (?),
so that the universal embeddings are available to a broad audience. The
platform777https://embeddings.cc/ consists of a RESTful, TLS-secured API along
with a website and an interactive documentation. The API contains a hidden
webservice for developers to maintain data and a public webservice with eight
methods providing RDF entity identifiers and related embeddings.

Regarding the FAIR principles, the data is _findable_ as existing entity
identifiers are used and can be accessed and explored with an autocomplete
feature on the website. The big data amount is _accessible_ as data subsets
can be retrieved using API methods such as random or autocomplete, which
explore the full data. The accessibility is additionally enhanced by meta
queries like the size of the offered datasets. _Interoperability_ is given by
reusing existing RDF namespaces and identifiers. In addition, the API is
versioned and uses the lightweight JSON format in Python and JavaScript.

##  6 Conclusion and Outlook

In this paper, we discuss the challenges related to computing embeddings for
entities shared across multiple knowledge graphs. In particular, we note the
lack of such embeddings for large knowledge graphs and propose a simple but
effective approach to compute embeddings for shared entities. Given a
countable set of knowledge graphs, our approach iterates over all triples and
assigns a unique ID to all matching entities (i.e. shared entities). An
embedding model is then applied to learn embeddings on the resulting graph—our
embeddings are called universal knowledge graph embeddings. We use our
approach to compute embeddings for recent versions of DBpedia and Wikidata,
and provide them as an open service via a convenient API. Experiments on link
prediction suggest that our universal embeddings are better than those
computed on separate knowledge graphs. Regarding the API, we currently provide
embeddings via autocomplete search and random entity selection. In future
releases, we will integrate an approximation of embedding-level nearest
neighbour search to support real-time queries of similar entities over the
complete data. We will also collect more large-scale knowledge graphs from the
Linked Open Data cloud to update our universal embeddings.

## References

  * Auer et al. 2007 Auer, S.; Bizer, C.; Kobilarov, G.; Lehmann, J.; Cyganiak, R.; and Ives, Z. G.  2007\.  Dbpedia: A nucleus for a web of open data.  In ISWC/ASWC, volume 4825 of Lecture Notes in Computer Science, 722–735.  Springer.
  * Bordes et al. 2013 Bordes, A.; Usunier, N.; García-Durán, A.; Weston, J.; and Yakhnenko, O.  2013\.  Translating embeddings for modeling multi-relational data.  In NIPS, 2787–2795.
  * Chen et al. 2017 Chen, M.; Tian, Y.; Yang, M.; and Zaniolo, C.  2017\.  Multilingual knowledge graph embeddings for cross-lingual knowledge alignment.  In IJCAI, 1511–1517.
  * Dai et al. 2020 Dai, Y.; Wang, S.; Xiong, N. N.; and Guo, W.  2020\.  A survey on knowledge graph embedding: Approaches, applications and benchmarks.  Electronics 9(5):750.
  * Demir and Ngomo 2021 Demir, C., and Ngomo, A. N.  2021\.  Convolutional complex knowledge graph embeddings.  In ESWC, volume 12731 of Lecture Notes in Computer Science, 409–424.  Springer.
  * Demir and Ngomo 2022 Demir, C., and Ngomo, A. N.  2022\.  Hardware-agnostic computation for large-scale knowledge graph embeddings.  Softw. Impacts 13:100377.
  * Demir et al. 2021 Demir, C.; Moussallem, D.; Heindorf, S.; and Ngomo, A. N.  2021\.  Convolutional hypercomplex embeddings for link prediction.  In ACML, volume 157 of Proceedings of Machine Learning Research, 656–671.  PMLR.
  * Jain et al. 2021 Jain, N.; Kalo, J.; Balke, W.; and Krestel, R.  2021\.  Do embeddings actually capture knowledge graph semantics?  In ESWC, volume 12731 of Lecture Notes in Computer Science, 143–159.  Springer.
  * Nickel, Tresp, and Kriegel 2011 Nickel, M.; Tresp, V.; and Kriegel, H.  2011\.  A three-way model for collective learning on multi-relational data.  In ICML, 809–816.  Omnipress.
  * Ristoski and Paulheim 2016 Ristoski, P., and Paulheim, H.  2016\.  Rdf2vec: RDF graph embeddings for data mining.  In ISWC (1), volume 9981 of Lecture Notes in Computer Science, 498–514.
  * Sun et al. 2018a Sun, Z.; Hu, W.; Zhang, Q.; and Qu, Y.  2018a.  Bootstrapping entity alignment with knowledge graph embedding.  In IJCAI, 4396–4402.  ijcai.org.
  * Sun et al. 2018b Sun, Z.; Deng, Z.-H.; Nie, J.-Y.; and Tang, J.  2018b.  Rotate: Knowledge graph embedding by relational rotation in complex space.  In International Conference on Learning Representations.
  * Sun, Hu, and Li 2017 Sun, Z.; Hu, W.; and Li, C.  2017\.  Cross-lingual entity alignment via joint attribute-preserving embedding.  In International Semantic Web Conference, 628–644.  Springer.
  * Trisedya, Qi, and Zhang 2019 Trisedya, B. D.; Qi, J.; and Zhang, R.  2019\.  Entity alignment between knowledge graphs using attribute embeddings.  In AAAI, 297–304.  AAAI Press.
  * Trouillon et al. 2016 Trouillon, T.; Welbl, J.; Riedel, S.; Gaussier, É.; and Bouchard, G.  2016\.  Complex embeddings for simple link prediction.  In ICML, volume 48 of JMLR Workshop and Conference Proceedings, 2071–2080.  JMLR.org.
  * Vrandecic and Krötzsch 2014 Vrandecic, D., and Krötzsch, M.  2014\.  Wikidata: a free collaborative knowledgebase.  Commun. ACM 57(10):78–85.
  * Wang et al. 2017 Wang, Q.; Mao, Z.; Wang, B.; and Guo, L.  2017\.  Knowledge graph embedding: A survey of approaches and applications.  IEEE Trans. Knowl. Data Eng. 29(12):2724–2743.
  * Wang et al. 2018 Wang, Z.; Lv, Q.; Lan, X.; and Zhang, Y.  2018\.  Cross-lingual knowledge graph alignment via graph convolutional networks.  In EMNLP, 349–357.  Association for Computational Linguistics.
  * Wang et al. 2019 Wang, M.; Shen, H.; Wang, S.; Yao, L.; Jiang, Y.; Qi, G.; and Chen, Y.  2019\.  Learning to hash for efficient search over incomplete knowledge graphs.  In 2019 IEEE International Conference on Data Mining (ICDM), 1360–1365.  IEEE.
  * Wilkinson et al. 2016 Wilkinson, M. D.; Dumontier, M.; Aalbersberg, I. J.; Appleton, G.; Axton, M.; Baak, A.; Blomberg, N.; Boiten, J.-W.; da Silva Santos, L. B.; Bourne, P. E.; et al.  2016\.  The fair guiding principles for scientific data management and stewardship.  Scientific data 3(1):1–9.
  * Yang et al. 2015 Yang, B.; Yih, W.; He, X.; Gao, J.; and Deng, L.  2015\.  Embedding entities and relations for learning and inference in knowledge bases.  In ICLR (Poster).
  * Zhang et al. 2019 Zhang, Q.; Sun, Z.; Hu, W.; Chen, M.; Guo, L.; and Qu, Y.  2019\.  Multi-view knowledge graph embedding for entity alignment.  In IJCAI, 5429–5435.
  * Zhang et al. 2023 Zhang, X.; Xin, X.; Li, D.; Liu, W.; Ren, P.; Chen, Z.; Ma, J.; and Ren, Z.  2023\.  Variational reasoning over incomplete knowledge graphs for conversational recommendation.  In Proceedings of the Sixteenth ACM International Conference on Web Search and Data Mining, 231–239.
  * Zhao et al. 2022 Zhao, F.; Li, Y.; Hou, J.; and Bai, L.  2022\.  Improving question answering over incomplete knowledge graphs with relation prediction.  Neural Computing and Applications 1–18.
  * Zheng et al. 2020 Zheng, D.; Song, X.; Ma, C.; Tan, Z.; Ye, Z.; Dong, J.; Xiong, H.; Zhang, Z.; and Karypis, G.  2020\.  DGL-KE: training knowledge graph embeddings at scale.  In SIGIR, 739–748.  ACM.