# CEERS Key Paper. IX. Identifying Galaxy Mergers in CEERS NIRCam Images Using
Random Forests and Convolutional Neural Networks

Caitlin Rose Jeyhan S. Kartaltepe Laboratory for Multiwavelength Astrophysics,
School of Physics and Astronomy, Rochester Institute of Technology, 84 Lomb
Memorial Drive, Rochester, NY 14623, USA  Gregory F. Snyder  Space Telescope
Science Institute, 3700 San Martin Dr., Baltimore, MD 21218, USA  Marc
Huertas-Company Instituto de Astrofísica de Canarias, La Laguna, Tenerife,
Spain  Universidad de la Laguna, La Laguna, Tenerife, Spain  Université Paris-
Cité, LERMA - Observatoire de Paris, PSL, Paris, France  L. Y. Aaron Yung
Space Telescope Science Institute, 3700 San Martin Dr., Baltimore, MD 21218,
USA  Pablo Arrabal Haro NSF’s National Optical-Infrared Astronomy Research
Laboratory, 950 N. Cherry Ave., Tucson, AZ 85719, USA  Micaela B. Bagley
Department of Astronomy, The University of Texas at Austin, Austin, TX, USA
Laura Bisigello INAF, Istituto di Radioastronomia, Via Piero Gobetti 101,
40129 Bologna, Italy  Dipartimento di Fisica e Astronomia ”G.Galilei”,
Universitá di Padova, Via Marzolo 8, I-35131 Padova, Italy  Antonello Calabrò
INAF – Osservatorio Astronomico di Roma, via di Frascati 33, 00078 Monte
Porzio Catone, Italy  Nikko J. Cleri Department of Physics and Astronomy,
Texas A&M University, College Station, TX, 77843-4242 USA  George P. and
Cynthia Woods Mitchell Institute for Fundamental Physics and Astronomy, Texas
A&M University, College Station, TX, 77843-4242 USA  Mark Dickinson NSF’s
National Optical-Infrared Astronomy Research Laboratory, 950 N. Cherry Ave.,
Tucson, AZ 85719, USA  Henry C. Ferguson Space Telescope Science Institute,
3700 San Martin Dr., Baltimore, MD 21218, USA  Steven L. Finkelstein
Department of Astronomy, The University of Texas at Austin, Austin, TX, USA
Adriano Fontana INAF – Osservatorio Astronomico di Roma, via di Frascati 33,
00078 Monte Porzio Catone, Italy  Andrea Grazian INAF – Osservatorio
Astronomico di Padova, Vicolo dell’Osservatorio 5, I-35122, Padova, Italy
Norman A. Grogin Space Telescope Science Institute, 3700 San Martin Dr.,
Baltimore, MD 21218, USA  Benne W. Holwerda Physics & Astronomy Department,
University of Louisville, 40292 KY, Louisville, USA  Kartheik G. Iyer Dunlap
Institute for Astronomy & Astrophysics, University of Toronto, Toronto, ON M5S
3H4, Canada  Lisa J. Kewley Center for Astrophysics — Harvard & Smithsonian,
60 Garden Street, Cambridge, MA 02138, USA  Allison Kirkpatrick Department of
Physics and Astronomy, University of Kansas, Lawrence, KS 66045, USA  Dale D.
Kocevski Department of Physics and Astronomy, Colby College, Waterville, ME
04901, USA  Anton M. Koekemoer Space Telescope Science Institute, 3700 San
Martin Dr., Baltimore, MD 21218, USA  Jennifer M. Lotz Gemini
Observatory/NSF’s National Optical-Infrared Astronomy Research Laboratory, 950
N. Cherry Ave., Tucson, AZ 85719, USA  Ray A. Lucas Space Telescope Science
Institute, 3700 San Martin Dr., Baltimore, MD 21218, USA  Lorenzo Napolitano
INAF – Osservatorio Astronomico di Roma, via di Frascati 33, 00078 Monte
Porzio Catone, Italy  Dipartimento di Fisica, Università di Roma Sapienza,
Città Universitaria di Roma - Sapienza, Piazzale Aldo Moro, 2, 00185, Roma,
Italy  Casey Papovich Department of Physics and Astronomy, Texas A&M
University, College Station, TX, 77843-4242 USA  George P. and Cynthia Woods
Mitchell Institute for Fundamental Physics and Astronomy, Texas A&M
University, College Station, TX, 77843-4242 USA  Laura Pentericci INAF –
Osservatorio Astronomico di Roma, via di Frascati 33, 00078 Monte Porzio
Catone, Italy  Pablo G. Pérez-González Centro de Astrobiología (CAB), CSIC-
INTA, Ctra. de Ajalvir km 4, Torrejón de Ardoz, E-28850, Madrid, Spain  Nor
Pirzkal ESA/AURA Space Telescope Science Institute  Swara Ravindranath
Astrophysics Science Division, NASA Goddard Space Flight Center, 8800
Greenbelt Road, Greenbelt, MD 20771, USA  Center for Research and Exploration
in Space Science and Technology II, Department of Physics, Catholic University
of America, 620 Michigan Ave N.E., Washington DC 20064, USA  Rachel S.
Somerville Center for Computational Astrophysics, Flatiron Institute, 162 5th
Avenue, New York, NY, 10010, USA  Amber N. Straughn Astrophysics Science
Division, NASA Goddard Space Flight Center, 8800 Greenbelt Road, Greenbelt, MD
20771, USA  Jonathan R. Trump Department of Physics, 196 Auditorium Road, Unit
3046, University of Connecticut, Storrs, CT 06269, USA  Stephen M. Wilkins
Astronomy Centre, University of Sussex, Falmer, Brighton BN1 9QH, UK
Institute of Space Sciences and Astronomy, University of Malta, Msida MSD
2080, Malta  Guang Yang Nanjing Institute of Astronomical Optics and
Technology, Nanjing 210042, China

###### Abstract

A crucial yet challenging task in galaxy evolution studies is the
identification of distant merging galaxies, a task which suffers from a
variety of issues ranging from telescope sensitivities and limitations to the
inherently chaotic morphologies of young galaxies. In this paper, we use
random forests and convolutional neural networks to identify high-redshift
JWST CEERS galaxy mergers. We train these algorithms on simulated $3<z<5$
CEERS galaxies created from the IllustrisTNG subhalo morphologies and the
Santa Cruz SAM lightcone. We apply our models to observed CEERS galaxies at
$3<z<5$. We find that our models correctly classify $\sim 60-70\%$ of
simulated merging and non-merging galaxies; better performance on the merger
class comes at the expense of misclassifying more non-mergers. We could
achieve more accurate classifications, as well as test for the dependency on
physical parameters such as gas fraction, mass ratio, and relative orbits, by
curating larger training sets. When applied to real CEERS galaxies using
visual classifications as ground truth, the random forests correctly
classified $40-60\%$ of mergers and non-mergers at $3<z<4$, but tended to
classify most objects as non-mergers at $4<z<5$ (misclassifying $\sim 70\%$ of
visually-classified mergers). On the other hand, the CNNs tended to classify
most objects as mergers across all redshifts (misclassifying $80-90\%$ of
visually-classified non-mergers). We investigate what features the models find
most useful, as well as characteristics of false positives and false
negatives, and also calculate merger rates derived from the identifications
made by the models.

††facilities: JWST (NIRCam)††software: Source Extractor (Bertin & Arnouts,
1996), Galapagos-2 (Häußler et al., 2013), statmorph (Rodriguez-Gomez et al.,
2019), DeepMerge (Ćiprijanović et al., 2020)

##  1 Introduction

Galaxy merger studies are important for understanding the evolution of
galaxies over cosmic time, as mergers can affect multiple physical phenomena
and processes such as star formation and AGN activity (e.g., Ellison et al.,
2008; Patton et al., 2011; Larson et al., 2016), the morphological evolution
of spiral galaxies to elliptical galaxies (e.g., Toomre, 1977; Cox et al.,
2006; Kormendy et al., 2009; Rodriguez-Gomez et al., 2017), and the build up
of massive structures (e.g., Tacchella et al., 2015; Zolotov et al., 2015;
Costantin et al., 2021, 2022). To better understand galaxy evolution, it is
necessary to be able to accurately identify merging galaxies over a wide range
of mass ratios (Lotz et al., 2010a, b) and redshifts.

However, at high redshift, merger identification can be difficult. The limited
sensitivity of modern telescopes can result in samples biased toward the most
massive galaxies and major mergers (mass ratio $<$4:1; e.g., Ellison et al.,
2013; Mantha et al., 2018). Cosmological surface brightness dimming and poor
angular resolution can result in the loss of faint, detailed structures. As
rest-frame optical light is shifted towards the infrared, infrared telescopes
like JWST are needed to observe the stellar population. Finally, high redshift
galaxies can inherently have irregular morphologies that are difficult to
discern from merger signatures (e.g., Dekel et al., 2009; Kartaltepe et al.,
2012, 2015).

Classic methods of identifying mergers – visual classifications (e.g., Lintott
et al., 2008; Kartaltepe et al., 2015), the close pair method (e.g., Lin et
al., 2004; Kartaltepe et al., 2007; Ventou et al., 2017; Shah et al., 2020),
and quantitative morphology parameters (e.g., Conselice, 2003; Lotz et al.,
2004; Freeman et al., 2013) – all have limitations. The close pair method
requires that both companions have accurate redshifts in order to identify
pairs that are physically close in space, and is most sensitive to galaxies
that have not yet merged. Visual classifications are qualitative and
subjective, as well as time consuming. Finally, individual morphology
parameters are each sensitive to different merger phases as well as noise
(e.g., Lotz et al., 2010a, b). Recently, studies have begun investigating
machine learning methods for merger identification tasks, as they show promise
for exploiting complex data (whether from multi-band images or from pre-
computed morphology parameters) to better detect merger signatures in high
redshift images than classic methods (e.g., Snyder et al., 2019; Ferreira et
al., 2022). Furthermore, the use of deep, high-resolution near-infrared JWST
imaging will also help in the detection of merger features at high redshifts.
One such JWST survey is the Cosmic Evolution Early Release Science Survey
(CEERS; Finkelstein et al., 2022), which surveyed galaxies at $0.5<z<13$ over
$100$ arcmin2 in the Extended Groth Strip HST legacy field (EGS; Davis et al.,
2007).

In Rose et al. (2023), we trained random forests to identify merging galaxies
at $0.5<z<4$ in simulated CEERS images from IllustrisTNG, using a suite of
morphology parameters measured using Galapagos-2 (Bamford et al., 2011;
Häußler et al., 2013; Vika et al., 2013) and statmorph (Rodriguez-Gomez et
al., 2019), as well as merger history information from IllustrisTNG. We found
that the forests correctly classified $\sim$60% of mock CEERS mergers and non-
mergers across all redshift bins. In this follow-up study, we investigate
using convolutional neural networks trained directly on the mock images for
the task of merger identification at high redshift, as well as test our
trained models on real galaxies from CEERS (described in §2.1).

In recent years, many studies have trained neural networks to classify
galaxies by morphology in general (e.g., Dieleman et al., 2015; Huertas-
Company et al., 2018, 2019; Barchi et al., 2020; Cheng et al., 2020; Walmsley
et al., 2022; Huertas-Company & Lanusse, 2023; Huertas-Company et al., 2023)
or to identify mergers specifically (e.g., Ackermann et al., 2018; Bottrell et
al., 2019; Pearson et al., 2019a, b; Ćiprijanović et al., 2020, 2021; Bickley
et al., 2021; Ferreira et al., 2020, 2022). However, most studies regarding
merger identification train and test on galaxy samples at low redshift. At
high redshift, Ćiprijanović et al. (2020, 2021), Ferreira et al. (2020, 2022),
and Pearson et al. (2019b) each demonstrate that convolutional neural networks
show promise for accurately identifying high redshift mergers. In particular,
Ćiprijanović et al. (2020) apply their convolutional neural network
(“DeepMerge”) to $z=2$ simulated Illustris-1 HST F814W and F160W images
created in Snyder et al. (2019) and achieve an accuracy of $76\%$ on the
merger class for noisy data.

In this paper, we train DeepMerge111https://github.com/deepskies/deepmerge-
public to identify galaxy mergers from simulated CEERS images from
IllustrisTNG. Then we explore using our trained random forest from Rose et al.
(2023) and DeepMerge algorithms to identify merging galaxies in real CEERS
imaging. In §2, we describe both the simulated JWST CEERS images and observed
images, and how our datasets were curated. In §3, we describe training
DeepMerge, as well as measuring quantitative parameters for observed galaxies.
In §4 and §5, we present our DeepMerge results on simulated data, then show
our attempts to identify observed mergers using both DeepMerge and random
forests. We summarize and conclude in §6.

##  2 Data

###  2.1 CEERS Observations and Data Reduction

The Cosmic Evolution Early Release Science Survey (CEERS; Finkelstein et al.,
2022) is a JWST Early Release Science program (Proposal ID #1345) that
observed the EGS deep field in 10 NIRCam pointings. The first four NIRCam
pointings – hereafter CEERS1, CEERS2, CEERS3, and CEERS6 – were taken on June
21, 2022 in seven filters (F115W, F150W, F200W, F277W, F356W, F410M, and
F444W) with a typical exposure time of 2835 seconds per exposure. We use these
four pointings in this work. A full description of CEERS and the data
reduction steps can be found in Finkelstein et al. (2022) and Bagley et al.
(2023), respectively. To summarize, the CEERS NIRCam images were reduced with
version 1.7.2 of the JWST Calibration Pipeline222jwst-pipeline.readthedocs.io
(Bushouse et al., 2022) with custom modifications and with current NIRCam
reference files333jwst-crds.stsci.edu, jwst 0989.pmap, jwst nircam 0232.imap.
The images were processed through Stages 1 and 2 of the pipeline, where
reduction steps included detector-level corrections; removal of $1/f$ noise,
wisps, snowballs; and astrometric calibration. The images were distortion-
corrected and combined into final mosaics in Stage 3 of the JWST pipeline
using the drizzle algorithm with inverse variance map weighting (Fruchter &
Hook, 2002; Casertano et al., 2000). The final mosaics for each pointing in
each filter are background-subtracted and have pixel scales of
0$\farcs$03/pixel. The publicly released images are available at
https://ceers.github.io/releases.html and on MAST via doi:
https://dx.doi.org/10.17909/z7p0-8481 (catalog 10.17909/z7p0-8481).

In this work, we use the sample of 850 galaxies at $z>3$, from the four CEERS
pointings, that were selected by Kartaltepe et al. (2023) as well as their
visual classifications (described in §3.1).

###  2.2 Simulated CEERS Images

The simulated mock CEERS images used in this work were created by Rose et al.
(2023). The images were constructed from IllustrisTNG100-1 (Springel et al.,
2018; Naiman et al., 2018; Nelson et al., 2018; Pillepich et al., 2018;
Marinacci et al., 2018) and the Santa Cruz SAM lightcone (Somerville et al.,
2021; Yung et al., 2022). We refer the reader to Rose et al. (2023) for
further details on the creation of the pristine, noiseless images. The images
are available in six filters (F115W, F150W, F200W, F277W, F356W, F444W) and
cover $\sim$ 100 square arcmin (with pixel scales of 0$\farcs$03/pixel) and
contain over 100,000 galaxies, to mimic the size of the complete CEERS mosaic
of the EGS field. The images were convolved with model PSFs, then noise was
added on top to mimic CEERS noise (estimated using the JWST Exposure Time
Calculator; Pontoppidan et al., 2016). The mock CEERS images are publicly
available at https://ceers.github.io/ancillary_data.html.

These images are accompanied by catalogs with simulation information such as
redshift, star formation rate, and stellar mass. Additionally, the merger
history catalogs for IllustrisTNG galaxies (Rodriguez-Gomez et al., 2015;
Nelson et al., 2019) were also available. These catalogs give the IllustrisTNG
snapshot numbers for each galaxy’s most recent past merger and next future
merger (both major and minor). This merger history information makes
IllustrisTNG an ideal dataset for training and testing machine learning
algorithms to identify galaxy mergers at different stages. From the full
simulated mosaic, we obtained morphology measurements for 5438 galaxies with
$z>3$. Figure 1 shows the redshift and mass distribution for these objects in
gray. This sample also peaks at $z\sim 3$ and extends out to $z\sim 9$. This
sample also contains more low-mass galaxies than in the observed CEERS sample.
While Pillepich et al. (2018) note that the IllustrisTNG galaxy stellar mass
function (GSMF) is in better agreement with observations than the Illustris
GSMF, expecially at $z\leq 1$, IllustrisTNG may still overproduce low-mass
galaxies at higher redshifts. Stellar masses for the observed CEERS galaxies
were computed by Kartaltepe et al. (2023) using the EGS photometry (Stefanon
et al., 2017) from CANDELS (Grogin et al., 2011; Koekemoer et al., 2011) and
estimated redshifts (Kodra et al., 2023).

Figure 1: Stellar mass versus redshift for objects from the mock CEERS sample
at $3<z<5$ (grey), and from the observed CEERS sample with $3<z<5$ (blue).
Above and to the right are the distributions of redshift and stellar mass,
respectively. These samples do include the S/N and flag cuts discussed in §3.3
and §4.1.

##  3 Methods

###  3.1 Visual Classifications of Observed Galaxies

Visual classifications were reported by Kartaltepe et al. (2023), and were
carried out by members of the CEERS team. Images of the galaxies in our sample
were hosted on the Zooniverse project builder444
https://www.zooniverse.org/lab. Volunteer classifiers viewed stamps of the
galaxies in seven JWST NIRCam filters and three HST ACS/WFC3 filters, as well
as RGB stamps and JWST segmentation map stamps (see Figure 14 in the Appendix
of Kartaltepe et al., 2023) from Source Extractor (SE v2.23.2; Bertin &
Arnouts, 1996). Each object was classified by three volunteer classifiers who
followed a morphology classification scheme based on that used in Kartaltepe
et al. (2015). The five tasks in this scheme ask volunteer classifiers to
select main morphology classes, interaction classes, structural and quality
flags, and, if the classifier wishes, to leave comments about the object. The
main morphology class options were: Disk, Spheroid, Irregular / Peculiar,
Point Source / Unresolved, and Unclassifiable / Junk. There were four
interaction class options.

(1) Merger: Galaxies (that are single objects) that appear to have experienced
a merger due to the presence of structures like tidal tails, double nuclei, or
other asymmetries. All mergers should have Irregular/Peculiar selected as
their main classification, but not all galaxies classified as irregulars are
mergers.

(2) Interaction within SE segmentation map: Two or more galaxies that appear
to be interacting within the same segmentation map. There must be clear
evidence of an interaction (tidal arms, bridges, etc.).

(3) Interaction beyond SE segmentation map: Two or more galaxies that appear
to be interacting that have their own distinct segmentation maps.

(4) Non-interacting companion: The main galaxy has a nearby companion, yet has
no evidence of tidal interaction.

The structural and quality flags question included flags to mark galaxies with
merger features such as tidal tails and double nuclei. Figure 2 shows examples
of galaxies from the different interaction classes.

Figure 2: Examples of CEERS galaxies (F150W+F200W+F356W stamps) for each
visually-classified interaction class at $3.5<z<4$, $3.5<z<4$, and $4<z<5$.

###  3.2 Morphology Parameters for Observed Galaxies

SE (Bertin & Arnouts, 1996) was used to detect $z>3$ galaxies in each of the
four pointings. Weighted F150W$+$F200W$+$F277W$+$F356W images were used as the
detection images. We ran SE in both “cold” and “hot” mode, in order to deblend
overlapping galaxies and detect faint galaxies, respectively. The “cold” and
“hot” detections were then combined following the logic of the Galapagos-2
program (Bamford et al., 2011; Barden et al., 2012; Häußler et al., 2013; Vika
et al., 2013).

Galapagos-2 (Galaxy Analysis over Large Areas: Parameter Assessment by Galfit-
ting Objects from SE) is an IDL program that runs the multiwavelength Sérsic
fitting program GalfitM (Peng et al., 2002, 2010; Peng, 2012) on large survey
images. Information from the output SE catalog was used as initial guesses for
the Sérsic fits from Galapagos-2. Empirical point-spread functions (PSFs) made
from stacking stars (Finkelstein et al., 2022) were used for these fits. We
ran Galapagos-2 on all seven filters simultaneously to obtain Sérsic indices
and related size and magnitude information. In addition to the output Sérsic
model catalog, Galapagos-2 also returns input image, model, and Sérsic
residual stamps for each galaxy in the catalog.

We also ran the non-parametric morphology program statmorph (Rodriguez-Gomez
et al., 2019) separately on each filter for each of the four pointings, for
the $z>3$ objects in the Galapagos-2 catalog, in order to obtain morphology
parameters such as concentration ($C$), asymmetry ($A$), and clumpiness ($S$)
(Bershady et al., 2000; Conselice et al., 2000; Conselice, 2003), and the Gini
coefficient ($G$) and the moment of light ($M_{20}$) (Abraham et al., 2003;
Lotz et al., 2004). The output SE segmentation map was used as input to
statmorph to aid in calculating these parameters, as well as the PSFs from
Finkelstein et al. (2022).

Finally, it has been shown that asymmetric or unusual features indicative of
mergers can be more prominent in Sérsic residuals (e.g., Mantha et al., 2019).
Therefore, we also run statmorph on the Sérsic residuals output by
Galapagos-2. To do so, we add an offset of $+1$ to every pixel of all residual
images, since statmorph requires that input images have positive flux which is
not necessarily true for residual images.

We begin with the catalog of visual classifications for 850 objects at $z>3$
from Kartaltepe et al. (2023). After combining all of the above catalogs
(matching by CANDELS ID number or by RA and Dec), we create a sample of 798
observed $z>3$ CEERS galaxies that have successful morphology measurements
from both the statmorph and Galapagos catalogs.

###  3.3 Training Random Forests on Simulated Galaxies

In Rose et al. (2023), we trained and tested the random forest algorithm (Ho,
1995; Breiman, 2001) on the simulated images described in §2.2. To do so, we
ran Galapagos-2 and statmorph on the mock CEERS data in a similar fashion as
described in §3.2. These morphology parameters were used as the input
“features” for the forests, which were the single Sérsic index $n$, the two-
component Sérsic indices $n_{bulge}$ and $n_{disk}$, and various statmorph
parameters (outlined in Rose et al., 2023), in all six filters, for both
science and residual images. From the merger history catalogs, we convert the
IllustrisTNG snapshot numbers to ages of the Universe in Gyr, then calculate
the difference in time between the galaxy of interest and the last and next
merger events. The resulting time since the last merger (both major and minor)
and the time until the next (both major and minor) are used as the “labels”
for the random forests. “Mergers” are objects that had, or will have, a
merging event within $\pm 250$ Myr, in accordance with Snyder et al. (2017),
as merger signatures from outside that time frame will have likely vanished.
“Non-mergers” are objects which either have a merging event outside that time
frame, or have not had merging events at all. In Rose et al. (2023), we
restrict the mock CEERS dataset to have statmorph signal-to-noise per pixel
(S/NF115W) $>3$ and Flag${}_{\texttt{Galfit}}$ $=2.0$ (which indicates
successful Galfit measurements). Then, we split the final sample of 40,391
mock galaxies into seven redshift bins from $0.5<z<4$ and trained a separate
forest for each bin, using BalancedRandomForestClassifier() from Python’s
imblearn package as the random forest algorithm and keras-tuner (O’Malley et
al., 2019) for hyperparameter optimization.

Following the same procedure in Rose et al. (2023), we additionally train a
random forest for $z=4-5$. In §4.3, we describe the outcome of training and
testing random forests for merger identification on the mock CEERS dataset. In
§4.5, we apply the $3<z<3.5$, $3.5<z<4$, and $4<z<5$ random forests to
observed CEERS galaxies.

Additionally, we compute the Spearman correlations between the morphology
parameters and the merger timescales using spearmanr from the SciPy package in
order to understand which features might be most important for the random
forests. The Spearman correlation describes the relationship between
variables, where $+1$ or $-1$ indicates perfect positive or negative
correlations, respectively, and 0 indicates no correlation. For each of the
redshift bins studied in this work ($3<z<3.5$, $3.5<z<4$, and $4<z<5$), the
strongest correlations ranged from $\sim|0.1|-|0.2|$. For example, the most
correlated features for the $4<z<5$ bin were: F356W concentration with a
correlation coefficient of $-0.21$, F200W residual flux (within an elliptical
aperture) with a correlation coefficient of $0.11$, F444W asymmetry with a
correlation coefficient of $0.27$, and F356W residual flux (within an
elliptical aperture) with a correlation coefficient of $0.07$.

###  3.4 Training the DeepMerge Algorithm on Simulated Galaxies

An alternative to using segmentation maps and morphology parameters is to
train a machine learning algorithm directly on the image data. Here, we
explore using the DeepMerge555https://github.com/deepskies/deepmerge-public
convolutional neural network (Ćiprijanović et al., 2020) to identify mock
CEERS mergers. Convolutional neural networks have special “convolutional”
layers which convolve the input image with a kernel, in order to extract
features from the input image while preserving spatial information from the
image. DeepMerge is a relatively simple network containing only three
convolutional layers.

Redshift Bin | $3-3.5$ | $3.5-4$ | $4-5$
---|---|---|---
Total Number of Galaxies | 2039 | 2367 | 3105
Number of Mergers | 668 | 839 | 1476
Table 1: Number of simulated CEERS mergers in each redshift bin after imposing
$S/N>3$ and Flag${}_{\texttt{Galfit}}$ $=2.0$.

As with the random forest analysis, we use the same mock CEERS sample (with
S/NF115W $>3$ and Flag${}_{\texttt{Galfit}}$ $=2.0$). This time, we restrict
our mock CEERS sample to three redshift bins at $3<z<3.5$, $3.5<z<4$, and
$4<z<5$ (see Table 1). We limit ourselves to these three redshift bins, since
we will apply our networks to our observed CEERS galaxies for which visual
classifications are only available for $z>3$. We cut out stamps of size 75 by
75 pixels of each galaxy. The time since last merger and time until next
merger were again used to determine the “labels” for DeepMerge.

We use stratified sampling to split our data into training, validation, and
test sets. The ratio of the train/validation and test split was 0.18, which
was smaller than the split used for the random forests in Rose et al. (2023),
in order to divert more examples to the training set. The training and
validation split was 0.3. Due to the low numbers of objects in each redshift
bin, we employ augmentation to create a larger training set. This included
flipping and rotating the images, as well as zooming in and zooming out while
maintaining a consistent stamp size of 75 by 75 pixels. For each object, we
provide DeepMerge with all six filters (F115W, F150W, F200W, F277W, F356W,
F444W). We also normalize all images based on one of the six filters and then
stretch the images using an arcsinh stretch to emphasize low surface
brightness features. Figure 3 shows an example of normalizing, stretching, and
augmenting a $3<z<3.5$ galaxy merger. Our data sets are imbalanced since there
are more non-mergers than mergers. In order to help prevent our classifier
from becoming biased toward the non-merger class, we maintain balanced classes
during training by randomly undersampling the non-merger class, which
decreases the size of the non-merger class. The validation and test sets are
left unbalanced to better represent proportions of real-world data.

Figure 3: Example of a normalized, stretched, and augmented $3<z<3.5$ galaxy
merger. For this redshift bin, normalization was based on the F115W filter.
After normalizing and stretching, the extended structures of this merger are
more distinct from the background.

We start with the architecture of DeepMerge, and train using our data. We
employ multiple strategies in order to properly train the DeepMerge networks.
In addition to normalizing, augmenting, and balancing the training set, we
also test:

  1. 1.

Using different ratios for the split between the training, validation, and
test sets.

  2. 2.

Using different subsets of augmentations (e.g., only rotations).

  3. 3.

Using only three filters, or only one rest-frame filter, rather than all six
available filters.

  4. 4.

Reducing the number of layers within the network, in order to reduce the
complexity of the network.

  5. 5.

Shifting the location of the dropout layers within the network. Dropout layers
mask contributions from some neurons to the next layer in an effort to prevent
overfitting.

  6. 6.

Using rectified linear unit (ReLU) activation rather than sigmoid activation
in the final layer. These activation functions determine how the result of the
last layer will be mapped to a class probability.

  7. 7.

Using different patience values for early
stopping666https://keras.io/api/callbacks/early_stopping/, which is a method
that stops training once a given metric (in this case, validation loss) has
stopped improving in order to combat overfitting (e.g., Prechelt, 2012).

  8. 8.

Tracking the difference between training and validation accuracy, rather than
validation loss, for early stopping.

Most of the above strategies resulted in networks that performed very well in
one class but very poorly in the other. We also attempted to narrow our merger
definition to include only major mergers, but the small number of labeled
mergers in the final data sets (with the $3<z<3.5$ bin having only 376
mergers) resulted in networks that classified all objects as non-mergers. We
also attempted to include objects with any Flag${}_{\texttt{Galfit}}$ (about
$\sim 10$ objects in each redshift bin), as this flag only pertains to the
quality of the Galapagos/Galfit measurements which are not relevant here, but
the inclusion of these objects also did not improve the performance of the
networks. The reason for this may be that these objects were typically found
in very crowded regions which may have caused DeepMerge to focus on the
surrounding galaxies.

For all redshift bins, normalizing and arcsinh stretching the images resulted
in improvements. We use all available filters. For the $3.5<z<4$ and $4<z<5$
redshift bins, we reduce the number of layers in DeepMerge to two sets of
Convolution2D, BatchNormalization, MaxPooling2D, and Dropout layers rather
than three sets. For the $3.0<z<3.5$ redshift bin, we also reduce the number
of layers to two sets of Convolution2D, BatchNormalization, MaxPooling2D;
additionally we use only one Dropout layer which we shifted to just before
DeepMerge’s Dense layers. We train for 500 epochs with a batch size of 32. For
the $3<z<3.5$ redshift bin, we normalize with respect to the F115W filter; for
the $3.5<z<4$ redshift bin, we normalize with respect to the F150W filter; and
for the $4<z<5$ redshift bin, we again normalize with respect to the F150W
filter. For each redshift bin, we perform the following augmentations on the
training set: flip up-down, flip left-right, flip up-down and left-right,
rotate at angles from $45^{\circ}$ to $270^{\circ}$, zoom-in by a factor of
1.5, and zoom-out by a factor of 0.75. For the $3<z<3.5$ bin, for early
stopping, we set patience to 50 and monitor the validation loss; for the
$3.5<z<4$ redshift bin, we set patience to 5 and monitor the ratio between
validation and training accuracy so that training stops once this ratio is
less than 0.8; for the $4<z<5$ redshift bin, we set patience to 50 and monitor
the validation loss.

In §4.4, we describe the best outcome of training and testing the DeepMerge
network for merger identification on the mock CEERS dataset at $3<z<5$. In
§4.6, we apply the DeepMerge to observed CEERS galaxies.

##  4 Analysis

###  4.1 Visual Classification Results for Observed CEERS Galaxies

We construct four possible “merger” definitions based on the visual
classifications. As described in §3.1, classifiers can select a main
morphology class, an interaction class, and structure and quality flags. Our
different merger definitions are motivated by these different categories:

Group 1 (G1): at least two out of the three classifiers assigned the Merger
class or Interaction within segmentation map class or Interaction beyond
segmentation map class.

Group 2 (G2): at least two out of the three classifiers assigned the Irregular
main morphology class, since many merging galaxies appear irregular, although
not all irregular galaxies are mergers.

Group 3 (G3): at least two out of the three classifiers assigned the tidal
tails or double nuclei structure flags, which are merger signatures.

Group 4 (G4): one out of the three classifiers assigned the Merger class or
Interaction within segmentation map class or Interaction beyond segmentation
map class.

For groups 1, 2, and 3, we examine objects for which at least two out of three
classifiers indicated that the object was potentially a merger, which means
classifiers were overall more confident that these objects actually are
mergers. For group 4, we examine objects where only only one classifier
indicated that a given object had merger signatures. All objects from Group 1
will also be in this group, as well as additional objects for which
classifiers were less confident about their merger status.

We start with the 798 objects that result from combining our morphology
catalogs in §3.2. Since we restricted our mock CEERS dataset to have S/NF115W
$>3$ and Flag${}_{\texttt{Galfit}}$ $=2.0$, we do the same to our real CEERS
dataset. Doing so results in a final sample of 369 galaxies at $3<z<5$. Table
2 shows the total number of objects from this restricted real CEERS data set,
as well as the number of mergers from each group, in our three redshift bins.
In this paper, we focus on merger definitions using Groups 1, 2, and 4, since
we found that most Group 3 objects are also in the other groups. Only two
objects were assigned the tidal tails structure flag without also being
assigned an interaction class by at least one person.

Redshift Bin | $3-3.5$ | $3.5-4$ | $4-5$ | $>5$
---|---|---|---|---
Total Number of Galaxies | 183 | 99 | 87 | 43
Group 1 Mergers | 38 | 23 | 15 | 6
Group 2 Mergers | 71 | 43 | 43 | 17
Group 3 Mergers | 28 | 20 | 18 | 11
Group 4 Mergers | 92 | 46 | 55 | 20
Table 2: Number of observed CEERS mergers in each redshift bin after imposing
$S/N>3$ and Flag${}_{\texttt{Galfit}}$ $=2.0$. Figure 4: F277W $G-M_{20}$
space for observed CEERS galaxies from $3<z<5$. In the left panel, observed
galaxies are color-coded by visual classifications as defined in §4.1 (Group 1
/ G1: at least two out of three volunteer classifiers assigned an interaction
class; Group 2 / G2: at least two out of three classifiers assigned the
Irregular morphology class; Group 4 / G4: one of out three classifiers
assigned an interaction class). In the middle panel and right panel, galaxies
are color-coded by the merger probability output by the random forest or
neural network, respectively.

###  4.2 Classical Techniques: Performance on Observed CEERS Galaxies

Past studies have often used classical techniques such as $G$ vs. $M_{20}$
(Lotz et al., 2004, 2008) to identify mergers. In such plots, the merger
discriminating line is

$G>-0.14M_{20}+0.33,$ (1)

defined by Lotz et al. (2008). The first panel of Figure 4 shows $G$ vs.
$M_{20}$ for the observed CEERS dataset, color-coded by visual classification
(Groups 1, 2, and 4 have been abbreviated as G1, G2, and G4 in the legend).
$G$ vs. $M_{20}$ does not appear to effectively separate visually classified
mergers and non-mergers, which is to be expected since $G-M_{20}$ is not
sensitive to all stages of a merger. The fraction of correctly classified
mergers in each group, based on the merger discriminating line, is only
$27.6\%$ for Group 1, $21.7\%$ for Group 2, and $19.2\%$ for Group 4. Of the
objects in the merger region above the merger discriminating line, only
$35.6\%$ are actually Group 1 mergers, $57.6\%$ are actually Group 2 mergers,
and $62.7\%$ are actually Group 4 mergers. Most objects tend to fall in the
late-type region of the plot. This performance of the $G-M_{20}$ method is
similar to that in Figure 6 of Rose et al. (2023) for the simulated CEERS
data, where only $19-23\%$ of the simulated mergers were correctly classified,
and only $4-50\%$ of predicted mergers were actually mergers.

$G-M_{20}$ has been shown to be unable to select a pure sample of mergers in
other studies. For example, Conselice et al. (2008) measured $G$ and $M_{20}$
for visually-classified HST UDF galaxies in several redshift bins from
$0.4<z<3$ and find that, besides peculiars/mergers, most other galaxy types
were actually also found above the merger discriminating line. In their last
redshift bin of $2.5<z<3$, they find that $\sim 93\%$ of peculiars are found
in the merger region, and of all objects in the merger region, $\sim 67\%$ are
peculiars. Pearson et al. (2019b) plot $G-M_{20}$ for $0.002<z<0.15$ KiDS-GAMA
galaxies with visual classifications from Galaxy Zoo. They report no specific
numbers for the fraction of mergers selected by $G-M_{20}$, but find that most
mergers and many non-mergers appear in the merger region. However, they then
also include visual classifications from Darg et al. (2010a, b), and find that
most Darg et al. (2010a, b) mergers lie in the non-merger region. Kartaltepe
et al. (2010) plot $G-M_{20}$ for $0.01<z<3.5$ visually-classified COSMOS
galaxies and also do not report specific numbers, but find that most (major
and minor) mergers appear in the non-merger region. They explored several
automated methods besides $G-M_{20}$ and find that no technique is sensitive
to all merger stages. Snyder et al. (2019) determine $G-M_{20}$ for simulated
Illustris-1 galaxies, with mergers defined by a timescale of $\pm 250$ Myr,
and also do not report specific numbers, but found that diagnostics including
$G-M_{20}$ select a highly incomplete sample of mergers, as most mergers
appeared in the non-merger region.

The second and third panels show the merger classifications predicted by the
random forest (§4.5) and the DeepMerge network (§4.6), respectively. These
panels are both color-coded by the color bar at the far right, which
represents the merger probability output by either the random forest or neural
network. Probabilities higher than 0.5 mean the object was classified as a
merger (and colored purple or light purple). Probabilities lower than 0.5 mean
the object was classified as a non-merger (and colored orange or yellow). See
§5.1 for further discussion of these two panels.

###  4.3 Random Forests: Performance on Simulated CEERS Galaxies

Here we summarize the results from training random forests on mock CEERS
galaxies from IllustrisTNG in Rose et al. (2023). Across all redshift bins,
the forests accurately classified about $\sim 60\%$ of merging and non-merging
galaxies (according to our merger definition of $\pm 250$ Myr). The forests
performed better than both a random classifier and classical techniques. For
lower redshift bins, rest-frame asymmetry features were more important and for
higher redshift bins, rest-frame bulge and clump features were more important
to the forests for merger identification. The feature importances were
calculated by the random forest and accessed via the feature_importances_
attribute. The forests misclassified non-mergers that had segmentation maps
contaminated with emission from nearby or background galaxies or had merger
signature persisting from outside the chosen merger time frame. The forests
also misclassified mergers that appeared visually undisturbed. Finally, the
merger fraction and merger rate calculated using random-forest-selected
mergers were underestimated compared to previous works that used Illustris-1.

Figure 5: Left: Random forest confusion matrix for $4<z<5$ objects from the
test set. The diagonal shows the fraction of objects correctly classified for
each class. Right: ROC curve for the $4<z<5$ random forest. The training set
(green) and test set (purple) curves lie in the region of “good” classifiers
(between the perfect (black) and no skill (red) classifiers).

In this work, we additionally trained a random forest for $z=4-5$. This forest
correctly classified $59\%$ of non-merging galaxies and $67\%$ of merging
galaxies. The confusion matrix and receiver operating characteristic (ROC)
curve are shown in Figure 5, which indicate similar performance to the forests
in Rose et al. (2023).

###  4.4 DeepMerge: Performance on Simulated CEERS Galaxies

Figure 6: Top: DeepMerge confusion matrices for the simulated CEERS test set,
for each redshift bin. Bottom: Corresponding ROC curves for the simulated
CEERS test set, for each redshift bin. The training set (green) and test set
(purple) curves lie in the region of “good” classifiers (between the perfect
(black) and no skill (red) classifiers).

Figure 6 shows the confusion matrices and ROC curves resulting from applying
our trained DeepMerge models to the simulated CEERS test set. Our DeepMerge
networks correctly classify $\sim 60-70\%$ of merging and non-merging
galaxies. The ROC curves show that the networks perform better than a random
classifier. As in Rose et al. (2023), we categorize the test set results into
four classes:

  * •

True Positives (TP): the number of true mergers correctly classified by the
random forest.

  * •

False Positives (FP): the number of non-mergers incorrectly classified as
mergers.

  * •

True Negatives (TN): the number of correctly classified non-mergers.

  * •

False Negatives (FN): the number of true mergers incorrectly classified as
non-mergers.

There are several metrics used to quantify the performance of the neural
networks:

  * •

True Positive Rate (TPR) – also known as recall or completeness:

$TPR=\textrm{recall}=\frac{TP}{TP+FN}.$ (2)

  * •

False Positive Rate (FPR) – also known as fall out:

$FPR=\frac{FP}{FP+TN}.$ (3)

  * •

Positive Predictive Value (PPV) – also known as precision:

$PPV=\textrm{precision}=\frac{TP}{TP+FP}.$ (4)

  * •

F1 Score – the harmonic mean of precision (P) and recall (R):

$F_{1}=2\frac{P\times R}{P+R}=\frac{TP}{TP+\frac{1}{2}(FP+FN)}.$ (5)

Since our simulated CEERS datasets are imbalanced, the F1 score is a more
useful metric than accuracy for comparing the performance of our models. Table
3 compares the F1 scores of the random forests and DeepMerge networks for the
simulated data. Based on the F1 scores, the DeepMerge networks technically
performed better than the random forests in each redshift bin, particularly
the $3.5<z<4$ bin.

Redshift Bin | $3-3.5$ | $3.5-4$ | $4-5$
---|---|---|---
RF Non-mergers | 0.67 | 0.65 | 0.62
NN Non-mergers | 0.70 | 0.75 | 0.64
RF Mergers | 0.52 | 0.54 | 0.63
NN Mergers | 0.53 | 0.59 | 0.66
Table 3: Comparison of F1 scores for the random forests (RF) and DeepMerge
neural networks (NN) on the simulated CEERS test sets.

As in Ćiprijanović et al. (2020), we investigate the use of Gradient-weighted
Class Activation Mapping (Grad-CAM) maps (Selvaraju et al., 2020). These
mappings, calculated from the gradient of the predicted class of the input
image with respect to the activations in the last convolutional layer, allow
us to validate that the network is “looking” at the correct regions in the
input image. However, for our complex data with six filters, the Grad-CAMs
were not directly interpretable and showed a six-by-six pattern of activations
all around the input image. Therefore, we investigate Grad-CAMs from one of
the previous tests described in §3.4, where we use only one rest-frame filter
rather than all six. Figure 7 shows some examples of Grad-CAMs from the
$z=4-5$ bin trained using only the F356W filter. For this test, the network
correctly classified $62\%$ of simulated CEERS non-mergers and $53\%$ of
simulated CEERS mergers. As shown in Figure 7, some Grad-CAMs from this F356W
filter test show that the network does in fact activate around the edges of
the central galaxies and around the edges of neighbors as we would expect, but
other Grad-CAMs show only activations in a few random pixels. The
distributions of the F356W magnitude and signal-to-noise per pixel indicate
that objects with these “bad” Grad-CAMs are not necessarily more likely to
have a fainter magnitude and lower S/N. The peak magnitude for objects with
“bad” Grad-CAMs is $28.8\pm 1.2$, while the peak magnitude for objects with
“good” Grad-CAMs is $28.1\pm 1.5$. Of objects with “bad” Grad-CAMs, $55.7\%$
were correctly classified (either true positives or true negatives) and
$44.4\%$ were incorrectly classified (either false positive or false
negative). Of the objects with “good” Grad-CAMs, $57.6\%$ were correctly
classified and $42.4\%$ were incorrectly classified.

Figure 7: Examples of simulated CEERS galaxies and their corresponding Grad-
CAMs from the single-filter DeepMerge test using F356W images. The bottom left
corner of the F356W image stamps shows the categorization (e.g., true
positive, false positive, etc.) of this object and the DeepMerge merger
probability. The red crosses show the location of the center of each image.

###  4.5 Random Forests: Performance on Observed CEERS Galaxies

After training on the simulated images, we apply the random forest algorithm
to observed CEERS galaxies, selected in §4.1, at the following redshift bins:
$3<z<3.5$, $3.5<z<4$, and $4<z<5$. Figure 8 shows the confusion matrix for the
$3<z<3.5$ redshift bin for Group 1 (Figure 13 in the Appendix shows all the
Group 1, 2, and 4 confusion matrices for each redshift bin). Here, we use the
visual classifications as the true merger labels for the comparison. Since the
visual classifications are subjective and volunteer classifiers may
misclassify objects or disagree on classifications, the true merger labels may
not be correct. In summary, the forests generally did not perform as well on
the observed galaxies as they did on the simulated galaxies in the test sets.
The only dataset where the random forest performed better than on the test set
was Group 1 at $3<z<3.5$, where the forest correctly classified $63\%$ of non-
mergers and $66\%$ of mergers. The forests performed the worst in the highest
redshift bin, where only $25\%-30\%$ of mergers were correctly classified.
Table 4 shows how the precision, recall, and F1-score change as a function of
redshift for these groups.

Figure 8: Random forest confusion matrix for observed CEERS galaxies at $3<z<3.5$.  Figure 9: DeepMerge neural network confusion matrix for observed CEERS galaxies at $3<z<3.5$.  |  |  | $3.0<z<3.5$ | $3.5<z<4$ | $4<z<5$
---|---|---|---|---|---
|  |  | Non-merger | Merger | Non-merger | Merger | Non-merger | Merger
|  | Precision | 0.88 | 0.32 | 0.71 | 0.19 | 0.84 | 0.24
| Group 1 | Recall | 0.63 | 0.66 | 0.38 | 0.48 | 0.82 | 0.27
|  | F1-score | 0.74 | 0.43 | 0.50 | 0.27 | 0.83 | 0.25
|  | Precision | 0.64 | 0.42 | 0.51 | 0.40 | 0.57 | 0.76
Random Forest | Group 2 | Recall | 0.60 | 0.46 | 0.38 | 0.53 | 0.91 | 0.30
|  | F1-score | 0.62 | 0.44 | 0.43 | 0.46 | 0.70 | 0.43
|  | Precision | 0.59 | 0.63 | 0.56 | 0.48 | 0.41 | 0.82
| Group 4 | Recall | 0.68 | 0.53 | 0.43 | 0.61 | 0.91 | 0.25
|  | F1-score | 0.63 | 0.58 | 0.49 | 0.54 | 0.57 | 0.39
|  | Precision | 0.72 | 0.19 | 0.64 | 0.22 | 0.73 | 0.14
| Group 1 | Recall | 0.16 | 0.76 | 0.09 | 0.83 | 0.22 | 0.60
|  | F1-score | 0.26 | 0.31 | 0.16 | 0.34 | 0.34 | 0.22
|  | Precision | 0.53 | 0.37 | 0.45 | 0.42 | 0.41 | 0.46
DeepMerge Network | Group 2 | Recall | 0.15 | 0.79 | 0.09 | 0.86 | 0.20 | 0.70
|  | F1-score | 0.24 | 0.50 | 0.15 | 0.56 | 0.27 | 0.56
|  | Precision | 0.44 | 0.49 | 0.27 | 0.43 | 0.23 | 0.58
| Group 4 | Recall | 0.15 | 0.80 | 0.06 | 0.83 | 0.16 | 0.69
|  | F1-score | 0.23 | 0.61 | 0.09 | 0.57 | 0.19 | 0.63
Table 4: Precision, recall, and F1-score for merger identification of observed
CEERS galaxies, for both random forests and DeepMerge. Group 1 (G1) mergers
are observed galaxies for which at least two out of three volunteer
classifiers assigned an interaction class. Group 2 (G2) mergers are observed
galaxies for which at least two out of three volunteer classifiers assigned
the Irregular morphology class. Group 4 (G4) mergers are observed galaxies for
which at least one out of three volunteer classifiers assigned an interaction
class.

###  4.6 DeepMerge: Performance on Observed CEERS Galaxies

After training on the simulated images, we apply our DeepMerge networks to
observed CEERS galaxies at the same redshift bins of $3<z<3.5$, $3.5<z<4$, and
$4<z<5$. Figure 9 shows the confusion matrix for the $3<z<3.5$ redshift bin
for Group 1 (Figure 14 in the Appendix shows all the Group 1, 2, and 4
confusion matrices for each redshift bin). In summary, the networks did not
perform as well on the observed galaxies as they did on the simulated galaxies
in the test sets. At $3<z<3.5$ and $4<z<5$ the networks classified $\sim
15-20\%$ of non-mergers correctly and $\sim 70-80\%$ of mergers correctly
across all visual classification groups. The networks performed the worst at
$3.5<z<4$, with only $6-9\%$ of non-mergers correctly classified and $\sim
85\%$ of mergers correctly classified for all visual classification groups.
Table 4 shows how the precision, recall, and F1-score change as a function of
redshift for these groups.

Based on the confusion matrices, the DeepMerge networks tend to incorrectly
classify more objects as mergers than the random forests. In addition to the
strategies employed in §3.4, we additionally tried the following strategies
during training to prevent underfitting and improve performance on the
observed CEERS data set:

  1. 1.

Giving DeepMerge only one rest-frame filter to train on, or summing the six
filters together into one image, in order to reduce the complexity of the
problem.

  2. 2.

Reducing the number of layers in the network, in order to reduce the
complexity of the network.

  3. 3.

Adjusting the probability threshold for merger classification (default 0.5),
such that only objects with merger probabilities higher than $0.55$, $0.6$, or
$0.65$ were identified as mergers.

The results of these tests did not significantly improve performance on the
observed CEERS data set. While some tests did result in confusion matrices
that were slightly more balanced, the networks still either classified almost
all objects as mergers or almost all objects as non-mergers.

##  5 Discussion

###  5.1 Classical Techniques: Observed CEERS Galaxies with Machine Learning
Classifications

The second and third panels of Figure 4 show $G$ vs. $M_{20}$ for the observed
CEERS dataset, color-coded by the merger probabilities predicted by the random
forests and the DeepMerge network, respectively. Objects were classified as
mergers if the merger probability was greater than 0.5, and classified as non-
merger if the merger probability was less than 0.5. As we saw in §4.2, the
fraction of correctly classified mergers in each group, based on the merger
discriminating line, ranged from only $19.2-27.6\%$. Thus, the machine
learning algorithms were able to correctly classify more mergers than the
$G-M_{20}$ method, and Figure 4 shows that many of these mergers are below the
$G-M_{20}$ merger discriminating line as expected given that $G-M_{20}$ only
selects mergers at a narrow portion of the merger phase. Therefore the
algorithms are able to identify mergers that the $G-M_{20}$ method misses due
$G-M_{20}$’s limited sensitivity. A visual inspection of the second panel of
Figure 4 reveals a potential trend where objects with very low probabilities
from the random forest are far below the merger discriminating line while
objects with higher probabilities are closer to or above the line. A visual
inspection of the third panel of Figure 4 shows that most galaxies were
classified as mergers with relatively high probabilities, and there appears to
be no clear trend in relation to the merger discriminating line.

###  5.2 Feature Importance

In Rose et al. (2023), we calculated the relative importance of the features
(morphology parameters) given to the random forests, which shows how useful a
given feature is for identifying simulated mergers. We found that at lower
redshift, bluer filters and asymmetry features were more useful, while at
higher redshift, redder filters and bulge and clump features were more useful.
Here we calculate feature importance for the $4<z<5$ random forest for
simulated galaxies, which are (in order of importance):

  1. 1.

F277W multimode ($M$)

  2. 2.

F444W moment of light ($M_{20}$)

  3. 3.

F356W Gini ($G$)

  4. 4.

F444W concentration ($C$)

  5. 5.

F356W concentration ($C$)

This continues the trend seen in Figure 11 in Rose et al. (2023) where redder
filter and bulge and clump features are more important at higher redshift.

Of the most correlated features identified by the Spearman correlation in
§3.3, we see that only one of those features was present in the top five most
important features calculated by the random forest, indicating that the
Spearman correlation is not necessarily a good predictor of the random forest
results.

It is important to note that morphology parameters are known to be sensitive
to different merger stages. For example, Snyder et al. (2015) find that, when
simulating a galaxy as observed with HST from $z\sim 3$ to $z\sim 1$ (with a
merger at $z=1.6$), the $G-M_{20}$ merger statistic is sensitive to the early
phases of the merger, while the $MID$ statistics are more sensitive after
coalescence. Future studies with larger datasets could benefit from
investigating random forest feature importance as a function of merger stage,
in addition to redshift.

###  5.3 Galaxy Categorization Results

Figure 10 shows the categorization of the mock CEERS test set at $3.5<z<4$
into true positives, false positives, true negatives, and false negatives as
output by DeepMerge. The top left hand corner of each stamp gives the
probability of the object being a merger as determined by the DeepMerge
network. The stamps are arranged in order of decreasing probability, so the
horizontal orange line effectively represents the 0.5 probability threshold
between merger and non-merger classifications. In each redshift bin, the
probabilities range from $\sim 0.3-0.7$, which is similar to the range of
probabilities output by the random forest.

In Figure 10, the bottom left hand corner of each stamp shows the merger
timescale for each galaxy, which includes both major and minor mergers. A
positive timescale indicates the time since a past merger, and a negative
timescale indicates the time until a future merger. Galaxies can have both
past and future mergers, so whichever timescale is smallest is shown here.
True positives and false negatives will have merger timescales $<0.25$ Gyr,
while false positives and true negatives will have merger timescales $>0.25$
Gyr. Finally, the segmentation map outlines are color-coded by whether the
given timescale corresponds to a major or minor merger.

Figure 10: Examples of true positive (TP), false positive (FP), true negative
(TN), and false negative (FP) (as defined in §4.4) simulated CEERS galaxies in
the F356W filter from the $3.5<z<4$ redshift bin. Each stamp is 3 x 3 arcsec.
Within each stamp, the merger probability output by the DeepMerge network is
in the upper left and the timescale since or until the most recent merger
(major or minor) is in the bottom left. The outlines show the segmentation
map, color-coded by major (magenta) and minor (green) mergers, respectively.

Figure 11: Examples of true positive (TP), false positive (FP), true negative
(TN), and false negative (FP) observed CEERS galaxies (from Group 1) in the
F356W filter from the $3.5<z<4$ redshift bin categorized by the random forest
(top) or the DeepMerge network (bottom). Each stamp is 3 x 3 arcsec. Within
each stamp, the merger probability output by the respective algorithm is in
the upper left. The white outlines are segmentation map outlines.

Like the false positives in Rose et al. (2023), the false positives in Figure
10 tend to have irregular segmentation maps due to neighboring or background
galaxies, or asymmetric features that are possibly due to a merging event
outside of our merger timescale of $\pm 250$ Myr. The average past and future
timescales of true negatives are $0.72\pm 0.30$ Gyr and $0.62\pm 0.24$ Gyr,
respectively. The average past and future timescales of false positives are
$0.49\pm 0.17$ Gyr and $0.41\pm 0.19$ Gyr, respectively. This indicates that
false positives are slightly more likely to be closer in time to a merging
event than other non-mergers. On the other hand, the false negatives appear
less disturbed, possibly due to minor mergers that would have a minor effect
on morphology. The fraction of minor mergers among true positives and false
negatives in the simulated CEERS test set is $42.9\%$ and $52.4\%$,
respectively, which indicates that false negatives are slightly more likely to
be minor mergers. This contrasts with the random forest results in Rose et al.
(2023) where false positives were not more likely to be closer in time to a
merging event and false negatives were not more likely to be minor mergers.

As with the simulated CEERS images, we also categorize the observed CEERS
images into true positives, false positives, true negatives, and false
negatives. Figure 11 shows this categorization resulting from the random
forest and from the DeepMerge network. In each case, the “true” label is based
on Group 1 visual classification (whether or not two or more people said the
object was a merger or interaction). As with Figure 10, the probability of the
object being a merger as determined by each algorithm is given in the upper
left hand corner of each stamp, and stamps are arranged in decreasing
probability.

One reason for the poorer performance of these algorithms on the observed data
is that the random forest and the DeepMerge network were both trained on data
where the ground truth labels were determined by the Illustris past or future
merger timescales within $\pm 250$ Myr, while the ground truth labels for the
observed CEERS galaxies were based on visual classifications. It is possible
that if the algorithms could have been trained on simulated data with labels
based on visual classifications, or applied to observed data where the labels
were somehow determined by merger timescales, then the algorithms may perform
better. Another reason is that the observed galaxies’ visual classifications
are subjective and were determined by only three classifiers. In some cases,
it appears that a different visual classification could be appropriate. For
example, the random forest’s second false positive in Figure 11 (or
DeepMerge’s first false positive in Figure 11) could in fact be a merger.
Likewise, DeepMerge’s second false negative in Figure 11 could be a non-
merger. If these objects had been assigned the opposite visual classification,
then the algorithms would have correctly classified them.

###  5.4 Merger Rates

Figure 12: Merger rate based on visual classifications (left panel), random
forest classifications (middle panel), and neural network classifications
(right panel) as compared to the theoretical Illustris merger rate (black
line) derived from Rodriguez-Gomez et al. (2015). The green shaded regions and
the observed galaxy error bars indicate the binomial 95% confidence interval.

Figure 12 shows the simulated CEERS and observed CEERS merger rates. In each
panel, the black line is the theoretical Illustris merger rate derived by
Rodriguez-Gomez et al. (2015) assuming a merger timescale of 0.5 Gyr, for
comparison. The second and third panels show the simulated CEERS merger rate
as determined by the random forests (§4.3) and by the DeepMerge network
(§4.4), respectively, and are in good agreement with each other.

For the DeepMerge-selected simulated CEERS mergers, we calculate the merger
rate in each of the three redshift bins. The merger rate was calculated
following the same procedure as in Rose et al. (2023), first using the
following equation to calculate the merger fraction:

$f_{\mathrm{merger}}(\mathrm{NN})=\frac{N_{NN}}{N}\frac{PPV}{TPR}<\frac{M}{N}>.$
(6)

Then we divide the merger fraction by our merger timescale of $0.5$ Gyr to
obtain the merger rate. Here, $N_{NN}$ is the number of mergers determined by
the DeepMerge network and $N$ is the total number of objects in our sample in
each redshift bin. PPV is the positive predictive value, or precision, of the
classifier. TPR is the true positive rate, or recall, of the classifier.
$PPV/TPR$ is meant to correct for the classifier’s known incompleteness and
purity based on the training set. $<M/N>$ is the average number of merging
events per true merger, and accounts for the possibility that mergers can
experience more than one merging event within the $\pm 250$ Myr timescale used
to determine the labels for the neural network. The DeepMerge simulated CEERS
merger rate is shown as the green line in the right panel of Figure 12, and is
in good agreement with the random forest simulated CEERS merger rate.

The first panel of Figure 12 shows the observed CEERS merger rate for each
visual classification group. This was calculated by first determining the
merger fraction using simply:

$f_{\mathrm{merger}}(\mathrm{obs})=\frac{N_{obs}}{N}.$ (7)

We then divide the merger fraction by our timescale of $0.5$ Gyr used for the
simulated data. Here, $N_{obs}$ is the number of mergers based on our visual
classification groups, and $N$ is the total number of objects in our observed
CEERS sample in each redshift bin. The second and third panels also give the
observed CEERS merger rate based on the machine learning outputs. These merger
rates were calculated as in Equation 6, where $PPV/TPR$ is from the respective
machine learning algorithm and $<M/N>$ is calculated from the simulated data.
Like the simulated merger rates, the observed merger rates in the first and
second panel are underestimated compared to the theoretical Illustris merger
rate. However, the DeepMerge observed merger rate is higher than the simulated
CEERS merger rates, the theoretical Illustris merger rate, and the observed
merger rates in the first panel due to the fact that DeepMerge overestimated
the number of mergers. The random forest observed merger rate is in better
agreement with the observed merger rates in the first panel. In addition, the
observed merger rates appear relatively flat while the simulated mergers rates
increase at $z>4$.

###  5.5 Comparison with Previous Works

As discussed in §1, few studies apply convolutional neural networks to the
task of merger identification at high redshift. We compare our work to
Ćiprijanović et al. (2020), Ferreira et al. (2020), and Pearson et al.
(2019b). The DeepMerge network is introduced in Ćiprijanović et al. (2020) to
identify $z=2$ simulated Illustris-1 HST mergers. They have access to four
different “camera” perspectives, which they use to create a sample of 8930
unique galaxies. They then augment the merger class to create a more balanced
sample of 8120 mergers and 7306 non-merger before they split their data into
train and test sets. Their merger class consists of mergers with a stellar
mass ratio of at least 0.1 and within a time frame of $\pm 250$ Myr. They find
that DeepMerge achieves an accuracy of $76\%$ when training and testing on
noisy data. They report correctly classifying $78\%$ of mergers and $73\%$ of
non-mergers in their noisy test set. In Ćiprijanović et al. (2021), they use
domain adaption techniques to apply their trained network to real data;
however, this is done at $0.005<z<0.1$ using data from the Galaxy Zoo project
(Lintott et al., 2008, 2011).

Ferreira et al. (2020) use simulated IllustrisTNG300-1 CANDELS major mergers
(up to $z\sim 3$) within $\pm 0.3$ Gyr and non-mergers. Additionally, they
impose a limit on the separation between each pair of galaxies. With these
criteria, they curate a sample of $\sim 30,000$ distinct major merger
candidates. For the non-mergers, they impose a timescale limit of $\pm 0.5$
Gyr (i.e., the galaxy must have a merger timescale outside this time frame).
Using these criteria, they build up a random sample of non-mergers that equals
the number of mergers in each redshift bin. They train two classifiers – one
to identify mergers vs. non-mergers, and a second to further subdivide mergers
into pre- and post-mergers – and are able to correctly classify $87\%$ and
$94\%$ or pre- and post-mergers, respectively. They apply their methodology to
a sample of 3759 real CANDELS galaxies with visual classifications given by
Kartaltepe et al. (2015), but do not report an accuracy for this test. They do
note that since their validation sample is balanced while the CANDELS sample
is not, the performance of the model on simulated data will not translate to
the same performance of the model on real data.

Finally, Pearson et al. (2019b) both train and test their convolutional neural
network on observed CANDELS galaxies up to $z<4$, in the F814W, F125W, and
F160W bands, using visual classifications from Kartaltepe et al. (2015). They
construct training samples of 694 objects in each class in each of their five
redshift bins from $0<z<4$. They also augment their images to create a larger
training set. They report a final accuracy of $81.8\%$ after testing on the
full redshift range of $0<z<4$.

Each of these studies reports better performance than we report in this work,
which could be due to a number of differences. First, Ćiprijanović et al.
(2020) and Ferreira et al. (2020) use Illustris-1 and IllustrisTNG300-1,
respectively, while we use IllustrisTNG100-1. Ferreira et al. (2020) impose
additional criteria to define their merger sample and non-merger samples.
Perhaps most importantly, both Ćiprijanović et al. (2020) and Ferreira et al.
(2020) are able to curate a large training set, even before augmentation. All
three studies also use balanced training, validation, and test sets, whereas
our validation and test sets are left unbalanced since real data will be
unbalanced. Additionally, Ćiprijanović et al. (2020) augment their data prior
to splitting their data into training, validation, and test sets, and it is
unclear if they ensured that all augmentations of a given galaxy end up in the
same subset. Otherwise, the training, validation, and test sets would be
correlated which could inflate the accuracy of the network. Ferreira et al.
(2020) do not specifically comment on the performance of their first
classifier, which separates mergers from non-mergers. Pearson et al. (2019b)
do not specifically report performance on their non-merger class. Thus these
papers may have focused on optimizing performance to correctly identify most
mergers, rather than focusing on the purity of the merger and non-merger
classifications, similarly to Snyder et al. (2019). Finally, Ćiprijanović et
al. (2020) and Ferreira et al. (2020) study galaxies at $z<3$ with HST images
while we push to $3<z<5$ with JWST. Pearson et al. (2019b) do extend out to
$z=4$, but their training set consists for data from the full range of
$0<z<4$.

##  6 Conclusions

In this work, we investigate using the DeepMerge convolutional neural network
algorithm to identify merging galaxies at $3<z<5$ in simulated CEERS images
constructed from IllustrisTNG and the Santa Cruz SAM. Additionally, we
calculate morphology parameters for observed CEERS galaxies using the
morphology programs Galapagos-2 and statmorph. We apply the trained random
forests from Rose et al. (2023), a $4<z<5$ random forest trained in this work,
and the DeepMerge neural network trained in this work to observed CEERS
galaxies, and verify using visual classifications from Kartaltepe et al.
(2023). We find the following results:

  1. 1.

The $4<z<5$ random forest performed similarly to those in Rose et al. (2023),
correctly classifying $59\%$ of non-merging galaxies and $67\%$ of merging
galaxies.

  2. 2.

The DeepMerge network correctly classified $\sim 60-70\%$ of simulated merging
and non-merging galaxies. Although Grad-CAMs from our tests using six filters
were not clearly interpretable, Grad-CAMs from our tests using one filter
showed that the networks usually activated around the edges of galaxies as
expected.

  3. 3.

The random forests and neural networks did not perform as well when applied to
real CEERS galaxies. In particular, the neural network tended to classify most
objects ($60-80\%$) as mergers. This could be due to a number of reasons, such
as the different methods of defining mergers for the simulated data and the
observed data.

  4. 4.

As in Rose et al. (2023), DeepMerge false positives tend to appear to have
close neighbors or asymmetric features, and false negatives appear less
disturbed. The false positives appear to be more likely to be closer in time
to merging events than the true negatives and false negatives appear more
likely to be minor mergers than true positives. The categorization of observed
CEERS galaxies shows that positive classifications do look more irregular and
asymmetric and negative classifications do look more undisturbed, but also
that visual classifications are subjective.

  5. 5.

Merger rates from (a) observed visual classifications only, (b) RF-selected
observed merger classifications, and (c) DeepMerge-selected simulated merger
classifications generally agree with each other and with the RF-selected
simulated merger rate from Rose et al. (2023). The merger rate calculated
using DeepMerge-selected observed merger classifications is overestimated
comparatively, due to the fact that DeepMerge misclassified almost all
observed non-mergers.

###  6.1 Future Work

One clear area of improvement is curating a larger sample of simulated mergers
in order to increase the number of objects in the training set which could
help the network better learn merger signatures. This could be done by
remaking the pristine simulated images using the larger IllustrisTNG300-1
simulation (as in Ferreira et al., 2020) or using another large volume, high
resolution simulation. A larger sample would also enable the performance as a
function of other physical parameters that can impact the observed merger
signatures, such as the gas fraction of the galaxies, their merger ratio, and
their relative orbits. Another area of improvement would be investigating the
use of transfer learning techniques to improve performance on the observed
simulated CEERS data. Ćiprijanović et al. (2021) show that using transfer
learning techniques improves the performance of their simulation-trained
networks on observed data by up to $\sim 20\%$. Finally, it would be
interesting to train and test machine learning techniques directly on observed
CEERS galaxies (similar to Pearson et al., 2019b) once more visual
classifications become available. It would also be interesting to compare the
Grad-CAM maps to the regions of the images for which the morphology parameters
are sensitive, in order to better understand the features that the neural
network has learned.

Support for this work was provided by NASA through grants JWST-ERS-01345.015-A
and HST-AR-15802.001-A awarded by the Space Telescope Science Institute, which
is operated by the Association of Universities for Research in Astronomy,
Inc., under NASA contract NAS 5-26555. This research is based in part on
observations made with the NASA/ESA Hubble Space Telescope obtained from the
Space Telescope Science Institute, which is operated by the Association of
Universities for Research in Astronomy, Inc., under NASA contract NAS 5–26555.
This work is based in part on observations made with the NASA/ESA/CSA James
Webb Space Telescope. The data were obtained from the Mikulski Archive for
Space Telescopes at the Space Telescope Science Institute, which is operated
by the Association of Universities for Research in Astronomy, Inc., under NASA
contract NAS 5-03127 for JWST. These observations are associated with program
#1345. The authors acknowledge Research Computing at the Rochester Institute
of Technology for providing computational resources and support that have
contributed to the research results reported in this publication (Rochester
Institute of Technology, 2019) The authors acknowledge the Texas Advanced
Computing Center (TACC) at The University of Texas at Austin for providing HPC
resources that have contributed to the research results reported within this
paper (http://www.tacc.utexas.edu).

##  Appendix A Confusion matrices for observed CEERS galaxies

Figure 13 shows the results of applying the trained random forests to observed
CEERS galaxies in §4.5. Figure 14 shows the results of applying the trained
DeepMerge neural networks to observed CEERS galaxies in §4.6.

Figure 13: Random forest confusion matrices for observed CEERS galaxies at
$3<z<3.5$ (top row), at $3.5<z<4$ (middle row), and at $4<z<5$ (bottom row).

Figure 14: DeepMerge neural network confusion matrices for observed CEERS
galaxies at $3<z<3.5$ (top row), at $3.5<z<4$ (middle row), and at $4<z<5$
(bottom row).

## References

  * Abraham et al. (2003) Abraham, R. G., van den Bergh, S., & Nair, P. 2003, ApJ, 588, 218, doi: 10.1086/373919
  * Ackermann et al. (2018) Ackermann, S., Schawinski, K., Zhang, C., Weigel, A. K., & Turp, M. D. 2018, MNRAS, 479, 415, doi: 10.1093/mnras/sty1398
  * Bagley et al. (2023) Bagley, M. B., Finkelstein, S. L., Koekemoer, A. M., et al. 2023, ApJ, 946, L12, doi: 10.3847/2041-8213/acbb08
  * Bamford et al. (2011) Bamford, S. P., Häußler, B., Rojas, A., & Borch, A. 2011, in Astronomical Society of the Pacific Conference Series, Vol. 442, Astronomical Data Analysis Software and Systems XX, ed. I. N. Evans, A. Accomazzi, D. J. Mink, & A. H. Rots, 479
  * Barchi et al. (2020) Barchi, P. H., de Carvalho, R. R., Rosa, R. R., et al. 2020, Astronomy and Computing, 30, 100334, doi: 10.1016/j.ascom.2019.100334
  * Barden et al. (2012) Barden, M., Häußler, B., Peng, C. Y., McIntosh, D. H., & Guo, Y. 2012, MNRAS, 422, 449, doi: 10.1111/j.1365-2966.2012.20619.x
  * Bershady et al. (2000) Bershady, M. A., Jangren, A., & Conselice, C. J. 2000, AJ, 119, 2645, doi: 10.1086/301386
  * Bertin & Arnouts (1996) Bertin, E., & Arnouts, S. 1996, A&AS, 117, 393, doi: 10.1051/aas:1996164
  * Bickley et al. (2021) Bickley, R. W., Bottrell, C., Hani, M. H., et al. 2021, MNRAS, 504, 372, doi: 10.1093/mnras/stab806
  * Bottrell et al. (2019) Bottrell, C., Hani, M. H., Teimoorinia, H., et al. 2019, MNRAS, 490, 5390, doi: 10.1093/mnras/stz2934
  * Breiman (2001) Breiman, L. 2001, Machine Learning, 45, 5, doi: 10.1023/A:1010933404324
  * Bushouse et al. (2022) Bushouse, H., Eisenhamer, J., Dencheva, N., et al. 2022, JWST Calibration Pipeline, 1.8.2, Zenodo, Zenodo, doi: 10.5281/zenodo.7325378
  * Casertano et al. (2000) Casertano, S., de Mello, D., Dickinson, M., et al. 2000, AJ, 120, 2747, doi: 10.1086/316851
  * Cheng et al. (2020) Cheng, T.-Y., Conselice, C. J., Aragón-Salamanca, A., et al. 2020, MNRAS, 493, 4209, doi: 10.1093/mnras/staa501
  * Ćiprijanović et al. (2020) Ćiprijanović, A., Snyder, G. F., Nord, B., & Peek, J. E. G. 2020, Astronomy and Computing, 32, 100390, doi: 10.1016/j.ascom.2020.100390
  * Ćiprijanović et al. (2021) Ćiprijanović, A., Kafkes, D., Downey, K., et al. 2021, MNRAS, 506, 677, doi: 10.1093/mnras/stab1677
  * Conselice (2003) Conselice, C. J. 2003, ApJS, 147, 1, doi: 10.1086/375001
  * Conselice et al. (2000) Conselice, C. J., Bershady, M. A., & Jangren, A. 2000, ApJ, 529, 886, doi: 10.1086/308300
  * Conselice et al. (2008) Conselice, C. J., Rajgor, S., & Myers, R. 2008, MNRAS, 386, 909, doi: 10.1111/j.1365-2966.2008.13069.x
  * Costantin et al. (2021) Costantin, L., Pérez-González, P. G., Méndez-Abreu, J., et al. 2021, ApJ, 913, 125, doi: 10.3847/1538-4357/abef72
  * Costantin et al. (2022) —. 2022, ApJ, 929, 121, doi: 10.3847/1538-4357/ac5a57
  * Cox et al. (2006) Cox, T. J., Dutta, S. N., Di Matteo, T., et al. 2006, ApJ, 650, 791, doi: 10.1086/507474
  * Darg et al. (2010a) Darg, D. W., Kaviraj, S., Lintott, C. J., et al. 2010a, MNRAS, 401, 1043, doi: 10.1111/j.1365-2966.2009.15686.x
  * Darg et al. (2010b) —. 2010b, MNRAS, 401, 1552, doi: 10.1111/j.1365-2966.2009.15786.x
  * Davis et al. (2007) Davis, M., Guhathakurta, P., Konidaris, N. P., et al. 2007, ApJ, 660, L1, doi: 10.1086/517931
  * Dekel et al. (2009) Dekel, A., Sari, R., & Ceverino, D. 2009, ApJ, 703, 785, doi: 10.1088/0004-637X/703/1/785
  * Dieleman et al. (2015) Dieleman, S., Willett, K. W., & Dambre, J. 2015, MNRAS, 450, 1441, doi: 10.1093/mnras/stv632
  * Ellison et al. (2013) Ellison, S. L., Mendel, J. T., Patton, D. R., & Scudder, J. M. 2013, MNRAS, 435, 3627, doi: 10.1093/mnras/stt1562
  * Ellison et al. (2008) Ellison, S. L., Patton, D. R., Simard, L., & McConnachie, A. W. 2008, AJ, 135, 1877, doi: 10.1088/0004-6256/135/5/1877
  * Ferreira et al. (2020) Ferreira, L., Conselice, C. J., Duncan, K., et al. 2020, ApJ, 895, 115, doi: 10.3847/1538-4357/ab8f9b
  * Ferreira et al. (2022) Ferreira, L., Conselice, C. J., Kuchner, U., & Tohill, C.-B. 2022, ApJ, 931, 34, doi: 10.3847/1538-4357/ac66ea
  * Finkelstein et al. (2022) Finkelstein, S. L., Bagley, M. B., Haro, P. A., et al. 2022, ApJ, 940, L55, doi: 10.3847/2041-8213/ac966e
  * Freeman et al. (2013) Freeman, P. E., Izbicki, R., Lee, A. B., et al. 2013, MNRAS, 434, 282, doi: 10.1093/mnras/stt1016
  * Fruchter & Hook (2002) Fruchter, A. S., & Hook, R. N. 2002, PASP, 114, 144, doi: 10.1086/338393
  * Grogin et al. (2011) Grogin, N. A., Kocevski, D. D., Faber, S. M., et al. 2011, ApJS, 197, 35, doi: 10.1088/0067-0049/197/2/35
  * Häußler et al. (2013) Häußler, B., Bamford, S. P., Vika, M., et al. 2013, MNRAS, 430, 330, doi: 10.1093/mnras/sts633
  * Ho (1995) Ho, T. K. 1995, in Proceedings of 3rd International Conference on Document Analysis and Recognition, Vol. 1, IEEE, 278–282
  * Huertas-Company & Lanusse (2023) Huertas-Company, M., & Lanusse, F. 2023, PASA, 40, e001, doi: 10.1017/pasa.2022.55
  * Huertas-Company et al. (2018) Huertas-Company, M., Primack, J. R., Dekel, A., et al. 2018, ApJ, 858, 114, doi: 10.3847/1538-4357/aabfed
  * Huertas-Company et al. (2019) Huertas-Company, M., Rodriguez-Gomez, V., Nelson, D., et al. 2019, MNRAS, 489, 1859, doi: 10.1093/mnras/stz2191
  * Huertas-Company et al. (2023) Huertas-Company, M., Iyer, K. G., Angeloudi, E., et al. 2023, arXiv e-prints, arXiv:2305.02478, doi: 10.48550/arXiv.2305.02478
  * Kartaltepe et al. (2007) Kartaltepe, J. S., Sanders, D. B., Scoville, N. Z., et al. 2007, ApJS, 172, 320, doi: 10.1086/519953
  * Kartaltepe et al. (2010) Kartaltepe, J. S., Sanders, D. B., Le Floc’h, E., et al. 2010, ApJ, 721, 98, doi: 10.1088/0004-637X/721/1/98
  * Kartaltepe et al. (2012) Kartaltepe, J. S., Dickinson, M., Alexander, D. M., et al. 2012, ApJ, 757, 23, doi: 10.1088/0004-637X/757/1/23
  * Kartaltepe et al. (2015) Kartaltepe, J. S., Mozena, M., Kocevski, D., et al. 2015, ApJS, 221, 11, doi: 10.1088/0067-0049/221/1/11
  * Kartaltepe et al. (2023) Kartaltepe, J. S., Rose, C., Vanderhoof, B. N., et al. 2023, ApJ, 946, L15, doi: 10.3847/2041-8213/acad01
  * Kodra et al. (2023) Kodra, D., Andrews, B. H., Newman, J. A., et al. 2023, ApJ, 942, 36, doi: 10.3847/1538-4357/ac9f12
  * Koekemoer et al. (2011) Koekemoer, A. M., Faber, S. M., Ferguson, H. C., et al. 2011, ApJS, 197, 36, doi: 10.1088/0067-0049/197/2/36
  * Kormendy et al. (2009) Kormendy, J., Fisher, D. B., Cornell, M. E., & Bender, R. 2009, ApJS, 182, 216, doi: 10.1088/0067-0049/182/1/216
  * Larson et al. (2016) Larson, K. L., Sanders, D. B., Barnes, J. E., et al. 2016, ApJ, 825, 128, doi: 10.3847/0004-637X/825/2/128
  * Lin et al. (2004) Lin, L., Koo, D. C., Willmer, C. N. A., et al. 2004, ApJ, 617, L9, doi: 10.1086/427183
  * Lintott et al. (2011) Lintott, C., Schawinski, K., Bamford, S., et al. 2011, MNRAS, 410, 166, doi: 10.1111/j.1365-2966.2010.17432.x
  * Lintott et al. (2008) Lintott, C. J., Schawinski, K., Slosar, A., et al. 2008, MNRAS, 389, 1179, doi: 10.1111/j.1365-2966.2008.13689.x
  * Lotz et al. (2010a) Lotz, J. M., Jonsson, P., Cox, T. J., & Primack, J. R. 2010a, MNRAS, 404, 575, doi: 10.1111/j.1365-2966.2010.16268.x
  * Lotz et al. (2010b) —. 2010b, MNRAS, 404, 590, doi: 10.1111/j.1365-2966.2010.16269.x
  * Lotz et al. (2004) Lotz, J. M., Primack, J., & Madau, P. 2004, AJ, 128, 163, doi: 10.1086/421849
  * Lotz et al. (2008) Lotz, J. M., Davis, M., Faber, S. M., et al. 2008, ApJ, 672, 177, doi: 10.1086/523659
  * Mantha et al. (2018) Mantha, K. B., McIntosh, D. H., Brennan, R., et al. 2018, MNRAS, 475, 1549, doi: 10.1093/mnras/stx3260
  * Mantha et al. (2019) Mantha, K. B., McIntosh, D. H., Ciaschi, C. P., et al. 2019, MNRAS, 486, 2643, doi: 10.1093/mnras/stz872
  * Marinacci et al. (2018) Marinacci, F., Vogelsberger, M., Pakmor, R., et al. 2018, MNRAS, 480, 5113, doi: 10.1093/mnras/sty2206
  * Naiman et al. (2018) Naiman, J. P., Pillepich, A., Springel, V., et al. 2018, MNRAS, 477, 1206, doi: 10.1093/mnras/sty618
  * Nelson et al. (2018) Nelson, D., Pillepich, A., Springel, V., et al. 2018, MNRAS, 475, 624, doi: 10.1093/mnras/stx3040
  * Nelson et al. (2019) —. 2019, MNRAS, 490, 3234, doi: 10.1093/mnras/stz2306
  * O’Malley et al. (2019) O’Malley, T., Bursztein, E., Long, J., et al. 2019, Keras Tuner, https://github.com/keras-team/keras-tuner
  * Patton et al. (2011) Patton, D. R., Ellison, S. L., Simard, L., McConnachie, A. W., & Mendel, J. T. 2011, MNRAS, 412, 591, doi: 10.1111/j.1365-2966.2010.17932.x
  * Pearson et al. (2019a) Pearson, W. J., Wang, L., Trayford, J. W., Petrillo, C. E., & van der Tak, F. F. S. 2019a, A&A, 626, A49, doi: 10.1051/0004-6361/201935355
  * Pearson et al. (2019b) Pearson, W. J., Wang, L., Alpaslan, M., et al. 2019b, A&A, 631, A51, doi: 10.1051/0004-6361/201936337
  * Peng (2012) Peng, C. Y. 2012, Galfit User’s Manual.  {https://users.obs.carnegiescience.edu/peng/work/galfit/README.pdf}
  * Peng et al. (2002) Peng, C. Y., Ho, L. C., Impey, C. D., & Rix, H.-W. 2002, AJ, 124, 266, doi: 10.1086/340952
  * Peng et al. (2010) Peng, C. Y., Ho, L. C., Impey, C. D., & Rix, H.-W. 2010, AJ, 139, 2097, doi: 10.1088/0004-6256/139/6/2097
  * Pillepich et al. (2018) Pillepich, A., Nelson, D., Hernquist, L., et al. 2018, MNRAS, 475, 648, doi: 10.1093/mnras/stx3112
  * Pontoppidan et al. (2016) Pontoppidan, K. M., Pickering, T. E., Laidler, V. G., et al. 2016, in Society of Photo-Optical Instrumentation Engineers (SPIE) Conference Series, Vol. 9910, Observatory Operations: Strategies, Processes, and Systems VI, 991016, doi: 10.1117/12.2231768
  * Prechelt (2012) Prechelt, L. 2012, Early Stopping — But When?, ed. G. Montavon, G. B. Orr, & K.-R. Müller (Berlin, Heidelberg: Springer Berlin Heidelberg), 53–67, doi: 10.1007/978-3-642-35289-8_5
  * Rochester Institute of Technology (2019) Rochester Institute of Technology. 2019, Research Computing Services, doi: 10.34788/0S3G-QD15
  * Rodriguez-Gomez et al. (2015) Rodriguez-Gomez, V., Genel, S., Vogelsberger, M., et al. 2015, MNRAS, 449, 49, doi: 10.1093/mnras/stv264
  * Rodriguez-Gomez et al. (2017) Rodriguez-Gomez, V., Sales, L. V., Genel, S., et al. 2017, MNRAS, 467, 3083, doi: 10.1093/mnras/stx305
  * Rodriguez-Gomez et al. (2019) Rodriguez-Gomez, V., Snyder, G. F., Lotz, J. M., et al. 2019, MNRAS, 483, 4140, doi: 10.1093/mnras/sty3345
  * Rose et al. (2023) Rose, C., Kartaltepe, J. S., Snyder, G. F., et al. 2023, ApJ, 942, 54, doi: 10.3847/1538-4357/ac9f10
  * Selvaraju et al. (2020) Selvaraju, R. R., Cogswell, M., Das, A., et al. 2020, Int. J. Comput. Vis., 128, 336, doi: 10.1007/s11263-019-01228-7
  * Shah et al. (2020) Shah, E. A., Kartaltepe, J. S., Magagnoli, C. T., et al. 2020, ApJ, 904, 107, doi: 10.3847/1538-4357/abbf59
  * Snyder et al. (2015) Snyder, G. F., Lotz, J., Moody, C., et al. 2015, MNRAS, 451, 4290, doi: 10.1093/mnras/stv1231
  * Snyder et al. (2017) Snyder, G. F., Lotz, J. M., Rodriguez-Gomez, V., et al. 2017, MNRAS, 468, 207, doi: 10.1093/mnras/stx487
  * Snyder et al. (2019) Snyder, G. F., Rodriguez-Gomez, V., Lotz, J. M., et al. 2019, MNRAS, 486, 3702, doi: 10.1093/mnras/stz1059
  * Somerville et al. (2021) Somerville, R. S., Olsen, C., Yung, L. Y. A., et al. 2021, MNRAS, 502, 4858, doi: 10.1093/mnras/stab231
  * Springel et al. (2018) Springel, V., Pakmor, R., Pillepich, A., et al. 2018, MNRAS, 475, 676, doi: 10.1093/mnras/stx3304
  * Stefanon et al. (2017) Stefanon, M., Yan, H., Mobasher, B., et al. 2017, ApJS, 229, 32, doi: 10.3847/1538-4365/aa66cb
  * Tacchella et al. (2015) Tacchella, S., Carollo, C. M., Renzini, A., et al. 2015, Science, 348, 314, doi: 10.1126/science.1261094
  * Toomre (1977) Toomre, A. 1977, in Evolution of Galaxies and Stellar Populations, ed. B. M. Tinsley & D. C. Larson, Richard B. Gehret, 401
  * Ventou et al. (2017) Ventou, E., Contini, T., Bouché, N., et al. 2017, A&A, 608, A9, doi: 10.1051/0004-6361/201731586
  * Vika et al. (2013) Vika, M., Bamford, S. P., Häußler, B., et al. 2013, MNRAS, 435, 623, doi: 10.1093/mnras/stt1320
  * Walmsley et al. (2022) Walmsley, M., Lintott, C., Géron, T., et al. 2022, MNRAS, 509, 3966, doi: 10.1093/mnras/stab2093
  * Yung et al. (2022) Yung, L. Y. A., Somerville, R. S., Ferguson, H. C., et al. 2022, MNRAS, 515, 5416, doi: 10.1093/mnras/stac2139
  * Zolotov et al. (2015) Zolotov, A., Dekel, A., Mandelker, N., et al. 2015, MNRAS, 450, 2327, doi: 10.1093/mnras/stv740