# Transforming Observations of Ocean Temperature
with a Deep Convolutional Residual Regressive Neural Network

Albert Larson
<EMAIL_ADDRESS>
Ali Shafqat Akanda
<EMAIL_ADDRESS>
0000-0003-3642-9565 0000-0003-2295-6794

###### Abstract

Sea surface temperature (SST) is an essential climate variable that can be
measured via ground truth, remote sensing, or hybrid “model” methodologies.
Here, we celebrate SST surveillance progress via the application of a few
relevant technological advances from the late 20th and early 21st century. We
further develop our existing water cycle observation framework, Flux to Flow
(F2F), to fuse AMSR-E and MODIS into a higher resolution product with the goal
of capturing gradients and filling cloud gaps that are otherwise unavailable.
Our neural network architecture is constrained to a deep convolutional
residual regressive neural network. We utilize three snapshots of twelve
monthly SST measurements in 2010 as measured by the passive microwave
radiometer AMSR-E, the visible and infrared monitoring MODIS instrument, and
the in situ Argo dataset ISAS. The performance of the platform and success of
this approach is evaluated using the root mean squared error (RMSE) metric. We
determine that the 1:1 configuration of input and output data and a large
observation region is too challenging for the single compute node and dcrrnn
structure as is. When constrained to a single 100 x 100 pixel region and a
small training dataset, the algorithm improves from the baseline experiment
covering a much larger geography. For next discrete steps, we envision the
consideration of a large input range with a very small output range.
Furthermore, we see the need to integrate land and sea variables before
performing computer vision tasks like those within. Finally, we see
parallelization as necessary to overcome the compute obstacles we encountered.

##  1 Introduction

Water is both an essential and abundant resource on earth; its availability
and quality are critical for sustaining life and ecosystems. Though it is
abundant, the majority of earth’s water, about 97%, is found in the ocean,
while the remaining 3% is freshwater found in glaciers, lakes, rivers, and
underground aquifers on land. Water is not only crucial for sustaining life,
but also plays a vital role in shaping the planet’s climate and weather
patterns. One significant but understudied climate variable that hydrologists
must consider is sea surface temperature. SST has a profound impact on the
water cycle, specifically evaporation [1]. Over-ocean-anomalies like
atmospheric rivers can lead to both anomalous and enormous quantities of
meteorological water falling on land [2]. The same is true in reverse, as
failure of the rains in India are influenced not only by orographic factors
from the towering Himalayan mountain range but also determinants of the Bay of
Bengal and eponymous nearby ocean [3]. Understanding the relationship between
sea surface temperature and the water cycle is essential for predicting and
adapting to extreme events, managing water resources, and sustainable the
global ecosystem [4].

Evidence shows that human beings through industrialization have modified and
are continuing to significantly modify the climate. However, the modern cause
for concern is the rate at which our climate has changed rather than the
Boolean of has it or has it not. Measurements of carbon dioxide (CO2) tell the
story: detected values of atmospheric CO2 have increased by 50% of the
starting value at the advent of industrialization [5, 6]. Invariant to
latitude and longitude, the impacts are felt everywhere. Earth’s response to
our stimuli manifests in the form of heat waves, stronger storms, longer
periods of drought, greater impulses of meteorological water accumulation over
land, and a general increase in environmental variability. While in wealthy
communities, modern civil infrastructure serves as a boundary layer to
environment-related catastrophes, the poor and powerless are unequally yoked
[7]. One must consider also the importance of the ecology itself. How is the
global health of all creatures of the atmosphere, land and oceans [8, 9, 10,
11]? What does the next five, ten, or one hundred years look like at the
current rate [12, 13]? What changes can be made to mitigate or adapt to
implications of past and present poor actions [14]? What are the global
environmental quality standards [15]? How can changes be promoted in unequal
nation states [16, 17, 18]?

SST is worthy of study for a number of reasons: its status as a key
observational characteristic of water in the environment; the importance of
SST in numerical weather and climate forecasting; SST’s detectability via
satellite-mounted RS instrumentation; and the availability of matched
continuous ground truth temporospatial measurements that can be studied for
intercomparison of dataset bias, variances, and uncertainties. We compare the
raw satellite observations to the lower resolution but more precise in situ
measurements of sea surface temperature (ISAS). We apply a treatment to the
lower resolution but generally more available satellite instrument (AMSR-E),
setting its target output to be the higher resolution MODIS product. Our
hypothesis is that fusing the AMSR-E data to MODIS data will create a product
that is closer in performance to MODIS than its AMSR-E input.

SST is a fascinating variable because not only is it a predictor of future
weather anomalies, but it represents a largely unexplored stage regarding the
capture and storage of CO2. Marine carbon dioxide removal (mCDR) is a crucial
strategy for mitigating climate change by extracting and sequestering carbon
dioxide (CO2) from the atmosphere. Natural processes, such as the biological
pump and carbon storage in marine ecosystems, contribute significantly to CO2
removal [19, 20]. Engineered approaches, including iron fertilization to
stimulate primary production [21], nutrient optimization for enhanced carbon
export [22], and ocean alkalinity enhancement to increase CO2 absorption
capacity [23], offer potential avenues for mCDR. However, challenges exist,
including environmental risks and unintended ecosystem disruptions [24].

One tool humans have in their arsenal is the ability to artificially model the
environment. Modeling is the combination of a priori measurement data with
empirically derived functions that simulate and represent the behavior of the
environment. Given observed information obtained by sensors or measurement
devices, we can use the collective knowledge to make better decisions. The
types of observation data we collect about the state of the climate now in a
given place, (e.g. tree ring cores, core samples) or via man-made data
collection devices (satellites, weather balloons, airplanes, drifters, buoys,
gauges), and have better insights to what tomorrow or the same time in five
years might bode given current inputs, trajectories, and these empirical
functions [25].

The use of neural networks as a tool for modeling has grown in frequency over
the last two decades alongside the increase in availability and speed of fast
mathematical computing hardware like graphics processing units. Nvidia has
just crossed the trillion dollar market capitalization in large part to the AI
boom driven by the proliferation of generative models like GPT and Stable
Diffusion [26]. Here, we build on an existing neural network architecture
called dcrrnn, which stands for a deep convolutional residual regressive
neural network and is pronounced "discern" [27, 28]. When applied to problems
involving the water cycle, dcrrnn is under the umbrella of Flux to Flow (F2F).
This is an expectorant measure developed during the creation process of the
first experiment. It is hypothesized that one might prefer to extract one or
the other depending on the subject environment. Specifically, the extraction
of just the dcrrnn structure for use in another vertical. Likewise,
development of F2F with a different but related structure and a focus
specifically on water-focused variables are anticipated to be better
categorized as Flux to Flow projects. Dcrrnn is narrow, whereas F2F is wide.

The premise and motivation of dcrrnn and F2F is to study measurements of
water, be them remotely sensed, gauged measurements, or hybrid datasets. From
ingestion, some preprocessing of the data occurs to prepare for training
either via statistical conformation or temporospatial constraint. The data is
used to train a neural network. Inferences on unseen data are performed to
evaluate the trained algorithm according to a relevant figure of merit for the
experiment. Our chosen metric in set of experiments is the dimensionless root
mean squared error (RMSE). This metric is a solid introductory measurement
because of its frequent use and acceptance within the scientific community.
Our stochastic framework was successful in mimicking the performance of
deterministic algorithms used to predict gauged river streamflow from
meteorological forcings. In light of this success and our interest in all
facets of the water cycle, we elected to next examine the ability of dcrrnn
and F2F to mimic the abilities of image enhancement and ocean modeling
software such as the Scatterometer Image Reconstruction (SIR) algorithm, the
regional ocean modeling system (ROMS) algorithm, and the Massachusetts
Institute of Technology’s General Circulation Model (MITgcm) [29, 30, 31, 32].
These algorithms take in a variety of different spatial datasets and perform
some deterministic process to provide a value-added output. We aimed to
evaluate the research questions: "Can dcrrnn, the supervised learning
structure, be used as a surrogate to other commonly used image optimization
algorithms and ocean modeling software? When applied to SST fields, does
dcrrnn improve an input dataset based solely on statistical optimization
associated with neural networks and the high resolution training data?"

From the outset, one important facet of dcrrnn and F2F to retain is the
integral relationship between the paper and the code used to perform the
experiments. A primary motivation behind the work writ large is to disseminate
the programming element so others will want to join. Frequently, frustrations
and disagreements arise about scientific discourse because the experiment
process is not documented, or there is no repeatable proof. We as much as
possible take a white box approach. Scripts for these experiments are made
available as jupyter notebooks at https://github.com/albertlarson/f2f_sst. As
a primer, we recommend the genesis dcrrnn paper [27]. Though the likely
audience is one with or approaching a graduate degree and a penchant for
education, our intention is that the language is such that it might be
introduced in an undergraduate or advanced K-12 classroom. Coincidingly,
pointing out pitfalls is a valuable exercise to help prevent others from
repeating the same mistakes. Here, we capture what occurs when a naive,
stochastic system is used on a single compute node to attempt a performance of
the classic data fusion problem via neural network methods with SST fields as
the subject matter. We determine that our approach has definite merit, but
requires further investigation with current datasets and other configurations.
A prerequisite for advancement of this effort should either be a larger amount
of compute time via single node, or via multiple compute nodes conducted in
parallel.

With this article, our contributions to the field are the following: 1, the
continued development of F2F as an open source water cycle measurement
framework; 2, to further consideration of dcrrnn as a viable neural network
architecture; 3, the active constraint of the work to a single compute node
and focus on the integration of the manuscript with the code behind the
experiments; 4, the consideration and focus on the importance of sea surface
temperature from a hydrologist’s point of view; 5, a fresh consideration of
neural network based data fusion and data engineering techniques; 6, an
illustration of the limitations of underparameterization; 7, a demonstration
of how the neural network improves its performance when the requirements of
the process are relaxed; and last but not least, 8, a biased focus on global
water resources.

##  2 Materials and Methods

###  2.1 Sea Surface Temperature (SST)

The origin of SST as a continuously monitored variable began when Benjamin
Franklin captured measurements of the ocean as he traversed the Atlantic,
acquiring data and synthesizing these observations into information about the
Gulf Stream [33]. Since then, the field of physical oceanographic research has
made great strides in a variety of advancements relevant here such as
observational techniques and data analysis methods. In recent decades,
satellite remote sensing has emerged as a crucial tool for observing the ocean
on a global scale, and for acquiring SST data. Satellites equipped with
infrared sensors provide accurate and high-resolution measurements of SST [34,
35]. These satellite-based measurements offer advantages over traditional in
situ measurements, as they provide comprehensive coverage of the oceans,
including remote and inaccessible regions. In situ devices are point source
measurements and have a relatively limited observation window compared to the
hundreds of kilometers sun-synchronous satellites capture at a single moment.

In addition to infrared sensors, satellite sensors that detect microwave
radiation, starting with the Nimbus-5 in 1972, have also been used to retrieve
SST [36]. Microwave-based SST retrieval methods have the benefit of low
sensitivity to atmospheric conditions, such as cloud cover and aerosols,
compared to infrared measurements. The availability of long-term satellite-
based SST datasets has led to significant advancements in climate research.
Scientists have utilized these fields to study various climate phenomena,
including oceanic oscillations such as the Pacific Decadal Oscillation (PDO)
and the Atlantic Multidecadal Oscillation (AMO) [37, 38]. Large-scale
oscillations influence the long-term variability of SST and have implications
for regional climate patterns.

Furthermore, satellite-based SST observations have been essential in
understanding the impacts of climate change on the oceans. Studies have shown
that global warming has led to widespread changes in SST, including increases
in average temperatures, alterations in temperature gradients, and shifts in
the distribution of warm and cold regions [39]. These changes have significant
implications for marine ecosystems, including shifts in species distributions,
changes in phenology, and coral reef bleaching events [40, 41]. Satellite-
derived SST data also contribute to the prediction and forecasting of weather
and climate events. The accurate representation of SST conditions is crucial
for weather prediction models, as it affects the development and intensity of
atmospheric phenomena such as hurricanes and tropical cyclones [42, 43]. The
integration of satellite-based SST data into numerical weather prediction
models has improved forecast accuracy, particularly in regions where in situ
observations are sparse or nonexistent.

In addition to weather forecasting, satellite-based SST data has practical
applications in fisheries management and marine resource monitoring. SST
information helps identify optimal fishing grounds by indicating areas with
suitable temperature conditions for target species [44, 45]. Furthermore,
monitoring changes in SST can provide insights into the health of marine
ecosystems and aid in the assessment and management of protected areas and
biodiversity hot spots [46, 47, 48].

###  2.2 Aqua

The Aqua satellite was launched on May 4, 2002. Upon it, two instruments sit:
AMSR-E and MODIS. Both, among other things, are designed to study the
temperature of the ocean. The measurements obtained as the satellite is moving
from South Pole towards North always crosses the equator at approximately 1:30
PM local time nadir (directly below the satellite). In the downward portion of
the orbit, the satellite crosses the equator at 1:30 AM local time nadir [49].
The AMSR-E instrument ceased functioning after ten years of service. MODIS
continues to operate, logging over twenty years of active surveillance. AMSR-E
has been succeeded by a follow-on instrument AMSR2. AMSR2 is aboard a Japanese
mission called GCOM-W, one of a series of global climate observation missions
[50]. AMSR2 has a slightly larger antenna than AMSR-E, but is similar in scope
to the AMSR-E instrument. The de facto replacement of MODIS is VIIRS, an
instrument series carried on the Suomi National Polar-orbiting Partnership
(SNPP), NOAA-20, and NOAA-21 satellites [51, 52]. Other global aerospace
mission datasets associated with SST are available, like the Chinese Haiyang
and Gaofen series [53, 54].

###  2.3 AMSR-E

AMSR-E was a passive microwave radiometer [55, 56]. The acronym stands for
Advanced Microwave Scanning Radiometer for Earth Observing System. There are
several products produced on top of the raw radiance data that were collected
by this instrument, and the AMSR-E data was processed by different ground
stations depending on the parameter of interest. The produced datasets contain
latitude, longitude, several physical parameters (e.g., SST, Albedo, soil
moisture) as well as other pertinent metadata. As it pertains to sea surface
temperature, AMSR-E is available in Level 3 products and as part of Level 4
assimilation system output. As of the writing of this document, Level 2 SST is
no longer publicly available. However, we were able to obtain Level 2 fields
from the Physical Oceanography Distributed Active Archive Center (PODAAC)
through the Jet Propulsion Laboratory before the sunsetting of the data
product.

To detail a sample, one single Level 2 netCDF (.nc) file containing AMSR-E
data was procured. The record selected is that of March 3rd, 2004, with a UTC
time of 01:07:01. The file contains three coordinates (latitude, longitude,
and time) and thirteen data variables. Each variable is a single matrix
comprised of columns and rows of measurements. The important distinction here
is that the data structure is stored to reflect the path of the orbit. See
Figure 1. When the sea surface temperature is plotted as it sits in the
matrix, it is difficult to discern what is transpiring. There appears to be
some curvature of the measurements, but other than that little is known to an
untrained eye beyond the title and colormap.

Figure 1: L2 AMSR-E SST field, March 3, 2004, no coordinate system Figure 2:
L2 AMSR-E SST, plotted with available coordinates and world map

Inclusion of the latitude and longitude coordinates, as well as a global
basemap generates a clearer picture as seen in Figure 2. A single Level 2
AMSR-E SST file contains matrices representing one full orbit around the
globe. Each file holds partially daytime and partially nighttime observations.
Because of diurnal warming, it is desirable to separate the nighttime and
daytime passes. Furthermore, many analyses are comprised of an ensemble of
satellite observations from different platforms such as this one. A grid makes
for more orderly computations at large spatial scales. Certainly, one could
elect to grid every observation to the AMSR-E or MODIS native product
coordinate system. With our experiments, we choose the path of rectangular
gridding. We consider the Level 3 product because of the interest in spatial
relationship across large geographic scales and variable time (daily, weekly,
seasonally, yearly, generationally).

AMSR-E is available (accessed 7 June 2023) via its producer, Remote Sensing
Systems of Santa Rosa, CA [57]. This daily product comes in 25 km resolution
and is delineated by daytime and nighttime passes of the satellite. The time
series runs from June of 2002 until October of 2011 when the AMSR-E instrument
ceased functioning. Figure 3 illustrates the point that even without
explicitly defining the coordinate system in the visualization, the matrix of
SST values is already placed in proper spatial order. Figure 4 reinforces the
fact that little change occurs with the inclusion of latitude and longitude
coordinates when plotted on a rectangular grid. Here, we simply mean each
month’s worth of daily daytime and nighttime passes on a pixel-wise basis. We
call these day and night in the experiments that follow. We also create a
hybrid dataset, where we average the monthly averages of day and night passes
together. Finally, we transform all three of these datasets from the native
AMSR-E grid system to the slightly different MODIS grid; this function is
carried out using the xESMF software [58].

Figure 3: L3 AMSR-E file plotted without supplied coordinate system Figure 4:
L3 AMSR-E file plotted with available coordinates and world map

###  2.4 MODIS

MODIS, or Moderate Resolution Imaging Spectroradiometer, measures thirty-six
different radiance bands in the infrared and visible ranges of the
electromagnetic spectrum [59]. Level 3 sea surface skin temperature as
obtained from MODIS comes in 4 kilometer and 9 km products, and is derived
from a subset of the thirty-six radiance bands. The products are available in
daily, average of eight days, and monthly products. They are also delineated
by daytime and nighttime passes of the Aqua’s polar-orbiting nature. SST
products deriving from MODIS are further specified by the length of the waves
within the thermal infrared range used to derive the measurement: longer waves
(11–12 microns) and middling waves (3–4 microns). The MODIS documentation
state that the 3–4 micron wave SST product is less uncertain, but only usable
at night because of the daytime sun glint impact on 3–4 micron waves. We use
the long wave 11–12 micron infrared measurements to keep constant the source
of both daytime and nighttime passes.

Figure 5: L3 daily MODIS file containing only high quality flagged pixels

The MODIS Aqua Level 3 SST Thermal IR Monthly 4km V2019.0 product [60, 61]
comes with latitude and longitude coordinates, SST values and per pixel
quality measurements denoting when contamination is likely. The grid is
equidistant rectangular, a match with the AMSR-E grid but at a finer original
resolution. Of the over thirty million pixels for an entire day of 4 km MODIS
pixels, 90% of them in the random sample selected here are deemed contaminated
and filtered out Figure 5. This contrasts with the 50% loss of AMSR-E pixels.
This great loss in pixels due to quality is attributed to cloud contamination.
To compensate, we use the monthly product Figure 6 where a greater amount of
time has transpired, allowing for a higher probability of clean global
coverage. A randomly sampled MODIS monthly image yields 50% loss, in line with
the AMSR-E daily product and much improved upon relative to the daily MODIS
observation files.

Figure 6: Monthly L3 MODIS image containing only high quality observations

###  2.5 Ground Truth Measurements

For a source of ground truth data, we selected the “In Situ Analysis System”
(ISAS) dataset obtained from the University of California’s Argo repository
and produced by a consortium of French institutions [62]. An important
constraint for this work was to obtain only the surface level measurement of
temperature at the highest frequency available during the years of both AMSR-E
and MODIS. These products are provided in a gridded format are used to observe
temperature measurements at many depth levels. In the publication attached to
the ISAS dataset [62], the target physical quantity is steric height and ocean
heat content; with these as their target output, gridded depth-dependent
temperature is stored as a byproduct. The 0.5 degree monthly dataset is
presented in a Mercator projection, slightly different than the AMSR-E and
MODIS grids. Mercator lines of longitude have a uniform distance in between
them; the distance between latitudes from the equator changes. Identical to
AMSR-E, we re-grid this data to the MODIS grid and coordinate system.

###  2.6 Treatment

This work is an extension of [27]. The F2F code base provides an step-by-step
approach to the fundamentals of the materials and methods applied within. As
such, it is a key of part of the work and has been made openly accessible at
https://github.com/albertlarson/f2f (accessed on 6 June 2023). The scripts
follow the logical order of that paper. Concretely, this work builds upon that
foundation. The notebooks found at https://github.com/albertlarson/f2f_sst
follow the simple process of extracting the data, transforming the data, feed
the data into a transformation system (dcrrnn), and then evaluating the
performance of the system.

The treatments we apply to the data are several configurations of one common
concept: neural networks. Neural networks are not new, but the growth of
graphical processing units (hardware) has enabled them to flourish in
software. Neural networks are a type of learned representation. A structure is
fed connected input and target pairs. Based on the predictive quality of the
initial network structure, an error between the neural network output and the
target occurs. This error is in turn fed to an optimization algorithm that
iteratively and slightly alters each “neuron” of the initial network structure
until it reaches a designated optimal state. Via many small calculations and
the simultaneous application of statistical mechanics, neural networks are
known to provide qualities like that of a brain, such as capturing spatial
eccentricities and temporal changes in sets of related images. Neural networks
are applied to a range of tasks from the more mundane such as learning a
quadratic equation, to the more cutting edge, like extreme event forecasting
or cancer detection.

Transfer learning has become commonplace in the field of machine learning
[63]. Transfer learning places an emphasis on creating reusable treatment
structures for others to build on top of without inadvertently causing the
audience to get lost in possibly unimportant details. We employ transfer
learning to create a complex configuration with a relatively short learning
curve. The neural network is characteristically deep, convolutional, residual,
and regressive. Our construct is inspired by the work of residual networks
[64]. However, our problem is one of a regressive nature. Sea surface
temperature has a continuous temperature range that it exists within. This is
a notable difference to some of the more common introductory neural network
examples, such as those associated with the MNIST and CIFAR datasets where the
number of possible outputs is very small [65, 66]. Loss functions associated
with regressive problems are constrained to just a couple: mean absolute error
(MAE) and mean squared error (MSE). The calculation of the loss function must
be differentiable. This is due to the optimization component of neural
networks. The literature is rich with publications regarding neural network
optimizers, as well as the general mechanics of neural nets [67, 68].

Once neural network architecture and hyperparameters are chosen, training and
validation data is loaded into the network. While training the neural network,
close observation is made of the reduction in error between training input and
output as the neural network begins to optimize or learn. We also monitor the
validation dataset at each training iteration. The learning process stops once
the training and validation data has been passed through the network a certain
number of times, or epochs. When prototyping or pilot-testing the experiment
set to be carried out, one should test with a very short number of epochs and
a larger sum of epochs to see where good performance meets fast time of
computation.

After training, the optimized neural network structure is intentionally
frozen. Before the point of freezing, the neurons of the network can be
adjusted for optimization, like a student asking a teacher for advice when
studying. The frozen state and inference imposed upon it is like a student
being prompted with a pop quiz and no teacher assistance. This test or input
data are similar enough to the training that the teacher believes the student
will have success in passing the test according to the selected merit (mean
squared error, the loss function). After the test, the performance of the
model is evaluated and a decision is made regarding next logical steps in the
research.

A neural network can become biased to its training inputs. It starts to
memorize the training dataset, which does not make for a generally applicable
algorithm. Avoidance of biasing comes at the cost of variance [69]. Applying
dropout is one technique to systematically prevent system bias by simply
“turning off” a certain percentage of random neurons at each iteration of the
algorithm [70, 71]. Another approach is the application of early stopping. The
loss function of a neural network typically looks like a very steep curve down
to a flat bottom. Rather than allow the network to persist in the flat bottom
for long and become overfit, simple logic can be employed to stop training
early when the network shows evidence that it has reached an optimal state.
Percentage of data split between training and testing proportions is another
relevant training hyperparameter. A larger proportion of the dataset being
part of the training portion could lead to overfitting of the model and lack
of generalized predictability. On the other hand, insufficient training data
might lead to an inability to adequately characterize the reality of the data
pairs.

Figure 7: Sample training monthly observation; January 2010 MODIS day
observation of the Hawaiian Islands; segmented into 100 x 100 pixel regions.

The image sets subjected to treatment are on the large side computationally.
Holding many one million or nine million pixel images within the memory of a
single graphical processing unit becomes intolerable to the device. One could
elect to use multiple GPUs or a compute node with a great provision of memory.
Here, we constrain the experiment to a single GPU and cut the images up into
smaller pieces of square data. Our patch size is fixed at 100 x 100, though
this is a tunable hyperparameter. Figure 7 shows a Pacific Ocean study region,
highlighting Hawaii and regions east. While this image is too large to process
directly in the neural network, we can solve this problem by creating the
eighteen patches of 100 x 100 pixels, representing the 300 x 600 pixel region
under observation.

Neural networks do not function when nan values are present in any of the
images. We enacted a broad treatment to the AMSR-E and MODIS images, computing
the mean of the entire image, excluding the nan values. Then, where the nan
values are present, we replace them with the mean value. This has the
convenient byproduct of introducing into the neural network many training
pairs where the input and output are simply comprised of the average global
SST value as obtained via the AMSR-E and MODIS instrument.

##  3 Results

We randomly selected a single calendar year from the available time series
where AMSR-E, MODIS, and ISAS overlap. Of this year’s worth of data, we
settled on nine different cases consisting of three different locations
(Atlantic Ocean, Pacific Ocean, and Indian Ocean) and three different
observation windows (day, night, mean averaged of day and night). We train the
neural network on the first ten months of the year, validate with the eleventh
month, and test with the twelfth month. However, we did not intend for this
system to be deterministic or biased in nature. Therefore, we shuffle the
training pairs to confuse the network and promote regularization [72]. A
training session runs for 100 epochs. Each image in the geographically
constrained time series is 300 pixels x 600 pixels in size, divided up into
eighteen 100 x 100 pixel segments to incrementally feed the neural net.

Figure 8: December 2010. The top panel shows the RMSE matrix for all datasets
(input and outputs) for each of the nine cases. Two bottom panels are MSE
plots of training and validation losses during neural network training.
Atlantic, Pacific, and Indian Oceans segmented by monthly Day, Night, and
Hybrid

Results of the nine cases are illustrated in Figure 8. The upper panel is a
test matrix. Each of the ten rows in the test matrix represent a tuple of
datasets that are compared. All of the different data products we use are
compared here to the ISAS data and the MODIS data as benchmarks. Columnwise,
the test matrix is delineated into each of the nine cases (experiments) that
we perform. For example, PD stands for Pacific Day, AN means Atlantic Night,
and IH Indian hybrid. The hybrid is simply a mean of the day and night images
for a given month. Take the PD column. We compute the RMSE between the
complete 300 x 600 Pacific scene for each of the comparison tuples. In row 1,
column 1, the RMSE value of the (Argo, Argo) comparison for the Pacific Day
experiment comes out to zero. This is expected, because there should be no
error between two identical datasets. Root mean squared error is not computed
in locations where land is present.

The bottom two panels are graphs of the in-the-loop performance of the dcrrnn
algorithm as the learning process occurs. The loss function, sometimes
otherwise referred to as the cost function, for dcrrnn is mean squared error
(MSE). This calculation is related to the RMSE metric used in the upper panel.
Though the computations are different, the results are not perfectly related.
The cause of this discrepancy is a function of a constraint for the training
process. If an input or target image has any non-numerical pixels present, we
have to replace them in some way. The cause of nans are largely attributed in
this case to a location that is partially or completely land, or is a pixel
contaminated by clouds or rainfall. We retain a global land mask for use
before and after the training process to enable the removal of the
temporarily-filled land pixels, which gives way to the clean RMSE test matrix.
Our fill mechanism for each of the training and validation squares is to use
the local mean of that square as the fill value. As such, there are always the
same number of pixels factored into each 100 x 100 square during the neural
network training process and in particular the calculation of in-the-loop
training loss.

Pred in the test matrix refers to the neural network’s prediction of the
unseen test data, the single month of data. It is fed into the trained network
as eighteen 100 x 100 squares, but then recombined into one 180,000 pixel
array. Land locations are added back on top. No other compute mechanism is
employed. The Optim dataset has an extremely faint bandpass filter on top of
the Pred dataset. Simply put, if there are any pixels in the Pred result from
the neural network that is outside of the three standard deviations from the
mean of the image, they are converted to nans and deemed erroneous. In this
experiment, this filter has little effect. One would see more drastic effects
should the trigger for filtering data from Pred to Optim change from three
standard deviations to two, one, or less standard deviations from the mean.
The risk of using a bandpass filter is that much of the interesting nuanced
information can be filtered away. This feature was implemented during the
experiment phase in response to the analyst’s acknowledgement that artefact in
the experiments were occurring along the coast.

Figure 9: Relatively “good” perceptual change, Pacific Night case, axes are
pixels Figure 10: Relatively “poor” perceptual change, Indian Day case, axes
are pixels

##  4 Discussion

With every case summarized in Figure 8, the RMSE between Optim and MODIS is
higher than the AMSR-E input. In occasional instances, the test case of
December does make an Optim output that is closer to Argo than the input
AMSR-E. In some cases, though, it makes a worse performing product with
regards to Argo than either AMSR-E or MODIS. See Figures 9 and 10 for samples
of how the RMSE translates to actual transformation of the images. In the
second row of the first column, one sees all green. This solid green color
demarcates a clear relationship between the two compared datasets. In this
case, that is because this square is the difference between Argo and itself.
Considering all images in row 2 of Figure 9, the second column where AMSR-E is
compared to Argo seems to be the most green, denoting a closeness. Pred and
Optim are both covered in green; however, the bandpass filter in Optim is not
being utilized. Therefore, we see the coastal artefact around the Hawaiian
islands present as a result of dealing with nan values due to land. We are
hesitant to use a liberal bandpass filter, because there’s a real risk of
deleting good dynamic data. This fact has lead us to an open question on the
most appropriate way to handle water cycle data. It is evident that a gridded
ocean product where land has no numerical representation isn’t acceptable, and
solutions like replacing values with local means isn’t a true solution but a
workaround. The most obvious answer is to create a unified global surface
temperature dataset. However, this is less than best for the hydrology
community. An adequate replacement specification for land surface temperature
might be soil moisture or calculated surface and subsurface flows.

Figure 10 represents a dcrrnn experiment conducted over the Arabian Sea and
Bay of Bengal; it also illustrates a less desirable outcome than that of
Figure 9. The telltale signal of underparameterization is evident in columns
four and five of Figure 10, most notably in rows 2 and 3. As mentioned
earlier, because of compute constraints, we can not load the entire image into
the neural network at one time. We have to break it up into squares. Here
there are clear horizontal and vertical boundary lines. Furthermore, there is
the appearance of much colder estimates of SST, driven by artefact from the
coastal regions. It appears that edges were systematically not able to be
resolved based on the configuration of the dcrrnn structure and quantity of
data fed into the system.

We determined that just looking at the nine cases in one constant way was not
considerate enough of potential unseen indirect effects or confounding
variables [73, 74]. As an additional measure, we dug deeper into the night
only observations of the Atlantic ocean. We selected a single 100 x 100 pixel
that is completely over ocean within the Atlantic region. Our selection is
seen in Figure 11. Figure 12 illustrates that all pixels in the selection are
real numbers (left). The presence of the color red would denote nan pixels
present. This point is reinforced on the right panel of Figure 12 with the
single vertical line histogram denoting all real pixels.

Figure 11: Atlantic Night, single patch selection overlain on top of the the
land mask (left) and SST field (right) Figure 12: Atlantic Night, single 100 x
100 pixel patch real/nan Boolean (left) and histogram of real/nan (right)
Figure 13: Atlantic Night, AMSR-E (col 1), Pred (col 2), MODIS (col 3),
histogram of differences (col 4)

We extend the number of epochs from 100 as compiled in Figure 8 to 500 in this
more-zoomed-in observation. A larger amount of training epochs, a smaller
dataset size, and no nans present are strategic measures away from the
baseline experiments to improve the outcome. Results of this experiment are
illustrated in Figure 13. With this configuration, we find that the network
reduces the RMSE difference between AMSR-E and MODIS by 20%, bringing the RMSE
between AMSR-E and MODIS from 53.4 down to 44.0 between Pred and MODIS.
However, there is no major perceptual change between the AMSR-E image and the
Pred image. It is a promising result that there is an improvement in the
results here relative to the nine larger cases. This performance indicates
that rectification of the performance issues we are seeing can be further
alleviated by shrinking the target size relative to the input, or by
increasing the input size. As the output size shrinks, the problem converges
with that of the earlier dcrrnn work [27].

Wherever land and coastal regions are present, the neural network alone
struggles. This is due to the nature of the land sea boundary layer in all
these datasets. At the presence of land, the raw data (as they are downloaded
as .nc files) are given a non-number (nan) designation. For the purpose of
training a neural network using the convolutional flavor, images with zero
nans are needed. Otherwise the software fails catastrophically. As referenced
earlier, steps were taken during the training process to circumvent the
presence of land by substituting those pixels temporarily with the local mean
value. Another option is the application of the substitution of the nan values
with the mean as computed by the entire “scene” or day. There is a risk that
the substitution of these values introduces a source of structured noise. This
noise might be one factor leading to the higher than zero RMSE scores denoted
in Figure 8. Furthermore, it is possible that this structured noise is
hindering training of the neural network process itself. This is unfortunate,
because coastal regions are the stage for a variety of interesting SST events
such as boundary currents. At the same time, hydrologists and global health
professionals have a growing stake in the influence of the ocean. Sea level
rise and saltwater intrusion are two hallmark conditions present at the
coastal interface [75]. Beyond sea surface temperature, ocean color and sea
surface height are variables intimately linked to the coastal environment [76,
77, 78]. We hypothesize that global datasets considering these parameters have
the same challenges when conformation to a gridded structure and neural
network process are prescribed. We believe that some harmony via integration
of land and data sets would be a next discrete step to evaluate the impact on
training and inferring with Flux to Flow. A good candidate is surface and
subsurface flow as producted by the Global Land Data Assimilation System
(GLDAS). GRACE, VIIRS, and SMAP all have land products available as well as
oceanic products. It’s possible that if the data is ocean focused, perhaps
leaving the land observations in their raw state with no physical scaling
would be the best practice.

Our images are single channel inputs, and can be considered grayscale
pictures. They are slightly different though; grayscale pictures are usually
digitized as pixels with numbers between 0 and 1. When displaying SST images,
measurements of physical properties, we use a colormap with minimum and
maximum based on what we know to be the physical limits of the parameter
itself. Nevertheless, it is closely related to a grayscale image. In grayscale
computer vision tasks such as this set of experiments, the use of mean squared
error as a loss function has audience of skeptics [79]. There are examples
where the loss function optimized in a neural network drops in value
significantly but sees no improvement in the quality of the image. Alternate
loss functions to the standards built into PyTorch are available [80]. Without
alteration, these functions require the inputs to be either between 0 and 1
(grayscale) or 0 and 255 (color images). Another avenue is the pursuit of
physics-based loss functions [81]. Some data engineering is definitely needed
in future iterations. As we have experience with the surface and subsurface
flow rasters and these are land viewing, we think the combination of SST and
GLDAS as predictors of streamflow in coastal regions is a suitable discrete
next step. This next step will force us to decrease the number of target
outputs, as there is a paucity of point measurements in streamflow monitoring
relative to that of the interpolated target ISAS data used herein.

Only a fraction of the available data was observed in this study. The ISAS
Argo dataset comes as a single compressed file; we extracted the surface layer
only. There is great value in consideration of SST depth layers; however, it
was outside the scope of our investigation. Furthermore, we studied monthly
time series images of all three raw datasets. AMSR-E and MODIS each have near
complete global pictures within two to three days. These datasets are then
transformed in different ways and can lose fidelity by various types of
decimation such as re-gridding from swaths to squares, uncertainty in formulas
used for conversion from base input to high level physical parameter, or by
forms of compression.

We took a naive approach to the problem. It is common to initialize a model
with a historical known bias, and make slight inferences based upon the long-
term mean. This tightly constrains the problem to the known past environment.
We did not do that. We actively attempt to root out and prevent any sort of
deterministic bias, and see under a tough conditioning what the dcrrnn
algorithm does. This leads to some less than desirable results; however, our
zoom in shows that when the complexity of the system is relaxed, the algorithm
improves according to the RMSE target. In a future study, it would be helpful
to start the training with a long-term bias of SST for an entire year based
on, for example, the Operational Sea Surface Temperature and Ice Analysis time
series dataset produced by the Group for High Resolution Sea Surface
Temperature [82].

##  5 Conclusions

Sea surface temperature (SST) is an essential climate variable. A better
understanding of SST equates to a better understanding of global hydrology and
the interactions of water as it moves around the hydrosphere in liquid, solid,
and vapor forms. The advent of satellite SST observation has allowed for the
study of large scale phenomena otherwise invisible. The beginning of the 21st
century marked a new frontier in the measurement of SST via the Aqua mission
and Argo program. Recently, neural networks have changed the way that
scientists consider modeling of the environment. In this study, we continued
to develop Flux to Flow, an extract, transform, load, treat, and evaluation
framework based around a deep convolutional residual regressive neural network
(dcrrnn). We extended its additional functionality of streamflow prediction to
transform one Aqua instrument dataset into another: AMSR-E observations
towards MODIS observations.

We focused on three large oceanic regions: Indian, Pacific, and Atlantic. With
each of the three locations comprised of eighteen 100 x 100 pixel image pairs
per month, and ten months of training data, the neural network struggles to
transform AMSR-E into MODIS. When we relax the amount of data fed into dcrrnn,
looking at a single 100 x 100 pixel image pair per month and ten months of
training data, the network is statistically better able to transform AMSR-E
into MODIS data. Provided these results, we believe that a next discrete step
is to focus on coastal areas where the hydrology and oceanography are closely
linked. We would like to investigate the performance of dcrrnn in predicting
the streamflow of a river when it does and does not consider oceanic behavior
in the adjacent ocean. We hypothesize that merging ocean and land datasets
will not only ease the challenges associated with handling non-numbers, but
that the streamflow prediction will benefit from the unique signatures present
in the ocean data alongside the land measurements of the water cycle.

## References

  * [1] Taikan Oki and Shinjiro Kanae.  Global hydrological cycles and world water resources.  science, 313(5790):1068–1072, 2006.
  * [2] Helen F Dacre, Peter A Clark, Oscar Martinez-Alvarado, Marc A Stringer, and David A Lavers.  How do atmospheric rivers form?  Bulletin of the American Meteorological Society, 96(8):1243–1255, 2015.
  * [3] NH Saji and TJCR Yamagata.  Possible impacts of indian ocean dipole mode events on global climate.  Climate Research, 25(2):151–169, 2003.
  * [4] Rainer Hollmann, Chris J Merchant, Roger Saunders, Catherine Downy, Michael Buchwitz, Anny Cazenave, Emilio Chuvieco, Pierre Defourny, Gerrit de Leeuw, René Forsberg, et al.  The esa climate change initiative: Satellite data records for essential climate variables.  Bulletin of the American Meteorological Society, 94(10):1541–1552, 2013.
  * [5] A Eldering, PO Wennberg, D Crisp, DS Schimel, MR Gunson, A Chatterjee, J Liu, FM Schwandner, Y Sun, CW O’dell, et al.  The orbiting carbon observatory-2 early science investigations of regional carbon dioxide fluxes.  Science, 358(6360):eaam5745, 2017.
  * [6] Josep G Canadell, Corinne Le Quéré, Michael R Raupach, Christopher B Field, Erik T Buitenhuis, Philippe Ciais, Thomas J Conway, Nathan P Gillett, RA Houghton, and Gregg Marland.  Contributions to accelerating atmospheric co2 growth from economic activity, carbon intensity, and efficiency of natural sinks.  Proceedings of the national academy of sciences, 104(47):18866–18870, 2007.
  * [7] Mesfin M Mekonnen and Arjen Y Hoekstra.  Four billion people facing severe water scarcity.  Science advances, 2(2):e1500323, 2016.
  * [8] Angela H Arthington, Anik Bhaduri, Stuart E Bunn, Sue E Jackson, Rebecca E Tharme, Dave Tickner, Bill Young, Mike Acreman, Natalie Baker, Samantha Capon, et al.  The brisbane declaration and global action agenda on environmental flows (2018).  Frontiers in Environmental Science, 6:45, 2018.
  * [9] Rebecca E Tharme.  A global perspective on environmental flow assessment: emerging trends in the development and application of environmental flow methodologies for rivers.  River research and applications, 19(5-6):397–441, 2003.
  * [10] Jeremy Dickens, Chris Dickens, Nishadi Eriyagama, Hua Xie, and David Tickner.  Towards a global river health assessment framework.  2022\.
  * [11] Abigail J Lynch, Steven J Cooke, Angela H Arthington, Claudio Baigun, Lisa Bossenbroek, Chris Dickens, Ian Harrison, Ismael Kimirei, Simone D Langhans, Karen J Murchie, et al.  People need freshwater biodiversity.  Wiley Interdisciplinary Reviews: Water, page e1633, 2023.
  * [12] Osvaldo E Sala, FIII Stuart Chapin, Juan J Armesto, Eric Berlow, Janine Bloomfield, Rodolfo Dirzo, Elisabeth Huber-Sanwald, Laura F Huenneke, Robert B Jackson, Ann Kinzig, et al.  Global biodiversity scenarios for the year 2100.  science, 287(5459):1770–1774, 2000.
  * [13] Charles J Vörösmarty, Peter B McIntyre, Mark O Gessner, David Dudgeon, Alexander Prusevich, Pamela Green, Stanley Glidden, Stuart E Bunn, Caroline A Sullivan, C Reidy Liermann, et al.  Global threats to human water security and river biodiversity.  nature, 467(7315):555–561, 2010.
  * [14] Brett F Sanders, Jochen E Schubert, Kristen A Goodrich, Douglas Houston, David L Feldman, Victoria Basolo, Adam Luke, Dani Boudreau, Beth Karlin, Wing Cheung, et al.  Collaborative modeling with fine-resolution data enhances flood awareness, minimizes differences in flood perception, and produces actionable flood maps.  Earth’s Future, 8(1):e2019EF001391, 2020.
  * [15] Maki Hatanaka, Carmen Bain, and Lawrence Busch.  Third-party certification in the global agrifood system.  Food policy, 30(3):354–369, 2005.
  * [16] Klaus Keller, Casey Helgeson, and Vivek Srikrishnan.  Climate risk management.  Annual Review of Earth and Planetary Sciences, 49:95–116, 2021\.
  * [17] Farah Nusrat, Musad Haque, Derek Rollend, Gordon Christie, and Ali S Akanda.  A high-resolution earth observations and machine learning-based approach to forecast waterborne disease risk in post-disaster settings.  Climate, 10(4):48, 2022.
  * [18] Mathis L Messager, Lise Comte, Thiago BA Couto, Elliot D Koontz, Lauren M Kuehne, Jane S Rogosch, Rebekah R Stiling, and Julian D Olden.  Course-based undergraduate research to advance environmental education, science, and resource management.  Frontiers in Ecology and the Environment, 20(7):431–440, 2022.
  * [19] S Honjo, TI Eglinton, CD Taylor, KM Ulmer, SM Sievert, A Bracher, CR German, V Edgcomb, R Francois, M Deboraiglesias-Rodriguez, et al.  The role of the biological pump in the global carbon cycle understanding an imperative for ocean science.  Oceanography, 27(3):10–16, 2014.
  * [20] Carlos M Duarte, Inigo J Losada, Iris E Hendriks, Ines Mazarrasa, and Nuria Marba.  The role of coastal plant communities for climate change mitigation and adaptation.  Nature Climate Change, 3(11):961–968, 2013.
  * [21] Philip W Boyd, Timothy Jickells, Cliff S Law, Stéphane Blain, Edward A Boyle, Ken O Buesseler, Kenneth H Coale, Jay T Cullen, Hein J de Baar, Michael Follows, et al.  Mesoscale iron enrichment experiments 1993-2005: synthesis and future directions.  Science, 315(5812):612–617, 2007.
  * [22] Raymond T Pollard, Ian Salter, Richard J Sanders, Mike I Lucas, C Mark Moore, Rachel A Mills, Peter J Statham, John T Allen, Alex R Baker, Dorothee CE Bakker, et al.  Southern ocean deep-water carbon export enhanced by natural iron fertilization.  Nature, 457(7229):577–580, 2009.
  * [23] Greg H Rau and Ken Caldeira.  Enhanced carbonate dissolution:: a means of sequestering waste co2 as ocean bicarbonate.  Energy Conversion and Management, 40(17):1803–1813, 1999.
  * [24] Philippe Ciais, Christopher Sabine, Govind Bala, Laurent Bopp, Victor Brovkin, Josep Canadell, Ankur Chhabra, Ruth DeFries, James Galloway, Martin Heimann, et al.  Carbon and other biogeochemical cycles.  Climate change 2013: the physical science basis. Contribution of Working Group I to the Fifth Assessment Report of the Intergovernmental Panel on Climate Change, 11.
  * [25] Christopher G Piecuch.  River effects on sea-level rise in the río de la plata<? xmltex$\backslash$break?> estuary during the past century.  Ocean Science, 19(1):57–75, 2023.
  * [26] Samrhitha A Akash Sriram.  Nvidia briefly joins $1 trillion valuation club — reuters.com.  https://www.reuters.com/technology/nvidia-sets-eye-1-trillion-market-value-2023-05-30/.  [Accessed 06-Jun-2023].
  * [27] Albert Larson, Abdeltawab Hendawi, Thomas Boving, Soni M Pradhanang, and Ali S Akanda.  Discerning watershed response to hydroclimatic extremes with a deep convolutional residual regressive neural network.  Hydrology, 10(6):116, 2023.
  * [28] Albert Larson.  A clearer view of earth’s water cycle via neural networks and satellite data.  Nature Reviews Earth & Environment, 3(6):361–361, 2022.
  * [29] A. F. Shchepetkin and J. C. McWilliams.  The regional oceanic modeling system (roms): A split-explicit, free-surface, topography-following-coordinate oceanic model.  Ocean Modelling, 9(4):347–404, 2005.
  * [30] A. Adcroft, C. Hill, J. Marshall, and R. J. Stouffer.  A conservative, multi-dimensional advection transport algorithm with application to the simulation of oceanic flows.  Ocean Modelling, 10(1-2):27–47, 2006.
  * [31] David G Long and Mary J Brodzik.  Optimum image formation for spaceborne microwave radiometer products.  IEEE Transactions on Geoscience and remote sensing, 54(5):2763–2779, 2015.
  * [32] David G Long, Perry J Hardin, and Peter T Whiting.  Resolution enhancement of spaceborne scatterometer data.  IEEE Transactions on Geoscience and Remote Sensing, 31(3):700–715, 1993.
  * [33] Peter J Minnett.  A brief history of sea surface temperature measurements.  Bulletin of the American Meteorological Society, 100(5):S3–S6, 2019\.
  * [34] David K Woolf.  Satellite observations of sea surface temperature.  Remote sensing of environment, 114(3):531–547, 2007.
  * [35] Christopher J Merchant.  Sea surface temperature and its relevance to climate change studies.  Remote Sensing, 4(6):1872–1901, 2012.
  * [36] Chelle L Gentemann, Frank J Wentz, Marty Brewer, Kyle Hilburn, and Deborah Smith.  Passive microwave remote sensing of the ocean: An overview.  Oceanography from Space: Revisited, pages 13–33, 2010.
  * [37] Jeff R Knight, Chris K Folland, and Adam A Scaife.  Climate impacts of the atlantic multidecadal oscillation.  Geophysical Research Letters, 33(17), 2006.
  * [38] Yi-Chun Kuo and Ming-An Lee.  Decadal variation of wintertime sea surface temperature in the taiwan strait.  Journal of Marine Science and Technology, 21(7):15, 2013.
  * [39] Sydney Levitus, J Antonov, and T Boyer.  Warming of the world ocean, 1955–2003.  Geophysical research letters, 32(2), 2005.
  * [40] Ove Hoegh-Guldberg, Peter J Mumby, Anthony J Hooten, Robert S Steneck, Paul Greenfield, Edgardo Gomez, C Drew Harvell, Peter F Sale, Alasdair J Edwards, Ken Caldeira, et al.  Coral reefs under rapid climate change and ocean acidification.  Science, 318(5857):1737–1742, 2007.
  * [41] Kristin M Kleisner, Michael J Fogarty, Sally McGee, Jonathan A Hare, Skye Moret, Charles T Perretti, and Vincent S Saba.  Marine species distribution shifts on the us northeast continental shelf under continued ocean warming.  Progress in Oceanography, 153:24–36, 2017.
  * [42] Frank M Monaldo, Todd D Sikora, Steven M Babin, and Raymond E Sterner.  Satellite imagery of sea surface temperature cooling in the wake of hurricane edouard (1996).  Monthly Weather Review, 125(10):2716–2721, 1997.
  * [43] Aneesh A Lotliker, T Srinivasa Kumar, Venkat Shesu Reddem, and Shailesh Nayak.  Cyclone phailin enhanced the productivity following its passage: evidence from satellite data.  Current Science, 106(3):360–361, 2014.
  * [44] Sally E Thorpe and Eugene J Murphy.  Spatial and temporal variability and connectivity of the marine environment of the south sandwich islands, southern ocean.  Deep Sea Research Part II: Topical Studies in Oceanography, 198:105057, 2022.
  * [45] Alistair J Hobday, Haritz Arrizabalaga, Karen Evans, Simon Nicol, Jock W Young, and Kevin C Weng.  Impacts of climate change on marine top predators: advances and future challenges, 2015.
  * [46] Maria T Kavanaugh, Tom Bell, Dylan Catlett, Megan A Cimino, Scott C Doney, Willem Klajbor, Monique Messié, Enrique Montes, Frank E Muller-Karger, Daniel Otis, et al.  Satellite remote sensing and the marine biodiversity observation network.  Oceanography, 34(2):62–79, 2021.
  * [47] Marie-Fanny Racault, Corinne Le Quéré, Erik Buitenhuis, Shubha Sathyendranath, and Trevor Platt.  Phytoplankton phenology in the global ocean.  Ecological Indicators, 14(1):152–163, 2012.
  * [48] Tiexi Chen, Shengjie Zhou, Chuanzhuang Liang, Daniel Fiifi Tawia Hagan, Ning Zeng, Jun Wang, Tingting Shi, Xin Chen, and A Johannes Dolman.  The greening and wetting of the sahel have leveled off since about 1999 in relation to sst.  Remote Sensing, 12(17):2723, 2020.
  * [49] Claire L Parkinson.  Aqua: An earth-observing satellite mission to examine water and other climate variables.  IEEE Transactions on Geoscience and Remote Sensing, 41(2):173–183, 2003.
  * [50] Chelle L Gentemann and Kyle A Hilburn.  In situ validation of sea surface temperatures from the gcom-w 1 amsr 2 rss calibrated brightness temperatures.  Journal of Geophysical Research: Oceans, 120(5):3567–3585, 2015\.
  * [51] Bruce Brasnett and Dorina Surcel Colan.  Assimilating retrievals of sea surface temperature from viirs and amsr2.  Journal of Atmospheric and Oceanic Technology, 33(2):361–375, 2016\.
  * [52] Irina Gladkova, Alexander Ignatov, Fazlul Shahriar, Yury Kihai, Don Hillger, and Boris Petrenko.  Improved viirs and modis sst imagery.  Remote Sensing, 8(1):79, 2016.
  * [53] Mingkun Liu, Lei Guan, Wei Zhao, and Ge Chen.  Evaluation of sea surface temperature from the hy-2 scanning microwave radiometer.  IEEE Transactions on Geoscience and Remote Sensing, 55(3):1372–1380, 2016.
  * [54] Xiangchen Meng and Jie Cheng.  Estimating land and sea surface temperature from cross-calibrated chinese gaofen-5 thermal infrared data using split-window algorithm.  IEEE Geoscience and Remote Sensing Letters, 17(3):509–513, 2019\.
  * [55] Toneo Kawanishi, Toshihiro Sezai, Yasuyuki Ito, Keiji Imaoka, Toshiaki Takeshima, Yoshio Ishido, Akira Shibata, Masaharu Miura, Hiroyuki Inahata, and Roy W Spencer.  The advanced microwave scanning radiometer for the earth observing system (amsr-e), nasda’s contribution to the eos for global energy and water cycle studies.  IEEE Transactions on Geoscience and Remote Sensing, 41(2):184–194, 2003.
  * [56] Dudley B Chelton and Frank J Wentz.  Global microwave satellite observations of sea surface temperature for numerical weather prediction and climate research.  Bulletin of the American Meteorological Society, 86(8):1097–1116, 2005.
  * [57] F.J. Wentz, T. Meissner, C. Gentemann, and M. Brewer.  Remote sensing systems AQUA AMSR-E.  Available online at www.remss.com/missions/amsr, 2014.  [Accessed 07 June 2023].
  * [58] Jiawei Zhuang.  xesmf documentation.  2019\.
  * [59] Wayne E Esaias, Mark R Abbott, Ian Barton, Otis B Brown, Janet W Campbell, Kendall L Carder, Dennis K Clark, Robert H Evans, Frank E Hoge, Howard R Gordon, et al.  An overview of modis capabilities for ocean science observations.  IEEE Transactions on Geoscience and Remote Sensing, 36(4):1250–1265, 1998.
  * [60] NASA/JPL.  Modis aqua level 3 sst thermal ir monthly 4km daytime v2019.0, 2020.  Accessed 07 June 2023.
  * [61] NASA/JPL.  Modis aqua level 3 sst thermal ir monthly 4km nighttime v2019.0, 2020\.  Accessed 07 June 2023.
  * [62] Fabienne Gaillard, Thierry Reynaud, Virginie Thierry, Nicolas Kolodziejczyk, and Karina von Schuckmann.  In situ–based reanalysis of the global ocean temperature and salinity with isas: Variability of the heat content and steric height.  Journal of Climate, 29(4):1305–1323, 2016.
  * [63] Karl Weiss, Taghi M Khoshgoftaar, and DingDing Wang.  A survey of transfer learning.  Journal of Big data, 3(1):1–40, 2016.
  * [64] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun.  Deep residual learning for image recognition.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016.
  * [65] Han Xiao, Kashif Rasul, and Roland Vollgraf.  Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms.  arXiv preprint arXiv:1708.07747, 2017.
  * [66] Alex Krizhevsky and Geoff Hinton.  Convolutional deep belief networks on cifar-10.  Unpublished manuscript, 40(7):1–9, 2010.
  * [67] Marcin Andrychowicz, Misha Denil, Sergio Gomez, Matthew W Hoffman, David Pfau, Tom Schaul, Brendan Shillingford, and Nando De Freitas.  Learning to learn by gradient descent by gradient descent.  Advances in neural information processing systems, 29, 2016.
  * [68] Diederik P Kingma and Jimmy Ba.  Adam: A method for stochastic optimization.  arXiv preprint arXiv:1412.6980, 2014.
  * [69] Stuart Geman, Elie Bienenstock, and René Doursat.  Neural networks and the bias/variance dilemma.  Neural computation, 4(1):1–58, 1992.
  * [70] Geoffrey E Hinton, Nitish Srivastava, Alex Krizhevsky, Ilya Sutskever, and Ruslan R Salakhutdinov.  Improving neural networks by preventing co-adaptation of feature detectors.  arXiv preprint arXiv:1207.0580, 2012.
  * [71] Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov.  Dropout: a simple way to prevent neural networks from overfitting.  The journal of machine learning research, 15(1):1929–1958, 2014\.
  * [72] Sudhakar Kumawat, Gagan Kanojia, and Shanmuganathan Raman.  Shuffleblock: Shuffle to regularize convolutional neural networks.  In 2022 National Conference on Communications (NCC), pages 36–41. IEEE, 2022.
  * [73] Judea Pearl and Dana Mackenzie.  The book of why: the new science of cause and effect.  Basic books, 2018.
  * [74] Judea Pearl.  Direct and indirect effects.  In Probabilistic and causal inference: The works of Judea Pearl, pages 373–392. 2022.
  * [75] Matthew J McCarthy, Benjamin Dimmitt, and Frank E Muller-Karger.  Rapid coastal forest decline in florida’s big bend.  Remote Sensing, 10(11):1721, 2018.
  * [76] Antarpreet S Jutla, Ali S Akanda, and Shafiqul Islam.  Tracking cholera in coastal regions using satellite observations 1.  JAWRA Journal of the American Water Resources Association, 46(4):651–662, 2010.
  * [77] Jeeban Panthi, Soni M Pradhanang, Annika Nolte, and Thomas B Boving.  Saltwater intrusion into coastal aquifers in the contiguous united states—a systematic review of investigation approaches and monitoring networks.  Science of the Total Environment, page 155641, 2022.
  * [78] Christopher G Piecuch.  Likely weakening of the florida current during the past century revealed by sea-level observations.  Nature Communications, 11(1):3973, 2020.
  * [79] Zhou Wang and Alan C Bovik.  Mean squared error: Love it or leave it? a new look at signal fidelity measures.  IEEE signal processing magazine, 26(1):98–117, 2009.
  * [80] Segrey Kastryulin, Jamil Zakirov, Nicola Pezzotti, and Dmitry V Dylov.  Image quality assessment for magnetic resonance imaging.  arXiv preprint arXiv:2203.07809, 2022.
  * [81] Apostolos F Psaros, Kenji Kawaguchi, and George Em Karniadakis.  Meta-learning pinn loss functions.  Journal of Computational Physics, 458:111121, 2022.
  * [82] NASA/JPL.  Ghrsst level 4 ostia global historical reprocessed foundation sea surface temperature analysis produced by the uk meteorological office, 2023.  Accessed 08 June 2023.


capbtabboxtable[][]

DoE

    _Design of Experiment_
BO

    Bayesian Optimization
AF

    acquisition function
EI

    Expected Improvement
PI

    Probability of Improvement
UCB

    Upper Confidence Bound
ELA

    Exploratory Landscape Analysis
GP

    Gaussian Process
TTEI

    Top-Two Expected Improvement
TS

    Thompson Sampling
DAC

    Dynamic Algorithm Configuration
AC

    Algorithm Configuration
CMA-ES

    CMA-ES

# PI is back! Switching Acquisition Functions
in Bayesian Optimization

Carolin Benjamins  Institute of AI, Leibniz University Hannover, Germany
Elena Raponi  Sorbonne Université, CNRS, LIP6, Paris, France  TU München,
Germany  Anja Jankovic  Sorbonne Université, CNRS, LIP6, Paris, France  Koen
van der Blom  Sorbonne Université, CNRS, LIP6, Paris, France  Maria Laura
Santoni  University of Camerino, Italy  Marius Lindauer  Institute of AI,
Leibniz University Hannover, Germany  Carola Doerr  Sorbonne Université, CNRS,
LIP6, Paris, France

###### Abstract

Bayesian Optimization (BO) is a powerful, sample-efficient technique to
optimize expensive-to-evaluate functions. Each of the BO components, such as
the surrogate model, the acquisition function (AF), or the initial design, is
subject to a wide range of design choices. Selecting the right components for
a given optimization task is a challenging task, which can have significant
impact on the quality of the obtained results.
In this work, we initiate the analysis of which AF to favor for which
optimization scenarios. To this end, we benchmark SMAC3 using Expected
Improvement (EI) and Probability of Improvement (PI) as acquisition functions
on the 24 BBOB functions of the COCO environment. We compare their results
with those of schedules switching between AFs. One schedule aims to use EI’s
explorative behavior in the early optimization steps, and then switches to PI
for a better exploitation in the final steps. We also compare this to a random
schedule and round-robin selection of EI and PI.
We observe that dynamic schedules oftentimes outperform any single static one.
Our results suggest that a schedule that allocates the first
$25\text{\,}\mathrm{\char 37\relax}$ of the optimization budget to EI and the
last $75\text{\,}\mathrm{\char 37\relax}$ to PI is a reliable default.
However, we also observe considerable performance differences for the 24
functions, suggesting that a per-instance allocation, possibly learned on the
fly, could offer significant improvement over the state-of-the-art BO designs.

##  1 Introduction

Bayesian Optimization (BO) [Mockus, 2012] is a technique to find the global
optimum of black-box problems when each evaluation of the objective function
is expensive or otherwise resource-intensive, e.g., problems requiring
elaborated physical experiments or computationally-heavy simulations. BO is
widely known as one of the most efficient black-box approaches in terms of
number of function evaluations required, hence a promising tool for the
optimization of low-budget problems.

BO is defined as a modular framework which consists of  (i) defining an
initial set of points where the true (unknown, expensive) objective function
is evaluated,  (ii) approximating the objective function based on the initial
design by placing a prior probability distribution on it (i.e., building a
_surrogate model_),  (iii) selecting where to sample next by optimizing an
_acquisition function_ (also known as _infill criterion_) that takes care of
the trade-off between exploration and exploitation,  (iv) evaluating new
samples on the true objective function, and finally  (v) computing the
posterior probability distribution on the true objective function based on new
knowledge, hence updating the surrogate model.  Steps (iii) to (v) are
repeated iteratively until the evaluation budget is exhausted.

Figure 1: Switching AFs is beneficial. Scaled log-regret averaged over all
$24$ BBOB functions for $5$ dimensions with the $95\text{\,}\mathrm{\char
37\relax}$ confidence interval for the schedules from Figure 2.

Name | Schedule
---|---
static (EI) |
static (PI) |
random |
round robin |
explore-exploit ($0.25$) |
explore-exploit ($0.5$) |
explore-exploit ($0.75$) |

Figure 2: Schedule portfolio of PI, EI and their combinations.

Each component of the sequential BO pipeline is subject to a wide range of
design choices, which in turn affects the overall performance of the BO
procedure [Lindauer et al., 2019; Cowen-Rivers et al., 2021; Bossek et al.,
2020]. For example, each acquisition function (AF) exhibits different
behavior, with some being more explorative and others more exploitative. To
date, design choices have been predominantly _static_ during BO execution.
However, other areas of optimization indicate that _dynamic_ choices can lead
to performance gains Karafotias et al. [2015]; Doerr and Doerr [2020];
Adriaensen et al. [2022].

In this work, we investigate whether the same applies to BO. Specifically, we
propose a _dynamic schedule_ that switches between two AFs, namely Probability
of Improvement (PI) and Expected Improvement (EI) Forrester et al. [2008],
_during_ the BO procedure. This simple approach is already quite effective and
outperforms any single static schedule on most problems. We assess our method
on the BBOB test suite from the COCO library [Hansen et al., 2021], which is a
standard benchmark in black-box optimization. We purposely opt for such a
benchmark in order to shift focus from a purely performance-oriented view
towards _understanding_ the problem characteristics that favor one or the
other AF, which is a stark difference between this paper and previous works on
dynamic AF selection. We thus observe that the choice of schedule is largely
problem-dependent and that further investigation in this direction is of
utmost importance to fully understand and exploit the untapped potential of
adaptive AF selection.

Background  Since our approach is focused on PI and EI, we briefly recall
their formulations:

$\text{PI}(x)=\Phi\left(\frac{\Delta(x)}{\sigma(x)}\right)\quad\text{and}\quad\text{EI}(x)=\begin{cases}\Delta(x)\
\Phi\left(\frac{\Delta(x)}{\sigma(x)}\right)+\sigma(x)\
\phi\left(\frac{\Delta(x)}{\sigma(x)}\right)&\mbox{if }\sigma(x)>0\\\
0&\mbox{if }\sigma(x)=0\end{cases}$

with $\Delta(x)=y_{\text{min}}-\mu(x)$, where $y_{\text{min}}$ is the best
observed value so far, and the mean $\mu(x)$ and the variance $\sigma^{2}(x)$
are queried from the posterior of the surrogate model – in our work a Gaussian
Process (GP). $\Phi$ and $\phi$ are the Gaussian cumulative distribution
function and probability density function, respectively. By definition, in a
minimization problem, the maximization of PI drives the selection of new
candidates for evaluation towards areas of the domain where the mean value of
the GP model is low. On the contrary, EI balances exploitation (thanks to the
first summation term) and exploration (second summation term) by alternating
sampling in promising areas of the domain according to the mean value and
unexplored ones, where the variance of the GP model is high. Comparing both,
EI is more explorative and PI more exploitative, see Appendix A for an
example.

Related Work  We first present existing work on AFs and focus on different
approaches considering AFs. Important findings are reported when the main
target is to improve and develop better AFs, from enhancing EI [Qin et al.,
2017; Balandat et al., 2020] to meta-learning neural AFs [Volpp et al., 2020].
A different line of work is concerned with combining different AFs, e.g. by
building a portfolio of AFs (EI, P̧I, Upper Confidence Bound (UCB) with
different hyperparameter settings) and then using an online multi-armed bandit
strategy to assign probabilities of which AF to use at which step, called GP-
Hedge [Hoffman et al., 2011]. Their work indicates that the performance of GP-
Hedge highly varies with the number of arms and their respective
hyperparameter settings. Similarly to GP-Hedge, Kandasamy et al. [2020] update
weights of their portfolio (UCB, EI, Thompson Sampling (TS) [Thompson, 1933],
Top-Two Expected Improvement (TTEI) [Qin et al., 2017]) in an online manner.
They do not include PI as they observe it exhibits inferior performance
compared to other single static AFs. In addition, robust versions of EI, PI
and UCB can be combined to a multi-objective AF combining the strengths of the
individual ones [Cowen-Rivers et al., 2021]. In contrast to the existing
literature, we here take a step back and ask ourselves what we could achieve
by employing a simplistic approach of switching between AFs through a dynamic
schedule.

It has also been shown in other optimization-related areas that dynamic
choices are beneficial in terms of performance, e.g. in evolutionary
computation Karafotias et al. [2015]; Doerr and Doerr [2020], planning [Speck
et al., 2021] and deep learning [Adriaensen et al., 2022]. Recently, the
introduction of Dynamic Algorithm Configuration (DAC) [Biedenkapp et al.,
2020] underlines the potential of training dynamic schedules (as opposed to
selecting algorithm components _on the fly_ , as is usually done in
evolutionary computation Hansen et al. [2003]).

##  2 Proposed Method

To evaluate the potential of dynamic AF schedules, we consider several
different variants, summarized in Figure 2. As baselines, we use static
schedules, namely EI and PI. Two types of dynamic schedules are considered:
alternating and switching. The former go back and forth between EI and PI,
whereas the latter make a single switch from EI to PI after a pre-defined
percentage of the budget has been spent. We introduce two alternating
schedules as baselines for a dynamic approach: random and round robin. For the
random schedule the AF is chosen uniformly at random before a new solution is
sampled, while round robin simply alternates the AFs for each sample. Finally,
three different switching schedules are considered, making a switch after
$25\text{\,}\mathrm{\char 37\relax}$, $50\text{\,}\mathrm{\char 37\relax}$ and
$75\text{\,}\mathrm{\char 37\relax}$ of the budget remaining after the initial
design (i.e., surrogate-based function evaluations), respectively. This makes
it possible to investigate whether the conceptual idea of switching is
beneficial and additionally gives insight into the effects of switching at
different stages of the optimization process.

##  3 Experiments

We evaluate our schedules on the $24$ single-objective noiseless BBOB
functions of the COCO benchmark [Hansen et al., 2021] in dimensions $5$ with
$60$ seeds (i.e., we perform $60$ independent runs on each problem and for
each AF schedule). We optimize our functions with an initial design (or:
_Design of Experiment_ (DoE)) of $3d=15$ data points and BO length of
$20d=100$ function evaluations. Per seed, the initial design is the same for
every schedule. To create the dynamic BO, SMAC3 [Lindauer et al., 2022] is
adapted accordingly and we use a GP as surrogate model. All experiments were
conducted on a Slurm CPU cluster with $1592$ CPUs available across nodes. For
visualization, we consider the log-regret of the incumbent (best evaluated
search point) and only visualize the evaluations proposed by BO, omitting the
initial design. In the violin plots the log-regrets of the final incumbents of
all $60$ seeds are normalized to $[0,1]$ per BBOB function. In the convergence
plots the log-regret of the incumbents of all $60$ seeds over all evaluations
are normalized to $[0,1]$ per BBOB function and the means with the
$95\text{\,}\mathrm{\char 37\relax}$ confidence intervals are shown. The plots
for each function can be found in Appendix B. The code can be found here:
https://github.com/automl/pi_is_back.

Observations

(a) Landscape of Weierstrass (F16) [Hansen et al., 2021]

(b) Log-Regret (Scaled) per Step

Figure 3: BBOB Function 16: Highly rugged and moderately repetitive landscape
with many local optima. Dynamic schedules outperform static ones and we can
clearly observe the boost gained after switching from EI to PI.

A first observation is that switching from EI to PI is in general beneficial
when the function landscape has an adequate global structure (F1-F19). Here,
the only exceptions are F5 and F19. F5 is a purely linear function, where EI
performs best as it explores fast in the beginning and exploits sufficiently
fast later on because of the simplicity of the landscape. For F19, we
hypothesize that only PI is able to exploit a lucky initial design landing
close to optima as indicated by the individual runs in Figure 23(c).

In addition, we can see that PI works well for uni-modal and quite smooth
functions (F1-F14), and even better when used after the switch. This
observation is in line with PI’s exploitative behavior. In these cases
exploiting does not miss any other optima further away in the landscape.

In contrast, PI is in general worse than EI and also not beneficial after the
switch for multi-modal functions with weak global structure (F20-F24). Again,
this is in line with our intuition because for these functions we have many
important basins of attraction that have to be discovered before starting
exploitation. The only exception is F23 (Figure 27), a rugged function with a
high number of global optima, where the probability of starting in a basin of
attraction is high and thus exploitation is a viable strategy. Also, the
flatter the landscape the worse PI performs which can be also seen in F7
(slope with step, Figure 11).

Besides the static and the switching schedules, round robin switches from EI
to PI and vice versa for each new function evaluation. The round robin
schedule creates a step-wise progress, but is never the best strategy.
Apparently, less frequent switching is preferable in order to take advantage
of the strengths of each AF. On average, the random schedule performs
similarly to round robin, but with a smoother progression. Most likely, the
random schedule still switches too frequently for the AFs to work effectively.

On F16 we observe that a late switch from EI to PI performs best, see Figure
3(b). We can also nicely spot the general boosts after switching after
$25\text{\,}\mathrm{\char 37\relax}$, $50\text{\,}\mathrm{\char 37\relax}$ and
$75\text{\,}\mathrm{\char 37\relax}$. F16 has a highly rugged and moderately
repetitive landscape with many local optima with evident quality difference.
Therefore, PI can be trapped in a bad local optimum if applied too early.

Key Insights  To summarize our observations, the general optimal choice of
static or dynamic AF schedules depends on the problem at hand. Plus, switching
from EI to PI after a certain percentage of the budget of total number of
function evaluations is beneficial and can boost performance. However, the
optimal switching point depends on the problem. On average, switching after
$25\text{\,}\mathrm{\char 37\relax}$ seems to be best and is a viable schedule
if the problem’s properties are unknown.

##  4 Conclusions and Future Work

In this work we acknowledge the modular structure of Bayesian Optimization
(BO) and explore different acquisition function schedules composed of Expected
Improvement (EI) and Probability of Improvement (PI), with the premise that a
dynamic choice would boost BO performance. We evaluate our approach on a
standard benchmark for black-box optimization, and observe that dynamic
acquisition function (AF) schedules often outperform single static ones,
although their performance depends on the problem at hand. As a general
recommendation, if the properties of the problem to optimize are unknown,
switching from EI to PI after $25\text{\,}\mathrm{\char 37\relax}$ of the
budget of surrogate-based evaluations should be favored. We also highlight
some key insights into the interplay of the nature of the problem and the
performance of the proposed approach.

These preliminary findings suggest the potential of the dynamic choices in a
BO pipeline merit further investigation. As future steps in this direction,
finding the most appropriate schedule of different AFs is an open problem we
could tackle with meta-learning. One way would be to train models recommending
which AF to use at which step based on problem representation, i.e., typically
Exploratory Landscape Analysis (ELA) features [Mersmann et al., 2011]. In
addition, finding meaningful state features could enable an online selection
of the AF via heuristics or Reinforcement Learning [Adriaensen et al., 2022].
To this end, besides elapsed time and progress being default choices as state
features, we could incorporate properties of the surrogate model to serve as a
guide to select the AF. Furthermore, including more AFs and their
hyperparameters in the portfolio, as well as dynamically changing other
components of BO, like the surrogate model, the GP kernel, or the optimizer of
the AF, might increase overall performance and sample-efficiency of BO.

### Acknowledgments

Our work has been financially supported by the the ANR T-ERC project
_VARIATION_ (ANR-22-ERCS-0003-01), by the CNRS INS2I project _RandSearch_ ,
and by the PRIME programme of the German Academic Exchange Service (DAAD) with
funds from the German Federal Ministry of Education and Research (BMBF), and
by RFBR and CNRS, project number 20-51-15009. Carolin Benjamins and Marius
Lindauer acknowledge funding by the German Research Foundation (DFG) under LI
2801/4-1.

## References

  * Mockus [2012] Jonas Mockus.  _Bayesian Approach to Global Optimization: Theory and Applications_.  Springer Science & Business Media, December 2012.  ISBN 978-94-009-0909-0.
  * Lindauer et al. [2019] M. Lindauer, M. Feurer, K. Eggensperger, A. Biedenkapp, and F. Hutter.  Towards assessing the impact of bayesian optimization’s own hyperparameters.  In _IJCAI’19 DSO Workshop_ , 2019.
  * Cowen-Rivers et al. [2021] A. Cowen-Rivers, W. Lyu, R. Tutunov, Z. Wang, A. Grosnit, R. Griffiths, A. Maraval, H. Jianye, J. Wang, J. Peters, and H. Ammar.  An empirical study of assumptions in Bayesian optimisation.  _arXiv:2012.03826 [cs.LG]_ , 2021.
  * Bossek et al. [2020] Jakob Bossek, Carola Doerr, and Pascal Kerschke.  Initial design strategies and their effects on sequential model-based optimization: an exploratory case study based on BBOB.  In _Proc. of Genetic and Evolutionary Computation Conference (GECCO’20)_ , pages 778–786. ACM, 2020.  doi: 10.1145/3377930.3390155.  URL https://doi.org/10.1145/3377930.3390155.
  * Karafotias et al. [2015] G. Karafotias, M. Hoogendoorn, and Á. Eiben.  Parameter control in evolutionary algorithms: Trends and challenges.  _IEEE Trans. Evolutionary Computation_ , 19(2):167–187, 2015.
  * Doerr and Doerr [2020] Benjamin Doerr and Carola Doerr.  Theory of parameter control mechanisms for discrete black-box optimization: Provable performance gains through dynamic parameter choices.  In _Theory of Evolutionary Computation: Recent Developments in Discrete Optimization_ , pages 271–321. Springer, 2020.  doi: 10.1007/978-3-030-29414-4˙6.  Also available online at https://arxiv.org/abs/1804.05650.
  * Adriaensen et al. [2022] S. Adriaensen, A. Biedenkapp, G. Shala, N. Awad, T. Eimer, M. Lindauer, and F. Hutter.  Automated dynamic algorithm configuration.  _arXiv:2205.13881 [cs.AI]_ , 2022.
  * Forrester et al. [2008] Alexander I. J. Forrester, András Sóbester, and Andy J. Keane.  _Engineering Design via Surrogate Modelling - A Practical Guide_.  John Wiley & Sons Ltd., 2008.  ISBN 978-0-470-06068-1.
  * Hansen et al. [2021] N. Hansen, A. Auger, R. Ros, O. Mersmann, T. Tušar, and D. Brockhoff.  COCO: A platform for comparing continuous optimizers in a black-box setting.  _Optimization Methods and Software_ , 36:114–144, 2021\.  doi: 10.1080/10556788.2020.1808977.
  * Qin et al. [2017] Chao Qin, Diego Klabjan, and Daniel Russo.  Improving the expected improvement algorithm.  In Isabelle Guyon, Ulrike von Luxburg, Samy Bengio, Hanna M. Wallach, Rob Fergus, S. V. N. Vishwanathan, and Roman Garnett, editors, _Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, December 4-9, 2017, Long Beach, CA, USA_ , pages 5381–5391, 2017.  URL https://proceedings.neurips.cc/paper/2017/hash/b19aa25ff58940d974234b48391b9549-Abstract.html.
  * Balandat et al. [2020] M. Balandat, B. Karrer, D. Jiang, S. Daulton, B. Letham, A. Wilson, and E. Bakshy.  Botorch: A framework for efficient monte-carlo Bayesian optimization.  In _Proc. of NeurIPS’20_ , 2020.
  * Volpp et al. [2020] M. Volpp, L. Fröhlich, K. Fischer, A. Doerr, S. Falkner, F. Hutter, and C. Daniel.  Meta-learning acquisition functions for transfer learning in bayesian optimization.  In _Proc. of ICLR’20_ , 2020.  URL https://openreview.net/forum?id=ryeYpJSKwr.
  * Hoffman et al. [2011] Matthew Hoffman, Eric Brochu, and Nando de Freitas.  Portfolio allocation for Bayesian optimization.  In _Proceedings of the Twenty-Seventh Conference on Uncertainty in Artificial Intelligence_ , UAI’11, pages 327–336, Arlington, Virginia, USA, 2011. AUAI Press.  ISBN 9780974903972.
  * Kandasamy et al. [2020] Kirthevasan Kandasamy, Karun Raju Vysyaraju, Willie Neiswanger, Biswajit Paria, Christopher R Collins, Jeff Schneider, Barnabas Poczos, and Eric P Xing.  Tuning hyperparameters without grad students: Scalable and robust Bayesian optimisation with dragonfly.  _J. Mach. Learn. Res._ , 21(81):1–27, 2020.
  * Thompson [1933] W. Thompson.  On the likelihood that one unknown probability exceeds another in view of the evidence of two samples.  _Biometrika_ , 25(3/4):285–294, 1933.
  * Speck et al. [2021] David Speck, André Biedenkapp, Frank Hutter, Robert Mattmüller, and Marius Lindauer.  Learning heuristic selection with dynamic algorithm configuration.  In Susanne Biundo, Minh Do, Robert Goldman, Michael Katz, Qiang Yang, and Hankz Hankui Zhuo, editors, _Proceedings of the Thirty-First International Conference on Automated Planning and Scheduling, ICAPS 2021, Guangzhou, China (virtual), August 2-13, 2021_ , pages 597–605. AAAI Press, 2021\.  URL https://ojs.aaai.org/index.php/ICAPS/article/view/16008.
  * Biedenkapp et al. [2020] A. Biedenkapp, H. F. Bozkurt, T. Eimer, F. Hutter, and M. Lindauer.  Dynamic algorithm configuration: Foundation of a new meta-algorithmic framework.  In _Proc. of ECAI’20_ , pages 427–434, 2020.
  * Hansen et al. [2003] N. Hansen, S. D. Müller, and P. Koumoutsakos.  Reducing the time complexity of the derandomized evolution strategy with covariance matrix adaptation (CMA-ES).  _Evolutionary Computing_ , 11(1):1–18, 2003.
  * Lindauer et al. [2022] M. Lindauer, K. Eggensperger, M. Feurer, A. Biedenkapp, D. Deng, C. Benjamins, T. Ruhkopf, R. Sass, and F. Hutter.  SMAC3: A versatile bayesian optimization package for Hyperparameter Optimization.  _Journal of Machine Learning Research (JMLR) – MLOSS_ , 23(54):1–9, 2022.
  * Mersmann et al. [2011] Olaf Mersmann, Bernd Bischl, Heike Trautmann, Mike Preuss, Claus Weihs, and Günter Rudolph.  Exploratory landscape analysis.  In Natalio Krasnogor and Pier Luca Lanzi, editors, _13th Annual Genetic and Evolutionary Computation Conference, GECCO 2011, Proceedings, Dublin, Ireland, July 12-16, 2011_ , pages 829–836. ACM, 2011.  doi: 10.1145/2001576.2001690.  URL https://doi.org/10.1145/2001576.2001690.

##  Appendix A Exploration vs. Exploitation

We choose EI and PI to build our AF schedules because of their different
search behavior. In Figure 4 we exemplarily show EI’s explorative character
and PI’s exploitative character. Both behaviors can be beneficial depending on
the problem to optimize.

Figure 4: EI vs. PI on BBOB function 12 (upper) and 21 (lower). The dots
indicate the visited configuration at a certain step. We visualize all $20$
seeds with an initial design size of $10$ (not plotted) and $40$ BO
evaluations.

##  Appendix B Additional Results

(a) Final Log Regret (Scaled)

(b) Log-Regret (Scaled) per Step

(c) Log-Regret (Scaled) per Step and Seed

Figure 5: BBOB Function 1

(a) Final Log Regret (Scaled)

(b) Log-Regret (Scaled) per Step

(c) Log-Regret (Scaled) per Step and Seed

Figure 6: BBOB Function 2

(a) Final Log Regret (Scaled)

(b) Log-Regret (Scaled) per Step

(c) Log-Regret (Scaled) per Step and Seed

Figure 7: BBOB Function 3

(a) Final Log Regret (Scaled)

(b) Log-Regret (Scaled) per Step

(c) Log-Regret (Scaled) per Step and Seed

Figure 8: BBOB Function 4

(a) Final Log Regret (Scaled)

(b) Log-Regret (Scaled) per Step

(c) Log-Regret (Scaled) per Step and Seed

Figure 9: BBOB Function 5

(a) Final Log Regret (Scaled)

(b) Log-Regret (Scaled) per Step

(c) Log-Regret (Scaled) per Step and Seed

Figure 10: BBOB Function 6

(a) Final Log Regret (Scaled)

(b) Log-Regret (Scaled) per Step

(c) Log-Regret (Scaled) per Step and Seed

Figure 11: BBOB Function 7

(a) Final Log Regret (Scaled)

(b) Log-Regret (Scaled) per Step

(c) Log-Regret (Scaled) per Step and Seed

Figure 12: BBOB Function 8

(a) Final Log Regret (Scaled)

(b) Log-Regret (Scaled) per Step

(c) Log-Regret (Scaled) per Step and Seed

Figure 13: BBOB Function 9

(a) Final Log Regret (Scaled)

(b) Log-Regret (Scaled) per Step

(c) Log-Regret (Scaled) per Step and Seed

Figure 14: BBOB Function 10

(a) Final Log Regret (Scaled)

(b) Log-Regret (Scaled) per Step

(c) Log-Regret (Scaled) per Step and Seed

Figure 15: BBOB Function 11

(a) Final Log Regret (Scaled)

(b) Log-Regret (Scaled) per Step

(c) Log-Regret (Scaled) per Step and Seed

Figure 16: BBOB Function 12

(a) Final Log Regret (Scaled)

(b) Log-Regret (Scaled) per Step

(c) Log-Regret (Scaled) per Step and Seed

Figure 17: BBOB Function 13

(a) Final Log Regret (Scaled)

(b) Log-Regret (Scaled) per Step

(c) Log-Regret (Scaled) per Step and Seed

Figure 18: BBOB Function 14

(a) Final Log Regret (Scaled)

(b) Log-Regret (Scaled) per Step

(c) Log-Regret (Scaled) per Step and Seed

Figure 19: BBOB Function 15

(a) Final Log Regret (Scaled)

(b) Log-Regret (Scaled) per Step

(c) Log-Regret (Scaled) per Step and Seed

Figure 20: BBOB Function 16

(a) Final Log Regret (Scaled)

(b) Log-Regret (Scaled) per Step

(c) Log-Regret (Scaled) per Step and Seed

Figure 21: BBOB Function 17

(a) Final Log Regret (Scaled)

(b) Log-Regret (Scaled) per Step

(c) Log-Regret (Scaled) per Step and Seed

Figure 22: BBOB Function 18

(a) Final Log Regret (Scaled)

(b) Log-Regret (Scaled) per Step

(c) Log-Regret (Scaled) per Step and Seed

Figure 23: BBOB Function 19

(a) Final Log Regret (Scaled)

(b) Log-Regret (Scaled) per Step

(c) Log-Regret (Scaled) per Step and Seed

Figure 24: BBOB Function 20

(a) Final Log Regret (Scaled)

(b) Log-Regret (Scaled) per Step

(c) Log-Regret (Scaled) per Step and Seed

Figure 25: BBOB Function 21

(a) Final Log Regret (Scaled)

(b) Log-Regret (Scaled) per Step

(c) Log-Regret (Scaled) per Step and Seed

Figure 26: BBOB Function 22

(a) Final Log Regret (Scaled)

(b) Log-Regret (Scaled) per Step

(c) Log-Regret (Scaled) per Step and Seed

Figure 27: BBOB Function 23

(a) Final Log Regret (Scaled)

(b) Log-Regret (Scaled) per Step

(c) Log-Regret (Scaled) per Step and Seed

Figure 28: BBOB Function 24

  *[BO]: Bayesian Optimization
  *[AF]: acquisition function
  *[PI]: Probability of Improvement
  *[EI]: Expected Improvement
  *[GP]: Gaussian Process
  *[UCB]: Upper Confidence Bound
  *[TS]: Thompson Sampling
  *[TTEI]: Top-Two Expected Improvement
  *[DAC]: Dynamic Algorithm Configuration
  *[DoE]: Design of Experiment
  *[ELA]: Exploratory Landscape Analysis