# AutoLoss: Automated Loss Function Search in Recommendations

Xiangyu Zhao1,2, Haochen Liu2, Wenqi Fan3∗, Hui Liu2, Jiliang Tang2, Chong
Wang4 1City University of Hong Kong, 2Michigan State University, 3The Hong
Kong Polytechnic University, 4Bytedance
<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>
(2021)

###### Abstract.

Designing an effective loss function plays a crucial role in training deep
recommender systems. Most existing works often leverage a predefined and fixed
loss function that could lead to suboptimal recommendation quality and
training efficiency. Some recent efforts rely on exhaustively or manually
searched weights to fuse a group of candidate loss functions, which is
exceptionally costly in computation and time. They also neglect the various
convergence behaviors of different data examples. In this work, we propose an
AutoLoss framework that can automatically and adaptively search for the
appropriate loss function from a set of candidates. To be specific, we develop
a novel controller network, which can dynamically adjust the loss
probabilities in a differentiable manner. Unlike existing algorithms, the
proposed controller can adaptively generate the loss probabilities for
different data examples according to their varied convergence behaviors. Such
design improves the model’s generalizability and transferability between deep
recommender systems and datasets. We evaluate the proposed framework on two
benchmark datasets. The results show that AutoLoss outperforms representative
baselines. Further experiments have been conducted to deepen our
understandings of AutoLoss, including its transferability, components and
training efficiency.

AutoML; Recommender Systems; Loss Functions

* Corresponding author.

††journalyear: 2021††copyright: acmcopyright††conference: Proceedings of the
27th ACM SIGKDD Conference on Knowledge Discovery and Data Mining; August
14–18, 2021; Virtual Event, Singapore††booktitle: Proceedings of the 27th ACM
SIGKDD Conference on Knowledge Discovery and Data Mining (KDD ’21), August
14–18, 2021, Virtual Event, Singapore††price: 15.00††doi:
10.1145/3447548.3467208††isbn: 978-1-4503-8332-5/21/08††ccs: Information
systems Recommender systems

##  1\. Introduction

In the era of information explosion, recommender systems play a pivotal role
in alleviating information overload, which vastly enhance user experiences in
many commercial applications, such as generating playlists in video and music
services (Zhao et al., 2021; Zhao et al., 2020d), recommending products in
online stores (Zhao et al., 2020c; Zou et al., 2020; Fan et al., 2020; Zhao et
al., 2018a, b), and suggesting locations for geo-social events (Liu et al.,
2017; Zhao et al., 2016; Guo et al., 2016). With the recent growth of deep
learning techniques, there have been increasing interests in developing deep
recommender systems (DRS) (Nguyen et al., 2017; Wu et al., 2016a). DRS has
improved the recommendation quality since they can effectively learn feature
representations and capture the nonlinear relationships between users and
items via deep architectures (Zhang et al., 2019). Aside from developing
sophisticated neural network architectures, well-designed loss functions have
also been demonstrated to be effective in improving the performance in
different recommendation tasks, such as item rating prediction (regression)
(Ravi and Vairavasundaram, 2016), click-through rate prediction (binary
classification) (Guo et al., 2017; Ge et al., 2021), user behavior prediction
(multi-class classification) (Zhao et al., 2019), and item retrieval
(clustering) (Gao et al., 2020).

To optimize DRS frameworks, most existing works are based on a predefined and
fixed loss function, such as mean-squared-error (MSE) or mean-absolute-error
(MAE) loss for regression tasks. Then DRS frameworks are optimized in a back-
propagation manner, which computes gradients effectively and efficiently to
minimize the given loss on the training dataset. During this process, the key
step is to calculate gradients of network parameters for minimizing loss
functions. However, it is often unclear whether the gradients generated from a
given loss function are optimal. For example, in regression tasks, the MSE
loss can ensure that the trained model has no outlier predictions with huge
errors, while MAE performs better if we only want a well-rounded model that
performs well on the majority (Fahrmeir et al., 2007; Chatterjee and Hadi,
2015). Therefore, solely utilizing a predefined and fixed loss function for
all data examples, i.e., user-item interactions, cannot guarantee the optimal
gradients throughout, especially when the interactions have varied convergence
behaviors in the non-stationary environment of online recommendation
platforms. In addition, there is often a gap between the model training and
evaluation performance in real-world recommender systems. For instance, we
usually train a predictive model by minimizing cross-entropy loss in online
advertising, and evaluate the model performance by click-through rate (CTR).
Consequently, it naturally raises a question - can we incorporate more loss
functions in the training phase to enhance the model performance?

Figure 1. Overview of the AutoLoss framework.

Efforts have been made to develop strategies to fuse multiple loss functions,
which can take advantage of multiple loss functions in a weighted sum fashion.
For example, Panoptic FPN (Kirillov et al., 2019) leverages a grid search to
find better loss weights; and UPSNet (Xiong et al., 2019) carefully
investigates the weighting scheme of loss functions. However, these works rely
on exhaustively or manually search for loss weights from a large candidate
space, which would be an extremely costly execution in both computing power
and time. Also, they aim to learn a set of unified and static weights over the
loss functions, which entirely overlook the different convergence behaviors of
data examples. Finally, retraining loss weights is always desired when
switching among different DRS frameworks or recommendation datasets, which
reduces their generalizability and transferability.

In order to obtain more accurate gradients to improve the recommendation
performance and the training efficiency, we propose an automated loss function
search framework, AutoLoss, which can dynamically and adaptively select
appropriate loss functions for training DRS frameworks. Different from
existing searching models with predefined and fixed loss functions, or the
loss weights exhaustively or manually searched, the optimal loss function in
AutoLoss is automatically selected for each data example in a differentiable
manner. The experiments on two datasets demonstrate the effectiveness of the
proposed framework. We summarize our major contributions as follows:

  * •

We propose an end-to-end framework, AutoLoss, which can automatically select
the proper loss functions for training DRS frameworks with better
recommendation performance and training efficiency;

  * •

A novel controller network is developed to adaptively adjust the probabilities
over multiple loss functions according to different data examples’ dynamic
convergence behaviors during training, which enhances the model
generalizability between different DRS frameworks and datasets;

  * •

We empirically demonstrate the effectiveness of the proposed framework on
real-world benchmark datasets. Extensive studies verify the importance of
model components and the transferability of AutoLoss.

The rest of this paper is organized as follows. In Section 2, we detail the
framework of automatically searching the probabilities over multiple loss
functions, the architecture of the main DRS network and controller network,
and propose an AutoML-based optimization algorithm. Section 3 carries out
experiments based on real-world datasets and presents experimental results.
Section 4 briefly reviews related work. Finally, Section 5 concludes this work
and discusses future work.

##  2\. The Proposed Framework

In this section, we will present an end-to-end framework, AutoLoss, which
effectively tackles the aforementioned challenges in Section 1 via
automatically and adaptively searching the optimal loss function from several
candidates according to data examples’ convergence behaviors. We will first
provide an overview of the framework; next detail the architectures of the
main DRS network; then introduce the loss function search method with a novel
controller network; and finally provide an AutoML-based optimization
algorithm.

###  2.1. An Overview

In this subsection, we will give an overview of the AutoLoss framework.
AutoLoss aims to automatically select appropriate loss functions from a set of
candidates for different data examples (i.e., user-item interactions). We
demonstrate the framework in Figure 1. With a DRS network, a controller
network and a set of pre-defined candidate loss functions, the learning
process of AutoLoss mainly consists of two major steps.

The forward-propagation step. Given a mini-batch of data examples, the main
DRS network first generates predictions $\boldsymbol{\hat{y}}$ based on the
input features $\boldsymbol{x}$. Then, we can calculate the losses
$\\{\ell_{i}\\}$ for each candidate loss function according to the ground
truth labels $\boldsymbol{y}$ and predictions $\boldsymbol{\hat{y}}$.
Meanwhile, the controller network takes $\boldsymbol{(y,\hat{y})}$ and outputs
the probabilities $\boldsymbol{p}$ over loss functions for each data example.
Finally, the overall loss $\mathcal{L}$ can be calculated according to the
losses from $\\{\ell_{i}\\}$ and the probabilities $\boldsymbol{p}$.

The backward-propagation step. We first fix the parameters of the controller
network and update the main DRS network parameters upon the training data
examples. Then, we fix the DRS parameters and optimize the controller network
parameters based on a mini-batch of validation data examples. This alternative
updating approach enhances the generalizability, and prevents AutoLoss from
selecting probabilities that overfit the training data examples (Pham et al.,
2018; Liu et al., 2018). Next, we will introduce the details of AutoLoss.

Figure 2. Architectures of DeepFM and IPNN.

###  2.2. Deep Recommender System Network

AutoLoss is quite general for most existing deep recommender system frameworks
(Rendle, 2010; Guo et al., 2017; Lian et al., 2018; Qu et al., 2016). As
visualized in Figure 2, they typically have four components: embedding layer,
interaction layer, MLP layer and output layer. We now briefly introduce these
components.

####  2.2.1. Embedding Layer

The raw input features of users and items are usually categorical or numeric,
and in the form of multiple fields. Most DRS works first transform the input
features into binary vectors, and then embed them into continuous vectors
using a field-wise embedding. In this way, a user-item interaction data
example
$\boldsymbol{x}=\left[\boldsymbol{x}_{1},\boldsymbol{x}_{2},\cdots,\boldsymbol{x}_{m}\right]$
can be represented as the concatenation of binary vectors from all feature
fields:

$[\underbrace{0,1,0,0,\ldots,0}_{\boldsymbol{x}_{1}:\,\text{userid
}}][\underbrace{1,0}_{\boldsymbol{x}_{2}:\,\text{gender
}}][\underbrace{0,1,0,0}_{\boldsymbol{x}_{3}:\,\text{age
}}]\overbrace{\cdots\cdots}^{\text{other
fields}}\underbrace{[0,1,0,1,\ldots,0]}_{\boldsymbol{x}_{m}:\,\text{itemid}}$

where $m$ is the number of feature fields and $\boldsymbol{x}_{i}$ is the
binary vector of the $i^{th}$ field. The categorical data are transformed into
binary vectors via one-hot encoding, e.g., $[0,1]$ for ${gender=Female}$ and
$[1,0]$ for ${gender=Male}$. The numeric data are first partitioned into
buckets, and then we have a binary vector for each bucket, e.g., we can use
$[0,0,0,1]$ for child whose $age{\in}[0{,}14]$, $[0,0,1,0]$ for youth whose
$age{\in}[15{,}24]$, $[0,1,0,0]$ for adult whose $age{\in}[25{,}64]$, and
$[1,0,0,0]$ for seniors whose $age{\geq}65$.

Since vector $\boldsymbol{x}$ is high-dimensional and very sparse, and
different feature fields have various lengths, DRS models usually introduce an
embedding layer to transform each binary vector $\boldsymbol{x}_{i}$ into a
low-dimensional continuous vector as:

(1) $\boldsymbol{e}_{i}=\boldsymbol{v}_{i}\boldsymbol{x}_{i}$

where $\boldsymbol{v}_{i}\in R^{d\times u_{i}}$ is the weight matrix with
$u_{i}$ the number of unique feature values in the $i^{th}$ feature field, and
$d$ is the pre-defined size of low-dimensional vectors111For multi-valued
features (e.g.,“Interest=Movie, Sports”), the feature embedding is the sum or
average of multiple embeddings (Covington et al., 2016).. Finally, the
embedding layer will output the concatenation of embedding vectors from all
feature fields:

(2)
$\boldsymbol{E}=\left[\boldsymbol{e}_{1},\boldsymbol{e}_{2},\ldots,\boldsymbol{e}_{m}\right]$

####  2.2.2. Interaction Layer

After representing the input features as low-dimensional embeddings, DRS
models usually develop an interaction layer to explicitly capture the
interactions among feature fields. The most widely used method is
factorization machine (FM) (Rendle, 2010). In addition to the linear
interactions among features, FM can explicitly model the pairwise (second-
order) feature interactions via the inner product of feature embeddings:

(3)
$\left[\left\langle\boldsymbol{e}_{1},\boldsymbol{e}_{2}\right\rangle,\left\langle\boldsymbol{e}_{1},\boldsymbol{e}_{3}\right\rangle,\ldots,\left\langle\boldsymbol{e}_{m-1},\boldsymbol{e}_{m}\right\rangle\right]$

where $\langle\cdot,\cdot\rangle$ is the inner product of two embeddings, and
the number of pairwise feature interactions is $\boldsymbol{C}_{m}^{2}$. Then,
the interaction layer will output:

(4)
$l_{fm}=\langle\boldsymbol{w},\boldsymbol{x}\rangle+\sum_{i=1}^{m}\sum_{j>i}^{m}\left\langle\boldsymbol{e}_{i},\boldsymbol{e}_{j}\right\rangle$

Where $\boldsymbol{w}$ is the weight over the binary vector $\boldsymbol{x}$
of input features. The first term represents the impact of first-order feature
interactions, and the second term reflects the impact of second-order feature
interactions. FM can explicitly model even higher order interactions, such as
$\sum_{i=1}^{m}\sum_{j>i}^{m}\sum_{t>j}^{m}\left\langle\boldsymbol{e}_{i},\boldsymbol{e}_{j},\boldsymbol{e}_{t}\right\rangle$
for third-order, but this will add a lot of computation.

####  2.2.3. MLP Layer

MLP Layer combines and transforms the features, e.g., $\boldsymbol{E}$ and
$l_{fm}$, with several fully-connected layers and activations. The output of
each layer is:

(5)
$\boldsymbol{h}_{l+1}=\operatorname{relu}\left(\boldsymbol{W}_{l}\boldsymbol{h}_{l}+\boldsymbol{b}_{l}\right)$

where $\boldsymbol{W}_{l}$ is the weight matrix and $\boldsymbol{b}_{l}$ is
the bias vector for the $l^{th}$ hidden layer. $\boldsymbol{h}_{0}$ is the
input of first fully-connected layer, and we denote the final output of MLP
layer as $\operatorname{MLP}(\boldsymbol{h}_{0})$.

####  2.2.4. Output Layer

Finally, the output layer, which is subsequent to the previous layers, will
generate the prediction $\boldsymbol{\hat{y}}$ of a user-item interaction data
example. The input $\boldsymbol{h}_{out}$ of output layer can be different in
different DRS models, e.g.,
$\boldsymbol{h}_{out}=[l_{fm}+\mathrm{MLP}(\boldsymbol{E})]$ in DeepFM (Guo et
al., 2017) and $\boldsymbol{h}_{out}=\mathrm{MLP}(l_{fm},\boldsymbol{E})$ in
IPNN (Qu et al., 2016), shown in Figure 2. The output layer will yield the
prediction $\boldsymbol{\hat{y}}$ of the user-item interaction as:

(6)
${\boldsymbol{\hat{y}}=\sigma\left(\boldsymbol{W}_{o}\boldsymbol{h}_{out}+\boldsymbol{b}_{o}\right)}$

where $\boldsymbol{W}_{o}$ and $\boldsymbol{b}_{o}$ are the weight matrix and
bias vector for the output layer. Activation function $\sigma(\cdot)$ is
selected based on different recommendation tasks, such as sigmoid for binary
classification (Guo et al., 2017), and softmax for multi-class classification
(Tan et al., 2016). Finally, given a set of candidate loss functions, such as
mean-squared-error, categorical hinge and cross-entropy, we can compute the
candidate losses $\mathcal{L_{C}}$:

(7)
$\mathcal{L_{C}}=[\ell_{1}(\boldsymbol{y},\boldsymbol{\hat{y}}),\ell_{2}(\boldsymbol{y},\boldsymbol{\hat{y}}),\cdots,\ell_{n}(\boldsymbol{y},\boldsymbol{\hat{y}})]$

where $\boldsymbol{y}$ is the ground truth label and $n$ is the number of
candidate loss functions.

###  2.3. Loss Function Search

AutoLoss aims to adaptively and automatically search the optimal loss
function, which can enhance the prediction quality and training efficiency of
the DRS network. This is naturally challenging because of the complex
relationship between the DRS parameters and the probabilities over candidate
loss functions. To address this challenge, many existing works have focused on
developing the fusing strategies for multiple loss functions, which can take
advantage of multiple loss functions in a weighted sum manner:

(8)
$\displaystyle\mathcal{L}(\boldsymbol{y},\boldsymbol{\hat{y}};\boldsymbol{\alpha})=\sum_{i=1}^{n}\alpha_{i}\cdot\ell_{i}(\boldsymbol{y},\boldsymbol{\hat{y}})$
s.t. $\displaystyle\sum_{i=1}^{n}\alpha_{i}=1,\quad\alpha_{i}>0\;\forall
i\in[1,n]$

where $\boldsymbol{y}$ is the ground truth, $\boldsymbol{\hat{y}}$ is the
prediction from DRS network, and $\ell_{i}$ is the $i^{th}$ candidate loss
function. The continuous loss weights
$\boldsymbol{\alpha}=[\alpha_{1},\alpha_{2},\cdots,\alpha_{n}]$ measure the
candidates’ contributions in the final loss function $\mathcal{L}$. However,
this method relies on exhaustively or manually search of loss weights from a
large search space, which is extremely costly. Also, this soft fusing strategy
cannot completely eliminate the impact of suboptimal candidate loss functions
on the final loss function $\mathcal{L}$, thus, a hard selection method is
desired. However, hard selection usually leads to the training framework not
end-to-end differentiable.

Reinforcement learning (RL) is a potential solution to tackle the hard
selection problem. However, since the RL is generally built upon the Markov
decision process, it utilizes temporal-difference to make sequential actions.
Consequently, the agent can only receive the reward until the optimal loss
function is selected and the DRS is evaluated. In other words, the temporal-
difference setting can suffer from delayed rewards. To address this issue, we
introduce the Gumbel-softmax operation to simulate the hard selection over
candidate loss functions, where the non-differentiable sampling is
approximated from a categorical distribution based on a differentiable
sampling from the Gumbel-softmax distribution (Jang et al., 2016).

Given the continuous loss weights $[\alpha_{1},\cdots,\alpha_{n}]$ over
candidate loss functions, we can draw a hard selection $z$ through the Gumbel-
max trick (Gumbel, 1948) as:

(9) $\displaystyle z=$
$\displaystyle\;\text{one\\_hot}\left(\arg\max_{i\in[1,n]}\left[\log\alpha_{i}+g_{i}\right]\right)$

where $g_{i}=-\log\left(-\log\left(u_{i}\right)\right)$ and $u_{i}\sim
Uniform(0,1)$. The independent and identically distributed (i.i.d) gumbel
noises $\\{g_{i}\\}$ disturb the $\\{\log\alpha_{i}\\}$ terms. Also, they make
the $\arg\max$ operation equivalent to drawing a sample from loss weights
$\alpha_{1},\cdots,\alpha_{n}$. However, because of the $\arg\max$ operation,
this sampling method is non-differentiable. We tackle this problem by
straight-through Gumbel-softmax (Jang et al., 2016), which leverages a softmax
function as a differentiable approximation to the $\arg\max$ operation:

(10)
$p_{i}=\frac{\exp\left(\left({\log\left(\alpha_{i}\right)+g_{i}}\right)/{\tau}\right)}{\sum_{j=1}^{n}\exp\left(\left({\log\left(\alpha_{j}\right)+g_{j}}\right)/{\tau}\right)},\quad\forall
i\in[1,n]$

where $p_{i}$ is the probability of selecting the $i^{th}$ candidate loss
function. The temperature parameter $\tau$ is introduced to manage the
smoothness of the Gumbel-softmax operation’s output. Specifically, the output
approaches a one-hot vector if $\tau$ is closer to zero. Then the final loss
function $\mathcal{L}$ can be reformulated as:

(11)
$\displaystyle\mathcal{L}(\boldsymbol{y},\boldsymbol{\hat{y}};\boldsymbol{p})=\sum_{i=1}^{n}p_{i}\cdot\ell_{i}(\boldsymbol{y},\boldsymbol{\hat{y}})$

In conclusion, the loss function search process becomes end-to-end
differentiable by introducing the Gumbel-softmax operation with a similar hard
selection performance. Next, we will discuss how to generate data example-
level loss weights $[\alpha_{1},\cdots,\alpha_{n}]$.

###  2.4. The Controller Network

As in Eq. (8), we suppose that $[\alpha_{1},\cdots,\alpha_{n}]$ are the
original (continuous) class probabilities over $n$ candidate loss functions
before the Gumbel-softmax operation. This assumption aims to learn a set of
unified and static probabilities over the candidate loss functions. However,
the environment of real-world commercial recommendation platforms is always
non-stationary, and different user-item interaction examples have varying
convergence behaviors. This cannot be handled by unified and static
probabilities, resulting in suboptimal model performance, generalizability and
transferability.

We propose a controller network to address this challenge, which learns to
generate original class probabilities for each data example. Motivated by
curriculum learning (Bengio et al., 2009; Jiang et al., 2014), the original
class probabilities should be generated according to the ground truth labels
$\boldsymbol{y}$ and the output of DRS network $\boldsymbol{\hat{y}}$.
Therefore, the input of the controller network is a mini-batch
$\boldsymbol{(y,\hat{y})}$, followed by the MLP layer with several fully-
connected layers like Eq. (5). Afterwards, the controller’s output layer
generates continuous class probabilities
$[\alpha_{1}^{b},\cdots,\alpha_{n}^{b}]\;\forall b\in[1,B]$ for each data
example in the mini-batch via a standard softmax activation, where $B$ is the
size of mini-batch. In other word, each data example has individual
probabilities. The controller can enhance the recommendation quality, model
generalizability and transferability, which is validated by the extensive
experiments.

###  2.5. An Optimization Method

In above subsections, we formulate the loss function search as an
architectural optimization problem and introduce the Gumbel-softmax that makes
the framework end-to-end differentiable. Now, we discuss the optimization for
the AutoLoss framework.

In AutoLoss, the parameters to be optimized are from two networks. We denote
the main DRS network’s parameters as $\boldsymbol{W}$, and the controller
network’s parameters as $\boldsymbol{V}$. Note that $\boldsymbol{p}$ are
directly generated by the Gumbel-softmax operation based on the controller’s
output $\boldsymbol{\alpha}$ as in Eq. (10). Inspired by automated machine
learning techniques (Pham et al., 2018), $\boldsymbol{W}$ and $\boldsymbol{V}$
should not be updated on the same training data batch like traditional
supervised learning methods. This is because the optimization of them is
highly dependent on each other. As a result, updating $\boldsymbol{W}$ and
$\boldsymbol{V}$ on the same training batch can lead to the model over-fitting
on the training examples.

Algorithm 1 An Optimization Algorithm for AutoLoss via DARTS.

Input: features $\boldsymbol{x}$ and ground-truth labels $\boldsymbol{y}$
Output: well-learned parameters $\boldsymbol{W}^{*}$ and $\boldsymbol{V}^{*}$

1: while not converged do

2: Sample a mini-batch of validation data examples

3: Estimate the approximation of $\boldsymbol{W}^{*}(\boldsymbol{V})$ via
Eq.(LABEL:equ:approximation)

4: Update $\boldsymbol{V}$ by descending
$\nabla_{\boldsymbol{V}}\;\mathcal{L}_{val}\big{(}\boldsymbol{W}^{*}(\boldsymbol{V}),\boldsymbol{V}\big{)}$

5: Sample a mini-batch of training data examples

6: Update $\boldsymbol{W}$ by descending
$\nabla_{\boldsymbol{W}}\mathcal{L}_{train}(\boldsymbol{W},\boldsymbol{V})$

7: end while

According to the end-to-end differentiable property of AutoLoss, we update
$\boldsymbol{W}$ and $\boldsymbol{V}$ through gradient descent utilizing the
differentiable architecture search (DARTS) techniques (Liu et al., 2018). To
be specific, $\boldsymbol{W}$ and $\boldsymbol{V}$ are alternately updated on
training and validation batches by minimizing the training loss
$\mathcal{L}_{train}$ and validation loss $\mathcal{L}_{val}$, respectively.
This forms a bi-level optimization problem (Pham et al., 2018), where
controller parameters $\boldsymbol{V}$ and DRS parameters $\boldsymbol{W}$ are
considered as the upper- and lower-level variables:

(12) $\displaystyle\min_{\boldsymbol{V}}$
$\displaystyle\mathcal{L}_{val}\big{(}\boldsymbol{W}^{*}(\boldsymbol{V}),\boldsymbol{V}\big{)}$
$\displaystyle s.t.$
$\displaystyle\boldsymbol{W}^{*}(\boldsymbol{V})=\arg\min_{\boldsymbol{W}}\mathcal{L}_{train}(\boldsymbol{W},\boldsymbol{V}^{*})$

where directly optimizing $\boldsymbol{V}$ thoroughly via Eq.(12) is
intractable since the inner optimization of $\boldsymbol{W}$ is extremely
costly. To tackle this issue, we use an approximation scheme for the inner
optimization:

(13)
$\displaystyle\boldsymbol{W}^{*}(\boldsymbol{V})\approx\boldsymbol{W}-\xi\nabla_{\boldsymbol{W}}\mathcal{L}_{train}(\boldsymbol{W},\boldsymbol{V})$

where $\xi$ is the predefined learning rate. This approximation scheme
estimates $\boldsymbol{W}^{*}(\boldsymbol{V})$ by descending only one step
toward the gradient
$\nabla_{\boldsymbol{W}}\mathcal{L}_{train}(\boldsymbol{W},\boldsymbol{V})$,
rather than optimizing $\boldsymbol{W}(\boldsymbol{V})$ thoroughly. To further
enhance the computation efficiency, we can set $\xi=0$, i.e., the first-order
approximation.

We detail the AutoLoss optimization via DARTS in Algorithm 1. More
specifically, in each iteration, we first sample a mini-batch validation data
examples of user-item interactions (line 2); next, we estimate (but do not
update) $\boldsymbol{W}^{*}(\boldsymbol{V})$ via the approximation scheme in
Eq.(LABEL:equ:approximation) (line 3); then, we update the controller
parameters $\boldsymbol{V}$ by one step based on the estimation (line 4);
afterward, we sample a mini-batch training data examples (line 5); and
finally, we update the $\boldsymbol{W}$ via descending
$\nabla_{\boldsymbol{W}}\mathcal{L}_{train}(\boldsymbol{W},\boldsymbol{V})$ by
one step (line 6).

##  3\. Experiment

This section will conduct extensive experiments using various datasets to
evaluate the effectiveness of AutoLoss. We first introduce the experimental
settings, then compare AutoLoss with representative baselines, and finally
conduct model component and transferability analysis.

###  3.1. Datasets

We evaluate our model on two datasets, including Criteo and ML-20m. Below we
introduce these datasets and more statistics about them can be found in Table
1.

Criteo222https://www.kaggle.com/c/criteo-display-ad-challenge/: It is a real-
world commercial dataset to assess click-through rate prediction models for
online ads. It consists of 45 million data examples, i.e., users’ click
records on displayed ads. Each example contains $m=39$ anonymous feature
fields, where 13 fields are numerical and 26 fields are categorical. 13
numerical fields are converted into categorical features through bucketing.

ML-20m333https://grouplens.org/datasets/movielens/20m/: This is a benchmark
dataset to evaluate recommendation algorithms, which contains 20 million
users’ 5-star ratings on movies. The dataset includes 27,278 movies and
138,493 users, i.e., $m=2$ feature fields, where each user has at least 20
ratings.

###  3.2. Evaluation Metrics

AutoLoss is general for many recommendation tasks. To evaluate its
effectiveness, we conduct binary classification (i.e., click-through rate
prediction) on Criteo, and multi-class classification (i.e., 5-star ratings)
on ML-20m. The two classification experiments are evaluated by AUC444We
evaluate the AUC for multiclass classification in a one-vs-rest manner. and
Logloss, where higher AUC or lower Logloss mean better performance. It is
worth noting that slightly higher AUC and lower Logloss at 0.001-level are
considered significant in recommendations (Guo et al., 2017).

Table 1. Statistics of the datasets. Data | Criteo | ML-20m
---|---|---
# Interactions | 45,840,617 | 20,000,263
# Feature Fields | 39 | 2
# Feature Values | 1,086,810 | 165,771
# Behavior | click or not | rating 1$\sim$5

###  3.3. Implementation

We implement AutoLoss based on a public
library555https://github.com/rixwew/pytorch-fm, which contains 16
representative recommendation models. We develop AutoLoss as an independent
class, so we can easily apply our framework for all these models. In this
paper, we only show the results on DeepFM (Guo et al., 2017) and IPNN (Qu et
al., 2016) due to the page limitation. To be specific, AutoLoss framework
mainly contains two networks, i.e., the DRS network and the controller
network.

Table 2. Performance comparison of different loss function search methods.

Dataset | Model | Metric | Methods
---|---|---|---
Focal | KL | Hinge | CE | MeLU | BOHB | DARTS | SLF | AutoLoss
Criteo | DeepFM | AUC $\uparrow$ | 0.8046 | 0.8042 | 0.8049 | 0.8056 | 0.8063 | 0.8065 | 0.8067 | 0.8081 | 0.8092*
Logloss $\downarrow$ | 0.4466 | 0.4469 | 0.4463 | 0.4457 | 0.4436 | 0.4435 | 0.4433 | 0.4426 | 0.4416*
Criteo | IPNN | AUC $\uparrow$ | 0.8077 | 0.8072 | 0.8079 | 0.8085 | 0.8090 | 0.8092 | 0.8093 | 0.8098 | 0.8108*
Logloss $\downarrow$ | 0.4435 | 0.4437 | 0.4432 | 0.4428 | 0.4423 | 0.4422 | 0.4423 | 0.4418 | 0.4409*
ML-20m | DeepFM | AUC $\uparrow$ | 0.7681 | 0.7682 | 0.7685 | 0.7692 | 0.7695 | 0.7695 | 0.7696 | 0.7705 | 0.7717*
Logloss $\downarrow$ | 1.2320 | 1.2317 | 1.2316 | 1.2310 | 1.2307 | 1.2305 | 1.2305 | 1.2299 | 1.2288*
ML-20m | IPNN | AUC $\uparrow$ | 0.7721 | 0.7722 | 0.7725 | 0.7733 | 0.7735 | 0.7734 | 0.7736 | 0.7745 | 0.7756*
Logloss $\downarrow$ | 1.2270 | 1.2269 | 1.2266 | 1.2260 | 1.2256 | 1.2257 | 1.2255 | 1.2249 | 1.2236*

“*” indicates the statistically significant improvements (i.e., two-sided
t-test with $p<0.05$) over the best baseline.
$\uparrow$: the higher the better; $\downarrow$: the lower the better.

For the DRS network, (a) Embedding layer: we set the embedding size as 16
following the existing works (Zhu et al., 2020). (b) Interaction layer: we
leverage factorization machine and inner product network to capture the
interactions among feature fields for DeepFM and IPNN, respectively. (c) MLP
layer: we have two fully-connected layers, and the layer size is 128. We also
employ batch normalization, dropout ($rate=0.2$) and ReLU activation for both
layers. (d) Output layer: original DeepFM and IPNN are designed for click-
through rate prediction, which use sigmoid activation for negative log-
likelihood function. To fit the 5-class classification task on ML-20m, we
modify the output layer correspondingly. i.e., the output layer is
5-dimensional with softmax activation.

For the controller network, (a) Input layer: the inputs are the ground truth
labels $\boldsymbol{y}$ and the predictions $\boldsymbol{\hat{y}}$ from DRS
network. (b) MLP layer: we also use two fully-connected layers with the layer
size 128, batch normalization, dropout ($rate=0.2$) and ReLU activation. (3)
Output layer: the controller network will output continuous loss probabilities
$\boldsymbol{\alpha}$ with softmax activation, whose dimension equals to the
number of candidate loss functions.

For other hyper-parameters, (a) Gumbel-softmax: we use an annealing scheme for
temperature $\tau=\max(0.01,1-0.00005\cdot t)$, where $t$ is the training
step. (b) Optimization: we set the learning rate as $0.001$ for updating both
DRS network and controller network with Adam optimizer and batch-size 2000.
(c) We select the hyper-parameters of the AutoLoss framework via cross-
validation, and we also do parameter-tuning for baselines correspondingly for
a fair comparison.

###  3.4. Overall Performance Comparison

AutoLoss is compared with the following loss function design and search
methods:

  * •

Fixed loss function: the first group of baselines leverages a predefined and
fixed loss function. We utilize Focal loss, KL divergence, Hinge loss and
cross-entropy (CE) loss for both classification tasks.

  * •

Fixed weights over loss functions: this group of baselines aims to learn fixed
weights over the loss functions in the first group, without considering the
difference among data examples. In this group, we use the meta-learning method
MeLU (Lee et al., 2019), as well as automated machine learning methods BOHB
(Falkner et al., 2018) and DARTS (Liu et al., 2018).

  * •

Data example-wise loss weights: this group learns to assign different loss
weights for different data examples according to their convergence behaviors.
One existing work, stochastic loss function (SLF) (Liu and Lai, 2020), belongs
to this group.

The overall performance is shown in Table 2. It can be observed that: (i) The
first group of baselines achieves the worst recommendation performance in both
recommendation tasks. Their optimizations are based on predefined and fixed
loss functions during the training stage. This result demonstrates that
leveraging a predefined and fixed loss function throughout can downgrade the
recommendation quality. (ii) The methods in the second group outperform those
in the first group. These methods try to learn weights over candidate loss
functions according to their contributions to the optimization, and then
combine them in a weighted sum manner. This validates that incorporating
multiple loss functions in optimization can enhance the performance of deep
recommender systems. (iii) The second group performs worse than the SLF, since
the weights they learned are unified and static, which completely overlooks
the various behaviors among different data examples. Therefore, SLF performs
better by taking this factor into account. (iv) The decision network of SLF is
optimized on the same training batch with the main DRS network via back-
propagation, which can lead to over-fitting on the training batch. AutoLoss
updates the DRS network on the training batch while updating the controller on
the validation batch, which improves the model generalizability and results in
better recommendation performance.

To summarize, AutoLoss achieves significantly better performance than state-
of-the-art baselines on both datasets and tasks, which demonstrates its
effectiveness.

###  3.5. Transferability Study

In this subsection, we study the transferability of the controller.
Specifically, we want to investigate (i) whether the controller trained with
one DRS model can be applied to other DRS models; and (ii) whether the
controller learned on one dataset can be directly used on other datasets.

Figure 3. Transferability study results.

To study the transferability across different DRS models, we leverage the
controller trained via DeepFM and AutoLoss on Criteo, fix its parameters and
apply it to train NFM (He and Chua, 2017) and AutoInt (Song et al., 2019) on
Criteo. The results are demonstrated in Figure 3 (a)-(d), where (i) “${CE}$”
means that we directly train the new DRS model via minimizing the cross-
entropy (CE) loss, which is the best single and fixed loss function in Table
2; (ii) “${SLF}$” is that we use the controller upon DeepFM and SLF, which is
the best baseline in Table 2; and (iii) “${AL}$” denotes that we use the
controller based on DeepFM and AutoLoss. From the figures, we can observe that
${SLF}$ performs superior to ${CE}$, which indicates that a pre-trained
controller can improve other DRS models’ training performance. More
importantly, ${AL}$ outperforms ${SLF}$, which validates AutoLoss’s better
transferability across different DRS models.

To study the transferability between different datasets, we train a controller
upon Criteo dataset with DeepFM and AutoLoss, fix its parameters and apply it
to train a new DeepFM on the Avazu dataset666Avazu is another benchmark
dataset for CTR prediction, which contains 40 million user clicking behaviors
in 11 days with $M=22$ categorical feature fields.
https://www.kaggle.com/c/avazu-ctr-prediction/, i.e., “${AL}$”. Also, we
denote that (i) “${CE}$”: DeepFM is directly optimized by minimizing cross-
entropy (CE) loss on Avazu dataset; and (ii) “${SLF}$”: DeepFM is optimized on
the new dataset with the assistance of a controller pre-trained with
DeepFM+SLF on Criteo. In Figure 3 (e)-(f), ${AL}$ shows superior performance
over ${CE}$ and ${SLF}$, which proves its better transferability between
different datasets.

In summary, AutoLoss has better transferability across different DRS models
and different recommendation datasets, which improves its usability in real-
world recommender systems.

###  3.6. Impact of Model Components

In this subsection, in order to understand the contributions of important
model components of AutoLoss, we systematically eliminate each component and
define the following variants:

  * •

AL-1: This variant aims to assess the contribution of the controller. Thus, we
assign equivalent weights on four candidate loss functions, i.e.,
$[0.25,0.25,0.25,0.25]$.

  * •

AL-2: In this variant, we eliminate the Gumbel-softmax operation, and directly
use the controller’s output, i.e., the continuous loss probabilities
$\boldsymbol{\alpha}$ from standard softmax activation, which aims to evaluate
the impact of Gumbel-softmax.

Table 3.  Impact of model components.

Dataset | Model | Metric | Methods
---|---|---|---
AL-1 | AL-2 | AutoLoss
Criteo | DeepFM | AUC $\uparrow$ | 0.8052 | 0.8083 | 0.8092*
Logloss $\downarrow$ | 0.4460 | 0.4422 | 0.4416*
Criteo | IPNN | AUC $\uparrow$ | 0.8081 | 0.8102 | 0.8108*
Logloss $\downarrow$ | 0.4431 | 0.4416 | 0.4409*

“*” indicates the statistically significant improvements (i.e., two-sided
t-test with $p<0.05$) over the best baseline.
$\uparrow$: the higher the better; $\downarrow$: the lower the better.

The results on the Criteo dataset are shown in Table 3. First, AL-1 has worse
performance than AutoLoss, which validates the necessity to introduce the
controller network. It is noteworthy that, AL-1 performs worse than all loss
function search methods, and even the fixed cross-entropy (CE) loss in Table
2, which indicates that equally incorporating all candidate loss functions
cannot guarantee better performance. Second, AutoLoss outperforms AL-2. The
main reason is that AL-2 always generates gradients based on all the loss
functions, which introduces some noisy gradients from the suboptimal candidate
loss functions. In contrast, AutoLoss can obtain appropriate gradients by
filtering out suboptimal loss functions via Gumbel-softmax, which enhances the
model robustness.

Figure 4. Efficiency study results.

###  3.7. Efficiency Study

This section compares AutoLoss’s training efficiency with other loss function
searching methods, which is an important metric to deploy a DRS model in real-
world applications. Our experiments are based on one GeForce GTX 1060 GPU.

The results of DeepFM on Criteo dataset are illustrated in Figure 4 (a). We
can observe that AutoLoss achieves the fastest training speed. The reasons are
two-fold. First, AutoLoss can generate the most appropriate gradients to
update DRS, which increases the optimization efficiency. Second, we update the
controller once after every 7 times DRS is updated, i.e., the controller
updating frequency $f=7$. This trick not only reduces the training time ($\sim
60\%$ in this case) with fewer computations, but also enhances the
performance. In Figure 4 (b)-(c) where $x$-axis is $f$, we find that DeepFM
performs the best when $f=7$, while updating too frequently/infrequently can
lead to suboptimal AUC/Logloss.

To summarize, AutoLoss can efficiently achieve better performance, making it
easier to be launched in real-world recommender systems.

##  4\. Related Work

In this section, we briefly introduce the works related to our study. We first
go over the latest studies in loss function search and then review works about
AutoML for recommendations.

###  4.1. Loss Function Search

The loss function plays an essential part in a deep learning framework. The
choice of the loss function significantly affects the performance of the
learned model. A lot of efforts have been made to design desirable loss
functions for specific tasks. For example, in the field of image processing,
Rahman and Wang (2016) argued that the typical cross-entropy loss for semantic
segmentation shows great limitations in aligning with evaluation metrics other
than global accuracy. Ronneberger et al. (2015); Wu et al. (2016b) designed
loss functions by taking class frequency into consideration to cater to the
mIoU metric. Caliva et al. (2019); Qin et al. (2019) designed losses with
larger weights at boundary regions to improve the boundary F1 score. Liu et
al. (2016) proposed to replace the traditional Softmax loss with large margin
Softmax (L-Softmax) loss to improve feature discrimination in classification
tasks. Fan et al. (2019) used sphere Softmax loss for the person re-
identification task and obtained state-of-the-art results. The loss functions
mentioned above are all designed manually, requiring ample expert knowledge,
non-trivial time, and many human efforts.

Recently, automated loss function search draws increasing interests of
researchers from various machine learning (ML) fields. Xu et al. (2018)
investigated how to automatically schedule iterative and alternate
optimization processes for ML models. Liu and Lai (2020) proposed to optimize
the stochastic loss function (SLF), where the loss function of an ML model was
dynamically selected. During training, model parameters and the loss
parameters are learned jointly. Li et al. (2020) proposed automatically
searching specific surrogate losses to improve different evaluation metrics in
the image semantic segmentation task. Jin et al. (2021) composed multiple
self-supervised learning tasks to jointly encode multiple sources of
information and produce more generalizable representations, and developed two
automated frameworks to search the task weights. Besides, Li et al. (2019);
Wang et al. (2020) designed search spaces for a series of existing loss
functions and developed algorithms to search for the best parameters of the
probability distribution for sampling loss functions.

###  4.2. AutoML for Recommendation

AutoML techniques are now widely used to automatically design deep
recommendation systems. Previous works mainly focused on the design of the
embedding layer and the selection of feature interaction patterns.

In terms of the embedding layer, Joglekar et al. (2020); Zhao et al. (2020a);
Liu et al. (2021) proposed novel methods to automatically select the best
embedding size for different feature fields in a recommendation system. Zhao
et al. (2020b); Liu et al. (2020a) proposed to dynamically search embedding
sizes for users and items based on their popularity in the streaming setting.
Similarly, Ginart et al. (2019) proposed to use mixed dimension embeddings for
users and items based on their query frequency. Kang et al. (2020) proposed a
multi-granular quantized embeddings (MGQE) technique to learn impact
embeddings for infrequent items. Cheng et al. (2020) proposed to perform
embedding dimension selection with a soft selection layer, making the
dimension selection more flexible. Guo et al. (2020) focused on the embeddings
of numerical features. They proposed AutoDis, which automatically discretizes
features in numerical fields and maps the resulting categorical features into
embeddings.

As for feature interaction, Luo et al. (2019) proposed AutoCross that produces
high-order cross features by performing beam search in a tree-structure
feature space. Song et al. (2020); Khawar et al. (2020); Liu et al. (2020b);
Xue et al. (2020) proposed to automatically discover feature interaction
architectures for click-through rate (CTR) prediction. Tsang et al. (2020)
proposed a method to interpret the feature interactions from a source
recommendation model and apply them in a target recommendation model.

To the best of our knowledge, we are the first to investigate the automated
loss function search for deep recommendation systems.

##  5\. Conclusion

We propose a novel end-to-end framework, AutoLoss, to enhance recommendation
performance and deep recommender systems’ training efficiency by selecting
appropriate loss functions in a data-driven manner. AutoLoss can automatically
select the proper loss function for each data example according to their
varied convergence behaviors. To be specific, we first develop a novel
controller network, which generates continuous loss weights based on the
ground truth labels and the DRS’ predictions. Then, we introduce a Gumbel-
softmax operation to simulate the hard selection over candidate loss
functions, which filters out the noisy gradients from suboptimal candidates.
Finally, we can select the optimal candidate according to the output from
Gumbel-softmax. We conduct extensive experiments to validate the effectiveness
of AutoLoss on two widely used benchmark datasets. The results show that our
framework can improve recommendation performance and training efficiency with
excellent transferability.

## ACKNOWLEDGEMENTS

This work is supported by National Science Foundation (NSF) under grant
numbers IIS1907704, IIS1928278, IIS1714741, IIS1715940, IIS1845081,
CNS1815636, and an internal research fund from the Hong Kong Polytechnic
University (project no. P0036200).

## References

  * (1)
  * Bengio et al. (2009) Yoshua Bengio, Jérôme Louradour, Ronan Collobert, and Jason Weston. 2009\.  Curriculum learning. In _Proceedings of the 26th annual international conference on machine learning_. 41–48.
  * Caliva et al. (2019) Francesco Caliva, Claudia Iriondo, Alejandro Morales Martinez, Sharmila Majumdar, and Valentina Pedoia. 2019.  Distance map loss penalty term for semantic segmentation.  _arXiv preprint arXiv:1908.03679_ (2019).
  * Chatterjee and Hadi (2015) Samprit Chatterjee and Ali S Hadi. 2015.  _Regression analysis by example_.  John Wiley & Sons.
  * Cheng et al. (2020) Weiyu Cheng, Yanyan Shen, and Linpeng Huang. 2020.  Differentiable Neural Input Search for Recommender Systems.  _arXiv preprint arXiv:2006.04466_ (2020).
  * Covington et al. (2016) Paul Covington, Jay Adams, and Emre Sargin. 2016\.  Deep neural networks for youtube recommendations. In _Proceedings of the 10th ACM conference on recommender systems_. 191–198.
  * Fahrmeir et al. (2007) Ludwig Fahrmeir, Thomas Kneib, Stefan Lang, and Brian Marx. 2007\.  _Regression_.  Springer.
  * Falkner et al. (2018) Stefan Falkner, Aaron Klein, and Frank Hutter. 2018\.  BOHB: Robust and efficient hyperparameter optimization at scale. In _International Conference on Machine Learning_. PMLR, 1437–1446.
  * Fan et al. (2020) Wenqi Fan, Tyler Derr, Xiangyu Zhao, Yao Ma, Hui Liu, Jianping Wang, Jiliang Tang, and Qing Li. 2020\.  Attacking Black-box Recommendations via Copying Cross-domain User Profiles.  _arXiv preprint arXiv:2005.08147_ (2020).
  * Fan et al. (2019) Xing Fan, Wei Jiang, Hao Luo, and Mengjuan Fei. 2019\.  Spherereid: Deep hypersphere manifold embedding for person re-identification.  _Journal of Visual Communication and Image Representation_ 60 (2019), 51–58.
  * Gao et al. (2020) Weihao Gao, Xiangjun Fan, Jiankai Sun, Kai Jia, Wenzhi Xiao, Chong Wang, and Xiaobing Liu. 2020.  Deep Retrieval: An End-to-End Learnable Structure Model for Large-Scale Recommendations.  _arXiv preprint arXiv:2007.07203_ (2020).
  * Ge et al. (2021) Yingqiang Ge, Shuchang Liu, Ruoyuan Gao, Yikun Xian, Yunqi Li, Xiangyu Zhao, Changhua Pei, Fei Sun, Junfeng Ge, Wenwu Ou, et al. 2021\.  Towards Long-term Fairness in Recommendation.  _arXiv preprint arXiv:2101.03584_ (2021).
  * Ginart et al. (2019) Antonio Ginart, Maxim Naumov, Dheevatsa Mudigere, Jiyan Yang, and James Zou. 2019.  Mixed Dimension Embeddings with Application to Memory-Efficient Recommendation Systems.  _arXiv preprint arXiv:1909.11810_ (2019).
  * Gumbel (1948) Emil Julius Gumbel. 1948\.  _Statistical theory of extreme values and some practical applications: a series of lectures_. Vol. 33.  US Government Printing Office.
  * Guo et al. (2020) Huifeng Guo, Bo Chen, Ruiming Tang, Zhenguo Li, and Xiuqiang He. 2020.  AutoDis: Automatic Discretization for Embedding Numerical Features in CTR Prediction.  _arXiv preprint arXiv:2012.08986_ (2020).
  * Guo et al. (2016) Hao Guo, Xin Li, Ming He, Xiangyu Zhao, Guiquan Liu, and Guandong Xu. 2016\.  CoSoLoRec: Joint Factor Model with Content, Social, Location for Heterogeneous Point-of-Interest Recommendation. In _International Conference on Knowledge Science, Engineering and Management_. Springer, 613–627.
  * Guo et al. (2017) Huifeng Guo, Ruiming Tang, Yunming Ye, Zhenguo Li, and Xiuqiang He. 2017.  DeepFM: a factorization-machine based neural network for CTR prediction. In _Proceedings of the 26th International Joint Conference on Artificial Intelligence_. 1725–1731.
  * He and Chua (2017) Xiangnan He and Tat-Seng Chua. 2017.  Neural factorization machines for sparse predictive analytics. In _Proceedings of the 40th International ACM SIGIR conference on Research and Development in Information Retrieval_. 355–364.
  * Jang et al. (2016) Eric Jang, Shixiang Gu, and Ben Poole. 2016.  Categorical reparameterization with gumbel-softmax.  _arXiv preprint arXiv:1611.01144_ (2016).
  * Jiang et al. (2014) Lu Jiang, Deyu Meng, Shoou-I Yu, Zhenzhong Lan, Shiguang Shan, and Alexander Hauptmann. 2014.  Self-paced learning with diversity. In _Advances in Neural Information Processing Systems_. 2078–2086.
  * Jin et al. (2021) Wei Jin, Xiaorui Liu, Xiangyu Zhao, Yao Ma, Neil Shah, and Jiliang Tang. 2021\.  Automated Self-Supervised Learning for Graphs.  arXiv:2106.05470 [cs.LG]
  * Joglekar et al. (2020) Manas R Joglekar, Cong Li, Mei Chen, Taibai Xu, Xiaoming Wang, Jay K Adams, Pranav Khaitan, Jiahui Liu, and Quoc V Le. 2020.  Neural input search for large scale recommendation models. In _Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining_. 2387–2397.
  * Kang et al. (2020) Wang-Cheng Kang, Derek Zhiyuan Cheng, Ting Chen, Xinyang Yi, Dong Lin, Lichan Hong, and Ed H Chi. 2020\.  Learning Multi-granular Quantized Embeddings for Large-Vocab Categorical Features in Recommender Systems.  _arXiv preprint arXiv:2002.08530_ (2020).
  * Khawar et al. (2020) Farhan Khawar, Xu Hang, Ruiming Tang, Bin Liu, Zhenguo Li, and Xiuqiang He. 2020\.  AutoFeature: Searching for Feature Interactions and Their Architectures for Click-through Rate Prediction. In _Proceedings of the 29th ACM International Conference on Information & Knowledge Management_. 625–634.
  * Kirillov et al. (2019) Alexander Kirillov, Ross Girshick, Kaiming He, and Piotr Dollár. 2019.  Panoptic feature pyramid networks. In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_. 6399–6408.
  * Lee et al. (2019) Hoyeop Lee, Jinbae Im, Seongwon Jang, Hyunsouk Cho, and Sehee Chung. 2019.  MeLU: meta-learned user preference estimator for cold-start recommendation. In _Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining_. 1073–1082.
  * Li et al. (2019) Chuming Li, Xin Yuan, Chen Lin, Minghao Guo, Wei Wu, Junjie Yan, and Wanli Ouyang. 2019.  Am-lfs: Automl for loss function search. In _Proceedings of the IEEE/CVF International Conference on Computer Vision_. 8410–8419.
  * Li et al. (2020) Hao Li, Chenxin Tao, Xizhou Zhu, Xiaogang Wang, Gao Huang, and Jifeng Dai. 2020\.  Auto Seg-Loss: Searching Metric Surrogates for Semantic Segmentation.  _arXiv preprint arXiv:2010.07930_ (2020).
  * Lian et al. (2018) Jianxun Lian, Xiaohuan Zhou, Fuzheng Zhang, Zhongxia Chen, Xing Xie, and Guangzhong Sun. 2018\.  xdeepfm: Combining explicit and implicit feature interactions for recommender systems. In _Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining_.
  * Liu et al. (2020b) Bin Liu, Chenxu Zhu, Guilin Li, Weinan Zhang, Jincai Lai, Ruiming Tang, Xiuqiang He, Zhenguo Li, and Yong Yu. 2020b.  AutoFIS: Automatic Feature Interaction Selection in Factorization Models for Click-Through Rate Prediction.  _arXiv preprint arXiv:2003.11235_ (2020).
  * Liu et al. (2018) Hanxiao Liu, Karen Simonyan, and Yiming Yang. 2018\.  Darts: Differentiable architecture search.  _arXiv preprint arXiv:1806.09055_ (2018).
  * Liu et al. (2020a) Haochen Liu, Xiangyu Zhao, Chong Wang, Xiaobing Liu, and Jiliang Tang. 2020a.  Automated Embedding Size Search in Deep Recommender Systems. In _Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval_. 2307–2316.
  * Liu and Lai (2020) Qingliang Liu and Jinmei Lai. 2020.  Stochastic Loss Function. In _Proceedings of the AAAI Conference on Artificial Intelligence_ , Vol. 34. 4884–4891.
  * Liu et al. (2021) Siyi Liu, Chen Gao, Yihong Chen, Depeng Jin, and Yong Li. 2021.  Learnable Embedding Sizes for Recommender Systems.  _arXiv preprint arXiv:2101.07577_ (2021).
  * Liu et al. (2016) Weiyang Liu, Yandong Wen, Zhiding Yu, and Meng Yang. 2016\.  Large-margin softmax loss for convolutional neural networks.. In _ICML_ , Vol. 2. 7\.
  * Liu et al. (2017) Yiding Liu, Tuan-Anh Nguyen Pham, Gao Cong, and Quan Yuan. 2017.  An experimental evaluation of point-of-interest recommendation in location-based social networks.  _Proceedings of the VLDB Endowment_ 10, 10 (2017), 1010–1021.
  * Luo et al. (2019) Yuanfei Luo, Mengshuo Wang, Hao Zhou, Quanming Yao, Wei-Wei Tu, Yuqiang Chen, Wenyuan Dai, and Qiang Yang. 2019\.  Autocross: Automatic feature crossing for tabular data in real-world applications. In _Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining_. 1936–1945.
  * Nguyen et al. (2017) Hanh TH Nguyen, Martin Wistuba, Josif Grabocka, Lucas Rego Drumond, and Lars Schmidt-Thieme. 2017\.  Personalized Deep Learning for Tag Recommendation. In _Pacific-Asia Conference on Knowledge Discovery and Data Mining_. Springer.
  * Pham et al. (2018) Hieu Pham, Melody Guan, Barret Zoph, Quoc Le, and Jeff Dean. 2018.  Efficient Neural Architecture Search via Parameters Sharing. In _International Conference on Machine Learning_. 4095–4104.
  * Qin et al. (2019) Xuebin Qin, Zichen Zhang, Chenyang Huang, Chao Gao, Masood Dehghan, and Martin Jagersand. 2019\.  Basnet: Boundary-aware salient object detection. In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_. 7479–7489.
  * Qu et al. (2016) Yanru Qu, Han Cai, Kan Ren, Weinan Zhang, Yong Yu, Ying Wen, and Jun Wang. 2016.  Product-based neural networks for user response prediction. In _2016 IEEE 16th International Conference on Data Mining (ICDM)_. IEEE, 1149–1154.
  * Rahman and Wang (2016) Md Atiqur Rahman and Yang Wang. 2016.  Optimizing intersection-over-union in deep neural networks for image segmentation. In _International symposium on visual computing_. Springer, 234–244.
  * Ravi and Vairavasundaram (2016) Logesh Ravi and Subramaniyaswamy Vairavasundaram. 2016\.  A collaborative location based travel recommendation system through enhanced rating prediction for the group of users.  _Computational intelligence and neuroscience_ 2016 (2016).
  * Rendle (2010) Steffen Rendle. 2010\.  Factorization machines. In _Data Mining (ICDM), 2010 IEEE 10th International Conference on_. IEEE, 995–1000.
  * Ronneberger et al. (2015) Olaf Ronneberger, Philipp Fischer, and Thomas Brox. 2015\.  U-net: Convolutional networks for biomedical image segmentation. In _International Conference on Medical image computing and computer-assisted intervention_. Springer, 234–241.
  * Song et al. (2020) Qingquan Song, Dehua Cheng, Hanning Zhou, Jiyan Yang, Yuandong Tian, and Xia Hu. 2020\.  Towards automated neural interaction discovery for click-through rate prediction. In _Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining_. 945–955.
  * Song et al. (2019) Weiping Song, Chence Shi, Zhiping Xiao, Zhijian Duan, Yewen Xu, Ming Zhang, and Jian Tang. 2019.  Autoint: Automatic feature interaction learning via self-attentive neural networks. In _Proceedings of the 28th ACM International Conference on Information and Knowledge Management_. 1161–1170.
  * Tan et al. (2016) Yong Kiam Tan, Xinxing Xu, and Yong Liu. 2016.  Improved recurrent neural networks for session-based recommendations. In _Proceedings of the 1st Workshop on Deep Learning for Recommender Systems_. 17–22.
  * Tsang et al. (2020) Michael Tsang, Dehua Cheng, Hanpeng Liu, Xue Feng, Eric Zhou, and Yan Liu. 2020\.  Feature interaction interpretability: A case for explaining ad-recommendation systems via neural interaction detection.  _arXiv preprint arXiv:2006.10966_ (2020).
  * Wang et al. (2020) Xiaobo Wang, Shuo Wang, Cheng Chi, Shifeng Zhang, and Tao Mei. 2020.  Loss function search for face recognition. In _International Conference on Machine Learning_. PMLR, 10029–10038.
  * Wu et al. (2016a) Sai Wu, Weichao Ren, Chengchao Yu, Gang Chen, Dongxiang Zhang, and Jingbo Zhu. 2016a.  Personal recommendation using deep recurrent neural networks in NetEase. In _Data Engineering (ICDE), 2016 IEEE 32nd International Conference on_. IEEE, 1218–1229.
  * Wu et al. (2016b) Zifeng Wu, Chunhua Shen, and Anton van den Hengel. 2016b.  Bridging category-level and instance-level semantic image segmentation.  _arXiv preprint arXiv:1605.06885_ (2016).
  * Xiong et al. (2019) Yuwen Xiong, Renjie Liao, Hengshuang Zhao, Rui Hu, Min Bai, Ersin Yumer, and Raquel Urtasun. 2019.  Upsnet: A unified panoptic segmentation network. In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_. 8818–8826.
  * Xu et al. (2018) Haowen Xu, Hao Zhang, Zhiting Hu, Xiaodan Liang, Ruslan Salakhutdinov, and Eric Xing. 2018\.  Autoloss: Learning discrete schedules for alternate optimization.  _arXiv preprint arXiv:1810.02442_ (2018).
  * Xue et al. (2020) Niannan Xue, Bin Liu, Huifeng Guo, Ruiming Tang, Fengwei Zhou, Stefanos P Zafeiriou, Yuzhou Zhang, Jun Wang, and Zhenguo Li. 2020.  AutoHash: Learning Higher-order Feature Interactions for Deep CTR Prediction.  _IEEE Transactions on Knowledge and Data Engineering_ (2020).
  * Zhang et al. (2019) Shuai Zhang, Lina Yao, Aixin Sun, and Yi Tay. 2019\.  Deep learning based recommender system: A survey and new perspectives.  _ACM Computing Surveys (CSUR)_ 52, 1 (2019), 1–38.
  * Zhao et al. (2021) Xiangyu Zhao, Changsheng Gu, Haoshenglun Zhang, Xiwang Yang, Xiaobing Liu, Hui Liu, and Jiliang Tang. 2021.  DEAR: Deep Reinforcement Learning for Online Advertising Impression in Recommender Systems. In _Proceedings of the AAAI Conference on Artificial Intelligence_ , Vol. 35. 750–758.
  * Zhao et al. (2020a) Xiangyu Zhao, Haochen Liu, Hui Liu, Jiliang Tang, Weiwei Guo, Jun Shi, Sida Wang, Huiji Gao, and Bo Long. 2020a.  Memory-efficient Embedding for Recommendations.  _arXiv preprint arXiv:2006.14827_ (2020).
  * Zhao et al. (2020b) Xiangyu Zhao, Chong Wang, Ming Chen, Xudong Zheng, Xiaobing Liu, and Jiliang Tang. 2020b.  AutoEmb: Automated Embedding Dimensionality Search in Streaming Recommendations.  _arXiv preprint arXiv:2002.11252_ (2020).
  * Zhao et al. (2019) Xiangyu Zhao, Long Xia, Zhuoye Ding, Dawei Yin, and Jiliang Tang. 2019.  Toward Simulating Environments in Reinforcement Learning Based Recommendations.  _arXiv preprint arXiv:1906.11462_ (2019).
  * Zhao et al. (2018a) Xiangyu Zhao, Long Xia, Liang Zhang, Zhuoye Ding, Dawei Yin, and Jiliang Tang. 2018a.  Deep Reinforcement Learning for Page-wise Recommendations. In _Proceedings of the 12th ACM Recommender Systems Conference_. ACM, 95–103.
  * Zhao et al. (2020c) Xiangyu Zhao, Long Xia, Lixin Zou, Hui Liu, Dawei Yin, and Jiliang Tang. 2020c.  Whole-Chain Recommendations. In _Proceedings of the 29th ACM International Conference on Information & Knowledge Management_. 1883–1891.
  * Zhao et al. (2016) Xiangyu Zhao, Tong Xu, Qi Liu, and Hao Guo. 2016\.  Exploring the Choice Under Conflict for Social Event Participation. In _International Conference on Database Systems for Advanced Applications_. Springer, 396–411.
  * Zhao et al. (2018b) Xiangyu Zhao, Liang Zhang, Zhuoye Ding, Long Xia, Jiliang Tang, and Dawei Yin. 2018b.  Recommendations with Negative Feedback via Pairwise Deep Reinforcement Learning. In _Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining_. ACM, 1040–1048.
  * Zhao et al. (2020d) Xiangyu Zhao, Xudong Zheng, Xiwang Yang, Xiaobing Liu, and Jiliang Tang. 2020d.  Jointly learning to recommend and advertise. In _Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining_. 3319–3327.
  * Zhu et al. (2020) Jieming Zhu, Jinyang Liu, Shuai Yang, Qi Zhang, and Xiuqiang He. 2020.  FuxiCTR: An Open Benchmark for Click-Through Rate Prediction.  _arXiv preprint arXiv:2009.05794_ (2020).
  * Zou et al. (2020) Lixin Zou, Long Xia, Yulong Gu, Xiangyu Zhao, Weidong Liu, Jimmy Xiangji Huang, and Dawei Yin. 2020.  Neural Interactive Collaborative Filtering. In _Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval_. 749–758.


# Quantum Computing for Climate Resilience and Sustainability Challenges
††thanks: Corresponding author<EMAIL_ADDRESS>
Kin Tung Michael Ho  Department of Civil Engineering
Imperial College London
London, UK
<EMAIL_ADDRESS>Kuan-Cheng Chen  Department of Materials
Imperial College London
London, UK
<EMAIL_ADDRESS>Lily Lee  Department of Civil Engineering
Imperial College London
London, UK
<EMAIL_ADDRESS>Felix Burt  Department of EEE
Imperial College London
London, UK
<EMAIL_ADDRESS>Shang Yu  Department of Physics
Imperial College London
London, UK
<EMAIL_ADDRESS>Po-Heng (Henry) Lee  Department of Civil Engineering
Imperial College London
London, UK
<EMAIL_ADDRESS>
###### Abstract

The escalating impacts of climate change and the increasing demand for
sustainable development and natural resource management necessitate innovative
technological solutions. Quantum computing (QC) has emerged as a promising
tool with the potential to revolutionize these critical areas. This review
explores the application of quantum machine learning and optimization
techniques for climate change prediction and enhancing sustainable
development. Traditional computational methods often fall short in handling
the scale and complexity of climate models and natural resource management.
Quantum advancements, however, offer significant improvements in computational
efficiency and problem-solving capabilities. By synthesizing the latest
research and developments, this paper highlights how QC and quantum machine
learning can optimize multi-infrastructure systems towards climate neutrality.
The paper also evaluates the performance of current quantum algorithms and
hardware in practical applications and presents realistic cases, i.e., waste-
to-energy in anaerobic digestion, disaster prevention in flooding prediction,
and new material development for carbon capture. The integration of these
quantum technologies promises to drive significant advancements in achieving
climate resilience and sustainable development.

###### Index Terms:

Quantum Computing, Quantum Machine Learning, Optimization, Climate Change,
Renewable Energy, Water Engineering

##  I Introduction

The increase of greenhouse gas emissions and subsequent rise in global
temperatures has incurred severe environmental and socio-economic
consequences. Both mitigation and adaptation strategies are required to
achieve net-zero emissions. Consequently, Global Climate Models (GCMs) have
become a crucial tool in understanding and predicting the impacts of a
changing environment [1]. These models simulate the interactions between the
atmosphere, oceans, land surface, and ice using mathematical representations
to forecast future scenarios based on different greenhouse gas emission
pathways.

However, GCMs face substantial challenges that limit their dynamic
predictability [1]. One critical issue is the need to accurately simulate the
complex mosaic of physical, chemical, and biological processes that influence
climate over long periods. Many of these processes are either absent or poorly
represented in current models, which results in various biases. Additionally,
classical computing capacity is constrained at lower resolutions, failing to
capture critical regional phenomena such as cloud formation. Using higher-
resolution models through downscaling often introduces boundary interactions
that propagate errors. The wide range of projected climate responses to
increased CO2 highlights significant errors and limitations in these models.
Furthermore, GCMs often inadequately account for multiplier effects that can
amplify initial climatic impacts, such as those related to solar variations.
Despite some improvements from Coupled Model Intercomparison Project 3 (CMIP3)
to CMIP5 models, little advancements were made to improve the accuracy of
critical climate parameters, highlighting ongoing challenges in climate
modeling.

Figure 1: Overview of Quantum Computing applications in climate change
mitigation.

Machine Learning (ML) and artificial intelligence (AI) are increasingly being
applied to various aspects of weather and climate modeling. However, research
on the use of ML and AI using classical computers is still limited compared to
its use for nowcasting and weather prediction. This gap is because the
available observation period for the earth system is too short to provide
enough training data for most applications [2]. Furthermore, integrating ML
into climate models, and more generally natural resource management models,
introduces challenges such as addressing instabilities in physical models,
increasing the ease of technical integration, and managing memory and
computational concerns [2]. In order to fully realize the impact of ML and AI
for enhancing climate predictions and climate-resilience systems, such
significant challenges must be addressed alongside the limitations of
traditional GCMs.

Quantum computing (QC), with its enhanced computing power, offers new tools
and methodologies to tackle climate challenges more effectively [3]. QC is
characterized by its potential to solve problems exponentially faster than
classical computers[4]. The development of scalable quantum hardware requires
the development of quantum algorithms to fully realize the impact of QC for
real-world applications [5]. Limited studies have been conducted on utilizing
QC to either model climate change and better understand future impacts, or
model processes that could improve current natural resource management systems
in the face of increasing instability from climate change.

In this paper, we review the current state-of-the-art in utilizing QC to
address challenges in optimization, ML, and simulations, with a particular
emphasis on its potential to enhance climate modeling and mitigation
strategies for climate change (see Fig. 1). Section 2 delves into the
applications of Quantum Optimization in sustainable water treatment and
biomass management. Section 3 examines the role of Quantum Machine Learning
(QML) in disaster prevention, addressing critical issues such as flooding,
heatwaves, and storms. Section 4 investigates the contributions of Quantum
Simulation to materials science engineering, focusing on advancements in
carbon capture.

##  II Quantum Computing for Optimization

###  II-A Algorithm Review

QC holds significant promise for solving complex optimization problems, which
are integral to addressing climate change. Among the notable quantum
approaches to optimization are quantum annealing (QA) and gate-based quantum
algorithms. Quantum annealers, such as those developed by D-Wave Systems, have
shown promise in various optimization tasks. It was reported that the Quantum
Annealing Continuous Optimization algorithm in [6] was able to optimize
biomethane production in anaerobic digestion for the largest biogas producer
in Europe, showing advantages over classical computing.

Quantum annealers are specialized devices designed to solve optimization
problems by finding the lowest energy state of a physical system, known as the
ground state [7]. Problems are encoded using the Ising model, where qubits and
their interactions form an energy landscape. The annealing process involves
gradually reducing quantum fluctuations to guide the system to the ground
state, leveraging the adiabatic theorem. This approach allows quantum
annealers to tackle complex combinatorial optimization problems more
efficiently than classical methods.

However, while QA excels in specific applications, it is less versatile than
gate-based quantum algorithms. Gate-based quantum algorithms, particularly the
Quantum Approximate Optimization Algorithm (QAOA), offer a more flexible and
powerful framework for tackling a broader range of optimization problems[8].
Unlike quantum annealers, which are specialized for certain tasks, QAOA can be
adapted to various problem structures by adjusting its quantum circuit
parameters, making it advantageous for complex, multi-objective optimization
challenges.

As a hybrid, iterative method, QAOA is designed to solve combinatorial
optimization problems. A notable example is the Max-Cut graph problem, which
aims to maximize the number of edges between two sets by a partition of the
vertices. As an NP-complete graph, reducing the complexity of Max-Cut would
help improve the complexity for all problems in NP[8, 9, 10]. QAOA works by
alternating between two types of operations: one that encodes the problem’s
constraints (the cost Hamiltonian), and one that induces quantum superposition
and entanglement (the mixing Hamiltonian). The cost Hamiltonian encodes the
objective function of the problem formulated as a ground state corresponds to
the optimal solution of the problem. For instance, in the Max-Cut problem, the
cost Hamiltonian assigns higher energy to configurations with fewer edges
between the two sets, thus incentivizing cuts that maximize the number of
edges between them. The mixing Hamiltonian, on the other hand, introduces
quantum superposition and entanglement into the system. This enables
transitions between different states to prevent the algorithm from getting
trapped in a local minimum. By applying these Hamiltonians in alternation,
QAOA navigates the solution space more effectively than classical algorithms.

While research into the application of QAOA and other quantum optimizers in
water resources management is still limited, there is considerable potential
for future research in this domain with substantial impact. We highlight
several areas below.

####  II-A1 Hydraulic Engineering

Quantum computers can simulate fluid dynamics using the hydrodynamic
Schrödinger equation, which could be used to model three-dimensional turbulent
flows in hydraulic engineering to understand and predict fluid dynamics [11].
There are also existing works on using Hamiltonian Simulation to solve the
advection equation [12] and various QC methods aimed at approximating Navier-
Stokes solutions[13, 14, 15].

####  II-A2 Water Supply Management

QC can help optimize logistics and supply chain management, which can be
applied to water supply management. For example, QAOA can enhance the
efficiency of delivery routes, schedule fleet operations, or manage inventory
levels more effectively [16]. This can lead to more efficient water
distribution, minimizing costs and delivery times.

####  II-A3 Wastewater and Waste Treatment Modeling

Wastewater and waste treatment processes involve numerous parameters that need
to be optimized for efficient operation, such as temperature, pH, nutrient
concentration, and stirring speed. QAOA can be used to find the optimal
combination of these parameters that maximizes the treatment efficiency of the
bioreactor.

###  II-B Use Case: Biogas Production in Anaerobic Digestion

Although QC offers significant advantages on optimizations, there is a notable
gap in their application to the fields of water engineering and climate
modelling. Limited research has been conducted on the use of either quantum
annealers or QAOA for these areas. To the best of our knowledge, [6] is one of
the few to investigate the use of QC for optimizing the biomass mix-ratio
selection for waste-to-energy in anaerobic digestion. Their work introduces
the innovative Quantum Annealing Continuous Optimization (QuAnCO) method to
tackle non-discrete optimization problems. QuAnCO utilizes quantum annealing
to address the continuous nature of decision spaces in many real-world
optimization tasks, such as those related to renewable energy, that
traditional QA struggles to handle directly.

The QuAnCO algorithm integrates Trust Region (TR) methods with QA by
transforming the TR Newton sub-problem into a Quadratic Unconstrained Binary
Optimization (QUBO) problem. This transformation is achieved through three key
steps: using a hyper-rectangular shape for the TR, representing each
continuous dimension as interval-bounded integers, and binary encoding of
these bounded integers. By doing so, QuAnCO allows the application of Ising
solvers like D-Wave’s quantum annealer to continuous optimization problems,
which are otherwise not directly solvable by such quantum hardware.

The practicality and effectiveness of QuAnCO are demonstrated with a real-
world application: optimizing the biomass mix selection for Nature Energy,
Europe’s largest biogas producer. Biomass mix optimization is a relevant and
challenging problem due to the complex, non-convex nature of the cost
functions involved. This case study provides strong evidence of the
feasibility and performance advantages of using QuAnCO in bio-energy
production, showcasing its potential beyond solar energy to broader
applications in renewable energy.

Experiments reveal that QuAnCO significantly outperforms traditional
optimization methods such as Trust Region Newton (TRN), Conjugate Gradient
(CG), and Broyden–Fletcher–Goldfarb–Shanno (BFGS) methods, particularly for
complex, non-convex cost functions. For example, QuAnCO-Exact produces a
solution that is, on average, approximately 9% closer to the true minimum
(measured from starting point) compared to TRN (p-value: $4\times 10^{-5}$).
This performance advantage is most pronounced in the most challenging
scenarios studied, indicating that QuAnCO may offer similar benefits for more
realistic and complex cost functions, such as the widely used on Anaerobic
Digestion Models (ADM1) [17] for wastewater engineering and biogas production.

##  III Quantum Machine Learning

###  III-A Algorithm Review

Quantum Machine Learning (QML) holds significant promise for addressing
complex problems with high-dimensional large-scale data in civil engineering,
such as predicting the impacts of climate change and managing water resources.
Among the various QML algorithms in this domain, the Quantum Support Vector
Machine (QSVM)[18] and Quantum-Enhanced Convolutional Neural Networks
(QCNN)[19] stand out as potent tools for supervised learning, adaptive
decision-making, and pattern recognition tasks, which are essential for
predictive modeling and dynamic system management in environmental sciences.
Furthermore, Quantum Reinforcement Learning (QRL)[20] and Quantum Train (QT)
[21, 22] can be leveraged for complex decision-making processes, offering
efficient solutions for large-scale real-world problems through its trial-and-
error learning mechanism, which is neither purely supervised nor unsupervised
but rather a distinct paradigm of learning based on interactions with the
environment.

####  III-A1 Quantum Support Vector Machine

The QVSM is a quantum algorithm designed to classify data points into one of
the categories by finding the optimal hyperplane that separates the
classes[23, 24]. In classical settings, this involves constructing a maximum-
margin hyperplane with a normal vector $\mathbf{w}$ that maximizes the
distance between the closest points of the two classes, formulated as a
quadratic programming problem. The complexity of solving this in the classical
setting is polynomial in the size of the data. However, the QSVM leverages QC
principles to achieve significant computational advantages. Using quantum
techniques such as matrix inversion via the Harrow-Hassidim-Lloyd (HHL)
algorithm, QVSM can classify data with a run time that scales logarithmically
with the number of features $N$ and training examples $M$[18].

The core advantage of QSVM is rooted in its ability to perform efficient inner
product evaluations and matrix operations. One of the critical equations
illustrating this is the dual form of the SVM optimization problem:

$L(\boldsymbol{\alpha})=\sum_{j=1}^{M}\alpha_{j}-\frac{1}{2}\sum_{j,k=1}^{M}\alpha_{j}K_{jk}\alpha_{k},$
(1)

where $\alpha_{j}$ are the Lagrange multipliers and
$K_{jk}=\mathbf{x}_{j}\cdot\mathbf{x}_{k}$ is the kernel matrix. The quantum
version efficiently approximates this matrix inversion, solving for
$\boldsymbol{\alpha}$ with the matrix inversion algorithm. This process
involves preparing quantum states that represent the training data and
exploiting quantum parallelism to perform operations in $\mathcal{O}(\log
N\log M)$ time[25]. This results in an exponential speedup over classical
methods, particularly beneficial for large-scale datasets common in
environmental science applications.

Figure 2: Schematic of a QSVM for flood prediction, illustrating the process
from training data through PCA and quantum kernel estimation to final
classification.

####  III-A2 Quantum Convolutional Neural Networks

QCNN are designed to leverage QC for enhancing the capabilities of classical
Convolutional Neural Networks (CNNs), widely used in image and pattern
recognition tasks[19]. QCNNs apply quantum circuits to perform convolution
operations, allowing for the extraction of features from input data with high
precision. This enables QCNNs to capture intricate patterns in climate and
hydrological data, improving the model’s performance in recognizing complex
environmental phenomena.

In addition, QCNNs use quantum operations for pooling, a process that reduces
the dimensionality of data while preserving essential features. This approach
enhances the efficiency and effectiveness of data reduction, facilitating the
handling of large-scale environmental datasets. Often, QCNNs combine quantum
and classical layers to utilize the strengths of both computing paradigms,
processing data more efficiently and accurately. QCNNs’ ability to enhance
feature extraction and pattern recognition makes them highly valuable for
climate change and water engineering applications, such as identifying trends
in climate data and predicting environmental changes.

Figure 3: Schematic of a QCNN architecture for flood prediction, illustrating
the process from training data through convolution and pooling layers to final
flood or normal classification.

###  III-B Use Case: Flood Prediction

In this section, we utilize a real-world dataset comprising 33 years of time
series data from January 1, 1990, to December 31, 2023. This extensive dataset
includes measurements from 10 river and dam stations as well as 3 weather
stations along the Wupper River in Germany. It is pivotal for developing a
predictive model for flood events. In this study, we employ QSVM and QCNN
alongside traditional methods such as linear regression and K-Nearest
Neighbors (KNN) for benchmarking. Given the large and asymmetric nature of the
training data, with only a few instances of flood cases, our benchmarking
focuses on sensitivity, particularly the prediction of flood occurrences.

Our benchmarking pipeline incorporates 12 different features related to
various reservoirs and accumulated precipitation data, as well as forecast
values for 1 day, 2 days, 3 days, 7 days, and 14 days, amounting to a total of
18 features. Principal Component Analysis (PCA) is employed to select the 12
most relevant features for supervised learning tasks. The benchmarking results
indicate that KNN and linear regression achieve 0% sensitivity, highlighting
the limitations of traditional supervised learning methods in handling large-
scale and high-dimensional unbalanced data. In contrast, QSVM and QCNN achieve
sensitivities of 40% and 80%, respectively. This demonstrates the potential of
quantum-enhanced machine learning for predictions using real-world climate
data.

Our study shows that QCNN performs better than QSVM when the number of
features (equivalent to the required number of qubits) is small. Furthermore,
the validation process for large-scale QSVM can be conducted using the tensor
network approach[26].

##  IV Quantum Simulation

###  IV-A Algorithm Review

One of the most promising near-term noisy intermediate scale quantum (NISQ)
algorithms is the variational quantum eigensolver (VQE). VQE is primarily used
to simulate and understand physical systems at the quantum level, such as
atoms or molecules and their interactions[27]. Current classical computational
techniques are inadequate for accurately simulating such systems due to high
error rates that scale with the size of the modeled system[28].

The general goal of VQE is to calculate the the ground-state energy of a
Hamiltonian, which can be used to solve the electronic Schrodinger
equation[29]. A Hamiltonian is an operator that describes the system’s energy,
and the ground-state energy relays the lowest possible energy state of that
system. Thus, this calculation is particularly apt for materials science and
chemical engineering as the ground state can provide fundamental information
regarding the overall energy spectrum of a system and thermodynamic and
chemical properties, such as the bonding characteristics of a molecule[30]. As
such, the use of VQE could accelerate the development of new materials to both
mitigate the effects of climate change and support the transition to a low-
carbon economy[31].

VQE is implemented via gate-based quantum circuits comprised of the ansatz,
parameters, and a cost function. The purpose of the circuit is to model the
physics of a trial wave function using constraints such that the final
measured energy approximates the ground-state energy of the Hamiltonian [29].
The process begins with initialization of a register of qubits, which is then
applied to the ansatz. The ansatz is the structure of ordered gates through
which the wave function is encoded into the circuit; it is initially
configured with adjustable parameters that are variationally optimized to
minimize the ground-state energy. As a result, the ansatz design and initial
preparation greatly influences the model accuracy[32]. The optimization
process typically utilizes classical techniques, such as COBYLA or gradient
descent, to minimize the cost function. Finally, upon convergence of the cost
function, the final measured energy represents the ground-state energy of the
Hamiltonian, and the final parameter values represent the parameterized
wavefunction used to approximate the ground-state energy [30].

###  IV-B Use Case: Carbon Capture Materials

The transition to a low-carbon economy requires extensive research into the
use of different materials not only for energy generation and storage, but
also for carbon capture. Carbon capture via adsorption to solid sorbents
including metal-organic frameworks (MOFs) is a promising method as the energy
requirements are low compared to standard post-combustion techniques [33, 34].
However, the optimization of these materials requires understanding the
specific adsorption mechanisms of CO2 with different metals, which represents
many-body interactions that are difficult to model with classical computers.
As a result, current methods used to predict the properties of new MOFs rely
on empirical assumptions of thermodynamic properties that are limited to a
small portion of physical scenarios.

QC could be used to fully simulate the electronic structure of these materials
and, in doing so, improve the accuracy and efficiency of the screening of
potential MOFs. As such, Greene-Diniz et al. applied VQE to model the
interaction of CO2 with aluminum nanopores[35] and better understand the
adsorption energy between CO2 and the aluminum cluster (an aluminum atom
surrounded by oxygen atoms). This metric that determines how much CO2 can be
captured and therefore provides information regarding the efficiency of
certain materials over others.

The VQE algorithm utilized the unitary coupled cluster (UCC) ansatz that was
applied to a register of qubits representing the electronic orbitals of the
adsorbing aluminum atom. The wave function was initially prepared according to
a dissociation energy between CO2 and the aluminum complex approximated using
classical methods. Additionally, different circuits were constructed using 4,
8, and 12 qubits, each modeling a different number of electron orbitals and
therefore representing different active spaces with which the CO2 could
physically adsorb to the aluminum. The results from high-level classical and
quantum models were comparable, with predicted bond dissociation energies
exhibiting minimal differences (¡1 mHa). The authors also found a non-trivial
dependence of CO2 binding and the non-active sites surrounding the active
site, underscoring the versatility of VQE for testing a broad range of
physical scenarios that could lead to improved models of this complex binding
process in the future. Thus, beyond providing precise output estimations, the
application of VQE for materials science could widen the range of physical
interactions modeled, which could ultimately expedite the process of
understanding and selecting the optimal materials for carbon capture.

##  V Discussion and conclusion

The possibilities afforded by QC for climate issues are extensive, yet the
amount of research dedicated to these applications is minimal compared to
other areas (examples). It is important to start a shift in focus toward both
the application of existing QC and QML methods as well as the development of
novel QC methods to tackle specific climate-resilience challenges. QuAnCO is a
great example of limitations in current methods catalyzing further
development. The scope of QuAnCO extends far beyond the original intended use
case, as there a various applications of continuous optimization problems that
could benefit from transformation to annealing problems, such as the
operational parameters in treatment processes, or other, complex, renewable
energy sources such as solar and geothermal. In addition, the scope of
problems addressable by QAOA is large, particularly concerning route
optimization in supply chains for water and food distribution, as well as
human transport.

Similarly, traditional machine learning methods have their limitations,
particularly when faced with large volumes of high-dimensional data, such as
in kernel transformations for SVM or deep neural networks. It is believed that
QML models will be able to express and decode complex patterns at lower
computational cost, due inherent complexity of Hilbert space [36], potentially
allowing us to increase the processing efficiency of high-dimension, high-
volume ML models. Climate-resilience problems, such as the flood prediction
use case, tend to require such large volumes of data, leading to difficulties
for classical ML methods in uncovering the behaviour of systems preceding
catastrophe. This highlights the need for QC methods which can perform
calculations more efficiently in matrix inversion or pooling operations. We
expect the methods used to build flood prediction models could be extended to
other disastrous climate events such as severe storms, wildfires and more, as
they all occur as a result of complex combinations of various different
meteorological and environmental factors, making good targets for QML models.

Quantum simulation methods face issues of their own. Quantum phase estimation
(QPE) poses complete solutions to many chemical and material simulation
problems, but is out of reach for current quantum computers, hence the
reliance on hybrid variational algorithms such as VQE for near-term problems,
as variational methods are expected to be more resistant to noise than QPE
[29]. VQE has issues of its own, however —VQE is known to work reasonably well
when good prior knowledge of the problem is available, most importantly a good
initial ansatz [37]. If such an ansatz is not available, VQE is unlikely to
provide useful results. To optimise algorithms like VQE for climate related
problems, this prior knowledge is essential, and will come only from attempted
implementation. As QC resources increase, more advanced methods are expected
to become feasible. These methods are largely unexplored for climate-related
chemical and material problems and hold strong future research prospects.
Quantum simulation possibilities also extend beyond material and chemical
challenges, showing promise also in improving simulation methods in
computational fluid dynamics [12, 13, 14, 15]. Simulating complex fluid
dynamic systems presents a strong path for improving our best weather and
climate models.

In this work, we have provided a brief review on the potential applications of
QC in tackling future climate-resilience problems, notably those relating to
renewable energy and water management.

## References

  * [1] A. Lupo, W. Kininmonth, J. Armstrong and K. Green, Global climate models and their limitations, Climate change reconsidered II: Physical science 9 (2013), p. 148.
  * [2] C. O. de Burgh-Day and T. Leeuwenburg, Machine learning for numerical weather and climate modelling: a review, Geoscientific Model Development 16 (2023), pp. 6433–6477.
  * [3] F. Tennie and T. Palmer, Quantum computers for weather and climate prediction: The good, the bad, and the noisy, Bulletin of the American Meteorological Society 104 (2023), pp. E488–E500.
  * [4] R. Jozsa and N. Linden, On the role of entanglement in quantum-computational speed-up, Proceedings of the Royal Society of London. Series A: Mathematical, Physical and Engineering Sciences 459 (2003), pp. 2011–2032.
  * [5] H. Liu, G. H. Low, D. S. Steiger, T. Häner, M. Reiher and M. Troyer, Prospects of quantum computing for molecular sciences, Materials Theory 6 (2022), p. 11.
  * [6] M. T. Sharabiani, V. B. Jakobsen, M. Jeppesen and A. S. Mahani, Quantum annealing continuous optimisation in renewable energy, arXiv preprint arXiv:2105.11322 (2021).
  * [7] A. Rajak, S. Suzuki, A. Dutta and B. K. Chakrabarti, Quantum annealing: An overview, Philosophical Transactions of the Royal Society A 381 (2023), p. 20210417.
  * [8] E. Farhi, J. Goldstone and S. Gutmann, A quantum approximate optimization algorithm, arXiv preprint arXiv:1411.4028 (2014).
  * [9] L. Zhou, S.-T. Wang, S. Choi, H. Pichler and M. D. Lukin, Quantum approximate optimization algorithm: Performance, mechanism, and implementation on near-term devices, Physical Review X 10 (2020), p. 021067.
  * [10] X. Xu, K.-C. Chen and R. Wille, Hamiltoniq: An open-source benchmark toolkit for quantum computers, arXiv preprint arXiv:2404.13971 (2024).
  * [11] Z. Meng and Y. Yang, Quantum computing of fluid dynamics using the hydrodynamic schrödinger equation, Physical Review Research 5 (2023), p. 033182.
  * [12] P. Brearley and S. Laizet, Quantum Algorithm for Solving the Advection Equation using Hamiltonian Simulation, Physical Review A 110 (2024), p. 012430.  arXiv:2312.09784 [quant-ph].  url: http://arxiv.org/abs/2312.09784, doi:10.1103/PhysRevA.110.012430.
  * [13] N. Ray, T. Banerjee, B. Nadiga and S. Karra, Towards Solving the Navier-Stokes Equation on Quantum Computers, Apr. 2019.  arXiv:1904.09033 [physics].  url: http://arxiv.org/abs/1904.09033, doi:10.48550/arXiv.1904.09033.
  * [14] F. Gaitan, Finding flows of a Navier–Stokes fluid through quantum computing, npj Quantum Information 6 (2020), pp. 1–6.  Publisher: Nature Publishing Group.  url: https://www.nature.com/articles/s41534-020-00291-0, doi:10.1038/s41534-020-00291-0.
  * [15] L. Budinski, Quantum algorithm for the Navier Stokes equations by using the streamfunction vorticity formulation and the lattice Boltzmann method, International Journal of Quantum Information 20 (2022), p. 2150039.  arXiv:2103.03804 [quant-ph].  url: http://arxiv.org/abs/2103.03804, doi:10.1142/S0219749921500398.
  * [16] F. Phillipson, Quantum computing in logistics and supply chain management-an overview, arXiv preprint arXiv:2402.17520 (2024).
  * [17] D. J. Batstone et al., The iwa anaerobic digestion model no 1 (adm1), Water Science and technology 45 (2002), pp. 65–73.
  * [18] P. Rebentrost, M. Mohseni and S. Lloyd, Quantum support vector machine for big data classification, Physical review letters 113 (2014), p. 130503.
  * [19] I. Cong, S. Choi and M. D. Lukin, Quantum convolutional neural networks, Nature Physics 15 (2019), pp. 1273–1278.
  * [20] D. Dong, C. Chen, H. Li and T.-J. Tarn, Quantum reinforcement learning, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics) 38 (2008), pp. 1207–1220.
  * [21] C.-Y. Liu, C.-H. A. Lin, C.-H. H. Yang, K.-C. Chen and M.-H. Hsieh, Qtrl: Toward practical quantum reinforcement learning via quantum-train, arXiv preprint arXiv:2407.06103 (2024).
  * [22] C.-H. A. Lin, C.-Y. Liu and K.-C. Chen, Quantum-train long short-term memory: Application on flood prediction problem, arXiv preprint arXiv:2407.08617 (2024).
  * [23] Z. Li, X. Liu, N. Xu and J. Du, Experimental realization of a quantum support vector machine, Physical review letters 114 (2015), p. 140504.
  * [24] K.-C. Chen, X. Xu, H. Makhanov, H.-H. Chung and C.-Y. Liu, Quantum-enhanced support vector machine for large-scale stellar classification with gpu acceleration, arXiv preprint arXiv:2311.12328 (2023).
  * [25] G. Gentinetta, A. Thomsen, D. Sutter and S. Woerner, The complexity of quantum support vector machines, Quantum 8 (2024), p. 1225.
  * [26] K.-C. Chen et al., cutn-qsvm: cutensornet-accelerated quantum support vector machine with cuquantum sdk, arXiv preprint arXiv:2405.02630 (2024).
  * [27] B. Bauer, S. Bravyi, M. Motta and G. K.-L. Chan, Quantum algorithms for quantum chemistry and quantum materials science, Chemical Reviews 120 (2020), p. 12685–12717.  url: http://dx.doi.org/10.1021/acs.chemrev.9b00829, doi:10.1021/acs.chemrev.9b00829.
  * [28] Y. Zhang et al., Variational quantum eigensolver with reduced circuit complexity, npj Quantum Information 8 (2022).  url: http://dx.doi.org/10.1038/s41534-022-00599-z, doi:10.1038/s41534-022-00599-z.
  * [29] J. Tilly et al., The variational quantum eigensolver: A review of methods and best practices, Physics Reports 986 (2022), pp. 1–128.  The Variational Quantum Eigensolver: a review of methods and best practices.
  * [30] J. M. Clary, E. B. Jones, D. Vigil-Fowler, C. Chang and P. Graf, Exploring the scaling limitations of the variational quantum eigensolver with the bond dissociation of hydride diatomic molecules, International Journal of Quantum Chemistry 123 (2023), p. e27097.  doi:https://doi.org/10.1002/qua.27097.
  * [31] D. Fedorov, B. Peng, N. Govind and Y. Alexeev, Vqe method: a short survey and recent developments, Materials Theory 6 (2022).  doi:10.1186/s41313-021-00032-6.
  * [32] M. Sapova and A. Fedorov, Variational quantum eigensolver techniques for simulating carbon monoxide oxidation, Communications Physics 5 (2022), p. 199.  doi:10.1038/s42005-022-00982-4.
  * [33] K. C. Stylianou and W. L. Queen, Recent Advances in Carbon Capture with Metal–Organic Frameworks, CHIMIA 69 (2015), pp. 274–274.  Number: 5.  url: https://www.chimia.ch/chimia/article/view/2015_274, doi:10.2533/chimia.2015.274.
  * [34] K. Sumida et al., Carbon Dioxide Capture in Metal–Organic Frameworks, Chemical Reviews 112 (2012), pp. 724–781.  Publisher: American Chemical Society.  url: https://doi.org/10.1021/cr2003272, doi:10.1021/cr2003272.
  * [35] G. Greene-Diniz et al., Modelling carbon capture on metal-organic frameworks with quantum computing, EPJ Quantum Technology 9 (2022).  doi:10.1140/epjqt/s40507-022-00155-w.
  * [36] J. Biamonte, P. Wittek, N. Pancotti, P. Rebentrost, N. Wiebe and S. Lloyd, Quantum Machine Learning, Nature 549 (2017), pp. 195–202.  arXiv:1611.09347 [cond-mat, physics:quant-ph, stat].  url: http://arxiv.org/abs/1611.09347, doi:10.1038/nature23474.
  * [37] W. Li et al., Toward practical quantum embedding simulation of realistic chemical systems on near-term quantum computers, Chem. Sci. 13 (2022).