# Local Latent Space Bayesian Optimization over Structured Inputs

Natalie Maus  Haydn T. Jones  Juston S. Moore  Matt J. Kusner  John Bradshaw
Jacob R. Gardner

###### Abstract

Bayesian optimization over the latent spaces of deep autoencoder models (DAEs)
has recently emerged as a promising new approach for optimizing challenging
black-box functions over structured, discrete, hard-to-enumerate search spaces
(e.g., molecules). Here the DAE dramatically simplifies the search space by
mapping inputs into a continuous latent space where familiar Bayesian
optimization tools can be more readily applied. Despite this simplification,
the latent space typically remains high-dimensional. Thus, even with a well-
suited latent space, these approaches do not necessarily provide a complete
solution, but may rather shift the structured optimization problem to a high-
dimensional one. In this paper, we propose LOL-BO, which adapts the notion of
trust regions explored in recent work on high-dimensional Bayesian
optimization to the structured setting. By reformulating the encoder to
function as both an encoder for the DAE globally and as a deep kernel for the
surrogate model within a trust region, we better align the notion of local
optimization in the latent space with local optimization in the input space.
LOL-BO achieves as much as 20 times improvement over state-of-the-art latent
space Bayesian optimization methods across six real-world benchmarks,
demonstrating that improvement in optimization strategies is as important as
developing better DAE models.

Machine Learning, ICML

##  1 Introduction

Many challenges across the physical sciences and engineering require the
optimization of _structured objects_. For example, in the drug design setting,
it is common to seek a molecule that maximizes some property, e.g., binding
affinity for a target protein (Huang et al., 2021), or similarity to a known
drug molecule under some metric (Brown et al., 2019). These structured
optimization problems are particularly challenging, as the objective function
is now typically over a large combinatorial space of discrete objects.

These challenging optimization problems have motivated a large amount of very
recent interest in methods that perform _Bayesian optimization (BO) over
latent spaces_. Using drug design as a running example, a deep auto-encoder
(DAE) is first trained on a large set of unsupervised molecules. Bayesian
optimization is then run as normal in the continuous latent space of the
autoencoder to produce candidate latent codes. These candidate latent codes
are then decoded back to molecules, and the objective function is evaluated.
Bayesian optimization then proceeds as normal, with the surrogate model
trained on a dataset of latent codes and their resulting objective values
after decoding.

Rapid progress has been made in this emerging topic. In large part, this has
been to improve the DAE model. The early work of Kusner et al. (2017) and Jin
et al. (2018) investigated domain specific autoencoder architectures that much
more reliably produced valid molecules. The latest work of Tripp et al. (2020)
and Grosnit et al. (2021) investigate methods to inject supervised signal from
objective function evaluations into the latent space, enabling better
surrogate modeling.111For the interested reader see these recent surveys for
more details (Öztürk et al., 2020; Coley et al., 2020). However, the
_optimization_ component of latent space Bayesian optimization remains under-
explored: indeed, it is common practice to directly apply standard Bayesian
optimization with expected improvement once a latent space has been
constructed (all of the above cited methods do this). We conjecture that,
despite the ability of a deep encoder to dramatically simplify a structured
input space, optimization remains challenging due to the high-dimensionality
of the extracted latent space.

In this paper, we propose to jointly co-design the latent space model and a
local Bayesian optimization strategy to produce a method for structured
optimization. Concretely, we make the following contributions:

  1. 1.

We develop local latent Bayesian optimization, LOL-BO, for the structured
optimization setting, addressing the inherent mismatch between the notion of a
trust region in the latent space and a trust region in the original structured
input space.

  2. 2.

We derive a semisupervised DAE model that better satisfies the implicit
assumptions of trust region methods by performing joint variational inference
over the DAE and sparse GP surrogate model (this model is similar in spirit to
Kingma et al. (2014); Eissman et al. (2018)).

  3. 3.

We propose a deep autoencoder architecture using transformer layers over the
recently proposed SELFIES representation for molecules (Krenn et al., 2019)
that additionally improves latent space organization.

  4. 4.

We show the potential of co-adapting latent space modeling with local Bayesian
optimization in an empirical evaluation of LOL-BO against top-performing
baselines, achieving up to $22\times$ improvements in objective value on
commonly used benchmark tasks over prior latent space BO approaches.

##  2 Background

In black-box optimization, we seek to find the minimizer of a function,
$\operatorname*{arg\,min}_{\mathbf{x}\in\mathcal{X}}f(\mathbf{x})$. Commonly,
$f(\mathbf{x})$ is assumed to be expensive to evaluate and unknown (i.e., a
“black box”). It is therefore desirable to use fewer function evaluations to
achieve a given objective value.

Most classically, this problem has been considered in the setting where the
search space $\mathcal{X}$ is continuous (e.g.,
$\mathcal{X}\subseteq\mathbb{R}$). However, many applications across the
natural sciences and engineering require optimizing over discrete and
structured input spaces. De novo molecule design using machine learning has
received particular attention, where $\mathcal{X}$ is a search space of small
organic molecules and $f(x)$ is some desirable property to optimize (Mouchlis
et al., 2021).

#### Bayesian optimization

(Osborne et al., 2009; Mockus, 1982; Snoek et al., 2012) is a framework for
solving these black-box optimization problems, most commonly when
$\mathcal{X}$ is a continuous, real-valued search space. A surrogate model is
trained on a set of inputs and corresponding objective values
$\mathcal{D}_{\ell}=[\mathbf{x}_{i},y_{i}]_{i=1}^{n}$. This surrogate model is
used to suggest candidate inputs $\hat{\mathbf{x}}$ to query next. The
candidates are then labeled, added to the dataset, and the process repeats.
Gaussian process models
$f(\mathbf{x})\sim\mathcal{GP}(m(\mathbf{x}),k(\mathbf{x},\mathbf{x}^{\prime}))$
are commonly used as surrogate models, as access to calibrated predictive
(co-)variances enables search strategies that effectively trade off
exploration and exploitation.

#### Deep autoencoder models.

A deep autoencoder model consists of an encoder
$\Phi:\mathcal{X}\to\mathcal{Z}$ that maps from a structured and discrete
space (e.g., the space of molecules) $\mathcal{X}$ to a continuous, real-
valued _latent space_ $\mathcal{Z}$, and a decoder that does the reverse
$\Gamma(\cdot):\mathcal{Z}\to\mathcal{X}$. A variational autoencoder (Kingma &
Welling, 2013) uses a probabilistic form of these two functions, a variational
posterior $\Phi(\mathbf{Z}\mid\mathbf{X})$ and a likelihood
$\Gamma(\mathbf{X}\mid\mathbf{Z})$. The parameters
$\theta_{\Phi},\theta_{\Gamma}$ of $\Phi$ and $\Gamma$ are trained by
maximizing the evidence lower bound (ELBO):

$\displaystyle\mathcal{L}_{\textrm{VAE}}$
$\displaystyle=\textrm{ELBO}(\theta_{\Phi},\theta_{\Gamma})$ (1)
$\displaystyle=\mathbb{E}_{\Phi(\mathbf{Z}\mid\mathbf{X})}[\log\Gamma(\mathbf{X}\mid\mathbf{Z})]-\textrm{KL}(\Phi(\mathbf{Z}\mid\mathbf{X})\,||\,p(\mathbf{Z}))$

Commonly, the (amortized) variational posterior
$\Phi(\mathbf{Z}\mid\mathbf{X})$ and the prior $p(\mathbf{Z})$ are chosen to
be Gaussian, which leads to a convenient analytic expression for the KL
divergence. Note that, with respect to specific optimization tasks we might
want to perform over the structured space $\mathcal{X}$, the ELBO as defined
above is unsupervised.

#### Latent space Bayesian optimization.

Latent space Bayesian optimization seeks to leverage the representation
learning capabilities of deep autoencoders models (DAEs) to reduce a discrete,
structured search space $\mathcal{X}$ to a continuous one $\mathcal{Z}$ (Tripp
et al., 2020; Deshwal & Doppa, 2021; Gómez-Bombarelli et al., 2018; Grosnit et
al., 2021; Eissman et al., 2018; Kajino, 2019). Because the ELBO above is
unsupervised with respect to an optimization task, a large set of unlabeled
inputs $\mathcal{D}_{u}\subseteq\mathcal{X}$ is first used to train the DAE. A
Gaussian process prior is again placed over the objective function, but here
as a function of the _latent codes_ ,
$f(\mathbf{z})\sim\mathcal{GP}(m(\mathbf{z}),k(\mathbf{z},\mathbf{z}^{\prime}))$.
To train the GP initially, we collect a small set of $n$ inputs
$\mathcal{D}_{\ell}\subseteq\mathcal{X},n\ll|\mathcal{D}_{u}|$ with known
labels $\mathbf{y}_{\ell}$. The inputs are passed through the encoder to
obtain latent representations $\mathbf{Z}=[\mathbf{z}_{i}]_{i=1}^{n}$, and the
surrogate model is fit to the dataset $[(\mathbf{z}_{i},y_{i})]_{i=1}^{n}$.
Standard Bayesian optimization strategies may then be applied using this
surrogate model to select a candidate latent vector $\hat{\mathbf{z}}$, which
is then decoded back to an input $\hat{\mathbf{x}}$ and evaluated.

##  3 Methods

Recent latent space Bayesian optimization (LS-BO) approaches have focused on
improving the architecture of the VAE used to compress the search space
(Kusner et al., 2017; Jin et al., 2018), and on reorganizing the subsequent
latent space so that it is more amenable to BO (Tripp et al., 2020; Grosnit et
al., 2021; Eissman et al., 2018). However, it is commonplace to apply standard
BO with expected improvement once the latent space has been constructed,
despite the fact that this latent space often remains high-dimensional (Jin et
al., 2018), on the order of tens or hundreds of dimensions. In this paper, we
explore whether lessons learned in the high-dimensional BO setting can be
adapted to improve optimization in high-dimensional latent spaces.

###  3.1 Local Bayesian optimization – Fixed latent space

One approach to adapting high-dimensional BO strategies is to apply them in
the extracted latent space $\mathcal{Z}$. Because $\mathcal{Z}$ is a synthetic
latent space, it’s unlikely that the objective function decomposes additively
over this space without explicitly encouraging this (Kandasamy et al., 2015;
Gardner et al., 2017; Wang et al., 2018). Likewise, $\mathcal{Z}$ is already a
compression of the much larger $\mathcal{X}$, so methods that exploit low-
dimensional substructure may not be effective (Wang et al., 2016; Nayebi et
al., 2019; Eriksson & Jankowiak, 2021; Garnett et al., 2014).

_Local Bayesian optimization_ is a strategy that avoids over-exploration in
these search spaces _without_ simplifying assumptions about the objective.
Commonly this is done by restricting the search in each iteration to a small
region. For example, Eriksson et al. (2019) proposes TuRBO, which maintains a
rectangular _trust region_ centered at the current best input. Candidates are
restricted to be in the trust region.

Adapting trust regions to LS-BO is straightforward. Suppose we have a VAE with
encoder $\Phi(\cdot)$ and decoder $\Gamma(\cdot)$ trained on an unsupervised
dataset $\mathcal{D}_{u}$ of structured inputs, and a supervised set
$\mathbf{Z}$ of latent codes corresponding to labeled inputs
$\mathcal{D}_{\ell}$ and their objective values $\mathbf{y}$. Denote by
$\hat{\mathbf{z}}$ the _incumbent_ latent code corresponding to the input
$\hat{\mathbf{x}}$ with best observed objective value $\hat{y}$ so far.

The trust region is an axis aligned hyper-rectangular subset of the latent
space $\mathcal{T}_{\hat{z}}\subseteq\mathcal{Z}$, centered at the incumbent
latent vector $\hat{\mathbf{z}}$. It has a base side length $L$, with the
actual side length for each dimension rescaled according to its lengthscale
$\lambda_{i}$ in the GP’s ARD covariance function,
$L_{i}=\frac{\lambda_{i}L}{(\prod_{j=1}^{d}\lambda_{j})^{1/d}}$. Candidate
latent vectors to evaluate next are restricted to be selected from
$\mathcal{T}_{\hat{z}}$. Following Nelder & Mead (1965); Eriksson et al.
(2019), the base trust region length is halved after $\tau_{\textrm{fail}}$
iterations where no progress is made and doubled after $\tau_{\textrm{succ}}$
iterations in a row where progress is made. When a new incumbent
$\hat{\mathbf{z}}^{\prime}$ is found with $\hat{y^{\prime}}<\hat{y}$, the
trust region is recentered at $\hat{\mathbf{z}}^{\prime}$.

###  3.2 Local Bayesian optimization – Adaptive latent space

By defining “locality” in terms of a small rectangular region centered at the
incumbent, trust region methods implicitly assume smoothness, that points
close to the center have highly correlated objective values with it. In the
standard BO setting with GP surrogates, this assumption matches that made by
the GP, as many covariance functions measure covariance as a function of
distance.

However, the latent space $\mathcal{Z}$ of a DAE is only trained so that a
latent vector $\mathbf{z}=\Phi(\mathbf{x})$ reconstructs the original input
when passed through the decoder, $\Gamma(\mathbf{z})\approx\mathbf{x}$.
Therefore, it is possible that small changes to the current incumbent _latent
vector_ $\hat{\mathbf{z}}$ may result in large changes to the corresponding
_input_ $\hat{\mathbf{x}}$, violating the assumptions of both the Gaussian
process surrogate model and the trust region method.

#### Adapting the latent space to the GP prior.

We propose to solve this mismatch by performing inference jointly over both
the Gaussian process surrogate defined over the latent space
$p(\mathbf{f}\mid\mathbf{Z})$ and the Bayesian VAE model
$p(\mathbf{X},\mathbf{Z})=p(\mathbf{X}\mid\mathbf{Z})p(\mathbf{Z})$, while
assuming in the prior conditional independence between $\mathbf{f}$ and
$\mathbf{X}$ given $\mathbf{Z}$:

$p(\mathbf{f},\mathbf{X},\mathbf{Z})=p(\mathbf{f}\mid\mathbf{Z})\Gamma(\mathbf{X}\mid\mathbf{Z})p(\mathbf{Z})$

By performing inference over $\mathbf{f}$ and $\mathbf{Z}$ jointly in the
above model in an end-to-end fashion, we explicitly encourage the latent space
to organize in a way that matches the assumptions of the GP prior
$p(\mathbf{f}\mid\mathbf{Z})$.

#### Relation to prior work.

Learning the VAE and supervised model jointly in a semisupervised fashion is
related to prior work on semisupervised VAE modelling as first proposed in
Kingma et al. (2014). This was adapted for BO in Eissman et al. (2018), who
use an auxiliary supervised neural network to inject supervision before
fitting the GP. Here we consider joint variational inference over the VAE and
a sparse GP, directly encouraging the latent space to satisfy the assumptions
of the GP prior. In the global BO setting, Siivola et al. (2021) finds
semisupervised adaptation of VAEs to not be critical; however, in the local
setting, we experimentally find this results in significant improvement due to
the assumptions underlying trust region methods.

#### Sparse GP augmentation.

Following Hensman et al. (2013), the above prior $p(\mathbf{f}\mid\mathbf{Z})$
can be augmented with a global set of learned _inducing variables_
$\mathbf{u}$ representing latent function values at a set of _inducing
locations_ $\mathbf{V}$222Canonically inducing locations are often denoted
$\mathbf{Z}$. However, in this work $\mathbf{Z}$ is already used to denote
latent vectors.:

$p(\mathbf{f}\mid\mathbf{Z})\to
p(\mathbf{f}\mid\mathbf{u},\mathbf{Z},\mathbf{V})p(\mathbf{u}\mid\mathbf{V})$

resulting in the following full joint density over the latent function
$\mathbf{f}$, structured inputs $\mathbf{X}$, latent representation
$\mathbf{Z}$, objective values $\mathbf{y}$, and inducing variables
$\mathbf{u}$ (ommitting dependencies on inducing locations $\mathbf{V}$ for
compactness):

$\displaystyle p(\mathbf{y},\mathbf{X},$
$\displaystyle\mathbf{f},\mathbf{u},\mathbf{Z})=p(\mathbf{y}\mid\mathbf{f})p(\mathbf{f}\mid\mathbf{u},\mathbf{Z})p(\mathbf{u})\Gamma(\mathbf{X}\mid\mathbf{Z})p(\mathbf{Z})$
(2)

#### ELBO derivation.

Our goal is to derive an evidence lower bound (ELBO) for the above model to
perform variational inference over both the inducing variables $\mathbf{u}$
and VAE latent variables $\mathbf{Z}$. The log evidence
$p(\mathbf{y},\mathbf{X})$ for this model is:

$\log p(\mathbf{y},\mathbf{X})=\log\int
p(\mathbf{y},\mathbf{X},\mathbf{f},\mathbf{u},\mathbf{Z})d\mathbf{f}d\mathbf{u}d\mathbf{Z}$

To aid in lower bounding the log evidence, we introduce a variational
distribution
$q(\mathbf{u},\mathbf{Z})=q(\mathbf{u})\Phi(\mathbf{Z}\mid\mathbf{X})$ and
write:

$\displaystyle\log p(\mathbf{y},\mathbf{X})=\log\int
p(\mathbf{y},\mathbf{X},\mathbf{f},\mathbf{u},\mathbf{Z})d\mathbf{f}\frac{q(\mathbf{u})}{q(\mathbf{u})}d\mathbf{u}\frac{\Phi(\mathbf{Z}\mid\mathbf{X})}{\Phi(\mathbf{Z}\mid\mathbf{X})}d\mathbf{Z}$
$\displaystyle=\log\mathbb{E}_{\Phi(\mathbf{Z}\mid\mathbf{X})}\left[\frac{1}{\Phi(\mathbf{Z}|\mathbf{X})}\mathbb{E}_{q(\mathbf{u})}\left[\frac{1}{q(\mathbf{u})}\int
p(\mathbf{y},\mathbf{X},\mathbf{f},\mathbf{u},\mathbf{Z})d\mathbf{f}\right]\right]$
(3)

Because the joint distribution
$p(\mathbf{y},\mathbf{X},\mathbf{f},\mathbf{u},\mathbf{Z})$ factors as in
Equation 2, we can further rewrite the integral of the joint distribution over
$\mathbf{f}$ as:

$\int
p(\mathbf{y},\mathbf{X},\mathbf{f},\mathbf{u},\mathbf{Z})d\mathbf{f}=\mathbb{E}_{p(\mathbf{f}\mid\mathbf{u},\mathbf{Z})}\left[p(\mathbf{y}\mid\mathbf{f})p(\mathbf{u})\Gamma(\mathbf{X}\mid\mathbf{Z})p(\mathbf{Z})\right]$

Plugging this expected value form of the joint distribution integral into
Equation 3 and repeatedly applying Jensen’s inequality, we derive an evidence
lower bound

$\displaystyle\log p(\mathbf{y},\mathbf{X})$
$\displaystyle\geq\mathbb{E}_{p(\mathbf{f}\mid\mathbf{u},\mathbf{Z})q(\mathbf{u})\Phi(\mathbf{Z}\mid\mathbf{X})}\left[\log
p(\mathbf{y}\mid\mathbf{f})\right]$ (4)
$\displaystyle+\mathbb{E}_{\Phi(\mathbf{Z}\mid\mathbf{X})}\left[\log\Gamma(\mathbf{X}\mid\mathbf{Z})\right]$
$\displaystyle-\textrm{KL}(\Phi(\mathbf{Z}\mid\mathbf{X})\,||\,p(\mathbf{Z}))$
$\displaystyle-\textrm{KL}(q(\mathbf{u})\,||\,p(\mathbf{u}))$

#### The ELBO in practice.

Rephrasing Equation 4 in terms of the standard SVGP (stochastic variational
GP; Hensman et al., 2013) and VAE ELBOs, we see that:

$\displaystyle\mathcal{L}_{\textrm{joint}}$
$\displaystyle=\mathbb{E}_{\Phi(\mathbf{Z}\mid\mathbf{X})}\left[\mathcal{L}_{\textrm{SVGP}}(\theta_{\mathcal{GP}},\theta_{\Phi};\mathbf{y},\mathbf{Z})\right]$
$\displaystyle+\mathcal{L}_{\textrm{VAE}}(\theta_{\Phi},\theta_{\Gamma};\mathbf{X})$

The ELBO can be estimated as follows. Select a minibatch of inputs (e.g.,
molecules) $\mathbf{X}$, some of which have known labels $\mathbf{y}$, and
some of which are unlabeled. Pass the minibatch $\mathbf{X}$ through the
encoder $\Phi$ and generate latent codes $\mathbf{Z}$. For latent codes
$\mathbf{Z}_{\ell}$ corresponding to labeled inputs, compute
$\mathcal{L}_{\textrm{SVGP}}$ on the supervised dataset
$(\mathbf{Z}_{\ell},\mathbf{y}_{\ell})$. For _all_ latent codes $\mathbf{Z}$,
pass through the decoder and add the standard VAE loss. Because the
variational posteriors $\Phi(\mathbf{Z}\mid\mathbf{X})$ and $q(\mathbf{u})$
support reparameterization, computing derivatives of the ELBO with respect to
$\theta_{\Phi},\theta_{\Gamma},\theta_{\mathcal{GP}}$ is straightforward.
Because the encoder parameters $\theta_{\Phi}$ are updated through the SVGP
loss, $\Phi$ can be viewed as simultaneously acting as an encoder for the VAE
and as a (stochastic) deep kernel (Wilson et al., 2016a, b) for the GP model.

#### Recentering

Under this joint model, local BO can still be applied using the same setup and
hyperparameters discussed in subsection 3.1. However, when using the above
loss, the VAE–and therefore the latent space–is now modified after each
iteration of BO. Therefore, the center of the trust region $\hat{\mathbf{z}}$
and latent codes of inputs within or near the trust region may change. To
remedy this, we pass observed data $\mathbf{X}$ back through the updated
encoder to determine the new location of the corresponding latent points
$\mathbf{Z}$ before selecting candidates, and update the trust region
accordingly.

#### Computational considerations.

Updating the full encoder $\Phi(\cdot)$ can be significantly more expensive
than training the surrogate model only. A practical extension to the above
idealized process is to only train in a joint fashion periodically. Thus, the
variational GP parameters $\theta_{\mathcal{GP}}$ are updated in each
iteration of optimization, but not necessarily
$\theta_{\Phi},\theta_{\Gamma}$.

We propose to tie updates to $\theta_{\Phi},\theta_{\Gamma}$ to the sequential
successes and failures already tracked by the trust region method. If the
optimization procedure is making rapid progress, updating
$\theta_{\Phi},\theta_{\Gamma}$ may be an unnecessary expense. Therefore, we
update the full set of parameters
$\theta_{\Phi},\theta_{\Gamma},\theta_{\mathcal{GP}}$ after
$\tau_{\textrm{retrain}}$ successive failures. Experimentally we use
$\tau_{\textrm{retrain}}\ll\tau_{\textrm{fail}}$ as this allows for multiple
joint model updates for each time a trust region shrinks.

##  4 Related Work

Bayesian optimization for molecule discovery can be performed without DAEs
over fixed lists of molecules (Williams et al., 2015; Hernández-Lobato et al.,
2017; Graff et al., 2021). However, the need to pre-define the molecules that
can be queried upfront puts a fundamental limit to the chemical space such
methods can consider: the largest virtual libraries are of the order of
$\approx 10^{10}-10^{20}$ in size (van Hilten et al., 2019), which is tiny
compared to the space of all possible compounds (Kirkpatrick & Ellis, 2004).

Latent-space Bayesian optimization (Gómez-Bombarelli et al., 2018; Eissman et
al., 2018; Tripp et al., 2020; Grosnit et al., 2021; Siivola et al., 2021; Jin
et al., 2018) ties the optimization algorithm to a DAE so that, in theory, it
can generate any molecule. We broadly group many recent advancements in this
approach into two categories: (a) those that investigate new training losses
to improve surrogate learning of the objective (Grosnit et al., 2021; Tripp et
al., 2020; Eissman et al., 2018; Deshwal & Doppa, 2021) and (b) those that
develop new decoder architectures, for instance improving the validity of the
generated molecules by building in grammars or working explicitly in the graph
domain (Kusner et al., 2017; Jin et al., 2018; Samanta et al., 2019; Kajino,
2019; Dai et al., 2018). Decoder architectures have also been designed to
apply these ideas to different discrete domains such as neural network
architectures (Zhang et al., 2019) or Gaussian process kernels (Lu et al.,
2018).

Components of our method including the joint training procedure described in
subsection 3.2 are similar to Eissman et al. (2018) which considers injecting
semisupervision into the VAE via an auxiliary neural network and Grosnit et
al. (2021) which does so using a triplet loss. In particular, the triplet loss
of Grosnit et al. (2021) likely encourages similar latent space organization
to the method described in subsection 3.2 due to the use of distances in both
the triplet loss and the GP covariance function. However, these works use off-
the-shelf global Bayesian optimization algorithms, such as expected
improvement, as the latent optimizer.

Alternative optimization algorithms have also been proposed for molecule
discovery, such as those based on genetic algorithms (Jensen, 2019; Nigam et
al., 2020), Markov chain Monte Carlo (Xie et al., 2021), or ideas from
reinforcement learning (RL) (Olivecrona et al., 2017; You et al., 2018; Zhou
et al., 2019). Segler et al. (2018) use the cross entropy method to guide the
generation of complete SMILES strings from an autoregressive recurrent neural
network. You et al. (2018) train a policy to build molecules up from atoms and
bonds using a RL algorithm. While such methods are often able to find
molecules with better property scores than early LS-BO approaches, these
methods frequently neglect sample efficiency, impeding their use on practical
problems.

##  5 Experiments

Figure 1: Optimization results for the log P and arithmetic expressions tasks.
We compare to recent competitive latent BO methods (Grosnit et al., 2021;
Tripp et al., 2020) using the same DAE (LOL-BO (JTVAE)) and the new SELFIES
VAE (LOL-BO (SELFIES)).

We apply LOL-BO to six high-dimensional, structured BO tasks over molecules
(Irwin & Shoichet, 2005; Irwin et al., 2020) and arithmetic expressions. We
additionally investigate the impact of various components of our method in
subsection 5.4 and subsection 5.5. As a roadmap for this section, log P and
Expressions (subsection 5.1) are tasks that are typically evaluated in low
budget settings. The Guacamol benchmarks (subsection 5.2) afford larger
function evaluation budgets, and the protein docking task (subsection 5.3) is
both low budget and the oracle is expensive. Error bars in all plots show
standard error over runs.

#### Deep autoencoder models.

A variety of deep autoencoder models have been used for the benchmarks tasks
we consider below. For the expressions task, we follow recent work (Kusner et
al., 2017; Grosnit et al., 2021; Tripp et al., 2020; Notin et al., 2021) in
using a GrammarVAE model (Kusner et al., 2017). For the remaining _de novo_
molecule design tasks, we consider two models.

#### Junction Tree VAE (JTVAE).

Molecules are commonly represented initially using the Simplified Molecular-
Input Line-Entry System (SMILES) (Weininger, 1988) string representation.
SMILES representations are well-known to be brittle, in that single character
mutations can easily result in strings that no longer encode a valid molecule.
The JTVAE model Jin et al. (2018) addresses the problem of generating valid
strings by using a molecular graph structure to build each molecule out of
valid sub-components so that the generated molecules are almost always valid.
As a result, the JTVAE has been widely used as a VAE model used for latent
space optimization due to its much higher rate of decoding to valid molecules
(Grosnit et al., 2021; Tripp et al., 2020; Deshwal & Doppa, 2021; Notin et
al., 2021). We follow prior work, using a latent dimensionality of 56.

#### SELFIES VAE

The SELFIES string representation for molecules is a recently proposed
alternative to the SMILES string representation (Krenn et al., 2019). Notably,
nearly every permutation of SELFIES tokens encodes a valid molecule. As a
result, purely sequence based models using the SELFIES representation are
capable of generating valid molecules at a rate competitive with the JT-VAE
while avoiding expensive computation involving graph structures.

In our molecule design experiments below, we therefore evaluate a VAE
architecture based on the SELFIES string representation in addition to the
commonly used JTVAE. We train a VAE model with 6 transformer encoder and
transformer decoder layers and a latent space dimensionality of 256 (Vaswani
et al., 2017). We use our SELFIES VAE for all molecular optimization tasks
below. We also use the JTVAE model to enable direct comparison to prior work.

#### Implementation details.

We implement LOL-BO leveraging BoTorch (Balandat et al., 2020) and GPyTorch
(Gardner et al., 2018). Code and model weights are available at
https://bit.ly/3KJNWP7.

#### Hyperparameters.

For the trust region dynamics, all hyperparameters including the initial base
and minimum trust region lengths $L_{\textrm{init}},L_{\textrm{min}}$, and
success and failure thresholds $\tau_{\textrm{succ}},\tau_{\textrm{fail}}$ are
set to the TuRBO defaults as used in Eriksson et al. (2019). Our method
introduces only one new hyperparameter, $\tau_{\textrm{retrain}}$, which we
set to $10$ in all experiments.

###  5.1 log P and Expressions

In this section, we consider two tasks commonly used in the latent space BO
literature–optimizing the penalised water-octanol partition coefficient (log
P) over molecules, and generating single variable arithmetic expressions
(e.g., $x\times sin(x\times x)$) (Grosnit et al., 2021; Tripp et al., 2020;
Deshwal & Doppa, 2021; Notin et al., 2021; Kusner et al., 2017; Ahn et al.,
2020). Because of the small evaluation budget typically considered for these
benchmarks, we use a batch size of $1$ for each iteration of BO.

#### Baselines.

For these tasks, we compare to standard latent space BO, as well as a variety
of recent prior work in this area. We compare to LS-BO with weighted
retraining (W-LBO) as described in Tripp et al. (2020) and LS-BO with triplet
loss retraining (T-LBO) from Grosnit et al. (2021).

A number of techniques outside of the BO literature consider this task as
well. These papers report final objective values $\geq 30$, but do not
consider sample efficiency. For example, GEGL (Ahn et al., 2020), which we do
compare to in subsection 5.2, uses more than 500 evaluations per iteration.
See Ahn et al. (2020) for an evaluation on log P.

#### Results.

In Figure 1, we plot the cumulative best objective value for the penalized log
P and expressions tasks, averaged over three runs. For log P, LOL-BO using the
SELFIES transformer VAE achieves an average log P score of 545.17 after 500
iterations. Using a JTVAE, which is more directly comparable to prior work, we
still achieve an average score of 123.54–higher than previous state-of-the-art
scores using latent space BO of 26.11 (Grosnit et al., 2021). For the
expressions task, LOL-BO achieves a final regret of 0.079 on average, versus
0.194 achieved by T-LBO.

#### Molecule quality.

While molecules found using LOL-BO for the log P task are “valid” according to
software commonly used to compute these scores, the molecules produced by this
search clearly abandon any notion of reality. A number of methods have been
proposed to evaluate molecule quality (Brown et al., 2019), and indeed even
some recent work in LS-BO has explored encouraging higher quality solutions
(Notin et al., 2021). However, our primary goal in this paper is to develop
stronger optimization routines in this space, and indeed the baselines we
consider also treat log P as an unconstrained optimization problem. Therefore,
this result should be taken primarily as evidence that the log P objective can
be exploited by a sufficiently strong optimizer, rather than as evidence of
novel interesting molecules.

###  5.2 Guacamol Benchmark Tasks

Figure 2: Optimization on three Guacamol benchmark tasks. We compare to T-LBO,
the most competitive latent space BO method from above, using the same DAE
((LOL-BO (JTVAE))) as well as the new SELFIES transformer ((LOL-BO
(SELFIES))). We additionally compare to two recent orthogonal methods for
these tasks that do not use deep autoencoders or BO (GraphGA, GEGL).

Figure 3: We evaluate (1) the gain in optimization performance achieved in
moving from global BO to local (Standard LS-BO vs TuRBO), and (2) the gain in
optimization performance achieved adapting the latent space to be suitable for
the trust region method (TuRBO vs LOL-BO). Improving the optimization
algorithm results in significant improvements using either the JT-VAE or the
SELFIES VAE.

The Guacamol benchmark suite (Brown et al., 2019) contains scoring oracles for
a variety of molecule design tasks, with scores ranging between 0 and 1–1
indicating the task can be solved perfectly. We select three tasks among the
most challenging in terms of the best objective value achieved: Perindopril
MPO, Ranolazine MPO, and Zaleplon MPO. The goal of each of these tasks is to
design a molecule with high fingerprint similarity to a known target drug but
that differs in a specified target way. The task definitions are discussed in
Brown et al. (2019).

#### Baselines.

We compare to T-LBO, the most competitive Bayesian optimization method overall
from subsection 5.1. Additionally, a variety of techniques from beyond the BO
literature using reinforcement learning and genetic programming have been
evaluated on these tasks. We choose to compare to GEGL and GraphGA, with GEGL
(Ahn et al., 2020) proposed recently and GraphGA (Jensen, 2019) among the top
methods on the Guacamol leaderboard. We also report the best score achieved
over the $\approx$1.27 million molecules in the full Guacamol dataset, which
serves as a performance cap for virtual screening approaches (see section 4).

#### Results.

The results of optimization on these three benchmarks are shown in Figure 2.
We again report results for LOL-BO with both a JTVAE and SELFIES VAE deep
autoencoder model. LOL-BO with a SELFIES VAE optimizes significantly faster
than other methods. Note that GEGL does eventually achieve scores competitive
with LOL-BO(SELFIES), but requires millions of oracle calls to do so – see Ahn
et al. (2020) for details.

Table 1: Comparing SELFIES VAE and JTVAE on metrics computed over molecules in
the GuacaMol test dataset (Brown et al., 2019). Decode stddev is the standard
deviation in Perindopril MPO objective value when decoding a fixed latent
$\mathbf{z}$ 20 times, averaged over a subset of 3500 latent vectors from the
dataset.
| SELFIES VAE | JTVAE
---|---|---
Reconstruction Acc. | 0.913 | 0.767
Avg. Decode Time (s) | 0.02 | 3.14
Decode Stddev | 0.04 | 0.1
Validity | 1.0 | 1.0
GP Test NLL (Perindopril) | -2.156 | -1.481
GP Test NLL(Zaleplon) | -2.705 | -1.781
GP Test NLL (Ranolazine) | -2.127 | -1.649

###  5.3 DRD3 Receptor Docking Affinity

Table 2: Docking scores achieved by various methods after the four evaluation
budgets considered on the TDC leaderboard (lower is better). We compare to
three BO baselines and a variety of orthogonal approaches from the TDC
leaderboard.

Method | Best Score Found in Evaluation Budget
---|---
| 100 evals | 500 evals | 1000 evals | 5000 evals
LOL-BO | $\bf{-13.1\pm 0.3}$ | $\bf{-13.3\pm 0.1}$ | $\bf{-13.8\pm 0.5}$ | $-15.4\pm 1.3$
TuRBO | $-12.6\pm 0.0$ | $-12.7\pm 0.3$ | $-12.9\pm 0.4$ | $-13.0\pm 0.4$
T-LBO | $-12.6\pm 0.0$ | $-12.6\pm 0.0$ | $-12.6\pm 0.0$ | $-12.8\pm 0.2$
LS-BO | $-12.6\pm 0.0$ | $-12.6\pm 0.0$ | $-12.6\pm 0.0$ | $-12.6\pm 0.0$
Graph-GA | $-11.8\pm 1.1$ | $-12.5\pm 0.7$ | $-13.2\pm 0.7$ | $\bf{-16.5\pm 0.3}$
SMI-LSTM | $-11.1\pm 0.6$ | $-11.4\pm 0.6$ | $-12.0\pm 0.2$ | $-14.5\pm 0.5$
MARS | $-9.1\pm 0.7$ | $-9.8\pm 0.3$ | $-11.1\pm 0.1$ | $-11.4\pm 0.5$
MolDQN | $-7.0\pm 0.2$ | $-7.6\pm 0.2$ | $-7.8\pm 0.04$ | $-10.0\pm 0.2$
GCPN | $-11.6\pm 2.2$ | $-12.0\pm 0.7$ | $-12.0\pm 0.6$ | $-12.3\pm 1.0$

The Therapeutics Data Commons (TDC) benchmark suite (Huang et al., 2021)
contains oracles that evaluate the binding propensity of a ligand (small
molecule) to a target protein. Unlike the oracles considered in subsection 5.2
and subsection 5.1, these docking oracles incur nontrivial computation cost,
requiring on the order of minutes to evaluate, making them ideal benchmark
tasks for Bayesian optimization. In this paper, we focus on designing ligands
that bind to dopamine receptor D3 (DRD3), as the TDC website maintains a
leaderboard of results for DRD3
333https://tdcommons.ai/benchmark/docking_group/drd3/.

#### Baselines.

We compare LOL-BO on the DRD3 docking task to the methods presented on the TDC
leaderboard and three BO methods–T-LBO, standard LS-BO, and fixed latent space
TuRBO on this task.

#### Results.

Results of optimization are in Table 2. LOL-BO achieves the best performance
among the BO baselines, with only TuRBO making significant progress on the
task by 5000 evaluations. Compared to the baselines from the TDC leaderboard,
LOL-BO is significantly more sample efficient, achieving results comparable to
the best method with a factor of $10\times$ fewer oracle calls. At the top end
of the evaluation budget, Graph-GA (Jensen, 2019) outperforms LOL-BO,
suggesting there is still room for improvement.

###  5.4 Benefits of adaptive latent spaces

In this section, we evaluate the components of LOL-BO described in subsection
3.2 beyond the direct application of TuRBO in the latent space described in
subsection 3.1. We run TuRBO on the three Guacamol tasks considered in
subsection 5.2 using a fixed latent space extracted by the pretrained SELFIES
VAE and JTVAE. To run TuRBO, we use the same experimental setup as for LOL-BO,
but $\Phi(\cdot)$ and $\Gamma(\cdot)$ are fixed at the start of optimization.

We compare LS-BO, TuRBO, and LOL-BO on all three tasks in Figure 3. LOL-BO
significantly outperforms TuRBO across all DAE models and tasks. While these
results demonstrate the impact of aligning locality in the input space
$\mathcal{X}$ with the latent space $\mathcal{Z}$, it is also interesting to
note that TuRBO in a static, pre-trained latent space is a very strong
baseline that already consistently outperforms prior state-of-the-art. This
result emphasizes that optimization problems remain high-dimensional in the
latent space.

###  5.5 Comparing the SELFIES VAE and JTVAE

LOL-BO achieves better performance with the SELFIES VAE than with the JTVAE.
Here, we evaluate a series of metrics for both DAE models. In Table 1, we
compare reconstruction error and validity, as well as the standard deviation
in Perindopril MPO score from decoding a fixed latent vector $\mathbf{z}$ 20
times, averaged over 3500 distinct $\mathbf{z}$. We report supervised learning
performance for a GP trained on $10,000$ molecules taken from the initial
Guacamol dataset, using the _pretrained_ latent spaces. In Figure 4, we plot a
histogram of the objective value variance, as well as the average time taken
to decode with each model.

Across standard metrics, the SELFIES VAE achieves excellent performance.
Beyond these, the improvement in objective variance for a single $\mathbf{z}$
is notable: because Guacamol objective values are in the range $[0,1]$, a
standard deviation of $0.1$ represents significant additional noise. Thus, the
DAE can be a significant source of noise even for an otherwise deterministic
objective function.

Figure 4: (_Left_) Histogram of variance in Perindopril MPO objective value
across 20 molecules decoded from a single a latent code $\mathbf{z}$.
(_Right_) Time taken to decode a batch of points.

##  6 Conclusion

Designing latent spaces for Bayesian optimization is challenging because the
VAE and the optimizer often compete: too small of a latent space renders the
VAE incapable of capturing the structure present in the input space, while too
large a latent space makes optimization challenging.

The empirical results we present here demonstrate the importance of viewing
these latent spaces as high-dimensional and tackling optimization with high-
dimensional BO strategies, even though these latent spaces are often
significantly smaller than the input space. Furthermore, we show that high-
dimensional BO methods can be significantly improved when translating them to
the latent space setting, demonstrating the potential of further research at
the intersection of these areas.

## References

  * Ahn et al. (2020) Ahn, S., Kim, J., Lee, H., and Shin, J.  Guiding deep molecular optimization with genetic exploration, 2020.
  * Balandat et al. (2020) Balandat, M., Karrer, B., Jiang, D. R., Daulton, S., Letham, B., Wilson, A. G., and Bakshy, E.  BoTorch: A Framework for Efficient Monte-Carlo Bayesian Optimization.  In _Advances in Neural Information Processing Systems 33_ , 2020.  URL http://arxiv.org/abs/1910.06403.
  * Brown et al. (2019) Brown, N., Fiscato, M., Segler, M. H., and Vaucher, A. C.  Guacamol: Benchmarking models for de novo molecular design.  _Journal of Chemical Information and Modeling_ , 59(3):1096–1108, Mar 2019.  ISSN 1549-960X.  doi: 10.1021/acs.jcim.8b00839.  URL http://dx.doi.org/10.1021/acs.jcim.8b00839.
  * Coley et al. (2020) Coley, C. W., Eyke, N. S., and Jensen, K. F.  Autonomous discovery in the chemical sciences part i: Progress.  _Angewandte Chemie International Edition_ , 59(51):22858–22893, 2020.
  * Dai et al. (2018) Dai, H., Tian, Y., Dai, B., Skiena, S., and Song, L.  Syntax-Directed variational autoencoder for structured data.  In _International Conference on Learning Representations 2018_ , 2018\.
  * Deshwal & Doppa (2021) Deshwal, A. and Doppa, J. R.  Combining latent space and structured kernels for Bayesian optimization over combinatorial spaces.  _CoRR_ , abs/2111.01186, 2021.  URL https://arxiv.org/abs/2111.01186.
  * Eissman et al. (2018) Eissman, S., Levy, D., Shu, R., Bartzsch, S., and Ermon, S.  Bayesian optimization and attribute adjustment.  In _Proc. 34th Conference on Uncertainty in Artificial Intelligence_ , 2018.
  * Eriksson & Jankowiak (2021) Eriksson, D. and Jankowiak, M.  High-dimensional Bayesian optimization with sparse axis-aligned subspaces.  _arXiv preprint arXiv:2103.00349_ , 2021.
  * Eriksson et al. (2019) Eriksson, D., Pearce, M., Gardner, J., Turner, R. D., and Poloczek, M.  Scalable global optimization via local Bayesian optimization.  In Wallach, H., Larochelle, H., Beygelzimer, A., d'Alché-Buc, F., Fox, E., and Garnett, R. (eds.), _Advances in Neural Information Processing Systems_ , volume 32. Curran Associates, Inc., 2019.  URL https://proceedings.neurips.cc/paper/2019/file/6c990b7aca7bc7058f5e98ea909e924b-Paper.pdf.
  * Gardner et al. (2017) Gardner, J., Guo, C., Weinberger, K., Garnett, R., and Grosse, R.  Discovering and exploiting additive structure for Bayesian optimization.  In _Artificial Intelligence and Statistics_ , pp. 1311–1319. PMLR, 2017.
  * Gardner et al. (2018) Gardner, J. R., Pleiss, G., Bindel, D., Weinberger, K. Q., and Wilson, A. G.  Gpytorch: Blackbox matrix-matrix Gaussian process inference with gpu acceleration.  _arXiv preprint arXiv:1809.11165_ , 2018.
  * Garnett et al. (2014) Garnett, R., Osborne, M. A., and Hennig, P.  Active learning of linear embeddings for Gaussian processes.  _Proceedings of the Thirtieth Conference on Uncertainty in Artificial Intelligence_ , 2014.
  * Gómez-Bombarelli et al. (2018) Gómez-Bombarelli, R., Wei, J. N., Duvenaud, D., Hernández-Lobato, J. M., Sánchez-Lengeling, B., Sheberla, D., Aguilera-Iparraguirre, J., Hirzel, T. D., Adams, R. P., and Aspuru-Guzik, A.  Automatic chemical design using a data-driven continuous representation of molecules.  _ACS central science_ , 4(2):268–276, 2018.
  * Graff et al. (2021) Graff, D. E., Shakhnovich, E. I., and Coley, C. W.  Accelerating high-throughput virtual screening through molecular pool-based active learning.  _Chemical science_ , 12(22):7866–7881, April 2021\.
  * Grosnit et al. (2021) Grosnit, A., Tutunov, R., Maraval, A. M., Griffiths, R., Cowen-Rivers, A. I., Yang, L., Zhu, L., Lyu, W., Chen, Z., Wang, J., Peters, J., and Bou-Ammar, H.  High-dimensional Bayesian optimisation with variational autoencoders and deep metric learning.  _CoRR_ , abs/2106.03609, 2021.  URL https://arxiv.org/abs/2106.03609.
  * Hensman et al. (2013) Hensman, J., Fusi, N., and Lawrence, N. D.  Gaussian processes for big data.  _Proceedings of the Twenty-Ninth Conference on Uncertainty in Artificial Intelligence_ , 2013.
  * Hernández-Lobato et al. (2017) Hernández-Lobato, J. M., Requeima, J., Pyzer-Knapp, E. O., and Aspuru-Guzik, A.  Parallel and distributed Thompson sampling for large-scale accelerated exploration of chemical space.  In Precup, D. and Teh, Y. W. (eds.), _Proceedings of the 34th International Conference on Machine Learning_ , volume 70, pp. 1470–1479. PMLR, 2017.
  * Huang et al. (2021) Huang, K., Fu, T., Gao, W., Zhao, Y., Roohani, Y., Leskovec, J., Coley, C. W., Xiao, C., Sun, J., and Zitnik, M.  Therapeutics data commons: Machine learning datasets and tasks for drug discovery and development.  _Proceedings of Neural Information Processing Systems, NeurIPS Datasets and Benchmarks_ , 2021.
  * Irwin & Shoichet (2005) Irwin, J. J. and Shoichet, B. K.  Zinc- a free database of commercially available compounds for virtual screening.  _Journal of chemical information and modeling_ , 45(1):177–182, 2005.
  * Irwin et al. (2020) Irwin, J. J., Tang, K. G., Young, J., Dandarchuluun, C., Wong, B. R., Khurelbaatar, M., Moroz, Y. S., Mayfield, J., and Sayle, R. A.  Zinc20—a free ultralarge-scale chemical database for ligand discovery.  _Journal of chemical information and modeling_ , 60(12):6065–6073, 2020.
  * Jensen (2019) Jensen, J.  A graph-based genetic algorithm and generative model/monte carlo tree search for the exploration of chemical space. chem sci 10 (12): 3567–3572, 2019\.
  * Jin et al. (2018) Jin, W., Barzilay, R., and Jaakkola, T. S.  Junction tree variational autoencoder for molecular graph generation.  In _International Conference on Machine Learning_. PMLR, 2018.
  * Kajino (2019) Kajino, H.  Molecular hypergraph grammar with its application to molecular optimization.  In _International Conference on Machine Learning_ , pp. 3183–3191. PMLR, 2019.
  * Kandasamy et al. (2015) Kandasamy, K., Schneider, J., and Póczos, B.  High dimensional Bayesian optimisation and bandits via additive models.  In _International conference on machine learning_ , pp. 295–304. PMLR, 2015.
  * Kingma & Welling (2013) Kingma, D. P. and Welling, M.  Auto-encoding variational Bayes.  _arXiv preprint arXiv:1312.6114_ , 2013.
  * Kingma et al. (2014) Kingma, D. P., Mohamed, S., Rezende, D. J., and Welling, M.  Semi-supervised learning with deep generative models.  In _Advances in neural information processing systems_ , pp. 3581–3589, 2014.
  * Kirkpatrick & Ellis (2004) Kirkpatrick, P. and Ellis, C.  Chemical space.  _Nature_ , 432(7019):823–823, December 2004.
  * Krenn et al. (2019) Krenn, M., Häse, F., Nigam, A., Friederich, P., and Aspuru-Guzik, A.  SELFIES: a robust representation of semantically constrained graphs with an example application in chemistry.  _CoRR_ , abs/1905.13741, 2019.  URL http://arxiv.org/abs/1905.13741.
  * Kusner et al. (2017) Kusner, M. J., Paige, B., and Hernández-Lobato, J. M.  Grammar variational autoencoder.  In _International Conference on Machine Learning_ , pp. 1945–1954. PMLR, 2017.
  * Lu et al. (2018) Lu, X., Gonzalez, J., Dai, Z., and Lawrence, N.  Structured variationally auto-encoded optimization.  In Dy, J. and Krause, A. (eds.), _Proceedings of the 35th International Conference on Machine Learning_ , volume 80 of _Proceedings of Machine Learning Research_ , pp. 3267–3275, Stockholmsmässan, Stockholm Sweden, 2018. PMLR.
  * Mockus (1982) Mockus, J.  The Bayesian approach to global optimization.  In _System Modeling and Optimization_ , pp. 473–481. Springer, 1982\.
  * Mouchlis et al. (2021) Mouchlis, V. D., Afantitis, A., Serra, A., Fratello, M., Papadiamantis, A. G., Aidinis, V., Lynch, I., Greco, D., and Melagraki, G.  Advances in de novo drug design: from conventional to machine learning methods.  _International journal of molecular sciences_ , 22(4):1676, 2021.
  * Nayebi et al. (2019) Nayebi, A., Munteanu, A., and Poloczek, M.  A framework for bayesian optimization in embedded subspaces.  In _International Conference on Machine Learning_ , pp. 4752–4761. PMLR, 2019.
  * Nelder & Mead (1965) Nelder, J. A. and Mead, R.  A simplex method for function minimization.  _The computer journal_ , 7(4):308–313, 1965.
  * Nigam et al. (2020) Nigam, A., Friederich, P., Krenn, M., and Aspuru-Guzik, A.  Augmenting genetic algorithms with deep neural networks for exploring the chemical space.  In _International Conference on Learning Representations_ , 2020.
  * Notin et al. (2021) Notin, P., Hernández-Lobato, J. M., and Gal, Y.  Improving black-box optimization in vae latent space using decoder uncertainty.  _Advances in Neural Information Processing Systems_ , 34, 2021.
  * Olivecrona et al. (2017) Olivecrona, M., Blaschke, T., Engkvist, O., and Chen, H.  Molecular de-novo design through deep reinforcement learning.  _Journal of cheminformatics_ , 9(1):48, September 2017.
  * Osborne et al. (2009) Osborne, M. A., Garnett, R., and Roberts, S. J.  Gaussian processes for global optimization.  In _3rd international conference on learning and intelligent optimization (LION3)_ , pp. 1–15, 2009.
  * Öztürk et al. (2020) Öztürk, H., Özgür, A., Schwaller, P., Laino, T., and Ozkirimli, E.  Exploring chemical space using natural language processing methodologies for drug discovery.  _Drug discovery today_ , 25(4):689–705, 2020\.
  * Samanta et al. (2019) Samanta, B., De, A., Jana, G., Chattaraj, P. K., Ganguly, N., and Gomez-Rodriguez, M.  NeVAE: A deep generative model for molecular graphs.  In _Proceedings of the AAAI Conference on Artificial Intelligence_ , volume 33 of _01_ , pp. 1110–1117, 2019.
  * Segler et al. (2018) Segler, M. H. S., Kogej, T., Tyrchan, C., and Waller, M. P.  Generating focused molecule libraries for drug discovery with recurrent neural networks.  _ACS Central Science_ , 4(1):120–131, 2018.  doi: 10.1021/acscentsci.7b00512.  URL https://doi.org/10.1021/acscentsci.7b00512.  PMID: 29392184.
  * Siivola et al. (2021) Siivola, E., Paleyes, A., González, J., and Vehtari, A.  Good practices for Bayesian optimization of high dimensional structured spaces.  _Applied AI Letters_ , 2(2):e24, 2021.
  * Snoek et al. (2012) Snoek, J., Larochelle, H., and Adams, R. P.  Practical Bayesian optimization of machine learning algorithms.  _Advances in neural information processing systems_ , 25, 2012.
  * Tripp et al. (2020) Tripp, A., Daxberger, E. A., and Hernández-Lobato, J. M.  Sample-efficient optimization in the latent space of deep generative models via weighted retraining.  In _Advances in Neural Information Processing Systems 33_ , 2020.
  * van Hilten et al. (2019) van Hilten, N., Chevillard, F., and Kolb, P.  Virtual compound libraries in Computer-Assisted drug discovery.  _Journal of chemical information and modeling_ , 59(2):644–651, February 2019.
  * Vaswani et al. (2017) Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, Ł., and Polosukhin, I.  Attention is all you need.  In _Advances in neural information processing systems_ , pp. 5998–6008, 2017.
  * Wang et al. (2016) Wang, Z., Hutter, F., Zoghi, M., Matheson, D., and de Feitas, N.  Bayesian optimization in a billion dimensions via random embeddings.  _Journal of Artificial Intelligence Research_ , 55:361–387, 2016.
  * Wang et al. (2018) Wang, Z., Gehring, C., Kohli, P., and Jegelka, S.  Batched large-scale Bayesian optimization in high-dimensional spaces.  In _International Conference on Artificial Intelligence and Statistics_ , pp. 745–754. PMLR, 2018.
  * Weininger (1988) Weininger, D.  SMILES, a chemical language and information system. 1. introduction to methodology and encoding rules.  _Journal of Chemical Information and Computer Sciences_ , 28(1):31–36, 1988.  doi: 10.1021/ci00057a005.  URL https://pubs.acs.org/doi/abs/10.1021/ci00057a005.
  * Williams et al. (2015) Williams, K., Bilsland, E., Sparkes, A., Aubrey, W., Young, M., Soldatova, L. N., De Grave, K., Ramon, J., de Clare, M., Sirawaraporn, W., Oliver, S. G., and King, R. D.  Cheaper faster drug development validated by the repositioning of drugs against neglected tropical diseases.  _Journal of the Royal Society, Interface / the Royal Society_ , 12(104):20141289, March 2015.
  * Wilson et al. (2016a) Wilson, A. G., Hu, Z., Salakhutdinov, R., and Xing, E. P.  Deep kernel learning.  In _Artificial intelligence and statistics_ , pp. 370–378. PMLR, 2016a.
  * Wilson et al. (2016b) Wilson, A. G., Hu, Z., Salakhutdinov, R. R., and Xing, E. P.  Stochastic variational deep kernel learning.  _Advances in Neural Information Processing Systems_ , 29:2586–2594, 2016b.
  * Xie et al. (2021) Xie, Y., Shi, C., Zhou, H., Yang, Y., Zhang, W., Yu, Y., and Li, L.  MARS: Markov molecular sampling for multi-objective drug discovery.  In _International Conference on Learning Representations_ , 2021.
  * You et al. (2018) You, J., Liu, B., Ying, Z., Pande, V., and Leskovec, J.  Graph convolutional policy network for Goal-Directed molecular graph generation.  In Bengio, S., Wallach, H., Larochelle, H., Grauman, K., Cesa-Bianchi, N., and Garnett, R. (eds.), _Advances in Neural Information Processing Systems 31_ , pp. 6410–6421. Curran Associates, Inc., 2018.
  * Zhang et al. (2019) Zhang, M., Jiang, S., Cui, Z., Garnett, R., and Chen, Y.  D-VAE: A variational autoencoder for directed acyclic graphs.  In _Advances in Neural Information Processing Systems 32_ , pp. 1588–1600. Curran Associates, Inc., 2019.
  * Zhou et al. (2019) Zhou, Z., Kearnes, S., Li, L., Zare, R. N., and Riley, P.  Optimization of molecules via deep reinforcement learning.  _Scientific reports_ , 9(1):10752, July 2019.