# Diffusion Models in De Novo Drug Design

Amira Alakhdar Email<EMAIL_ADDRESS>Department of Chemistry,
Carnegie Mellon University, Pittsburgh, PA, 15213, USA  Barnabas Poczos Email:
<EMAIL_ADDRESS>Department of Machine Learning, Carnegie Mellon
University, Pittsburgh, PA, 15213, USA  Newell Washburn Corresponding author.
Email<EMAIL_ADDRESS>Department of Chemistry, Carnegie Mellon
University, Pittsburgh, PA, 15213, USA  Department of Biomedical Engineering,
Carnegie Mellon University, Pittsburgh, PA, 15213, USA

###### Abstract

Diffusion models have emerged as powerful tools for molecular generation,
particularly in the context of 3D molecular structures. Inspired by non-
equilibrium statistical physics, these models can generate 3D molecular
structures with specific properties or requirements crucial to drug discovery.
Diffusion models were particularly successful at learning 3D molecular
geometries’ complex probability distributions and their corresponding chemical
and physical properties through forward and reverse diffusion processes. This
review focuses on the technical implementation of diffusion models tailored
for 3D molecular generation. It compares the performance, evaluation methods,
and implementation details of various diffusion models used for molecular
generation tasks. We cover strategies for atom and bond representation,
architectures of reverse diffusion denoising networks, and challenges
associated with generating stable 3D molecular structures. This review also
explores the applications of diffusion models in de novo drug design and
related areas of computational chemistry, such as structure-based drug design,
including target-specific molecular generation, molecular docking, and
molecular dynamics of protein-ligand complexes. We also cover conditional
generation on physical properties, conformation generation, and fragment-based
drug design. By summarizing the state-of-the-art diffusion models for 3D
molecular generation, this review sheds light on their role in advancing drug
discovery as well as their current limitations.

##  1 Introduction

Generative models have been increasingly integrated with molecular science in
modern drug discovery to help develop new therapeutics such as small molecules
[1], antibodies [2], gene therapy and mRNA vaccines [3]. The generation of
small molecules in the form of strings is well-established [1]. However, the
generation of 3D molecular structures is still lagging due to the complexity
of the shapes of molecules and the E(3) and SE(3) equivariance requirements
for any model, which hold the molecules identical under rotations and
translations [4]. The 3D geometry is the main determining factor of the
electrochemical properties of the molecule that consequently determines its
pharmacology, pharmacokinetics, metabolism, toxicity, and immune response by
governing its interaction with the biological target pocket as well as various
biological molecules such as (enzymes, receptors, antibodies, etc.). Hence,
learning molecules in the 3D space can improve probing of the chemical space
in structure-based drug design applications for possible protein and DNA
ligands. Moreover, it can help advance the material discovery by conditioning
on specific structural or quantum mechanics properties of the molecules.
Historically, computational molecular design or de novo drug design (DNDD) was
first approached by simpler methods such as growth and evolutionary algorithms
[5]. However, with the advancement of the deep generative models, they have
taken over and were used to generate molecules in 1D, 2D, and more recently,
in the 3D space. Several deep learning (DL) architectures were deployed for
that end, including recurrent neural networks (RNNs), variational autoencoder
(VAE), reinforcement learning (RL), generative adversarial network (GAN),
convolutional neural network (CNNs), and Graph Neural Networks (GNNs). RNNs
were popularly used with text-based representations such as SMILES and
SELFIES, where they run in an “autoregressive” way to predict the next token
of the sequence representing the molecule. RNNs were also used to generate
molecular graphs in GraphNet [6], MolRNN [7], and MRNN [8], and those
sequence-based models were usually combined with more complex architectures
such as RL, VAE, and CNNs [9]. Moreover, VAE-based methods such as GraphVAE
[10], CGVAE [11], NeVAE [12], GAN-based models such as MOLGAN [13], GNN-based
models such as GraphINVENT [14], and flow-based models such as GraphNVP [15],
GraphAF [16], MoFlow [17] and GraphDF [18] were used to generate the adjacency
matrix of the 2D graph and in some cases, the 3D atomic coordinates. Several
algorithms were developed specifically to generate the molecules in the 3D
space, including autoregressive models or latent representation-based models
that can be decoded from a VAE, an equivariant normalizing flow, or a
diffusion process, and they are nicely summarized in a recently published
review by Baillif at al [4]. Other reviews have focused on the implementation
details for those models up to May 2022 [19]; however, given that the
advancement of diffusion models in generating 3D molecular structures has
advanced quickly after that, most of the models with unprecedented results
were not included in that review.
Inspired by non-equilibrium statistical physics, diffusion models were first
introduced in 2015 by Sohl-Dickstein et al to learn complex probability
distributions through forward and reverse processes [20]. However, they became
very popular in 2020 after achieving unprecedented results in image generation
[21], and many studies followed to improve on those results [22, 23, 24]
including the score-based method that was introduced by Song et al [25].
Diffusion models also became popular in graph generation and they were used
for several applications in computational biology and chemistry such as
confirmation generation, molecular docking, protein generation and modeling,
protein-ligand complex structure prediction, molecular generation, and
molecular dynamics [26, 27]. The first deployment of diffusion models for 3d
molecular generation was the E(3) Equivariant Diffusion Model (EDM) [28] where
they used an E(n) equivariant graph neural network (EGNN) developed originally
for discriminative tasks to denoise and learn molecular structures
distributions [29]. Subsequently, they became widely adapted for 3D molecular
generation tasks combined with GNNs or transformer-based models for encoding
and learning molecule structures [30].
In this review, we aim to summarize the technical implementation aspects of
the diffusion models used for 3D molecular generation in general and for
specific applications in drug discovery, such as de novo drug design. There
are several reviews in the literature about the use of deep generative models
for de novo drug design in general, and one of the very comprehensive examples
is a recently published review by Xie et al. [1]. There are also several
reports on the applications of diffusion models in bioinformatics and
computational biology [26, 27]. However, the literature lacks a review that
dives into the recently developed diffusion models and compares their
performance, evaluation methods, and implementation details. A related survey
covers diffusion model applications in chemistry, including drug discovery
until April 2023 [30]. They also covered antibody design, protein design, and
material design applications and the emerging challenges in the field. In this
review, we dive deeper into the representation and encoding of atoms and bonds
and how those models aimed to learn both in synchronicity during the reverse
diffusion to avoid generating unstable molecules with inconsistent atoms and
bond structures. We cover the different strategies used for forward diffusion,
the architectures employed for reverse diffusion, and the various applications
of those models in the drug discovery process. An overview of the molecular
generation process using diffusion models is shown in Figure. 1.

Figure 1: Overview of the process of generating molecules using diffusion
models. First, the relevant dataset is acquired, the molecules are expressed
in an appropriate molecular representation, and diffusion conditions are
determined. Next, the diffusion framework (DDPM, SGM, Score SDE) is selected,
and the forward and reverse diffusion strategies are designed. Denoising
architectures may include transformers, GNNs, CNNs, and hybrid architectures.
The output results are obtained, and the generated molecules are evaluated
using multiple evaluation metrics according to the specific task in the drug
discovery process [After[1]].

##  2 Diffusion Models

Diffusion models are probabilistic generative models that add noise to distort
the data and then reverse the process to generate samples. Current diffusion
model research revolves around three main formulations: denoising diffusion
probabilistic models (DDPMs), score-based generative models (SGMs), and models
motivated by stochastic differential equations (score SDEs). Research on
diffusion models focuses on improving several aspects of those three
formulations, such as faster and more efficient sampling, accurate likelihood
and density estimation, and handling data with special structures (e.g.,
permutation invariance, manifold structures, discrete data) [31]. The most
popular formulation for molecular generations and DNDD applications is DDPMs,
but other formulations are also used [27]. An illustration of the diffusion
process applied to a 3D molecular graph is shown in Figure. 2.

Figure 2: Overview of the diffusion process applied to 3D molecules. In the
forward diffusion process, noise is added gradually to molecules by sampling
from the distribution
$\mathbf{q(x_{t}|x_{t-1})=\mathcal{N}(x_{t};\sqrt{1-\beta_{t}}x_{t-1},\beta_{t}\mathbb{I})}$
where $\mathbf{\beta_{t}\in(0,1)}$ is a hyperparameter specified before model
training, $\mathbf{\mathbb{I}}$ is the identity matrix and
$\mathbf{t\in\\{1,2,\dots,T\\}}$ is the time step. To generate molecules,
starting from standard normal noise $\mathbf{x_{T}}$, samples are drawn from
the distributions $\mathbf{p_{\theta}(x_{t-1}|x_{t})}$ iteratively. Those
distributions are learned by the pretrained denoising neural networks.

###  2.1 Denoising Diffusion Probabilistic Models (DDPMs)

The denoising diffusion probabilistic model (DDPM) [21, 20] incorporates two
Markov chains: a forward chain that transforms data into standard Gaussian
noise and a reverse chain that converts noise back to data by learning
denoising transformations parameterized by deep neural networks. The
generation of new data points from the same distribution can be achieved by
sampling a random vector from the Gaussian distribution, followed by employing
the reverse chain for ancestral sampling. In formal terms, given a data
distribution $\mathrm{x}_{0}\sim q(\mathrm{x}_{0})$, the forward Markov
process will use a Gaussian perturbation transition kernel
$q(\mathrm{x}_{t}|\mathrm{x}_{t-1})$ to incrementally transform the data
distribution $q(\mathrm{x}_{0})$ into a tractable prior distribution
$q(\mathrm{x}_{T})$ where $T$ is the number of time steps as:

$q(\mathrm{x}_{t}|\mathrm{x}_{t-1})=\mathcal{N}(\mathrm{x}_{t};\sqrt{1-\beta_{t}}\mathrm{x}_{t-1},\beta_{t}\mathbb{I}).$
(1)

Here $\beta_{t}\in(0,1)$ is a hyperparameter specified before model training,
$\mathbb{I}$ is the identity matrix and $t\in\\{1,2,\dots,T\\}$ is the time
step. With $\alpha_{t}:=1-\beta_{t}$, and
$\bar{\alpha_{t}}:=\prod_{s=0}^{t}\alpha_{s}$, it is easy to see that we can
calculate $\mathrm{x}_{t}$ directly from $\mathrm{x}_{0}$ using the Gaussian
kernel:

$q(\mathrm{x}_{t}|\mathrm{x}_{0})=\mathcal{N}(\mathrm{x}_{t};\sqrt{\bar{\alpha_{t}}}\mathrm{x}_{0},(1-\bar{\alpha_{t}})\mathbb{I})$
(2)

This process is followed by a learnable reverse denoising chain that reverses
the time step and learns to retrieve the data gradually. The reverse chain
takes the prior distribution $p(\mathrm{x}_{T})\sim\mathcal{N}(0,\mathbb{I})$
and learnable transition kernel $p_{\theta}(\mathrm{x}_{t-1}|\mathrm{x}_{t})$
such that:

$q(\mathrm{x}_{t-1}|\mathrm{x}_{t})=\mathcal{N}(\mathrm{x}_{t-1};\mu_{\theta}(\mathrm{x}_{t},t)\mathrm{x}_{0},\Sigma_{\theta}(\mathrm{x}_{t},t))$
(3)

where $\theta$ denotes model parameters, and the mean
$\mu_{\theta}(\mathrm{x}_{t},t)$ and variance
$\Sigma_{\theta}(\mathrm{x}_{t},t)$ are parameterized by deep neural networks.
New samples can be generated by sampling from the Gaussian distribution
$p(\mathrm{x}_{T})$ and then iteratively sampling from the kernel in Eq.3
until we reach $t=1$. Parameters of the neural network $\theta$ are learned by
minimizing the Kullback-Leibler (KL) divergence between the distributions of
the forward $q(\mathrm{x}_{0},\mathrm{x}_{1},\dots,\mathrm{x}_{T})$ and the
reverse $p_{\theta}(\mathrm{x}_{0},\mathrm{x}_{1},\dots,\mathrm{x}_{T})$
Markov chains, which is equivalent to maximizing the variational lower bound
(VLB) of the log-likelihood of the data $\mathrm{x}_{0}$ as shown in Eq.4.

$\displaystyle\mathbb{E}[-\log p_{\theta}(\mathrm{x}_{0})]$ $\displaystyle\leq
KL(q(\mathrm{x}_{0},\mathrm{x}_{1},\dots,\mathrm{x}_{T})\|p_{\theta}(\mathrm{x}_{0},\mathrm{x}_{1},\dots,\mathrm{x}_{T}))$
(4)
$\displaystyle=-\mathbb{E}_{q(\mathrm{x}_{0},\mathrm{x}_{1},\dots,\mathrm{x}_{T})}[\log
p_{\theta}(\mathrm{x}_{0},\mathrm{x}_{1},\dots,\mathrm{x}_{T})]+\text{const}$
$\displaystyle=-\mathbb{E}_{q(\mathrm{x}_{0},\mathrm{x}_{1},\dots,\mathrm{x}_{T})}[-\log
p_{\theta}(\mathrm{x}_{T})-\sum_{t=1}^{T}\log\frac{p_{\theta}(\mathrm{x}_{t-1}|\mathrm{x}_{t})}{q(\mathrm{x}_{t}|\mathrm{x}_{t-1})}]+\text{const}$
$\displaystyle=-\mathcal{L}_{VLB}(\mathrm{x}_{0})$

Ho et al. [21] proposed an alternative $L_{VLB}$ loss by optimizing the neural
network for predicting the Gaussian noise instead, as shown in Eq. 5.

$\mathbb{E}_{t\sim\mathcal{U}[1,T],\mathrm{x}_{0}\sim
q(\mathrm{x}_{0}),\epsilon\sim\mathcal{N}(0,1)}[\lambda(t)\|\epsilon-\epsilon_{\theta}(\mathrm{x}_{t},t)\|^{2}]$
(5)

where $\lambda(t)$ denotes a positive weighting function, $x_{t}$ can be
calculated as
$\mathrm{x}_{t}=\sqrt{\bar{\alpha_{t}}}\mathrm{x}_{0}+\sqrt{1-\bar{\alpha_{t}}}\epsilon$,
$\epsilon_{\theta}$ is the neural network that will be trained to estimate the
Gaussian noise, and $\mathcal{U}[1,T]$ is a uniform distribution over the set
$\\{1,2,\dots,T\\}$.
DDPMs were adopted in a wide variety of molecular diffusion models such as EDM
[28], MiDi [32], GCDM [33], GeoDIff [34] and several others [35, 36, 27].

###  2.2 Score-Based Generative Models (SGMs)

The concept of (Stein) score (also known as a score or score function) is a
key quantity to score-based generative models. It is is defined as the
gradient of the log probability density $\nabla_{\mathrm{x}}\log
p(\mathrm{x})$ [25, 37]. Score-based generative models (SGMs), also known as a
noise-conditional score network (NCSN) [38], use a sequence of increasing
Gaussian noise to perturb the data, then train a deep neural network model
conditioned on noise levels to predict the score function.
To describe SGMs formally, we assume a data distribution $q(\mathrm{x}_{0})$,
and a sequence of increasing noise levels
$0<\sigma_{1}<\sigma_{2}<\dots<\sigma_{t}<\dots<\sigma_{T}$. The data will be
perturbed from $\mathrm{x}_{0}$ to $\mathrm{x}_{t}$ by the Gaussian noise
distribution
$q(\mathrm{x}_{t}|\mathrm{x}_{0})=\mathcal{N}(\mathrm{x}_{t};\mathrm{x}_{0},\sigma_{t}^{2}\mathbb{I})$
to get the noisy data densities
$q(\mathrm{x}_{1}),q(\mathrm{x}_{2}),\dots,q(\mathrm{x}_{T})$ where
$q(\mathrm{x}_{t}):=\int
q(\mathrm{x}_{t}|\mathrm{x}_{0})q(\mathrm{x}_{0})\mathrm{d}\mathrm{x}_{0}$,
and the score function the score function $\nabla_{\mathrm{x}_{t}}\log
q(\mathrm{x}_{t})$ will be estimated using a noise-conditioned deep neural
network $s_{\theta}(\mathrm{x},t)$. Therefore, the objective loss function
becomes:

$\displaystyle\mathbb{E}_{t\sim\mathcal{U}\|1,T\|,\mathrm{x}_{0}\sim
q(\mathrm{x}_{0}),\mathrm{x}_{t}\sim
q(\mathrm{x}_{t}|\mathrm{x}_{0})}\left[\lambda(t)\sigma_{t}^{2}\|\nabla_{\mathrm{x}_{t}}\log
q(\mathrm{x}_{t})-s_{\theta}(\mathrm{x}_{t},t)\|^{2}\right]$ (6)
$\displaystyle=$
$\displaystyle\mathbb{E}_{t\sim\mathcal{U}\|1,T\|,\mathrm{x}_{0}\sim
q(\mathrm{x}_{0}),x_{t}\sim
q(\mathrm{x}_{t}|\mathrm{x}_{0})}\left[\lambda(t)\|-\frac{\mathrm{x}_{t}-\mathrm{x}_{0}}{\sigma_{t}}-\sigma_{t}s_{\theta}(\mathrm{x}_{t},t)\|^{2}\right]+\text{const}$
$\displaystyle=$
$\displaystyle\mathbb{E}_{t\sim\mathcal{U}\|1,T\|,\mathrm{x}_{0}\sim
q(x_{0}),\epsilon\sim\mathcal{N}(0,1)}\left[\lambda(t)\|\epsilon+\sigma_{t}s_{\theta}(\mathrm{x}_{t},t)\|^{2}\right]+\text{const}$

Similar to Eq.5, $\lambda(t)$ is a positive weighting function, and $x_{t}$
can be calculated as $\mathrm{x}_{t}=\mathrm{x}_{0}+\sigma_{t}\epsilon$. By
comparing the loss functions in Eq.5 and Eq.6, we can see that the training
objectives of DDPMs and SGMs are equivalent. For sampling, starting from
Gaussian noise, SGMs use a sequential chain of
$s_{\theta}(\mathrm{x}_{T},T),s_{\theta}(\mathrm{x}_{T-1},T-1),\dots,s_{\theta}(\mathrm{x}_{0},0)$
to produce new data instances. Annealed Langevin dynamics (ALD) is one of the
most commonly used methods for sample generation in SGMs, but other methods,
such as stochastic differential equations, ordinary differential equations,
and their combinations with ALD have also been studied [31].

###  2.3 Stochastic Differential Equations (Score SDEs)

Score SDEs [25] are an extension of DDPMs and SGMs to include infinite time
steps or noise levels, with perturbation and denoising processes and they
involve solving stochastic differential equations (SDEs) where SDEs are used
for noise perturbation and sample generation, and the denoising is
accomplished by estimating the score function of noisy data distributions. The
data perturbation in Score SDEs-based diffusion is governed by the following
SDE:

$\mathrm{d}\mathrm{x}=\mathbf{f}(\mathrm{x},t)\mathrm{d}t+g(t)\mathrm{d}\boldsymbol{\mathrm{w}}$
(7)

where $\mathbf{f}(\mathrm{x},t)$ is the SDE diffusion function, $g(t)$ is the
SDE drift function, and $\boldsymbol{\mathrm{w}}$ defines the standard Wiener
process or Brownian motion. DDPMs and SGMs are both discretizations of this
SDE in time, and the SDE of DDPM can be represented in [25] as:

$\mathrm{d}\mathrm{x}=-\frac{1}{2}\beta(t)\mathrm{x}\mathrm{d}t+\sqrt{\beta(t)}\mathrm{d}\boldsymbol{\mathrm{w}}$
(8)

Any diffusion process, taking the form of Eq. 7 can be reversed by solving the
following SDE [39].:

$\mathrm{d}\mathrm{x}=\left[\mathbf{f}(\mathrm{x},t)-g(t)^{2}\nabla_{\mathrm{x}}\log
q_{t}(\mathrm{x})\right]\mathrm{d}t+g(t)\mathrm{d}\boldsymbol{\bar{\mathrm{w}}}$
(9)

Where $\boldsymbol{\bar{\mathrm{w}}}$ represents a standard Wiener process in
the reverse-time diffusion process, and $dt$ is an infinitesimal negative step
in the opposite time direction. The solution of the reverse SDE has the same
marginal densities as the forward SDE but in the opposite time direction.
After estimating the score function $\nabla_{\mathrm{x}}\log
q_{t}(\mathrm{x})$ at each time step $t$, the reverse-time SDE (eq. 9) is
solved and sampling can be achieved using numerical methods including
numerical SDE/ODE solvers [25, 40, 41, 42], annealed Langevin dynamics [38]
and combination of MCMC with those methods (predictor-corrector methods) [25].
Similar to SGMs, a time-dependent score neural network
$s_{\theta}(\mathrm{x},t)$ is trained to estimate the score function at each
time step using the following objective function in Eq. 10 which is a
generalization to the loss function in Eq. 6 to continuous time,

$\mathbb{E}_{t\sim\mathcal{U}[0,T],\mathrm{x}_{0}\sim
q(\mathrm{x}_{0}),\mathrm{x}_{t}\sim
q(\mathrm{x}_{t}|\mathrm{x}_{0})}\left[\lambda(t)\sigma_{t}^{2}\|s_{\theta}(\mathrm{x}_{t},t)-\nabla_{\mathrm{x}_{t}}\log
q_{t}(\mathrm{x}_{t}|\mathrm{x}_{0})\|^{2}\right].$ (10)

Here $\mathcal{U}[0,T]$ denotes the uniform distribution over $[0,T]$. Similar
to Eq.5 and Eq.6, $\lambda(t)$ is a positive weighting function, and $\theta$
are the trainable parameters of the neural network.
Score-based models, including SGMs and Score SDEs, have been used in several
diffusion models for molecule generation, such as DiffBridges [43], GDSS [44],
and several others [45, 46, 27].

##  3 Molecular Representations

Over the years, there have been several ways to represent molecules such as
coulomb matrices, molecular fingerprints, bag-of-bonds, International Chemical
Identifier (InChI), Simplified molecular-input line-entry system (SMILEs), and
graphs [1], however, diffusion models have only been reportedly used with
SMILEs, 2D and 3D graphs and achieved unprecedented results in molecular graph
generation [47, 27]. Hence, we will focus specifically on those two molecular
data representations. Also, an overview of the datasets commonly used in
molecular generation using the diffusion model is summarized in Table. 1.

SMILEs is a notation that translates a molecular structure into a one-
dimensional string of symbols. Although sequence-based autoregressive models
such as RNNs have achieved successful results with text-based representations,
including SMILEs and SELF-referencIng Embedded Strings (SELFIES), diffusion
models were investigated to apply SMILEs in more complex and tailored
applications. For example, the DIFFUMOL model [47] combined a diffusion model
with Transformer architecture to tokenize SMILEs and generate molecules with
specified scaffolds and properties.

Molecular Graph Representation has become a widely used representation in
generative models where the molecular structure is represented as a graph
$G=(V,E)$ with where nodes $v_{i}\in V$ represent the atoms and edges
$(v_{i},v_{j})\in E$ represent the bonds or interatomic interactions between
the atoms $v_{i}$ and $v_{j}$. In 2D graphs, bonds are represented as the
edges, and atom types are represented as node features. In contrast, molecules
represented in 3D graphs also have both the atom positions and atom types as
node features, while the edges can be explicitly represented as edge features
or implicitly encoded in the interatomic distances of the 3D atom coordinates
while the edge features represent other information such as interatomic
interactions in the fully connected graph as in the EDM model [28]. Both atom
types and Bond types are encoded using one-hot embedding, where each channel
represents an atom or bond type.

Dataset |  No. Molecules |  Data |  Task
---|---|---|---
QM9 [48] |  133,885 |  Organic molecules with up to nine atoms and their DFT calculated quantum chemical properties. |  Unconditioned and conditioned 3D molecular generation.
GEOM-DRUG [49] |  over 450,000 |  37 million molecular conformations for over 450,000 molecules. |  Unconditioned and conditioned 3D molecular generation.
ZINC250k [50] |  249,455 |  A drug-like subset of the ZINC database with bioactivity data for a portion of the molecules. |  Unconditioned and conditioned generation of drug-like molecules.
MOSES [51] |  1,936,962 |  filtered from ZINC Clean Leads collection. |  Unconditioned and conditioned generation of 2D molecules.
CrossDocked [52] |  18,450 complexes, 22.6 million poses |  A collection of ligand-protein complexes where ligands are docked against several protein targets similar to one another. |  SBDD tasks: target-aware 3D generation, molecular docking, etc.
PDBbind [53] |  23,496 complexes |  Biomolecular complexes in PDB with experimentally measured binding affinities, including 19,443 protein-ligand, 2,852 protein-protein, 1,052 protein-nucleic acid, and 149 nucleic acid-ligand complexes. |  protein-protein docking and SBDD tasks: target-aware 3D generation, molecular docking, etc.
Table 1: Datasets commonly used in molecular generation using diffusion models

##  4 Essential Requirements for Diffusion Models in Molecular Graph
Generation

E(3) Invariance and SE(3) Equivariance: The model is considered E(3)
equivariant when it is invariant to translations, rotations, and reflections
of the 3D structure of the molecule. At the same time, SE(3) are only
invariant to rotations and translations, which means that they are sensitive
to chirality, which alters the 3D geometry of a molecule [54]. The main
advantage of graph-based representation is they can easily meet the E(3)
equivariance requirement by combining with geometric graph neural networks
(GNNs) such as EGNN [29] and the rEGNN introduced in MiDi model [32] for
rotational invariance and centering the molecule by setting the center-of-mass
to zero for translational invariance.

Permutation Invariant Graph Generation: Graphs are permutation invariant,
meaning that they remain unchanged by permutation actions such as changing the
order of the rows or columns in the adjacency matrix or the order of nodes
corresponding to atoms and edges corresponding to bonds in the molecular graph
representation $G=(V,E)$. Therefore, permutation invariance is one of the main
requirements for any graph-based generation, including molecular generation.
Diffusion models overcame the permutation invariant graph generation issue,
surpassing autoregressive models that exhibit dependence on the sequence of
generated nodes [55].

Accounting for Discreteness: Accounting for Discreteness: Another critical
issue with the graph representation is that the atom-type and bond-type
features are discrete, making it challenging to use Gaussian noise diffusion
to train a denoising neural network to learn the distribution of the molecular
structures. Various forward and reverse diffusion methods applied to molecular
graphs will be discussed in sections 5 and 6, respectively.

Capturing Underlying Data Distributions: The generative model needs to learn
the underlying distribution of the data within the chemical space and generate
various data points accurately representing that distribution, including the
2D chemical graphs and their conformation in the 3D space. The molecular graph
(2D structure) dictates the distribution of atom and bond types and the
scaffolds and functional groups within compounds, and the 3D confirmation
represents more sophisticated distributions such as bond angles, dihedral
angles, and cases of stereo-isomerism.

Generated Sample Fidelity: Ensuring the validity of the generated molecules
can have so many levels, including the connectivity of the entire molecular
graph, chemical stability, and adherence to established rules of molecular
structure such as atom valency, possible ionic charges based on atoms’ groups
within the periodic table, stable ring structure and tolerable levels of angle
and torsional strains that can change according to the underlying data
distribution. For example, a set of drug-like molecules should have different
underlying distributions from more reactive transition states that occur
during a chemical synthesis. In section. 6, we will discuss how different
architectures were employed to improve the fidelity of the generated
molecules.

##  5 Forward Diffusion Process (Discrete vs Continuous)

In forward diffusion, the data is corrupted by injecting noise gradually until
it reaches standard Gaussian noise. Given the categorical variable nature of
atom and bond types, they are represented as discrete features in the
molecular graph after one-hot encoding. Suppose we have a categorical array
$h$ representing the categories ${c_{1},\dots,c_{d}}$. In that case, it can be
one-hot encoded to the array $h^{\text{onehot}}$ using the one-hot function
$h_{i,j}^{\text{onehot}}=\mathbbm{1}_{h_{i}=c_{j}}$ and then noise can be
applied to the array $h^{\text{onehot}}$ innforward diffusion. Yet, it can be
tricky to apply Gaussian noise to those features and then denoise the graph
and maintain the same distribution of those categories in the generated data.
Nonetheless, several models applied Gaussian noise to discrete features,
including EDM [28] that applies the continuous diffusion to the one-hot
encoded vector using a predefined noise scaling schedule
$q(z_{t}^{(h)}|h)=\mathcal{N}(z_{t}^{(h)};\alpha_{t}h^{\text{onehot}},\sigma_{t}^{2}\mathbb{I})$,
where a probability distributions parameter $p$ is defined to be proportional
to the normal distribution integrated from $1-\frac{1}{2}$ to $1+\frac{1}{2}$:

$\displaystyle p(h|z_{0}^{(h)})=\mathcal{C}(h|p),\quad
p\propto\int_{1-\frac{1}{2}}^{1+\frac{1}{2}}\mathcal{N}(\boldsymbol{u};z_{0}^{(h)},\sigma_{0})du$

Here $\mathcal{C}(h|p)$ is a categorical distribution, and $p$ is normalized
to sum to one. This ensures that one category will be active in each row of
$z_{0}^{(h)}$ and in practice, the parameters of the normal noise distribution
are tuned so that it is guaranteed that the sampled class from the reverse
diffusion process matches the original active atom type category.
Moldiff [56] follows a similar forward diffusion process. However, they add
one more none-type to the original element space and all-atom types or bond
types will gradually be perturbed to this new type. They call it the absorbing
type because atom or bond types are gradually absorbed to this specific type
at the end of the forward diffusion process. Therefore, this perturbed
probability distribution will be the starting point of the reverse process.
Also, the forward diffusion was carried out in two stages, focusing on
perturbing bond types in the first stage and atom types and coordinates in the
second stage.
Other models, such as GFMDiff [57], apply Gaussian noises with learnable
parameters controlling the strength of noises. Similarly, JODO [45] projects
the noise through learnable sinusoidal positional embeddings, and then it’s
used as a conditional feature in the denoising process. Several studies favor
continuous diffusion because it provides classifier-free guidance, i.e., an
explicit classifier is not needed to guide the generative process, leading to
more efficient training and sampling. Continuous diffusion can also provide
uncertainty modeling and efficient sampling algorithms with faster designs
based on advanced ODE solvers [58, 59].
On the other hand, other research suggested that discrete graph diffusion can
generate higher-quality samples with distributions closer to the original data
distribution indicated by the lower Maximum Mean Discrepancy (MMD) values
between the two distributions [60].MMD is a distance metric between two
probability distributions discussed formally in the evaluation metrics section
below. Discrete graph diffusion was also introduced in some molecular
generation models, such as DiGress [35], to get better marginal distributions
of node and edge types during diffusion. Simply, discrete forward diffusion is
defined in DiGress [35] as a categorical distribution applied to each node and
edge type using a transition probability matrix. Given a one-hot encoded nodes
(atoms) matrix $\mathcal{X}\in R^{n\times a}$ where $n$ is the number of nodes
(atoms), and $a$ is the number of atoms categories, and a one-hot encoded edge
tensor $\mathcal{E}\in R^{n\times n\times b}$ where $b$ is the number of edge
types including the absence of edge as a particular edge type. In that case,
the transition probabilities can be defined by the matrices
$[\mathbf{Q}^{t}_{\mathcal{X}}]{i}j=q(x^{t}=j|x^{t-1}=i)$ and
$[\mathbf{Q}^{t}_{\mathcal{E}}]_{ij}=q(e^{t}=j|e^{t-1}=i)$, and hence noise
can be applied to the graph $G^{t}=(\mathcal{X}^{t},\mathcal{E}^{t})$ by
sampling each node and edge type from a categorical distribution:

$\displaystyle
q(G^{t}|G^{t-1})=(\mathcal{X}^{t-1}\mathbf{Q}^{t}_{\mathcal{X}},\mathcal{E}^{t-1}\mathbf{Q}^{t}_{\mathcal{E}})\quad\text{and}\quad
q(G^{t}|G)=(\mathcal{X}\bar{\mathbf{Q}}^{t}_{\mathcal{X}},\mathcal{E}\bar{\mathbf{Q}}^{t}_{\mathcal{E}})$

Here
$\bar{\mathbf{Q}}^{t}_{\mathcal{X}}=\mathbf{Q}^{1}_{\mathcal{X}}\dots\mathbf{Q}^{t}_{\mathcal{X}}$
and
$\bar{\mathbf{Q}}^{t}_{\mathcal{E}}=\mathbf{Q}^{1}_{\mathcal{E}}\dots\mathbf{Q}^{t}_{\mathcal{E}}$.
The same discrete forward diffusion process was also used in a subsequent
model called MiDi [32]. However, it was combined with continuous diffusion for
the 3D coordinates corrupted with Gaussian noise. Also, MiDi followed an
adaptive noise schedule specifically tuned to make the model predict the bond
types and atom coordinates before the atom types in the denoising process
[32].

###  5.1 Latent Graph Diffusion

Some models tend to embed the data in continuous space to apply stable
diffusion, such as GEOLDM [61], which uses geometric autoencoders to convert
molecular structures into 3D equivariant latent features and then apply stable
diffusion in the latent space. Another model called 3M-Diffusion [62] used a
graph encoder to encode molecular graphs to a continuous space (latent space)
aligned with their corresponding text descriptions, and then employs a decoder
retrieve the molecular graph based on a given text descriptions. Hence, the
model can simultaneously learn molecular structures and their text description
and generate novel molecules based on a given textual description. An
illustration of latent space diffusion is shown in Figure. 3.

Figure 3: Overview of latent space diffusion process. First, the molecules are
encoded to a continuous latent space, then stable diffusion is applied on the
latent space. To generate molecules, they are first sampled from the latent
space, then retrieved to the original discrete space using the decoder [After
[61]].

##  6 Reverse Diffusion Denoising Neural Networks Architectures

The reverse diffusion is responsible for learning the distribution of the data
using a denoising neural network architecture that gradually removes the noise
added in the forward diffusion. Several architectures of denoising neural
networks have been investigated for molecular generation in combination with
diffusion models, and those architectures fall into one of the three
categories that will be discussed in this section: transformers, GNNs, and
CNNs (Figure. 4 and Table. 2) or a combination of them. EDM also managed to
generate molecules conditioned on prosperity $c$ by adding the condition as an
input from which the EGNN could learn.

Figure 4: Simple illustrations of the neural network architectures commonly
used in reverse diffusion: A- A-transformers, B-GNNs, and C-CNNs (in 3D).

###  6.1 Graph Neural Networks (GNNs)

GNNs [63] is a type of deep learning architecture specifically designed to
work with data structured as graphs by passing messages between nodes,
allowing each node to understand its neighbors and the broader network. Hence,
they can operate on graph representations of molecules and have been widely
adopted in predicting molecular properties and deep generative molecular
models. Several variations of GNNs were also used in different models such as
Graph Diffusion via the System of Stochastic differential equations (GDSS)
[44], E(n) Equivariant Graph Neural Networks (EGNNs) [29], Geometry-Complete
Perceptron Neural Networks (GCPNET) [64] and ShapeMol [65]. GNNs have also
been used to encode the discrete graph structures of molecules in the
3M-Diffusion model [62] where they employ Graph Isomorphism Networks (GINs)
[66] to map molecules to a continuous latent space.

####  6.1.1 EGNNs

The EGNNs model developed originally for discriminative tasks, is the most
popular GNNs architecture used in molecular generation using diffusion models
[29]. It was first used in the EDM model to generate 3D molecules by message-
passing through several Equivariant Convolutional Layers (EGCL) [28]. In the
EDM model, the EGNNs used a fully connected graph $G$ with nodes $v_{i}\in V$,
and Each node $v_{i}$ has the coordinates $x_{i}$ as its features, and hence
the covalent bonds were not stated explicitly but inferred from the 3D
coordinates. Implicit representation of covalent bonds caused several issues
because the model cannot learn the valency of different atom types and the
constraint relations between adjacent atoms that govern the bond formation and
distributions. This issue was more clear in datasets of large drug-like
molecules such as the GEOM-DRUGs [49] with up to 181 atoms and 44.4 atoms on
average (24.9 heavy atoms) that generates more unstable molecules compared to
the QM9 dataset [48] with up to 29 atoms (9 heavy atoms) and an average of 18
atoms per molecule. EDM also managed to generate molecules conditioned on
prosperity $c$ by adding the condition as an input from which the EGNNs can
learn.
Several attempts have been made to improve various aspects of the EDM model
relying on the same EGNN with modified models. For example, EEGSDE [67] used
an SDE framework and added the guidance of energy function according to
consistency between the molecule $z$ and the property $c$, which can help
capture the dependency between the two variables and generate molecules with
desired properties. DiffBridges [43] used a similar approach by designing
physically informed diffusion bridges based on a Lyapunov function method. An
energy function was introduced to the diffusion bridge inspired by the AMBER
force field [68] and molecular geometric statistics such as bond lengths, bond
angles, torsional angles, etc, calculated from the data. The DiffBridges
achieved better results on the QM9 dataset than EDM. However, the issue with
inconsistent atom types and bonds leading to unstable molecules remained.
The MolDiff [56] model was developed to solve this issue of atom-bond
inconsistency in EGNNs-based reverse diffusion, and hence, a two-stage
diffusion model was implemented. In the first stage, bond types are perturbed
with higher noise than atom types and positions that are only slightly
perturbed. Then, the noised atom types and positions are used to recover bonds
from the atom’s information, and by that, the model learns the valency of
molecules by relating bond types to atom types and coordinates, and the slight
noise adds robustness to the model. In the following time step, atoms are
adjusted based on the recovered atoms before bonds and are perturbed again
until all bonds are labeled as the absorbing type. In the second stage, atom
types and coordinates are perturbed until they reach the prior distribution of
the none-type. Moldiff generated a higher percentage of valid molecules than
EDM; however, it did not report on the stability of the generated 3D
structures and atoms’ valencies.
HierDiff [36] was designed to generate higher-quality drug-like molecules, and
it followed a completely different strategy to solve the atom-bond
inconsistency issue in local environments using a hierarchical coarse-grained
generation model where each node encodes a fragment. First, a vanilla EGNN
obtains the embeddings for all links and nodes, and then the nodes are
assembled to generate the 3d molecule using bottom-up or top-down EGNN that
predicts the links between those fragments. The coarse-grained nodes are
subsequently decoded into fine-grained fragments using another two EGNN
modules: a message passing bottom-up EGNN and an iterative refined sampling
bottom-up EGNN module. Similarly, GEOLDM used EGNN modules to implement the
encoder, latent diffusion, and decoder [61]. HierDiff and GEOLDM avoided the
atom-bond inconsistency issue by embedding the structure into smaller domains
using EGNNs.

####  6.1.2 GCPNET of the GCDM

GCPNET [64] is another SE(3)-equivariant graph neural network designed for
discriminative tasks. It showed superior performance in 3D geometry-dependant
predictive tasks on 3D molecular graphs such as protein-ligand binding
affinity and chirality recognition on the rectus/sinister (RS) 3D molecular
dataset [54]. It was repurposed for generative tasks in the GCDM model [33].
Like EGNNs, message passing is carried out through geometry-complete graph
convolution layers called GCPConv; however, those layers are preceded by a
Geometry-Complete Perceptron embedding layer $\textbf{GCP}_{e}$ to encode the
input node and edge features into scalar and vector-valued values. The GCPConv
layers are also followed by another GCP that outputs the final predictions of
$\textbf{GCP}_{p}$. Although GCPNET has several advantages over EGNN, such as
supporting geometry-complete and chirality-aware message-passing, it failed to
improve the 3d stability of larger molecules such as the GEOM-DRUGs dataset
[49] in the GCDM model [33].

####  6.1.3 ShapeMol

ShapeMol generates 3D molecules conditioned on molecular shape using two GNNs:
an invariant graph neural network, denoted as INV-GNN, to predict atom
features and an equivariant graph neural network, denoted as EQ-GNN to predict
atomic coordinates [65].

####  6.1.4 MACE of the SiMGen Model

The MACE model [69] is an equivariant message-passing neural networks (MPNNs)
architecture designed to create computationally efficient and accurate ML-
based force fields. Hence, it was used to extract features from the SPICE [70]
dataset with 1 million molecules of sizes ranging from 3 to 100 atoms for the
zero-shot molecular generation model, SiMGen [71]. The QM force field features
were combined with time-varying local similarity Kernels to define a score-
based diffusion model that can generate molecules of any arbitrary size and
execute conditional generation without altering the model.

####  6.1.5 LigandDiff

LigandDiff [72] is a conditional diffusion model aiming to generate 3D
transition metal complexes employing message-passing neural networks (MPNNs)
for denoising combined with Geometric vector perceptrons (GVPs) for embedding
molecules into molecular representation. LigandDiff conditionally generates
molecules under a fixed context; for example, the number of heavy atoms and
the transition metal used can be used as a condition.

###  6.2 Convolutional Neural Networks (CNN)

CNNs are deep neural networks especially proficient at processing data
organized into a grid-like structure, like images. CNNs are frequently
utilized for computer vision applications, including object detection, image
classification, and image recognition [73].

####  6.2.1 SchNet of the MDM Model

SchNet [74] is a continuous-filter CNN that learns a representation of atoms
in a molecule analogous to pixels in an image where atoms are embedded based
on the atom type with three interaction blocks modeling the interatomic
interactions. The model is equivariant and rotational invariant because it
uses interatomic distances to model molecules in the filter network. SchNet
[74] was adapted in the MDM model [75] to generate 3D molecular structures. It
was used to extract node embeddings along with Dual Equivariant Score Neural
Networks: one operates on the molecular graph to tackle covalent bonding
interactions using local edges, and one operates on the fully connected
molecular graph to tackle interatomic van der Waals forces using global edges.
MDM [75] managed to improve the 3d stability of the GEOM-DRUGs [49] dataset
compared to the EDM model.

####  6.2.2 3D U-Net of the VoxMol model

The VoxMol [76], a score-based model, employed a CNN architecture designed for
volumetric segmentation tasks called 3D U-Net for the denoising process. The
3D U-Net is an expansion of the wildly successful conventional U-Net
architecture for 2D image segmentation [77]. The atomic densities of atoms are
depicted as continuous Gaussian-like values within a 3D grid space where each
atom type (element) is represented by a different grid channel. Molecules are
then generated by discretizing the 3D space around the atoms into voxel grids.
Voxelized molecules can later be recovered from the generated voxel grids
using a simple peak detection algorithm. Given the voxelized representation of
molecules, the CNNs-based VoxMol model was able to achieve remarkable results
with the GEOM-DRUGs dataset [49] in terms of 3D molecular stability.

###  6.3 Transformers

Transformers are a type of neural network design that has gained popularity in
natural language processing (NLP). Typical transformer architecture consists
of multi-head attention (MHA), layer normalization (LN), and feed-forward
networks (FFN). Their ability to capture long-range dependencies between word
sequences using self-attention mechanisms allows them to analyze the
relationships between different parts of a text input simultaneously [78].
Similarly, they have shown a significant ability to capture long-range
dependencies between atoms and bonds and, hence, learn the 3D molecular
structures efficiently. Message-passing transformers can be tailored for
graph-structured data like molecules and have been employed in several
denoising architectures such as JODO [45] and MiDi [32] that were highly
successful in the 3D graph generation compared to the early EGNN-based models
such as EDM [28]. Moreover, in the DIFFUMOL [47] model, the transformer
architecture was used to model the semantic relationship between the user-
guided instructions and the generated molecules.

####  6.3.1 Diffusion Graph Transformer (DGT) of the JODO Model

The JODO model [45] employs a DGT for parameterizing the data prediction
architecture. DGT adopts the typical transformer architecture; however, it
interacts intricately with node (atoms) and edge (bonds) representations to
learn the dependencies between them. DGT employs an adaptive layer
normalization (AdaLN) [79] to project edge and atom features to continuous
embeddings. Moreover, the model uses a self-conditioning mechanism [59] where
predictions from the previous sampling step are utilized as an extra condition
where a 2D adjacency matrix and a 3D adjacency matrix calculated from distance
cut-offs are derived from the previous sampling step to serve as a condition
for the ongoing sampling step, therefore enhancing the model’s capture the
precise graph discreteness along with its connectivity and spatial arrangement
dependencies on atom types. Besides self-conditioning, they used different
methods to augment the data, such as the m-step random walk matrix from the
discrete adjacency matrix and discrete atomic features. JODO achieved highly
successful results with conditional and joint unconditional 2D and 3D
molecular graph design [45].

####  6.3.2 Dual Track Transformer Network (DTN) of the GFMDiff Model

DTN, the E(n) equivariant denoising kernel of the GFMDiff [57] model, is
designed to specifically learn the 3D geometry of the molecules by taking
embedding inputs derived from atom features, pairwise distance features, and
triple-wise angle features, and predicts atom types, valencies, and
coordinates as an output. The spatial information was used to learn multi-body
interactions among atoms. Moreover, the GFMDiff model introduced a new loss
function, the Geometric-Facilitated Loss (GFLoss), that takes the valencies
predicted by the DTN and aims to minimize the difference between those
valencies and valencies calculated from molecular geometries. This
architecture of the DTN, combined with the GFLoss, demonstrated a notable
efficiency in learning atomic valencies and achieved favorable molecular and
atomic stabilities of the generated molecules.

####  6.3.3 Molecule Unified Transformer (MUformer) of the MUDiff Model

Similar to the DGT of the JODO model [45], MUformer [80] is an E(n)
equivariant transformer architecture that jointly learns the 2D and the 3D
graph structures of the molecule. It relies on encoding atomic, positional,
and structural information using six encoding functions, three of which are
message-passing based. Those six channels generate atom, bond, and graph
encodings besides 2D and 3D neighborhood encoding that combine into two
channels: one that represents 2D molecular structures and predicts atoms and
edge features and one that represents the 3D geometric structure and predicts
the atom features and the 3D geometry. Finally, an output network merges the
atom features from the two channels and outputs the final molecule. The
MUformer successfully generated valid and stable 3D geometries and property
prediction tasks on the QM9 dataset [48].

####  6.3.4 DiGress

DiGress employed a discrete denoising architecture to retrieve the 2D
molecular graph as categorical atom and bond features. The denoising
architecture is based on a graph transformer network from [81] that uses FiLM
layers [82] to combine edge features and global features. Similar to MUformer
[80], theoretic structural features such as cycles and spectral features, as
well as molecular features such as the current valency of each atom and the
current molecular weight of the whole molecule, are used to enhance the
representations of molecules to improve the denoising process.

####  6.3.5 MiDi

MiDi also employs a graph-denoising transformer architecture that jointly
predicts molecules’ 2D and 3D graphs. MiDi allows molecule atoms to hold
formal charges and become ions; a carbon atom, for example, can hold charges
of -1, 0, and 1. The denoising architecture learns to predict the molecular
structure gradually, where atom coordinates, and bond types are predicted
first, followed by the atom types and formal charges. MiDi’s denoising
architecture integrates relaxedEGNN (rEGNN) layers into its update block. MiDi
uses edge, node, pairwise, and global features pooled into node
representations using the Principal Neighbourhood Aggregation (PNA) layers
[83].

###  6.4 Hybrid Architectures

Combining elements from various architectures, such as transformers and GNNs,
can leverage the strengths of each approach to enhance denoising performance.
A good example of that is the Conditional Diffusion model based on discrete
Graph Structures (CDGS) [84], where a hybrid message passing block (HMPB) was
employed for the denoising architecture. HMPB includes two variations of
message-passing layers: a GNN layer called GINE [66] for discrete data types
to aggregate local neighbor node-edge features and a fully connected
attention-based transformer [78] for global graph features learning.
Similar to ShapeMol [65], Diff-Shape [85] proposed a pre-trained GNN called
Graph ControllNet (GrCN) to condition the diffusion process on 3D molecular
shapes. Graph ControllNet is inspired by ControlNet [86], which was pre-
trained with billions of images to guide text-to-image diffusion models.
Similarly, a pre-trained Graph ControllNet (GrCN) was combined with a pre-
trained unconditioned model of MiDi [32] to generate molecules constrained by
3d shape. While MiDi [32] contains a transformer architecture, and GrCN is
GNN-based, the model can be considered a hybrid architecture. Diff-Shape [85]
can also be employed for structure-based drug design by conditioning on the 3D
shape of ligands pre-docked to the target protein with high docking scores.

Denoising Architecture |  Model |  Denoising Model |  Condition |  Framework |  Datasets
---|---|---|---|---|---
GNNs |  EDM [28] |  EGNNs [29] |  Conditioned, unconditioned |  DDPM |  QM9 [48], GEOM-Drugs [49]
DiffBridges [43] |  EGNNs [29] |  Unconditioned |  SMLDa |  QM9 [48], GEOM-Drugs [49]
EEGSDE [67] |  EGNNs [29] |  Conditioned, unconditioned |  Score SDE |  QM9 [48], GEOM-Drugs [49]
GEOLDM [61] |  EGNNs [29] |  Conditioned, unconditioned |  DDPM |  QM9 [48], GEOM-Drugs [49]
Moldiff [56] |  EGNNs [29] |  Unconditioned |  DDPM |  QM9 [48], GEOM-Drugs [49]
HierDiff [36] |  EGNNs [29] |  Conditioned |  DDPM |  GEOM-Drugs [49], CrossDocked2020 [52]
GCDM [33] |  GCPNET [64] |  Conditioned, unconditioned |  DDPM |  QM9 [48], GEOM-Drugs [49], RS [54]
ShapeMol [65] |  INV-GNNs, EQ-GNNs |  Conditioned |  DDPM |  MOSES [51]
GDSS [44] |  |  Unconditioned |  Score SDE |  QM9 [48], ZINC250k [50]
LigandDiff [72] |  MMPNs, GVPs |  Conditioned |  DDPM |  Subset of Cambridge Structural Database (CSD) [87, 88]
CNNs |  MDM [75] |  Schnet [74] |  Conditioned, unconditioned |  SMLDa |  QM9 [48], GEOM-Drugs [49]
VoxMol [76] |  3D U-Net [77] |  Unconditioned |  SGM |  QM9[48], GEOM-Drugs [49]
Transformer |  DiGress [35] |  |  Conditioned, unconditioned |  DDPM |  QM9[48], GuacaMol [89], MOSES [51]
MiDi [32] |  |  Unconditioned |  DDPM |  QM9[48], GEOM-Drugs [49]
DIFFUMOL [47] |  |  Conditioned |  DDPM |  GuacaMol[89], MOSES [51], ZINC250K [50]
JODO [45] |  DGT |  Conditioned, unconditioned |  Score SDE |  QM9[48], GEOM-Drugs [49]
MUDiff [80] |  MUformer |  Conditioned, unconditioned |  DDPM |  QM9[48]
GFMDiff [57] |  DTN |  Conditioned, unconditioned |  DDPM |  QM9 [48], GEOM-Drugs [49]
Other |  CDGS [84] |  HMPB (hybrid) |  Unconditioned |  Score SDE |  QM9 [48], ZINC250K [50]
Diff-Shape [85] |  hybrid |  Conditioned |  DDPM |  GEOM-Drugs [49], PDBBind dataset [53]
SiMGen [71] |  |  Conditioned |  Score SDE |  QM9 [48], SPICE [70]
3M-Diffusion [62] |  |  Conditioned |  DDPM |  ChEBI-20 [90], PubChem [91], PCDes [92], MoMu [93]
GSDM [94] |  |  Unconditioned |  Score SDE |  QM9 [48], ZINC250k [50]

Table 2: Summery of the denoising architectures used in diffusion models for
molecular generation, and the datasets used for training those models.

a SMLD: score matching with Langevin dynamics, a sub category of SGMs where
Langevin dynamics are used

##  7  Molecular Generation for Structure-Based Drug Design (SBDD)

Molecular generation using diffusion models can be guided using the protein
target binding pocket as a condition for the generation. In the case of 3D
generation, the model can also generate the pose of the molecule within the
pocket (Figure. 5 and Table. 3). DiffSBDD [95] and DiffBP [96] were the first
two models to introduce SBDD-based diffusion models, and they both used an
equivariant EGNN for modeling the molecules; however, the DiffSBDD [95] study
was later expanded to explore more cases. Currently, it includes two different
approaches. In the first approach, a fixed pocket representation is used as a
condition for the 3D molecular generation, while in the second approach, the
joint distribution of ligand-pocket complexes was unconditionally
approximated. They also studied the impainting strategies where the context
can be injected into the sampling process at the probabilistic transition
steps. Impainting allows for masking, replacing, or fixing arbitrary parts of
the ligand-pocket system. Hence, it can be combined with the second strategy
for full 3D molecular generation or fixing parts of the molecule and
performing partial molecular generation or fragment generation. This can allow
for optimizing candidate drug molecules (leads) by exploring the local
chemical space while maintaining the rest of the 3D molecular structure, as
well as for applying techniques widely used in drug design, such as fragment
generation, scaffold hopping, fragment growing, and linker design.
TargetDiff [97] is another target-aware molecular generative model that
employs graph attention layers for parameterization. It also studied the case
of sub-structure or fragment generation and binding affinity ranking and
prediction. Both models were able to achieve better Vina Scores [98] on the
CrossDocked2020 dataset [52] than state-of-the-art non-diffusion-based models
such as Pocket2Mol [99] and GraphBP [100]. However, their performance needs
more improvement regarding other drug design metrics, such as SAS and QED
scores.
Moreover, two models relying on the idea of decomposing ligands were
developed: DECOMPOPT [101] and DECOMPDIFF [102] where ligands are decomposed
into two parts arms, and scaffold with arms are responsible for forming
interactions with the pocket amino acid residues and the scaffold connects all
arms to form a complete molecule. DECOMPDIFF [102] uses structural priors over
arms and scaffold, including priors estimated from the reference molecule and
pocket prior derived from the subpockets within the target binding site
extracted by AlphaSpace2 [103, 104]. They showed that pocket prior can enhance
the binding of generated molecules compared to TargetDiff [97]. DECOMPOPT
[101] applies iterative and controllable optimization by conditioning on local
substructures applying techniques like R-group optimization and scaffold
hopping. For example, the model was used to enhance the drug-likeness metrics
such as QED and SAS of the generated molecules while maintaining the binding
affinity to the target by conditioning on the molecular arms, making
interactions with the bonding pocket [101].
Binding Adaptive Diffusion Models (BindDM) [105] is another model that
adaptively generates molecules in reverse diffusion by extracting the
essential binding subcomplex graphs at each time step from the protein-ligand
complex graph with a learnable structural pooling. The two hierarchies of the
complex graph and its subcomplex graph interact during that process through
two cross-hierarchy interaction nodes: complex-to-subcomplex (C2S) and
subcomplex-to-complex (S2C). BindDM [105] has a similar performance to
TargetDiff [97] and DecompDiff [102]. KGDiff [106] also integrates the
chemical knowledge of protein-ligand binding affinity to direct the denoising
process at each step. Compared to TargetDiff [97], it achieved an average Vina
Score higher by 46.2% on the CrossDocked2020 dataset [52]. IPDiff [107]
applies a similar strategy to BindDM [105] where a protein-ligand interaction
prior network (IPNET) is pre-trained to learn ligand-protein interactions from
the chemical properties and 3D structures. Then, the pretrained IPNET serves
as a prior to enhance the generative diffusion process. They also introduce
two prior strategies: prior-shifting, where the diffusion process is shifted
based on protein-molecule interactions learned by IPNET, and prior-
conditioning, where the diffusion process is conditioned on previously
estimated protein-ligand complexes.
PROMPTDIFF [108] is also a target-aware generative model that uses a selected
set of ligand prompts—that is, compounds with desired characteristics like
high binding affinity to the target, drug-likeness, and synthesizability—to
direct the generative process towards generating similar molecules meeting the
design requirements. The model employs a geometric protein-molecule
interaction network (PMINet) that extracts information about the interactions
between protein-ligand pairs as embeddings that can be used as a prompt to
steer the diffusion process in desirable directions. The model proposes two
approaches for promoting the generative process: Self-Promoting, where the
embeddings are extracted from the generated ligand at each time step, and
Exemplar Prompting, where embeddings extracted from exemplar ligands with the
desired properties by pre-trained PMINet are used to guide the reverse
generation process [108].
PMDM [109] adopted a novel approach where they conditioned molecular
generation on both local and global molecular dynamics trajectories. The model
employed an EGNN to model the pocket ligand structures where the pocket
geometry served as the condition and a SchNet to generate the conditional
protein semantic information encodings and to embed the ligand atom feature
into an intermediate representation where the two are later fused using a
cross-attention mechanism.
The Molecular Out-Of-distribution Diffusion (MOOD) [46] model is a score-based
diffusion model that aims to generate molecules with desired chemical
properties such as binding affinity, drug-likeness, and synthesizability. The
model trains a separate network to predict a particular property, and then the
gradients from the property predictor are used to guide the reverse-time
diffusion process. This Out-Of-distribution guidance is controlled by a
hyperparameter $\lambda\in[0,1)$ that can be adapted to different magnitudes
based on the primary goal of generation. MOOD [46] was able to generate
molecules satisfying several constraints at the same time, and that was
reflected by the high Novel top $5\%$ docking score (the average DS of the top
$5\%$ unique molecules with $\text{QED}>0.5$ and $\text{SA}<5$) of the
generated compounds for five protein targets. PILOT [110] also aims to achieve
multi-objective conditioning using importance sampling. The model combines
pocket conditioning with synthetic accessibility (SA) guidance. First, the
model is unconditionally pre-trained on the Enamine Real Diversity subset
present in the ZINC database, and then the model is fine-tuned using the
pocket-conditioned CrossDocked2020 dataset [52]. Finally, importance sampling
is used at inference to guide the diffusion process towards a specific
property such as SA.
Similar to VoxMol [76], a score-based conditional model, VoxBind [111] was
developed to perform target-aware molecular generation using 3D atomic density
grids. VoxBind applied a two-step approach where a Langevin MCMC samples noisy
ligands, and then clean molecules are predicted using a conditional neural
empirical Bayes (NEB) denoiser [112, 76]. The model managed to achieve
successful Vina Scores on the n CrossDocked2020 dataset [52], surpassing
DiffSBDD [\cite[cite]{[\@@bibref{}{schneuing2023structurebased}{}{}]}],
TargetDiff [97], and DecompDif [102]. MolSnapper [113] is another recently
published model where the diffusion process is guided by 3D pharmacophores
incorporating features crucial for making interactions with the target protein
such as hydrogen bonds, charge interactions, and lipophilic contacts. The
model used Moldiff [56] for molecule generation with further guidance from the
target protein.
Moreover, some models were designed to use fragment-based drug design for
generating 3D compounds conditioned on protein pockets using fragments such as
DiffLinker [114], FragDiff [115] and AutoFragDiff [116] which will be
discussed in the following section.

Figure 5: Generation of 3D molecules conditioned on protein pocket using
diffusion models.

##  8 Other Applications

In addition to generating molecules in the 3D space, diffusion models have
been developed for other applications crucial for the drug design process,
such as conformations generation, molecular docking, molecular dynamics, and
fragment-based drug design or linker generation (Figure. 6 and Table. 3).

Figure 6: Some applications of diffusion models in the drug discovery process.

###  8.1 Fragment-Based Drug Design and Linker Design

DiffLinker [114] is an EGNNs-based 3D-conditional diffusion model for linker
generation. The model applies the concept of fragment-based drug design, where
it takes a set of disconnected fragments as a condition and connects them. The
linker attachment sites and their size, specifically the required number of
atoms, can be predicted automatically by DiffLinker. Similar to the mentioned
above SBDD models that applied linker design such as DiffSBDD [95] and PMDM
[109], DiffLinker was also used with protein pockets as a condition to the
linker design.
FragDiff [115] is an autoregressive model that generates 3D molecules
conditioned on protein pockets fragment-by-fragment using an E(3)-equivariant
diffusion generative model. AutoFragDiff [116] follows a similar approach, but
it employs geometric vector perceptrons (GVPs) to predict molecular fragments
conditioned on molecular scaffolds and protein pockets.
Selective iterative latent variable refinement (SILVR) [117] model generated
molecules conditioned on fragment hits screened against a protein binding
site. Hit fragments first go through the forward diffusion process by adding
Gaussian noise at each time step, generating a set of reference fragments at
different noise levels for different time steps. Then, the set of noisy
fragments is used to condition the EDM during the reverse diffusion process to
generate new samples similar to the reference fragments.

###  8.2 Conformations Generation

Conformer generation is a crucial step in drug design, and it involves
predicting the various spatial arrangements, or conformations, that a molecule
can adopt. Several diffusion models have been designed to efficiently sample
molecules’ conformational space efficiently. Dynamic Graph Score Matching
(DGSM) [118] is a score-based based that aims to predict equilibrium molecular
conformations by modeling both local and long-range interactions using Graph
Neural Networks (GNNs). Molecular graphs are constructed using fully connected
graphs with a cutoff distance where each node represents an atom within a
molecule and is assumed to interact with all the atoms within the sphere
constructed by the cutoff distance. Geodiff [34] employs an equivariant
convolutional layer, called graph field network (GFN), to model the molecules
where each atom is treated as a particle, and the reverse diffusion process
trains a Markov chain to generate stable conformations of the molecules.
Geodiff outperformed DGSM [118] in terms of matching and coverage scores for
the GEOM-DRUGs dataset [49].
Torsional diffusion [119] generates conformations using torsional Boltzmann
generators by operating only on a hypertorus defined by torsional angles, and
a diffusion model is scored using an extrinsic-to-intrinsic model that
predicts a torsional score (extrinsic coordinates) unique to a molecule and
takes the 3D point cloud representation of its conformer in Euclidean space
(intrinsic coordinates) as input.
SDEGen [120] aims to model low-energy conformations using a stochastic
differential equations (SDE) model that predicts the injected Gaussian noise.
Molecular conformations are represented as molecular graphs extended by adding
auxiliary bonds such as the two-hop edges (1–3 angle interaction) and three-
hop edges (1–4 dihedral angle interaction). Multilayer Perceptrons (MLPs) are
used to embed Gaussian noise, interatomic distances matrix, high-dimensional
space, and the edge information and add them together, forming distance
embedding. Then, Graph Isomorphism Networks (GINs) [66], a type of GNNs,
combine the distance embeddings with atom features embeddings and perform the
reverse denoising diffusion process by updating the distance embedding until
it gets mapped to the dimensional space of the Gaussian noise. SDEGen reported
high coverage and matching scores on the QM9 [48] and GEOM-DRUGs [49] datasets
compared to other models.

###  8.3 Molecular Docking

Molecular docking aims to predict the optimal binding orientation (pose) of
the ligand in the binding pocket of the receptor by simulating the interaction
between the two molecules and then scoring the pose based on those
interactions. DiffDock [121] is an SDE score-based generative model (Score
SDE) trained to predict ligand poses. Starting from a random pose, the reverse
diffusion process operates over translations, rotations, and torsion angles
and samples several poses. Another model called a confidence model is trained
to score those poses and rank them according to a confidence score that
estimates the most likely sample. DiffDock was able to achieve significant
accuracy on the PDBBind dataset [53] by retrieving $38.2\%$ of the ligand
poses with root-mean-square deviation (RMSD) values $<2$ Å and $63.2\%$ of the
ligands with RMSD $<5$ Å outperforming the traditional docking methods that
require a parameterized physics-based scoring function and a search algorithm
such as VINA [98] as well as deep learning-based methods such as GNINA [122].
DiffDock-PP [123] is similar to DiffDock [121], but it performs rigid protein-
protein docking instead.
PLANTAIN [124] is a physics-inspired diffusion model that aims to predict and
score ligands’ pose within the protein pocket by minimizing a score function.
The model used the protein binding pocket encodings, the 2D ligand
representation, and ligand-ligand and ligand-residue interatomic distances to
iteratively refine a ligand pose and score it. PLANTAIN could predict $21.5\%$
of the ligands’ poses within protein pockets with RMSD values $<2$ Å on the
CrossDocked dataset [52], also outperforming VINA [98] and GNINA [122]. DiffEE
[125] takes a similar approach where a large-scale protein language model
(PLM) encodes input protein sequences, and an iterative process updates the 3D
molecular graph and sample ligand poses. DiffEE had comparable results to VINA
[98] and GNINA [122] on the PDBBind benchmark [53] for the $2$ Å and $5$ Å
RMSD cutoffs.
PocketCFDM [126] used data augmentation of binding pockets where several
artificial binding pockets are created around the ligand molecule to
statistically simulate the non-bond interactions found in actual protein-
ligand complexes. PocketCFDM’s statistical approach was able to achieve an
accuracy of $23.9\%$ with an RMSD cutoff of $2$ Å on the PDBbind dataset [53],
scoring below DiffDock; however, it performed better on the non-bond
interactions, steric clash numbers, and inference speed.
NeuralPLexer [127] is an SDE-based diffusion model that can predict protein-
ligand complex structures at atomistic resolution by training a primary
network composed of an Equivariant Structure Denoising Module (ESDM) and
coarse-grained, auto-regressive contact prediction module (CPM). Protein
language model (PLM) features and structure templates are first retrieved from
the input protein sequence. Next, using molecular graph representations of the
input ligands and the set of PLM and template features as inputs, NeuralPLexer
samples a set of binding protein-ligand complex conformations. The model was
able to outperform DiffDock in rigid blind protein-ligand docking and predict
up to $78\%$ of the ligands’ poses within protein pockets with RMSD values
$<2$ Å from the reference PDBBind2020 benchmark [53].
Neural Euler’s Rotation Equations (NERE) [128] is an unsupervised denoising
score matching (DSM) energy-based diffusion model that simulates the force and
torque between the atoms of the ligand and the protein to predict rotation.
The model’s log-likelihood is defined as the binding affinity of the ligand-
protein complex and trains an SE(3) equivariant rotation prediction network
where force is the gradient of the energy function with respect to atom
coordinates. NERE was able to achieve Pearson correlation values of 0.656 and
0.651 between true binding affinity and predicted energy on the crystal and
docked structures of the PDBBind benchmark, respectively [53].
The aforementioned diffusion-based docking methods report comparable or better
performance than traditional approaches. However, a study by Yu et al. [129]
argued that deep learning-based models are superior in docking on the whole
protein. At the same time, traditional methods are better at docking on given
pockets. Therefore, when pocket searching was performed before docking,
traditional methods achieved better results for docking in the predetermined
pockets than in the entire protein compared to deep learning methods.
Moreover, PoseBusters [130] found that deep-learning-based tools, including
DiffDock [121], do not generate physically valid poses. However, those studies
only covered DiffDock [121] and not the following models. With the fast-paced
advancement of diffusion-based models, there is a massive potential for
further improvement to surpass traditional models in pocket searching and
given pocket docking.

###  8.4 Molecular Dynamics

Molecular dynamics (MD) is a computational modeling technique employed to
investigate the time-dependent behavior of atoms and molecules by simulating
their physical motions through classical mechanics. It has several
applications, such as understanding binding mechanisms between drug molecules
with their target proteins, protein-protein interactions, and Investigating
the dynamics of biomolecules such as proteins and nucleic acid. DiffMD [131]
is a score-based diffusion model aiming to accelerate molecules’ MD. DiffMD
employs an equivariant geometric transformer to calculate the gradient of the
log density of molecular conformations where the directions and velocities of
atomic motions are represented by 3D spherical Fourier-Bessel bases. DiffMD
outperforms state-of-the-art baselines such as EGNNs [29] on MD17 [132] and
isomers of $\text{C}_{7}\text{O}_{2}\text{H}_{10}$ datasets. Another model
[133] aims to estimate a coarse-grained (CG) force field as a Denoising Force
Field (DFF) using a score-based diffusion model. The model successfully
enhanced performance on various protein simulations for up to 56 amino acid
systems.

Application |  Model name |  Condition/Guidance |  Framework |  Network architecture
---|---|---|---|---
Target-conditioned molecular generation |  DiffSBDD [95] |  Protein pockets |  DDPM |  EGNNs [29]
DiffBP [96] |  Protein pockets |  DDPM |  EGNNs [29]
TargetDiff [97] |  Protein pockets |  DDPM |  EGNNs [29]
DECOMPOPT [101] |  Protein pockets |  DDPM |  EGNNs [29]
DECOMPDIFF [102] |  Protein pockets |  DDPM |  GNNs
BindDM [105] |  Protein pockets |  DDPM |  Hybird
PROMPTDIFF [108] |  Ligand prompt set and target protein |  DDPM |  PMINeta
IPDiff [107] |  Protein-ligand interaction prior |  DDPM |  Hybird
PMDM [109] |  Protein pockets |  SGM |  Hybird
MOOD [46] |  Chemical properties |  Score SDE |  Hybird (GNNs and GCNb)
PILOT [110] |  Protein pockets and SAS |  SGM |  GNNs
KGDiff [106] |  Protein–ligand binding affinity |  DDPM |  GNNs
|  VoxBind [111] |  Protein pockets |  SGM |  3D U-Net [77]
|  MolSnapper [113] |  3D pharmacophores and Target protein |  DDPM |  EGNNs variant
Fragment-Based Drug Design and Linker Design |  DiffLinker [114] |  Protein pockets |  DDPM |  EGNNs [29]
FragDiff [115] |  Protein pockets |  DDPM |  EGNNs [29]
AutoFragDiff [116] |  Target protein |  DDPM |  GVPsc
SILVR [117] |  Hit fragments |  DDPM |  EGNNs [29]
Conformations Generation |  DGSM [118] |  Molecular graph |  SGM |  GNNs
Geodiff [34] |  Molecular graph |  DDPM |  GFNd
Torsional diffusion [119] |  Molecular graph |  Score SDE |  Hybird
SDEGen [120] |  Molecular graph |  Score SDE |  GNNs
Molecular Docking |  DiffDock [121] |  Protein-ligand complex |  Score SDE |  GNNs variant (Hybird)
DiffDock-PP [123] |  Protein-protein complex |  Score SDE |  GNNs variant (Hybird)
PLANTAIN [124] |  Protein-ligand complex |  SGM |  GNNs variant (Hybird)
DiffEE [125] |  Protein sequence and ligand |  DDPM |  PLM and EGNNs [29]
PocketCFDM [126] |  Unconditioned |  Score SDE |  GNNs variant (Hybird)
NeuralPLexer [127] |  Protein-ligand complex |  Score SDE |  GNNs variant (Hybird)
NERE [128] |  Unconditioned |  SGM |  MPNNe [134]
Molecular Dynamics |  DiffMD [131] |  Unconditioned |  Score SDE |  Transformer
DFF [133] |  Unconditioned |  SGM |  Transformer

Table 3: Applications of diffusion models in drug design

a Protein-molecule interaction network, b Graph Convolutional Network, c
Geometric Vector Perceptrons, d Graph Field Network, e MPNN: Message Passing
Neural Network

##  9 Evaluation Metrics

Assessing diffusion models for molecular generation can include various
strategies depending on the task. However, the quality of the generated
molecules is the most crucial aspect, specifically for the 3D generation, and
it requires a multifaceted evaluation strategy. Some evaluation metrics can be
used on any molecular generation, whether in 1D, 2D, or 3D, and some metrics
are engineered specifically for 3D molecular graphs or drug-like molecules.
This section will cover evaluation metrics usually used with molecular
generation using diffusion models.

###  9.1 Evaluation Metrics for Molecular Generation

Validity -  The percentage of valid molecules, i.e., chemically correct in
terms of atoms’ valency and consistency of bonds in aromatic rings. This
metric is often used with 2D molecular graphs and SMILEs and is evaluated by
RDKit.

Novelty -  The percentage of molecules not contained within the training
dataset; this metric measures how well the model can generate molecules
outside of the dataset in question.

Uniqueness -  The percentage of unique and valid molecules in a sample of
generated molecules.

Atom Stability -  The percentage of atoms with the correct valency (usually
used in 3D molecular generation).

Molecule Stability -  The percentage of molecules whose atoms are all stable
(usually used in 3D molecular generation).

Connectivity/Connected -  The percentage of connected molecules, i.e.,
generated molecules with a single connected component (usually used in
molecular graph generation).

###  9.2 Similarity Metrics

Distance metrics are crucial in quantifying the similarity between generated
molecules’ data distribution and training data distribution in molecular
generation tasks. Two widely used metrics for this purpose are the Maximum
Mean Discrepancy (MMD) and the Jensen-Shannon (JS) divergence. They are
employed to evaluate the distributions of bond lengths, bond angles, and
dihedral angles in generated samples compared to the original dataset.

Maximum mean discrepancy (MMD) -  A formulation that measures the distance
between two probability distributions by embedding probabilities in a
reproducing kernel Hilbert space (RKHS) [135]. Given two distributions $P$ and
$Q$, with corresponding feature maps $\phi(x)$ and $\psi(y)$, the MMD can be
expressed as:

$\displaystyle MMD^{2}(P,Q)=\left\lVert\mathbb{E}_{x\sim
P}[\phi(x)]-\mathbb{E}_{y\sim Q}[\psi(y)]\right\rVert^{2}_{\mathcal{H}}$

where $\mathbb{E}_{x\sim P}[\phi(x)]$ and $\mathbb{E}_{y\sim Q}[\psi(y)]$ are
the expected value of the feature map of samples drawn from distributions $P$,
and $Q$ respectively, and $\lVert\cdot\rVert^{2}_{\mathcal{H}}$ is the norm in
the RKHS [135].

Jensen Shannon (JS) divergence -  A metric that quantifies the similarity or
dissimilarity between two probability distributions based on the Kullback-
Leibler (KL) divergence. Hence, given two distributions $P$ and $Q$, the JS
divergence between them should be:

$\displaystyle JS(P\parallel Q)=\frac{1}{2}\cdot KL\left(P\parallel
M\right)+\frac{1}{2}\cdot KL\left(Q\parallel M\right)$

where $M=\frac{1}{2}(P+Q)$ is the average distribution, and $KL$ is the
Kullback-Leibler divergence and it takes the form:

$KL(P\parallel
Q)=\int_{-\infty}^{\infty}P(x)\cdot\log\left(\frac{P(x)}{Q(x)}\right)dx$

###  9.3 Drug-Likeness Metrics

logP - the logarithm of the partition coefficient A measurement of the
compound’s distribution between two immiscible phases, usually between an
organic phase (usually octanol) and an aqueous phase, it is frequently used to
assess a compound’s lipophilicity, which is an important aspect in determining
its absorption, distribution, metabolism, and excretion (ADME) properties.

QED - Quantitative Estimate of Druglikeness An integrative score to assess a
compound’s likelihood of becoming a drug, it combines a variety of main
molecular properties that fall within the range of known drugs.

SAS - synthetic accessibility score Evaluates the synthetic feasibility of the
generated molecules, and it is calculated as a sum of fragment scores and
complexity penalty [136].

Lipinski -  Average number of rules satisfied from the Lipinski rule of five,
a loose rule for evaluating the drug-likeness of molecules according to five
rules the molecule should satisfy: no more than five hydrogen bond donors, no
more than ten hydrogen bond acceptors. Molecular mass less than 500 Da, and a
logP value less than five [137].

###  9.4 Evaluation Metrics for Propriety-Based Conditional Generation

Conditional generation can be performed with any property. However, it is
often used with the QM9 dataset as a proof of concept where a property
classifier network, usually an EGNN, is trained on half of the QM9 dataset,
while the model is trained on the other half, and then the model is given the
property value as an input and is asked to sample molecules with that property
value. The mean absolute error (MAE) between the input property values and the
predicted values of the generated molecules is used to evaluate the model’s
ability of conditional generation. The six OM9 properties often conditioned on
are:

  * •

$\mathbf{\alpha}$ \- Polarizability in cubic Bohr radius ($\text{Bohr}^{3}$).

  * •

$\mathbf{\mu}$ \- Dipole moment in debyes ($D$).

  * •

$\mathbf{\epsilon_{HOMO}}$ \- Highest Occupied Molecular Orbit Energy in
millielectron volts ($meV$).

  * •

$\mathbf{\epsilon_{LUMO}}$ \- Lowest Unoccupied Molecular Orbit Energy in
millielectron volts ($meV$).

  * •

$\mathbf{\Delta\epsilon}$ \- Difference between $\mathbf{\epsilon_{HOMO}}$ and
$\mathbf{\epsilon_{LUMO}}$ in millielectron volts ($meV$).

  * •

$\mathbf{C_{v}}$ \- Molar Heat Capacity at $298.15K$ in calories per Kelvin
per mole ($\frac{cal}{mol}K$).

###  9.5 Evaluation Metrics for Structure-Based Generation

Vina Score -  A scoring function that estimates the binding affinity between a
ligand (small molecule) and a protein target in molecular docking simulations
using Autodock vina [98].

Vina Min -  The same scoring function as Vina Score, but the Vina platform
conducts a local structure minimization before estimation [98].

Vina Dock -  The same scoring function as Vina Score, but molecules are re-
docked before being scored, and it reflects the best possible binding affinity
for the generated molecules [98].

Vina Energy -  An energy estimation of the binding affinity between a ligand
(small molecule) and a protein target in molecular docking simulations using
Autodock vina [98].

High-Affinity Percentage -  Percentage of compounds that, upon binding to the
target protein, have a lower Vina energy than the reference (ground-truth)
molecule.

Diversity -  The average pairwise dissimilarity between all generated
molecules for each target pocket is calculated as (1 - Tanimoto similarity)
between pairs of Morgan fingerprints to measure the diversity of generated
molecules for each pocket.

###  9.6 Evaluation Metrics for Conformations Generation

Evaluation metrics for evaluating conformers are all based on the root-mean-
square deviation (RMSD), which can be calculated as the normalized Frobenius
norm of the atoms of two superimposed molecules aligned using the Kabsch
algorithm [138] using the following formula:

$\displaystyle\text{RMSD}(C,\hat{C})=\sqrt{\frac{\sum_{i=1}^{N}(x_{i}-x_{i}^{ref})^{2}+(y_{i}-y_{i}^{ref})^{2}+(z_{i}-z_{i}^{ref})^{2}}{N}}$

Where $\hat{C}$ is the predicted molecule conformer with a set of
$i\in\\{1,2,\dots,N\\}$ atomic coordinates $(x_{i},y_{i},z_{i})$ and $C$ is
the reference molecule conformer with the corresponding reference atomic
positions $(x_{i}^{ref},y_{i}^{ref},z_{i}^{ref})$, and $N$ is the number of
atoms. Then, metrics like Average Minimum RMSD (AMR) or Matching (MAT) and
Coverage (COV)can be calculated for Precision (P) and Recall (R) from RMSD
values of the generated molecular conformations [34].

COV -  The percentage of structures in one set covered by another, where
covering denotes the RMSD between two conformations falling within a specified
threshold $\delta$. Given two sets of conformers, $S_{g}$ as the generated set
and $S_{r}$ as the reference set, coverage for recall $COV-R$ measures the
model’s ability to find all the reference conformers (percentage of recalled
or predicted reference conformers) while coverage for precision $COV-P$
measures the percentage of the conformers the model generates that are
relevant, i.e. can be found in the reference set [34]. Formally, it can be
calculated as:

$\displaystyle\text{COV-R}(S_{g},S_{r})=\frac{1}{|S_{r}|}\left|\left\\{C\in
S_{r}|\text{RMSD}(C,\hat{C})\leq\delta,\hat{C}\in S_{g}\right\\}\right|$

MAT/AMR -  The average RMSD between conformers in one set and their nearest
neighbor in another. i.e., the conformers’ average minimum RMSD (AMR), where
lower MAT scores reflect the generation of more realistic conformations.
Matching for recall (MAT-R) and precision (MAT-P) are defined similarly to the
coverage metric [34].

$\displaystyle\text{MAT-R}(S_{g},S_{r})=\frac{1}{|S_{r}|}\sum_{C\in
S_{r}}\underset{\hat{C}\in S_{g}}{\text{min}}\text{RMSD}(C,\hat{C})$

##  10 Current Limitations

Diffusion models have significantly advanced molecular generation in general,
specifically 3D molecular graph generation. However, there are still some
challenges to advance the field further.
Chirality is one of the current limitations of the 3D generative models as
most of the diffusion models are E(3) equivariant, meaning they are not
sensitive to chirality. This issue was only considered when designing the
reverse diffusion neural network of the GCDM model [33], GCPNET [64], and
similarly, chirality should be considered when designing novel models or
improvements over the current state-of-the-art models.
A second limitation shared by molecular generation models, in general, is the
disconnect between metrics and real-world performance in different
applications like drug discovery and material design. While metrics like
synthetic accessibility score (SAS) [136] and Vina docking scores [98] or
binding affinities serve as good estimates, it still does not guarantee that
this molecule can be found in libraries of compounds or that it may be
synthesized using current methods of synthetic chemistry or the cost
efficiency of synthesizing those molecules compared to other possible
candidates from other regions of the chemical space. Moreover, the black-box
nature of deep generative models in general, including diffusion models, makes
it challenging to trace why diffusion models generate specific molecules or
conformations. This lack of explainability complicates identifying and
addressing potential biases towards certain chemical spaces or limitations in
the generation process.
A third limitation is related to data availability in chemical applications.
Due to the higher cost of experimental validation, such as structure-based
applications in drug design and material design applications, the data
available for training is still limited and does not cover important aspects
such as pharmacokinetics and toxicity properties of the generated molecules.
The data can also introduce bias toward known space, limiting the discovery of
truly novel and groundbreaking structures, especially when the model is
optimized using metrics that reward that bias, such as the Maximum Mean
Discrepancy (MMD) distance and other similarity scores. The lack of data is a
general problem in several drug design applications, not only in molecular
generation. To solve this issue for some applications, Hu et al. used
diffusion models to generate synthetic data specific to certain tasks such as
pharmacokinetic properties, toxicity, and hERG [139].
A more specific limitation to the 3D generation is the lack of a unified
benchmark; even the widely used metrics, such as 3D stability, are defined
differently in each model. For example, some models such as MiDi [32]
calculate stability over the adjacency matrix generated by the model for
covalent bond types, and other models such as JODO [45] calculate the covalent
bonds based on distance cutoffs from the generated 3D coordinates and then
evaluate the percentage of stable atoms accordingly. However, even simple
definitions like what is considered a stable atom are not unified. For
example, some models such as MiDi [32] consider ionized atoms stable, so a
carbon atom is allowed to have a valency of 4 (formal charge (FC) = 0), 3 (FC
= 1, -1), or 5 (FC = -1), and other models such as JODO [45] and EDM [28] only
consider an atom stable if its formal charge is equal to zero. Moreover,
conditional generation of molecules on a quantum mechanics’ property such as
the QM9 polarizability ($\alpha$), and dipole moment ($\mu$), $\dots$, etc.,
is evaluated based on an EGNN [29] predictive model. In contrast, the EGNN
model itself has a margin of error from the ground truth (Density-functional
theory (DFT) calculations). Given the advancement of computational resources,
it would be better to consider generating a reasonable sample of molecules and
evaluating the generated molecules using DFT calculations directly.
A further limitation of diffusion models is the high training and sampling
cost, especially for complex data like molecules. Those computational demands
scale significantly with increasing the size of molecules. However, developing
new model architectures and training strategies can help achieve better
training and sampling from those models.
PoseCheck [140], a benchmarking study designed to evaluate deep generative
models designed for SBDD, reported another limitation of diffusion models in
target-aware generation where the two diffusion-based models (DiffSBDD [95]
and TargetDiff [97]) generate highly-strained molecules with high steric
clashes with the protein pocket and suggested penalizing steric clashes to
avoid this issue.

##  11 Conclusions

Diffusion models hold immense potential for various chemical applications. The
early successes motivate intense activity to address challenges in data
availability, model performances and relevance to different applications
persist. By addressing these limitations and actively researching new
approaches and model architectures, diffusion models can be used effectively
in various chemical applications. Given the large datasets available and
decades of intensive research, drug discovery represents an ideal platform for
further development of diffusion models.

##  12 Acknowledgements

I, Amira, thank Sijie Fu, Calvin Gang, Chenghui Zhou, and Euxhen Hasanaj for
the helpful discussions and comments on diffusion models that led to this
review. Also, my special thanks go to Arav Agarwal for helping with data
collection and for his valuable feedback and comments. AI tools, including
ChatGPT, Gemini, and Quillbot, were used in limited tasks while writing this
article.

##  13 Conflict of Interest

The authors declare no competing interests.

## References

  * [1] Chao Pang, Jianbo Qiao, Xiangxiang Zeng, Quan Zou, and Leyi Wei.  Deep generative models inde novodrug molecule generation.  Journal of Chemical Information and Modeling, Nov 2023.
  * [2] Wiktoria Wilman, Sonia Wróbel, Weronika Bielska, Piotr Deszynski, Paweł Dudzic, Igor Jaszczyszyn, Jędrzej Kaniewski, Jakub Młokosiewicz, Anahita Rouyan, Tadeusz Satława, Sandeep Kumar, Victor Greiff, and Konrad Krawczyk.  Machine-designed biotherapeutics: opportunities, feasibility and advantages of deep learning in computational antibody discovery.  Briefings in Bioinformatics, 23(4):bbac267, 07 2022.
  * [3] Yoo-Ah Kim, Kambiz Mousavi, Amirali Yazdi, Magda Zwierzyna, Marco Cardinali, Dillion Fox, Thomas Peel, Jeff Coller, Kunal Aggarwal, and Giulietta Maruggi.  Computational design of mrna vaccines.  Vaccine, 42(7):1831–1840, Mar 2024.
  * [4] Benoit Baillif, Jason Cole, Patrick McCabe, and Andreas Bender.  Deep generative models for 3d molecular structure.  Current Opinion in Structural Biology, 80:102566, Jun 2023.
  * [5] Peter S Kutchukian and Eugene I Shakhnovich.  De novo design: balancing novelty and confined chemical space.  Expert Opinion on Drug Discovery, 5(8):789–812, 2010.  PMID: 22827800.
  * [6] Yujia Li, Oriol Vinyals, Chris Dyer, Razvan Pascanu, and Peter Battaglia.  Learning deep generative models of graphs, 2018.
  * [7] Yibo Li, Liangren Zhang, and Zhenming Liu.  Multi-objective de novo drug design with conditional graph generative model - journal of cheminformatics, Jul 2018.
  * [8] Mariya Popova, Mykhailo Shvets, Junier Oliva, and Olexandr Isayev.  Molecularrnn: Generating realistic molecular graphs with optimized properties, 2019.
  * [9] Daniel C. Elton, Zois Boukouvalas, Mark D. Fuge, and Peter W. Chung.  Deep learning for molecular design—a review of the state of the art.  Molecular Systems Design & Engineering, 4(4):828–849, 2019.
  * [10] Martin Simonovsky and Nikos Komodakis.  Graphvae: Towards generation of small graphs using variational autoencoders, 2018.
  * [11] Qi Liu, Miltiadis Allamanis, Marc Brockschmidt, and Alexander L. Gaunt.  Constrained graph variational autoencoders for molecule design, 2019.
  * [12] Bidisha Samanta, Abir De, Gourhari Jana, Pratim Kumar Chattaraj, Niloy Ganguly, and Manuel Gomez-Rodriguez.  Nevae: A deep generative model for molecular graphs, 2019.
  * [13] Nicola De Cao and Thomas Kipf.  Molgan: An implicit generative model for small molecular graphs, 2022.
  * [14] Rocío Mercado, Tobias Rastemo, Edvard Lindelöf, Günter Klambauer, Ola Engkvist, Hongming Chen, and Esben Jannik Bjerrum.  Graph networks for molecular design.  ChemRxiv, 2020.
  * [15] Kaushalya Madhawa, Katushiko Ishiguro, Kosuke Nakago, and Motoki Abe.  Graphnvp: An invertible flow model for generating molecular graphs, 2019.
  * [16] Chence Shi, Minkai Xu, Zhaocheng Zhu, Weinan Zhang, Ming Zhang, and Jian Tang.  Graphaf: a flow-based autoregressive model for molecular graph generation, 2020.
  * [17] Chengxi Zang and Fei Wang.  Moflow: An invertible flow model for generating molecular graphs.  In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, KDD ’20. ACM, August 2020.
  * [18] Youzhi Luo, Keqiang Yan, and Shuiwang Ji.  Graphdf: A discrete flow model for molecular graph generation, 2021.
  * [19] Weixin Xie, Fanhao Wang, Yibo Li, Luhua Lai, and Jianfeng Pei.  Advances and challenges in de novo drug design using three-dimensional deep generative models.  Journal of Chemical Information and Modeling, 62(10):2269–2279, May 2022.
  * [20] Jascha Sohl-Dickstein, Eric A. Weiss, Niru Maheswaranathan, and Surya Ganguli.  Deep unsupervised learning using nonequilibrium thermodynamics, 2015.
  * [21] Jonathan Ho, Ajay Jain, and Pieter Abbeel.  Denoising diffusion probabilistic models, 2020.
  * [22] Diederik P. Kingma, Tim Salimans, Ben Poole, and Jonathan Ho.  Variational diffusion models, 2023.
  * [23] Alex Nichol and Prafulla Dhariwal.  Improved denoising diffusion probabilistic models, 2021.
  * [24] Fan Bao, Chongxuan Li, Jun Zhu, and Bo Zhang.  Analytic-dpm: an analytic estimate of the optimal reverse variance in diffusion probabilistic models, 2022.
  * [25] Yang Song, Jascha Sohl-Dickstein, Diederik P. Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole.  Score-based generative modeling through stochastic differential equations, 2021.
  * [26] Zhiye Guo, Jian Liu, Yanli Wang, Mengrui Chen, Duolin Wang, Dong Xu, and Jianlin Cheng.  Diffusion models in bioinformatics and computational biology.  Nature Reviews Bioengineering, Oct 2023.
  * [27] Chengyi Liu, Wenqi Fan, Yunqing Liu, Jiatong Li, Hang Li, Hui Liu, Jiliang Tang, and Qing Li.  Generative diffusion models on graphs: Methods and applications, 2023.
  * [28] Emiel Hoogeboom, Victor Garcia Satorras, Clément Vignac, and Max Welling.  Equivariant diffusion for molecule generation in 3d, 2022.
  * [29] Victor Garcia Satorras, Emiel Hoogeboom, and Max Welling.  E(n) equivariant graph neural networks, 2022.
  * [30] Mengchun Zhang, Maryam Qamar, Taegoo Kang, Yuna Jung, Chenshuang Zhang, Sung-Ho Bae, and Chaoning Zhang.  A survey on graph diffusion models: Generative ai in science for molecule, protein and material.  2023\.
  * [31] Ling Yang, Zhilong Zhang, Yang Song, Shenda Hong, Runsheng Xu, Yue Zhao, Wentao Zhang, Bin Cui, and Ming-Hsuan Yang.  Diffusion models: A comprehensive survey of methods and applications, 2024.
  * [32] Clement Vignac, Nagham Osman, Laura Toni, and Pascal Frossard.  Midi: Mixed graph and 3d denoising diffusion for molecule generation, 2023.
  * [33] Alex Morehead and Jianlin Cheng.  Geometry-complete diffusion for 3d molecule generation and optimization, 2024.
  * [34] Minkai Xu, Lantao Yu, Yang Song, Chence Shi, Stefano Ermon, and Jian Tang.  Geodiff: a geometric diffusion model for molecular conformation generation, 2022.
  * [35] Clement Vignac, Igor Krawczuk, Antoine Siraudin, Bohan Wang, Volkan Cevher, and Pascal Frossard.  Digress: Discrete denoising diffusion for graph generation, 2023.
  * [36] Bo Qiang, Yuxuan Song, Minkai Xu, Jingjing Gong, Bowen Gao, Hao Zhou, Weiying Ma, and Yanyan Lan.  Coarse-to-fine: a hierarchical diffusion model for molecule generation in 3d, 2023.
  * [37] Yang Song, Sahaj Garg, Jiaxin Shi, and Stefano Ermon.  Sliced score matching: A scalable approach to density and score estimation, 2019.
  * [38] Yang Song and Stefano Ermon.  Generative modeling by estimating gradients of the data distribution, 2020.
  * [39] Brian DO Anderson.  Reverse-time diffusion equation models.  Stochastic Processes and their Applications, 12(3):313–326, 1982.
  * [40] Tero Karras, Miika Aittala, Timo Aila, and Samuli Laine.  Elucidating the design space of diffusion-based generative models, 2022.
  * [41] Jiaming Song, Chenlin Meng, and Stefano Ermon.  Denoising diffusion implicit models, 2022.
  * [42] Cheng Lu, Yuhao Zhou, Fan Bao, Jianfei Chen, Chongxuan Li, and Jun Zhu.  Dpm-solver: A fast ode solver for diffusion probabilistic model sampling in around 10 steps, 2022.
  * [43] Lemeng Wu, Chengyue Gong, Xingchao Liu, Mao Ye, and Qiang Liu.  Diffusion-based molecule generation with informative prior bridges, 2022.
  * [44] Jaehyeong Jo, Seul Lee, and Sung Ju Hwang.  Score-based generative modeling of graphs via the system of stochastic differential equations, 2022.
  * [45] Han Huang, Leilei Sun, Bowen Du, and Weifeng Lv.  Learning joint 2d & 3d diffusion models for complete molecule generation, 2023.
  * [46] Seul Lee, Jaehyeong Jo, and Sung Ju Hwang.  Exploring chemical space with score-based out-of-distribution generation, 2023.
  * [47] Xinmiao Peng and Fei Zhu.  Hitting stride by degrees: Fine grained molecular generation via diffusion model.  Expert Systems with Applications, 244:122949, 2024.
  * [48] Raghunathan Ramakrishnan, Pavlo O. Dral, Matthias Rupp, and O. Anatole von Lilienfeld.  Quantum chemistry structures and properties of 134 kilo molecules.  Scientific Data, 1, 2014.
  * [49] Simon Axelrod and Rafael Gomez-Bombarelli.  Geom: Energy-annotated molecular conformations for property prediction and molecular generation, 2022.
  * [50] John J. Irwin, Teague Sterling, Michael M. Mysinger, Erin S. Bolstad, and Ryan G. Coleman.  Zinc: A free tool to discover chemistry for biology.  Journal of Chemical Information and Modeling, 52(7):1757–1768, Jun 2012.
  * [51] Daniil Polykovskiy, Alexander Zhebrak, Benjamin Sanchez-Lengeling, Sergey Golovanov, Oktai Tatanov, Stanislav Belyaev, Rauf Kurbanov, Aleksey Artamonov, Vladimir Aladinskiy, Mark Veselov, and et al.  Molecular sets (moses): A benchmarking platform for molecular generation models.  Frontiers in Pharmacology, 11, Dec 2020.
  * [52] Paul G. Francoeur, Tomohide Masuda, Jocelyn Sunseri, Andrew Jia, Richard B. Iovanisci, Ian Snyder, and David R. Koes.  Three-dimensional convolutional neural networks and a cross-docked data set for structure-based drug design.  Journal of Chemical Information and Modeling, 60(9):4200–4215, Aug 2020.
  * [53] Zhihai Liu, Minyi Su, Li Han, Jie Liu, Qifan Yang, Yan Li, and Renxiao Wang.  Forging the basis for developing protein–ligand interaction scoring functions.  Accounts of Chemical Research, 50(2):302–309, 2017.  PMID: 28182403.
  * [54] Keir Adams, Lagnajit Pattanaik, and Connor W. Coley.  Learning 3d representations of molecular chirality with invariance to bond rotations, 2021.
  * [55] Han Huang, Leilei Sun, Bowen Du, Yanjie Fu, and Weifeng Lv.  Graphgdp: Generative diffusion processes for permutation invariant graph generation, 2022.
  * [56] Xingang Peng, Jiaqi Guan, Qiang Liu, and Jianzhu Ma.  Moldiff: Addressing the atom-bond inconsistency problem in 3d molecule diffusion generation, 2023.
  * [57] Can Xu, Haosen Wang, Weigang Wang, Pengfei Zheng, and Hongyang Chen.  Geometric-facilitated denoising diffusion model for 3d molecule generation, 2024.
  * [58] Sander Dieleman, Laurent Sartran, Arman Roshannai, Nikolay Savinov, Yaroslav Ganin, Pierre H. Richemond, Arnaud Doucet, Robin Strudel, Chris Dyer, Conor Durkan, Curtis Hawthorne, Rémi Leblond, Will Grathwohl, and Jonas Adler.  Continuous diffusion for categorical data, 2022.
  * [59] Ting Chen, Ruixiang Zhang, and Geoffrey Hinton.  Analog bits: Generating discrete data using diffusion models with self-conditioning, 2023.
  * [60] Kilian Konstantin Haefeli, Karolis Martinkus, Nathanaël Perraudin, and Roger Wattenhofer.  Diffusion models for graphs benefit from discrete state spaces, 2023.
  * [61] Minkai Xu, Alexander Powers, Ron Dror, Stefano Ermon, and Jure Leskovec.  Geometric latent diffusion models for 3d molecule generation, 2023.
  * [62] Huaisheng Zhu, Teng Xiao, and Vasant G Honavar.  3m-diffusion: Latent multi-modal diffusion for text-guided generation of molecular graphs, 2024.
  * [63] Jie Zhou, Ganqu Cui, Shengding Hu, Zhengyan Zhang, Cheng Yang, Zhiyuan Liu, Lifeng Wang, Changcheng Li, and Maosong Sun.  Graph neural networks: A review of methods and applications, 2021.
  * [64] Alex Morehead and Jianlin Cheng.  Geometry-complete perceptron networks for 3d molecular graphs, 2023.
  * [65] Ziqi Chen, Bo Peng, Srinivasan Parthasarathy, and Xia Ning.  Shape-conditioned 3d molecule generation via equivariant diffusion models, 2023.
  * [66] Weihua Hu, Bowen Liu, Joseph Gomes, Marinka Zitnik, Percy Liang, Vijay Pande, and Jure Leskovec.  Strategies for pre-training graph neural networks, 2020.
  * [67] Fan Bao, Min Zhao, Zhongkai Hao, Peiyao Li, Chongxuan Li, and Jun Zhu.  Equivariant energy-guided sde for inverse molecular design, 2023.
  * [68] Jay W Ponder and David A Case.  Force fields for protein simulations.  Advances in Protein Chemistry, 66:27–85, 2003.
  * [69] Ilyes Batatia, Dávid Péter Kovács, Gregor N. C. Simm, Christoph Ortner, and Gábor Csányi.  Mace: Higher order equivariant message passing neural networks for fast and accurate force fields, 2023.
  * [70] Peter Eastman, Pavan Kumar Behara, David L. Dotson, Raimondas Galvelis, John E. Herr, Josh T. Horton, Yuezhi Mao, John D. Chodera, Benjamin P. Pritchard, Yuanqing Wang, and et al.  Spice, a dataset of drug-like molecules and peptides for training machine learning potentials.  Scientific Data, 10(1), Jan 2023.
  * [71] Rokas Elijošius, Fabian Zills, Ilyes Batatia, Sam Walton Norwood, Dávid Péter Kovács, Christian Holm, and Gábor Csányi.  Zero shot molecular generation via similarity kernels, 2024.
  * [72] Hongni Jin and Kenneth M. Jr. Merz.  Liganddiff: de novo ligand design for 3d transition metal complexes with diffusion models.  Journal of Chemical Theory and Computation, 0(0):null, 0.  PMID: 38743854.
  * [73] Keiron O’Shea and Ryan Nash.  An introduction to convolutional neural networks, 2015.
  * [74] Kristof T. Schütt, Pieter-Jan Kindermans, Huziel E. Sauceda, Stefan Chmiela, Alexandre Tkatchenko, and Klaus-Robert Müller.  Schnet: A continuous-filter convolutional neural network for modeling quantum interactions, 2017.
  * [75] Lei Huang, Hengtong Zhang, Tingyang Xu, and Ka-Chun Wong.  Mdm: Molecular diffusion model for 3d molecule generation, 2022.
  * [76] Pedro O. Pinheiro, Joshua Rackers, Joseph Kleinhenz, Michael Maser, Omar Mahmood, Andrew Martin Watkins, Stephen Ra, Vishnu Sresht, and Saeed Saremi.  3d molecule generation by denoising voxel grids, 2024.
  * [77] Olaf Ronneberger, Philipp Fischer, and Thomas Brox.  U-net: Convolutional networks for biomedical image segmentation, 2015.
  * [78] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Łukasz Kaiser, and Illia Polosukhin.  Attention is all you need.  In Advances in Neural Information Processing Systems, pages 6000–6010. Curran Associates, Inc., 2017.
  * [79] Yunhui Guo, Chaofeng Wang, Stella X. Yu, Frank McKenna, and Kincho H. Law.  Adaln: A vision transformer for multidomain learning and predisaster building information extraction from images.  Journal of Computing in Civil Engineering, 36(5), Sep 2022.
  * [80] Chenqing Hua, Sitao Luan, Minkai Xu, Rex Ying, Jie Fu, Stefano Ermon, and Doina Precup.  Mudiff: Unified diffusion for complete molecule generation, 2024.
  * [81] Vijay Prakash Dwivedi and Xavier Bresson.  A generalization of transformer networks to graphs, 2021.
  * [82] Ethan Perez, Florian Strub, Harm de Vries, Vincent Dumoulin, and Aaron Courville.  Film: Visual reasoning with a general conditioning layer, 2017.
  * [83] Gabriele Corso, Luca Cavalleri, Dominique Beaini, Pietro Liò, and Petar Veličković.  Principal neighbourhood aggregation for graph nets, 2020.
  * [84] Han Huang, Leilei Sun, Bowen Du, and Weifeng Lv.  Conditional diffusion based on discrete graph structures for molecular graph generation, 2023.
  * [85] Jie Lin, Mingyuan Xu, and Hongming Chen.  Diff-shape: A novel constrained diffusion model for shape based de novo drug design.  ChemRxiv, 2024.
  * [86] Lvmin Zhang, Anyi Rao, and Maneesh Agrawala.  Adding conditional control to text-to-image diffusion models, 2023.
  * [87] Naveen Arunachalam, Stefan Gugler, Michael G. Taylor, Chenru Duan, Aditya Nandy, Jon Paul Janet, Ralf Meyer, Jonas Oldenstaedt, Daniel B. Chu, and Heather J. Kulik.  Ligand additivity relationships enable efficient exploration of transition metal chemical space.  The Journal of Chemical Physics, 157(18), Nov 2022.
  * [88] Colin R. Groom, Ian J. Bruno, Matthew P. Lightfoot, and Suzanna C. Ward.  The cambridge structural database.  Acta Crystallographica Section B Structural Science, Crystal Engineering and Materials, 72(2):171–179, Apr 2016.
  * [89] Nathan Brown, Marco Fiscato, Marwin H.S. Segler, and Alain C. Vaucher.  Guacamol: Benchmarking models for de novo molecular design.  Journal of Chemical Information and Modeling, 59(3):1096–1108, March 2019.
  * [90] Janna Hastings, Gareth Owen, Adriano Dekker, Marcus Ennis, Namrata Kale, Venkatesh Muthukrishnan, Steve Turner, Neil Swainston, Pedro Mendes, and Christoph Steinbeck.  Chebi in 2016: Improved services and an expanding collection of metabolites.  Nucleic Acids Research, 44(D1), Oct 2015.
  * [91] Zhiyuan Liu, Sihang Li, Yanchen Luo, Hao Fei, Yixin Cao, Kenji Kawaguchi, Xiang Wang, and Tat-Seng Chua.  MolCA: Molecular graph-language modeling with cross-modal projector and uni-modal adapter.  In Houda Bouamor, Juan Pino, and Kalika Bali, editors, Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 15623–15638, Singapore, December 2023. Association for Computational Linguistics.
  * [92] Zheni Zeng, Yuan Yao, Zhiyuan Liu, and Maosong Sun.  A deep-learning system bridging molecule structure and biomedical text with comprehension comparable to human professionals.  Nature Communications, 13(1), Feb 2022.
  * [93] Bing Su, Dazhao Du, Zhao Yang, Yujie Zhou, Jiangmeng Li, Anyi Rao, Hao Sun, Zhiwu Lu, and Ji-Rong Wen.  A molecular multimodal foundation model associating molecule graphs with natural language, 2022.
  * [94] Tianze Luo, Zhanfeng Mo, and Sinno Jialin Pan.  Fast graph generation via spectral diffusion, 2022.
  * [95] Arne Schneuing, Yuanqi Du, Charles Harris, Arian Jamasb, Ilia Igashov, Weitao Du, Tom Blundell, Pietro Lió, Carla Gomes, Max Welling, Michael Bronstein, and Bruno Correia.  Structure-based drug design with equivariant diffusion models, 2023.
  * [96] Haitao Lin, Yufei Huang, Meng Liu, Xuanjing Li, Shuiwang Ji, and Stan Z. Li.  Diffbp: Generative diffusion of 3d molecules for target protein binding, 2022.
  * [97] Jiaqi Guan, Wesley Wei Qian, Xingang Peng, Yufeng Su, Jian Peng, and Jianzhu Ma.  3d equivariant diffusion for target-aware molecule generation and affinity prediction, 2023.
  * [98] Oleg Trott and Arthur J. Olson.  Autodock vina: improving the speed and accuracy of docking with a new scoring function, efficient optimization, and multithreading.  Journal of Computational Chemistry, 31(2):455–461, 2010.
  * [99] Xingang Peng, Shitong Luo, Jiaqi Guan, Qi Xie, Jian Peng, and Jianzhu Ma.  Pocket2mol: Efficient molecular sampling based on 3d protein pockets, 2022.
  * [100] Meng Liu, Youzhi Luo, Kanji Uchino, Koji Maruhashi, and Shuiwang Ji.  Generating 3d molecules for target protein binding, 2022.
  * [101] Xiangxin Zhou, Xiwei Cheng, Yuwei Yang, Yu Bao, Liang Wang, and Quanquan Gu.  Decompopt: Controllable and decomposed diffusion models for structure-based molecular optimization.  In The Twelfth International Conference on Learning Representations, 2024.
  * [102] Jiaqi Guan, Xiangxin Zhou, Yuwei Yang, Yu Bao, Jian Peng, Jianzhu Ma, Qiang Liu, Liang Wang, and Quanquan Gu.  Decompdiff: Diffusion models with decomposed priors for structure-based drug design, 2024.
  * [103] David Rooklin, Cheng Wang, Joseph Katigbak, Paramjit S Arora, and Yingkai Zhang.  Alphaspace: fragment-centric topographical mapping to target protein–protein interaction interfaces.  Journal of chemical information and modeling, 55(8):1585–1599, 2015.
  * [104] Joseph Katigbak, Haotian Li, David Rooklin, and Yingkai Zhang.  Alphaspace 2.0: representing concave biomolecular surfaces using $\beta$-clusters.  Journal of chemical information and modeling, 60(3):1494–1508, 2020.
  * [105] Zhilin Huang, Ling Yang, Zaixi Zhang, Xiangxin Zhou, Yu Bao, Xiawu Zheng, Yuwei Yang, Yu Wang, and Wenming Yang.  Binding-adaptive diffusion models for structure-based drug design, 2024.
  * [106] Hao Qian, Wenjing Huang, Shikui Tu, and Lei Xu.  Kgdiff: Towards explainable target-aware molecule generation with knowledge guidance.  Briefings in Bioinformatics, 25(1), Nov 2023.
  * [107] Zhilin Huang, Ling Yang, Xiangxin Zhou, Zhilong Zhang, Wentao Zhang, Xiawu Zheng, Jie Chen, Yu Wang, Bin CUI, and Wenming Yang.  Protein-ligand interaction prior for binding-aware 3d molecule diffusion models.  In The Twelfth International Conference on Learning Representations, 2024.
  * [108] Ling Yang, Zhilin Huang, Xiangxin Zhou, Minkai Xu, Wentao Zhang, Yu Wang, Xiawu Zheng, Wenming Yang, Ron O. Dror, Shenda Hong, and Bin CUI.  Prompt-based 3d molecular diffusion models for structure-based drug design, 2024.
  * [109] Lei Huang, Tingyang Xu, Yang Yu, Peilin Zhao, Xingjian Chen, Jing Han, Zhi Xie, Hailong Li, Wenge Zhong, Ka-Chun Wong, and et al.  A dual diffusion model enables 3d molecule generation and lead optimization based on target pockets.  Nature Communications, 15(1), Mar 2024.
  * [110] Julian Cremer, Tuan Le, Frank Noé, Djork-Arné Clevert, and Kristof T. Schütt.  Pilot: Equivariant diffusion for pocket conditioned de novo ligand generation with multi-objective guidance via importance sampling, 2024.
  * [111] Pedro O. Pinheiro, Arian Jamasb, Omar Mahmood, Vishnu Sresht, and Saeed Saremi.  Structure-based drug design by denoising voxel grids, 2024.
  * [112] Saeed Saremi and Aapo Hyvarinen.  Neural empirical bayes, 2020.
  * [113] Yael Ziv, Brian Marsden, and Charlotte M. Deane.  Molsnapper: Conditioning diffusion for structure based drug design.  Mar 2024.
  * [114] Ilia Igashov, Hannes Stärk, Clément Vignac, Victor Garcia Satorras, Pascal Frossard, Max Welling, Michael Bronstein, and Bruno Correia.  Equivariant 3d-conditional diffusion models for molecular linker design, 2022.
  * [115] Xingang Peng, Jiaqi Guan, Jian Peng, and Jianzhu Ma.  Pocket-specific 3d molecule generation by fragment-based autoregressive diffusion models, 2023.
  * [116] Mahdi Ghorbani, Leo Gendelev, Paul Beroza, and Michael J. Keiser.  Autoregressive fragment-based diffusion for pocket-aware ligand design, 2023.
  * [117] Nicholas T. Runcie and Antonia S.J.S. Mey.  Silvr: Guided diffusion for molecule generation.  Journal of Chemical Information and Modeling, 63(19):5996–6005, September 2023.
  * [118] Shitong Luo, Chence Shi, Minkai Xu, and Jian Tang.  Predicting molecular conformation via dynamic graph score matching.  In Neural Information Processing Systems, 2021.
  * [119] Bowen Jing, Gabriele Corso, Jeffrey Chang, Regina Barzilay, and Tommi Jaakkola.  Torsional diffusion for molecular conformer generation, 2023.
  * [120] Haotian Zhang, Shengming Li, Jintu Zhang, Zhe Wang, Jike Wang, Dejun Jiang, Zhiwen Bian, Yixue Zhang, Yafeng Deng, Jianfei Song, Yu Kang, and Tingjun Hou.  Sdegen: learning to evolve molecular conformations from thermodynamic noise for conformation generation.  Chem. Sci., 14:1557–1568, 2023.
  * [121] Gabriele Corso, Hannes Stärk, Bowen Jing, Regina Barzilay, and Tommi Jaakkola.  Diffdock: Diffusion steps, twists, and turns for molecular docking, 2023.
  * [122] Andrew T. McNutt, Paul Francoeur, Rishal Aggarwal, Tomohide Masuda, Rocco Meli, Matthew Ragoza, Jocelyn Sunseri, and David Ryan Koes.  Gnina 1.0: Molecular docking with deep learning.  Journal of Cheminformatics, 13(1), Jun 2021.
  * [123] Mohamed Amine Ketata, Cedrik Laue, Ruslan Mammadov, Hannes Stärk, Menghua Wu, Gabriele Corso, Céline Marquet, Regina Barzilay, and Tommi S. Jaakkola.  Diffdock-pp: Rigid protein-protein docking with diffusion models, 2023.
  * [124] Michael Brocidiacono, Konstantin I. Popov, David Ryan Koes, and Alexander Tropsha.  Plantain: Diffusion-inspired pose score minimization for fast and accurate molecular docking, 2023.
  * [125] Shuya Nakata, Yoshiharu Mori, and Shigenori Tanaka.  End-to-end protein–ligand complex structure generation with diffusion-based generative models.  BMC Bioinformatics, 24(1), Jun 2023.
  * [126] Taras Voitsitskyi, Volodymyr Bdzhola, Roman Stratiichuk, Ihor Koleiev, Zakhar Ostrovsky, Volodymyr Vozniak, Ivan Khropachov, Pavlo Henitsoi, Leonid Popryho, Roman Zhytar, and et al.  Augmenting a training dataset of the generative diffusion model for molecular docking with artificial binding pockets.  RSC Advances, 14(2):1341–1353, 2024.
  * [127] Zhuoran Qiao, Weili Nie, Arash Vahdat, Thomas F. Miller III au2, and Anima Anandkumar.  State-specific protein-ligand complex structure prediction with a multi-scale deep generative model, 2023.
  * [128] Wengong Jin, Siranush Sarkizova, Xun Chen, Nir Hacohen, and Caroline Uhler.  Unsupervised protein-ligand binding energy prediction via neural euler’s rotation equation, 2023.
  * [129] Yuejiang Yu, Shuqi Lu, Zhifeng Gao, Hang Zheng, and Guolin Ke.  Do deep learning models really outperform traditional approaches in molecular docking?, 2023.
  * [130] Martin Buttenschoen, Garrett M. Morris, and Charlotte M. Deane.  Posebusters: Ai-based docking methods fail to generate physically valid poses or generalise to novel sequences, 2023.
  * [131] Fang Wu and Stan Z. Li.  Diffmd: A geometric diffusion model for molecular dynamics simulations, 2023.
  * [132] Stefan Chmiela, Alexandre Tkatchenko, Huziel E. Sauceda, Igor Poltavsky, Kristof T. Schütt, and Klaus-Robert Müller.  Machine learning of accurate energy-conserving molecular force fields.  Science Advances, 3(5), May 2017.
  * [133] Marloes Arts, Victor Garcia Satorras, Chin-Wei Huang, Daniel Zuegner, Marco Federici, Cecilia Clementi, Frank Noé, Robert Pinsler, and Rianne van den Berg.  Two for one: Diffusion models and force fields for coarse-grained molecular dynamics, 2023.
  * [134] Kevin Yang, Kyle Swanson, Wengong Jin, Connor Coley, Philipp Eiden, Hua Gao, Angel Guzman-Perez, Timothy Hopper, Brian Kelley, Miriam Mathea, Andrew Palmer, Volker Settels, Tommi Jaakkola, Klavs Jensen, and Regina Barzilay.  Analyzing learned molecular representations for property prediction, 2019.
  * [135] Ilya O Tolstikhin, Bharath K. Sriperumbudur, and Bernhard Schölkopf.  Minimax estimation of maximum mean discrepancy with radial kernels.  In D. Lee, M. Sugiyama, U. Luxburg, I. Guyon, and R. Garnett, editors, Advances in Neural Information Processing Systems, volume 29. Curran Associates, Inc., 2016.
  * [136] Peter Ertl and Ansgar Schuffenhauer.  Estimation of synthetic accessibility score of drug-like molecules based on molecular complexity and fragment contributions.  Journal of Cheminformatics, 1:1–11, 2009.
  * [137] Christopher A Lipinski, Franco Lombardo, Brian W Dominy, and Patrick J Feeney.  Experimental and computational approaches to estimate solubility and permeability in drug discovery and development settings.  Advanced drug delivery reviews, 64:4–17, 2012.
  * [138] Wolfgang Kabsch.  A solution for the best rotation to relate two sets of vectors.  Acta Crystallographica Section A: Crystal Physics, Diffraction, Theoretical and General Crystallography, 32(5):922–923, 1976.
  * [139] Fang Du, Haibo Yu, Beiyan Zou, Joseph Babcock, Shunyou Long, and Min Li.  hergcentral: a large database to store, retrieve, and analyze compound-human ether-a-go-go related gene channel interactions to facilitate cardiotoxicity assessment in drug development.  Assay and drug development technologies, 9(6):580–588, 2011.
  * [140] Charles Harris, Kieran Didi, Arian R. Jamasb, Chaitanya K. Joshi, Simon V. Mathis, Pietro Lio, and Tom Blundell.  Benchmarking generated poses: How rational is structure-based drug design with generative models?, 2023.