# De novo antibody design with SE(3) diffusion

Daniel Cutting1∗ Frédéric A. Dreyer1∗ David Errington1 Constantin Schneider1
Charlotte M. Deane1,2
1Exscientia, Oxford Science Park, Oxford, OX4 4GE, UK
2Department of Statistics, University of Oxford, Oxford OX1 3LB, UK
<EMAIL_ADDRESS>
<EMAIL_ADDRESS>
###### Abstract

We introduce IgDiff, an antibody variable domain diffusion model based on a
general protein backbone diffusion framework which was extended to handle
multiple chains. Assessing the designability and novelty of the structures
generated with our model, we find that IgDiff produces highly designable
antibodies that can contain novel binding regions. The backbone dihedral
angles of sampled structures show good agreement with a reference antibody
distribution. We verify these designed antibodies experimentally and find that
all express with high yield. Finally, we compare our model with a state-of-
the-art generative backbone diffusion model on a range of antibody design
tasks, such as the design of the complementarity determining regions or the
pairing of a light chain to an existing heavy chain, and show improved
properties and designability.

**footnotetext: Equal contribution.

##  1 Introduction

Engineering novel proteins that can satisfy specified functional properties is
the central aim of rational protein design. While sequence-based methods have
seen some success (Wu et al., 2021), they are intrinsically limited by the
fact that most properties of a molecule, such as binding or solubility, are
determined by their three-dimensional structure. Recent advances in diffusion
models (Ho et al., 2020; Song et al., 2021), a class of deep probabilistic
generative models, have shown promise as a data-driven alternative to more
computationally expensive physics-based methods (Alford et al., 2017) in
tackling de novo protein design. Most approaches focus on modelling only the
backbone (Watson et al., 2022; Lin and AlQuraishi, 2023), while the sequence
is inferred through an inverse folding model, though some full-atom models
have been explored (Chu et al., 2023; Martinkus et al., 2023).

An application of particular therapeutic relevance is the design of
immunoglobulin proteins, which play a central role in helping the adaptive
immune system identify and neutralise pathogens. They consist of two heavy and
two light chains. These are separated into constant domains that specify
effector function, and a variable domain that contains six hypervariable
loops, known as the complementarity determining regions (CDRs), which control
binding specificity. The third loop of the heavy chain, CDR-H3, is the most
structurally diverse domain of the antibody, and often determines antigen
recognition (Narciso et al., 2011; Tsuchiya and Mizuguchi, 2016). Monoclonal
antibodies are an emerging drug modality with the potential for applications
in a wide range of therapeutic areas, for example oncogenic, infectious and
autoimmune diseases. They can be adapted to target specific antigens or
receptors through engineering of the binding site (Chiu et al., 2019).

In this article, we consider the recent backbone diffusion model FrameDiff
(Yim et al., 2023) and adapt it for applications to antibody variable regions.
We then fine-tune it on synthetic antibody structures from the ImmuneBuilder
dataset (Abanades et al., 2022). Our IgDiff model is trained to generate the
paired variable region backbone, with a paired heavy and light chain. We study
the designability and novelty of the structures generated by our heavy chain
model and predict the corresponding sequences with AbMPNN (Dreyer et al.,
2023), an antibody-specific inverse folding model based on ProteinMPNN
(Dauparas et al., 2022). We then consider a range of design tasks and show how
IgDiff can outperform existing backbone diffusion methods on antibody
engineering tasks.

##  2 Related work

Denoising diffusion probabilistic models (Sohl-Dickstein et al., 2015; Ho et
al., 2020) have shown promising results to the task of de novo protein design.
Recent models based on a $C_{\alpha}$-only backbone representation of proteins
(Trippe et al., 2023) have already been adapted to account for the full frame
representation of backbone residues (Yim et al., 2023; Lin and AlQuraishi,
2023) through the application of diffusion on Riemannian manifolds (Bortoli et
al., 2022). Notably, tools such as RFDiffusion (Watson et al., 2022), which
rely on a fine-tuned RoseTTAFold structure prediction network for the reverse
diffusion process, can tackle a number of complex protein engineering tasks
such as protein binder design, enzyme active site scaffolding or topology-
constrained protein monomer design. These diffusion models often operate in
structural space, and an inverse folding model such as ProteinMPNN (Ingraham
et al., 2019; Dauparas et al., 2022) is needed to recover sequences
corresponding to the predicted structures. To reconstruct the placement of
side-chain atoms, physics-based methods packing tools such as Rosetta (Alford
et al., 2017) can be used. AI-driven side-chain packing methods based on SE(3)
transformers (Fuchs et al., 2020), or a diffusion model on the joint
distribution of torsional angles (Zhan et al., 2023) can also offer an
efficient alternative.

Alternatively, protein diffusion models have been applied in sequence space to
directly generate amino acid sequences (Lisanza et al., 2023) which can allow
for sequence specific attributes and functional properties. Classifier-guided
discrete diffusion models such as NOS (Gruver et al., 2023) can for example
leverage a discriminative model and the large amount of available sequence
data to predict sequences with high fitness.

Antibody design has seen rapid progress and will benefit from these advances
in generative protein models. This task was traditionally tackled using
energy-based optimization strategies (Li et al., 2014; Adolf-Bryfogle et al.,
2018), and more recently with sequence-based language models (Liu et al.,
2019; Saka et al., 2021). Several structure-based approaches relying on graph
neural networks, such as RefineGNN (Jin et al., 2022b), HSRN (Jin et al.,
2022a), MEAN (Kong et al., 2023b) and its subsequent end-to-end design model
dyMEAN (Kong et al., 2023a), or sequence and structure co-design diffusion
models Luo et al. (2022), have shown promising results in predicting the CDR
design of antibodies to target a specific epitope on an antigen. The FrameDiff
framework has also been recently explored for inpainting CDR loops of T-cell
receptors (Zhang et al., 2023), while other recent work has explored the
application of structure-based diffusion models to the design of nanobodies
(Bennett et al., 2024) and antibodies (Martinkus et al., 2023).

##  3 SE(3) protein backbone diffusion model

We review the SE(3) diffusion framework introduced by Yim et al. (2023), which
constructs an explicit framework for the diffusion of protein backbones based
on the Riemannian score-based generative modeling approach of Bortoli et al.
(2022).

For the backbone frame parametrisation we adopt the same formalism as in
AlphaFold2 (Jumper et al., 2021), using a collection of $N$ orientation
preserving rigid transformations to represent an $N$ residue backbone, as
shown in figure 1. These frames map from fixed coordinates of the four heavy
atoms $N^{*},C_{\alpha}^{*},C^{*},O^{*}\in\mathbb{R}^{3}$ centered at
$C_{\alpha}^{*}=\vec{0}$, assuming experimentally measured bond lengths and
angles (Engh and Huber, 2012). The main backbone atomic coordinates for a
residue $i$ are given through

$[N_{i},C_{i},C_{\alpha,i}]=T_{i}\cdot[N^{*},C^{*},C^{*}_{\alpha}]\,,$ (1)

where $T_{i}\in\mathrm{SE}(3)$ is a member of the special Euclidean group, the
set of valid translations and rotations in Euclidean space. A backbone
consists of $N$ frames $[T_{1},\dots T_{N}]\in\mathrm{SE}(3)^{N}$, with the
oxygen atom $O$ being reconstructed from an additional torsion angle
$\psi\in\mathrm{SO}(2)$ around the $C_{\alpha}$ and $C$ bond. Each frame is
decomposed into $T_{i}=(r_{i},x_{i})$, where $x_{i}\in\mathbb{R}^{3}$ is the
$C_{\alpha}$ translation and $r_{i}\in\mathrm{SO}(3)$ is a $3\times 3$
rotation matrix which can be derived from relative atom positions with the
Gram-Schmidt process. A diffusion process over $\mathrm{SE}(3)^{N}$ can be
constructed to achieve global SE(3) invariance by keeping the diffusion
process centered at the origin.

Figure 1: Left: Schematic representation of an antibody. Centre: Backbone
atoms of the variable region, showing both a heavy (green) and light (blue)
chain domain. Right: The parametrisation of residues into frames used by the
diffusion model, with each frame consisting of four heavy atoms connected by
rigid covalent bonds.

We model the distribution over $\mathrm{SE}(3)^{N}$ through Riemannian score-
based generative modeling, which aims to sample from a distribution supported
on a Riemmanian manifold $\mathcal{M}$ by reversing a forward process that
evolves from the data distribution $p_{0}$ towards an invariant density
$p_{T}$ through

$d\mathbf{X}_{t}=-\tfrac{1}{2}\nabla
U(\mathbf{X}_{t})dt+d\mathbf{B}_{t,\mathcal{M}}\,,\quad\mathbf{X}_{0}\sim
p_{0}\,,$ (2)

where $\mathbf{B}_{t,\mathcal{M}}$ is the Brownian motion on $\mathcal{M}$,
$U(x)$ is a continuously differentiable variable defining the invariant
density $p_{T}\propto e^{-U(x)}$, $\nabla$ is the Riemannian gradient, and
$t\in[0,T]$ is a continuous time variable. The time-reversed process for
$\mathbf{Y}_{t}=\mathbf{X}_{T-t}$ also satisfies a stochastic differential
equation given by

$d\mathbf{Y}_{t}=\big{[}\tfrac{1}{2}\nabla U(\mathbf{Y}_{t})+\nabla\log
p_{T-t}(\mathbf{Y}_{t})\big{]}dt+d\mathbf{B}_{t,\mathcal{M}}\,,\quad\mathbf{Y}_{0}\sim
p_{T}\,,$ (3)

where $p_{t}$ is the density of $\mathbf{X}_{t}$. The Riemannian gradients and
Brownian motion depend on a choice of inner product on $\mathcal{M}$, which
for SE(3) can simply be derived from the canonical inner products on SO(3) and
$\mathbb{R}^{3}$. The invariant density on SE(3) is chosen as
$p_{T}\propto\mathcal{U}^{\mathrm{SO}(3)}(r)\,\mathcal{N}(x)$.

The Stein score $\nabla\log p_{t}$ itself is intractable and is therefore
approximated with a score network $s_{\theta}$ which is trained with a
denoising score matching loss given by

$\mathcal{L}_{\mathrm{DSM}}(\theta)=\mathbb{E}\big{[}\lambda_{t}\|\nabla\log
p_{t|0}(\mathbf{X}_{t}|\mathbf{X}_{0})-s_{\theta}(t,\mathbf{X}_{t})\|^{2}\big{]}\,,$
(4)

where $\lambda_{t}$ is a weighting schedule, $p_{t|0}$ is the density of
$\mathbf{X}_{t}$ given $\mathbf{X}_{0}$, and the expectation is taken over $t$
and the distribution of $(\mathbf{X}_{0},\mathbf{X}_{t})$. The loss on SE(3)
is decomposed into its translation and rotation components as
$\mathcal{L}_{\mathrm{DSM}}=\mathcal{L}^{x}_{\mathrm{DSM}}+\mathcal{L}^{r}_{\mathrm{DSM}}$.

To mitigate chain breaks or steric clashes and to learn the torsion angle
$\psi$, two auxiliary losses are used. The first one is a direct mean squared
error on the backbone positions $\mathcal{L}_{bb}$, while the second one is a
local neighbourhood loss on pairwise atomic distances $\mathcal{L}_{2D}$.
These losses are applied with a weight $w$ when sampling $t$ near 0, when
fine-grained characteristics of the protein backbone emerge, such that the
full training loss is expressed as

$\mathcal{L}=\mathcal{L}_{\mathrm{DSM}}+w\,\Theta\big{(}\tfrac{T}{4}-t\big{)}\big{(}\mathcal{L}_{bb}+\mathcal{L}_{2D}\big{)}\,.$
(5)

The score network is based on the structure module of AlphaFold2 (Jumper et
al., 2021) and performs iterative updates over $L$ layers by combining spatial
and sequence based attention modules using an Invariant Point Attention and a
Transformer (Vaswani et al., 2017), considering a fully connected graph
structure. As well as a denoised frame, the network also predicts the torsion
angle $\psi$ for each residue, from which the positions of the backbone oxygen
atoms can be reconstructed.

Sampling is achieved through an Euler-Maruyama discretisation of equation (3)
which is approximated with a geodesic random walk (Jørgensen, 1975). To avoid
destabilisation of the backbone in the final sampling steps, trajectories are
instead truncated at a time $\epsilon>0$. For all numerical applications, we
use identical parameters to the original FrameDiff model (Yim et al., 2023).

We train this SE(3) diffusion model on synthetic antibody structural data,
specifically targeting the variable domains whose CDR loops play a key role in
defining the binding properties of the antibody. Our dataset consists of
148,832 variable regions from the Observed Antibody Space (OAS) (Kovaltsuk et
al., 2018; Olsen et al., 2022), a database of antibody sequences, for which
structures were predicted with ABodyBuilder2 (ABB2) (Abanades et al., 2022),
an antibody structure prediction model based on the structure module of
AlphaFold-Multimer (Evans et al., 2022).

Our IgDiff model is obtained by fine-tuning the updated FrameDiff weights from
July 2023 for 18 hours on 8 NVIDIA A10G GPUs, using an Adam optimizer (Kingma
and Ba, 2017) with a learning rate of $10^{-4}$ and a batch size of 64. To
encode the chain break between the heavy and light domains, we add a jump of
50 in the residue index across chains. We also include a heavy or light chain
label as an additional node feature. The training data is obtained by
clustering the concatenated CDR heavy and light chain sequences at 80%
similarity using CDHIT (Fu et al., 2012). For each epoch, we train on one
representative from each cluster. Batches are constructed from equal length
antibodies, where we complete a batch with previously seen antibodies if
necessary.

In Figure 2, we show examples of unconditioned structures obtained with
IgDiff, as well as conditioned samples for each of the design tasks considered
in this study. In order to determine the CDR regions, we use the IMGT
numbering scheme (Lefranc et al., 2003) and North definition (North et al.,
2011) throughout our analysis.

Figure 2: Examples of IgDiff generated antibody structures. Light chains are
highlighted in blue, heavy chains in green. (A-B) Unconditionally generated
antibodies. (A) IgDiff generated antibody (dark green/dark blue) compared to
ABodyBuilder2 prediction on the lowest self-consistency RMSD sequence
predicted by AbMPNN (light green/cyan) and the closest match in the training
set by TM-score (grey). (B) Multiple antibodies generated by IgDiff with the
same heavy and light chain length settings. (C-E) Conditionally generated
antibodies, designed regions in orange, non-designed regions in dark blue/dark
green, original input structure in cyan/light green. (C) Conditional
generation of all CDR loops. (D) Conditional generation of CDR H3 with a
different length compared to the input structure. (E) Conditional generation
of the entire light chain.

##  4 Generating de novo antibody variable domains

Let us now consider samples generated with IgDiff and study their properties.
We first analyse the ability of our IgDiff model to sample plausible, novel,
and diverse antibodies, when generating the entire variable domain backbone of
both heavy and light chains.

Using our fine-tuned model, we start by sampling unconditioned paired heavy
and light variable region structures. We obtain 100 unique combinations of
chain lengths by sampling uniformly for the heavy and light chain
independently between the 2.5 and 97.5 percentiles of the corresponding
lengths found in the paired OAS. We then sample 8 structures for each of the
100 unique combinations for a total of 800 unique structures generated with
IgDiff.

Sequences are predicted using the antibody-specific inverse folding model
AbMPNN (Dreyer et al., 2023), an adaptation of the general protein model
ProteinMPNN (Dauparas et al., 2022) to antibodies. We sample 20 sequences for
each generated structure with temperature $T=0.2$. To determine the best
sequence for a given structure, we fold the sequence with ABB2 and compare it
to the IgDiff structure. The root mean squared deviation (RMSD) between the
model predicted for a sequence and the initial generated structure is referred
to as the self-consistency RMSD (scRMSD). For each generated sample, we keep
the sequence that results in the lowest scRMSD.

For consistency and diversity metrics, we compare to a baseline of 800
structures drawn from the training set of ABB2 predictions on paired OAS. For
a fair comparison, we select 8 structures for each unique length combination
of light and heavy chains in the IgDiff unconditioned dataset.

###  4.1 Consistency

Figure 3: Left: Ramachandran plot of the dihedral angle distribution comparing
the heavy chain residues from the predicted structures of OAS to IgDiff.
Right: Same comparison for the light chain.

We demonstrate that the IgDiff generated structures follow consistent
structural distributions by comparing their backbone dihedral angle
distributions in Ramachandran space to that of the OAS dataset. In Fig. 3, we
show that the IgDiff $\phi$ and $\psi$ angle distribution closely matches that
of the structures of naturally occurring antibody sequences from the paired
OAS predicted with the modelling tool ABB2.

All generated antibodies have an scRMSD for the best predicted sequence that
is below 2Å across the entire antibody, while 88% pass this threshold across
each region independently, including all CDR loops. We also assess the
consistency by considering the canonical clusters assumed by the CDR loops
(Chothia and Lesk, 1987) and comparing those with the repredicted ABB2
structures, finding that across each region, more than 91% are assigned to the
same canonical form. Further studies of the scRMSD of generated antibodies and
the distribution of canonical clusters in the CDR loops are given in Appendix
A.

When ABB2 outputs a prediction of a structure from a sequence, it first
predicts an ensemble of four structures, and then selects the structure
closest to the mean structure. This also allows it to provide a predicted
error metric for each residue, computed from the mean deviation of the
predicted residue position across the ensemble. When averaged across a set of
residues this metric is referred to as the root mean square predicted error or
RMSPE. We use the RMSPE evaluated across the full variable domain as an
additional check for consistency, for which a generated structure will pass if
the RMSPE is less than the 90th percentile of the RMSPE on the ABB2 testset.
We find that 79.1% of our generated structures pass this test.

###  4.2 Novelty and diversity

We investigate the novelty of antibodies generated by IgDiff without
conditioning by calculating the CDR H3 RMSD to the closest match by TM-score
(Zhang and Skolnick, 2004) with identical CDR H3 loop length in the training
dataset. This is shown in Fig. LABEL:fig:novelty-diversity (left), comparing
the distribution of CDR H3 RMSD of IgDiff generated antibodies to that of a
randomly drawn set of antibodies from the training dataset with the same
distribution of CDR H3 loop lengths. Generated antibodies have similar novelty
as randomly drawn antibodies from the training dataset, with mean CDR H3 RMSD
to the closest match of 1.39A $\pm$ 0.56 for IgDiff generated antibodies and
1.50A $\pm$ 0.74 for naturally observed antibodies, demonstrating that the
model can produce novel H3 loops.

Figure 4: Left: Novelty of generated CDR H3 structures compared to random
samples from the OAS dataset. For each structure, the RMSD shown is the CDR H3
RMSD to the closest match in the OAS dataset by TM-score with the same CDR H3
length. Right: Pairwise RMSD between IgDiff generated CDR loops of the same
length. fig:novelty-diversity

We consider also the diversity of antibodies generated by IgDiff, an important
consideration for antibody engineering. Taking CDRs with the same loop length,
regardless of specified chain length, we calculate the pairwise RMSD between
loops. This is shown in Fig. LABEL:fig:novelty-diversity (right), where we can
observe diversity of generated CDR loops across different samples.
Interestingly, we observe a bimodal distribution of pairwise RMSDs for CDRs
L1, L3 and H2, arising from two generated loops either assuming the same or
different canonical conformations. Further studies of the distribution of loop
lengths as a function of specified chain length and exploration of sequence
diversity are discussed in Appendix A.

###  4.3 Experimental validation

In order to demonstrate that IgDiff generated antibodies are able to express
and exhibit favorable designability characteristics, we selected 28 antibody
sequences for experimental validation. Prior to selection we first excluded
any of the sequences that had higher than the median CDRH3 scRMSD or higher
than the median overall scRMSD. As detailed in Appendix B all selected
antibodies expressed and yielded sufficiently high concentration for
downstream characterisation.

##  5 Antibody design tasks

Targeted design of specific regions of an antibody is of interest in a number
of important applications. The engineering of the interface region, notably
the CDR loops, can be of particular relevance for therapeutic applications
(Akbar et al., 2021; Shanehsazzadeh et al., 2023). Furthermore, the pairing of
an appropriate heavy or light chain to a given domain is of importance in
antibody discovery and in shaping antibody repertoires (DeKosky et al., 2015).

We thus consider the performance of IgDiff on conditional sampling for several
inpainting design tasks of particular relevance to antibody engineering. The
tasks we consider are (i) the design of all CDR loops given a fixed heavy and
light chain framework, (ii) the design of a complete light chain given a heavy
chain, and (iii) the design of a CDRH3 loop with varying length given the
remaining variable region, thus allowing for additional contact interactions
from the longer loop.

We generate 10 structures per reference structure and CDR length. All starting
point antibodies are taken from the ABB2 test set. In order to perform
inpainting, at each time $t$ during inference we replace each frame in the
fixed region with the corresponding frame from the reference structure after
applying the forward noising process to time $t$.

In table 1, we show self-consistency metrics and compare our IgDiff model
against RFDiffusion on each design task. None of the light chains designed by
RFDiffusion can be parsed by Anarci (Dunbar and Deane, 2015) after inverse
folding, while IgDiff always generates valid light chains, of which 93.3% pass
our confidence and scRMSD test. The CDRH3 length change task leads to 74% of
IgDiff generated structures that pass our combined test, compared with only 6%
of RFDiffusion designs. For the task of designing all CDR loops, none of the
RFDiffusion structures pass the scRMSD test, while 4% of the IgDiff structures
have scRMSD below 2Å across all regions. Note here that this low success rate
in the case of IgDiff is primarily driven by a high scRMSD in the CDRH1 loop,
while the remaining regions have a success rate above 75%. In contrast
RFDiffusion also has poor self-consistency in CDRH1, CDRH3, CDRL1, and CDRL3.
Further analysis of the scRMSD distribution and canonical clusters of designs,
as well as details of the starting point antibodies used for each design task,
are given in Appendix B.

| IgDiff | RFDiffusion
---|---|---
Task | scRMSD | Confidence | Combined | scRMSD | Confidence | Combined
Unconditioned | 0.88 | 0.79 | 0.76 | - | - | -
Design all CDRs | 0.04 | 0.80 | 0.02 | 0.00 | 0.54 | 0.00
Change CDRH3 length | 0.76 | 0.91 | 0.74 | 0.08 | 0.69 | 0.06
Design light chain | 0.93 | 1 | 0.93 | - | - | -

Table 1: Success rate of inpainted samples for each design task and three
different tests, comparing IgDiff and RFDiffusion. The scRMSD test requires
all regions to have scRMSD < 2Å independently. The confidence test requires
the ABB2 RMSPE averaged over all residues to be less than the 90th percentile
of the same metric evaluated on the ABB2 test set. The combined test metric
reports the fraction of designs passing both previous tests.

##  6 Conclusions

In this article, we introduce a model for de novo antibody generation, IgDiff.
This model is derived from the recent SE(3) diffusion framework FrameDiff, by
fine-tuning on antibody variable domains. The weights of our IgDiff model are
made publicly available (Dreyer and Cutting, 2024).

We show that our antibody backbone model is able to recapitulate the expected
backbone dihedral distribution, and studied the validity of the sequences
recovered from generated samples using an antibody-specific inverse folding
model. Studying the designability of the generated structures by comparing
them with structure predictions based on the corresponding sequences, we found
excellent agreement. We probed our model for novelty by finding the closest
match in the training data for each sampled structure and found it could
generate structures distinct from those in the training set. We selected a
number of designs for experimental validation, finding that all generated
antibodies express with high yield. We further found that IgDiff generates
diverse antibodies, particularly in the CDR loops that are important
determinants of binding affinity, making it well suited for antibody design
tasks.

Finally, we considered examples of antibody engineering tasks, such as the
redesign of CDR loops or the generation of a light chain paired to a specified
heavy chain, and demonstrated the applicability of our approach in practical
use cases. We designed metrics to assess the quality of generated structures,
and showed substantial improvement over existing state-of-the-art protein
diffusion models.

Diffusion models trained on antibodies offer a promising approach to
accelerate drug design through data-driven generative AI. In this article, we
provide a key step towards the generation of viable therapeutic antibodies
through structure-based diffusion. Conditioning the generation of samples to
express desired properties, as well as to target specified antigens, will be a
crucial elements towards facilitating their application in therapeutic
development.

## Acknowledgements

We are grateful to Henry Kenlay, Claire Marks, Douglas Pires, Aleksandr
Kovaltsuk and Newton Wahome for useful discussions.

##  Appendix A Designability and novelty of unconditioned generation

In Figure 5, we show the expression yields for 28 selected antibody sequences
predicted with AbMPNN from full unconditioned variable structures produced
with IgDiff.

Figure 5: Expression yields (mg/L) of a validation set of IgDiff generated
antibodies.

We assess the scRMSD of IgDiff generated antibody structures to corresponding
ABB2 predicted structures. To this end, we predict 20 amino acid sequences for
each IgDiff generated backbone structure using AbMPNN and use ABB2 to predict
the structure those sequences are likely to assume, reporting the scRMSDs for
the AbMPNN prediction that achieves the lowest overall scRMSD across the 20
predictions for each IgDiff output, as shown in Fig. 6. A lower scRMSD in this
scenario indicates that the IgDiff generated antibody structures represent
realistic and designable antibody structures. We show that the structures are
designable in that the scRMSD in all loops are typically lower than the 2 Å
cutoff used to define designability in the original FrameDiff paper (Yim et
al., 2023). All IgDiff generated antibodies pass the 2Å threshold across the
entire antibody and 88% of IgDiff generated antibodies also pass for every
sub-region assessed independently in Fig. 6, including all CDR loops.

Figure 6: scRMSD of IgDiff structures against ABB2 re-predictions, compared to
inherent ABB2 RMSD. (Left) IgDiff scRMSD to the ABB2 predicted structure for
the AbMPNN sequence producing the lowest overall scRMSD to the input IgDiff
structure. The red line indicates the 2Å cutoff used to designate a designable
structure in FrameDiff (Yim et al., 2023). (Right) RMSD of ABB2 predictions on
the ABB2 testset against the ground truth crystal structures. Mean IgDiff
scRMSD on each region are lower than the corresponding ABB2 test set RMSD.

We further assessed self-consistency through the canonical clusters assumed by
the non-H3 CDR loops in both IgDiff generated structures and ABB2 repredicted
structures (generated as described for the scRMSD calculation). To this end,
each loop is assigned to the closest PyIGClassify cluster centre by RMSD
Adolf-Bryfogle et al. (2015). If a loop has an unusual length with no clusters
or has $\text{RMSD}>1.5$Å to the closest cluster centre then it is denoted as
unclassified.

For non-H3 loops, IgDiff structures and ABB2 repredicted structures were
assigned the same canonical class in between 93% (CDR L1) and 98% (CDR H1) of
samples for the less variable CDRs 1 and 2 and in 85% of samples for the more
variable CDR L3. Up to 8% (CDRH2) of generated designs fell outside of the
known canonical class for that loop both in the IgDiff generated structure and
in the ABB2 repredicted structure (see Table 2). We further show the RMSD to
the assigned cluster centre for each loop generated by IgDiff and ABB2
respectively, as well as for a baseline of OAS paired ABB2 predicted
structures (see Fig. 7). We demonstrate that RMSD to the closest cluster
centre depends heavily on the canonical class of the loop and that IgDiff
generated antibodies follow a similar distribution of distances to the closest
cluster centre as the ABB2 repredicted structures and a baseline set of OAS
derived, ABB2 predicted structures.

Region |  Matching clusters |  Mismatching clusters |  Both unclassified
---|---|---|---
CDRH1 | 0.98 | 0.01 | 0.01
CDRH2 | 0.92 | 0.00 | 0.08
CDRL1 | 0.93 | 0.03 | 0.04
CDRL2 | 0.95 | 0.01 | 0.04
CDRL3 | 0.85 | 0.09 | 0.06
Table 2: Canonical cluster analysis of antibody structures generated
unconditionally using IgDiff and the ABB2 repredicted structures. Figure 7:
RMSD to the central cluster representative in the PyIGClassify dataset for all
classified canonical loops. Each panel refers to a different CDR loop. For
each panel we show the result for the generated IgDiff structures on the left,
the results for the ABB2 structures predicted on the IgDiff sequences in the
centre, and a baseline of paired OAS structure predictions on the right. In
each plot the individual datapoints that make up a box-plot are shown as a
swarm-plot with cluster class indicated by the point hue.

For heavy or light chains generated with the same length specification, we
investigate the distribution of loop length for each CDR, as CDRs with
different lengths can necessarily be considered diverse. This is shown in Fig.
8. We observe that IgDiff generates diverse CDR loop lengths largely
independent of the pre-specified chain length. For the light chain, diversity
of CDR loop lengths increases with increasing chain length, while for the
heavy chain, the diversity of generated CDR loops remains broadly constant
across chain lengths (with the exception of CDR H1 at short chain lengths). We
further observe that IgDiff drives chain length of the heavy chain primarily
via CDR H3 length and light chain length via CDR L1 and L3 length, mirroring
the natural distributions of CDR loop lengths.

Figure 8: Distribution of CDR loop lengths depending on the generated heavy or
light chain length.

To explore sequence diversity, we find the minimum pairwise Levenshtein
distance (also known as edit distance) between the sequence of a generated
IgDiff structure and the sequences in the rest of the generated structures.
The resulting distribution of pairwise distances is shown in Fig. 9. Each
generated antibody has at least 3 edits between the closest sequence, and at
most 56 edits. The median number of edits to the closest sequence is 15. For a
comparison we also show the minimum pairwise edit distances between 800
example predicted structures taken from the paired OAS dataset with the same
chain lengths as the IgDiff unconditioned dataset. Both distributions are
similar, with IgDiff producing fewer antibodies with very low and very high
minimum edit distances and more antibodies with medium edit distances than the
OAS baseline.

Figure 9: Histogram of the minimum pairwise Levenshtein distance between 800
unconditioned IgDiff generated structures (blue) and 800 paired OAS sequences
(orange).

##  Appendix B Benchmarking of design tasks

The starting point antibodies used in each design task are given in Table 3.
In this appendix, we consider also the task of redesigning the CDRL3 loop with
varying lengths, to study the recapitulation of the canonical clusters of
designs with modified loop lengths. This task is left out of the main text for
conciseness but achieves 100% success on the scRMSD test for IgDiff compared
to 85% for RFDiffusion, with both models obtaining 100% success on the
confidence test.

Design task | Subtasks | Reference antibody | Total samples
---|---|---|---
CDRH3 length change | H3 lengths 10-19 | 7seg | 100
CDRL3 length change | L3 lengths 8-11 | 7sem | 40
Design all CDRs | Different antibodies | 7ps6, 7q4q, 7rp2, 7ttm, 7u8c | 50
Design light chain | Different heavy chains | 7qf0, 7rxl, 7zf6 | 30
Table 3: Conditional design tasks. We sample 10 structures for each inpainting
subtask. The CDRL3 length change design task is limited to this Appendix.

In Figure 10, we show the scRMSD of each region across the different design
tasks. We compare the performance of IgDiff to the baseline of RFDiffusion. We
see that in all design tasks IgDiff has a lower mean and median scRMSD in the
region that is being inpainted. The most challenging task for IgDiff is to
design all of the CDRs. Interestingly, the region that IgDiff most struggles
to model in this conditional task appears to be CDRH1, although this region
was well modelled in unconditional generation. For all other tasks, we show
that almost all designed antibodies retain a scRMSD below 2Å across all
regions, indicating that the conditioned antibodies retain the designability
properties of the unconditioned model. Figure 11 shows the RMSD with the input
structure across the fixed regions of the design task. This is below 1Å for
all generated structures, demonstrating that the motifs are well preserved
during the inpainting procedure.

Figure 10: scRMSD of antibodies generated conditionally using IgDiff and
RFDiffusion with different design tasks. The red line indicates the 2Å cutoff
used to designate designable structures. Figure 11: RMSD of the fixed regions
during conditional design tasks using IgDiff and RFDiffusion. For each design
task, the RMSD is evaluated on the fixed region and between the generated
design and the reference antibody

In Table 4, we give the fraction of matching canonical clusters with
repredicted ABB2 structures for each design task. Here we note good agreement
across all CDR loops except CDR H1. Notably IgDiff outperforms RFDiffusion on
the fraction of matching clusters on all tasks.

|  | IgDiff | RFDiffusion
---|---|---|---
Task | Region |  Matching clusters |  Mismatching clusters |  Both unclassified |  Matching clusters |  Mismatching clusters |  Both unclassified
Design all CDRs | CDRH1 | 0.02 | 0.96 | 0.02 | 0.06 | 0.84 | 0.10
Design all CDRs | CDRH2 | 1.00 | 0.00 | 0.00 | 1.00 | 0.00 | 0.00
Design all CDRs | CDRL1 | 0.70 | 0.24 | 0.06 | 0.02 | 0.74 | 0.24
Design all CDRs | CDRL2 | 0.98 | 0.02 | 0.00 | 0.02 | 0.98 | 0.00
Design all CDRs | CDRL3 | 0.90 | 0.10 | 0.00 | 0.46 | 0.54 | 0.00
CDRL3 length change | CDRL3 | 0.70 | 0.28 | 0.03 | 0.68 | 0.33 | 0.00
Design light chain | CDRL1 | 0.80 | 0.17 | 0.03 | – | – | –
Design light chain | CDRL2 | 1.00 | 0.00 | 0.00 | – | – | –
Design light chain | CDRL3 | 0.87 | 0.07 | 0.07 | – | – | –

Table 4: Fraction of matching and mismatching canonical clusters between the
antibody structures generated conditionally using IgDiff and the ABB2
repredicted structures. Note that RFDiffusion did not produce antibody light
chains in the light chain design task.

## References

  * Abanades et al. (2022) Brennan Abanades, Wing Ki Wong, Fergus Boyles, Guy Georges, Alexander Bujotzek, and Charlotte M. Deane.  Immunebuilder: Deep-learning models for predicting the structures of immune proteins.  _bioRxiv_ , 2022.  doi: 10.1101/2022.11.04.514231.  URL https://www.biorxiv.org/content/early/2022/11/04/2022.11.04.514231.
  * Adolf-Bryfogle et al. (2015) Jared Adolf-Bryfogle, Qifang Xu, Benjamin North, Andreas Lehmann, and Roland L Dunbrack Jr.  Pyigclassify: a database of antibody cdr structural classifications.  _Nucleic acids research_ , 43(D1):D432–D438, 2015.
  * Adolf-Bryfogle et al. (2018) Jared Adolf-Bryfogle, Oleks Kalyuzhniy, Michael Kubitz, Brian D. Weitzner, Xiaozhen Hu, Yumiko Adachi, William R. Schief, and Roland L. Dunbrack, Jr.  Rosettaantibodydesign (rabd): A general framework for computational antibody design.  _PLOS Computational Biology_ , 14(4):1–38, 04 2018.  doi: 10.1371/journal.pcbi.1006112.  URL https://doi.org/10.1371/journal.pcbi.1006112.
  * Akbar et al. (2021) Rahmad Akbar, Philippe A. Robert, Milena Pavlović, Jeliazko R. Jeliazkov, Igor Snapkov, Andrei Slabodkin, Cédric R. Weber, Lonneke Scheffer, Enkelejda Miho, Ingrid Hobæk Haff, Dag Trygve Tryslew Haug, Fridtjof Lund-Johansen, Yana Safonova, Geir K. Sandve, and Victor Greiff.  A compact vocabulary of paratope-epitope interactions enables predictability of antibody-antigen binding.  _Cell Reports_ , 34(11):108856, 2021.  ISSN 2211-1247.  doi: https://doi.org/10.1016/j.celrep.2021.108856.  URL https://www.sciencedirect.com/science/article/pii/S2211124721001704.
  * Alford et al. (2017) Rebecca F. Alford, Andrew Leaver-Fay, Jeliazko R. Jeliazkov, Matthew J. O’Meara, Frank P. DiMaio, Hahnbeom Park, Maxim V. Shapovalov, P. Douglas Renfrew, Vikram K. Mulligan, Kalli Kappel, Jason W. Labonte, Michael S. Pacella, Richard Bonneau, Philip Bradley, Roland L. Dunbrack, Rhiju Das, David Baker, Brian Kuhlman, Tanja Kortemme, and Jeffrey J. Gray.  The rosetta all-atom energy function for macromolecular modeling and design.  _Journal of Chemical Theory and Computation_ , 13(6):3031–3048, June 2017.  ISSN 1549-9618.  doi: 10.1021/acs.jctc.7b00125.
  * Bennett et al. (2024) Nathaniel R. Bennett, Joseph L. Watson, Robert J. Ragotte, Andrew J. Borst, Déjenaé L. See, Connor Weidle, Riti Biswas, Ellen L. Shrock, Philip J. Y. Leung, Buwei Huang, Inna Goreshnik, Russell Ault, Kenneth D. Carr, Benedikt Singer, Cameron Criswell, Dionne Vafeados, Mariana Garcia Sanchez, Ho Min Kim, Susana Vázquez Torres, Sidney Chan, and David Baker.  Atomically accurate de novo design of single-domain antibodies.  _bioRxiv_ , 2024.  doi: 10.1101/2024.03.14.585103.  URL https://www.biorxiv.org/content/early/2024/03/18/2024.03.14.585103.
  * Bortoli et al. (2022) Valentin De Bortoli, Emile Mathieu, Michael Hutchinson, James Thornton, Yee Whye Teh, and Arnaud Doucet.  Riemannian score-based generative modelling, 2022.
  * Chiu et al. (2019) Mark L Chiu, Dennis R Goulet, Alexey Teplyakov, and Gary L Gilliland.  Antibody structure and function: the basis for engineering therapeutics.  _Antibodies_ , 8(4):55, 2019.
  * Chothia and Lesk (1987) Cyrus Chothia and Arthur M. Lesk.  Canonical structures for the hypervariable regions of immunoglobulins.  _Journal of Molecular Biology_ , 196(4):901–917, 1987.  ISSN 0022-2836.  doi: https://doi.org/10.1016/0022-2836(87)90412-8.  URL https://www.sciencedirect.com/science/article/pii/0022283687904128.
  * Chu et al. (2023) Alexander E. Chu, Lucy Cheng, Gina El Nesr, Minkai Xu, and Po-Ssu Huang.  An all-atom protein generative model.  _bioRxiv_ , 2023.  doi: 10.1101/2023.05.24.542194.  URL https://www.biorxiv.org/content/early/2023/05/25/2023.05.24.542194.
  * Dauparas et al. (2022) J. Dauparas, I. Anishchenko, N. Bennett, H. Bai, R. J. Ragotte, L. F. Milles, B. I. M. Wicky, A. Courbet, R. J. de Haas, N. Bethel, P. J. Y. Leung, T. F. Huddy, S. Pellock, D. Tischer, F. Chan, B. Koepnick, H. Nguyen, A. Kang, B. Sankaran, A. K. Bera, N. P. King, and D. Baker.  Robust deep learning based protein sequence design using proteinmpnn.  _bioRxiv_ , 2022.  doi: 10.1101/2022.06.03.494563.  URL https://www.biorxiv.org/content/early/2022/06/04/2022.06.03.494563.
  * DeKosky et al. (2015) Brandon J DeKosky, Takaaki Kojima, Alexa Rodin, Wissam Charab, Gregory C Ippolito, Andrew D Ellington, and George Georgiou.  In-depth determination and analysis of the human paired heavy- and light-chain antibody repertoire.  _Nature Medicine_ , 21(1):86–91, 2015.  doi: 10.1038/nm.3743.  URL https://doi.org/10.1038/nm.3743.
  * Dreyer et al. (2023) Frédéric A. Dreyer, Daniel Cutting, Constantin Schneider, Henry Kenlay, and Charlotte M. Deane.  Inverse folding for antibody sequence design using deep learning.  In _2023 ICML Workshop on Computational Biology_ , 2023.  URL https://arxiv.org/abs/2310.19513.
  * Dreyer and Cutting (2024) Frédéric A. Dreyer and Daniel Cutting.  De novo antibody design with se(3) diffusion, May 2024.  URL https://doi.org/10.5281/zenodo.11184374.
  * Dunbar and Deane (2015) James Dunbar and Charlotte M. Deane.  ANARCI: antigen receptor numbering and receptor classification.  _Bioinformatics_ , 32(2):298–300, 09 2015.  ISSN 1367-4803.  doi: 10.1093/bioinformatics/btv552.  URL https://doi.org/10.1093/bioinformatics/btv552.
  * Engh and Huber (2012) R. A. Engh and R. Huber.  _Structure quality and target parameters_ , chapter 18.3, pages 474–484.  John Wiley & Sons, Ltd, 2012.  ISBN 9780470685754.  doi: https://doi.org/10.1107/97809553602060000857.  URL https://onlinelibrary.wiley.com/doi/abs/10.1107/97809553602060000857.
  * Evans et al. (2022) Richard Evans, Michael O’Neill, Alexander Pritzel, Natasha Antropova, Andrew Senior, Tim Green, Augustin Žídek, Russ Bates, Sam Blackwell, Jason Yim, Olaf Ronneberger, Sebastian Bodenstein, Michal Zielinski, Alex Bridgland, Anna Potapenko, Andrew Cowie, Kathryn Tunyasuvunakool, Rishub Jain, Ellen Clancy, Pushmeet Kohli, John Jumper, and Demis Hassabis.  Protein complex prediction with alphafold-multimer.  _bioRxiv_ , 2022.  doi: 10.1101/2021.10.04.463034.  URL https://www.biorxiv.org/content/early/2022/03/10/2021.10.04.463034.
  * Fu et al. (2012) Limin Fu, Beifang Niu, Zhengwei Zhu, Sitao Wu, and Weizhong Li.  CD-HIT: accelerated for clustering the next-generation sequencing data.  _Bioinformatics_ , 28(23):3150–3152, 10 2012.  ISSN 1367-4803.  doi: 10.1093/bioinformatics/bts565.  URL https://doi.org/10.1093/bioinformatics/bts565.
  * Fuchs et al. (2020) Fabian B. Fuchs, Daniel E. Worrall, Volker Fischer, and Max Welling.  Se(3)-transformers: 3d roto-translation equivariant attention networks.  In _Advances in Neural Information Processing Systems 34 (NeurIPS)_ , 2020.
  * Gruver et al. (2023) Nate Gruver, Samuel Stanton, Nathan C. Frey, Tim G. J. Rudner, Isidro Hotzel, Julien Lafrance-Vanasse, Arvind Rajpal, Kyunghyun Cho, and Andrew Gordon Wilson.  Protein design with guided discrete diffusion, 2023.
  * Ho et al. (2020) Jonathan Ho, Ajay Jain, and Pieter Abbeel.  Denoising diffusion probabilistic models, 2020.
  * Ingraham et al. (2019) John Ingraham, Vikas Garg, Regina Barzilay, and Tommi Jaakkola.  Generative models for graph-based protein design.  In H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alché-Buc, E. Fox, and R. Garnett, editors, _Advances in Neural Information Processing Systems_ , volume 32. Curran Associates, Inc., 2019.  URL https://proceedings.neurips.cc/paper_files/paper/2019/file/f3a4ff4839c56a5f460c88cce3666a2b-Paper.pdf.
  * Jin et al. (2022a) Wengong Jin, Dr.Regina Barzilay, and Tommi Jaakkola.  Antibody-antigen docking and design via hierarchical structure refinement.  In Kamalika Chaudhuri, Stefanie Jegelka, Le Song, Csaba Szepesvari, Gang Niu, and Sivan Sabato, editors, _Proceedings of the 39th International Conference on Machine Learning_ , volume 162 of _Proceedings of Machine Learning Research_ , pages 10217–10227. PMLR, 17–23 Jul 2022a.  URL https://proceedings.mlr.press/v162/jin22a.html.
  * Jin et al. (2022b) Wengong Jin, Jeremy Wohlwend, Regina Barzilay, and Tommi S. Jaakkola.  Iterative refinement graph neural network for antibody sequence-structure co-design.  In _International Conference on Learning Representations_ , 2022b.  URL https://openreview.net/forum?id=LI2bhrE_2A.
  * Jørgensen (1975) Erik Jørgensen.  The central limit problem for geodesic random walks.  _Zeitschrift für Wahrscheinlichkeitstheorie und Verwandte Gebiete_ , 32(1):1–64, 1975.  doi: 10.1007/BF00533088.  URL https://doi.org/10.1007/BF00533088.
  * Jumper et al. (2021) John Jumper, Richard Evans, Alexander Pritzel, Tim Green, Michael Figurnov, Olaf Ronneberger, Kathryn Tunyasuvunakool, Russ Bates, Augustin Žídek, Anna Potapenko, Alex Bridgland, Clemens Meyer, Simon A. A. Kohl, Andrew J. Ballard, Andrew Cowie, Bernardino Romera-Paredes, Stanislav Nikolov, Rishub Jain, Jonas Adler, Trevor Back, Stig Petersen, David Reiman, Ellen Clancy, Michal Zielinski, Martin Steinegger, Michalina Pacholska, Tamas Berghammer, Sebastian Bodenstein, David Silver, Oriol Vinyals, Andrew W. Senior, Koray Kavukcuoglu, Pushmeet Kohli, and Demis Hassabis.  Highly accurate protein structure prediction with alphafold.  _Nature_ , 596(7873):583–589, 2021.  doi: 10.1038/s41586-021-03819-2.  URL https://doi.org/10.1038/s41586-021-03819-2.
  * Kingma and Ba (2017) Diederik P. Kingma and Jimmy Ba.  Adam: A method for stochastic optimization, 2017.
  * Kong et al. (2023a) Xiangzhe Kong, Wenbing Huang, and Yang Liu.  End-to-end full-atom antibody design, 2023a.
  * Kong et al. (2023b) Xiangzhe Kong, Wenbing Huang, and Yang Liu.  Conditional antibody design as 3d equivariant graph translation.  In _The Eleventh International Conference on Learning Representations_ , 2023b.  URL https://openreview.net/forum?id=LFHFQbjxIiP.
  * Kovaltsuk et al. (2018) Aleksandr Kovaltsuk, Jinwoo Leem, Sebastian Kelm, James Snowden, Charlotte M. Deane, and Konrad Krawczyk.  Observed Antibody Space: A Resource for Data Mining Next-Generation Sequencing of Antibody Repertoires.  _The Journal of Immunology_ , 201(8):2502–2509, 10 2018.  ISSN 0022-1767.  doi: 10.4049/jimmunol.1800708.  URL https://doi.org/10.4049/jimmunol.1800708.
  * Lefranc et al. (2003) Marie-Paule Lefranc, Christelle Pommié, Manuel Ruiz, Véronique Giudicelli, Elodie Foulquier, Lisa Truong, Valérie Thouvenin-Contet, and Gérard Lefranc.  Imgt unique numbering for immunoglobulin and t cell receptor variable domains and ig superfamily v-like domains.  _Developmental & Comparative Immunology_, 27(1):55–77, 2003.  ISSN 0145-305X.  doi: https://doi.org/10.1016/S0145-305X(02)00039-3.  URL https://www.sciencedirect.com/science/article/pii/S0145305X02000393.
  * Li et al. (2014) Tong Li, Robert J. Pantazes, and Costas D. Maranas.  Optmaven – a new framework for the de novo design of antibody variable region models targeting specific antigen epitopes.  _PLOS ONE_ , 9(8):1–17, 08 2014.  doi: 10.1371/journal.pone.0105954.  URL https://doi.org/10.1371/journal.pone.0105954.
  * Lin and AlQuraishi (2023) Yeqing Lin and Mohammed AlQuraishi.  Generating novel, designable, and diverse protein structures by equivariantly diffusing oriented residue clouds, 2023.
  * Lisanza et al. (2023) Sidney Lyayuga Lisanza, Jake Merle Gershon, Sam Tipps, Lucas Arnoldt, Samuel Hendel, Jeremiah Nelson Sims, Xinting Li, and David Baker.  Joint generation of protein sequence and structure with rosettafold sequence space diffusion.  _bioRxiv_ , 2023.  doi: 10.1101/2023.05.08.539766.  URL https://www.biorxiv.org/content/early/2023/05/10/2023.05.08.539766.
  * Liu et al. (2019) Ge Liu, Haoyang Zeng, Jonas Mueller, Brandon Carter, Ziheng Wang, Jonas Schilz, Geraldine Horny, Michael E Birnbaum, Stefan Ewert, and David K Gifford.  Antibody complementarity determining region design using high-capacity machine learning.  _Bioinformatics_ , 36(7):2126–2133, 11 2019.  ISSN 1367-4803.  doi: 10.1093/bioinformatics/btz895.  URL https://doi.org/10.1093/bioinformatics/btz895.
  * Luo et al. (2022) Shitong Luo, Yufeng Su, Xingang Peng, Sheng Wang, Jian Peng, and Jianzhu Ma.  Antigen-specific antibody design and optimization with diffusion-based generative models for protein structures.  In Alice H. Oh, Alekh Agarwal, Danielle Belgrave, and Kyunghyun Cho, editors, _Advances in Neural Information Processing Systems_ , 2022.  URL https://openreview.net/forum?id=jSorGn2Tjg.
  * Martinkus et al. (2023) Karolis Martinkus, Jan Ludwiczak, Kyunghyun Cho, Wei-Ching Liang, Julien Lafrance-Vanasse, Isidro Hotzel, Arvind Rajpal, Yan Wu, Richard Bonneau, Vladimir Gligorijevic, and Andreas Loukas.  Abdiffuser: Full-atom generation of in-vitro functioning antibodies, 2023.
  * Narciso et al. (2011) Jo Erika Narciso, Iris Uy, April Cabang, Jenina Chavez, Juan Pablo, Gisela Padilla-Concepcion, and Eduardo Padlan.  Analysis of the antibody structure based on high-resolution crystallographic studies.  _New biotechnology_ , 28:435–47, 04 2011.  doi: 10.1016/j.nbt.2011.03.012.
  * North et al. (2011) Benjamin North, Andreas Lehmann, and Roland L. Dunbrack.  A new clustering of antibody cdr loop conformations.  _Journal of Molecular Biology_ , 406(2):228–256, 2011.  ISSN 0022-2836.  doi: https://doi.org/10.1016/j.jmb.2010.10.030.  URL https://www.sciencedirect.com/science/article/pii/S0022283610011496.
  * Olsen et al. (2022) Tobias H. Olsen, Fergus Boyles, and Charlotte M. Deane.  Observed antibody space: A diverse database of cleaned, annotated, and translated unpaired and paired antibody sequences.  _Protein Science_ , 31(1):141–146, 2022.  doi: https://doi.org/10.1002/pro.4205.  URL https://onlinelibrary.wiley.com/doi/abs/10.1002/pro.4205.
  * Saka et al. (2021) Koichiro Saka, Taro Kakuzaki, Shoichi Metsugi, Daiki Kashiwagi, Kenji Yoshida, Manabu Wada, Hiroyuki Tsunoda, and Reiji Teramoto.  Antibody design using lstm based deep generative model from phage display library for affinity maturation.  _Scientific Reports_ , 11(1):5852, 2021.  doi: 10.1038/s41598-021-85274-7.  URL https://doi.org/10.1038/s41598-021-85274-7.
  * Shanehsazzadeh et al. (2023) Amir Shanehsazzadeh, Julian Alverio, George Kasun, Simon Levine, Jibran A. Khan, Chelsea Chung, Nicolas Diaz, Breanna K. Luton, Ysis Tarter, Cailen McCloskey, Katherine B. Bateman, Hayley Carter, Dalton Chapman, Rebecca Consbruck, Alec Jaeger, Christa Kohnert, Gaelin Kopec-Belliveau, John M. Sutton, Zheyuan Guo, Gustavo Canales, Kai Ejan, Emily Marsh, Alyssa Ruelos, Rylee Ripley, Brooke Stoddard, Rodante Caguiat, Kyra Chapman, Matthew Saunders, Jared Sharp, Douglas Ganini da Silva, Audree Feltner, Jake Ripley, Megan E. Bryant, Danni Castillo, Joshua Meier, Christian M. Stegmann, Katherine Moran, Christine Lemke, Shaheed Abdulhaqq, Lillian R. Klug, Sharrol Bachas, and Absci Corporation.  In vitro validated antibody design against multiple therapeutic antigens using generative inverse folding.  _bioRxiv_ , 2023.  doi: 10.1101/2023.12.08.570889.  URL https://www.biorxiv.org/content/early/2023/12/09/2023.12.08.570889.
  * Sohl-Dickstein et al. (2015) Jascha Sohl-Dickstein, Eric Weiss, Niru Maheswaranathan, and Surya Ganguli.  Deep unsupervised learning using nonequilibrium thermodynamics.  In Francis Bach and David Blei, editors, _Proceedings of the 32nd International Conference on Machine Learning_ , volume 37 of _Proceedings of Machine Learning Research_ , pages 2256–2265, Lille, France, 07–09 Jul 2015. PMLR.  URL https://proceedings.mlr.press/v37/sohl-dickstein15.html.
  * Song et al. (2021) Yang Song, Jascha Sohl-Dickstein, Diederik P. Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole.  Score-based generative modeling through stochastic differential equations, 2021.
  * Trippe et al. (2023) Brian L. Trippe, Jason Yim, Doug Tischer, David Baker, Tamara Broderick, Regina Barzilay, and Tommi S. Jaakkola.  Diffusion probabilistic modeling of protein backbones in 3d for the motif-scaffolding problem.  In _The Eleventh International Conference on Learning Representations_ , 2023.  URL https://openreview.net/forum?id=6TxBxqNME1Y.
  * Tsuchiya and Mizuguchi (2016) Yuko Tsuchiya and Kenji Mizuguchi.  The diversity of h3 loops determines the antigen-binding tendencies of antibody cdr loops.  _Protein science : a publication of the Protein Society_ , 25, 01 2016.  doi: 10.1002/pro.2874.
  * Vaswani et al. (2017) Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin.  Attention is all you need, 2017.
  * Watson et al. (2022) Joseph L. Watson, David Juergens, Nathaniel R. Bennett, Brian L. Trippe, Jason Yim, Helen E. Eisenach, Woody Ahern, Andrew J. Borst, Robert J. Ragotte, Lukas F. Milles, Basile I. M. Wicky, Nikita Hanikel, Samuel J. Pellock, Alexis Courbet, William Sheffler, Jue Wang, Preetham Venkatesh, Isaac Sappington, Susana Vázquez Torres, Anna Lauko, Valentin De Bortoli, Emile Mathieu, Regina Barzilay, Tommi S. Jaakkola, Frank DiMaio, Minkyung Baek, and David Baker.  Broadly applicable and accurate protein design by integrating structure prediction networks and diffusion generative models.  _bioRxiv_ , 2022.  doi: 10.1101/2022.12.09.519842.  URL https://www.biorxiv.org/content/early/2022/12/10/2022.12.09.519842.
  * Wu et al. (2021) Zachary Wu, Kadina E. Johnston, Frances H. Arnold, and Kevin K. Yang.  Protein sequence design with deep generative models.  _Current Opinion in Chemical Biology_ , 65:18–27, 2021.  ISSN 1367-5931.  doi: https://doi.org/10.1016/j.cbpa.2021.04.004.  URL https://www.sciencedirect.com/science/article/pii/S136759312100051X.  Mechanistic Biology * Machine Learning in Chemical Biology.
  * Yim et al. (2023) Jason Yim, Brian L Trippe, Valentin De Bortoli, Emile Mathieu, Arnaud Doucet, Regina Barzilay, and Tommi Jaakkola.  Se(3) diffusion model with application to protein backbone generation.  _arXiv preprint arXiv:2302.02277_ , 2023.
  * Zhan et al. (2023) Yangtian Zhan, Zuobai Zhang, Bozitao Zhong, Sanchit Misra, and Jian Tang.  Diffpack: A torsional diffusion model for autoregressive protein side-chain packing, 2023.
  * Zhang et al. (2023) Cheng Zhang, Adam Leach, Thomas Makkink, Miguel Arbesú, Ibtissem Kadri, Daniel Luo, Liron Mizrahi, Sabrine Krichen, Maren Lang, Andrey Tovchigrechko, Nicolas Lopez Carranza, Uğur Şahin, Karim Beguir, Michael Rooney, and Yunguan Fu.  Framedipt: Se(3) diffusion model for protein structure inpainting.  _bioRxiv_ , 2023.  doi: 10.1101/2023.11.21.568057.  URL https://www.biorxiv.org/content/early/2023/11/21/2023.11.21.568057.
  * Zhang and Skolnick (2004) Yang Zhang and Jeffrey Skolnick.  Scoring function for automated assessment of protein structure template quality.  _Proteins_ , 57(4):702—710, December 2004.  ISSN 0887-3585.  doi: 10.1002/prot.20264.  URL https://doi.org/10.1002/prot.20264.