# Generative AI for Controllable Protein Sequence Design: A Survey

Author Name Affiliation<EMAIL_ADDRESS>Yiheng Zhu1 Zitai Kong1 Jialu Wu1
Weize Liu1 Yuqiang Han2 Mingze Yin1 Hongxia Xu1 Chang-Yu Hsieh1&Tingjun Hou1
1Zhejiang University
2The Chinese University of Hong Kong
{zhuyiheng2020, kongzitai, jialuwu, weizeliu, 22218878, Einstein, kimhsieh,
<EMAIL_ADDRESS><EMAIL_ADDRESS>
###### Abstract

The design of novel protein sequences with targeted functionalities underpins
a central theme in protein engineering, impacting diverse fields such as drug
discovery and enzymatic engineering. However, navigating this vast
combinatorial search space remains a severe challenge due to time and
financial constraints. This scenario is rapidly evolving as the transformative
advancements in AI, particularly in the realm of generative models and
optimization algorithms, have been propelling the protein design field towards
an unprecedented revolution. In this survey, we systematically review recent
advances in generative AI for controllable protein sequence design. To set the
stage, we first outline the foundational tasks in protein sequence design in
terms of the constraints involved and present key generative models and
optimization algorithms. We then offer in-depth reviews of each design task
and discuss the pertinent applications. Finally, we identify the unresolved
challenges and highlight research opportunities that merit deeper exploration.

##  1 Introduction

Proteins are indispensable biomolecules that execute the myriad of biological
processes fundamental to life, such as catalyzing enzymatic reactions and
mediating immune responses. Motivated by the versatility exhibited by natural
proteins, designing novel amino acid sequences that encode proteins with
desired functions has been a central challenge in bioengineering.
Nevertheless, due to the vast exploration space of possible proteins,
conventional experimental methods, such as directed evolution Arnold (1998),
are prohibitively time-intensive and money-consuming. To overcome these
challenges, recent years have witnessed the trend of leveraging generative AI
technologies to explore the biochemical space intelligently. Fueled by
advances in generative AI, the field of protein design is experiencing an
unprecedented revolution.

Evolutionary-scale protein sequence databases, as opposed to limited and
static structural data that yield a biased subset of the protein design space,
lay the groundwork for sequence-based protein design. Deep generative models
offer a promising tool for capturing the distribution underlying natural
protein sequences to generate novel and diverse proteins. Moreover, making
protein design controllable is crucial for the transition from theory to
practice. Ideally, a practical method ought to generate proteins that adhere
to task-specific structural and functional constraints. To this end,
researchers have utilized conditional generative models and optimization
algorithms to realize such tailored protein design.

Figure 1: Illustration for controllable protein sequence design. (a) The upper
diagram delineates the four integral tiers of the protein central dogma. Amino
acid sequences fold to form specific protein structures, which determine
protein functions. These varied functions integrate to perform higher-level
actions. Then protein sequence design tasks can be bifurcated into structure-
to-sequence, function-to-sequence, and higher-level sequence design. The lower
diagram highlights that generative AI empowers designers to sidestep intricate
intermediate steps associated with traditional methods, facilitating protein
design in an end-to-end manner. (b) Related generative AI technologies
primarily incorporate (i) deep generative models and (ii) optimization methods
(see Section 2 for more details).

Protein sequence design stands as a rapidly growing field of scientific
discovery, necessitating a thorough review of the extensive literature to
sketch the contours of the field. Despite certain surveys detailing
advancements Wu et al. (2021) and highlighting controllability Ferruz and
Höcker (2022), they may fall short in igniting enthusiasm for controllable
protein sequence design among the machine learning community due to two main
reasons: 1) the majority are framed from a biochemical standpoint, potentially
inaccessible for researchers without deep domain expertise; 2) they often
focus on selected aspects of the literature in terms of methodologies and
tasks. For instance, compared to de novo design, the equally important theme
of mutation-based optimization tends to be overlooked. In contrast, our survey
endeavors to bridge this gap. We seek to demystify protein sequence design for
machine learning researchers by systematically categorizing various
controllable design tasks according to the constraints involved. We hope this
survey reaches both the machine learning and biochemistry fields and inspires
further collaboration towards developing more sophisticated and controllable
algorithms.

0-1=dj0-1= Protein Sequence Design
---
0-1= Distribution Learning (§3) 0-1= Local Space 0-1= MSA VAE Hawkins-Hooker
et al. (2021), ProteinGAN Repecka et al. (2021). 0-1= Global Space 0-1=
ProtGPT2 Ferruz et al. (2022), ProGen2 Nijkamp et al. (2023), xTrimoPGLM Chen
et al. (2024), RITA Hesslow et al. (2022), EvoDiff Alamdari et al. (2023).
0-1= Controllable Design (§4) 0-1= Structure 0-1= AR Generation 0-1=
StructTrans and StructGNN Ingraham et al. (2019), GVP-GNN Jing et al. (2021),
ESM-IF Hsu et al. (2022), ProteinMPNN Dauparas et al. (2022). 0-1= One-shot
Generation 0-1= PiFold Gao et al. (2023b), VFN-IF Mao et al. (2023). 0-1=
Iterative Refinement 0-1= LM-Design Zheng et al. (2023), KWDesign Gao et al.
(2023a), GRADE-IF Yi et al. (2023). 0-1= Function 0-1= De Novo Generation 0-1=
ProGen Madani et al. (2023), MSA Transformer Sgarbossa et al. (2023). 0-1= De
Novo Optimization 0-1= DyNA-PPO Angermueller et al. (2019), GFlowNet-AL Jain
et al. (2022), ProteinRL Sternke and Karpiak (2023), NOS Gruver et al. (2023).
0-1= Mutation-based Optimization 0-1= AdaLead Sinai et al. (2020), DES Swersky
et al. (2020), ReLSO Castro et al. (2022), PEX Ren et al. (2022), LaMBO
Stanton et al. (2022), EvoPlay Wang et al. (2023a). 0-1= Higher level 0-1=
Target Protein 0-1= LaMBO2 Gruver et al. (2023), TCR-dWAE Li et al. (2023).
0-1= Text 0-1= ProteinDT Liu et al. (2023), InstructProtein Wang et al.
(2023b). 0-1= Application (§5) 0-1= Peptide 0-1= SPGen Wu et al. (2020),
PepWAE Das et al. (2021), HydrAMP Szymczak et al. (2023). 0-1= Antibody 0-1=
LaMBO-2 Gruver et al. (2023), xTrimoPGLM-Ab Chen et al. (2024). 0-1= Enzyme
0-1= EvoPlay Wang et al. (2023a), ProteinGAN Repecka et al. (2021), ProGen
Madani et al. (2023).

Figure 2: A taxonomy of protein sequence design methods with representative
examples.

The remainder of this paper is organized as follows: We first outline the
foundational tasks involving controllable protein sequence design, along with
relevant generative models and optimization algorithms (Section 2). We then
encapsulate the prevalent paradigms for distribution learning on natural
sequence space (Section 3). Next, we provide an extensive review of each
controllable design task (Section 4). We also examine the pertinent
applications (Section 5). Finally, we conclude with the open challenges and
point to potential opportunities for further exploration (Section 6). A
detailed breakdown is depicted in Figure 2.

##  2 Preliminaries

###  2.1 Problem Formulation

Generally, a protein can be represented as a sequence of amino acids
$x=[x_{1},x_{2},\dots,x_{L}]$ of length $L$, where each amino acid $x_{i}$ is
drawn from a vocabulary $\mathcal{V}$ consisting of 20 standard amino acids.
Controllable protein sequence design aims to generate plausible proteins that
satisfy some desired constraints $c$, which may vary across different tasks.
Formally, we pose the controllable protein sequence design as the problem of
sampling $x$ from $p(x|c)$.

A fundamental tenet of molecular biology is that “sequence determines
structure, structure determines function” Branden and Tooze (2012).
Consequently, we divide the meaningful control conditions into three
categories, i.e., structural, functional, and higher-level constraints, as
illustrated in Figure 1.

###  2.2 Deep Generative Models

Herein, we concisely outline the core principles behind representative
generative frameworks.

Autoregressive models (ARMs) Bengio et al. (2000) learn the high-dimensional
joint distribution $p(x)$ by factorizing it into the product of univariate
conditionals as follows:

$p(x)=\prod_{i=1}^{L}p(x_{i}|x_{1},x_{2},\dots,x_{i-1})$ (1)

ARMs require a predefined order in which to sequentially generate $x$. While
the choice of ordering can be clear for some modalities (e.g., text and
audio), it is not obvious for others such as images and graphs.

Variational Autoencoders (VAEs) Kingma and Welling (2013) optimize the
evidence lower bound (ELBO) of the intractable likelihood $p(x)$. VAEs
introduce a latent variable $z$ to derive the ELBO:

$\displaystyle\begin{aligned} \log{p(x)}&=\log{\int p(z)p(x|z)\mathrm{d}z}\\\
&\geq\mathbb{E}_{q_{\phi}(z|x)}\big{[}\log{p(x|z)}\big{]}-D_{KL}(q_{\phi}(z|x)||p(z))\\\
&\triangleq\text{ELBO}\end{aligned}$

(2)

where $p(z)$ is the prior distribution, $q_{\phi}(z|x)$ is the amortized
variational posterior. The first term of the ELBO is referred to as the
(negative) reconstruction error, while the second term is the KL-divergence
which quantifies the discrepancy between the learned distribution
$q_{\phi}(z|x)$ and prior distribution $p(z)$.

Generative Adversarial Networks (GANs) Goodfellow et al. (2020) are a class of
likelihood-free models, aiming to generate vivid fake data indistinguishable
from real data. GANs are composed of a discriminator $D$ that tries to
distinguish the generated data from real data, and a generator $G$ that aims
to generate data convincing enough to deceive the discriminator. This
objective is achieved through adversarial training:

$\displaystyle\min_{G}\max_{D}\mathcal{L}(D,G)$ (3) $\displaystyle=$
$\displaystyle\,\mathbb{E}_{x\sim p_{data}}[\log D(x)]+\mathbb{E}_{z\sim
p(z)}[\log(1-D(G(z))]$

Diffusion models Sohl-Dickstein et al. (2015); Ho et al. (2020) are latent
variable models inspired by non-equilibrium thermodynamics. The essential idea
is to define a forward diffusion process that gradually corrupts the data into
pure noise and learn the reverse process to reconstruct the data from noise
distributions.

Most works in continuous spaces define the forward process as
$q(x_{t}|x_{t-1})={\mathcal{N}}(x_{t}|\sqrt{1-\beta_{t}}x_{t-1},\beta_{t}{\mathbb{I}})$
to corrupts the data $x_{0}\sim p(x)$ into a sequence of increasingly noisy
latent variables $x_{1:T}=x_{1},x_{2},\dots,x_{T}$. The reverse process can
then be computed as
$q(x_{t-1}|x_{t},x_{0})=\mathcal{N}(x_{t-1}|{\widetilde{\mu}}_{t}(x_{t},x_{0}),{\widetilde{\beta}}_{t}\mathbb{I})$
when conditioned on $x_{0}$ using Bayes theorem. Let $\alpha_{t}=1-\beta_{t}$
and $\hat{\alpha}_{t}=\prod_{s=1}^{t}{\alpha_{t}}$, the parameters
$\widetilde{\mu_{t}}(x_{t},x_{0})=\frac{\sqrt{{\hat{\alpha}}_{t-1}}\beta_{t}}{1-\hat{\alpha}_{t}}x_{0}+\frac{\sqrt{\alpha_{t}}(1-\hat{\alpha}_{t-1})}{1-\hat{\alpha}_{t}}x_{t}$
and
${\widetilde{\beta}}_{t}=\frac{1-\hat{\alpha}_{t-1}}{1-\hat{\alpha}_{t}}\beta_{t}$
are analytical. The objective of diffusion models is derived from the
variational lower bound of $p(x)$, delineated as follows:

$\displaystyle\begin{aligned}
&\mathcal{L}_{\text{VLB}}=\underbrace{KL[q(x_{T}|x_{0})||p_{\theta}(x_{T})]}_{\mathcal{L}_{T}}\\\
&~{}~{}~{}+\sum_{t=2}^{T}\underbrace{KL[q(x_{t-1}|x_{t},x_{0})||p_{\theta}(x_{t-1}|x_{t})]}_{\mathcal{L}_{t-1}}\underbrace{-\mathbb{E}_{q}[\log
p_{\theta}(x_{0}|x_{1})]}_{\mathcal{L}_{0}}\end{aligned}$

(4)

where $\mathcal{L}_{T}$ is a constant and $\mathcal{L}_{0}$ can be estimated
using a separate model. Diffusion models commonly adopt a neural network
${\epsilon}_{\theta}$ to predict the noise such that
$\\{\mathcal{L}_{t-1}\\}_{t=2}^{T}$ is reduced to
$\mathbb{E}_{x_{0},t}\Big{[}\|\epsilon_{t}-\epsilon_{\theta}(\sqrt{\bar{\alpha}_{t}}x_{0}+\sqrt{1-\bar{\alpha}_{t}}\epsilon_{t},t)\|^{2}\Big{]}$.

While diffusion models with Gaussian perturbation excel in generating
continuous signals, their direct application to discrete data generation is
not feasible. To bridge this gap, researchers have explored two main
strategies for adapting diffusion models to discrete domains: the first embeds
discrete symbols into a continuous space Li et al. (2022), and the second
defines a diffusion process that inherently operates within discrete space
Austin et al. (2021).

###  2.3 Optimization algorithms

We overview the widely-used optimization algorithms designed to optimize the
objective function or reward.

Latent space optimization (LSO) Gómez-Bombarelli et al. (2018) circumvents the
discrete search space by performing optimization over the low-dimensional
continuous latent space learned by generative models. Following the navigation
of the latent space, the optimized latent representations are then decoded
back into sequences.

Reinforcement learning (RL) Sutton and Barto (2018) seeks to learn an optimal
policy that maximizes the reward. The RL problem is typically formalized
within the Markov Decision Process (MDP) framework, which consists of states,
actions, transitions, and a reward function. In RL, agents investigate their
environment via a process of trial and error, learning to take actions that
lead to the highest rewards. Recently, the RL-like algorithm GFlowNet Bengio
et al. (2021) has emerged, offering a method to learn a stochastic policy that
sequentially constructs objects with probability proportional to reward,
carving a niche for itself in scientific discovery Jain et al. (2022); Zhu et
al. (2023).

Evolution algorithms are popular heuristic algorithms inspired by natural
evolutionary processes. Beginning with a population of individuals, where each
represents a potential solution within the search space of a given problem,
evolution algorithms mimic the stochastic processes of mutation and/or
recombination to facilitate exploration within the search space. Individuals
with higher fitness values are preferentially selected in this adaptive
environment.

Bayesian optimization (BO) Shahriari et al. (2015) provides a sample-efficient
active learning framework for the global optimization of expensive black-box
functions. During each iteration, the principal strategy is to build a
probabilistic surrogate model that approximates the actual black-box functions
on the observed data and then optimize an acquisition function, built upon the
surrogate model, to identify informative candidates which offer the greatest
utility for the next cycle of evaluations.

##  3 Distribution Learning of Natural Sequences

Comprehending the underlying data distributions of natural protein sequences
is crucial for exploring the protein landscape. Deep generative models serve
as powerful tools for learning these distributions and generating novel
sequences that extend beyond the currently explored regions of the protein
sequence space. We delineate the models specifically developed to characterize
either the local or global protein space.

###  3.1 Local Sequence Space

Early works train generative models on multiple sequence alignment (MSA) of a
specific family of interest to capture statistical patterns of amino-acid
relationships and generate novel proteins belonging to that family. The Potts
model is a well-defined statistical energy model for capturing the first- and
second-order evolutionary dependencies. Representatively, the Potts model
fitted with direct coupling analysis is used to design chorismate mutase
enzymes Russ et al. (2020). Beyond conventional statistical approaches,
various deep generative models have also been reasonably applied to capture
higher-order dependencies. Successful examples include MSA VAE Hawkins-Hooker
et al. (2021), and ProteinGAN Repecka et al. (2021).

###  3.2 Global Sequence Space

##### Protein language models.

The advent of autoregressive large language models (LLMs) such as GPT-3 Brown
et al. (2020), which can generate text with human-like capabilities, has
opened new avenues in protein sequence design due to the structural parallels
between natural language and protein sequences—both are sequential systems of
fundamental units. Building on the flexible and powerful Transformer
architecture Vaswani et al. (2017), renowned for its proficiency in capturing
long-range dependencies within sequences, protein language models (PLMs) pre-
trained on a large-scale corpus of protein sequences succeed in capturing
intrinsic sequence features and generating novel protein sequences that
exhibit properties analogous to those of natural proteins. ProtGPT2 Ferruz et
al. (2022) is an autoregressive PLM with 738 million parameters, showcasing
the GPT-2 architecture’s capabilities in generating novel protein sequences de
novo. It utilizes the BPE algorithm for tokenizing sequences rather than
directly employing amino acids as tokens. Having been pre-trained on nearly 50
million protein sequences, ProtGPT2 is capable not only of generating
sequences that conform to the patterns of natural proteins but also of
sampling unexplored regions of protein space. ProGen2 Nijkamp et al. (2023)
models are based on the GPT-J architecture and have been scaled up to 6.4
billion parameters, exhibiting enhanced proficiency in capturing the
distribution of observed evolutionary sequences and making zero-shot fitness
predictions. xTrimoPGLM Chen et al. (2024) adopts the backbone of the general
language model and utilizes in-place token prediction alongside autoregressive
blank infilling as training objectives. This innovative pre-training framework
enables xTrimoPGLM to perform both de novo design and targeted sequence
infilling. Scaling laws for LLMs Kaplan et al. (2020) delineate the
correlation between model performance and size, thereby providing the
theoretical basis for the strategic enlargement of these models. Hesslow et
al. Hesslow et al. (2022) present RITA, a suite of generative PLMs with up to
1.2 billion parameters, and conduct the first systematic analysis of how
capabilities evolve relative to model size within the protein domain,
revealing benefits associated with scaling up. More recently, xTrimoPGLM-100B
significantly outperforms other advanced PLMs across both protein
understanding and generation tasks, further emphasizing scaling effects.

##### Diffusion models.

The emergence of diffusion models as dominant frameworks in image synthesis
has attracted considerable interest in their application to protein design,
offering distinct advantages over left-to-right ARMs, including flexible
iterative refinement enabled by order-agnostic decoding and the trade-off
between sample quality and inference efficiency. EvoDiff Alamdari et al.
(2023) is the first foundation diffusion model for protein design trained on
evolutionary-scale protein sequence data. EvoDiff presents two variants,
EvoDiff-Seq and EvoDiff-MSA, which are grounded in two famous discrete
diffusion frameworks: D3PM Austin et al. (2021) and OADM Hoogeboom et al.
(2022). Extensive experiments show that EvoDiff can generate diverse and
structurally plausible proteins encompassing the breadth of natural sequence
and functional spaces. Owing to the universal applicability of sequence-based
design paradigm, EvoDiff can produce proteins inaccessible to structure-based
models, especially those with disordered regions, while preserving the
capacity to engineer scaffolds for functional structural motifs.

###  3.3 Databases

UniProt Consortium (2019) serves as an extensive protein repository,
documenting hundreds of millions of proteins spanning all branches of life. It
offers three unique databases: UniProtKB, UniRef, and UniParc. Among these,
the UniRef databases, which cluster sequence sets at various levels of
sequence identity, are broadly utilized in pre-training PLMs. Besides, the OAS
database Olsen et al. (2022) is specifically designed to curate and annotate
immune repertoires suitable for extensive analytical research. It furnishes
comprehensive nucleotide and amino acid profiles for each antibody, enriched
with additional sequence annotations. OAS features a vast collection of more
than 2.4 billion unpaired and over 1.5 million paired antibody sequences.

##  4 Controllable Protein Sequence Design

###  4.1 Structure-to-sequence Design

Structure-to-sequence design, which is often referred to as the protein
inverse folding problem, aims to automatically identify the protein sequences
that fold into the given backbone structures. Formally, the inverse folding
problem is represented as learning the conditional distribution $p(x|r)$,
where $r=[r_{1},r_{2},\dots,r_{n}]\in\mathbb{R}^{n\times 4\times 3}$ denotes
the 3D coordinates of backbone atoms in each residue, namely N,
$\text{C}_{\alpha}$, C, and O.

Typically, Protein inverse folding models consist of an encoder that
represents the protein backbone and a decoder tasked with generating the
protein sequence. The backbone structure is commonly represented as a
proximity graph over amino acid nodes. According to the generative manner,
modern inverse folding models are categorized into autoregressive, one-shot,
and iterative models.

##### Autoregressive generation.

Ingraham et al. Ingraham et al. (2019) firstly framed inverse folding as a
structure-to-sequence learning problem, proposing to predict protein sequences
from N to C terminus in an autoregressive manner using backbone features
extracted by the structure encoder. They define a local coordinate system at
each amino acid node to derive relative distance, direction, and orientation
for edge features, and compute the backbone dihedral angles to constitute node
features. Further, they proposed two variants, StructTrans and StructGNN, to
serve as the structure encoders. Following Ingraham et al. Ingraham et al.
(2019), a succession of studies has adopted the autoregressive framework,
achieving further enhancement in performance. For instance, GVP-GNN Jing et
al. (2021) replaces the standard multi-layer perceptrons in GNNs with
geometric vector perceptrons, which can effectively operate on both scalar and
geometric features. ESM-IF Hsu et al. (2022) augments training data by
incorporating 12M structures predicted by AlphaFold2, and examines three model
architectures, namely GVP-GNN, GVP-GNN-large, and GVP-Transformer. Building on
the foundation of StructGNN, ProteinMPNN Dauparas et al. (2022) verifies that
incorporating distances between N, $\text{C}_{\alpha}$, C, O, and a virtual
$\text{C}_{\beta}$ as extra input features and employing edge updates in the
encoder can lead to a marked enhancement of performance. To facilitate a wide
array of applications, ProteinMPNN adopts an order-agnostic ARM where a random
decoding order is sampled.

##### One-shot generation.

Given that the autoregressive models tend to have low inference speed, some
researchers have investigated one-shot methods that facilitate parallel
generation of multiple tokens. PiFold Gao et al. (2023b) replaces the
autoregressive decoder with stacked PiGNNs and improves the quality and
efficiency simultaneously, providing the first evidence that the expressive
encoder could play a more crucial role than the autoregressive decoder in
inverse folding. To be precise, PiFold integrates a broader range of features,
along with learnable virtual atoms designed to capture information not
accounted for by real atoms, and employs PiGNN layers to effectively learn
from multi-scale residue interactions by accounting for feature dependencies
at the node, edge, and global levels. Advancing in this direction, VFN-IF Mao
et al. (2023) introduces a more expressive encoder called Vector Field
Networks, which extract hierarchical residual representations at both the real
atom and frame-anchored virtual atom levels, thereby surpassing PiFold.

##### Iterative refinement.

Another promising approach involves generating an initial sequence then
iteratively refining the generated sequence. Motivated by the insight that
PLMs can learn evolutionary knowledge from the universe of natural protein
sequences, LM-Design Zheng et al. (2023) reprograms sequence-based PLMs, such
as ESM-1b Rives et al. (2021), for inverse folding. By integrating a
lightweight structural adapter, LM-Design equips PLMs with structural
comprehension. LM-Design undertakes training through conditional masked
language modeling, which allows it to reconstruct a native sequence from a
corrupted version, facilitating the iterative refinement of the predicted
sequence. KWDesign Gao et al. (2023a) also capitalizes on the common protein
knowledge extracted from PLMs to refine low-quality residues. Specifically,
KWDesign introduces a confidence-aware knowledge-tuning module that fuses
multimodal pre-trained knowledge to generate more rational sequences. GRADE-IF
Yi et al. (2023) controls the intermediate refinement process by applying the
principles of discrete diffusion diffusion probabilistic models (D3PM) Austin
et al. (2021). To constrain the generative space to a reduced subspace
reflecting evolutionary pressures, the Blocks Substitution Matrix (BLOSUM) is
employed as the transition matrix to guide the forward diffusion process. A
primary benefit of GRADE-IF is its capacity to encompass diverse plausible
solutions.

####  4.1.1 Datasets and Benchmarks

Conventionally, inverse folding models are benchmarked using the CATH4.2 and
CATH 4.3 datasets Orengo et al. (1997), with perplexity and recovery rate
serving as the key evaluation metrics. Additionally, the TS50 and TS500
datasets Li et al. (2014) are commonly used as independent test sets for
evaluating the generalization ability. Advanced challenges encompass the
design of multi-chain complexes, antibodies, and de novo proteins Zheng et al.
(2023). Recently, Gao et al. Gao et al. (2023c) established ProteinInvBench,
the first comprehensive benchmark for inverse folding, aiming at extending
state-of-the-art models to new tasks and evaluating them based on a variety of
metrics such as confidence, diversity, and sc-TM.

###  4.2 Function-to-sequence Design

The goal of function-to-sequence design is to generate novel, diverse, and
biologically plausible proteins with desired functions. Based on the manner in
which the objective is achieved, we roughly divided existing methods into
three distinct categories: De novo generation, De novo optimization, and
mutation-based optimization. Note that most optimization methods can be
adopted as the acquisition function optimizer in BO, enabling online model-
based optimization. For the sake of conciseness, we will focus solely on the
optimization mechanisms of these methods without delving into the specifics of
their application in BO.

####  4.2.1 De novo generation

The most direct approach is to train generative models on known active
proteins. The family-specific generative models discussed in Section 3.1
exemplify this methodology. Despite that they are capable of generating
functional proteins, their generalizability to sequence space outside the
scope of the training family is limited. Consequently, later works tend to
train a unified conditional model across diverse protein families. Inspired by
CTRL Keskar et al. (2019), ProGen Madani et al. (2023) designs a decoder-only
transformer for autoregressive generation and incorporates control tags (e.g.,
protein family, biological process, and molecular function), which are
metadata prepended to the protein during training, to enable controllable
protein design. MSA Transformer Rao et al. (2021) pre-trained with masked
language modeling, has been leveraged to generate synthetic MSAs by performing
an iterative mask-then-fill procedure on the input natural MSAs Sgarbossa et
al. (2023).

####  4.2.2 De novo optimization

Another promising direction is to guide an unconditional generative model with
a discriminative surrogate model to produce protein sequences that exhibit
elevated levels of functionality, referred to as fitness. Given the intensive
demands of laboratory experiments, surrogate sequence-function models are
commonly built to act as substitutes for wet-lab validation.

RL provides a flexible framework that guides modern deep generative models to
navigate the protein fitness landscape. DyNA-PPO Angermueller et al. (2019) is
a pioneer in employing model-based policy optimization and diversity-promoting
reward functions for biological sequence design. DyNA-PPO formulates the
design problem as an autoregressive MDP, where each action appends an acid
amino token to a partially constructed sequence. Likewise, GFlownet-AL Jain et
al. (2022) strives to learn a stochastic policy that can sequentially generate
sequences proportionally to a reward function, proposing a batch of diverse
candidates. Fine-tuning the pre-trained PLMs with RL has the potential to
enhance performance. Inspired by the achievements of ChatGPT, ProteinRL
Sternke and Karpiak (2023) has emerged as the first work to design full-length
proteins for specific properties by fine-tuning a generative PLM, namely
ProGen2.

Diffusion models can be guided to generate continuous samples with the desired
conditions, circumventing the necessity for fine-tuning by employing
classifier guidance Dhariwal and Nichol (2021). Recent advancements have
broadened this strategy to encompass general discrete diffusion models,
facilitating guided protein design. NOS Gruver et al. (2023) skews the
categorical sampling distribution of the diffusion models by exerting gradient
guidance upon the continuous representations of discrete sequences, which are
expressed in the form of hidden states.

####  4.2.3 Mutation-based Optimization

Contrary to de novo optimization, mutation-based optimization focuses on
improving a fitness function by modifying an existing wild type, defined as
the sequence occurring in nature. In general, the wild type demonstrates
decent fitness under the natural selection evolutionary process towards the
target function. Traditionally, biologists have approached this problem using
a model-free paradigm known as directed evolution, which involves iterative
cycles of obtaining protein variants by random mutagenesis, followed by
screening and selecting high-fitness variants. However, this approach is
limited by its reliance on high-throughput functional assays and a greedy
hill-climbing search algorithm, which can lead to substantial costs and
convergence to local optima. Recent efforts have concentrated on proposing
more efficient optimization methods to improve search efficiency.

In classical evolutionary algorithms, only top-fitness sequences are selected
and mutated within the evolution cycle, leading to limited diversity. To
alleviate this issue, AdaLead Sinai et al. (2020) incorporates recombination
of mutations to promote diversity and avoid local optima, establishing itself
as both an accessible and robust baseline method. To leverage the natural
property of the fitness landscape that a concise set of mutations upon the
wild type are usually sufficient to enhance the desired function, PEX Ren et
al. (2022) prioritizes the evolutionary search for high-fitness mutants with
low mutation counts.

Besides random mutations, an alternative approach is to leverage RL to
intelligently guide the search. DES Swersky et al. (2020) employs a
convolutional neural network as the policy network to propose specific edits
on a target sequence, and these edits are defined as a combination of intended
mutation site along with the choice of replacing amino acids. The policy
network is subsequently trained using a policy gradient-based RL algorithm.
EvoPlay Wang et al. (2023a) also mutates a single-site residue as an action to
optimize protein sequences. Likening the protein optimization process to
playing pieces on a chessboard, EvoPlay implements an iterative look-ahead
Monte Carlo tree search procedure to generate samples for training a policy-
value neural network, which in turn is then used to guide the search.

By employing gradient ascent, LSO methods can progress from wild type towards
regions of higher fitness. For instance, ReLSO Castro et al. (2022) trains a
transformer-based autoencoder with a fitness prediction network jointly,
producing a latent space organized by sequence-function relationships. It also
introduces norm-based negative sampling and an interpolative sampling penalty
to regularize the latent space, further enhancing the optimization process.
LaMBO Stanton et al. (2022) maps discrete sequences to continuous
representations using a denoising autoencoder. During the optimization phase,
LaMBO follows the gradient in the latent space to reconstruct sequences that
have been partially masked.

###  4.3 High-level constraints

Beyond explicit structural and functional constraints, there exists tremendous
knowledge in other formats describing proteins’ higher-level properties.

##### Target protein.

Target-specific protein design paves the way for the design of functional
proteins that possess distinct specificity. A prime example is the engineering
of antibodies tailored to bind selectively to peptide antigens. LaMBO2 Gruver
et al. (2023) integrates BO and NOS to optimize the hu4D5 antibody for
improved expression yield and binding affinity to the HER2 antigen. TCR-dWAE
Li et al. (2023) is a disentangled Wasserstein autoencoder, can isolate the
function-related patterns from the rest and is utilized for engineering T-cell
receptors (TCRs) to bind specifically to peptide antigens.

##### Text.

Text-guided protein design allows practitioners to describe their demands in
natural language without formal specifications. ProteinDT Liu et al. (2023) is
a pioneering method that leverages textual descriptions for protein design.
ProteinDT consists of three modules: ProteinCLAP that aligns the
representation of two modalities, a facilitator that generates the protein
representation from the text modality, and a decoder that generates the
protein sequences from the representation. InstructProtein Wang et al. (2023b)
equips LLMs with the ability to comprehend protein language and further
improve the performance of LLMs on protein design tasks through knowledge
instruction tuning.

##  5 Applications

###  5.1 Peptide

Peptides, characterized as short chains of amino acids with lengths varying
from 2 to 50 units, exhibit a wide range of biological activities. Rather than
simply screening an existing library, peptides with desired properties can be
designed through generative models. Landmark studies include the design of
signal peptides Wu et al. (2020) and antimicrobial peptides Das et al. (2021);
Szymczak et al. (2023).

###  5.2 Antibody

Antibodies are large, Y-shaped immunoproteins with both highly conserved parts
and highly variable regions, formed by a heavy and a light chain, each
comprising one variable domain and several constant domains. The variable
domain is partitioned into a framework region and three complementarity
determining regions (CDRs). The diversity inherent in CDRs empowers antibodies
to confer immunity by specifically targeting and binding to a wide spectrum of
pathogens. Thus, the majority of antibody design methods focus on redesigning
or infilling CDRs while keeping the remainder unchanged Gruver et al. (2023);
Chen et al. (2024).

###  5.3 Enzyme

Enzymes, which are biological catalysts made primarily of proteins, may
greatly accelerate the rate of chemical reactions. These substances are
essential for life, performing numerous critical roles within the body, such
as aiding in digestion and metabolism. For enzyme design, one may either
locally modify an existing enzyme to enhance its activity Wang et al. (2023a)
or synthesize diverse enzymes from scratch, thereby expanding the functional
enzyme sequence space Repecka et al. (2021); Madani et al. (2023).

##  6 Challenges and Opportunities

In this section, we discuss the open challenges that we face in protein
sequence design in terms of data, algorithms, and evaluation, and identify
potential opportunities.

###  6.1 Challenges

  * •

Lack of assay-labeled data: The acquisition of assay-labeled data poses
considerable challenges and incurs high costs, stemming from the complexity of
in vitro experimental procedures. For regimes where such labeled data are
scarce, supervised predictive models are not accurate enough to guide the
optimization process.

  * •

Lack of interpretability: Owing to the ‘black box’ nature of deep learning
models, which serve as the foundation for current protein sequence design
approaches, there is a notable interpretability challenge. Consequently,
practitioners struggle to elucidate the critical functional sites involved in
the protein generation or optimization processes, nor can they provide
effective remedies for failures encountered during in silico experiments.

  * •

Lack of reliable evaluation protocols: Currently, the evaluation of protein
sequence design models often relies on distribution-based computational
metrics that may lack biological relevance and robustness. The development of
a suite of task-specific, biologically meaningful, and robust evaluation
metrics to identify promising protein candidates persists as an unresolved
challenge.

###  6.2 Opportunities

  * •

Build comprehensive benchmarks: Scientific progress can be substantially
accelerated if comprehensive and universally recognized benchmarks are
established. We should therefore also strive for more reliable and comparable
in silico benchmarks. Within the realm of protein sequence design, researchers
have recently established the first benchmark for protein inverse folding,
ProteinInvBench Gao et al. (2023c). However, there remains an urgent need to
create more realistic benchmarks to ensure equitable comparisons among various
models tailored to the expansive spectrum of protein design challenges.

  * •

Leverage PLMs: RL is a prevalent methodology for the task-directed tuning of
LLMs, with ChatGPT exemplifying its success. The integration of RL with PLMs
for controllable design is nascent, yet holds immense promise. As PLMs gain
prominence, there emerges a compelling opportunity to leverage RL in steering
these pre-trained models to discover sequences that manifest predefined
characteristics, while preserving naturalness.

  * •

Explore more generative frameworks: Considering the evolutionary origins of
proteins, it is insightful to explore the effectiveness of diverse non-
autoregressive generative modeling approaches beyond diffusion models.
Innovative frameworks including GFlowNet and Schrödinger Bridge present viable
alternatives that merit further investigation within this scientific field.

##  7 Conclusion

In this paper, we systematically review the landscape of generative AI
technologies and their recent progress in controllable protein sequence
design. We present typical tasks, principal methodologies, and promising
future directions in the field. We anticipate that this survey will provide a
clear picture of this emerging field, encourage cross-disciplinary
partnerships, and set a roadmap for both researchers and practitioners to move
forward.

## References

  * Alamdari et al. [2023] Sarah Alamdari, Nitya Thakkar, et al.  Protein generation with evolutionary diffusion: sequence is all you need.  In NeurIPS 2023 Generative AI and Biology (GenBio) Workshop, 2023.
  * Angermueller et al. [2019] Christof Angermueller, David Dohan, et al.  Model-based reinforcement learning for biological sequence design.  In ICLR, 2019.
  * Arnold [1998] Frances H Arnold.  Design by directed evolution.  Accounts of chemical research, 31(3):125–131, 1998.
  * Austin et al. [2021] Jacob Austin, Daniel D Johnson, et al.  Structured denoising diffusion models in discrete state-spaces.  NeurIPS, 34:17981–17993, 2021.
  * Bengio et al. [2000] Yoshua Bengio, Réjean Ducharme, et al.  A neural probabilistic language model.  NeurIPS, 13, 2000.
  * Bengio et al. [2021] Emmanuel Bengio, Moksh Jain, et al.  Flow network based generative models for non-iterative diverse candidate generation.  NeurIPS, 34:27381–27394, 2021.
  * Branden and Tooze [2012] Carl Ivar Branden and John Tooze.  Introduction to protein structure.  Garland Science, 2012.
  * Brown et al. [2020] Tom Brown, Benjamin Mann, et al.  Language models are few-shot learners.  NeurIPS, 33:1877–1901, 2020.
  * Castro et al. [2022] Egbert Castro, Abhinav Godavarthi, et al.  Transformer-based protein generation with regularized latent space optimization.  Nature Machine Intelligence, 4(10):840–851, 2022.
  * Chen et al. [2024] Bo Chen, Xingyi Cheng, et al.  xtrimopglm: unified 100b-scale pre-trained transformer for deciphering the language of protein.  arXiv preprint arXiv:2401.06199, 2024.
  * Consortium [2019] UniProt Consortium.  Uniprot: a worldwide hub of protein knowledge.  Nucleic acids research, 47(D1):D506–D515, 2019.
  * Das et al. [2021] Payel Das, Tom Sercu, et al.  Accelerated antimicrobial discovery via deep generative models and molecular dynamics simulations.  Nature Biomedical Engineering, 5(6):613–623, 2021.
  * Dauparas et al. [2022] Justas Dauparas, Ivan Anishchenko, et al.  Robust deep learning–based protein sequence design using proteinmpnn.  Science, 378(6615):49–56, 2022.
  * Dhariwal and Nichol [2021] Prafulla Dhariwal and Alexander Nichol.  Diffusion models beat gans on image synthesis.  NeurIPS, 34:8780–8794, 2021.
  * Ferruz and Höcker [2022] Noelia Ferruz and Birte Höcker.  Controllable protein design with language models.  Nature Machine Intelligence, 4(6):521–532, 2022.
  * Ferruz et al. [2022] Noelia Ferruz, Steffen Schmidt, et al.  Protgpt2 is a deep unsupervised language model for protein design.  Nature communications, 13(1):4348, 2022.
  * Gao et al. [2023a] Zhangyang Gao, Cheng Tan, et al.  Knowledge-design: Pushing the limit of protein deign via knowledge refinement.  arXiv preprint arXiv:2305.15151, 2023.
  * Gao et al. [2023b] Zhangyang Gao, Cheng Tan, et al.  Pifold: Toward effective and efficient protein inverse folding.  In ICLR, 2023.
  * Gao et al. [2023c] Zhangyang Gao, Cheng Tan, et al.  Proteininvbench: Benchmarking protein inverse folding on diverse tasks, models, and metrics.  In NeurIPS Datasets and Benchmarks Track, 2023.
  * Gómez-Bombarelli et al. [2018] Rafael Gómez-Bombarelli, Jennifer N Wei, et al.  Automatic chemical design using a data-driven continuous representation of molecules.  ACS central science, 4(2):268–276, 2018.
  * Goodfellow et al. [2020] Ian Goodfellow, Jean Pouget-Abadie, et al.  Generative adversarial networks.  Communications of the ACM, 63(11):139–144, 2020.
  * Gruver et al. [2023] Nate Gruver, Samuel Stanton, et al.  Protein design with guided discrete diffusion.  arXiv preprint arXiv:2305.20009, 2023.
  * Hawkins-Hooker et al. [2021] Alex Hawkins-Hooker, Florence Depardieu, et al.  Generating functional protein variants with variational autoencoders.  PLoS computational biology, 17(2):e1008736, 2021.
  * Hesslow et al. [2022] Daniel Hesslow, Niccoló Zanichelli, et al.  Rita: a study on scaling up generative protein sequence models.  arXiv preprint arXiv:2205.05789, 2022.
  * Ho et al. [2020] Jonathan Ho, Ajay Jain, et al.  Denoising diffusion probabilistic models.  NeurIPS, 33:6840–6851, 2020.
  * Hoogeboom et al. [2022] Emiel Hoogeboom, Alexey A. Gritsenko, et al.  Autoregressive diffusion models.  In ICLR, 2022.
  * Hsu et al. [2022] Chloe Hsu, Robert Verkuil, et al.  Learning inverse folding from millions of predicted structures.  In ICML, pages 8946–8970. PMLR, 2022.
  * Ingraham et al. [2019] John Ingraham, Vikas Garg, et al.  Generative models for graph-based protein design.  NeurIPS, 32, 2019.
  * Jain et al. [2022] Moksh Jain, Emmanuel Bengio, et al.  Biological sequence design with gflownets.  In ICML, pages 9786–9801. PMLR, 2022.
  * Jing et al. [2021] Bowen Jing, Stephan Eismann, et al.  Learning from protein structure with geometric vector perceptrons.  In ICLR, 2021.
  * Kaplan et al. [2020] Jared Kaplan, Sam McCandlish, et al.  Scaling laws for neural language models.  arXiv preprint arXiv:2001.08361, 2020.
  * Keskar et al. [2019] Nitish Shirish Keskar, Bryan McCann, et al.  Ctrl: A conditional transformer language model for controllable generation.  arXiv preprint arXiv:1909.05858, 2019.
  * Kingma and Welling [2013] Diederik P Kingma and Max Welling.  Auto-encoding variational bayes.  arXiv preprint arXiv:1312.6114, 2013.
  * Li et al. [2014] Zhixiu Li, Yuedong Yang, et al.  Direct prediction of profiles of sequences compatible with a protein structure by neural networks with fragment-based local and energy-based nonlocal profiles.  Proteins: Structure, Function, and Bioinformatics, 82(10):2565–2573, 2014.
  * Li et al. [2022] Xiang Li, John Thickstun, et al.  Diffusion-lm improves controllable text generation.  NeurIPS, 35:4328–4343, 2022.
  * Li et al. [2023] Tianxiao Li, Hongyu Guo, et al.  Disentangled wasserstein autoencoder for t-cell receptor engineering.  In NeurIPS, 2023.
  * Liu et al. [2023] Shengchao Liu, Yutao Zhu, et al.  A text-guided protein design framework.  arXiv preprint arXiv:2302.04611, 2023.
  * Madani et al. [2023] Ali Madani, Ben Krause, et al.  Large language models generate functional protein sequences across diverse families.  Nature Biotechnology, pages 1–8, 2023.
  * Mao et al. [2023] Weian Mao, Muzhi Zhu, et al.  De novo protein design using geometric vector field networks.  arXiv preprint arXiv:2310.11802, 2023.
  * Nijkamp et al. [2023] Erik Nijkamp, Jeffrey A Ruffolo, et al.  Progen2: exploring the boundaries of protein language models.  Cell Systems, 14(11):968–978, 2023.
  * Olsen et al. [2022] Tobias H Olsen, Fergus Boyles, et al.  Observed antibody space: A diverse database of cleaned, annotated, and translated unpaired and paired antibody sequences.  Protein Science, 31(1):141–146, 2022.
  * Orengo et al. [1997] Christine A Orengo, Alex D Michie, et al.  Cath–a hierarchic classification of protein domain structures.  Structure, 5(8):1093–1109, 1997.
  * Rao et al. [2021] Roshan M Rao, Jason Liu, et al.  Msa transformer.  In ICML, pages 8844–8856. PMLR, 2021.
  * Ren et al. [2022] Zhizhou Ren, Jiahan Li, et al.  Proximal exploration for model-guided protein sequence design.  In ICML, pages 18520–18536. PMLR, 2022.
  * Repecka et al. [2021] Donatas Repecka, Vykintas Jauniskis, et al.  Expanding functional protein sequence spaces using generative adversarial networks.  Nature Machine Intelligence, 3(4):324–333, 2021.
  * Rives et al. [2021] Alexander Rives, Joshua Meier, et al.  Biological structure and function emerge from scaling unsupervised learning to 250 million protein sequences.  Proceedings of the National Academy of Sciences, 118(15):e2016239118, 2021.
  * Russ et al. [2020] William P Russ, Matteo Figliuzzi, et al.  An evolution-based model for designing chorismate mutase enzymes.  Science, 369(6502):440–445, 2020.
  * Sgarbossa et al. [2023] Damiano Sgarbossa, Umberto Lupo, et al.  Generative power of a protein language model trained on multiple sequence alignments.  Elife, 12:e79854, 2023.
  * Shahriari et al. [2015] Bobak Shahriari, Kevin Swersky, et al.  Taking the human out of the loop: A review of bayesian optimization.  Proceedings of the IEEE, 104(1):148–175, 2015.
  * Sinai et al. [2020] Sam Sinai, Richard Wang, et al.  Adalead: A simple and robust adaptive greedy search algorithm for sequence design.  arXiv preprint arXiv:2010.02141, 2020.
  * Sohl-Dickstein et al. [2015] Jascha Sohl-Dickstein, Eric Weiss, et al.  Deep unsupervised learning using nonequilibrium thermodynamics.  In ICML, pages 2256–2265. PMLR, 2015.
  * Stanton et al. [2022] Samuel Stanton, Wesley Maddox, et al.  Accelerating bayesian optimization for biological sequence design with denoising autoencoders.  In ICML, pages 20459–20478. PMLR, 2022.
  * Sternke and Karpiak [2023] Matt Sternke and Joel Karpiak.  Proteinrl: Reinforcement learning with generative protein language models for property-directed sequence design.  In NeurIPS 2023 Generative AI and Biology (GenBio) Workshop, 2023.
  * Sutton and Barto [2018] Richard S Sutton and Andrew G Barto.  Reinforcement learning: An introduction.  MIT press, 2018.
  * Swersky et al. [2020] Kevin Swersky, Yulia Rubanova, et al.  Amortized bayesian optimization over discrete spaces.  In UAI, pages 769–778. PMLR, 2020.
  * Szymczak et al. [2023] Paulina Szymczak, Marcin Możejko, et al.  Discovering highly potent antimicrobial peptides with deep generative model hydramp.  Nature Communications, 14(1):1453, 2023.
  * Vaswani et al. [2017] Ashish Vaswani, Noam Shazeer, et al.  Attention is all you need.  NeurIPS, 30, 2017.
  * Wang et al. [2023a] Yi Wang, Hui Tang, et al.  Self-play reinforcement learning guides protein engineering.  Nature Machine Intelligence, 5(8):845–860, 2023.
  * Wang et al. [2023b] Zeyuan Wang, Qiang Zhang, et al.  Instructprotein: Aligning human and protein language via knowledge instruction.  arXiv preprint arXiv:2310.03269, 2023.
  * Wu et al. [2020] Zachary Wu, Kevin K Yang, et al.  Signal peptides generated by attention-based neural networks.  ACS Synthetic Biology, 9(8):2154–2161, 2020.
  * Wu et al. [2021] Zachary Wu, Kadina E Johnston, et al.  Protein sequence design with deep generative models.  Current opinion in chemical biology, 65:18–27, 2021.
  * Yi et al. [2023] Kai Yi, Bingxin Zhou, et al.  Graph denoising diffusion for inverse protein folding.  In NeurIPS, 2023.
  * Zheng et al. [2023] Zaixiang Zheng, Yifan Deng, et al.  Structure-informed language models are protein designers.  In ICML, 2023.
  * Zhu et al. [2023] Yiheng Zhu, Jialu Wu, et al.  Sample-efficient multi-objective molecular optimization with gflownets.  arXiv preprint arXiv:2302.04040, 2023.


# Dynamic scaling and stochastic fractal in nucleation and growth processes

Amit Lahiri†, Md. Kamrul Hassan†, Bernd Blasius$ and Jürgen Kurths‡ †
University of Dhaka, Department of Physics, Theoretical Physics Division,
Dhaka 1000, Bangladesh
$ Institute for Chemistry and Biology of the Marine Environment (ICBM) Carl-
von-Ossietzky University Oldenburg PF 2503, 26131 Oldenburg, Germany
‡ Potsdam Institute for Climate Impact Research (PIK), Postfach 601203, 14412
Potsdam, Germany

###### Abstract

A class of nucleation and growth models of a stable phase (S-phase) is
investigated for various different growth velocities. It is shown that for
growth velocities $v\sim s(t)/t$ and $v\sim x/\tau(x)$, where $s(t)$ and
$\tau$ are the mean domain size of the metastable phase (M-phase) and the mean
nucleation time respectively, the M-phase decays following a power law.
Furthermore, snapshots at different time $t$ are taken to collect data for the
distribution function $c(x,t)$ of the domain size $x$ of M-phase are found to
obey dynamic scaling. Using the idea of data-collapse we show that each
snapshot is a self-similar fractal. However, for $v={\rm const.}$ like in the
classical Kolmogorov-Johnson-Mehl-Avrami (KJMA) model and for $v\sim 1/t$ the
decay of the M-phase are exponential and they are not accompanied by dynamic
scaling. We find a perfect agreement between numerical simulation and
analytical results.

###### pacs:

68.55.Ac, 64.70.Kb, 61.50.Ks

A class of nucleation and growth process is studied analytically by solving an
integro-partial differential equation and verified numerically by Monte Carlo
simulation. The growth velocity is defined as the ratio of the distance
traveled $s$ by the new phase and the magnitude of time $t$ is needed to
travel that distance. We first choose constant growth velocity by assuming
both $s$ and $t$ equal to constant and reproduce the results of the much
studied classical Kolmogorov-Johnson-Mehl-Avrami (KJMA) model. Choosing one of
them a constant still makes the exponential decay of the meta-stable phase
like the KJMA model. However, the growth velocity is so chosen that neither
$s$ nor $t$ is constant we find that the meta-stable phase decays following a
power-law. Such power-law is also accompanied by the emergence of fractal. The
self-similar property of fractal is verified by showing that the system
exhibits dynamic scaling revealing a self-similar symmetry along the
continuous time axis. According to Noether’s theorem there must exist a
conserved quantity and we do find that the $d_{f}$th moment, where $d_{f}$ is
the fractal dimension, is always conserved.

##  I Introduction

The formation of any phase is usually a process that starts first by
nucleation of new phase on old phase, which is called metastable phase,
followed by growth of the new phase which eventually becomes a stable phase
(S-phase). This mechanism of nucleation and growth represents one of the most
fundamental topics of interest in both science and technology kn.christian .
It plays a key role in metallurgical applications as well as in many seemingly
unrelated fields of research. Phase separation and coarsening ref.bray ,
electrodeposition of metals at electrodes via nucleation and growth ref.kong ,
melting of stable glasses ref.jack , dendritic growth ref.langer , kinetics of
crystal growth kn.crystallization , the domain switching phenomena in
ferroelectrics kn.ferroelectrics , the spread of ecological invaders ref.arim
and growth of breath figures ref.meakin  are just a few examples of such
systems. Besides, G. Giacomelli et al. studied an interesting case in which
nucleation occurs in time not in space ref.giacomelli . The nature of phase
transition, which is governed by nucleation and growth of domains of S-phase,
is well-known as first order phase transition. One of the characteristic
features is that during the transition stable and metastable regions coexist
at the same time. Much of our theoretical understanding of such phenomena is
provided by the Kolmogorov-Johnson-Mehl-Avrami (KJMA) model, which has been
formulated independently by Kolmogorov, Johnson and Mehl and Avrami in and
around the $1940$s kn.kolmogorov . This model still remains one of the most
studied theories of nucleation and growth processes as it is often the only
means of interpreting the experimental data that help gain insights into the
process.

The KJMA theory is valid under the assumptions that (a) nucleation events are
Poissonian in nature, (b) seeds grow with constant velocity while keeping
fixed geometrical shape and orientation, and (c) the system is homogeneous in
space and time. In the context of nucleation and growth phenomena, one of the
central quantities of interest is the fraction of the M-phase $\Phi(t)$ that
still survives at time $t$. According to the KJMA theory, this quantity in $d$
dimensions follows an exponential decay known as the Kolmogorov-Avrami law

$\Phi(t)=\exp\Big{[}-{{\Omega_{d}}\over{d+1}}\Gamma v_{s}^{d}t^{a}\Big{]},$
(1)

where, the Avrami exponent $a=d+1$ kn.kolmogorov , $\Gamma$ describes the
constant nucleation rate per unit volume and $\Omega_{d}$ is the constant
volume factor of the $d$ dimensional hypersphere e.g.,
$\Omega_{d}=1,\pi,4\pi/3$ for $d=1,2,3$ respectively.

The derivation of correlation functions and their connection to the scattering
cross-section kn.sekimoto ; kn.sekimoto_ng , the theory of grain-size
populations kn.population , and the generalization of the KJMA theory to
multiple stable phases kn.multiphase  etc. have played a significant role in
understanding the phenomena. Besides, these studies have provided a better
means of interpreting the experimental data. On the other hand, there have
been reports that the experimental data in some cases do not fit a straight
line in the plot of $\log(-\log[\Phi(t)])$ against $\log[t]$ revealing that it
violates the Kolmogorov-Avrami law kn.price ; kn.sessa . Recently, it has also
been observed that nucleation and growth processes results in the emergence of
fractal kn.fractal_nucleation_1 ; kn.fractal_nucleation_2 ;
kn.fractal_nucleation_3 ; kn.fractal_tieng ; ref.riedel . Obviously, constant
growth velocity does not result in fractal. However, the nature of growth
mechanism responsible for the emergence of fractal is not yet fully
understood. This observation clearly raises some concerns and hence it
requires further theoretical interest in order to find variants of the model
which would be suitable under different physical situations. This is exactly
the purpose of the present work.

In this article, we investigate a class of Kolmogorov-Johnson-Mehl-Avrami
nucleation and growth model in one dimension for four different choices of the
growth velocities. In particular, we choose (A) $v=const.$, (B) $v=\sigma/t$,
(C) $v=ks(t)/t$ and (D) $v=mx/\tau(x)$ where $\sigma,k$ and $m$ are constants,
$s(t)$ is the mean domain size and $\tau(x)$ is the mean nucleation time
kn.maslov . The idea to move away from the constant growth velocity is born by
the observation that a growth velocity which may change in the course of time
and space. Such choices may provides a more realistic description and hence
will become more suitable under various natural conditions than constant
growth velocity. Usually, as the time proceeds, size and shape of both phases
change continuously, altering the condition that ultimately determines the
growth velocity. We solve the corresponding rate equation for each choice of
the growth velocity and obtain the exact solution for the domain size
distribution function $c(x,t)$ of M-phase. Alongside, we also give exact
algorithm for each case to solve them by numerical simulation. Using the idea
of data collapse we show that the analytical results are in perfect agreement
with our numerical simulations. It has been found that the M-phase decays
exponentially with time for models (A) and (B) whereas in the case of models
(C) and (D), it decays following a power-law. Moreover, the domain size
distribution function $c(x,t)$ of models (C) and (D) is found to obey dynamic
scaling $c(x,t)\sim t^{\theta z}\phi(x/t^{z})$ revealing that it evolves with
time in a self-similar fashion and eventually emerges as fractal. We find the
kinetic exponent $z=1$ and the mass exponent $\theta=1+d_{f}$. Their numerical
values are fixed by the dimensional consistency and the conservation
principle.

The rest of this article is organized as follows. Section II formulates the
rate equation for nucleation and growth model in one dimension. We also
attempt to solve the rate equation for generalized growth velocity. In Section
III-VI, we solve the rate equation for four different growth velocities and
discuss the corresponding scaling theory. For each choice of the growth
velocity exact algorithm is given and extensive Monte Carlo simulation is
performed to corroborate all the analytical results namely decay of the
M-phase, fractal dimension, a conserved moment and dynamic scaling. In Section
VII, we make concluding remarks and leave some open questions.

##  II Formulation of the Rate equation

The right choice of the growth velocity should depend on the detailed nature
of the system under investigation. In the present work, we make a few simple
choices so that we can handle the problem analytically. Further, we took extra
care on the dimensional consistency with the terms describing the nucleation
mechanism. Our aim is to learn how much effect the growth velocity alone has
on the dynamics of the system. To obtain a mathematical formalism of the
phenomena, we find it useful to treat nucleation and growth mechanisms
separately and derive their respective governing equations. To do it, we
define the distribution function $c(x,t)$ as the concentration of domain size
$x$ of M-phase at time $t$. First, we derive the rate equation for the
nucleation mechanism. The random nucleation of seeds of S-phase can be thought
of as a random sequential adsorption (RSA) of monodisperse, size-less
particles on a substrate of M-phase. The distribution function $c(x,t)$ then
obeys the following well known rate equation for RSA kn.sekimoto_ng ; kn.naim

$\left.{{\partial c(x,t)}\over{\partial t}}\right|_{{\rm
nucleation}}=-x\,c(x,t)+2\int_{x}^{\infty}c(y,t)dy.$ (2)

The two terms on the right hand side of Eq. (2) represent the decrease and
increase of M-phase intervals of size $x$ due to the nucleation of point-like
seeds of S-phase at locations $x$, and $y>x$, respectively. The factor $2$ in
the integral term takes into account that every nucleation event creates two
new domains of M-phase either of which can be of size $x$.

To derive an expression for the growth mechanism, we consider that once a seed
of S-phase has nucleated at some point on M-phase, it keeps growing with some
velocity $v(x,t)$. Here we assume a simplified generalized case where the
growth velocity may depend on the size $x$ of the M-phase and the time $t$. In
general, the growth velocity may also depend on the size of the S-phase
segment, but that goes beyond the scope of the present work. To derive the
growth equations, note that the growth of the S-phase occurs at the expense of
the decay of the M-phase. That is, the size of the M-phase on either side of
the growing seeds are shrinking with the same velocity $v$ with which seeds of
the S-phase are growing. The growth term can be deduced by taking into account
all the possible ways in which the distribution function $c(x,t)$ remains in
the domain size range $[x,x+dx]$ after a span of infinitesimal time $dt$

$\displaystyle c(x,t+dt)$ $\displaystyle=$
$\displaystyle\big{[}1-2v(x,t)dt/dx\big{]}c(x,t)+$
$\displaystyle\big{[}2v(x+dx,t)dt/dx\big{]}c(x+dx,t).$

Here, $v(x,t)dt/dx$ is the fraction of concentration $c(x,t)$ that is lost in
time $dt$ due to the growth of the S-phase with velocity $v(x,t)$, and the
factor 2 takes into account of the fact that M-phase segments are reduced from
both sides. Thus

$\left.{{\partial c(x,t)}\over{\partial t}}\right|_{{\rm
growth}}=2{{\partial}\over{\partial x}}\big{[}v(x,t)c(x,t)\big{]}$ (4)

Putting the nucleation and growth terms together, the general rate equation
for the random nucleation followed by space-time dependent continuous growth
therefore is

$\displaystyle{{\partial c(x,t)}\over{\partial t}}$ $\displaystyle=$
$\displaystyle-xc(x,t)+2\int_{x}^{\infty}c(y,t)dy$ $\displaystyle+$
$\displaystyle 2{{\partial}\over{\partial x}}\big{[}v(x,t)c(x,t)\big{]}.$

We now attempt to solve Eq. (II) subject to the initial conditions

$c(x,0)={{1}\over{L}}\delta(x-L),\hskip
17.07182pt\lim_{L\longrightarrow\infty}\int_{0}^{\infty}c(x,0)dx=0.$ (6)

This ensures that there is no seed of the S-phase at $t=0$. Once we know
$c(x,t)$, we can immediately find the fraction of the M-phase that remained
un-transformed

$\Phi(t)=\int_{0}^{\infty}xc(x,t)dx$ (7)

and the number density of the domain size of the M-phase

$N(t)=\int_{0}^{\infty}c(x,t)dx.$ (8)

The fraction of the M-phase covered by the S-phase is related to $\Phi(t)$ via
$\theta(t)=1-\Phi(t)$ that evolves with time as

${{d\theta(t)}\over{dt}}=\int_{0}^{\infty}v(x,t)c(x,t)dx.$ (9)

The quantities $\Phi(t)$ and $N(t)$ can also be used to obtain the time
dependence of the average interval size $s(t)=\Phi(t)/N(t)$.

In the context of the present work, it is the choice of the growth velocity
$v(x,t)$ that will define different models and accordingly we study the
following four different choices: (A) $v(x,t)=v_{0}$, the traditional constant
growth velocity (classical KJMA model), (B) $v(x,t)=\sigma/t$ where $\sigma$
is a constant and bears the dimension of length, (C) $v(x,t)=s(t)/t$ where
$s(t)$ is the mean interval size of the M-phase at time $t$ and finally (D)
$v(x,t)=mx/\tau(x)$ where $\tau(x)$ is the mean nucleation time and $m$ is a
dimensionless positive constant kn.maslov .

For growth velocities $v=v(t)$ which are independent of the variable $x$ (as
in our models (A) \- (C)), the general solution of Eq. (II) can be obtained by
substituting the ansatz

$c(x,t)=A(t)\exp[-xt].$ (10)

Here, the time dependent pre-factor $A(t)$ obeys the following ordinary
differential equation

${{d\ln A(t)}\over{dt}}=2/t-2v(t)t.$ (11)

We have to solve Eq. (11) subject to the initial condition $A(0)=0$, followed
by Eq. (6). To proceed further and for clarity, we will treat for every choice
of growth velocity separately and independently.

First, we note that the time dependence of $s(t)$ is independent of $A(t)$

$s(t)=\frac{\Phi(t)}{N(t)}=\frac{A(t)\int_{0}^{\infty}x\exp[-xt]\,dx}{A(t)\int_{0}^{\infty}\exp[-xt]\,dx}.$
(12)

Solving the integrals we find that in this case of $v=v(t)$ the mean interval
size of M-phase shrinks as

$s(t)=1/t.$ (13)

In choosing the growth velocity, we shall make use of this relation.
Interestingly, the same expression also holds for a model without the growth
term which is known as random scission model kn.ziff .

##  III Model A

(a)

(b)

Figure 1:  (a) We plot $\log(\phi(t))$ versus $t^{2}$ for different constant
growth velocities to see how the $M$-phase $\phi(t)$ decays with time. (b)
Plots of $N(t)$ versus $t$ is shown for different constant growth velocities.

We first consider the classical KJMA model defined by the constant velocity of
domain of S-phase $v(t)=v_{0}$. The goal is to reproduce the known results so
that they help build confidence on the rate equation approach and help
appreciate its simplicity. Solving the problem rest on finding the
distribution function $c(x,t)$ which means coding all the essential
information regarding this model. After substitution of $v(t)=v_{0}$ into Eq.
(11) and a straightforward integration we immediately obtain
$A(t)=t^{2}\exp[-v_{0}t^{2}]$ and the solution of Eq. (II) for the classical
KJMA model is given by

$c(x,t)=t^{2}\exp[-xt-v_{0}t^{2}].$ (14)

Using this in the definition of $\Phi(t)$, we immediately find that

$\Phi(t)=\exp[-v_{0}t^{2}].$ (15)

It implies that the plots of $\log(\Phi(t))$ versus $t^{2}$ should give a
straight line with slope equal to $-v_{0}$. This is indeed the case as shown
in Fig. (1a). Note that $v_{s}=2v_{0}$ where $v_{0}$ is the velocity of front
of the S-phase that moves forward transforming the M-phase into S-phase within
it. Thus, the velocity with which each seed grows is $v_{s}=2v_{0}$ and
substituting it into the expression for $\Phi(t)$, we can immediately recover
the celebrated Kolmogorov-Avrami law in one dimension as given in Eq.(1),
except the factor $\Gamma$. Note that Eq. (2) describes the random sequential
nucleation of one seed at each time step and hence $\Gamma=1$ in the context
of the present rate equation approach. The rate equation approach thus clearly
demonstrates its simplicity and brevity. This is however true only for one
dimension. In the context of nucleation and growth phenomena, the 1d model can
in fact fully capture the qualitative behavior of the key quantities of
interest, namely the decay of the M-phase. The Kolmogorov-Avrami law itself is
a testament to its justification which clearly shows that the exponential
decay of the M-phase is common to all dimensions.

Integration of Eq. (14) gives the number density of domain of M-phase
$N(t)=te^{-v_{0}t^{2}}$. This expression reveals that during the very early
stages $N(t)$ rises linearly due to the nucleation of the S-phase, whereas at
the late stages $N(t)$ decreases exponentially, which reflects a fast
coalescence of the neighboring stable phases as seen in Fig. (1b) kn.naim . By
taking the ratio $\Phi(t)/N(t)$ we confirm that the mean domain size $s(t)$ of
the M-phase decays as $s(t)=t^{-1}$. To verify these results we have done
Monte Carlo simulation based on the following elaborate algorithm. One time
unit of the process in one dimension can be defined as follows:

  * (i)

At the $j$th step say there are $n$ number of domains of M-phase of size
$x_{1},x_{2},...,x_{n}$.

  * (ii)

Generate a random number $R$ with a uniform distribution within the unit
interval $[0,1]$.

(_a_)

Check which of the domains of M-phase contain $R$. Say, it is the $k$th domain
whose size is $x$. Then sow a point-like seed exactly at $xR$ and go to step
(iii).

(_b_)

If $R$ falls within S-phase then increase the iteration step by one unit, and
go back to step $(ii)$.

  * (iii)

Increase the size of the seed of S-phase on either side by the constant value
$v_{0}$ independently. If the domain size of M-phase in any side is less than
$v_{0}$ then the growth of the S-phase ceases immediately at the point of
contact, while it is continuing elsewhere.

  * (iv)

Increase all the existing domains of S-phase in the same way as described in
step (iii).

  * (v)

Increase the iteration step and the number of domains of M-phase by one unit.

  * (vi)

Go to step (ii) and continue the process ad infinitum.

In Fig. (1) we show the plots of our various results and find that both
analytical solutions of the KJMA-process are in perfect agreement to direct
numerical simulations.

##  IV Model B

(a)

(b)

Figure 2:  (a) Plots of $\log(\phi(t))$ versus $t$ of model B are drawn for
different $\sigma$ value that determine the extent of growth velocities $v$
since $v=\sigma/t$. We find straight lines with slope always equal to
$-2\sigma$ which is in complete agreement with our analytical solution given
by Eq. (17). (b) The number of domains $N(t)$ of $M$-phase versus $t$ are
shown for model B as a function of time for different constant growth
velocities.

We now solve Eq. (11) for the growth velocity $v(t)=\sigma/t$ to give

$c(x,t)=t^{2}\exp[-(x+2\sigma)t].$ (16)

In this case too, we find that the fraction of the M-phase decays
exponentially

$\Phi(t)=\exp[-2\sigma t],$ (17)

but slower than that for constant velocity. Here the Avrami exponent
corresponds to the one that typically known for heterogeneous nucleation and
growth processes despite the fact in the present case it strictly describes
the homogeneous nucleation kn.heterogeneous1 . In Fig. (2a) we show plots of
$\log(\Phi(t))$ versus $t$ and find a set of straight lines with slope always
equal to $2\sigma$ which are in perfect agreement with Eq. (17). This proves
that the Avrami exponent not only depends on the nature of the nucleation
process but also on the exact choice of the growth velocity. In a similar way
we find that the number density of the M-phase varies as

$N(t)=te^{-2\sigma t}.$ (18)

This reveals that at the early stage $N(t)$ rises linearly like the model (A).
However, at the late stage, coalescence events take place less frequently than
in the model (A) as shown in Fig. (2b). On the other hand, $N(t)/t$ varies
following the same relation with time as $\Phi(t)$ given by Eq. (17). Also for
model (B) analytical and numerical results show perfect agreement.

To verify these results we have done Monte Carlo simulation based on the
algorithm described in Model A except step (iii) which is replaced as follows.

  * •

Increase the size of the seed of S-phase on either side by $\sigma/j$
independently provided domain of S-phase is greater than $\sigma$ in both
sides. However, if the domain size of S-phase in any side is less than
$\sigma/j$ then the growth ceases immediately at the point of contact, while
it continues elsewhere.

Plots in Fig. (2) are drawn using numerical data based on the above algorithm.
We find that our analytical solutions and numerical simulations are in perfect
agreement.

##  V Model C

(a)

(b)

Figure 3:  (a) Plots of $\log(\Phi(t))$ versus $\log(t)$ are drawn for
different $k$ values. The result is a set of straight lines with slopes equal
to $-2k$ as predicted by our analytical results. In (b) We show plots of
$\log(N(s))$ versus $\log(s)$ are shown for different $k$ values and find
straight lines with slopes $-(1-2k)$ as predicted by Eq. (25).

Next we solve the rate equations for the case $v(x,t)=ks(t)/t$, where $k$ is a
dimensionless parameter. Here, we make use of the fact that the mean domain
size of the M-phase decays as $s(t)=1/t$ which is confirmed by numerical
simulations and found that such a behaviour is independent of precise choice
of the growth velocity. Incorporating this time dependence into the definition
(C) gives $v(x,t)=k/t^{2}$. Solving Eq. (11) for this front velocity yields

$c(x,t)=t^{2(1-k)}\exp[-xt].$ (19)

Unlike the previous two cases, this model is of particular interest for the
following reasons. First, all the moments, $M_{n}(t)$ of $c(x,t)$ where

$M_{n}(t)=\int_{0}^{\infty}x^{n}c(x,t)dx,$ (20)

exhibit a power-law

$M_{n}(t)\sim t^{-(n-(1-2k))}.$ (21)

Using this relation we find that the mean domain size $s(t)$ of the M-phase
decays as

$s(t)={{M_{1}(t)}\over{M_{0}(t)}}=t^{-1}.$ (22)

We find that the mean domain size decays following exactly the same way as for
Model (A) and (B). Secondly, unlike the previous two cases, the fraction of
the M-phase decays following a power-law

$\Phi(t)=t^{-2k}.$ (23)

To verify this we plot $\log(\Phi(t))$ versus $\log(t)$ for different $k$
values in Fig. (3a) using simulation data and we find straight lines with
slopes equal to $2k$ in each case. It suggests that our numerical results
match perfectly with our analytical solution given by Eq. (23).

The physical constraints of the model restrict the feasible range of $k$
values according to Eq. (21). For instance, the lower bound is fixed by the
behaviour of $\Phi(t)$ that must be an increasing function of time and hence
it demands $k>0$. On the other hand, the upper bound is fixed by the
constraint that the zeroth moment or the number density should be an
increasing function of time or at least during the early and/or at
intermediate stage. This immediately provides the upper bound $k<0.5$ since
according to Eq. (21) we find

$M_{0}(t)=N(t)\sim t^{1-2k}$ (24)

Thus, the only non-trivial and physically interesting $k$ value are the ones
that stay within the interval $[0,0.5]$. One significant result of the present
model is that unlike the previous two models here the number density of
domains of M-phase keeps increasing at all time (in the scaling regime). This
is due to fact that the front velocity is a decreasing function of time
resulting in a decelerated motion of the front and hence if the size of the
M-phase is large enough the front will effectively stop at some stage leaving
a finite sized interface. In passing, we note that in this model again the
average domain size $s(t)$ of M-phase decays as $s(t)=t^{-1}$. The existence
of scaling and the non-trivial conservation law provides an extra motivation
to go beyond the simple scaling description. This can be done by invoking the
idea of fractal analysis, as it has been a very useful tool to obtain a global
exponent called fractal dimension. To do so, we need a proper yard-stick to
measure the size of the set created in the long time limit. The most
convenient one is the mean domain size $s(t)$. Using the relation for $s(t)$
we can eliminate $t$ from Eq. (24) and find that the number $N(s)$ scales with
yard-stick size or average domain size $s$ following a power-law

$N(s)\sim s^{-d_{f}},$ (25)

where $d_{f}=(1-2k)$ provided $k<0.5$. The exponent $d_{f}$ is known as the
fractal dimension or the Hausdorff-Besicovitch dimension of the resulting set
created by the nucleation and growth process ref.hassan_hassan ; kn.fractal ;
ref.hassan_santo ; kn.hassan ; ref.fractals_95 ; ref.hassan_kurths ;
ref.fractal_self . In Fig. (3b) we plot $\log(N(s)))$ versus $\log(s)$ using
numerical simulation data for different $k$ and find straight line with slope
as it should be according to Eq. (25). Once again it proves that numerical
simulation and analytical results match perfectly.

(a)  Figure 4:  Plots of the $d_{f}$th moment of $c(x,t)$ versus time $t$ are
shown for different values of $k<0.5$. It clearly demonstrates that the
$d_{f}$th moment is always a conserved quantity suggesting that numerical and
analytical results are in perfect agreement.

Despite the numerical values of the major variables of the system, interval
sizes $x_{i}$, are changing with time yet according to Eq. (21) we observe
that $(1-2k)$th moment of $c(x,t)$ is a conserved quantity. That is, at any
given time if there are $m$ number of intervals of M-phase of size
$x_{1},x_{2},\cdots,x_{m}$ then

$M_{1-2k}(t)=x_{1}^{1-2k}+x_{2}^{1-2k}+\cdots+x_{m}^{1-2k}={\rm const.},$ (26)

regardless of the time and the value of $m$ provided $k<0.5$ (see Fig. (4)).
Note that self-similarity along the continuous time axis is a kind of
symmetry. Thus, on one hand, we find that the system enjoys dynamical scaling
symmetry that manifests itself through data collapse and on the other, we have
the $(1-2k)$th moment of the interval size of the M-phase is a conserved
quantity in time. The emergence of non-trivial conserved quantity accompanied
by continuous self-similar symmetry in time is reminiscent of Noether’s
theorem as it states that for every symmetry there is a corresponding
conservation law and vice versa ref.fractal_conserved ; ref.fractal_rakib .

(a)

(b)

(c)

(d)

Figure 5:  The plots of the distribution function $c(x,t)$ vs $x$ are shown in
(a) and (b) for $k=0.1$ and $k=0.2$ respectively. Each plot represents four
different times where each set of data represents ensemble average over $1000$
independent realizations. In (c) and (d) we plot $\log[c(x,t)/t^{2-2k}$ vs
$xt$ using the data of (a) and (b) respectively and find that all the data
collapse into a straight line with slope equal to $-1$ as suggested by Eq.
(19).

We know that if a function $f(x,t)$ of a time varying phenomena satisfies the
condition

$f(x,t)\sim t^{\theta}\phi(x/t^{z}),$ (27)

then it is said to obey dynamic scaling ref.hassan_ba_dc ;
ref.hassan_sarkar_liana_dc ; ref.fractal_cda . The solution for $c(x,t)$ given
by Eq. (19) has exactly the same form as in Eq. (27) provided we choose
$\theta=2(1-k)$ and $z=-1$. Here, the exponent $\theta$ and $z$ are fixed by
the dimensional requirement that $[f]=[t^{\theta}]$ and $[x]=t^{z}]$
respectively. Thus, the model (c) exhibits dynamic scaling with
$\theta=2(1-k)$ and $z=-1$. It implies that the numerical value of $c(x,t)$
varies with $x$ for a given time $t$. That is, the data of $c(x,t)$ versus $x$
will be distinct for fixed time and for each different $k$ value which can be
seen in Figs. (5a) and (5b). However, if we plot $c(x,t)t^{-2(1-k)}$ or
$c(x,t)t^{-(1+d_{f})}$ versus $xt$ they should collapse into one universal
scaling curve. Indeed, they do so, however to show that the scaling function
$\phi(\xi)$ is exponential we plot them in the $\log$-linear scaling and find
that all the distinct curves in Figs. (5a) and (5b) collapse superbly onto
their respective universal curves which are shown respectively in Figs. (5c)
and (5d). It means that the solution for the scaling function is exponential.
Besides, the data collapse means that the system evolves with time and that
the snapshots taken at different times are similar. Since the same system at
different times are similar the solutions are self-similar. Self-similarity in
this problem manifests, only statistically, through dynamical scaling. This is
one of the key properties of fractal too.

To verify these results we have done Monte Carlo simulation based on the
algorithm described in Model A except step (iii) which is replaced as follows.

  * •

Increase the size of the seed of S-phase on either side by $k/j^{2}$
independently provided domain of S-phase is greater than $\sigma$ in both
sides. However, if the domain size of S-phase in any side is less than
$\sigma/j^{2}$ then the growth ceases immediately at the point of contact,
while it continues elsewhere.

The data for all the plots shown for this model are obtained by Monte Carlo
simulation based on the algorithm described above. The perfect matching with
our analytical solution suggests that the algorithm is ideal for describing
the integro-differential equation given by Eq. (II) with growth velocity
$v=k/t^{2}$.

##  VI Model D

(a)

(b)

Figure 6: Plots of $\log(N(s))$ versus $\log(s)$, where $N$ is number of
domains of M-phase, are shown in (a) for different $m$ values. In (b) we show
plots of $\log(\Phi(t))$ versus $\log(t)$ for different $m$ values. In each
case it results in a straight line with slopes always equal to $-d_{f}$ in (a)
and $-(1-d_{f})$ in (b). These results match perfectly with our analytical
results.

Finally, we consider the case where $v(x,t)=mx/\tau$ which implies that the
S-phase travels a distance $x$ of the M-phase in time $\tau$ transforming $x$
into the S-phase kn.maslov ; ref.fractal_cda . However, according to Eq.(II),
the typical time between two nucleation events on an interval of size $x$ is
$\tau=x^{-1}$ and therefore $v(x,t)=mx^{2}$ revealing that the growth velocity
decreases increasingly fast as the domain size of the M-phase decreases since
$x$ itself is a decreasing quantity with time. To solve the model we
incorporate the definition of $M_{n}(t)$ into Eq. (II) and then, after some
simple algebraic manipulations, we are able to write the following rate
equation for $M_{n}(t)$

${{dM_{n}(t)}\over{dt}}=-\Big{[}{{n^{2}+n(1+{{1}\over{2m}})-{{1}\over{2m}}}\over{(n+1)}}\Big{]}M_{n+1}(t).$
(28)

Note that for $m=0$ the total mass or $M_{1}(t)$ is a conserved quantity.
However, for $m>0$ the system violates this conservation of mass principle due
to continuous growth of $S$-phase at the expense of $M$-phase. The interesting
feature of the above equation is that for each value of $m$ there exists a
unique conserved quantity. We can find the value $n=\gamma$ for which the
moment $M_{\gamma}(t)$ is a conserved quantity and it is done simply by
setting $dM_{n}(t)/dt=0$ ref.krapivsky_naim_fractal . Note that
$dM_{n}(t)/dt=0$ can be equal to zero either due to $M_{n+1}(t)=0$ of Eq. (28)
or due to its co-factor equal to zero. If the former is zero then it leads to
trivial result. On the other hand, if the latter is true then the problem
rests on solving a quadratic equation in $\gamma$ whose real positive root is

$\gamma(m)=-{{1}\over{2}}\Big{(}1+{{1}\over{2m}}\Big{)}+{{1}\over{2}}\sqrt{\Big{(}1+{{1}\over{2m}}\Big{)}^{2}+{{2}\over{m}}}.$
(29)

It implies that the corresponding moment $M_{\gamma(m)}(t)$ is independent of
time. It has been verified by numerical simulation as shown in Fig. (7). To
find the significance of the value of $\gamma$ we once again invoke the idea
of fractal. Like in model (C), we can use $s(t)$ as a yard-stick and obtain
the $N(s)$ we need to cover the system.

(a)  Figure 7:  Here we show that the $d_{f}$th moment of $c(x,t)$ is always a
conserved quantity for all $m$ values of model D.

We can solve Eq. (28) by assuming a trial solution

$M_{n}(t)\sim t^{\alpha(n)},$ (30)

such that $\alpha(\gamma)=0$ since $M_{\gamma}(t)$ is a conserved quantity.
Substituting it in Eq. (28) and demanding dimensional consistency we find a
recurrence relation

$\alpha(n+1)=\alpha(n)-1.$ (31)

Iterating it over and over again subject to the condition that
$\alpha(\gamma)=0$ as required by the conservation law we find that

$M_{n}(t)\sim t^{-(n-\gamma(m))}.$ (32)

Using this we find that the average domains of the M-phase
$s(t)=M_{1}(t)/M_{0}(t)$ decays exactly like all previous three cases i.e.,
$s(t)=t^{-1}$. We then find that $N(s)$ scales as

$N(s)\sim s^{-d_{f}},$ (33)

where $d_{f}(m)$ is given by Eq. (29). The plots of $\log(N(s))$ versus
$\log(s)$ for different $m$ are shown in Fig. (6a) using data from extensive
Monte Carlo simulation and find excellent straight lines with slopes equal to
$d_{f}$ ref.hassan_hassan ; ref.fractal_self . The exponent $d_{f}$ is known
as the fractal dimension or the Hausdorff-Besicovitch dimension of the
resulting set created by the nucleation and growth process kn.fractal .
Besides, like in model (C), the Kolmogorov-Avrami formula in this case is no
longer valid. Instead, it is replaced by the following general power-law decay
of the M-phase

$\Phi(t)\sim t^{-(1-d_{f})}.$ (34)

In Fig. (6b) we show that plots of $\log(\Phi(t)$ versus $\log(t)$ for
different $m$ and find that they all obey Eq. (34). This reveals a generalised
exponent $(1-d_{f})$ that can quantify the extent of the decay of the M-phase.
On the other hand, the coverage by the S-phase $\theta(t)$ reaches its
asymptotic value $\theta(\infty)=1$ as

$\theta(\infty)-\theta(t)\sim t^{-(1-d_{f})},$ (35)

which is reminiscent of the Feder’s law in RSA processes kn.fractal .

(a)

(b)

(c)

(d)

Figure 8: Plots of $c(x,t)$ versus $x$ for (a) $m=0.05$ and (b) $m=0.1$ are
drawn as a representative values of $m$. In (c) and (d) we plot
$c(x,t)t^{-(1+d_{f})}$ versus $xt$ and we find excellent data collapse of the
same data of (a) and (b) respectively which confirms that the model D obeys
dynamic scaling.

Finally, we show that the solution for $C(x,t)$ exhibits dynamic scaling. To
do this, we check if the solution for $c(x,t)$ obeys dynamic scaling

$c(x,t)\sim t^{(1+d_{f})}\phi(xt).$ (36)

To verify this we plot $c(x,t)$ versus $x$ in Figs. (8a) and (8b) for
different $m$ value. According to Eq. (36) the same data would collapse if we
divide $c(x,t)$ by $t^{\sqrt{2}}$ and $x$ by $t^{-1}$. Indeed, we find
excellent data collapse as shown in Figs. (8c) and (8d) respectively for
$m=0.05$ and $m=0.1$. It provides a clear litmus test of our solution that it
obeys dynamic scaling and hence like model C, the snapshots of the model D too
taken at different times are similar. It is worth to mention that the number
density in (C) and (D) increases for all time - a sharp contrast to the models
(A) and (B) where $N(t)$ increases only at the early stage. This is due to the
fact that in models (C) and (D), the growth velocity decreases in time in such
a way that two growing phases from opposite direction hardly coalesce. In
fact, the growth of the S-phase virtually stops at some stage.

To verify these results we have done Monte Carlo simulation based on the
algorithm described in Model A except step (iii) which is replaced as follows.

  * •

Increase the size of the seed of S-phase on each side by an amount equal to
square of the respective domain size of the M-phase and the growth ceases
immediately at the point of contact, while it continues elsewhere.

Model | $\Phi(t)$ | $N(t)$ | $s(t)$ | Scaling | Fractal
---|---|---|---|---|---
$v=v_{0}$ | $e^{-v_{0}t^{2}}$ | $t\,e^{-v_{0}t^{2}}$ | $t^{-1}$ | violates | $\times$
$v=\sigma/t$ | $e^{-2\sigma t}$ | $t\,e^{-2\sigma t}$ | $t^{-1}$ | violates | $\times$
$v=ks(t)/t$ | $t^{-(1-d_{f})}$ | $t^{d_{f}}$ | $t^{-1}$ | obeys | $(1-2k)$
$v=mx^{2}$ | $t^{-(1-d_{f})}$ | $t^{d_{f}}$ | $t^{-1}$ | obeys | $d_{f}(m)$
Table 1: Summary of the various model results

##  VII Discussion of findings and summary

In this article, we have studied a class of random nucleation and growth
processes of a stable phase with three different growth velocities which can
be space and time dependent. Also, we revisit the classical KJMA model. The
three new growth velocities describe a process in which the growth velocity
depends on the local configuration of the system at every instant of time. We
have solved the models analytically and verified the results by extensive
Monte Carlo simulation. One interesting finding is that in all four cases the
mean domain size $s(t)$ of M-phase decays following a power-law $s(t)\sim
t^{-\alpha}$ with the same exponent $\alpha=1$ irrespective of the detailed
choice of the growth velocity. There must be a common mechanism behind this
common nature. Indeed, all the four models share one common thing, i.e. the
nucleation process is described by the random secession model. However, the
fraction of the untransformed space (M-phase) and the corresponding number
density are very sensitive to the specific choice of the growth velocity.

The most striking result, though, is the emergence of fractal for either fully
time dependent or fully size dependent growth velocities in the sense that the
growth velocities are either proportional to inverse square of time $t$ or
simply proportional to the square of size $x$. The system is called fractal if
the exponent of the power-law decay of $N(s)$ as a function of $s$ is less
than the dimension $d=1$ of the space where the system is embedded and at the
same time the system must be self-similar. One of the ways to test whether the
systems that evolves probabilistically with time is self-similar or not is
that it must exhibit dynamic scaling. Testing of dynamic scaling means that
the numerical values of the dimensional quantities such as $c(x,t)$ and $x$ at
different time will be different but the corresponding dimensionless
quantities $c(x,t)/t^{\theta z}$ and $x/t^{z}$ would coincide. In other words,
the plots of $c(x,t)$ versus $x$ for time will be distinct but all these
distinct plots would collapse if we plot $c(x,t)/t^{\theta z}$ versus
$x/t^{z}$ instead. We find $\theta=1+d_{f}$ and $z=-1$ for both model C and D
albeit the fractal dimension $d_{f}$ is different. The self-similarity is also
a kind of continuous symmetry along the continuous time axis. Interestingly,
we also find that during the evolution the $d_{f}$th moment is always a
conserved quantity. These two results are reminiscent of Noether’s theorem
that states that for every continuous symmetry there must exist a conserved
quantity. We have also shown that when either the temporal variable $t$ or
both spatial and the temporal in the definition of the velocity are assumed
constant, the decay of the metastable phase is always exponential and is also
accompanied by the violation of scaling. One of the key observables in
nucleation and growth processes is how the fraction of the M-phase decays to
let the new S-phase grow. We find that both in model C and D, it decays
following a power-law with a unique generalised exponent $(1-d_{f})$ though
the $d_{f}$ values are different.

Thus, the present work represents the most obvious natural extension of the
classical KJMA theory. We believe that this extension is a realistic
description for many natural processes. Furthermore, our model sketches the
process of invasion and growth of a biological colony with a random
immigration of pioneer individuals or seed arrival. One important example is
the spread of diseases such as fungal spots on plant leaves. Here one assumes
that spores are transmitted by wind or insects and randomly arrive on the
leave surface. Once an infected spot is established it stays and begins to
grow. In many studies of such biological processes the growth has been found
to deviate from the constant velocity assumption of the KJMA model. Instead it
is typically observed that the velocity increases with the increase in
proportion of non-infected areas, i.e. the M-phase (Vanderplank principle
Vander ). Therefore we believe that the present investigation of the
consequences of state-dependent growth velocities will have important
applications.

We conclude with the following words. The power-law decay of the M-phase can
be assumed to be a generalized formula replacing the classical Kolmogorov-
Avrami law, provided, the distribution of the M-phase in the late stage
describes a scale-free fractal. The fractal dimension is the quantitative
measure of the notion that the density of the M-phase is less at larger length
scale. We believe that the present work will have a significant impact in
changing the way we intended to interpret the experimental data as we are now
aware of the fascinating results due to the decelerating growth velocity.

## References

  * (1) J. W Christian, The Theory of Phase Transformations in Metals and Alloys, (Pergamon Press, New York, 1981).
  * (2) A. J. Bray, Adv. Phys. 43 357 (1994).
  * (3) D. Kong, Z. Zheng, F. Meng, N. Li, D. Li, J. of the Electrochem. Soc. 165 (2018).
  * (4) R. L. Jack and L. Berthier, J. Chem. Phys. 144 244506 (2016).
  * (5) Y. Pomeau and M. Ben Amar in Solids Far From Equilibrium, edited by C. Godréche (Cambridge University Press, Cambridge, 1992).
  * (6) Y. Yamada, N. Hamaya, J. D. Axe, and M. Shapiro, Phys. Rev. Lett. 53, 1665 (1984); H. Hermann, N. Mattern, S. Roth, and P. Uebele, Phys. Rev. B 56, 13888 (1997).
  * (7) K. Matyjasek, J. Phys. D: Appl. Phys. 34, 2211 (2001); Y. Ishibashi and Y. Takagi, J. Phys. Soc. Jpn. 31, 506 (1971).
  * (8) M. Arim, S. R. Abades, P. E. Neill, M. Lima, and P. A. Marquet, PNAS 103 374 (2005).
  * (9) P. Meakin, Rep. Prog. Phys. 55 157 (1992).
  * (10) G. Giacomelli, F. Marino, M. A. Zaks and S. Yanchuk, Phys. Rev. E 88 062920 (2013).
  * (11) A. N. Kolmogorov, Bull. Acad. Sci. USSR, Phys. Ser. 1, 355 (1937); M. Avrami, J. Chem. Phys. 7, 212 (1940) ; M. Avrami, ibid. 9, 177 (1941); W. A. Jhonson and P. A Mehl, Trans. AIMME 135, 416 (1939).
  * (12) K. Sekimoto, Phys. Lett. A 105, 390 (1984); Ken Sekimoto, Physica A 135A, 328 (1986).
  * (13) K. Sekimoto, Physica 125A 261 (1984); K. Sekimoto, Physica 128A 132 (1984);
  * (14) D. Crespo and T. Pradell, Phys. Rev. B 54, 3101 (1996).
  * (15) R. M. Bradley and P. N. Strenski, Phys. Rev. B 40, 8967 (1989).
  * (16) C. W. Price, Acta Metal. Mater. 38, 727 (1990).
  * (17) M. Castro, F. Dominguez-Adame, A. Sanchez, and T. Rodriguez, Appl. Phys. Letters, 75, 2205 (1999); M. Fanfoni and M. Tomellini, Phys. Rev. B 54, 9828 (1996); V. Erukhimovitch and J. Baram, ibid. 50, 5854 (1994).
  * (18) H. Su, Y. Wang, L. Ren, P. Yuan, B. K. Teo, S. Lin, L. Zheng, and N. Zheng, Inorg. Chem. 58 259 (2019).
  * (19) M. R. Riedel, S. Karato, Geophys. J. Int. 125 397 (1996).
  * (20) S. Labidi, Z. Jia, M. B. Amar, K. Chhora and A. Kanaev, Phys. Chem. Chem. Phys. 17 2651 (2015).
  * (21) S. Tieng, O. Brinza, K. Chhor and A Kanaev, J Sol-Gel Sci Technol 64 145 (2012).
  * (22) M. R. Riedel and S. Karato, Geophys. J. lnt. 125,397 (1996).
  * (23) D. L. Maslov, Phys. Rev. Lett. 71, 1268 (1993).
  * (24) E. Ben-Naim and P. L. Krapivsky, Phys. Rev. E 54, 3562 (1996).
  * (25) R. M. Ziff and E. D. McGrady, J. Phys. A 18, 3027 (1985).
  * (26) E. D. Zanotto and M. C. Weinberg, J. Non-Crys. Solids 105, 53 (1988).
  * (27) M. K. Hassan, M. Z. Hassan, Phys. Rev. E 79 021406 (2009).
  * (28) M. K. Hassan, M. Z. Hassan and N. Islam, Phys. Rev. E 88 042137 (2013).
  * (29) J. Feder, Fractals, (Plenum, New York, 1988).
  * (30) S. Banerjee, M. K. Hassan, S. Mukherjee and A Gowrisankar, Fractal Patterns in Nonlinear Dynamics and Applications (CRS press, Tayor & Francis group, New York, 2020).
  * (31) M. K. Hassan and J. Kurths, Phys. Rev. E 64 016119 (2001).
  * (32) M. K. Hassan and G. J. Rodgers, Phys. Lett. A 208 95 (1995).
  * (33) M. K. Hassan and J. Kurths, Physica A 315 342 (2002).
  * (34) M K Hassan, Eur. Phys. J Spec. Top. 228 209 (2019).
  * (35) R. Rahman, F. Nowrin, M. S. Rahman, J. A. Wattis, M. K. Hassan, Phys. Rev. E 103 022106 (2021).
  * (36) M. K. Hassan, M Z. Hassan and N. I Pavel, J. Phys. A: Math. Gen. 44 175101 (2011).
  * (37) D. Sarker, L. Islam, M. K. Hassan, Chaos, Solitons & Fractals, 132 109591 (2020).
  * (38) M. K. Hassan and M. Z. Hassan, Phys. Rev. E 77 061404 (2008).
  * (39) P. L. Krapivsky and E. Ben-Naim, Phys. Lett. A 196 168 (1994).
  * (40) J. Vanderplank, Plant Diseases: Epidemics and Control (Acad. Press, NY, 1963).