# A Review of Large Language Models and Autonomous Agents in Chemistry

Mayk Caldas Ramos
Department of Chemical Engineering
University of Rochester, Rochester, NY
<EMAIL_ADDRESS>
& Christopher J. Collison
School of Chemistry and Materials Science
Rochester Institute of Technology, Rochester, NY
<EMAIL_ADDRESS>
& Andrew D. White
FutureHouse Inc., San Francisco, CA
Department of Chemical Engineering
University of Rochester, Rochester, NY
<EMAIL_ADDRESS>
Corresponding author

###### Abstract

Large language models (LLMs) have emerged as powerful tools in chemistry,
significantly impacting molecule design, property prediction, and synthesis
optimization. This review highlights LLM capabilities in these domains and
their potential to accelerate scientific discovery through automation. We also
review LLM-based autonomous agents: LLMs with a broader set of tools to
interact with their surrounding environment. These agents perform diverse
tasks such as paper scraping, interfacing with automated laboratories, and
synthesis planning. As agents are an emerging topic, we extend the scope of
our review of agents beyond chemistry and discuss across any scientific
domains. This review covers the recent history, current capabilities, and
design of LLMs and autonomous agents, addressing specific challenges,
opportunities, and future directions in chemistry. Key challenges include data
quality and integration, model interpretability, and the need for standard
benchmarks, while future directions point towards more sophisticated multi-
modal agents and enhanced collaboration between agents and experimental
methods. Due to the quick pace of this field, a repository has been built to
keep track of the latest studies: https://github.com/ur-whitelab/LLMs-in-
science.

_Keywords_ Large Language Model, LLM, LLM agent, agent, science, chemistry

###### Contents

  1. 1 Introduction
    1. 1.1 Challenges in chemistry
  2. 2 Large Language Models
    1. 2.1 The Transformer
    2. 2.2 Model training
    3. 2.3 Model types
      1. 2.3.1 Encoder-only models
      2. 2.3.2 Decoder-only models
      3. 2.3.3 Encoder-decoder models
      4. 2.3.4 Multi-task and multi-modal models
  3. 3 LLMs for chemistry and biochemistry
    1. 3.1 Molecular Representations, Datasets, and Benchmarks
    2. 3.2 Property Prediction and Encoder-only mol-LLMs
      1. 3.2.1 Property Prediction
      2. 3.2.2 Encoder-only Mol-LLMs
    3. 3.3 Property Directed Inverse Design and Decoder-only mol-LLMs
      1. 3.3.1 Property Directed Inverse Design
      2. 3.3.2 Decoder-only mol-LLMs
    4. 3.4 Synthesis Prediction and Encoder-decoder Mol-LLMs
      1. 3.4.1 Synthesis Prediction
      2. 3.4.2 Encoder-decoder mol-LLMs
    5. 3.5 Multi-Modal LLMs
    6. 3.6 Textual scientific LLMs
    7. 3.7 The use of ChatGPT in Chemistry
      1. 3.7.1 Automation
  4. 4 LLM-based autonomous agents
    1. 4.1 Memory module
    2. 4.2 Planning and reasoning modules
    3. 4.3 Profiling module
    4. 4.4 Perception
    5. 4.5 Tools
  5. 5 LLM-Based autonomous agents in scientific research
    1. 5.1 Agents for literature review
    2. 5.2 Agents for chemical innovation
    3. 5.3 Agents for experiments planning
    4. 5.4 Agents for automating cheminformatics tasks
    5. 5.5 Agents for hypothesis creation
  6. 6 Challenges and opportunities
  7. 7 Conclusions

##  1 Introduction

The integration of Machine Learning (ML) and Artificial Intelligence (AI) into
chemistry has spanned several decades1, 2, 3, 4, 5, 6, 7, 8, 9, 10. Although
applications of computational methods in quantum chemistry and molecular
modeling from the 1950s-1970s were not considered AI, they laid the
groundwork. Subsequently in the 1980s expert systems like DENDRAL11, 12 were
expanded to infer molecular structures from mass spectrometry data.13 At the
same time, Quantitative Structure-Activity Relationship (QSAR) Models were
developed5 that would use statistical methods to predict the effects of
chemical structure on activity.14, 15, 16, 17 In the 1990s, neural networks,
and associated Kohonen Self-Organizing Maps were introduced to domains such as
drug design,18, 19 as summarized well by Yang et al. 5 and Goldman and Walters
20, although they were limited by the computational resources of the time.
With an explosion of data from High-Throughput Screening (HTS)21, 22, models
then started to benefit from vast datasets of molecular structures and their
biological activities. Furthermore, ML algorithms such as Support Vector
Machines and Random Forests became popular for classification and regression
tasks in cheminformatics,1 offering improved performance over traditional
statistical methods.23

Deep learning transformed the landscape of ML in chemistry and materials
science in the 2010s.24 Recurrent Neural Networks (RNNs),25, 26, 27, 28, 29
Convolutional Neural Networks (CNNs)30, 31, 32 and later, Graph Neural
Networks (GNNs),33, 34, 35, 36, 37, 38 made great gains in their application
to molecular property prediction, drug discovery,39 and synthesis
prediction.40 Such methods were able to capture complex patterns in data, and
therefore enabled the identification of novel materials for high-impact needs
such as energy storage and conversion.41, 42

In this review, we explore the next phase of AI in chemistry, namely the use
of Large Language Models (LLMs) and autonomous agents. Inspired by successes
in natural language processing (NLP), LLMs were adapted for chemical language
(e.g., SMILES43) to tackle tasks from synthesis prediction to molecule
generation44, 45, 46. We will then explore the integration of LLMs into
autonomous agents as illustrated by M. Bran et al. 47 and Boiko et al. 48,
which may be used for data interpretation or, for example, to experiment with
robotic systems. We are at a crossroads where AI enables chemists to solve
major global problems faster and streamline routine lab tasks. This enables,
for instance, the development of larger, consistent experimental datasets and
shorter lead times for drug and material commercialization. As such, language
has been the preferred mechanism for describing and disseminating research
results and protocols in chemistry for hundreds of years.49

The review structure is as follows: Section 1.1 discusses current challenges
that need to be addressed to increase the impact of AI in chemistry. Section 2
introduces transformers, covering encoder-only, decoder-only, and encoder-
decoder architectures. Section 3 surveys work with LLMs, linking each
transformer architecture to suitable chemistry areas. Section 4 describes
autonomous agents and their applications in chemistry research. Section 5
surveys LLM-based agents in chemistry. Section 6 discusses future challenges
and opportunities, followed by the conclusion in Section 7. We distinguish
between “text-based” and “mol-based” inputs and outputs, with “text” referring
to natural language and “mol” referring to the chemical syntax for material
structures, as introduced by Zhang et al. 50.

###  1.1 Challenges in chemistry

With the potential to accelerate scientific discovery through the use of AI,
it makes sense to connect this potential to some of the more pressing
opportunities in the chemical sciences as may be categorized under three broad
umbrellas: Property Prediction, Property-Directed Molecule Generation, and
Synthesis Prediction. The first challenge is to predict a property for a given
compound to decide if it should be synthesized for a specific application,
such as an indicator,51 light harvester,52 or catalyst.53 To have better
models for property prediction, good quality data is crucial. We discuss the
caveats and issues with the current datasets in Section 3.1 and illustrate
state-of-the-art findings in Section 3.2.

The second challenge is to generate novel chemical structures that meet
desired chemical profiles or specific properties.54 Success would accelerate
progress in various chemical applications, but reliable reverse engineering
(inverse design)55 is not yet feasible over the vast chemical space.56 For
instance, inverse design coupled with automatic selection of novel structures
(de novo molecular design) could develop a drug targeting a specific protein
while retaining properties like solubility, toxicity, and blood-brain barrier
permeability.57 The complexity of connecting de novo design with property
prediction is high and raises ethical questions that need to be addressed to
guide the development of such models. We show how state-of-the-art models
currently perform in Section 3.3.

The third major challenge is predicting its optimal synthesis using
inexpensive, readily available, and non-toxic starting materials. There will
always likely be a solution within a large chemical space: an alternative
molecule with similar properties that is easier to synthesize. Exploring this
space to find a new molecule with the right properties and a high-yield
synthesis route unites these challenges. The number of possible stable
chemicals is estimated to be up to $10^{180}$.58, 59, 60, 61 Exploring this
vast space requires significant acceleration beyond current methods.62
Restrepo 58 emphasizes the need to catalog data on failed syntheses to build a
comprehensive dataset of chemical features. Autonomous chemical resources can
accelerate database growth and tackle this challenge. Thus, automation is
considered a fourth major challenge in chemistry.63, 64, 65, 66 The following
discussion explores how LLMs and autonomous agents can provide the most value.
Relevant papers are discussed in Section 3.4

##  2 Large Language Models

The prior state-of-the-art for seq2seq had been the Recurrent Neural Network
(RNN),67 typically as implemented in Hochreiter and Schmidhuber 68. The RNN
retains “memory” of previous steps in a sequence to predict later parts.
However, as sequence length increases, gradients can become vanishingly small
or explosively large69, 70, preventing effective use of earlier information in
long sequences. RNNs have thus fallen behind Large Language Models (LLMs).

LLMs are deep neural networks (NN) with billions of parameters. Mostly, LLMs
implement a transformer architecture, introduced by Vaswani et al. 71,
although other architectures for longer input sequences111These longer
sequence models are more important in biology, where very long genetic
sequences are relevant are being actively explored72, 73, 74, 75. Transformers
are well-developed in chemistry and are the dominant paradigm behind nearly
all state-of-the-art sequence modeling results and, thus, the focus. More
detailed discussion on current LLMs applied to general purposes can be found
elsewhere.76

###  2.1 The Transformer

Figure 1: a) The generalized encoder-decoder transformer: The encoder on the
left converts an input into a vector, while the decoder on the right predicts
the next token in a sequence. b) Encoder-decoder transformers are
traditionally used for translation tasks and, in chemistry, for reaction
prediction, translating reactants into products. c) Encoder-only transformers
provide a vector output and are typically used for sentiment analysis. In
chemistry, they are used for property prediction or classification tasks. d)
Decoder-only transformers generate likely next tokens in a sequence. In
chemistry, they are used to generate new molecules given an instruction and
description of molecules.

The transformer was introduced in, “Attention is all you need” by Vaswani et
al. 71 in 2017. A careful line-by-line review of the model can be found in
“The Annotated Transformer” 77. The transformer was the first sequence-to-
sequence (seq2seq) model based entirely on attention, although attention had
been a feature for Recurrent Neural Networks (RNNs) some years prior.78 Here,
a seq2seq model processes a text input, such as an English paragraph, and
produces a different text sequence, such as a French translation of that
paragraph. The concept of “attention” is a focus applied to certain words of
the input, which would convey the most importance, or the context of the
passage, and thereby would allow for better decision-making and greater
accuracy. However, in a practical sense, “attention” is implemented simply as
the dot-product between token embeddings with some non-linear learned
function, and is described further below.

The transformer architecture implements two main modules: the encoder and the
decoder. Figure 1a provides a cartoon diagram of the general encoder-decoder
transformer architecture. The input is first tokenized, from the model’s
vocabulary,79, 80, 81, 82, 83 into computable integers that are then converted
into numerical vectors using embedding layers. A single encoder stack is shown
here, where in reality there are many, with six in the original model.71 The
later stacks each use outputs from the prior stack, typically as a $\sim$512
per-token vector. Each encoder applies positional encoding typically using
sine and cosine functions, with a frequency depending on that word’s position,
to handle sequences of any length. Other functions can also be used.84 The
encoder stack then comprises a multi-headed self-attention mechanism, which
relates each “word” to others in the sequence by computing attention scores
based on queries, keys, and values. This is followed by normalization and
residual connections. Residual connection is represented in Figure 1a by the
“by-passing” arrows, which indicate how each sub-layer output is added to its
input, which helps mitigate the vanishing gradient problem.69, 70 Lastly, the
output is further refined through a pointwise feed-forward network with an
activation function (such as ReLU,85 SwiGLU,86 GELU,87 etc) resulting in a set
of vectors representing the input sequence with rich contextual understanding,
ready for the decoder.

Similarly, the decoder workflow begins with converting the initial input token
into numerical vectors, known as the output embedding step, as this initial
input serves as the start token for the model’s final output. Positional
encodings are then added, followed by a stack of approximately six identical
decoder layers. Each decoder layer includes a Masked Self-Attention Mechanism
to prevent positions from attending to subsequent ones and an Encoder-Decoder
Multi-Head Attention to align encoder outputs with decoder attention layer
outputs (indicated by the square connecting arrows in Figure 1a), focusing on
relevant parts of the input. A fully connected Feed-Forward Neural Network
refines the output further. The final layer acts as a linear classifier,
mapping the output to the vocabulary size, and a softmax layer converts this
output into probabilities, with the highest indicating the predicted next
word. Each sub-layer is followed by normalization and includes a residual
connection. The decoder generates output sequences autoregressively,
incorporating each newly predicted token until an end token signals
completion. Figure 1b shows a cartoon illustration of the encoder-decoder
model’s strength in translating between languages or from atoms in reactants
mapped into new positions in associated products. Figure 1c depicts the
ability of an encoder-only model to detect an additional meaning, or property,
from an input sequence, represented as a number description. Figure 1d
presents a cartoon depiction of a decoder-only architecture inferring and
generating a consequence, based on an original input. Hence, a new peptide
sequence has been inferred after the combined input of two smaller amino
acids, indicating how the decoder-only approach can be used to generate
potentially new compounds.

###  2.2 Model training

The common lifetime of an LLM consists of being first pretrained using
unsupervised or self-supervised techniques, generating what is called a base
model. This base model is then fine-tuned for specific applications using
supervised techniques. Finally, the supervised fine-tuned model is further
tuned with reward models to improve human preference or some other non-
differentiable and sparse desired character88.

##### Unsupervised pretraining

A significant benefit implied in all the transformer models described in this
review is that unsupervised learning takes place with a vast corpus of text.
Thus, the algorithm learns patterns from untagged data, which opens up the
model to larger datasets that may not have been explicitly annotated by
humans. The advantage is to discover underlying structures or distributions
without being provided with explicit instructions on what to predict, nor with
labels that might indicate the correct answer.

##### Supervised fine-tuning

After this pretraining, many models described herein are fine-tuned on
specific downstream tasks (e.g., text classification, question answering)
using supervised learning. In supervised learning, models learn from labeled
data, and map inputs to known outputs. Such fine-tuning allows the model to be
adjusted with a smaller, task-specific dataset to perform well on that
downstream task.

##### LLM alignment

A key task after model training is aligning the output with what human users
prefer. This can be done so as to improve the LLM output’s style and tone, or
to reduce harmful outputs because the training objectives used during the
pretraining and the fine-tuning may not include human values.89 Reinforcement
learning (RL) techniques are among the most common alignment techniques. RL is
a type of machine learning where actions are performed and feedback is
received through rewards, the goal being to learn a policy that maximizes the
cumulative reward. However, human preferences can only be evaluated on
complete outputs from models, preventing supervised fine-tuning (which
requires token-by-token losses).90, 91, 92 To apply RL, sampling tokens from
the model is recast as a Markov Decision Process (MDP) - actions are tokens
and the reward is zero until the EOS (end of sequence) token, at which point
the human preference reward is applied93 Reinforcement Learning with Human
Feedback (RLHF)94 is the most used alignment technique. However, alternative
strategies include reinforcement learning with synthetic feedback (RLSF)95,
proximal policy optimization (PPO)96, REINFORCE97, and others.90, 93, 98, 99.
A general discussion about RL potential for LLM fine-tuning was published by
Cao et al. 100 and Shen et al. 89.

There are ways to reformulate the RLHF process into a direct optimization
problem with a different loss. This is known as direct preference optimization
(DPO)101 and a popular competitor to PPO (the most commonly used approach) due
to its simplicity. It overcomes the lack of token-by-token loss signal by
comparing two completions at a time. The discussions about which technique is
superior remain very active in the literature.102

Finally, the alignment may not be to human preferences but to downstream tasks
that do not provide token-by-token rewards. For example, Bou et al. 103 and
Hayes et al. 104 both use RL on a language model for improving its outputs on
a downstream scientific task.

###  2.3 Model types

While the Vaswani Transformer71 employed an encoder-decoder structure for
sequence-to-sequence tasks, the encoder and decoder were ultimately seen as
independent models, leading to “encoder-only”, and “decoder-only” models
described below.

####  2.3.1 Encoder-only models

Beyond Vaswani’s transformer,71 used for sequence-to-sequence tasks, another
significant evolutionary step forward came in the guise of the Bidirectional
Encoder Representations from Transformers, or “BERT”, described in October
2018 by Devlin et al. 105 BERT utilized only the encoder component, achieving
state-of-the-art performance on sentence-level and token-level tasks,
outperforming prior task-specific architectures.105 The key difference was
BERT’s bidirectional transformer pretraining on unlabeled text, meaning the
model looks at the context both to the left and right of the word in question,
facilitated by a Masked Language Model (MLM). This encoder-only design meant
BERT focused on generating a deeper "understanding" of the input sequence,
rather than mapping input sequences to output sequences. In pretraining, BERT
also uses Next Sentence Prediction (NSP). “Sentence” here means an arbitrary
span of contiguous text. The MLM task randomly masks tokens and predicts them
by looking at preceding and following contexts simultaneously, inspired by
Taylor106. NSP predicts whether one sentence logically follows another,
training the model to understand sentence relationships. This bidirectional
approach allows BERT to recognize greater nuance and richness in the input
data.

Subsequent evolutions of BERT include, for example, RoBERTa, (Robustly
optimized BERT approach), described in 2019 by Facebook AI’s Liu et al. 107.
RoBERTa was trained on a larger corpus, for more iterations, with larger mini-
batches, and longer sequences, improving model understanding and
generalization. By removing the NSP task and focusing on the MLM task,
performance improved. RoBERTa dynamically changed masked positions during
training and used different hyperparameters. Evolutions of BERT also include
domain-specific pretraining and creating specialist LLMs for fields like
chemistry, as described below (see Section 3).

####  2.3.2 Decoder-only models

In June 2018, Radford et al. 108 from OpenAI proposed the Generative
Pretrained Transformer (GPT) in their paper, “Improving Language Understanding
by Generative Pretraining”. GPT used a decoder-only, left-to-right
unidirectional language model to predict the next word in a sequence based on
previous words, without an encoder. Unlike the Vaswani Transformer’s decoder,
GPT could predict the next sequence, applying general language understanding
to specific tasks with smaller annotated datasets.

GPT employed positional encodings to maintain word order in its predictions.
Unlike the Vaswani Transformer, GPT’s self-attention mechanism prevented
tokens from attending to future tokens, ensuring each word prediction depended
only on preceding words. Hence a decoder-only architecture represents a so-
called causal language model, one that generates each item in a sequence based
on the previous items; the generation of each subsequent output is causally
linked to the history of generated outputs and nothing ahead of the current
word affects its generation.

####  2.3.3 Encoder-decoder models

Evolving further, BART (Bidirectional and Auto-Regressive Transformers) was
introduced by Lewis et al. from Facebook AI in 2019.109 BART combined the
context learning strengths of the bidirectional BERT, and the autoregressive
capabilities of models like GPT, which excel at generating coherent text. BART
was thus a hybrid seq2seq model, consisting of a BERT-like bidirectional
encoder and a GPT-like autoregressive decoder. This is nearly the same
architecture as Vaswani et al. 71; the differences are in the pretraining.
BART was pretrained using a task that corrupted text by, for example, deleting
tokens, and shuffling sentences. It then learned to reconstruct the original
text with left-to-right autoregressive decoding as in GPT models.

####  2.3.4 Multi-task and multi-modal models

In previous sections, we discussed LLMs that take natural language text as
input and then they output either a learned representation or another text
sequence. These models traditionally perform tasks like translation,
summarization, and classification. Raffel et al. 110 developed the Text-to-
Text Transfer Transformer (T5) to demonstrate that various tasks can be
reframed as text-to-text tasks, allowing the same model architecture and
training procedure to be used with the same set of weights across different
tasks. This approach has inspired the development of multi-task models that
can adapt to different tasks at inference time. For example, Flan-T5111 used
instruction fine-tuning with chain-of-thought prompts, enabling it to
generalize to unseen tasks, such as generating rationales before answering.
More complex approaches have been proposed for robust multi-task models.112,
113, 114, 115

Additionally, LLMs have been extended to understand different input
modalities, despite initially receiving only text. For instance, Fuyu116 uses
linear projection to adapt image representations into the token space of an
LLM, enabling a decoder-only model to write figure captions. Scaling up, next-
GPT117 was developed as an “any-to-any” model, processing multiple
modalities—text, audio, image, and video—through modality-specific encoders.
The encoded representation is projected into a decoder-only token space, and
the LLM’s output is processed by a domain-specific diffusion model to generate
each modality’s output. Multitask or multimodel methods are further described
below as these methods start to connect LLMs with autonomous agents.

for tree= forked edges, grow’=0, draw, rounded corners, node
options=align=center, text width=2.7cm, s sep=6pt, calign=edge midpoint, ,
[SciLLMs, fill=blue, root [Text2Text, task [Question-answering, application
[BioMedLM118, BioMistral119, BiMediX120, EpilepsyLLM121, CheXagent122,
BioMedGPT-LM123, Darwin124, PMC-LLaMA125, Galactica126, BioGPT127, nacho0128,
etc., study] ] ] [Text2Mol, task [Conditional de-novo generation, application
[BioT5+129, Darwin124, Text+Chem T5130, MolT5131, etc., study] ] ]
[Text2Number, task [Document classification, application [MatSciBERT132,
Galactica126, PubMedBERT133, SciBERT134, BlueBERT135, etc, study] ] [Property
prediction, application [EpilepsyLLM121, CatBERTa136, ScholarBERT137,
ClinicalBERT138, etc, study] ] [Named entity recognition, application
[nacho0128, MaterialsBert139, MatBERT140, ChemBERT141, PubMedBERT133,
BioMegatron142, BioBERT143, SciBERT134, BlueBERT135, etc, study] ] ]
[Mol2Text, task [Molecule captioning, application [BioT5+129, BioT5144,
Text+Chem T5130, Galactica126, MolT5131, etc., study] ] ] [Mol2Mol, task
[Molecule tuning, application [nacho0128, Regression Transformer145,
ChemFormer146, etc., study] ] [Synthesis prediction, application
[ReactionT5147, Galactica126, ChemFormer146, T5Chem148, etc., study] ]
[Retrosynthesis, application [Text+Chem T5130, T5Chem148, etc., study] ] ]
[Mol2Number, task [Property prediction, application [nacho0128, Regression
Transformer145, CatBERTa136, Darwin124, SELFormer149, Galactica126,
ChemBERTa-2150, MolFormer151, Mol-BERT152, MTL-BERT153, ChemBERTa44,
MolBERT154, SMILES-BERT155, SMILES transformer156etc, study] ] ] ]

Figure 2: Classification of LLMs in chemistry and biochemistry according to
their application.

##  3 LLMs for chemistry and biochemistry

Integrating LLMs into chemistry and biochemistry has revolutionized molecular
design, synthesis prediction, and property analysis. These models, trained on
vast datasets, interpret chemical languages like SMILES and InChI to predict
molecular behavior accurately. This section explores the importance of
trustworthy datasets and the necessity of good benchmarks and LLM applications
in molecular representations, property prediction, inverse design, and
synthesis prediction. Figure 2 illustrates the capabilities of different LLMs
available currently, and Figure 4 shows a chronological map of LLMs in
chemistry and biology.

###  3.1 Molecular Representations, Datasets, and Benchmarks

Given the diversity of data in chemistry-based machine learning, and the many
different ways that a compound can be described, from molecular structure to
property description, there are multiple ways a molecule can be represented,
underlining this heterogeneity.157, 158, 159, 160 Some common forms include
molecular graphs161, 162, 163, 3D point clouds164, 165, 166, 167, and
quantitative feature descriptors.168, 169, 170, 171, 158 In this review, we
will focus on string-based representations of molecules, given the interest in
language models. Among the known string representations, we can cite IUPAC
names, SMILES,43 DeepSMILES,172 SELFIES,173 and InChI,174 as recently reviewed
by Das et al. 175.

Figure 3: Number of training tokens (on log scale) available from various
chemical sources compared with typical LLM training runs. The numbers are
drawn from ZINC176, PubChem177, Touvron et al. 178, ChEMBL179, and Kinney et
al. 180

.

Regarding datasets, there are two types of data used for training LLMs:
training data and evaluation data. Training data should be grounded on real
molecules to build the correct “prior belief” of a molecule definition during
the LLM training process. Similar care is taken when natural language training
data used in GPT-4, for instance, is built from real sentences or code; it is
important not to use random combinations of words, even if they are
grammatically correct. Figure 3 shows the number of tokens on common chemistry
datasets in comparison with the number of tokens used to train LLaMA2, based
on literature information.176, 177, 178, 179, 180 With this in mind, we note
the largest chemical training corpus, which largely comprises hypothetical
chemical structures, amounts to billions of tokens, almost two orders of
magnitude fewer than the trillions of tokens used to train LLaMA2. When not
including the large number of hypothetical structures found on ZINC176,
(Figure 3), that number of tokens associated with verifiably synthesized
compounds is over five orders of magnitude lower than the LLaMA2 training
data. Nonetheless, Segler et al. 181 demonstrated that even using the Reaxys
dataset, a very small human-curated set of chemical reactions, is enough to
enable state-of-the-art retrosynthesis results. Therefore, the lack of data is
not the only problem. Instead, the lack of good-quality data may be the
pivotal factor holding back the development of better scientific LLMs.

On the other hand, benchmark data evaluates the success of these models.
Numerous datasets, curated by the scientific community, are available for this
purpose182, 183. Among them, MoleculeNet57, first published in 2017, is the
most commonly used labeled dataset for chemistry. However, MoleculeNet has
several issues: it is small, contains errors and inconsistencies, and lacks
relevance to real-world chemistry problems184, 146, 185, 128. Pat Walters, a
leader in ML for drug discovery, stated, “I think the best way to make
progress on applications of machine learning to drug discovery is to fund a
large public effort that will generate high-quality data and make this data
available to the community”186.

Walters makes several constructive critiques such as, for example, how the
QM7, QM8, and QM9 datasets, intended for predicting quantum properties from 3D
structures, are often misused with predictions based incorrectly on 1D SMILES
strings, which inadequately represent 3D molecular conformations. Walters also
makes suggestions for more relevant benchmarks and also datasets with more
valid entries. He suggests the Absorption, Distribution, Metabolism, and
Excretion (ADME) data curated by Fang et al. 187, and the Therapeutic Data
Commons (TDC)188, 189 and TDC-2190. These datasets contain measurements of
real compounds, making them grounded in reality. Moreover, ADME is crucial for
determining a drug candidate’s success, while therapeutic results in diverse
modalities align with metrics used in drug development.

Here, we hypothesize that the lack of easily accessible, high-quality data in
the correct format for training foundational chemical language models is a
major bottleneck to the development of the highly desired “super-human” AI-
powered digital chemist. A more optimistic view is presented by Rich and
Birnbaum 191. They argue that we do not need to wait for the creation of new
benchmarks. Instead, the messy, available public data can be used to curate
benchmarks carefully curated for specific real-world approximation. In
addition, we argue that extracting data from scientific chemistry papers might
be an interesting commitment to generating data of high quality, grounded to
the truth, and on a large scale. Recently, a benchmark following these ideas —
LAB-Bench192 — was created for evaluating LLMs’ performance in biology.

Figure 4: Illustration of how Large Language Models (LLMs) evolved
chronologically.

###  3.2 Property Prediction and Encoder-only mol-LLMs

Table 1: Encoder-only scientific LLMs. $a$:“Model Size” is reported as the number of parameters. $b$: The authors report they not used as many encoder layers as it was used in the original BERT paper. But the total number of parameters was not reported. LLM |  Model Sizea |  Training Data |  Architecture |  Application |  Release date
---|---|---|---|---|---
CatBERTa136 |  355M |  OpenCatalyst2020 (OC20) |  RoBERTa |  Property prediction |  2023.11
SELFormer149 |  $\sim$86M |  $\sim$2M compounds from ChEMBL |  RoBERTa |  Property prediction |  2023.05
MaterialsBERT193 |  110M |  2.4M material science abstracts + 750 annotated abstract for NER |  BERT |  NER and property extraction |  2023.04
SolvBERT194 |  b |  1M SMILES of solute-solvent pairs from CombiSolv-QM and LogS from Boobier et al. 195 |  BERT |  Property prediction |  2023.01
MolFormer196 |  $b$ |  PubChem and ZINC |  BERT |  Property prediction |  2022.12
ChemBERTa-2150 |  5M - 46M |  77M SMILES from PubChem |  RoBERTa |  Property prediction |  2022.09
ScholarBERT137 |  340M, 770M |  Public.Resource.Org, Inc |  BERT |  Property prediction |  2022.05
BatteryBERT197 |  $\sim$ 110M |  $\sim$ 400k papers from RSC, Elsevier and Springer |  BERT |  Document classification |  2022.05
MatSciBERT132 |  110M |  $\sim$150K material science paper downloaded from Elsevier |  BERT |  NER and text classification |  2022.05
MatBERT140 |  110M |  Abstracts from solid state articles and abstracts and methods from gold nanoparticle articles |  BERT |  NER |  2022.04
PubMedBERT133 |  110M |  14M abstracts from PubMed |  BERT |  NER, QA, and document classification |  2021.10
Mol-BERT152 |  110M |  $\sim$4B SMILES from ZINC15 and ChEMBL27 |  BERT |  Property prediction |  2021.09
ChemBERT141 |  110M |  $\sim$200k extracted using ChemDataExtractor |  BERT |  NER |  2021.06
MolBERT154 |  $\sim$85M |  ChemBench |  BERT |  Property prediction |  2020.11
ChemBERTa44 |  |  10M SMILES from PubChem |  RoBERTa |  Property prediction |  2020.10
BioMegatron142 |  345M, 800M, 1.2B |  Wikipedia, CC-Stories, Real-News, and OpenWebtext |  Megatron-LM |  NER and QA |  2020-10
BioBERT143 |  110M |  PubMed and PMC |  BERT |  NER and QA |  2020.02
Molecule Attention Transformer198 |  $b$ |  ZINC15 |  Encoder with GCN features |  Property prediction |  2020.02
SciBERT134 |  110M |  1.14M papers from Semantic Scholar |  BERT |  NER and sentence classification |  2019.09
SMILES-BERT199 |  $b$ |  $\sim$18M SMILES from ZINC |  BERT |  Property prediction |  2019.09
BlueBERT135 |  110M |  PubMed and MIMIC-III |  BERT |  NER, and document classification |  2019.06
ClinicalBERT138 |  110M |  MIMIC-III |  BERT |  Patient readmission probability |  2019.04

Encoder-only transformer architectures solely comprise an encoder. Chemists
can leverage the simpler encoder-only architecture for tasks where the primary
goal is classification, efficient exploration of chemical space, and property
prediction. Since encoder-only architectures are mostly applied to property
prediction, we describe here the relative importance of this principal
chemical challenge. Sultan et al. 200 also discussed the high importance of
this task, the knowledge obtained in the last years, and the remaining
challenges regarding molecular property prediction using LLMs.

####  3.2.1 Property Prediction

The universal value of chemistry lies in identifying and understanding the
properties of compounds to optimize their practical applications. In the
pharmaceutical industry, therapeutic molecules interact with the body in
profound ways.201, 202, 203 Understanding these interactions and modifying
molecular structures to enhance those therapeutic benefits can lead to
significant medical advancements.204 Similarly, in polymer science, material
properties depend on chemical structure, polymer chain length, and packing,205
and a protein’s function similarly depends on its structure and folding.
Historically, chemists have identified new molecules from natural products206
and screened them against potential targets207 to test their properties for
diseases. Once a natural product shows potential, chemists synthesize scaled-
up quantities for further testing or derivatization,208, 209, 210 a costly and
labor-intensive process.211, 212 Traditionally, chemists have used their
expertise to hypothesize the properties of new molecules derived from those
natural products, hence aiming for the best investment of synthesis time and
labor. To support the chemical industry in more accurate property prediction,
computational chemistry has evolved.213 Quantum theoretical calculations
predict certain properties reliably, while force-field-based Molecular
Dynamics (MD)214 predict packing and crystal structures of large molecular
ensembles, though both require substantial computational resources.215, 216,
217, 218, 219 Property prediction can now be enhanced through machine learning
tools,220, 170, 221, 222 and more recent advancements in LLMs lead to
effective property prediction without the extensive computational demands of
quantum mechanics and MD calculations. Combined with human insight, AI can
revolutionize material development, enabling the synthesis of new materials
with a high likelihood of possessing desired properties for specific
applications.

####  3.2.2 Encoder-only Mol-LLMs

Encoder-only models are exemplified by the BERT architecture, which is
commonly applied in natural language sentiment analysis to extract deeper
patterns from prose.223 The human chemist has been taught to look at a 2D
image of a molecular structure and to recognize its chemical properties or
classify the compound. Therefore, encoder-only models would ideally convert
SMILES strings, empty of inherent chemical essence, into a vector
representation, or latent space, reflecting those chemical properties. This
vector representation can then be used directly for various downstream tasks.

Schwaller et al. 224 used a BERT model to more accurately classify complex
synthesis reactions by generating reaction fingerprints from raw SMILES
strings, without the need to separate reactants from reagents in the input
data, thereby simplifying data preparation. The BERT model achieved higher
accuracy (98.2%) compared to the encoder-decoder model (95.2%) for classifying
reactions. Accurate classification aids in understanding reaction mechanisms,
vital for reaction design, optimization, and retrosynthesis. Toniato et al.
225 also used a BERT architecture to classify reaction types for downstream
retrosynthesis tasks that enable manufacture of any molecular target. Further
examples of BERT use include unsupervised reaction atom-to-atom mapping 226,
227. These chemical classifications would accelerate research and development
in organic synthesis, described further below.

Due to limited labeled data for molecular property prediction, Wang et al.
proposed a semi-supervised SMILES-BERT model, pretrained on a large unlabeled
dataset with a Masked SMILES Recovery task.155 The model was then fine-tuned
for various molecular property prediction tasks, outperforming state-of-the-
art methods in 2019 on three chosen datasets varying in size and property.
This marked a shift from using BERT for reaction classification towards
property prediction and drug discovery. Maziarka et al. 198 also claimed
state-of-the-art performance in property prediction after self-supervised
pretraining in their Molecule Attention Transformer (MAT), which adapted BERT
to chemical molecules by augmenting the self-attention with inter-atomic
distances and molecular graph structure.

Zhang et al. 228 also tackled the issue of limited property-labeled data and
the lack of correlation between any two datasets labeled for different
properties, hindering generalizability. They introduced multitask learning
BERT (MTL-BERT), which used large-scale pretraining and multitask learning
with unlabeled SMILES strings from ChEMBL.179 This approach mined contextual
information and extracted key patterns from complex SMILES strings, improving
model interpretability. The model was fine-tuned for relevant downstream
tasks, achieving better performance than state-of-the-art methods in 2022 on
60 molecular datasets from ADMETlab229 and MoleculeNet.57

In 2021, Li and Jiang152 introduced Mol-BERT, pretrained on four million
unlabeled drug SMILES from the ZINC15230 and ChEMBL27179 databases to capture
molecular substructure information for property prediction. Their work
leveraged the underutilized potential of large unlabeled datasets like ZINC.
Mol-BERT consisted of three components: PretrainingExtractor, Pretraining Mol-
BERT, and Fine-Tuning Mol-BERT. It treated Morgan fingerprint fragments as
“words” and compounds as “sentences,” using RDKit and the Morgan algorithm for
canonicalization and substructure identification. This approach generated
comprehensive molecular fingerprints from SMILES strings, used in a Masked
Language Model (MLM) task for pretraining. Mol-BERT was fine-tuned on labeled
samples, providing outputs as binary values or continuous scores for
classification or regression, and it outperformed existing sequence and graph-
based methods by at least 2% in ROC-AUC scores on Tox21, SIDER, and ClinTox
datasets.57

Ross et al. 151 introduced MoLFormer, a large-scale unsupervised BERT model
for more accurate and faster molecular property predictions than Density
Functional Theory calculations or wet-lab experiments. They trained MoLFormer
with rotary positional embeddings on SMILES sequences of 1.1 billion unlabeled
molecules from PubChem177 and ZINC230. Rotary positional encoding captures
token positions more effectively than traditional methods, 71 improving
modeling of sequence relationships. MoLFormer outperformed state-of-the-art
GNNs on several classification and regression tasks from ten MoleculeNet57
datasets, while performing competitively on two others. It effectively learned
spatial relationships between atoms, predicting various molecular properties,
including quantum-chemical properties. Additionally, the authors stated how
MoLFormer represents an efficient and environment-friendly use of
computational resources, reducing GPU usage in training by a factor of 60 (16
GPUs instead of 1000).

With ChemBERTa, Chithrananda et al. 44 explored the impact of pretraining
dataset size, tokenization strategy, and the use of SMILES or SELFIES,
distinguishing their work from other BERT studies. They used HuggingFace’s
RoBERTa transformer,231 and referenced a DeepChem57 tutorial for
accessibility. Their results showed improved performance on downstream tasks
(BBBP, ClinTox, HIV, Tox21 from MoleculeNet57) as the pretraining dataset size
increased from 100K to 10M. Although ChemBERTa did not surpass state-of-the-
art GNN-based baselines like Chemprop (which used 2048-bit Morgan Fingerprints
from rdkit),232 the authors suggested that with expansion to larger datasets
they would eventually beat those baselines. The authors compared Byte-Pair
Encoder (BPE) with a custom SmilesTokenizer and its regex developed by 233
while exploring tokenization strategies. They found the SmilesTokenizer
slightly outperformed BPE, suggesting more relevant sub-word tokenization is
beneficial. No difference was found between SMILES and SELFIES, but the paper
highlighted how attention heads in transformers could be visualized with
BertViz,234 showing certain neurons selective for functional groups. This
study underscored the importance of appropriate benchmarking and addresses the
carbon footprint of AI in molecular property prediction.

In ChemBERTa-2, Ahmad et al. 150 aimed to create a foundational model
applicable across various tasks. They addressed a criticism that LLMs were not
so generalizable because the training data was biased or non-representative.
They addressed this criticism by training on 77M samples and adding a Multi-
Task Regression component to the pretraining. ChemBERTa-2 matched state-of-
the-art architectures on MoleculeNet.57 As with ChemBERTa, the work was
valuable because of additional exploration, in this case into how pretraining
improvements affected certain downstream tasks more than others, depending on
the type of fine-tuning task, the structural features of the molecules in the
fine-tuning task data set, or the size of that fine-tuning dataset. The result
was that pretraining the encoder-only model is important, but gains could be
made by considering the chemical application itself, and the associated fine-
tuning dataset.

In June 2023, Yuksel et al. 149 introduced SELFormer, building on ideas from
ChemBERTa2150 and using SELFIES for large data input. Yuksel et al. 149 argue
that SMILES strings have validity and robustness issues, hindering effective
chemical interpretation of the data, although this perspective is not
universally held.235 SELFormer uses SELFIES and is pretrained on two million
drug-like compounds, fine-tuned for diverse molecular property prediction
tasks (BBBP, SIDER, Tox21, HIV, BACE, FreeSolv, ESOL, PDBbind from
MoleculeNet).57 SELFormer outperformed all competing methods for some tasks
and produced comparable results for the rest. It could also discriminate
molecules with different structural properties. The paper suggests future
directions in multimodal models combining structural data with other types of
molecular information, including text-based annotations. We will discuss such
multimodal models below.

Also in 2023, Yu et al. 194 published SolvBERT, a multi-task BERT-based
regression model that could predict both solvation free energy and solubility
from the SMILES notations of solute-solvent complexes. It was trained on the
CombiSolv-QM dataset,236 a curation of experimental solvent free energy data
called CombiSolv-Exp-8780,237, 238, 239, 240 and the solubility dataset from
Boobier et al. 195. SolvBERT’s performance was benchmarked against advanced
graph-based models241, 242 This work is powerful because there is an
expectation that solvation free energy depends on 3-dimensional conformational
properties of the molecules, or at least 2D properties that would be well
characterized by graph-based molecular representations. It shows an
overachieving utility of using SMILES strings in property prediction, and
aligns with other work by Winter et al. 243, regarding activity coefficients.
SolvBERT showed comparable performance to DMPNN in predicting solvation free
energy, largely due to its effective clustering feature in the pretraining
phase as shown by TMAP visualizations. T Furthermore, SolvBERT outperformed
GROVER in predicting experimentally evaluated solubility data for new solute-
solvent combinations. This underscores the significance of SolvBERT’s ability
to capture the dynamic and spatial complexities of solvation interactions in a
text-based model.

Despite significant progress, Jiang et al. 244 highlighted the limitations of
labeled data in 2024 and introduced INTransformer,244 a transformer-based
method for predicting molecule properties. INTransformer enhances the capture
of global molecular information by adding perturbing noise and employing
contrastive learning to artificially augment smaller datasets. This method
achieved higher performance, and ongoing work continues to explore various
transformer strategies for smaller datasets.

###  3.3 Property Directed Inverse Design and Decoder-only mol-LLMs

Table 2:  Decoder-only scientific LLMs. $a$:“Model Size” is reported as the number of parameters. “PubMed” refer to the PubMed abstracts dataset, while PMC (PubMed Corpus) refers to the full-text corpus dataset. $b$: The total number of parameters was not reported. LLM |  Model Sizea |  Training Data |  Architecture |  Application |  Release date
---|---|---|---|---|---
Tx-LLM245 |  $b$ |  TDC datasets |  PaLM-2 |  Property prediction and retrosynthesis |  2024.06
LlasMol246 |  $\sim$ 7B |  SMolInstruct |  Galactica, LLaMa, Mistral |  Property prediction, molecule captioning, molecule generation, retrosynthesis, name conversion |  2024.04
BioMedLM118 |  2.7B |  PubMed abstracts and full articles |  GPT |  QA |  2024.03
BioMistral119 |  7B |  PubMed Central (PMC) |  Mistral |  QA |  2024.02
BiMediX120 |  8x7B |  1.3M Arabic-English instructions (BiMed) |  Mixtral |  QA |  2024.02
EpilepsyLLM121 |  7B |  Data from the Japan Epilepsy Association, Epilepsy Information Center, and Tenkan Net |  LLaMa |  QA |  2024.01
ChemSpaceAL247 |  $b$ |  ChEMBL 33, GuacaMol v1, MOSES, and BindingDB 08-2023 |  GPT |  Molecule Generation |  2024.01
CheXagent122 |  7B |  28 publicly available datasets, including PMC, MIMIC, wikipedia, PadChest, and BIMCV-COVID-19 |  Mistral |  QA, Image understanding |  2024.01
ChemGPT248 |  $\sim$ 1B |  10M molecules from PubChem |  GPT-neo |  Molecule generation |  2023.11
BioMedGPT-LM123 |  7B and 10B |  5.5M biomedical papers from S2ORC |  LLaMA2 |  QA |  2023.08
Darwin124 |  7B |  SciQ and Web of Science |  LLaMA |  QA, Property prediction, NER, and Molecule Generation |  2023.08
cMolGPT46 |  $b$ |  MOSES |  GPT |  Molecule Generation |  2023.04
PMC-LLaMA125 |  7B and 13B |  MedC-k and MedC-I |  LLaMA |  QA |  2023.04
Regression Transformer145 |  $\sim$27M |  ChEMBL, MoleculeNet, USPTO, etc |  XLNet |  Property prediction, Molecule tuning, Molecule generation |  2023.04
GPTChem249 |  175B |  Curation of multiple classification and regression benchmarks |  GPT-3 |  Property prediction and inverse design |  2023.02
Galactica126 |  125M, 1.3B, 6.7B, 30B, 120B |  The galactica corpus, a curation with 62B scientific documents |  Decoder-only |  QA, NER, Document Summarization, Property Prediction |  2022.11
BioGPT127 |  355M |  15M of Title and abstract from PubMed |  GPT-2 |  QA, NER, and Document Classification |  2022-09
SPT243 |  6.5M |  Synthetic data generated with the thermodynamic model COSMO-RS |  GPT-3 |  Property prediction |  2022.09
MolGPT250 |  6M |  MOSES and GuacaMol |  GPT |  Molecule Generation |  2021.10
Adilov2021251 |  13.4M |  5M SMILES from ChemBERTa’s PubChem-10M. |  GPT-2 |  Property prediction and molecule generation |  2021.09

Decoder-only GPT-like architectures offer significant value for property
directed molecule generation and de novo chemistry applications because they
excel in generating novel molecular structures by learning from vast datasets
of chemical compounds, and capture intricate patterns and relationships within
molecular sequences. They thus propose viable new compounds that adhere to
desired chemical properties and constraints, enabling rapid exploration and
innovation within an almost infinite chemical space. In addition, such large
general-purpose models have been proven to enable specific applications by
fine-tuning with a small amount of scientific data.249, 252 We first describe
property-directed inverse design from a chemistry perspective and then provide
examples of how decoder-only LLMs have propelled inverse design forward.

####  3.3.1 Property Directed Inverse Design

Nature has been a significant source of molecules that inhibit disease
proliferation, as organisms have evolved chemicals to protect themselves. This
has led to most pharmaceuticals being derived from natural products,253, 254
which offer advantages such as cell permeability, target specificity, and a
vast chemical diversity.255 However, despite these benefits, there are high
costs and complexities associated with high-throughput screening and synthesis
of natural products.255, 253 While building on the chemical diversity and
efficacy of natural products, AI also opens new avenues for synthesizing
unique compounds efficiently. Advanced in-silico molecular design allows for
rapid mutation256 towards valid de-novo molecular structures that are
synthesizable,222, 257 streamlining the iterative process of drug development.
Yet, the true innovation lies in the potential of LLM-driven target-directed
molecular design or “Inverse Design,” where we can start with a desired
property and directly generate molecules that manifest this attribute,
bypassing traditional stepwise modifications.258 This capability dramatically
accelerates the pathway from concept to viable therapeutic agents and aligns
well with decoder-only LLM architectures.

####  3.3.2 Decoder-only mol-LLMs

In 2021, Adilov 251 presented “Generative pretraining from Molecules,” one of
the first applications of decoder-only models to SMILES strings. It pretrained
a GPT-2-like causal transformer for self-supervised learning, introducing
“adapters” between attention blocks for task-specific fine-tuning.259 This
method, requiring minimal architectural changes, offered versatility in
molecule generation and property prediction, aiming to surpass ChemBERTa’s
encoder-only performance with a more scalable and resource-efficient approach.
Another early decoder-only SMILES-based model was Bagal et al’s MolGPT
model,250. MolGPT, with a mere 6 million parameters, advanced GPT-type LLMs
for molecular generation. Its decoder-only architecture with masked self-
attention facilitated learning long-range dependencies, enabling chemically
valid SMILES representations that met complex structural rules involving
valency and ring closures. The paper also used salience measures for
interpretability in predicting SMILES tokens. MolGPT outperformed many
existing Variational Auto-Encoder (VAE) based approaches260, 261, 262, 263,
264, 265, 266, 267 in predicting novel molecules with specified properties,
being trained on datasets like MOSES268 and GuacaMol.269 It showed good
performance with metrics like validity, uniqueness, Frechet ChemNet Distance
(FCD),270 and KL divergence,269. While MolGPT’s computational demands might be
higher than traditional VAEs, its ability to generate high-quality, novel
molecules justifies this trade-off, but future research could likely optimize
model efficiency or explore lighter versions. A brief summary of advancements
in transformer-based models for de-novo molecule generation from 2023 and 2024
follows.

Haroon et al. 271 further developed a GPT-based model with relative attention
for de novo drug design, showing improved validity, uniqueness, and novelty.
Frey et al. 248 introduced ChemGPT and explored the challenges of
hyperparameter tuning at scale in new domains, along with a consideration of
how the scale of the pretraining dataset will influence neural architecture
selection, relative to ”typical" chemical generative models. Both Wang et al.
272 and Mao et al. 273 presented work that surpassed MolGPT, with Mao et al.’s
T5-type model274 generating novel compounds using IUPAC names directly.275
Although T5-based, we include this here because of its relevance to de novo
drug design. In a similar vein, Zhang et al. 276 proposed including target 3D
structural information in molecular generative models, even though their
approach is not LLM-based. However, we nonetheless note its value for future
structure-based LLM drug design. They demonstrated that integrating additional
biological data significantly enhances the relevance and specificity of
generated molecules for targeted drug discovery. Wang et al. 277 discussed
PETrans, a deep learning method for generating target-specific ligands using
protein-specific encoding and transfer learning. This study emphasized the
application of transformer models for generating molecules with a high binding
affinity to specific protein targets.

In 2024, Yoshikai et al. 278 discussed the limitations of transformer
architectures in recognizing chirality from SMILES representations,
highlighting the challenges in learning overall molecular structures with, in
particular, chirality impacting the prediction accuracy of molecular
properties. They coupled a transformer with a VAE to address this. Qian et al.
279 introduced CONSMI, motivated by contrastive learning in NLP, to generate
new molecules using multiple SMILES representations, improving molecular
novelty and validity. Kyro et al. 247 presented ChemSpaceAL, an active
learning method for protein-specific molecular generation, efficiently
discovering molecules with desired characteristics without prior knowledge of
existing inhibitors. Yan et al. 280 proposed the GMIA framework, featuring a
graph mutual interaction attention decoder for drug-drug interaction
prediction, enhancing prediction accuracy and interpretability. Lastly, Shen
et al. 281 reported on AutoMolDesigner, an AI-based open-source software for
automated design of small-molecule antibiotics.

For a deeper dive into decoder-only transformer architecture in chemistry, we
highlight the May 2023 “Taiga” model by Mazuz et al. 92, and cMolGPT by Wang
et al. 46. Taiga first learns to map SMILES strings to a vector space, and
then refines that space using a smaller dataset of labeled molecules to
produce molecules with targeted attributes. It employs an autoregressive
mechanism, sequentially predicting each SMILES character based on the previous
ones. For optimizing molecular properties, Taiga uses the REINFORCE
algorithm97, which develops a strategy to enhance desired molecular features.
Although this fine-tuning with reinforcement learning (RL) slightly reduces
molecule validity, it significantly boosts their practical applicability.
Initially evaluated using the Quantitative Estimate of Drug-likeness (QED)
metric,282 Taiga has also demonstrated promising results in targeting IC50
values,179 the BACE protein,283 and anti-cancer activities they collected from
a variety of sources. This work underscores the importance of using new models
to address applications that require a higher level of chemical
sophistication, to illustrate how such models could ultimately be applied
outside of the available benchmark datasets. It also builds on the necessary
use of standardized datasets and train-validation-test splitting, to
demonstrate progress, as explained by Wu et al. 57. Yet, even the MoleculeNet
benchmarks57 are flawed, and we point the reader here to a more detailed
discussion on benchmarking,186 given that a significant portion of molecules
in the BACE dataset have undefined stereo centers, which, at a deeper level,
complicates the modeling and prediction accuracy.

Wang et al. 46 introduced “cMolGPT: A Conditional Generative Pre-Trained
Transformer for Target-Specific de novo Molecular generation,” which
underscores the importance of incorporating chemical domain knowledge to
effectively navigate the vast landscape of drug-like molecules. Using
unsupervised learning and an auto-regressive approach, cMolGPT generates
SMILES guided by predefined conditions based on target proteins and binding
molecules. Initially trained on the MOSES dataset268 without target
information, the model is fine-tuned with embeddings of protein-binder pairs,
focusing on generating compound libraries and target-specific molecules for
the EGFR, HTR1A, and S1PR1 protein datasets.284, 285, 286, 287

Their approach employs a QSAR model5 to predict the activity of generated
compounds, achieving a Pearson correlation coefficient over 0.75. However,
reliance on this QSAR model, which has its own limitations, underscores the
necessity for more extensive experimental datasets. Despite its capabilities,
the cMolGPT model tends to generate molecules within the confines of the same
sub-chemical space as the original dataset, indicating that while it
successfully identifies potential binders, it faces challenges in broadly
exploring the chemical space to discover novel solutions. This limitation
highlights the difficulty of generating molecules from significantly different
chemical families that are effective in binding or possess a combination of
desired properties, suggesting a need for approaches that can more effectively
expand the exploration of chemical diversity. Regardless, models capable of
this kind of conditional generation are typically trained on large datasets of
known protein-ligand interactions. Through this training, they learn the
complex relationship between protein structures and the chemical structures of
binding ligands. The model learns to predict which molecular features or
structures are important for interacting with proteins that have certain
characteristics (as captured in the embeddings). While both “Taiga” and
“cMolGPT” use transformer architectures and decoder-only setups for molecule
generation, they differ in approach. Taiga integrates reinforcement learning
to optimize its generative model, while cMolGPT uses target-specific
embeddings to guide the generation process. Yu et al. 246 follow a different
approach. LlaSMol246 starts from pretrained models (for instance Galactica,
LlaMa2, and Mistral) and performs parameter efficient fine-tuning (PEFT)288,
289. Specifically, they used LoRa290. They show that PEFT has the potential to
achieve state-of-the-art performance in property prediction when fine-tuning
on MoleculeNet57.

Outside of training on SMILES strings, Mao et al. 275 introduce iupacGPT,
which also uses a decoder-only architecture, modeled after GPT-2. It addresses
the limitations of SMILES strings by using IUPAC names to integrate human-
readable chemical semantics. It focuses on pretraining with a vast IUPAC
dataset and fine-tuning with lightweight networks, excelling in molecule
generation, classification, and regression tasks.

We also refer the reader to several other good reviews, 291, 292, 293, 175
with Goel et al. 294 exploring the efficiency of modern machine learning
methods in sampling drug-like chemical space for virtual screening and
molecular design. The work of Goel et al. 294 discussed generative models,
including LLM approaches, for approximating the entire drug-like chemical
space and highlighted models conditioned on specific properties or receptor
structures.

We provide a segue from this section by introducing the work by Jablonka et
al. 249, which showcases a decoder-only GPT model that, despite its training
on natural language rather than specialized chemical languages, competes
effectively with decoder-only LLMs tailored to chemical languages. The authors
use finetuned GPT-3 to answer complex chemistry questions and therefore
highlight its potential as a foundational tool in the field. This sets the
stage for integrating a natural language decoder-only LLM, like ChatGPT, into
chemical research, where it could serve as a central hub. This foreshadows
future developments to pair the LLM with specialized tools to enhance
capabilities, paving the way for the creation of autonomous agents that
leverage deep language understanding in scientific domains.

###  3.4 Synthesis Prediction and Encoder-decoder Mol-LLMs

Table 3:  Encoder-decoder scientific LLMs. $a$:“Model Size” is reported as the number of parameters. $b$: The total number of parameters was not reported. LLM |  Model Sizea |  Training Data |  Architecture |  Application |  Release date
---|---|---|---|---|---
MOLGEN295 |  $b$ |  ZINC15 |  BART |  Molecule Generation |  2024.03
BioT5+129 |  252M |  ZINC20, UniRef50, 33M PubMed articles, 339K mol-text pairs from PubChem, 569K FASTA-text pairs from Swiss-prot |  T5 |  Molecule Captioning, Molecule Generation, Property Prediction, |  2024-02
BioT5144 |  252M |  ZINC20, UniRef50, full-articles from BioRxiv and PubMed, mol-text-IUPAC information from PubChem |  T5 |  Molecule Captioning, Property Prediction |  2023-12
nach0128 |  250M |  MoleculeNet, USPTO, ZINC |  T5 |  Property prediction, Molecule generation, Question answering, NER |  2023.11
ReactionT5147 |  220M |  ZINC and ORD |  T5 |  Property prediction and Reaction prediction |  2023.11
Text+Chem T5130 |  60M, 220M |  11.5M or 33.5M samples curated from Vaucher et al. 296, Toniato et al. 225, and CheBI-20 |  T5 |  Molecule Captioning, Product Prediction, Retrosynthesis, Molecule Generation |  2023.01
MolT5131 |  60M, 770M |  C4 dataset |  T5 |  Molecule Captioning and Molecule Generation |  2022.11
T5Chem148 |  220M |  USPTO |  T5 |  Product Prediction, Retrosynthesis, Property Prediction |  2022.03
ChemFormer146 |  45M, 230M |  100M SMILES from ZINC-15 |  BART |  Product Prediction, Property Prediction, Molecular Generation |  2022.01
Text2Mol297 |  $b$ |  CheBI-20 |  SciBERT w/ decoder |  Molecule captioning and conditional molecule generation |  2021.
SMILES transformer156 |  $b$ |  ChEMBL24 |  Transformer |  Property prediction |  2019.11
Molecular Transformer233 |  12M |  USPTO |  Transformer |  Product prediction |  2019.08

The encoder-decoder architecture is designed for tasks involving the
translation of one sequence into another, making it ideal for predicting
chemical reaction outcomes or generating synthesis pathways from given
reactants. We begin with a background on optimal synthesis prediction and
describe how earlier machine learning has approached this challenge. Following
that, we explain how LLMs have enhanced chemical synthesis prediction and
optimization. Although, our context below is aptly chosen to be synthesis
prediction, other applications exist. For example, SMILES Transformer (ST)156
is worth a mention, historically, because it explored the benefits of
unsupervised pretraining to produce continuous, data-driven molecular
fingerprints from large SMILES-based datasets.

####  3.4.1 Synthesis Prediction

Once a molecule has been identified through property-directed inverse design,
the next challenge is to predict its optimal synthesis, including yield.
Shenvi 298 describe how the demanding and elegant syntheses of natural
products has contributed greatly to organic chemistry. However, in the past 20
years, the focus has shifted away from complex natural product synthesis
towards developing reactions applicable for a broader range of compounds,
especially in reaction catalysis.298 Yet, complex synthesis is becoming
relevant again as it can be digitally encoded, mined by LLMs,299 and applied
to new challenges. Unlike property prediction, reaction prediction is
challenging due to the involvement of multiple molecules. Modifying one
reactant requires adjusting all others, with different synthesis mechanisms or
conditions likely involved. Higher-level challenges exist for catalytic
reactions and complex natural product synthesis. Synthesis can be approached
in two ways. Forward synthesis involves building complex target molecules from
simple, readily available substances, planning the steps progressively.
Retrosynthesis, introduced by E.J. Corey in 1988,300 is more common. It
involves working backward from the target molecule, breaking it into smaller
fragments whose re-connection is most effective. Chemists choose small,
inexpensive, and readily available starting materials to achieve the greatest
yield and cost-effectiveness. For example, the first total synthesis of
discodermolide301 involved 36 such steps, a 24-step longest linear sequence,
and a 3.2% yield. There are many possible combinations for the total synthesis
of a target molecule, and the synthetic chemist must choose the most sensible
approach based on their expertise and knowledge. However, this approach to
total synthesis takes many years. LLMs can now transform synthesis such that
structure-activity relationship predictions can be coupled in lock-step with
molecule selection based on easier synthetic routes. This third challenge of
predicting the optimal synthesis can also lead to the creation of innovative,
non-natural compounds, chosen because of such an easier predicted sytnthesis
but for which the properties are still predicted to meet the needs of the
application. Thus, these three challenges introduced above are interconnected.

####  3.4.2 Encoder-decoder mol-LLMs

Before we focus on transformer use, some description is provided on the
evolution from RNN and Gated Recurrent Unit (GRU) approaches in concert with
the move from template-based to semi-template-based to template-free models.
Nam and Kim 302 pioneered forward synthesis prediction using a GRU-based
translation model, while Liu et al. 303 reported retro-synthesis prediction
with a Long Short-Term Memory (LSTM) based seq2seq model incorporating an
attention mechanism, achieving 37.4% accuracy on the USPTO-50K dataset.304 The
reported accuracies of these early models highlighted the challenges of
synthesis prediction, particularly retrosynthesis. Schneider et al. 304
further advanced retrosynthesis by assigning reaction roles to reagents and
reactants based on the product.

Evolving from RNNs and GRUs, the field progressed with the introduction of
template-based models. As a parallel to the separate development of the
Chematica305, 306 synthesis mapping tool, Segler and Waller 307 identified
that computational “rule-based systems” often failed because they ignored the
molecular context, resulting in “reactivity conflicts.” They prioritized
suitable transformation rules describing how atoms and bonds change during
reactions, applied in reverse for retrosynthesis. They trained a model on 3.5
million reactions, achieving 95% top-10 accuracy in retrosynthesis and 97% for
reaction prediction on a validation set of nearly 1 million reactions from the
Reaxys database (1771-2015). Though not transformer-based, their work paved
the way for LLMs in synthesis applications. However, template-based models
rely on explicit reaction templates from known reactions, which limits their
ability to predict novel reactions and they require manual updates for
learning new data.

Semi-template-based models offered a balance between rigid template-based
methods and flexible template-free approaches. They used interpolation or
extrapolation within template-defined spaces to predict a wider range of
reactions and to adjust based on new data. In 2021, Somnath et al. 308
introduced a graph-based approach recognizing that precursor molecule topology
is largely unchanged during reactions. Their model broke the product molecule
into “synthons” and added relevant leaving groups, making results more
interpretable.309 Training on the USPTO-50k dataset, 304 they achieved a top-1
accuracy of 53.7%, outperforming previous methods.

It is, however, the template-free approaches that align well with transformer-
based learning approaches because they learn retrosynthetic rules from raw
training data. This provides significant flexibility and generalizability
across various types of chemistry. Template-free models are not constrained by
template libraries and so can uncover novel synthetic routes that are
undocumented or not obvious from existing reaction templates. To pave the way
for transformer use in synthesis, Cadeddu et al. 310 drew an analogy between
fragments in a compound and words in a sentence due to their similar rank
distributions. Schwaller et al. 311 further advanced this with an LSTM network
augmented by an attention-mechanism-based encoder-decoder architecture, using
the USPTO dataset.304 They introduced a commonly used “regular expression” (or
“regex”) for tokenizing molecules, framing synthesis (or retrosynthesis)
predictions as translation problems with a data-driven, template-free
sequence-to-sequence model. They tracked which starting materials were actual
reactants, distinguishing them from other reagents like solvents or catalysts,
and used the regex to uniquely tokenize recurring reagents, as their atoms
were not mapped to products in the core reaction.

In 2019, going beyond the ”neural machine" work of Nam and Kim 302, Schwaller
et al. 233 first applied a transformer for synthesis prediction, framing the
task as translating reactants and reagents into the final product. Their model
inferred correlations between chemical motifs in reactants, reagents, and
products in the dataset (USPTO-MIT,312 USPTO-LEF, 313 USPTO-STEREO311). It
required no handcrafted rules and accurately predicted subtle chemical
transformations, outperforming all prior algorithms on a common benchmark
dataset. The model handled inputs without a reactant-reagent split, following
their previous work,311 and accounted for stereochemistry, making it valuable
for universal application. Then, in 2020, for automated retrosynthesis,
Schwaller et al. 314 developed an advanced Molecular Transformer model with a
hyper-graph exploration strategy. The model set a standard for predicting
reactants and other entities, evaluated with four new metrics: coverage, class
diversity, round-trip accuracy, and Jensen–Shannon divergence. Constructed
dynamically, the hypergraph allowed for efficient expansion based on Bayesian-
like probability scores, showing high performance despite training data
limitations. Notably, accuracy was improved after the re-synthesis of the
target product from the newly generated molecular precursors was included in
the model. This round-trip accuracy concept was also used by Chen and Jung 315
and Westerlund et al. 316. Also in 2020, Zheng et al. 317 developed a
“template-free self-corrected retrosynthesis predictor” (SCROP) using
transformer networks and a neural network-based syntax corrector, achieving
59.0% accuracy on a benchmark dataset.318, 304 This approach outperformed
other deep learning methods by over 2% and template-based methods by over 6%.

We now highlight advancements in synthesis prediction using the BART Encoder-
Decoder architecture, starting with Chemformer by Irwin et al. 146. This paper
emphasized the computational expense of training transformers on SMILES and
the importance of pretraining for efficiency. It showed that models pretrained
on task-specific datasets or using only the encoder stack were limited for
sequence-to-sequence tasks. After transfer learning, Chemformer achieved
state-of-the-art results in both sequence-to-sequence synthesis tasks and
discriminative tasks, such as optimizing molecular structures for specific
properties. They studied the effects of small changes on molecular properties
using pairs of molecules from the ChEMBL database179 with a single structural
modification. Chemformer’s performance was tested on the ESOL, Lipophilicity,
and Free Solvation datasets.57 Irwin et al. 146 also described their use of an
in-house property prediction model, but when models train on calculated data
for ease of access and uniformity, they abstract away from real-world chemical
properties. We again emphasize the importance of incorporating experimentally
derived data into Chemistry LLM research to create more robust and relevant
models. Continuously curating new, relevant datasets that better represent
real-world chemical complexities will enhance the applicability and
transferability of these models.

In 2023, Toniato et al. 225 also applied LLMs to single-step retrosynthesis as
a translation problem, but increased retrosynthesis prediction diversity by
adding classification tokens, or “prompt tokens,” to the target molecule’s
language representation, guiding the model towards different disconnection
strategies. Increased prediction diversity has high value by providing out-of-
the-box synthetic strategies to complement the human chemist’s work. To
measure retrosynthesis accuracy, Li et al. 319 introduced Retro-BLEU, a metric
adapted from the BLEU score used in machine translation.320 Despite progress
in computer-assisted synthesis planning (CASP), not all generated routes are
chemically feasible due to steps like protection and deprotection needed for
product formation. Widely accepted NLP metrics like BLEU320 and ROUGE321 focus
on precision and recall by computing n-gram overlaps between generated and
reference texts. Similarly, in retrosynthesis, reactant-product pairs can be
treated as overlapping bigrams. Retro-BLEU uses a modified BLEU score,
emphasizing precision over recall, as there is no absolute best route for
retrosynthesis. Although not yet applied to LLM-based predictions, this
approach has value by allowing future performance comparison with a single
standard.

Finally, by expanding the use of encoder-decoder architectures outside
synthesis prediction into molecular generation, Fang et al. 295 introduced
MOLGEN, a BART-based pretrained molecular language model, in a 2023 preprint
updated in 2024. MOLGEN addressed three key challenges: generating valid
SMILES strings, avoiding an observed bias that existed against natural
product-like molecules, and preventing hallucinations of molecules that didn’t
retain the intended properties. Pretrained on 100 million molecules using
SELFIES173 and a masked language model approach, MOLGEN predicts missing
tokens to internalize chemical grammar. An additional highlight of this work
is how MOLGEN uses “domain-agnostic molecular prefix tuning.” This technique
integrates domain knowledge directly into the model’s attention mechanisms by
adding molecule-specific prefixes, trained simultaneously with the main model
across various molecular domains. The model’s parameters would thus be
adjusted to better capture the complexities and diversities of molecular
structures, and domain-specific insights would be seamlessly integrated. To
prevent molecular hallucinations, MOLGEN employs a chemical feedback
mechanism, to autonomously evaluate generated molecules for appropriate
properties, to guide learning and optimization. Such feedback foreshadows a
core aspect of autonomous agents, which is their capacity for reflection. We
will explore this further below.

###  3.5 Multi-Modal LLMs

Multitask or multimodal methods applied to chemistry are now described. In
2021, Edwards et al. 297 proposed Text2Mol, which retrieved molecules using
natural language descriptions as queries. This task required integrating both
molecular and natural language modalities, making it a cross-lingual retrieval
problem due to the distinct grammar of molecular representations. The
researchers built a paired dataset of molecules with corresponding text
descriptions, and developed an aligned semantic embedding space for retrieval.
This was enhanced with a cross-modal attention-based model for explainability
and reranking. One stated aim was to improve retrieval metrics, which would
further advance the ability for machines to learn from chemical literature. In
their follow-up work in 2022, Edwards et al. 131 continued using SMILES string
representations along with text descriptions. The two key applications were
first, to generate captions for molecules from SMILES strings and second, to
output molecules based on textual descriptions of desired properties. With
this in mind, several important challenges remain. First, molecules can have
many possible descriptions based on their diverse properties. For example,
aspirin can be described by its pain-relieving effects, its use in preventing
heart attacks, its chemical structure (an ester and a carboxylic acid
connected to a benzene ring in ortho geometry), or its degradation into
salicylic acid and ethanoic acid in moist conditions 322. Describing such a
molecule therefore requires expertise in different chemistry domains,
depending on its functions and properties. In contrast, image captioning for
common objects like cats and dogs needs much less domain expertise. Thus,
obtaining a large, paired dataset of chemical representations and textual
descriptions is challenging. Another issue is that standard evaluation
metrics, like BLEU, are inadequate for molecule-language tasks. To address
this, Edwards et al. 131 developed MolT5, which pretrained a model on a large
corpus of unlabeled natural language text and molecule strings using a
denoising objective. They fine-tuned the model on gold standard annotations
and improved the metric based on their prior Text2Mol work 297. MolT5
effectively generated both molecule and caption outputs.

In other work, Seidl et al. 323 developed CLAMP. Trained on large biochemical
datasets, CLAMP adapts to new tasks using separate modules for chemical and
language inputs, and predicts biochemical activity for drug discovery. Xu et
al. 324 presented BioTranslator, a method for translating user-written text
descriptions into non-text biological data, facilitating the discovery of
novel cell types and applications in protein function prediction and drug
target identification. This method enabled data exploration beyond controlled
vocabularies using free text. ChatDrug, by Liu et al. 325, integrates
multimodal capabilities through a prompt module, a retrieval and domain
feedback module, and a conversation module for systematic drug editing. It
identifies and manipulates molecular structures for better interpretability in
pharmaceutical research. Christofidellis et al. 130 introduced a multi-domain,
multi-task language model capable of handling tasks across chemical and
natural language domains without specific pretraining for each. Shoghi et al.
326 describe Joint Multi-domain Pre-training (JMP), based on the hypothesis
that pre-training on a diverse set of chemical domains should lead to better
generalization towards a foundational model. The pre-training task is framed
as a multi-task supervised learning problem, where each label of each pre-
training dataset is treated as a separate task. Liu et al. 327 developed
MolXPT, a unified model that integrates text and molecules for enhanced
molecular property prediction and translation, demonstrating robust zero-shot
molecular generation capabilities. Zhang et al. 328 introduced PremuNet, a
pretrained multi-representation fusion network that enhances molecular
property prediction by integrating various molecular data representations. Liu
et al. 329 developed GIT-Mol, integrating graph, image, and text data,
significantly improving molecular property prediction and molecule generation
validity. Gao et al. 330 advanced targeted molecule generation with DockingGA,
combining transformer neural networks with genetic algorithms and docking
simulations for optimal molecule generation, utilizing Self-referencing
Chemical Structure Strings to represent and optimize molecules. Zhou et al.
331 developed TSMMG, a teacher-student LLM for multi-constraint molecular
generation, learning from a large set of text-molecule pairs and generating
molecules that meet complex property requirements. Zhu et al. 332 proposed
3M-Diffusion, a multi-modal molecular graph generation method that uses
natural language descriptions to generate diverse, novel molecular graphs.
Gong et al. 333 introduced TGM-DLM, a diffusion model for text-guided molecule
generation that overcomes limitations of autoregressive models in generating
precise molecules from textual descriptions. Fang et al. 334 developed MolTC,
a multimodal framework that effectively integrates graphical information of
molecules for improved prediction of molecular interactions. Soares et al. 335
describe MULTIMODAL-MOLFORMER, a multimodal language model that predicts
molecular properties by integrating chemical language representations with
physicochemical features. It uses a causal multistage feature selection method
to identify physicochemical features directly impacting target properties.
These features are combined with molecular embeddings from MOLFORMER196,
significantly enhancing prediction accuracy for complex tasks like
biodegradability and PFAS toxicity.

###  3.6 Textual scientific LLMs

LLMs are large deep neural network models. Therefore, their superior
performance against other smaller models in classical machine learning tasks
is somewhat expected. Addressing tasks that seemed impossible to tackle in the
past better illustrates LLM’s outstanding capabilities. That is, despite
accurately predicting properties given well-structured information such as
molecular features and descriptors, LLMs can access less structured data and
extract information from simple natural language. For instance, instead of
developing a restricted representation, Huang et al. 138 used clinical
annotations from the MIMIC-III336 dataset to predict patient readmission.
ClinicalBERT implements a pretrained BERT architecture using a combination of
masked language modeling and next-sentence prediction training schemes
followed by supervised fine-tuning on readmission prediction. Or, instead of
pretraining, Zhao et al. 121 fine-tuned LLaMA on epilepsy data using an
instruction-following approach to create EpilepsyLLM. Similarly, SciBERT134
and ScholarBERT137 adapted BERT to scientific literature. Beltagy et al. 134
created a tokenizer specific for scientific texts extracted from scientific
articles from semantic scholar180 and showed that SciBERT outperforms fine-
tuned BERT-base105 in every task investigated. It shows that vocabulary
quality plays an important role in the model’s performance. A few years later,
Hong et al. 137 pretrained BERT using RoBERTa337 optimizations to improve
pretraining performance. ScholarBERT was pretrained on scientific articles
from Public.Resource.Org, Inc and further fine-tuned on the tasks used for
evaluation. Despite using a much larger dataset, ScholarBERT did not
outperform other LLMs trained with narrower domain datasets. Even against
SciBERT, another LLM trained on multi-domain articles, ScholarBERT could not
perform relatively better in tasks from biomedical, computer science, and
material domains. However, ScholarBERT showed significant improvement in the
ScienceExamCER338 dataset, which contains semantic class annotations in a
training corpus of 3rd to 9th grade science exam questions for named entity
recognition (NER) tasks.

On the other hand, instead of pretraining LLMs on a large domain scope, models
focused on only one field were also developed. Guo et al. 141 argue that
manually curating structured datasets is a sub-optimal, time-consuming, and
labor-intensive task. Therefore, they automated data extraction and annotation
from scientific papers using ChemDataExtractor339 and their in-house
annotation tool340. ChemBERT141 was pretrained using a BERT model to encode
chemical reaction information, followed by fine-tuning a NER head. ChemBERT
outperformed other models such as BERT105 and BioBERT143 in the product
extraction task, presenting an improvement of $\sim$ 6% in precision. For the
product role labeling task – given the extracted compound, which role does it
play in the reaction? –, it increases precision in $\sim$ 5% when compared to
the same models. The increase in performance might be due to training in a
narrower domain, which allows the model to learn specific traits from the data
instead of being more generalist. MatSciBERT132 and MaterialsBERT139 followed
a similar strategy, and BatteryBERT,197 similarly outperformed original BERT
models with application to specific battery tasks. Gupta et al. 132 developed
MatSciBERT by fine-tuning SciBERT134 on the Material Science Corpus (MSC),
their curated dataset focused on materials extracted from Elsevier’s
scientific papers. MatSciBERT shows an accuracy improvement of $\sim$ 3%
compared to SciBERT on classifying an article subject based on its abstract.
Differently, MaterialsBERT139 fine-tuned PubMedBERT133 on 2.4M abstracts
extracted from material science articles using a masked training approach.
MaterialsBERT was then used to generate a contextual encoded representation of
the abstract to train a named entity relationship (NER) model. Reporting an
increase in precision on their test dataset of $\sim$ 1%, $\sim$ 2%, and
$\sim$ 10% for PubMedBERT, MatBERT, and ChemBERT, respectively.

Considerable effort was also employed to develop such models focused on
biology tasks. They follow a similar trend of training an LLM model on a large
training corpus like Wikipedia, scientific data extracted from a scientific
database, and textbooks, followed by fine-tuning specific downstream tasks.
Shin et al. 142 pretrained different Megatron-LM341, another BERT-like LLM,
models with different sizes, originating the BioMegatron family of models.
Considering models with 345M, 800M, and 1.2B parameters and vocabularies with
30k and 50k tokens, they compared the influence of model size on its
performance. These models were pretrained using abstracts from the PubMed
dataset and full-text scientific articles from PubMed Central (PMC), similar
to BioBERT143. The largest 1.2B training started from a BERT-uncased
checkpoint to save time. Surprisingly, the largest model did not perform
better than the smaller ones. In fact, the 345M parameters using the 50k
tokens vocabulary consistently outperformed other models in NER and relation
extraction (RE) tasks. Interestingly, the model size seems relevant for the
SQuAD342 dataset, showing that LLMs trained on small, domain-specific datasets
may lack generalization abilities. BioBERT143 used data from Wikipedia,
textbooks, PubMed abstracts, and PMC full-text corpus to pretrain a BERT
architecture. Outperforming the original BERT in all benchmarks tested and
even SOTA models in some benchmarks (NCBI disease, 2010 i2b2/VA, BC5CDR,
BC4CHEMD, BC2GM, JNLPBA, LINNAEUS, Species-800). Peng et al. 343 developed a
multi-task BERT evaluated on the Biomedical Language Understanding Evaluation
(BLUE)135 benchmark called BlueBERT. The model was pretrained on PubMed
abstracts and MIMIC-III336 and fine-tuned on BLUE tasks, achieving similar
results as BioBERT in multiple benchmarks. PubMedBERT133, following the
approach adopted in SciBERT, developed a domain-specific vocabulary. They used
14M abstracts extracted from PubChem to pretrain PubMedBERT. In addition, they
curated and grouped multiple biomedical datasets to develop BLURB, their
comprehensive benchmark for biomedical NLP tasks, such as NER, sentence
similarity, document classification, and question-answering. Gu et al. 133
showed that PubMedChem considerably outperformed other LLMs in the BLURB
benchmark, mainly in PubMedQA and BioQSA datasets. The second best-evaluated
model in these datasets was BioBERT, evidencing the importance of domain-
specific knowledge to high-performance LLMs.

The models discussed previously in the section use an encoder-only LLM to
encode information from scientific data. The learned encoded representation is
then further used to make predictions by feeding it to simpler models. A
different approach involves training a decoder model. Using a decoder model
aims to address the lack of generation ability of the encoder-only models. By
implementing a decoder, the model can address a different group of generative
tasks, such as question-answering and document classification, using generated
labels instead of following a pre-defined set of possible classes. Darwin124
aims to create domain-specific LLMs for natural science by first fine-tuning
LLaMA-7B on FAIR, a general Question-Answering (QA) dataset, and then fine-
tuning it on scientific QA datasets. Scientific QA instructions were obtained
from SciQ344 and generated by the authors using the Scientific Instruction
Generation (SIG) model also developed by the authors. Their SIG model is fine-
tuned from Vicuna-7B to convert full-text scientific papers into a set of QA
pairs. This approach presented an improved performance of DARWIN in several
regression and classification benchmarks. Interestingly, Using LLaMA-7B fine-
tuned on FAIR only – with no scientific-specific QA fine-tuning – led to
similar results in six out of nine benchmarks with metrics available for both
models.

In biology, BioGPT127 pretrained a GPT-2 model architecture using 15M
abstracts from PubChem corpus. BioGPT was evaluated on four tasks across five
benchmarks: end-to-end relation extraction on BC5CDR, KD-DTI, and DDI,
question answering on PubMedQA, document classification on HoC, and text
generation on the previously listed benchmarks. fine-tuning BioGPT on
downstream tasks (except text generation) showed its superiority against other
encoder-only LLMs in the biomedical domain, such as BioBERT and PubMedBERT.
Focusing on text generation, the authors qualitatively compared BioGPT and
GPT-2 generations. They conclude that BioGPT demonstrates better text
generation. However, no metric was used, making the comparison difficult to
measure. With a similar idea, Wu et al. 125 first pretrained an LLaMA2 model
with the MedC-k dataset, consisting of 4.8M academic papers and 30K textbooks.
In the sequence, the model was aligned through instruction tuning using the
MedC-I dataset, a set of medical QA problems. PMC-LLaMA125 outperforms LLaMa-2
and ChatGPT in multiple biomedical QA benchmarks despite its size $\sim$ 10
times smaller. Interestingly, the authors show that the performance of PMC-
LLaMA on MedQA345, MedMCQA346, and PubMedQA347 benchmarks progressively
increases as more knowledge is injected, the model size is increased, and more
instructions are included in instruction tuning.

###  3.7 The use of ChatGPT in Chemistry

With the rise of ChatGPT, we review here how many researchers have wanted to
test the capability of such an accessible decoder-only LLM. Castro Nascimento
and Pimentel 348 wrote the first notable paper on ChatGPT’s impact on
Chemistry. The authors emphasize that LLMs, trained on extensive, uncurated
datasets potentially containing errors or secondary sources, may include
inaccuracies limiting their ability to predict chemical properties or trends.
The paper highlighted that while LLMs could generate seemingly valid
responses, they lacked true reasoning or comprehension abilities and would
perpetuate existing errors from their training data. However, the authors
suggested that these limitations could be addressed in the future. The work
serves as a benchmark to qualitatively assess improvements in generative
pretrained transformers. For example, five tasks were given to ChatGPT
(GPT-3). The accuracy for converting compound names to SMILES representations
and vice versa was about 27%, with issues in differentiating alkanes and
alkenes, benzene and cyclohexene, or cis and trans isomers. ChatGPT found
reasonable octanol-water partition coefficients with a 31% mean relative
error, and a 58% hit rate for coordination compounds’ structural information.
It had a 100% hit rate for polymer water solubility and a 60% hit rate for
molecular point groups. Understandably, the best accuracies were achieved with
widely recognized topics. The authors concluded that neither experimental nor
computational chemists should fear the development of LLMs, nor task
automation. Instead, they encouraged the improvement of dedicated AI tools for
specific problems, with their integration into research, as facilitators.

The sole use of ChatGPT for chemistry and any apparent advancement in
chemistry seems somewhat limited. Humphry and Fuller 349, Emenike and Emenike
350, and Fergus et al. 351 address the use of ChatGPT in chemical education.
There are also a few papers that describe the use of ChatGPT on a specific
area of chemical research, namely with the synthesis and functional
optimization of Metal Organic Framewoks (MOFs), merging computational modeling
with empirical chemistry research.352, 353 There is also some overlap with
fine-tuned GPT models,354 within this field.355 Deb et al. 356 provide a
thorough assessment of ChatGPT’s effectiveness in material science through
various tasks in computational material science, though the perspective is
subjective. ChatGPT assisted in computational materials science by providing
crystal space groups for simple compounds, suggesting inputs for simulations,
refining analysis with specific rules, and identifying resources. The authors
highlight ChatGPT’s potential to write code for enhancing processes and as a
tool for those without prior expertise, particularly in catalyst development
for CO2 capture. We highlight three points about using ChatGPT alone. First,
reliable outputs require detailed and accurate input, as Deb et al. 356
showed, where ChatGPT struggled to mine or predict crystal structures. Second,
standardized methods for reproducing and evaluating GPT work are still
unclear. Third, this work naturally leads to the inference that additional
chemical tools or agents are needed to achieve more complex thinking, as
described by Bloom’s Taxonomy.357, 358 Bloom’s taxonomy categorizes
educational goals into hierarchical levels: Remembering, Understanding,
Applying, Analyzing, Evaluating, and Creating. Remembering involves recalling
facts, Understanding means restating a concept, and Applying uses a theory in
a new situation. Analyzing breaks down a mechanism into parts to understand
connections and differences. Evaluating involves making value judgments, and
Creating entails building new structures from diverse elements. Currently,
LLMs and autonomous agents are limited in replicating higher-level thinking
compared to human understanding. To better quantify their capabilities in this
area, we therefore recommend adopting a quality metric based on Bloom’s
taxonomy. This educational framework can serve as an effective tool for
evaluating the sophistication of LLMs and autonomous agents, especially when
tackling complex chemical challenges. Bloom’s taxonomy would thereby provide a
structured approach to assess higher-order thinking and reasoning.

####  3.7.1 Automation

The evolution of artificial intelligence in chemistry has fueled the potential
for automating scientific processes. For example, in 2019, Coley et al. 359
developed a flow-based synthesis robot proposing synthetic routes and
assembling flow reaction systems, tested on medically relevant molecules, and
in 2020, Gromski et al. 360 provided a useful exploration on how chemical
robots could outperform humans when executing chemical reactions and analyses.
They developed the Chemputer, a programmable batch synthesis robot handling
reactions like peptide synthesis and Suzuki coupling. In 2021, Grisoni et al.
361 combined deep learning-based molecular generation with on-chip synthesis
and testing. The Automated Chemical Design (ACD) framework by Goldman et al.
362 provides a useful taxonomy for automation and experimental integration
levels. Thus, automation promises to enhance productivity through increased
efficiency, error reduction, and the ability to handle complex problems, as
described in several excellent reviews regarding automation in chemistry,363,
364, 365, 366, 367, 368, 369

This increased productivity may be the only possible approach to exploring the
vastness of all chemical space. To fully leverage AI in property prediction,
inverse design, and synthesis prediction, it must be integrated with automated
synthesis, purification, and testing. This automation should be high-
throughput and driven by AI-based autonomous decision-making (sometimes called
“lights-out” automation). Janet et al. 364 highlighted challenges in multi-
step reactions with intermediate purifications, quantifying uncertainty, and
the need for standardized recipe formats. They also stated the limitations of
automated decision-making. Organa370 addresses some of these challenges. It
can significantly reduce physical workload and improve users’ lab experience
by automating diverse common lab routine tasks such as solubility assessment,
pH measurement, and recrystallization. Organa interacts with the user through
text and audio. The commands are converted into a detailed LLM prompt and used
to map the goal to the robot’s instructions. Interestingly, Organa is also
capable of reasoning over the instructions, giving feedback about the
experiments, and producing a written report with the results.

Other limitations exist, like machines’ restriction to pre-defined
instructions, their inability to originate new materials, and the lower
likelihood of lucky discoveries. Yet, when dedicated tools can be connected to
address each step of an automated chemical design, these limitations can be
systematically addressed through advancements in LLMs and autonomous agents,
discussed in the next section.

##  4 LLM-based autonomous agents

The term “agent” originates in philosophy, referring to entities capable of
making decisions371. Hence, in artificial intelligence, an “agent” is a system
that can perceive its environment, make decisions, and act upon them in
response to external stimuli372. Language has enabled humans to decide and act
to make progress in response to the environment and its stimuli, and so LLMs
are naturally ideal for serving as the core of autonomous agents. Thus, in
agreement with Gao et al. 373, we define a “language agent” as a model or
program (typically based on LLMs) that receives an observation from its
environment and executes an action in this environment. Here, environment
means a set of tools and a task. Hence, “LLM-based autonomous agents” refer to
language agents whose core is based on an LLM model. Comprehensive analyses of
these agents are available in the literature374, 372, 373, but this section
highlights key aspects to prepare the reader for future discussions.

There is no agreed definition of the nomenclature to be used to discuss
agents. For instance, Gao et al. 373 created a classification scheme that aims
to group agents by their autonomy in biological research. This means a level 0
agent has no autonomy and can only be used as a tool, while a level 3 agent
can independently create hypotheses, design experiments, and reason. Another
previously suggested framework highlights agents’ construction, application,
and evaluation as a parallel between agents and neural networks. Following
this perspective, Wang et al. 374 categorizes agent components into four
modules: profiling, memory, planning, and action. In contrast, Weng 375 also
identifies four elements — memory, planning, action, and tools — but with a
different emphasis. Meanwhile, Xi et al. 372 proposes a division into three
components: brain, perception, and action, integrating profiling, memory, and
planning within the brain component, where the brain is typically an LLM.
Recently, Sumers et al. 376 proposed Cognitive Architectures for Language
Agents (CoALA), a conceptual framework to generalize and ease the design of
general-purpose cognitive language agents. In their framework, a larger
cognitive architecture composed of modules and processes is defined. CoALA
defines a memory, decision-making, and core processing module, in addition to
an action space composed of both internal and external tools. While internal
tools mainly interact with the memory to support decision-making, external
tools compose the environment, as illustrated in Figure 5. Given a task that
initiates the environment, the “decision process” runs continuously in a loop,
receiving observations and executing actions until the task is completed. For
more details, read Reference [376].

Figure 5:  Agent’s architecture as defined in this review. According to our
definition, an agent is composed of a central program (typically an LLM and
the code to implement the agent’s dynamic behavior) and the agent modules. The
agent continuously receives observations from the environment and decides
which action should be executed to complete the task given to it. Here, we
define the agent as the set of elements whose decision is trainable, that is,
the LLM, the agent code, the decision process, and the agent modules. Given a
task, the agent uses the agent modules (memory, reasoning, planning,
profiling) and the LLM to decide which action should be executed. This action
is executed by calling a tool from the environment. After the action is
executed, an observation is produced and fed back to the agent. The agent can
use perception to receive inputs in different modalities from the environment.
A) Description of agent modules, B) illustration of the agent architecture, C)
illustration of the environment components, D) description of tools elements
present in the environment.

In this review, we define an autonomous agent system as a model (typically an
LLM) that continuously receives observations from the environment and executes
actions to complete a provided task, as described by Gao et al. 373.
Nevertheless, in contrast to CoALA373, we will rename “internal tools” as
“agent modules” and “external tools” simply as “tools”, for clarity. The agent
consists of trainable decision-making components such as the LLM itself,
policy, memory, and reasoning scheme. In contrast, the environment comprises
non-trainable elements like the task to be completed, Application Programming
Interface (API) access, interfaces with self-driving labs, dataset access, and
execution of external code. By referring to decision-making components as
agent modules, we emphasize their inclusion as parts of the agent. By
referring to non-trainable elements as tools, we highlight their role as part
of the environment. We discuss six main types of actions. As shown in Figure
5, four of the six, memory, planning, reasoning, and profiling are agent
modules. The remaining two actions (or tools) and perception are part of the
environment. Since the perception is how the agent interacts with the
environment and is not a trainable decision, we therefore included it as part
of the environment.

###  4.1 Memory module

The role of the memory module is to store and recall information from past
interactions and experiences to inform future decisions and actions. There are
multiple types of memory in agents, namely sensory memory, short-term memory,
and long-term memory. A major challenge in using agents is the limited context
window, which restricts the amount of in-context information and can lead to
information loss, thereby impacting the effectiveness of short-term and long-
term memory. Solutions involve summarizing memory content377, compressing
memories into vectors378, 379, 380, and utilizing vector databases381 or
combinations thereof382, with various databases available such as ChromaDB,
FAISS, Pinecone, Weaviate, Annoy, and ScaNN.383 Addressing these challenges to
enhance agent memory continues to be a significant area of research384.
Sensory, or procedural, memory is knowledge embedded into the model’s
parameters during pretraining and/or in heuristics implemented into the
agent’s code. Short-term, or working, memory includes the agent’s finite
knowledge during a task, incorporating interaction history and techniques like
in-context learning385 (ICL), which leverages the limited input’s context
length for information retention. Long-term memory involves storing
information externally, typically through an embedded vector representation in
an external database. In the original CoALA373 paper, long-term memory is
further divided into two different types of long-term memory: episodic, which
registers previous experiences, and semantic, which stores general information
about the world.

###  4.2 Planning and reasoning modules

The planning and reasoning module is made of two components. Planning involves
identifying a sequence of actions required to achieve a specified goal. In the
context of language agents, this means generating steps or strategies that the
model can follow to solve a problem or answer a question, which can be
enhanced with retrieval from previous experiences,386 and from feedback from
post-execution reasoning387, 388. Reasoning refers to the process of drawing
conclusions or making decisions based on available information and logical
steps. For example, there are studies that demonstrate the benefits of LLM
reasoning for question answering, where new context tokens can be integrated
in a step-by-step way to guide the model towards more accurate answers389,
390, 391, 392, 393, 394. One popular reasoning strategy is Chain-of-Thought
(CoT),391, 395, 396, 98, 397, 398 a reasoning strategy which substantially
boosts QA performance by generating intermediate reasoning steps. CoT thereby
reduces hallucinations and enhances interpretability, as demonstrated by
improved results in models like PaLM399 and GPT-3 with benchmarks like
GSM8K400, SVAMPs401, and MAWPS402.

In advanced reasoning, final tasks are often decomposed into intermediary ones
using a cascading approach, similar to Zero-shot-CoT390 and RePrompt387.
However, while CoT is considered as single-path reasoning, CoT extensions like
Tree-of-Thoughts393, Graph-of-Thoughts403, Self-consistent CoT392, and
Algorithm-of-Thoughts404 offer multi-path reasoning. Furthermore, other models
have pitted multiple agents against each other to debate or discuss various
reasoning paths,405, 406, 407 while others use external planners to create
plans408, 409. A feedback step during the execution of the plan was a further
extension of the CoT ideas; this enables agents to refine their actions based
on environmental responses adaptively, which is crucial for complex tasks.410,
411

Another interesting reasoning scheme is the Chain-of-Verification(CoVe), 388
where once an answer is generated, another LLM is prompted to generate a set
of verification questions to check for agreement between the original answer
and the answers to the verification questions such that the final answer can
be refined. The ReAct393 – Reason+Act – model proposes adding an observation
step after acting. This means the LLM first reasons about the task and
determines the necessary step for its execution, it performs the action and
then observes the action’s result. Reasoning on that result, it can
subsequently perform the following step. Similarly, Reflexion98 also
implements a reasoning step after executing an action. However, Reflexion
implements an evaluator and self-reflection LLMs to not only reason about each
step but also to evaluate the current trajectory the agent is following using
a long-term memory module. As the context increases, it may become challenging
for agents to deal with the long prompt. Aiming to solve this issue, the
Chain-of-Agents (CoA)412 extends reasoning schemes that leverage multi-agent
collaboration to reason over long contexts. This framework employs workers and
manager agents to process and synthesize information to generate the final
response. CoA demonstrated improvements of up to 10% when compared against an
RAG baseline.

ReAct and Reflexion are closed-ended approaches where the agent starts with
all the tools and must determine which to use. To address more open-world
challenges, Wang et al. 413 introduced the Describe, Explain, Plan, and Select
(DEPS) method, which extends this approach. Lastly, human inputs can also be
used to provide feedback to the agent. Providing feedback using a human-in-
the-loop approach is particularly interesting in fields where safety is a main
concern.

###  4.3 Profiling module

LLMs can be configured to perform in specific roles, such as coders,
professors, students, and domain experts, through a process known as
profiling. Language agents can thus incorporate the profile through the LLM or
through the agent code. The profiling approach involves inputting
psychological characteristics to the agent, significantly impacting its
decision-making process 414, 415, 416, 417. Profiling enables the creation of
multi-agent systems that simulate societal interactions, with each agent
embodying a unique persona within the group 386, 418. The most prevalent
technique for profiling, called “handcrafting”, requires manually defining the
agent’s profile, often through prompts or system messages 419, 420. While
profiling can also be automated with LLMs 421, that automation method may only
be suited for generating large numbers of agents since it offers less control
over their overall behavior. An interesting application of profiling is the
development of agent sets that reflect demographic distributions 422.

###  4.4 Perception

Perception is an analog to the human sensory system, which interprets
multimodal information such as text, images, or auditory data, transforming it
into a format comprehensible by LLMs, as demonstrated by SAM423, GPT4-V424,
LLaVa425, Fuyu8B116, and BuboGPT426. In our proposed architecture, the
perception is responsible for converting the task and the observations to a
data representation that can be understood by the agent. Moreover,
advancements in LLMs have led to the development of even more versatile
models, such as the any-to-any Next-GPT117 and the any-to-text Macaw-LLM427.
Employing multimodal LLMs, which are introduced above, in decision-making
processes can simplify perception tasks for agents, with several studies
exploring their use in autonomous systems428, 429.

###  4.5 Tools

In our proposed definition (see Figure 5b), tools or actions are part of the
environment. The agent can interact with this environment by deciding which
action to execute through the decision-making process. The set of all possible
actions that can be selected is also known as the “action space”.

The decision process is composed of three main steps: proposal, evaluation,
and selection. During the proposal, one or more action candidates are selected
using reasoning393, code structures430, 386, or simply by selecting every tool
available431.389, 432, 392 The evaluation process consists of evaluating each
selected action according to some metric to predict which action would bring
more value to the agent. Lastly, the action is selected and executed.

Given that pretrained parameters (sensory memory) are limited, the model must
use tools for complex tasks in order to provide reliable answers. However,
LLMs need to learn how to interact with the action space and how and when to
use those tools most accurately.433 LLMs can be pretrained or fine-tuned with
examples of tool use, enabling them to operate tools and directly retrieve
tool calls from sensory memory during a zero-shot generation.434 Recent
studies investigate this approach, particularly focusing on open-source
LLMs.435, 436, 437

As foundational AI models become more advanced, their abilities can be
expanded. It was shown that general-purpose foundation models can reason and
select tools even with no fine-tuning. For example, MRKL438 (pronounced
“miracle”) implements an extendable set of specialized tools known as neuro-
symbolic modules and a smart “router” system to retrieve the best module based
on the textual input. These neuro-symbolic modules are designed to handle
specific tasks or types of information, equipped with built-in capabilities
and task-relevant knowledge. This pre-specialization allows the model to
perform domain-specific tasks without needing a separate, domain-specific
dataset. This design addresses the problem of LLMs lacking domain-specific
knowledge and eliminates the need for the costly and time-consuming LLM fine-
tuning step, using specialized data annotation.439 The router can receive
support from a reasoning strategy to help select the tools439 or follow a
previously created plan413. Recent advances have shown that LLMs can develop
new tools of their own440, 441, 442, enabling agents to operate, as needed, in
dynamic and unpredictable “open-worlds”, on unseen problems as illustrated by
Voyager430. This capability allows agents to evolve and improve continually.

##  5 LLM-Based autonomous agents in scientific research

The previous section introduced key concepts relevant to any description of
the development of autonomous agents. Here, we now focus on which agents were
developed for scientific purposes, and ultimately for chemistry. Previous
sections of this review have discussed how LLMs could be powerful in
addressing challenges in molecular property prediction, inverse design, and
synthesis prediction. When we consider the value of agents in chemistry and
the ability to combine tools that, for example, search the internet for
established synthetic procedures, look up experimental properties, and control
robotic synthesis and characterization systems, we can see how autonomous
agents powerfully align with the broader theme of automation, which will lead
to an acceleration of chemical research and application.

Table 4: Scientific LLM systems and agents. We identify the studies we classified as an agent with the icon 443 and multi-agent systems with the icon . , , and 444 mean the agent bases his behavior on sensory, short, and long memory components, respectively. Besides the textual capabilities of LLM-based agents, 445 and 446 mean the agent has additional audio and visual perception, respectively. Agent | Memory | Planning | Reasoning |  Action | Release
---|---|---|---|---|---
LLaMP447 |  |  |  |  Tools for database access, literature search, and atomistic simulations | 2024.06
SGA448 |  |  |  |  Employ the LLMs in a optimization loop | 2024.05
CRISPR-GPT449 |  |  |  |  Tool for gene editing experiments design | 2024.04
TAIS450 |  |  |  |  Tools for gene expression data analysis | 2024.02
ChemReasoner451 |  |  |  |  Tools for heuristic search, 3D structure generation, and prediction using GNNs | 2024.02
SciAgent452 |  |  |  |  Trained Mistral for tool usage. Evaluated it using MathToolBench’s tools | 2024.02
STORM453 |  |  |  |  Article writing using retrieval from multi-LLM conversations and pre-generated outline | 2024.02
Völker et al. 454 |  |  |  |  Regression with ICL and text retrieval | 2024.02
ProtAgent455 |  |  |  |  Tools for proteins information retrieval, analyzing, de novo design, and 3D folded structure generation | 2024.01
Organa370 |  |  |  |  Tools for common lab procedures, reasoning about experimental results, and report writing | 2024.01
PaperQA456 |  |  |  |  Tools to search the scientific literature, gather evidence, and answer questions | 2023.12
WikiCrow457 |  |  |  |  Uses PaperQA as a tool | 2023.12
Coscientist48 |  |  |  |  Tools for running Python code, web-searching, and interacting with lab equipment | 2023.12
Eunomia458 |  |  |  |  Tools for literature and dataset searching and a chain-of–verification loop | 2023.12
CALMS459 |  |  |  |  Tools for using the Materials Project API, designing experiments, and using a hardware API to perform the experiment | 2023.12
CoQuest460 |  |  |  |  Research question generations and tools for literature visualization using a graph organization | 2023.10
eXpertAI309 |  |  |  |  Tools for applying XAI methods | 2023.11
BioPlanner461 |  |  |  |  Tool for protocol searching in the BioProt dataset | 2023.10
IBM ChatChem462 |  |  |  |  Tools for cheminformatics and accessing GT4SD and HuggingFace models | 2023.09
ChatMOF463 |  |  |  |  Tools for database search, property prediction, and MOF’s structure generation | 2023.08
AmadeusGPT47 |  |  |  |  Tools for writing and executing code for computer vision, machine learning, and spatial-temporal reasoning | 2023.07
i-Digest464 |  |  |  |  Uses the whisper model to process audio transcription from classes and write summaries and following up questions | 2023.06
BOLLaMa 465 |  |  |  |  Implements an LLM interface to ease the usage of their BO code | 2023.06
text2concrete466 |  |  |  |  Uses ICL to predict compressive strength from concrete formulation | 2023.06
MAPI_LLM467 |  |  |  |  Database access and LLM prediction using ICL | 2023.06
BO-LIFT468 |  |  |  |  Regression using ICL and text retrieval | 2023.04
ChemCrow47 |  |  |  |  Molecular, cheminformatics, search and critique tools | 2023.04

It was Hocky and White 469 who discussed the early stages of models that could
automate programming and, hence, the expected impacts in chemistry. Then,
early work by White et al. 470 applied LLMs that could generate code to a
benchmark set of chemical problems. In that case, not only were LLMs
demonstrated to possess a notable understanding of chemistry, based on
accurate question answering, but White et al. 470 imagined a potential to use
them as base models to control knowledge augmentation and a variety of other
tools. Thus, these LLMs could be used to execute routine tasks, optimize
procedures, and enhance the retrieval of information from scientific
literature across a range of scientific domains. To the best of our knowledge,
this is the first review of autonomous agents in chemistry that have evolved
since these two visionary conceptual perspectives. A deeper exploration
follows below. One driving motivation for the need to augment LLMs with a more
pertinent and dedicated knowledge base is the need to circumvent problems of a
limited context prompt window, and the restriction that once an LLM is
trained, any new information is beyond it’s reach since it necessarily has
fallen outside its corpus of training data. Furthermore, LLMs are also known
to hallucinate. Their predictions are probabilistic and, in science, if
experimental evidence is available, then there is great value in building from
known domain-specific information. Some improved prompt engineering can aid in
the generation of results that are more likely to be accurate, but the use of
autonomous agents may solve such problems completely in this next phase of AI
in chemistry. In fact, even adding one or two components when building an
agent, as opposed to a whole suite, has shown some significant gains.

Building on this foundation, Ramos et al. 468 illustrated that LLMs could
directly predict experimental outcomes from natural language descriptions, a
technique they incorporated into a Bayesian optimization (BO) algorithm to
streamline chemical processes. Because ICL was used, their approach did not
require additional model training or finetuning, thereby greatly simplifying
the optimization algorithm and removing the need to re-train or update models
as data is gathered. Recently, Kristiadi et al. 471 demonstrated that similar
results could also be achieved with a much smaller domain-specific model while
performing parameter-efficient fine-tuning (PEFT) instead of ICL. Ranković and
Schwaller 472 also explored BO using natural language. They used an LLM to
encode chemical reaction procedures, described using natural language, and
then trained a Gaussian process (GP) head to predict the reaction yield from
the latent encoded representation of the procedure. This effective transfer-
learning approach minimized training time by keeping the LLM frozen while
updating only the MLP. Extending these ideas, Völker et al. 454 suggested
sampling multiple model completions and adding a verifier model to select the
next best procedure to continue the BO algorithm. They also used ICL and a
short-term memory component to optimize alkali-activated concrete mix design.
The authors thus underscored how augmenting with knowledge-driven design can
outperform traditional data-driven design. These are examples of how agent-
based approaches can execute complex optimization algorithms in a step-by-step
approach, directly contributing to automation and leading to faster and more
efficient experimental design.

To better promote new ideas regarding AI in scientific research, Jablonka et
al. 473 organized a hackathon in March 2023. During this one-day hackathon, 14
innovative projects were developed to address chemical problems centered
around predictive modeling, automation, knowledge extraction, and education.
Below, we highlight agent-based approaches from that Hackathon. First,
MAPI_LLM467 is an agent with access to the Materials Project API (MAPI)
database that receives a query asking for a property of a material and then
retrieves the relevant information from the dataset. If the material is not
available on MAPI, the agent can search for similar materials and use in-
context learning (ICL) to provide a prediction of the requested property.
Additionally, MAPI_LLM also has a reaction module for synthesis proposal.
Second, Rankovic et al. 465 used LLMs to make BO algorithms more accessible to
a broader group of scientists; BOLLaMa implements a natural language interface
to easily interact with BO software developed by their group474. Third, and
similar to Ramos et al. 468 and Ranković and Schwaller 472 who employed LLMs
in BO, Weiser et al. 475 focused on genetic algorithms (GA), a different
optimization algorithm. In GA, pieces of information are stochastically
combined and evaluated to guide the algorithm during the optimization. For
chemistry, these pieces are often molecular fragments that are combined to
compose a final whole molecular structure. Thus, Weiser et al. 475 used LLMs
to implement common GA operators under the hypothesis that LLMs can generate
new combined molecules better than random cross-over due to their sensory
memory. Fourth, InsightGraph476 can draw general relationships between
materials and their properties from JSON files. Circi and Badhwar 476 showed
that LLMs can understand the structured data from a JSON format and reorganize
the information in a knowledge graph. Further refinement of this tool could
automate the process of describing relationships between materials across
various scientific reports, a task that remains labor-intensive today. Fifth,
Kruschwitz et al. 466 used ICL and LLMs to accurately predict the compressive
strength of concrete formulations; Text2Concrete achieved predictive accuracy
comparable with a Gaussian process regression (GPR) model, with the advantage
that design principles can be easily added as context. This model was
successfully applied in a BO algorithm following Ramos et al. 468 approach.454
For education purposes, multiple authors have raised the discussion about how
LLMs can be used to support educators’ and instructors’ daily work.477, 478,
479, 480, 481 Finally, in this direction, Mouriño et al. 464 developed
i-Digest, an agent whose perception module can understand audio tracks and
video recordings. These audio recordings are transcribed to text using the
Whisper482 model, and therefore, i-Digest is a digital tutor that generates
questions to help students test their knowledge about the course material.
These are just a few examples to showcase the capabilities of AI systems to
innovate and generate solutions rapidly.

More recently, Ma et al. 452 showed that agents can be trained to use tools.
SciAgent452 was developed under the premise that finetuning LLMs for domain-
specific applications is often impractical. Nevertheless, the agent can be
fine-tuned with a set of tools that will enable them to perform well in a
domain-specific task. These tools, typically Python functions, enable SciAgent
to plan, retrieve, and use these tools to facilitate reasoning and answer
domain-related questions effectively. The benchmark developed for SciAgent,
known as SciToolBench, includes five distinct domains, each equipped with a
set of questions and corresponding tools. The development of its retrieval and
planning modules involved finetuning different LLMs on the MathFunc benchmark,
resulting in a notable performance improvement of approximately $\sim$ 20%
across all domains in SciToolBench compared to other LLMs.

###  5.1 Agents for literature review

Another fantastic opportunity for automation in the sciences is associated
with high-quality literature review, a pivotal aspect of scientific research
that requires reading and selecting relevant information from large numbers of
papers, and thereby distilling the current state of knowledge relevant to a
particular research direction. This extremely time-consuming task is being
revolutionized by advanced AI tools designed to automate and enhance such
analysis and summarization.

PaperQA introduces a robust model that significantly reduces misinformation
while improving the efficiency of information retrieval. This agent retrieves
papers from online scientific databases, reasons about their content, and