performs question-answering (QA) tasks. Its mechanism involves three primary
components—“search”, “gather_evidence”, and “answer_question” and the authors
adapted the Retrieval-Augmented Generation (RAG)483 algorithm to include inner
loops on each step. For instance, PaperQA can perform multiple rounds of
search and gather_evidence if, with reflection, not have enough evidence has
been acquired to successfully answer_question. Further validating its
capabilities, a new benchmark called LitQA demonstrates that PaperQA not only
meets but exceeds human performance in solving complex, real-world scientific
questions. By applying the RAG technique to full-text scientific papers,
PaperQA sets a new standard in QA capabilities, achieving human-like
performance in curated datasets without hallucination or selecting irrelevant
citations.456 To further investigate PaperQA performance, the authors
developed a new benchmark called LitQA, which designs tasks that mimic the
complexity of real scientific inquiry. The benchmark comprises 50 multiple-
choice questions sourced from biomedical papers published post-September 2021,
ensuring they were not part of the LLM training data. In this setting, PaperQA
demonstrated a precision rate of 87.9% and surpassed typical human accuracy,
scoring 69.5% compared to 66.8%.456

Building on top of PaperQA, WikiCrow exemplifies the practical application of
AI in generating concise and relevant Wikipedia-style summaries. The authors
show that while 16% of a human-created Wikipedia article comprises irrelevant
statements, WikiCrow displays irrelevant information only 3% of the time.
Their system also added 5% more correct citations when compared with original
articles. Moreover, thanks to its foundation in the PaperQA framework456,
WikiCrow achieves remarkable cost-efficiency. The authors estimate that
WikiCrow can accomplish in a few days what would take humans approximately
60,000 hours, or about 6.8 years, thereby underscoring its ability to rapidly
produce extensive scientific content. This efficiency exemplifies the
reliability and transformative potential of AI in content creation 457.

Following a different approach, the STORM model also addressed the problem of
writing Wikipedia-like summaries, where the STORM acronym represents the
Synthesis of Topic Outlines through Retrieval and Multi-perspective
questions.453 This approach implements a two-step procedure. First, STORM
retrieves multiple articles on a topic and uses an LLM to integrate various
perspectives into a cohesive outline. Second, this outline is used to write
each section of the Wikipedia-like summary individually. To create the
outline, multiple articles discussing the topic of interest are retrieved by
an “expert” LLM, which processes each one to create $N$ perspectives. Each
perspective is then fed to a “writer” LLM, and a conversation is initiated
between writer and expert. Finally, the $N$ conversations are used to design
the final outline. The outline and the set of references, accessed by RAG, are
given to the writer LLM. The writer LLM is prompted to use these inputs to
generate each section of the article sequentially. Following this, all
sections are merged and refined to eliminate redundancies and enhance
coherence. Upon human evaluation, STORM is reported to be $\sim$ 25% more
organized and present $\sim$ 10% better coverage when compared to a pure RAG
approach. However, it was also less informative than human-written Wikipedia
pages, and STORM presented a transfer of internet-borne biases, producing
emotional articles, which is a major concern.

###  5.2 Agents for chemical innovation

Transitioning from literature synthesis to practical chemistry applications,
we next explore how LLM-based agents have proven their capabilities to
revolutionize routine chemical tasks towards an acceleration of molecular
discovery and scientific research. Agents are flexible entities capable of
developing prompt-specific workflows and executing a plan toward accomplishing
a specific task. ChemCrow47 introduced a significant shift in how LLMs would
be applied in chemistry, given that LLMs alone do not access information
outside of their training data nor can they directly perform chemistry-related
tasks.

By augmenting LLMs with common chemical tools, computational or robotic,
ChemCrow automates a broad spectrum of routine chemical tasks, demonstrating a
significant leap in LLM applicability. Under human evaluation, ChemCrow
consistently outperformed GPT-4, achieving an accuracy score of 9.24/10
compared to 4.79/10.47 The developers of ChemCrow have also considered the
ethical implications and potential risks associated with its capabilities.
ChemCrow’s high potential could be misused and exploited for malicious
objectives, and therefore the authors have implemented safety checks and
guidelines to prevent such misuse, or “dual usage”. Additionally, they
acknowledge that ChemCrow, relying on an LLM, may not always provide
completely accurate answers due to gaps in its chemical knowledge. As such,
they recommend careful and responsible use of the tool, along with thorough
scrutiny of its outputs. In summary, while ChemCrow presents a powerful new
chemical assistant47, oversight of its use is required, and this agent’s
access to tools has been deliberately limited to enhance security and avoid
misuse.

Similarly to ChemCrow47, Chemist-X484 uses RAG to get up-to-date literature
information and use it to reliably solve user’s questions. Nevertheless,
Chemist-X focuses on designing chemical reactions to achieve a given molecule.
It works in three phases: (1) First, the agent searches molecule databases for
similar molecules, then (2) there is a look-up for online literature searching
for chemical reactions capable of converting the list of similar molecules in
the target. Lastly, (3) machine learning models are used to propose the
reaction conditions. To validate their agent, the authors used Chemist-X to
design an HTS experiment aiming to produce 6-(1-methyl-1H-indazol-4-yl),
resulting in a maximum yield of 98.6%.

On the other hand, the Coscientist48 system exemplifies the integration of
semi-autonomous robots in planning, conceiving, and performing chemical
reactions with minimal human intervention. At its core, the system features a
main module named ‘PLANNER’, which is supported by four submodules. These
submodules, or tools, are responsible for performing actions such as searching
the web for organic synthesis, executing Python code, searching the hardware
documentation, and performing a reaction in an automated lab.48 Utilizing this
framework, the Coscientist successfully conducted two types of chemical
coupling reactions, Suzuki-Miyaura and Sonogashira, in a semi-automated
fashion, with manual handling of initial reagents and solvents. Additionally,
Coscientist was also used to optimize reaction conditions. In contrast to
Ramos et al. 468, who used LLMs within a Bayesian Optimization (BO) algorithm
as a surrogate model, Boiko et al. 48 approached the optimization task as a
strategic “game” aimed at maximizing reaction yield by selecting optimal
reaction conditions. This demonstrates the ability of GPT-4 to effectively
reason about popular chemical reactions – possibly via comprehensive coverage
in pretraining. The authors have indicated that the code for their agent will
be released following changes in U.S. regulations on AI and its scientific
applications. At the time of writing, the code remains unreleased, but a
simple example that calculates the square roots of random numbers has been
provided to illustrate their approach.48. These examples underscore the
transformative role of LLMs in enhancing and automating chemical processes,
which will likely accelerate chemical discovery.

Automated workflows in protein research have also been explored. ProtAgent455
is a multi-agent system designed to automate and optimize protein design with
minimal human intervention. This system comprises three primary agents:
Planner, Assistant, and Critic. The Planner is tasked with devising a strategy
to address the given problem, the Assistant executes the plan using
specialized tools and API calls, and the Critic supervises the entire process,
providing feedback and analyzing outcomes. These agents collaborate through a
dynamic group chat managed by a fourth agent, the Chat Manager. Tasks executed
by this team include protein retrieval and analysis, de novo protein design,
and conditioned protein design using Chroma485 and OmegaFold486.

Similarly to ProtAgent, Liu et al. 450 created a team of AI-made scientists
(TAIS) to scientific discovery without human intervention. However, their
agents have roles analog to human roles in scientific discovery, for instance,
project manager, data engineer, code reviewer, statistician, and domain
expert. While in ProtAgent455 agents interact through the Chat Manager only,
TAIS450 enables AI scientists to interact between themselves directly using
pre-defined collaboration pipelines. To evaluate TAIS, the authors curated the
Genetic Question Exploration (GenQEX) benchmark, which consists of 457
selected genetic data questions. As a case study, the authors show TAIS’s
answer to the prompt ‘‘What genes are associated with Pancreatic Cancer when
considering conditions related to Vitamin D Levels?’’. The system identified
20+ genes with a prediction accuracy of 80%.

Innovation can also be achieved by looking into data from a different point-
of-view to get new insights. Automating querying databases was investigated by
Ramos et al. 467 with a ReAct agent with access to the MAPI dataset. This
concept was extended by Chiang et al. 447. LLaMP447 is a RAG-based ReAct agent
that can interact with MAPI, arXiv, Wikipedia, and has access to atomistic
simulation tools. The authors showed that grounding the responses on high-
fidelity information (a well-known dataset) enabled the agent to perform
inferences without fine-tuning.

The agents in chemistry, as exemplified by ChemCrow47 and Coscientist48,
highlight a significant shift towards automation and enhanced efficiency in
molecular discovery and scientific research. These systems demonstrate the
potential of integrating LLMs with chemical tools and automation frameworks,
achieving impressive accuracy and effectiveness in tasks ranging from routine
chemical operations to complex reaction optimizations. Similarly, ProtAgent455
and TAIS450 systems showcase the versatility of multi-agent frameworks in
automating protein design and genetic research, pushing the boundaries of what
AI-driven scientific discovery can achieve. These studies collectively
showcase the incredible potential of agents in chemical and biological
research, promising automation of routine tasks, easing the application of
advanced techniques and analyses, and accelerating discoveries. However, they
also underscore the necessity for meticulous oversight and responsible
development to harness their full potential while mitigating risks.487

###  5.3 Agents for experiments planning

Building on the capabilities of ChemCrow and Coscientist in automating
chemistry-related tasks, recent advances have focused on bridging the gap
between virtual agents and physical laboratory environments. For example,
Context-Aware Language Models for Science (CALMS)459, BioPlanner461, and
CRISPR-GPT449 focus on giving support to researchers with wet-lab experimental
design and data analysis.

CALMS459 focuses on improving laboratory efficiency through the operation of
instruments and management of complex experiments, employing conversational
LLMs to interact with scientists during experiments. In addition, this agent
can perform actions using lab equipment after lab equipment APIs have been
provided to the agent as tools. CALMS was designed to enhance instrument
usability and speed up scientific discovery, providing on-the-spot assistance
for complex experimental setups, such as tomography scans, and enabling fully
automated experiments. For instance, its capability was showcased through the
operation of a real-world diffractometer. Although CALMS excelled in several
tasks, a comparison between GPT-3.5 and Vicuna 1.5 revealed Vicuna’s
limitations in handling tools.

In contrast, BioPlanner461 significantly improves the efficiency of scientific
experimentation by creating pseudocode representations of experimental
procedures, showcasing AI’s capacity to streamline scientific workflows.
Therefore, Rather than interacting directly with lab equipment through APIs,
BioPlanner creates innovative experimental protocols that can be expanded upon
within a laboratory setting. The initial step in BioPlanner’s process involves
assessing the capability of LLMs to produce structured pseudocode based on
detailed natural language descriptions of experimental procedures. In testing,
BioPlanner successfully generated correct pseudocode for 59 out of 100
procedures using GPT-4, although the most common errors involved omitted
units. Afterward, the authors used BioPlanned to generate a procedure for
culturing an E.coli bacteria colony and storing it with cryopreservation,
which ran successfully.

Focusing on gene editing experiments, CRISPR-GPT449 is an agent developed to
design experiments iteratively with constant human feedback. CRISPR-GPT449
aims to bridge the gap for non-experts by simplifying this process into
manageable steps solvable by an LLM with access to useful tools. This agent
operates in three modes based on user prompts: “Meta mode” provides predefined
pipelines for common gene-editing scenarios; “Auto mode” uses the LLM to plan
a sequence of tasks; and “Q&A mode” answers general questions about the
experimental design. The authors demonstrate that based on human evaluations,
CRISPR-GPT outperforms GPT-3.5 and GPT-4 in accuracy, reasoning, completeness,
and conciseness. Additionally, they applied CRISPR-GPT to design real-world
experiments for knocking out TGFBR1, SNAI1, BAX, and BCL2L1 in the human A375
cell line, achieving an editing efficiency of approximately 70% for each gene.

Following the ideas of developing agents for automating experimental protocol
generation, Ruan et al. 488 created a multi-agents system composed of 6
agents: Literature Scouter, Experiment Designer, Hardware Executor, Spectrum
Analyzer, Separation Instructor, and Result Interpreter. The Large Language
Models-based Reaction Development Framework (LLM-RDF)488 automates every step
of the synthesis workflow. While other studies focus on the literature
review489, 490, 491, HTS484, and reaction optimization492, 48, LLM-RDF can
support researchers from literature search until the product purification.
Using this system, the authors showed they could design a copper/TEMPO
catalyzed alcohol oxidation reaction, optimize reaction conditions, engineer a
scale-up, and purify the products, obtaining a yield of 86% and a purity >98%
while producing 1 gram of product.

Interestingly, despite acting on different fields and presenting different
goals (while CALMS459 and LLM-RDF488 can autonomously use laboratory
equipment, BioPlanner461 and CRISPR-GPT449 propose a protocol that needs to be
carried out by a human), all studies implemented a “human-in-the-loop”
approach. Maintaining the researcher as a vital piece of the development is a
way to enhance reliability and mitigate agent limitations, such as errors and
hallucinations. This approach also addresses risk and dual usage problems once
a human can decide whether the agent’s suggestions are safe. 374, 493 On a
slightly different direction, Organa370 fully automates the laboratory
workload while providing feedback to the researcher and producing reports with
the results, as discussed on Section 3.7.1.

###  5.4 Agents for automating cheminformatics tasks

Cheminformatics consists of applying information technology techniques to
convert physicochemical information into knowledge. The process of solving
cheminformatics problems commonly involves retrieving, processing, and
analyzing chemical data.494 Getting inspiration from ChemCrow47 ideas,
Chemistry Agent Connecting Tool Usage to Science (CACTUS)495 focused on
assisting scientists by automating cheminformatics tasks. CACTUS automates the
applications of multiple cheminformatics tools, such as property prediction
and calculation, while maintaining the human-in-the-loop for molecular
discovery. The authors investigated the performance of a diverse set of open-
source LLMs, where Gemma-7B and Mistral-7B demonstrated superior performance
against LLaMA-7B and Falcon-7B. In addition, the authors reported that adding
domain-specific information in the prompt to align the agent to chemistry
problems considerably increases model’s performance. For instance, predicting
drug-likeness with a Gemma-7B agent improves the accuracy of $\sim$ 60% when
aligning the agent. Prompt alignment improved the prediction of all properties
they studied.

Further illustrating the versatility of AI in scientific research and domain-
specific tools usage is ChatMOF,463 which focuses on the prediction and
generation of Metal-Organic Frameworks (MOFs). ChatMOF integrates MOF
databases with its MOFTransformer496 predictor module, thereby showcasing the
innovative use of genetic algorithms in guiding generative tasks from
associated predictions. The authors showed that ChatMOF achieved an accuracy
of $\sim$ 90% in search and prediction tasks while generative tasks have an
accuracy of $\sim$ 70%. The genetic algorithm used by ChatMOF allows for the
generation of a diverse array of MOF structures, which can be further refined
based on specific properties requested by users. For instance, when prompted
to, “generate structures with the largest surface area”, the system initially
generated a broad distribution of structures with surface area centered in
3784 m2/g, and the GA evolves it to a narrower distribution with a peak at
5554 m2/g after only three generations. It is important to note that even
though ChatMOF has access to a dataset of experimental values for MOFs,
language model predictions guide their GA, and no further validation has been
made. Lastly, Ansari and Moosavi 458 developed Eunomia, another domain-
specific autonomous AI agent that leverages existing knowledge to answer
questions about materials. Eunomia458 can use chemistry tools to access a
variety of datasets, scientific papers and unstructured texts to extract and
reason about material science information. The authors implemented a CoVe388
(Consistency Verification) scheme to evaluate the model’s answer and minimize
hallucination. The authors showed that including CoVe increased the model’s
precision by $\sim$ 20% when compared to previous methods such as an agent
using ReAct only.393

Promoting molecular discovery is a topic with great attention in the
literature devoted to it and, as described extensively above, LLMs have
leveraged a large amount of unstructured data to accelerate that discovery.
Janakarajan et al. 462 discuss the advantages of using LLMs in fields such as
de novo drug design, reaction chemistry, and property prediction, but they
augment the LLM in IBM ChemChat, a chatbot with the capability of using common
APIs and python packages commonly used daily by a cheminformatics researcher
to access molecular information. ChemChat has access to tools such as
Generative Toolkit for Scientific Discovery (GT4SD)497, a package with dozens
of trained models generative models for science, rxn4chemistry498, a package
for computing chemistry reactions tasks, HuggingMolecules499, a package
developed to aggregate molecular property prediction LMs, and RDKit500, a
package to manipulating molecules. Since ChemChat implements an agent in a
chat-like environment, users can interactively refine design ideas. Despite
being developed to target de novo drug design, ChemChat nonetheless is a
multi-purpose platform that can be more broadly used for molecular discovery.

In addition to the capabilities described above, LLM-based agents can empower
users to tackle tasks that typically require extensive technical knowledge. In
previous work, Wellawatte and Schwaller 309 and Gandhi and White 257 showed
that including natural language explanations (NLE) in explainable AI (XAI)
analysis can improve user understanding. More recently, Wellawatte and
Schwaller 309 developed XpertAI309 to seamlessly integrate XAI techniques with
LLMs to interpret and explain raw chemical data autonomously. Applying XAI
techniques is usually restricted to technical experts but by integrating such
techniques with an LLM-based agent to automate the workflow, the authors made
XAI accessible to a wider audience.

Their system receives raw data with labels for physicochemical properties. The
raw data is used to compute human-interpretable descriptors and then calculate
SHAP values or Z-scores for Local Interpretable Model-agnostic Explanations
(LIME). By calculating SHAP values (or SHapley Addictive exPlanations) a value
can be assigned to each feature, indicating its contribution to a model’s
output. LIME interprets a model by making a local approximation, around a
particular prediction, to indicate what factors have contributed to that
prediction in the model. It may use, for example, a surrogate local linear
regression fit to recognized features.257 In addition to XAI tools, XpertAI
can search and leverage scientific literature to provide accessible NLEs.
While ChatGPT provides scientific justifications with similar accuracy, its
explanation is often too broad. On the other hand, XpertAI provides data-
specific explanations and visual XAI plots to support its explanations.309
With a similar goal, Zheng et al. 501 prompted the LLM to generate explanatory
rules from data.

These developments signify a growing trend in the integration of tools and
LLMs in autonomous AI within scientific research. By automating routine tasks,
enhancing information retrieval and analysis, and facilitating
experimentation, AI is expanding the capabilities of researchers and
accelerating the pace of scientific discovery. This review underscores the
transformative impact of AI across various scientific domains, heralding a new
era of innovation and efficiency in chemical research.

###  5.5 Agents for hypothesis creation

Following the agent’s classification proposed by Gao et al. 373, the studies
we discussed previously lie mainly in level 1 — AI agents as a research
assistant. That is, such agents can support researchers in executing
predefined tasks, but they lack the autonomy to propose, test, and refine new
scientific hypotheses. New research has been focusing on making agents able to
refine scientists’ initial hypotheses collaboratively, which is a required
skill to achieve level 2 in Gao et al. 373 classification.

the idea of an “AI scientist” who can generate new, relevant research
questions (RQ) has been pursued. Wang et al. 502 developed a framework called
Scientific Inspiration Machines Optimized for Novelty (SciMON). It uses LLMs
to produce new scientific ideas grounded in existing literature. SciMON
retrieves inspirations from past papers and iteratively refines generated
ideas to optimize novelty by comparing them with prior work. Extending these
ideas, Gu and Krenn 503 used LLMs to search over a knowledge graph for
inspiration to propose new personalized research ideas. Aligned with this
vision, Liu et al. 460 developed CoQuest, partially automating the
brainstorming for new RQs process. This system uses a human-computing
interface (HCI) to allow the agent to create new RQs that can be further
enhanced by human feedback. They developed two strategies for RQ generation:
breadth-first, where the agent generates multiple RQs simultaneously following
the original user’s prompt, and depth-first, where multiple RQs are created
sequentially, building on the top of the previously generated RQ. For each RQ
generation, the agent implements a ReAct393 framework with tools for
literature discovery, hypothesis proposition, refinement, and evaluation. Upon
evaluation of 20 HCI doctoral researchers by a post-interaction survey, the
breadth-first approach was preferred by 60% of the evaluators. Interestingly,
despite the evaluators’ report that the breadth-first approach gave them more
control and resulted in more trustworthy RQs, the depth-first had better
scores for novelty and surprise. This difference might be caused by the fact
that the depth-first uses its own RQ to iterate. This process can introduce
new keywords that users have not considered.

Focusing on generating and testing hypotheses, ChemReasoner451 uses a domain-
specific reward function and computational chemistry feedback to validate
agent responses. The authors combined a Monte Carlo thought search504 for
catalysis with a reward function from atomistic GNNs trained to predict
adsorption energy or reaction energy barriers. While the search is responsible
for exploiting literature information and allowing the model to propose new
materials, the hypothetic material is further tested by the GNN. This
framework was applied to suggest materials for adsorbates, biofuel catalysts,
and catalysts for CO2 to methanol conversion. The LLM generated the top five
catalysts for each task, with ChemReasoner significantly outperforming GPT-4
based on the reward score.

Similarly, Ma et al. 448 developed Scientific Generative Agent (SGA) to
generate hypotheses and iteratively refine them through computational
simulations. Initially, the LLM generates a hypothesis. In the use cases
considered, it can be a code snippet or a molecule. In the sequence, a search
algorithm is used to find a better initial hypothesis for solving the initial
query. Finally, this hypothesis — code or molecule — is optimized using a
gradient-based algorithm. Lastly, the optimization output serves as feedback
to the LLM to iterate. In their molecule design task, the goal was to generate
a molecule with a specified HOMO-LUMO gap. The hypothesis is a molecule, that
is, a SMILES string and a set of atomic coordinates. The gap is predicted by
employing UniMol505. They showed that SGA could generate molecules based on
quantum mechanical properties, but the results were not validated.

##  6 Challenges and opportunities

LLMs hold great potential in chemistry due to their ability to both predict
properties and orchestrate existing computational and experimental tools.
These capabilities enhance the accuracy and efficiency of chemical research
and open up new avenues for discovery and innovation. By encapsulating AI
models, data analysis software, and laboratory equipment within agent-based
frameworks, researchers can harness these sophisticated tools through a
unified interface. This approach not only simplifies the interaction with
complex systems but also democratizes the immense capabilities of modern
computational tools, thereby maximizing their utility in advancing chemical
research and development.

##### Data Quality and Availability

Quality and availability of data are critical factors that influence the
efficacy of LLMs. Indeed, scaling both the model size and the amount of
training data used has proven to improve capabilities.506 However, current AI
models are not trained on large amounts of chemical data, which limits their
capabilities to reason about advanced chemical concepts.183

There are two types of datasets commonly used to train LLMs: unlabeled and
labeled datasets. Unlabeled datasets, or pretraining data, are used during the
semi-supervised training, which focuses on creating a “prior belief” about a
molecule. Currently, we have huge datasets composed of hypothetical and/or
theoretical data. When a model is trained on data that is not grounded in real
chemical information, this might cause the model to learn a wrong prior
belief.507

Labeled datasets, often used in benchmarks, also suffer from their inclusion
of hypothetical and calculated data. Benchmarks are necessary for quantifying
improvements in AI modeling and prediction within a competitive field.
However, dominant benchmarks like MoleculeNet,57 have significant limitations
that may restrict the generalizability and applicability of evolving models.
In his blog, Walters 186 brings to light numerous errors and inconsistencies
within the MoleculeNet data, which substantially impact model performance and
reliability.184, 146, 185, 128. Walters also argues that the properties
present in these benchmarks do not directly correlate with real chemistry
improvement. As such, new benchmarks need to translate to practical chemistry
problems directly. For instance, increasing accuracy in predicting LogP is not
necessarily mapped to drugs with greater bioavailability. Some promising work
has come from the Therapeutic Common Data (TDC)189, 188 includes data from
actual therapeutic essays, providing a more practical foundation for model
training.

The community continues to work to organize and curate datasets to make data
ready for LLMs training and evaluation. Scientific benchmarks135, 456, 508,
repositories with curated datasets182, and packages for model evaluation183
have been developed. However, the challenges concerning grounded truth and
consistent datasets remain. With advancements in scientific document
processing,339 there is now the opportunity to obtain new datasets from peer-
reviewed scientific papers. Due to the multi-modal capabilities of such AI
models, these new benchmarks can comprise multiple data types, potentially
enhancing the applicability and transferability of these models. The continual
curation of new, relevant datasets that represent the complexities of real-
world chemical problems will further enhance the robustness and relevance of
LLMs in chemistry.

##### Model Interpretability

Model interpretability is a significant challenge for LLMs due to their
“black-box” nature, which obscures the understanding of how predictions are
made. However, innovative approaches are being developed to enhance LLMs’
interpretability. For instance, Schwaller et al. 509 and Schilter et al. 510
used information from the different multi-attention heads. While Schwaller et
al. 509 connected atoms from reactants to atoms in the products, Schilter et
al. 510 assigned H-NMR peaks to specific hydrogens in a molecule to indicate
how spectra were comprehended, or structures deduced. Additionally, since the
LLMs use language, which is intrinsically interpretable, LLMs may be
incrementally modified to explain their reasoning processes directly,
exemplified with tools like eXpertAI309 and or simply adjusting prompting391,
439. These methods address the critical need for transparency in the mechanism
of understanding for a good prediction beyond the good prediction itself.

##### Integration with Domain Knowledge and Cross-disciplinary Applications

While LLMs excel at pattern recognition, integrating explicit chemical rules
and domain knowledge into these systems remains challenging. This integration
is essential to make predictions that are not only statistically valid but
also chemically reasonable. It was shown by Beltagy et al. 134 and Gu et al.
133 that better performance on common NLP tasks can be achieved by developing
a vocabulary and pretraining on a domain-specific training corpus. While,
pretrainng with domain-specific datasets that include chemical properties,
reaction mechanisms, and experimental results may better capture the nuances
of chemistry, the challenge to use AI to promote multi-disciplinary research
remains. The Galactica LLM 126 also used special tokens for delineating
chemical information, to relatively good success on chemistry tasks. Aryal et
al. 511 also progress by creating an ensemble of specialist agents with
different domains of knowledge, allowing them to interact to better answer the
user query. Specifically, Aryal et al. 511 used agents with chemistry,
physics, electrochemistry, and materials knowledge.

##### Tool development

The effectiveness of a combined LLM/autonomous agent approach hinges
significantly on the availability and quality of the tools, as well as on the
complexity and diversity of the chemical tasks at hand. Some emphasis should
be placed on refining standalone tools, with the confidence that overarching
frameworks, like a GPT-4-type wrapper, or “assistant”, will eventually
integrate these tools seamlessly. Developers should stay informed about
existing tools and design their tools to interface effectively with such a
wrapper. This ensures that each tool is ready to contribute its unique
capabilities to a cohesive agent system.

##### Reinforcement learning

RL has been successfully used in LLMs512, 513, 94, with a few applications
also proposed for use in agents514, 515. The next frontier is applying RL to
agents directly, to improve their ability on specific tasks. Bou et al. 103
provided a recent framework and example for generative molecular design when
viewed as an RL problem (similar to RLHF) and some early success has been seen
in applying the RLHF algorithm directly to protein language models where the
reward model comes from scientific tasks.104 Neither of these are direct RL on
language model agents, but are a step towards this goal.

##### Agent evaluation

Comparing different agent systems is challenging due to the lack of robust
benchmarks and evaluation schemes. Consequently, it is difficult to define
what constitutes a “superhuman” digital chemist and reach a consensus on the
criteria for success. This issue is similar to the ongoing discussions about
defining artificial general intelligence (AGI) and the expected capabilities
of cognitive architectures.516, 517 Once a reliable metric for evaluating such
AI systems is established, it is crucial for the AI scientific community to
set clear guidelines for conducting research. Currently, assessing success is
challenging because the goals are not well defined. Building on this, we
propose using Bloom’s taxonomy as a reference point for developing a metric to
evaluate more complex reasoning and tool use in autonomous agents. This
educational framework categorizes cognitive skills in a hierarchical manner,
from basic recall to creative construction, providing a structured approach to
assess higher-order thinking and reasoning capabilities in these systems. This
adaptation could significantly enhance the evaluation of LLMs and autonomous
agents, especially when tackling complex chemical challenges.

##### Ethical and Safety Concerns

As with all AI technologies, there are ethical considerations in deploying
LLMs, including bias in model predictions and the potential misuse of AI-
generated chemical knowledge. Ruan et al. 518 and Tang et al. 487 pointed out
the necessity of different levels of regulation in agent development, noting
that current alignment methods may be insufficient for ensuring safety, and
using human evaluation to address the risks of such systems is not scalable.
Therefore, the lack of specialized models for risk control and effective
safety evaluations poses perhaps the greatest challenges when assuring the
safety of LLMs with the ability to use tools. This underscores the urgency,
and the opportunity, in automating red-teaming of agents to enhance safety
protocols in AI technologies.

##### Human-AI Collaboration in Chemical Research

LLMs are poised to transform fields such as drug discovery, materials science,
and environmental chemistry due to their ability to predict chemical
properties and reactions with remarkable accuracy. Models based on
architectures like BERT have demonstrated their capability to achieve state-
of-the-art performance in various property prediction tasks.45, 155
Furthermore, studies by Jablonka et al. 249 and Born and Manica 145 have
showcased the predictive power of LLMs by reformulating traditional regression
and classification tasks as generative tasks, opening up new avenues for
chemical modeling. However, as emphasized by Weng 375, maintaining the
reliability of LLM outputs is essential, as inaccuracies in formatting,
logical reasoning, or content can significantly impede their practical
utility. Hallucination is also an intrinsic issue with LLMs.519 Though agents
can deal with hallucinations to some extent by implementing sanity-checking
tools, it does not make the response hallucination-proof. A possible approach
to address this issue is to use a human-in-the-loop approach, where steps of
human-agent interaction are added to the workflow to check if the agent is in
the correct pathway to solve the request.520, 521, 522

The potential of LLMs to design novel molecules and materials was highlighted
by the AI-powered robotic lab assistant, A-Lab, which synthesized 41 new
materials within just 17 days.523 Nonetheless, this achievement has sparked
debates about the experimental methods and the actual integration of atoms
into new crystalline materials, raising questions about the authenticity of
the synthesized structures.524 These controversies underline the necessity for
rigorous standards and the critical role of human expertise in validating AI-
generated results. Again, the integration of advanced AI tools with the
oversight of seasoned chemists is crucial, suggesting that a hybrid approach
could significantly enhance both the innovation and integrity of materials
science research.

In parallel, we have seen how LLM-based agents are increasingly capable of
automating routine tasks in chemical research, which traditionally consume
significant time and resources. These models excel in real-time data
processing, managing vast datasets, and even conducting comprehensive
literature reviews with minimal human intervention. Advances in AI technology
now allow agents not only to perform predefined tasks but also to adapt and
develop new tools for automating additional processes. For instance, tasks
such as data analysis, literature review, and elements of experimental design
are now being automated.309, 525, 526, 456, 461, 459, 47, 48 This automation
liberates chemists to focus on more innovative and intellectually engaging
aspects of their work, and the opportunity is to expand productivity and
creativity in their science.

##  7 Conclusions

Since this review is targeted in part to an audience of chemists, who may not
have yet embraced AI technology, we consider it valuable to point out our
perspective that AI in chemistry is definitely here to stay. We predict that
its use will only grow as a necessary tool that will inevitably lead to more
jobs and greater progress. We hope to facilitate the change by connecting the
technology to the chemical problems that our readership is already addressing
through more traditional methods.

Large Language Models (LLMs) have demonstrated remarkable potential in
reshaping chemical research and development workflows. These models have
facilitated significant advancements in molecular simulation, reaction
prediction, and materials discovery. In this review, we discussed the
evolution of LLMs in chemistry and biochemistry. Successful cases where LLMs
have proven their potential in promoting scientific discovery were shown with
caveats of such models.

Adopting LLM-based autonomous agents in chemistry has enhanced the accuracy
and efficiency of traditional research methodologies and introduced innovative
approaches to solving complex chemical problems. Looking forward, the
continued integration of LLMs promises to accelerate the field’s evolution
further, driving forward the frontiers of scientific discovery and
technological innovation in chemistry. We have shown how agents have been used
in chemistry and proposed a framework for thinking about agents as a central
LLM followed by interchangeable components.

However, despite the community’s astonishing advances in this field, many
challenges still require solutions. We identified the main challenges and
opportunities that need to be addressed to promote the further development of
agents in chemistry. Addressing the challenges related to model transparency,
data biases, and computational demands will be crucial for maximizing their
utility and ensuring their responsible use in future scientific endeavors.

While there are significant challenges to be addressed, the opportunities
presented by LLMs in chemistry are vast and have the potential to
fundamentally alter how chemical research and development are conducted.
Effectively addressing these challenges will be crucial for realizing the full
potential of LLMs in this exciting field. To keep pace with the ever-growing,
we will maintain a repository with an organized structure listing new studies
regarding LLMs and LLM-based agents focused on scientific purposes. The
repository can be found in https://github.com/ur-whitelab/LLMs-in-science

### Author contribution

All authors contributed to writing this review article.

### Competing interests

The authors have no conflicts to declare.

#### Acknowledgments

M.C.R. and A.D.W. acknowledge the U.S. Department of Energy, Grant No. DE-
SC0023354, and C.J.C. gratefully acknowledges the Jane King Harris Endowed
Professorship at Rochester Institute of Technology for the support provided
for this publication. We are grateful for feedback from early drafts of this
review from the following colleagues: Kevin Jablonka, Philippe Schwaller,
Michael Pieler, Ryan-Rhys Griffiths, Geemi Wellawatte, and Mario Krenn.

## References

  * Willett 2011 Peter Willett.  Chemoinformatics: a history.  _Wiley Interdisciplinary Reviews: Computational Molecular Science_ , 1(1):46–56, 2011.
  * Griffen et al. 2020 Edward J. Griffen, Alexander G. Dossetter, and Andrew G. Leach.  Chemists: AI Is Here; Unite To Get the Benefits.  _Journal of Medicinal Chemistry_ , 63(16):8695–8704, August 2020.  ISSN 0022-2623.  doi:10.1021/acs.jmedchem.0c00163.  URL https://doi.org/10.1021/acs.jmedchem.0c00163.  Publisher: American Chemical Society.
  * Baum et al. 2021 Zachary J. Baum, Xiang Yu, Philippe Y. Ayala, Yanan Zhao, Steven P. Watkins, and Qiongqiong Zhou.  Artificial Intelligence in Chemistry: Current Trends and Future Directions.  _Journal of Chemical Information and Modeling_ , 61(7):3197–3212, July 2021.  ISSN 1549-9596.  doi:10.1021/acs.jcim.1c00619.  URL https://doi.org/10.1021/acs.jcim.1c00619.  Publisher: American Chemical Society.
  * Ayres et al. 2021 Lucas B. Ayres, Federico J.V. Gomez, Jeb R. Linton, Maria F. Silva, and Carlos D. Garcia.  Taking the leap between analytical chemistry and artificial intelligence: A tutorial review.  _Analytica Chimica Acta_ , 1161:338403, May 2021.  ISSN 00032670.  doi:10.1016/j.aca.2021.338403.  URL https://linkinghub.elsevier.com/retrieve/pii/S0003267021002294.
  * Yang et al. 2019a Xin Yang, Yifei Wang, Ryan Byrne, Gisbert Schneider, and Shengyong Yang.  Concepts of Artificial Intelligence for Computer-Assisted Drug Discovery.  _Chemical Reviews_ , 119(18):10520–10594, September 2019a.  ISSN 0009-2665.  doi:10.1021/acs.chemrev.8b00728.  URL https://doi.org/10.1021/acs.chemrev.8b00728.  Publisher: American Chemical Society.
  * Mater and Coote 2019 Adam C Mater and Michelle L Coote.  Deep learning in chemistry.  _Journal of chemical information and modeling_ , 59(6):2545–2559, 2019.
  * Shi et al. 2023 Yun-Fei Shi, Zheng-Xin Yang, Sicong Ma, Pei-Lin Kang, Cheng Shang, P Hu, and Zhi-Pan Liu.  Machine learning for chemistry: basics and applications.  _Engineering_ , 2023.
  * Keith et al. 2021 John A Keith, Valentin Vassilev-Galindo, Bingqing Cheng, Stefan Chmiela, Michael Gastegger, Klaus-Robert Muller, and Alexandre Tkatchenko.  Combining machine learning and computational chemistry for predictive insights into chemical systems.  _Chemical reviews_ , 121(16):9816–9872, 2021.
  * Kuntz and Wilson 2022 David Kuntz and Angela K Wilson.  Machine learning, artificial intelligence, and chemistry: How smart algorithms are reshaping simulation and the laboratory.  _Pure and Applied Chemistry_ , 94(8):1019–1054, 2022.
  * Meuwly 2021 Markus Meuwly.  Machine learning for chemical reactions.  _Chemical Reviews_ , 121(16):10218–10239, 2021.
  * Lederberg et al. 1969 Joshua Lederberg, Georgia L Sutherland, Bruce G Buchanan, Edward A Feigenbaum, Alexander V Robertson, Alan M Duffield, and Carl Djerassi.  Applications of artificial intelligence for chemical inference. i. number of possible organic compounds. acyclic structures containing carbon, hydrogen, oxygen, and nitrogen.  _Journal of the American Chemical Society_ , 91(11):2973–2976, 1969.
  * Lindsay et al. 1993 Robert K Lindsay, Bruce G Buchanan, Edward A Feigenbaum, and Joshua Lederberg.  Dendral: a case study of the first expert system for scientific hypothesis formation.  _Artificial intelligence_ , 61(2):209–261, 1993.
  * Buchanan et al. 1976 B. G. Buchanan, D. H. Smith, W. C. White, R. J. Gritter, E. A. Feigenbaum, J. Lederberg, and Carl Djerassi.  Applications of artificial intelligence for chemical inference. 22. Automatic rule formation in mass spectrometry by means of the meta-DENDRAL program.  _Journal of the American Chemical Society_ , 98(20):6168–6178, September 1976.  ISSN 0002-7863.  doi:10.1021/ja00436a017.  URL https://doi.org/10.1021/ja00436a017.  Publisher: American Chemical Society.
  * Hansch et al. 1962 Corwin Hansch, Peyton P Maloney, Toshio Fujita, and Robert M Muir.  Correlation of biological activity of phenoxyacetic acids with hammett substituent constants and partition coefficients.  _Nature_ , 194(4824):178–180, 1962.
  * Hansch and Fujita 1964 Corwin Hansch and Toshio Fujita.  p-$\sigma$-$\pi$ analysis. a method for the correlation of biological activity and chemical structure.  _Journal of the American Chemical Society_ , 86(8):1616–1626, 1964.
  * Hansch and Leo 1995 Corwin Hansch and Albert Leo.  _Exploring QSAR.: Fundamentals and applications in chemistry and biology_ , volume 1.  American Chemical Society, 1995.
  * Topliss and Costello 1972 John G Topliss and Robert J Costello.  Chance correlations in structure-activity studies using multiple regression analysis.  _Journal of Medicinal Chemistry_ , 15(10):1066–1068, 1972.
  * Weinstein et al. 1992 John N. Weinstein, Kurt W. Kohn, Michael R. Grever, Vellarkad N. Viswanadhan, Lawrence V. Rubinstein, Anne P. Monks, Dominic A. Scudiero, Lester Welch, Antonis D. Koutsoukos, August J. Chiausa, and Kenneth D. Paull.  Neural Computing in Cancer Drug Development: Predicting Mechanism of Action.  _Science_ , 258(5081):447–451, October 1992.  doi:10.1126/science.1411538.  URL https://www.science.org/doi/10.1126/science.1411538.  Publisher: American Association for the Advancement of Science.
  * Van Osdol et al. 1994 William W Van Osdol, Timothy G Myers, Kenneth D Paull, Kurt W Kohn, and John N Weinstein.  Use of the kohonen self-organizing map to study the mechanisms of action of chemotherapeutic agents.  _JNCI: Journal of the National Cancer Institute_ , 86(24):1853–1859, 1994.
  * Goldman and Walters 2006 Brian B Goldman and W Patrick Walters.  Machine learning in computational chemistry.  _Annual Reports in Computational Chemistry_ , 2:127–140, 2006.
  * Pereira and Williams 2007 DA Pereira and JA Williams.  Origin and evolution of high throughput screening.  _British journal of pharmacology_ , 152(1):53–61, 2007.
  * Medina-Franco et al. 2013 José L. Medina-Franco, Marc A. Giulianotti, Gregory S. Welmaker, and Richard A. Houghten.  Shifting from the single to the multitarget paradigm in drug discovery.  _Drug Discovery Today_ , 18(9):495–501, May 2013.  ISSN 1359-6446.  doi:10.1016/j.drudis.2013.01.008.  URL https://www.sciencedirect.com/science/article/pii/S1359644613000251.
  * Butler et al. 2018 Keith T. Butler, Daniel W. Davies, Hugh Cartwright, Olexandr Isayev, and Aron Walsh.  Machine learning for molecular and materials science.  _Nature_ , 559(7715):547–555, July 2018.  ISSN 1476-4687.  doi:10.1038/s41586-018-0337-2.  URL https://www.nature.com/articles/s41586-018-0337-2.  Publisher: Nature Publishing Group.
  * Rupp et al. 2012 Matthias Rupp, Alexandre Tkatchenko, Klaus-Robert Müller, and O. Anatole von Lilienfeld.  Fast and Accurate Modeling of Molecular Atomization Energies with Machine Learning.  _Physical Review Letters_ , 108(5):058301, January 2012.  doi:10.1103/PhysRevLett.108.058301.  URL https://link.aps.org/doi/10.1103/PhysRevLett.108.058301.  Publisher: American Physical Society.
  * Olivecrona et al. 2017 Marcus Olivecrona, Thomas Blaschke, Ola Engkvist, and Hongming Chen.  Molecular de-novo design through deep reinforcement learning.  _J. Cheminform._ , 9(1):48, September 2017.  ISSN 1758-2946,1758-2946.  doi:10.1186/s13321-017-0235-x.  URL https://jcheminf.biomedcentral.com/articles/10.1186/s13321-017-0235-x.
  * Segler et al. 2018a Marwin H S Segler, Thierry Kogej, Christian Tyrchan, and Mark P Waller.  Generating focused molecule libraries for drug discovery with recurrent neural networks.  _ACS Cent. Sci._ , 4(1):120–131, January 2018a.  ISSN 2374-7943,2374-7951.  doi:10.1021/acscentsci.7b00512.  URL https://pubs.acs.org/doi/abs/10.1021/acscentsci.7b00512.
  * Segler et al. 2018b Marwin H. S. Segler, Thierry Kogej, Christian Tyrchan, and Mark P. Waller.  Generating Focused Molecule Libraries for Drug Discovery with Recurrent Neural Networks.  _ACS Central Science_ , 4(1):120–131, January 2018b.  ISSN 2374-7943.  doi:10.1021/acscentsci.7b00512.  URL https://doi.org/10.1021/acscentsci.7b00512.  Publisher: American Chemical Society.
  * Gupta et al. 2018 Anvita Gupta, Alex T Muller, Berend J H Huisman, Jens A Fuchs, Petra Schneider, and Gisbert Schneider.  Generative recurrent networks for de novo drug design.  _Mol. Inform._ , 37(1-2):1700111, January 2018.  ISSN 1868-1751,1868-1743.  doi:10.1002/minf.201700111.  URL https://onlinelibrary.wiley.com/doi/abs/10.1002/minf.201700111.
  * Karpov et al. 2020 Pavel Karpov, Guillaume Godin, and Igor V Tetko.  Transformer-CNN: Swiss knife for QSAR modeling and interpretation.  _J. Cheminform._ , 12(1):17, March 2020.  ISSN 1758-2946,1758-2946.  doi:10.1186/s13321-020-00423-w.  URL https://link.springer.com/articles/10.1186/s13321-020-00423-w.
  * Schütt et al. 2018 K. T. Schütt, H. E. Sauceda, P.-J. Kindermans, A. Tkatchenko, and K.-R. Müller.  SchNet – A deep learning architecture for molecules and materials.  _The Journal of Chemical Physics_ , 148(24):241722, March 2018.  ISSN 0021-9606.  doi:10.1063/1.5019779.  URL https://doi.org/10.1063/1.5019779.
  * Hirohara et al. 2018 Maya Hirohara, Yutaka Saito, Yuki Koda, Kengo Sato, and Yasubumi Sakakibara.  Convolutional neural network based on SMILES representation of compounds for detecting chemical motif.  _BMC Bioinformatics_ , 19(Suppl 19):526, December 2018.  ISSN 1471-2105,1471-2105.  doi:10.1186/s12859-018-2523-5.  URL https://bmcbioinformatics.biomedcentral.com/articles/10.1186/s12859-018-2523-5.
  * Coley et al. 2019a Connor W Coley, Wengong Jin, Luke Rogers, Timothy F Jamison, Tommi S Jaakkola, William H Green, Regina Barzilay, and Klavs F Jensen.  A graph-convolutional neural network model for the prediction of chemical reactivity.  _Chem. Sci._ , 10(2):370–377, January 2019a.  ISSN 2041-6520,2041-6539.  doi:10.1039/c8sc04228d.  URL https://pubs.rsc.org/en/content/articlelanding/2019/sc/c8sc04228d.
  * Dwivedi et al. 2023 Vijay Prakash Dwivedi, Chaitanya K Joshi, Anh Tuan Luu, Thomas Laurent, Yoshua Bengio, and Xavier Bresson.  Benchmarking graph neural networks.  _Journal of Machine Learning Research_ , 24(43):1–48, 2023.
  * Sanchez-Lengeling et al. 2021 Benjamin Sanchez-Lengeling, Emily Reif, Adam Pearce, and Alexander B Wiltschko.  A gentle introduction to graph neural networks.  _Distill_ , 6(9):e33, 2021.
  * Bronstein et al. 2017 Michael M Bronstein, Joan Bruna, Yann LeCun, Arthur Szlam, and Pierre Vandergheynst.  Geometric deep learning: going beyond euclidean data.  _IEEE Signal Processing Magazine_ , 34(4):18–42, 2017.
  * Wu et al. 2020 Zonghan Wu, Shirui Pan, Fengwen Chen, Guodong Long, Chengqi Zhang, and S Yu Philip.  A comprehensive survey on graph neural networks.  _IEEE transactions on neural networks and learning systems_ , 32(1):4–24, 2020.
  * Gilmer et al. 2017 Justin Gilmer, Samuel S Schoenholz, Patrick F Riley, Oriol Vinyals, and George E Dahl.  Neural message passing for quantum chemistry.  In _International conference on machine learning_ , pages 1263–1272. PMLR, 2017.
  * Gómez-Bombarelli et al. 2018 Rafael Gómez-Bombarelli, Jennifer N. Wei, David Duvenaud, José Miguel Hernández-Lobato, Benjamín Sánchez-Lengeling, Dennis Sheberla, Jorge Aguilera-Iparraguirre, Timothy D. Hirzel, Ryan P. Adams, and Alán Aspuru-Guzik.  Automatic Chemical Design Using a Data-Driven Continuous Representation of Molecules.  _ACS Central Science_ , 4(2):268–276, February 2018.  ISSN 2374-7943.  doi:10.1021/acscentsci.7b00572.  URL https://doi.org/10.1021/acscentsci.7b00572.  Publisher: American Chemical Society.
  * Gaudelet et al. 2021 Thomas Gaudelet, Ben Day, Arian R Jamasb, Jyothish Soman, Cristian Regep, Gertrude Liu, Jeremy BR Hayter, Richard Vickers, Charles Roberts, Jian Tang, et al.  Utilizing graph machine learning within drug discovery and development.  _Briefings in bioinformatics_ , 22(6):bbab159, 2021.
  * Choudhary et al. 2022 Kamal Choudhary, Brian DeCost, Chi Chen, Anubhav Jain, Francesca Tavazza, Ryan Cohn, Cheol Woo Park, Alok Choudhary, Ankit Agrawal, Simon J. L. Billinge, Elizabeth Holm, Shyue Ping Ong, and Chris Wolverton.  Recent advances and applications of deep learning methods in materials science.  _npj Computational Materials_ , 8(1):1–26, April 2022.  ISSN 2057-3960.  doi:10.1038/s41524-022-00734-6.  URL https://www.nature.com/articles/s41524-022-00734-6.  Publisher: Nature Publishing Group.
  * Fung et al. 2021 Victor Fung, Jiaxin Zhang, Eric Juarez, and Bobby G Sumpter.  Benchmarking graph neural networks for materials chemistry.  _Npj Comput. Mater._ , 7(1):1–8, June 2021.  ISSN 2057-3960,2057-3960.  doi:10.1038/s41524-021-00554-0.  URL https://www.nature.com/articles/s41524-021-00554-0.
  * Reiser et al. 2022 Patrick Reiser, Marlen Neubert, André Eberhard, Luca Torresi, Chen Zhou, Chen Shao, Houssam Metni, Clint van Hoesel, Henrik Schopmans, Timo Sommer, and Pascal Friederich.  Graph neural networks for materials science and chemistry.  _Commun. Mater._ , 3(1):93, November 2022.  ISSN 2662-4443,2662-4443.  doi:10.1038/s43246-022-00315-6.  URL https://www.nature.com/articles/s43246-022-00315-6.
  * Weininger 1988 David Weininger.  SMILES, a chemical language and information system. 1. Introduction to methodology and encoding rules.  _Journal of Chemical Information and Computer Sciences_ , 28(1):31–36, February 1988.  ISSN 0095-2338.  doi:10.1021/ci00057a005.  URL https://doi.org/10.1021/ci00057a005.  Publisher: American Chemical Society.
  * Chithrananda et al. 2020 Seyone Chithrananda, Gabe Grand, and Bharath Ramsundar.  ChemBERTa: Large-scale self-supervised pretraining for molecular property prediction.  _arXiv [cs.LG]_ , October 2020.  doi:10.48550/arXiv.2010.09885.  URL http://arxiv.org/abs/2010.09885.
  * Li and Jiang 2021a Juncai Li and Xiaofei Jiang.  Mol-BERT: An Effective Molecular Representation with BERT for Molecular Property Prediction.  _WIRELESS COMMUNICATIONS & MOBILE COMPUTING_, 2021, September 2021a.  ISSN 1530-8669.  doi:10.1155/2021/7181815.
  * Wang et al. 2023a Ye Wang, Honggang Zhao, Simone Sciabola, and Wenlu Wang.  cMolGPT: A conditional generative Pre-Trained transformer for Target-Specific de novo molecular generation.  _Molecules_ , 28(11), May 2023a.  ISSN 1420-3049.  doi:10.3390/molecules28114430.  URL http://dx.doi.org/10.3390/molecules28114430.
  * M. Bran et al. 2024 Andres M. Bran, Sam Cox, Oliver Schilter, Carlo Baldassari, Andrew D White, and Philippe Schwaller.  Augmenting large language models with chemistry tools.  _Nature Machine Intelligence_ , pages 1–11, May 2024.  ISSN 2522-5839, 2522-5839.  doi:10.1038/s42256-024-00832-8.  URL https://www.nature.com/articles/s42256-024-00832-8.
  * Boiko et al. 2023 Daniil A Boiko, Robert MacKnight, Ben Kline, and Gabe Gomes.  Autonomous chemical research with large language models.  _Nature_ , 624(7992):570–578, December 2023.  ISSN 0028-0836,1476-4687.  doi:10.1038/s41586-023-06792-0.  URL https://www.nature.com/articles/s41586-023-06792-0.
  * White 2023 Andrew D White.  The future of chemistry is language.  _Nature Reviews Chemistry_ , 7(7):457–458, July 2023.  ISSN 2397-3358.  doi:10.1038/s41570-023-00502-0.  URL https://www.nature.com/articles/s41570-023-00502-0.
  * Zhang et al. 2024a Qiang Zhang, Keyang Ding, Tianwen Lyv, Xinda Wang, Qingyu Yin, Yiwen Zhang, Jing Yu, Yuhao Wang, Xiaotong Li, Zhuoyi Xiang, Xiang Zhuang, Zeyuan Wang, Ming Qin, Mengyao Zhang, Jinlu Zhang, Jiyu Cui, Renjun Xu, Hongyang Chen, Xiaohui Fan, Huabin Xing, and Huajun Chen.  Scientific large language models: A survey on biological & chemical domains.  _arXiv [cs.CL]_ , January 2024a.  URL http://arxiv.org/abs/2401.14656.
  * Collison et al. 2008 Christopher J. Collison, Marc J. O’Donnell, and Jessica L. Alexander.  Complexation between Rhodamine 101 and Single-Walled Carbon Nanotubes Indicative of Solvent-Nanotube Interaction Strength.  _The Journal of Physical Chemistry C_ , 112(39):15144–15150, October 2008.  ISSN 1932-7447.  doi:10.1021/jp804359j.  URL https://doi.org/10.1021/jp804359j.  Publisher: American Chemical Society.
  * Wiegand et al. 2024 Tyler J. Wiegand, Juan S. Sandoval, Jeremy A. Cody, David W. McCamant, and Christopher J. Collison.  Directional Exciton Diffusion, Measured by Subpicosecond Transient Absorption as an Explanation for Squaraine Solar Cell Performance.  _The Journal of Physical Chemistry C_ , 128(11):4616–4630, March 2024.  ISSN 1932-7447, 1932-7455.  doi:10.1021/acs.jpcc.3c06361.  URL https://pubs.acs.org/doi/10.1021/acs.jpcc.3c06361.
  * Ahmadov et al. 2024 Rashad Ahmadov, Shane S. Michtavy, and Marc D. Porosoff.  Dual Functional Materials: At the Interface of Catalysis and Separations.  _Langmuir_ , 40(19):9833–9841, May 2024.  ISSN 0743-7463.  doi:10.1021/acs.langmuir.3c03888.  URL https://doi.org/10.1021/acs.langmuir.3c03888.  Publisher: American Chemical Society.
  * Fischer et al. 2019 Thomas Fischer, Silvia Gazzola, and Rainer Riedl.  Approaching Target Selectivity by De Novo Drug Design.  _Expert Opinion on Drug Discovery_ , 14(8):791–803, August 2019.  ISSN 1746-0441.  doi:10.1080/17460441.2019.1615435.  URL https://doi.org/10.1080/17460441.2019.1615435.  Publisher: Taylor & Francis _eprint: https://doi.org/10.1080/17460441.2019.1615435.
  * Wang et al. 2022a Zhuo Wang, Zhehao Sun, Hang Yin, Xinghui Liu, Jinlan Wang, Haitao Zhao, Cheng Heng Pang, Tao Wu, Shuzhou Li, Zongyou Yin, and Xue-Feng Yu.  Data-Driven Materials Innovation and Applications.  _Advanced Materials_ , 34(36):2104113, 2022a.  ISSN 1521-4095.  doi:10.1002/adma.202104113.  URL https://onlinelibrary.wiley.com/doi/abs/10.1002/adma.202104113.  _eprint: https://onlinelibrary.wiley.com/doi/pdf/10.1002/adma.202104113.
  * Sridharan et al. 2022 Bhuvanesh Sridharan, Manan Goel, and U. Deva Priyakumar.  Modern machine learning for tackling inverse problems in chemistry: molecular design to realization.  _Chemical Communications_ , 58(35):5316–5331, April 2022.  ISSN 1364-548X.  doi:10.1039/D1CC07035E.  URL https://pubs.rsc.org/en/content/articlelanding/2022/cc/d1cc07035e.  Publisher: The Royal Society of Chemistry.
  * Wu et al. 2018 Zhenqin Wu, Bharath Ramsundar, Evan N. Feinberg, Joseph Gomes, Caleb Geniesse, Aneesh S. Pappu, Karl Leswing, and Vijay Pande.  MoleculeNet: a benchmark for molecular machine learning.  _Chemical Science_ , 9(2):513–530, 2018.  ISSN 2041-6520, 2041-6539.  doi:10.1039/C7SC02664A.  URL http://xlink.rsc.org/?DOI=C7SC02664A.
  * Restrepo 2022 Guillermo Restrepo.  Chemical space: limits, evolution and modelling of an object bigger than our universal library.  _Digital Discovery_ , 1(5):568–585, October 2022.  ISSN 2635-098X.  doi:10.1039/D2DD00030J.  URL https://pubs.rsc.org/en/content/articlelanding/2022/dd/d2dd00030j.  Publisher: RSC.
  * Kirkpatrick and Ellis 2004 Peter Kirkpatrick and Clare Ellis.  Chemical space.  _Nature_ , 432(7019):823–824, 2004.
  * Mullard et al. 2017 Asher Mullard et al.  The drug-maker’s guide to the galaxy.  _Nature_ , 549(7673):445–447, 2017.
  * Llanos et al. 2019 Eugenio J Llanos, Wilmer Leal, Duc H Luu, Jürgen Jost, Peter F Stadler, and Guillermo Restrepo.  Exploration of the chemical space and its three historical regimes.  _Proceedings of the National Academy of Sciences_ , 116(26):12660–12665, 2019.
  * Schrier et al. 2023 Joshua Schrier, Alexander J. Norquist, Tonio Buonassisi, and Jakoah Brgoch.  In Pursuit of the Exceptional: Research Directions for Machine Learning in Chemical and Materials Science.  _Journal of the American Chemical Society_ , 145(40):21699–21716, October 2023.  ISSN 0002-7863.  doi:10.1021/jacs.3c04783.  URL https://doi.org/10.1021/jacs.3c04783.  Publisher: American Chemical Society.
  * Gromski et al. 2019 Piotr S Gromski, Alon B Henson, Jarosław M Granda, and Leroy Cronin.  How to explore chemical space using algorithms and automation.  _Nature Reviews Chemistry_ , 3(2):119–128, 2019.
  * Steiner et al. 2019 Sebastian Steiner, Jakob Wolf, Stefan Glatzel, Anna Andreou, Jarosław M Granda, Graham Keenan, Trevor Hinkley, Gerardo Aragon-Camarasa, Philip J Kitson, Davide Angelone, et al.  Organic synthesis in a modular robotic system driven by a chemical programming language.  _Science_ , 363(6423):eaav2211, 2019.
  * Burger et al. 2020 Benjamin Burger, Phillip M Maffettone, Vladimir V Gusev, Catherine M Aitchison, Yang Bai, Xiaoyan Wang, Xiaobo Li, Ben M Alston, Buyi Li, Rob Clowes, et al.  A mobile robotic chemist.  _Nature_ , 583(7815):237–241, 2020.
  * MacLeod et al. 2020 Benjamin P MacLeod, Fraser GL Parlane, Thomas D Morrissey, Florian Häse, Loïc M Roch, Kevan E Dettelbach, Raphaell Moreira, Lars PE Yunker, Michael B Rooney, Joseph R Deeth, et al.  Self-driving laboratory for accelerated discovery of thin-film materials.  _Science Advances_ , 6(20):eaaz8867, 2020.
  * Rumelhart et al. 1986 David E Rumelhart, Geoffrey E Hinton, and Ronald J Williams.  Learning internal representations by error propagation, parallel distributed processing, explorations in the microstructure of cognition, ed. de rumelhart and j. mcclelland. vol. 1. 1986.  _Biometrika_ , 71:599–607, 1986.
  * Hochreiter and Schmidhuber 1997 Sepp Hochreiter and Jürgen Schmidhuber.  Long short-term memory.  _Neural computation_ , 9(8):1735–1780, 1997.
  * Ribeiro et al. 2020 Antônio H. Ribeiro, Koen Tiels, Luis A. Aguirre, and Thomas B. Schön.  Beyond exploding and vanishing gradients: analysing RNN training using attractors and smoothness, March 2020.  URL http://arxiv.org/abs/1906.08482.  arXiv:1906.08482 [cs, math, stat].
  * Or 2023 Dr Barak Or.  The Exploding and Vanishing Gradients Problem in Time Series, December 2023.  URL https://medium.com/metaor-artificial-intelligence/the-exploding-and-vanishing-gradients-problem-in-time-series-6b87d558d22.
  * Vaswani et al. 2017 Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin.  Attention is all you need.  _arXiv [cs.CL]_ , June 2017.  URL http://arxiv.org/abs/1706.03762.
  * Gu and Dao 2023 Albert Gu and Tri Dao.  Mamba: Linear-time sequence modeling with selective state spaces.  _arXiv [cs.LG]_ , December 2023.  URL http://arxiv.org/abs/2312.00752.
  * Jelassi et al. 2024 Samy Jelassi, David Brandfonbrener, Sham M Kakade, and Eran Malach.  Repeat after me: Transformers are better than state space models at copying.  _arXiv [cs.LG]_ , February 2024.  URL http://arxiv.org/abs/2402.01032.
  * Peng et al. 2023 Bo Peng, Eric Alcaide, Quentin Anthony, Alon Albalak, Samuel Arcadinho, Huanqi Cao, Xin Cheng, Michael Chung, Matteo Grella, Kranthi Kiran GV, et al.  Rwkv: Reinventing rnns for the transformer era.  _arXiv preprint arXiv:2305.13048_ , 2023.
  * Beck et al. 2024 Maximilian Beck, Korbinian Pöppel, Markus Spanring, Andreas Auer, Oleksandra Prudnikova, Michael Kopp, Günter Klambauer, Johannes Brandstetter, and Sepp Hochreiter.  xLSTM: Extended Long Short-Term Memory, May 2024.  URL http://arxiv.org/abs/2405.04517.  arXiv:2405.04517 [cs, stat].
  * Minaee et al. 2024 Shervin Minaee, Tomas Mikolov, Narjes Nikzad, Meysam Chenaghlu, Richard Socher, Xavier Amatriain, and Jianfeng Gao.  Large language models: A survey.  _arXiv [cs.CL]_ , February 2024.  URL http://arxiv.org/abs/2402.06196.
  * ann 2022 The Annotated Transformer, 2022.  URL https://nlp.seas.harvard.edu/annotated-transformer/.
  * Bahdanau et al. 2016 Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio.  Neural Machine Translation by Jointly Learning to Align and Translate, May 2016.  URL http://arxiv.org/abs/1409.0473.  arXiv:1409.0473 [cs, stat].
  * Kudo 2018 Taku Kudo.  Subword regularization: Improving neural network translation models with multiple subword candidates.  _arXiv [cs.CL]_ , April 2018.  URL http://arxiv.org/abs/1804.10959.
  * Kudo and Richardson 2018 Taku Kudo and John Richardson.  SentencePiece: A simple and language independent subword tokenizer and detokenizer for neural text processing.  _arXiv [cs.CL]_ , August 2018.  URL http://arxiv.org/abs/1808.06226.
  * Song et al. 2020 Xinying Song, Alex Salcianu, Yang Song, Dave Dopson, and Denny Zhou.  Fast WordPiece tokenization.  _arXiv [cs.CL]_ , December 2020.  URL http://arxiv.org/abs/2012.15524.
  * Rust et al. 2020 Phillip Rust, Jonas Pfeiffer, Ivan Vulić, Sebastian Ruder, and Iryna Gurevych.  How good is your tokenizer? on the monolingual performance of multilingual language models.  _arXiv [cs.CL]_ , December 2020.  URL http://arxiv.org/abs/2012.15613.
  * Berglund and van der Merwe 2023 Martin Berglund and Brink van der Merwe.  Formalizing BPE tokenization.  _arXiv [cs.FL]_ , September 2023.  URL http://arxiv.org/abs/2309.08715.
  * Gehring et al. 2017 Jonas Gehring, Michael Auli, David Grangier, Denis Yarats, and Yann N Dauphin.  Convolutional sequence to sequence learning.  _arXiv [cs.CL]_ , pages 1243–1252, May 2017.  URL https://proceedings.mlr.press/v70/gehring17a.html.
  * Nair and Hinton 2010 Vinod Nair and Geoffrey E Hinton.  Rectified linear units improve restricted boltzmann machines.  In _Proceedings of the 27th international conference on machine learning (ICML-10)_ , pages 807–814, 2010.
  * Shazeer 2020 Noam Shazeer.  GLU Variants Improve Transformer, February 2020.  URL http://arxiv.org/abs/2002.05202.  arXiv:2002.05202 [cs, stat].
  * Hendrycks and Gimpel 2023 Dan Hendrycks and Kevin Gimpel.  Gaussian Error Linear Units (GELUs), June 2023.  URL http://arxiv.org/abs/1606.08415.  arXiv:1606.08415 [cs].
  * Stiennon et al. 2020 Nisan Stiennon, Long Ouyang, Jeffrey Wu, Daniel Ziegler, Ryan Lowe, Chelsea Voss, Alec Radford, Dario Amodei, and Paul F Christiano.  Learning to summarize with human feedback.  _Advances in Neural Information Processing Systems_ , 33:3008–3021, 2020.
  * Shen et al. 2023 Tianhao Shen, Renren Jin, Yufei Huang, Chuang Liu, Weilong Dong, Zishan Guo, Xinwei Wu, Yan Liu, and Deyi Xiong.  Large language model alignment: A survey.  _arXiv [cs.CL]_ , September 2023.  URL http://arxiv.org/abs/2309.15025.
  * Duan et al. 2016 Yan Duan, John Schulman, Xi Chen, Peter L Bartlett, Ilya Sutskever, and Pieter Abbeel.  RL2: Fast reinforcement learning via slow reinforcement learning.  _arXiv [cs.AI]_ , November 2016.  URL http://arxiv.org/abs/1611.02779.
  * Ziegler et al. 2019 Daniel M Ziegler, Nisan Stiennon, Jeffrey Wu, Tom B Brown, Alec Radford, Dario Amodei, Paul Christiano, and Geoffrey Irving.  Fine-tuning language models from human preferences.  _arXiv [cs.CL]_ , September 2019.  URL http://arxiv.org/abs/1909.08593.
  * Mazuz et al. 2023 Eyal Mazuz, Guy Shtar, Bracha Shapira, and Lior Rokach.  Molecule generation using transformers and policy gradient reinforcement learning.  _Sci. Rep._ , 13(1):8799, May 2023.  ISSN 2045-2322.  doi:10.1038/s41598-023-35648-w.  URL http://dx.doi.org/10.1038/s41598-023-35648-w.
  * Laskin et al. 2022 Michael Laskin, Luyu Wang, Junhyuk Oh, Emilio Parisotto, Stephen Spencer, Richie Steigerwald, D J Strouse, Steven Hansen, Angelos Filos, Ethan Brooks, Maxime Gazeau, Himanshu Sahni, Satinder Singh, and Volodymyr Mnih.  In-context reinforcement learning with algorithm distillation.  _arXiv [cs.LG]_ , October 2022.  URL http://arxiv.org/abs/2210.14215.
  * Ope 2022 Aligning language models to follow instructions.  https://openai.com/index/instruction-following, 2022.  URL https://openai.com/index/instruction-following.  Accessed: 2024-5-1.
  * Kim et al. 2023 Sungdong Kim, Sanghwan Bae, Jamin Shin, Soyoung Kang, Donghyun Kwak, Kang Min Yoo, and Minjoon Seo.  Aligning large language models through synthetic feedback.  _arXiv [cs.CL]_ , May 2023.  URL http://arxiv.org/abs/2305.13735.
  * Schulman et al. 2017 John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov.  Proximal policy optimization algorithms.  _arXiv [cs.LG]_ , July 2017.  URL http://arxiv.org/abs/1707.06347.
  * Zhang et al. 2020 Junzi Zhang, Jongho Kim, Brendan O’Donoghue, and Stephen Boyd.  Sample efficient reinforcement learning with REINFORCE.  _arXiv [cs.LG]_ , October 2020.  URL http://arxiv.org/abs/2010.11364.
  * Shinn et al. 2023 Noah Shinn, Federico Cassano, Edward Berman, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao.  Reflexion: Language agents with verbal reinforcement learning.  _arXiv [cs.AI]_ , March 2023.  URL http://arxiv.org/abs/2303.11366.
  * Akyurek et al. 2023 Afra Feyza Akyurek, Ekin Akyurek, Aman Madaan, Ashwin Kalyan, Peter Clark, Derry Wijaya, and Niket Tandon.  RL4F: Generating natural language feedback with reinforcement learning for repairing model outputs.  _arXiv [cs.CL]_ , May 2023.  URL http://arxiv.org/abs/2305.08844.
  * Cao et al. 2024 Yuji Cao, Huan Zhao, Yuheng Cheng, Ting Shu, Guolong Liu, Gaoqi Liang, Junhua Zhao, and Yun Li.  Survey on large language model-enhanced reinforcement learning: Concept, taxonomy, and methods.  _arXiv [cs.LG]_ , March 2024.  URL http://arxiv.org/abs/2404.00282.
  * Rafailov et al. 2023 Rafael Rafailov, Archit Sharma, Eric Mitchell, Stefano Ermon, Christopher D Manning, and Chelsea Finn.  Direct preference optimization: Your language model is secretly a reward model.  _arXiv [cs.LG]_ , May 2023.  URL http://arxiv.org/abs/2305.18290.
  * Xu et al. 2024 Shusheng Xu, Wei Fu, Jiaxuan Gao, Wenjie Ye, Weilin Liu, Zhiyu Mei, Guangju Wang, Chao Yu, and Yi Wu.  Is DPO superior to PPO for LLM alignment? a comprehensive study.  _arXiv [cs.CL]_ , April 2024.  URL http://arxiv.org/abs/2404.10719.
  * Bou et al. 2024 Albert Bou, Morgan Thomas, Sebastian Dittert, Carles Navarro Ramírez, Maciej Majewski, Ye Wang, Shivam Patel, Gary Tresadern, Mazen Ahmad, Vincent Moens, et al.  Acegen: Reinforcement learning of generative chemical agents for drug discovery.  _arXiv preprint arXiv:2405.04657_ , 2024.
  * Hayes et al. 2024 Thomas Hayes, Roshan Rao, Halil Akin, Nicholas James Sofroniew, Deniz Oktay, Zeming Lin, Robert Verkuil, Vincent Quy Tran, Jonathan Deaton, Marius Wiggert, Rohil Badkundri, Irhum Shafkat, Jun Gong, Alexander Derry, Raul Santiago Molina, Neil Thomas, Yousuf Khan, Chetan Mishra, Carolyn Kim, Liam J. Bartie, Patrick D. Hsu, Tom Sercu, Salvatore Candido, and Alexander Rives.  Simulating 500 million years of evolution with a language model.  _self-hosted preprint_ , 2024.  Preprint.
  * Devlin et al. 2018 Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova.  BERT: Pre-training of deep bidirectional Transformers for language understanding.  _arXiv [cs.CL]_ , October 2018.  doi:10.48550/arXiv.1810.04805.  URL http://arxiv.org/abs/1810.04805.
  * Taylor 1953 Wilson L. Taylor.  “Cloze Procedure”: A New Tool for Measuring Readability.  _Journalism Quarterly_ , 30(4):415–433, September 1953.  ISSN 0022-5533.  doi:10.1177/107769905303000401.  URL https://doi.org/10.1177/107769905303000401.  Publisher: SAGE Publications.
  * Liu et al. 2019a Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov.  RoBERTa: A Robustly Optimized BERT Pretraining Approach, July 2019a.  URL http://arxiv.org/abs/1907.11692.  arXiv:1907.11692 [cs].
  * Radford et al. 2018 Alec Radford, Karthik Narasimhan, Tim Salimans, and Ilya Sutskever.  Improving language understanding with unsupervised learning.  2018\.
  * Lewis et al. 2019 Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Ves Stoyanov, and Luke Zettlemoyer.  BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension, October 2019.  URL http://arxiv.org/abs/1910.13461.  arXiv:1910.13461 [cs, stat].
  * Raffel et al. 2019 Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu.  Exploring the limits of transfer learning with a unified text-to-text transformer.  _arXiv [cs.LG]_ , October 2019.  URL http://arxiv.org/abs/1910.10683.
  * Chung et al. 2022 Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus, Yunxuan Li, Xuezhi Wang, Mostafa Dehghani, Siddhartha Brahma, Albert Webson, Shixiang Shane Gu, Zhuyun Dai, Mirac Suzgun, Xinyun Chen, Aakanksha Chowdhery, Alex Castro-Ros, Marie Pellat, Kevin Robinson, Dasha Valter, Sharan Narang, Gaurav Mishra, Adams Yu, Vincent Zhao, Yanping Huang, Andrew Dai, Hongkun Yu, Slav Petrov, Ed H Chi, Jeff Dean, Jacob Devlin, Adam Roberts, Denny Zhou, Quoc V Le, and Jason Wei.  Scaling instruction-finetuned language models.  _arXiv [cs.LG]_ , October 2022.  URL http://arxiv.org/abs/2210.11416.
  * Tan et al. 2023 Bowen Tan, Yun Zhu, Lijuan Liu, Eric Xing, Zhiting Hu, and Jindong Chen.  Cappy: Outperforming and boosting large multi-task LMs with a small scorer.  _arXiv [cs.LG]_ , November 2023.  doi:10.48550/arXiv.2311.06720.  URL https://proceedings.neurips.cc/paper_files/paper/2023/file/b860c0c546f4a3a786f9c9468228c99f-Paper-Conference.pdf.
  * Shen et al. 2024a Junhong Shen, Neil Tenenholtz, James Brian Hall, David Alvarez-Melis, and Nicolo Fusi.  Tag-LLM: Repurposing general-purpose LLMs for specialized domains.  _arXiv [cs.LG]_ , February 2024a.  URL http://arxiv.org/abs/2402.05140.
  * Son et al. 2024 Guijin Son, Sangwon Baek, Sangdae Nam, Ilgyun Jeong, and Seungone Kim.  Multi-task inference: Can large language models follow multiple instructions at once?  _arXiv [cs.CL]_ , February 2024.  URL http://arxiv.org/abs/2402.11597.
  * Feng et al. 2024 Wenfeng Feng, Chuzhan Hao, Yuewei Zhang, Yu Han, and Hao Wang.  Mixture-of-LoRAs: An efficient multitask tuning for large language models.  _arXiv [cs.CL]_ , March 2024.  URL http://arxiv.org/abs/2403.03432.
  * FUY 2023 Fuyu-8B: A multimodal architecture for AI agents.  https://www.adept.ai/blog/fuyu-8b, 2023.  URL https://www.adept.ai/blog/fuyu-8b.  Accessed: 2023-11-8.
  * Wu et al. 2023a Shengqiong Wu, Hao Fei, Leigang Qu, Wei Ji, and Tat-Seng Chua.  NExT-GPT: Any-to-any multimodal LLM.  _arXiv [cs.AI]_ , September 2023a.  URL http://arxiv.org/abs/2309.05519.
  * Bolton et al. 2024 Elliot Bolton, Abhinav Venigalla, Michihiro Yasunaga, David Hall, Betty Xiong, Tony Lee, Roxana Daneshjou, Jonathan Frankle, Percy Liang, Michael Carbin, and Christopher D Manning.  BioMedLM: A 2.7B parameter language model trained on biomedical text.  _arXiv [cs.CL]_ , March 2024.  URL http://arxiv.org/abs/2403.18421.
  * Labrak et al. 2024 Yanis Labrak, Adrien Bazoge, Emmanuel Morin, Pierre-Antoine Gourraud, Mickael Rouvier, and Richard Dufour.  BioMistral: A collection of open-source pretrained large language models for medical domains.  _arXiv [cs.CL]_ , February 2024.  URL http://arxiv.org/abs/2402.10373.
  * Pieri et al. 2024 Sara Pieri, Sahal Shaji Mullappilly, Fahad Shahbaz Khan, Rao Muhammad Anwer, Salman Khan, Timothy Baldwin, and Hisham Cholakkal.  BiMediX: Bilingual medical mixture of experts LLM.  _arXiv [cs.CL]_ , February 2024.  URL http://arxiv.org/abs/2402.13253.
  * Zhao et al. 2024 Xuyang Zhao, Qibin Zhao, and Toshihisa Tanaka.  EpilepsyLLM: Domain-specific large language model fine-tuned with epilepsy medical knowledge.  _arXiv [cs.CL]_ , January 2024.  URL http://arxiv.org/abs/2401.05908.
  * Chen et al. 2024 Zhihong Chen, Maya Varma, Jean-Benoit Delbrouck, Magdalini Paschali, Louis Blankemeier, Dave Van Veen, Jeya Maria Jose Valanarasu, Alaa Youssef, Joseph Paul Cohen, Eduardo Pontes Reis, Emily B Tsai, Andrew Johnston, Cameron Olsen, Tanishq Mathew Abraham, Sergios Gatidis, Akshay S Chaudhari, and Curtis Langlotz.  CheXagent: Towards a foundation model for chest X-ray interpretation.  _arXiv [cs.CV]_ , January 2024.  URL http://arxiv.org/abs/2401.12208.
  * Luo et al. 2023 Yizhen Luo, Jiahuan Zhang, Siqi Fan, Kai Yang, Yushuai Wu, Mu Qiao, and Zaiqing Nie.  BioMedGPT: Open multimodal generative pre-trained transformer for BioMedicine.  _arXiv [cs.CE]_ , August 2023.  URL https://github.com/PharMolix/OpenBioMed.
  * Xie et al. 2023 Tong Xie, Yuwei Wan, Wei Huang, Zhenyu Yin, Yixuan Liu, Shaozhou Wang, Qingyuan Linghu, Chunyu Kit, Clara Grazian, Wenjie Zhang, Imran Razzak, and Bram Hoex.  DARWIN series: Domain specific large language models for natural science.  _arXiv [cs.CL]_ , August 2023.  URL http://arxiv.org/abs/2308.13565.
  * Wu et al. 2023b Chaoyi Wu, Weixiong Lin, Xiaoman Zhang, Ya Zhang, Yanfeng Wang, and Weidi Xie.  PMC-LLaMA: Towards building open-source language models for medicine.  _arXiv [cs.CL]_ , April 2023b.  URL http://arxiv.org/abs/2304.14454.
  * Taylor et al. 2022 Ross Taylor, Marcin Kardas, Guillem Cucurull, Thomas Scialom, Anthony Hartshorn, Elvis Saravia, Andrew Poulton, Viktor Kerkez, and Robert Stojnic.  Galactica: A large language model for science.  _arXiv [cs.CL]_ , November 2022.  URL http://arxiv.org/abs/2211.09085.
  * Luo et al. 2022 Renqian Luo, Liai Sun, Yingce Xia, Tao Qin, Sheng Zhang, Hoifung Poon, and Tie-Yan Liu.  BioGPT: generative pre-trained transformer for biomedical text generation and mining.  _Brief. Bioinform._ , 23(6), November 2022.  ISSN 1467-5463, 1477-4054.  doi:10.1093/bib/bbac409.  URL http://dx.doi.org/10.1093/bib/bbac409.
  * Livne et al. 2023 Micha Livne, Zulfat Miftahutdinov, Elena Tutubalina, Maksim Kuznetsov, Daniil Polykovskiy, Annika Brundyn, Aastha Jhunjhunwala, Anthony Costa, Alex Aliper, Alán Aspuru-Guzik, and Alex Zhavoronkov.  Nach0: Multimodal natural and chemical languages foundation model.  _arXiv [cs.CL]_ , November 2023.  URL http://arxiv.org/abs/2311.12410.
  * Pei et al. 2024 Qizhi Pei, Lijun Wu, Kaiyuan Gao, Xiaozhuan Liang, Yin Fang, Jinhua Zhu, Shufang Xie, Tao Qin, and Rui Yan.  BioT5+: Towards generalized biological understanding with IUPAC integration and multi-task tuning.  _arXiv [q-bio.QM]_ , February 2024.  URL http://arxiv.org/abs/2402.17810.
  * Christofidellis et al. 2023 Dimitrios Christofidellis, Giorgio Giannone, Jannis Born, Ole Winther, Teodoro Laino, and Matteo Manica.  Unifying molecular and textual representations via multi-task language modelling.  _arXiv [cs.LG]_ , January 2023.  URL http://arxiv.org/abs/2301.12586.
  * Edwards et al. 2022 Carl Edwards, Tuan Lai, Kevin Ros, Garrett Honke, Kyunghyun Cho, and Heng Ji.  Translation between molecules and natural language.  _arXiv [cs.CL]_ , April 2022.  URL http://arxiv.org/abs/2204.11817.
  * Gupta et al. 2022 Tanishq Gupta, Mohd Zaki, N M Anoop Krishnan, and Mausam.  MatSciBERT: A materials domain language model for text mining and information extraction.  _npj Computational Materials_ , 8(1):1–11, May 2022.  ISSN 2057-3960, 2057-3960.  doi:10.1038/s41524-022-00784-w.  URL https://www.nature.com/articles/s41524-022-00784-w.
  * Gu et al. 2021 Yu Gu, Robert Tinn, Hao Cheng, Michael Lucas, Naoto Usuyama, Xiaodong Liu, Tristan Naumann, Jianfeng Gao, and Hoifung Poon.  Domain-Specific language model pretraining for biomedical natural language processing.  _ACM Trans. Comput. Healthcare_ , 3(1):1–23, October 2021.  doi:10.1145/3458754.  URL https://doi.org/10.1145/3458754.
  * Beltagy et al. 2019 Iz Beltagy, Kyle Lo, and Arman Cohan.  SciBERT: A pretrained language model for scientific text.  _arXiv [cs.CL]_ , March 2019.  URL http://arxiv.org/abs/1903.10676.
  * Peng et al. 2019 Yifan Peng, Shankai Yan, and Zhiyong Lu.  Transfer learning in biomedical natural language processing: An evaluation of BERT and ELMo on ten benchmarking datasets.  _arXiv [cs.CL]_ , June 2019.  URL http://arxiv.org/abs/1906.05474.
  * Ock et al. 2023 Janghoon Ock, Chakradhar Guntuboina, and Amir Barati Farimani.  Catalyst Energy Prediction with CatBERTa: Unveiling Feature Exploration Strategies through Large Language Models.  _ACS CATALYSIS_ , 13(24):16032–16044, November 2023.  ISSN 2155-5435.  doi:10.1021/acscatal.3c04956.
  * Hong et al. 2022 Zhi Hong, Aswathy Ajith, Gregory Pauloski, Eamon Duede, Kyle Chard, and Ian Foster.  The diminishing returns of masked language models to science.  _arXiv [cs.CL]_ , May 2022.  URL http://arxiv.org/abs/2205.11342.
  * Huang et al. 2019 Kexin Huang, Jaan Altosaar, and Rajesh Ranganath.  ClinicalBERT: Modeling clinical notes and predicting hospital readmission.  _arXiv [cs.CL]_ , April 2019.  URL http://arxiv.org/abs/1904.05342.
  * Shetty et al. 2023 Pranav Shetty, Arunkumar Chitteth Rajan, Chris Kuenneth, Sonakshi Gupta, Lakshmi Prerana Panchumarti, Lauren Holm, Chao Zhang, and Rampi Ramprasad.  A general-purpose material property data extraction pipeline from large polymer corpora using natural language processing.  _NPJ Comput Mater_ , 9(1):52, April 2023.  ISSN 2057-3960.  doi:10.1038/s41524-023-01003-w.  URL http://dx.doi.org/10.1038/s41524-023-01003-w.
  * Trewartha et al. 2022 Amalie Trewartha, Nicholas Walker, Haoyan Huo, Sanghoon Lee, Kevin Cruse, John Dagdelen, Alexander Dunn, Kristin A Persson, Gerbrand Ceder, and Anubhav Jain.  Quantifying the advantage of domain-specific pre-training on named entity recognition tasks in materials science.  _Patterns (N Y)_ , 3(4):100488, April 2022.  ISSN 2666-3899.  doi:10.1016/j.patter.2022.100488.  URL http://dx.doi.org/10.1016/j.patter.2022.100488.
  * Guo et al. 2022 Jiang Guo, A Santiago Ibanez-Lopez, Hanyu Gao, Victor Quach, Connor W Coley, Klavs F Jensen, and Regina Barzilay.  Automated chemical reaction extraction from scientific literature.  _J. Chem. Inf. Model._ , 62(9):2035–2045, May 2022.  ISSN 1549-9596, 1549-960X.  doi:10.1021/acs.jcim.1c00284.  URL http://dx.doi.org/10.1021/acs.jcim.1c00284.
  * Shin et al. 2020 Hoo-Chang Shin, Yang Zhang, Evelina Bakhturina, Raul Puri, Mostofa Patwary, Mohammad Shoeybi, and Raghav Mani.  BioMegatron: Larger biomedical domain language model.  _arXiv [cs.CL]_ , October 2020.  URL http://arxiv.org/abs/2010.06060.
  * Lee et al. 2020 Jinhyuk Lee, Wonjin Yoon, Sungdong Kim, Donghyeon Kim, Sunkyu Kim, Chan Ho So, and Jaewoo Kang.  BioBERT: a pre-trained biomedical language representation model for biomedical text mining.  _Bioinformatics_ , 36(4):1234–1240, February 2020.  ISSN 1367-4803, 1367-4811.  doi:10.1093/bioinformatics/btz682.  URL http://dx.doi.org/10.1093/bioinformatics/btz682.
  * Pei et al. 2023 Qizhi Pei, Wei Zhang, Jinhua Zhu, Kehan Wu, Kaiyuan Gao, Lijun Wu, Yingce Xia, and Rui Yan.  BioT5: Enriching cross-modal integration in biology with chemical knowledge and natural language associations.  In Houda Bouamor, Juan Pino, and Kalika Bali, editors, _Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing_ , pages 1102–1123, Stroudsburg, PA, USA, December 2023. Association for Computational Linguistics.  doi:10.18653/v1/2023.emnlp-main.70.  URL https://aclanthology.org/2023.emnlp-main.70.
  * Born and Manica 2023 Jannis Born and Matteo Manica.  Regression Transformer enables concurrent sequence regression and generation for molecular language modelling.  _Nat. Mach. Intell._ , 5(4):432–444, April 2023.  ISSN 2522-5839,2522-5839.  doi:10.1038/s42256-023-00639-z.  URL https://www.nature.com/articles/s42256-023-00639-z.
  * Irwin et al. 2022 Ross Irwin, Spyridon Dimitriadis, Jiazhen He, and Esben Jannik Bjerrum.  Chemformer: a pre-trained transformer for computational chemistry.  _Machine Learning: Science and Technology_ , 3(1):015022, January 2022.  ISSN 2632-2153.  doi:10.1088/2632-2153/ac3ffb.  URL https://dx.doi.org/10.1088/2632-2153/ac3ffb.  Publisher: IOP Publishing.
  * Sagawa and Kojima 2023 Tatsuya Sagawa and Ryosuke Kojima.  ReactionT5: a large-scale pre-trained model towards application of limited reaction data.  _arXiv [physics.chem-ph]_ , November 2023.  URL http://arxiv.org/abs/2311.06708.
  * Lu and Zhang 2022 Jieyu Lu and Yingkai Zhang.  Unified Deep Learning Model for Multitask Reaction Predictions with Explanation.  _J. Chem. Inf. Model._ , 62(6):1376–1387, March 2022.  ISSN 1549-9596, 1549-960X.  doi:10.1021/acs.jcim.1c01467.  URL https://doi.org/10.1021/acs.jcim.1c01467.
  * Yuksel et al. 2023 Atakan Yuksel, Erva Ulusoy, Atabey Ünlü, and Tunca Doğan.  SELFormer: Molecular representation learning via SELFIES language models.  _arXiv [q-bio.QM]_ , April 2023.  URL http://arxiv.org/abs/2304.04662.
  * Ahmad et al. 2022 Walid Ahmad, Elana Simon, Seyone Chithrananda, Gabriel Grand, and Bharath Ramsundar.  ChemBERTa-2: Towards Chemical Foundation Models, September 2022.  URL http://arxiv.org/abs/2209.01712.  arXiv:2209.01712 [cs, q-bio].
  * Ross et al. 2022a Jerret Ross, Brian Belgodere, Vijil Chenthamarakshan, Inkit Padhi, Youssef Mroueh, and Payel Das.  Large-scale chemical language representations capture molecular structure and properties.  _Nature Machine Intelligence_ , 4(12):1256–1264, December 2022a.  ISSN 2522-5839, 2522-5839.  doi:10.1038/s42256-022-00580-7.  URL https://www.nature.com/articles/s42256-022-00580-7.
  * Li and Jiang 2021b Juncai Li and Xiaofei Jiang.  Mol-BERT: An effective molecular representation with BERT for molecular property prediction.  _Proc. Int. Wirel. Commun. Mob. Comput. Conf._ , 2021, September 2021b.  ISSN 1530-8669, 1530-8669.  doi:10.1155/2021/7181815.  URL https://www.hindawi.com/journals/wcmc/2021/7181815/.
  * Jia et al. 2021 Qinjin Jia, Jialin Cui, Yunkai Xiao, Chengyuan Liu, Parvez Rashid, and Edward F. Gehringer.  ALL-IN-ONE: Multi-Task Learning BERT models for Evaluating Peer Assessments, October 2021.  URL http://arxiv.org/abs/2110.03895.  arXiv:2110.03895 [cs].
  * Fabian et al. 2020 Benedek Fabian, Thomas Edlich, Héléna Gaspar, Marwin Segler, Joshua Meyers, Marco Fiscato, and Mohamed Ahmed.  Molecular representation learning with language models and domain-relevant auxiliary tasks.  _arXiv [cs.LG]_ , November 2020.  URL http://arxiv.org/abs/2011.13230.
  * Wang et al. 2019a Sheng Wang, Yuzhi Guo, Yuhong Wang, Hongmao Sun, and Junzhou Huang.  SMILES-BERT: Large Scale Unsupervised Pre-Training for Molecular Property Prediction.  In _Proceedings of the 10th ACM International Conference on Bioinformatics, Computational Biology and Health Informatics_ , BCB ’19, pages 429–436, New York, NY, USA, September 2019a. Association for Computing Machinery.  ISBN 978-1-4503-6666-3.  doi:10.1145/3307339.3342186.  URL https://doi.org/10.1145/3307339.3342186.
  * Honda et al. 2019 Shion Honda, Shoi Shi, and Hiroki R Ueda.  SMILES transformer: Pre-trained molecular fingerprint for low data drug discovery.  _arXiv [cs.LG]_ , November 2019.  URL http://arxiv.org/abs/1911.04738.
  * Lo et al. 2018 Yu-Chen Lo, Stefano E Rensi, Wen Torng, and Russ B Altman.  Machine learning in chemoinformatics and drug discovery.  _Drug discovery today_ , 23(8):1538–1546, 2018.
  * David et al. 2020 Laurianne David, Amol Thakkar, Rocío Mercado, and Ola Engkvist.  Molecular representations in AI-driven drug discovery: a review and practical guide.  _Journal of Cheminformatics_ , 12(1):56, September 2020.  ISSN 1758-2946.  doi:10.1186/s13321-020-00460-5.  URL https://doi.org/10.1186/s13321-020-00460-5.
  * Atz et al. 2021 Kenneth Atz, Francesca Grisoni, and Gisbert Schneider.  Geometric deep learning on molecular representations.  _arXiv [physics.chem-ph]_ , July 2021.  URL https://www.nature.com/articles/s42256-021-00418-8.
  * Walters and Barzilay 2021 W Patrick Walters and Regina Barzilay.  Applications of deep learning in molecule generation and molecular property prediction.  _Acc. Chem. Res._ , 54(2):263–270, January 2021.  ISSN 0001-4842,1520-4898.  doi:10.1021/acs.accounts.0c00699.  URL https://pubs.acs.org/doi/abs/10.1021/acs.accounts.0c00699.
  * Chen et al. 2019 Chi Chen, Weike Ye, Yunxing Zuo, Chen Zheng, and Shyue Ping Ong.  Graph networks as a universal machine learning framework for molecules and crystals.  _Chemistry of Materials_ , 31(9):3564–3572, 2019.
  * Hu et al. 2020 Weihua Hu, Matthias Fey, Marinka Zitnik, Yuxiao Dong, Hongyu Ren, Bowen Liu, Michele Catasta, and Jure Leskovec.  Open graph benchmark: Datasets for machine learning on graphs.  _Advances in neural information processing systems_ , 33:22118–22133, 2020.
  * Kearnes et al. 2016 Steven Kearnes, Kevin McCloskey, Marc Berndl, Vijay Pande, and Patrick Riley.  Molecular graph convolutions: moving beyond fingerprints.  _Journal of computer-aided molecular design_ , 30:595–608, 2016.
  * Wang et al. 2022b Yeji Wang, Shuo Wu, Yanwen Duan, and Yong Huang.  A point cloud-based deep learning strategy for protein–ligand binding affinity prediction.  _Briefings in Bioinformatics_ , 23(1):bbab474, 2022b.
  * Thomas et al. 2018 Nathaniel Thomas, Tess Smidt, Steven Kearnes, Lusann Yang, Li Li, Kai Kohlhoff, and Patrick Riley.  Tensor field networks: Rotation-and translation-equivariant neural networks for 3d point clouds.  _arXiv preprint arXiv:1802.08219_ , 2018.
  * Wang et al. 2021 Liguo Wang, Lu Zhao, Xian Liu, Jianjie Fu, and Aiqian Zhang.  Seppcnet: deeping learning on a 3d surface electrostatic potential point cloud for enhanced toxicity classification and its application to suspected environmental estrogens.  _Environmental Science & Technology_, 55(14):9958–9967, 2021.
  * Ahmadi et al. 2024 Soroush Ahmadi, Mohammad Amin Ghanavati, and Sohrab Rohani.  Machine learning-guided prediction of cocrystals using point cloud-based molecular representation.  _Chemistry of Materials_ , 2024.
  * Singh and Sunoj 2023 Sukriti Singh and Raghavan B. Sunoj.  Molecular Machine Learning for Chemical Catalysis: Prospects and Challenges.  _Accounts of Chemical Research_ , 56(3):402–412, February 2023.  ISSN 0001-4842.  doi:10.1021/acs.accounts.2c00801.  URL https://doi.org/10.1021/acs.accounts.2c00801.  Publisher: American Chemical Society.
  * Bran and Schwaller 2023 Andres M Bran and Philippe Schwaller.  Transformers and large language models for chemistry and drug discovery.  _arXiv [cs.LG]_ , October 2023.  URL http://arxiv.org/abs/2310.06083.
  * Shilpa et al. 2023 Shilpa Shilpa, Gargee Kashyap, and Raghavan B. Sunoj.  Recent Applications of Machine Learning in Molecular Property and Chemical Reaction Outcome Predictions.  _The Journal of Physical Chemistry A_ , 127(40):8253–8271, October 2023.  ISSN 1089-5639.  doi:10.1021/acs.jpca.3c04779.  URL https://doi.org/10.1021/acs.jpca.3c04779.  Publisher: American Chemical Society.
  * Wigh et al. 2022 Daniel S Wigh, Jonathan M Goodman, and Alexei A Lapkin.  A review of molecular representation in the age of machine learning.  _Wiley Interdiscip. Rev. Comput. Mol. Sci._ , 12(5), September 2022.  ISSN 1759-0876,1759-0884.  doi:10.1002/wcms.1603.  URL https://wires.onlinelibrary.wiley.com/doi/10.1002/wcms.1603.
  * O’Boyle and Dalke 2018 Noel O’Boyle and Andrew Dalke.  DeepSMILES: An Adaptation of SMILES for Use in Machine-Learning of Chemical Structures, September 2018.  URL https://chemrxiv.org/engage/chemrxiv/article-details/60c73ed6567dfe7e5fec388d.
  * Krenn et al. 2020 Mario Krenn, Florian Häse, AkshatKumar Nigam, Pascal Friederich, and Alán Aspuru-Guzik.  Self-Referencing Embedded Strings (SELFIES): A 100% robust molecular string representation.  _Machine Learning: Science and Technology_ , 1(4):045024, December 2020.  ISSN 2632-2153.  doi:10.1088/2632-2153/aba947.  URL http://arxiv.org/abs/1905.13741.  arXiv:1905.13741 [physics, physics:quant-ph, stat].
  * Heller et al. 2015 Stephen R. Heller, Alan McNaught, Igor Pletnev, Stephen Stein, and Dmitrii Tchekhovskoi.  InChI, the IUPAC International Chemical Identifier.  _Journal of Cheminformatics_ , 7(1):23, May 2015.  ISSN 1758-2946.  doi:10.1186/s13321-015-0068-4.  URL https://doi.org/10.1186/s13321-015-0068-4.
  * Das et al. 2024 Manajit Das, Ankit Ghosh, and Raghavan B Sunoj.  Advances in machine learning with chemical language models in molecular property and reaction outcome predictions.  _J. Comput. Chem._ , February 2024.  ISSN 0192-8651, 1096-987X.  doi:10.1002/jcc.27315.  URL http://dx.doi.org/10.1002/jcc.27315.
  * Irwin et al. 2012 John J Irwin, Teague Sterling, Michael M Mysinger, Erin S Bolstad, and Ryan G Coleman.  Zinc: a free tool to discover chemistry for biology.  _Journal of chemical information and modeling_ , 52(7):1757–1768, 2012.
  * Kim et al. 2016 Sunghwan Kim, Paul A Thiessen, Evan E Bolton, Jie Chen, Gang Fu, Asta Gindulyte, Lianyi Han, Jane He, Siqian He, Benjamin A Shoemaker, et al.  Pubchem substance and compound databases.  _Nucleic acids research_ , 44(D1):D1202–D1213, 2016.
  * Touvron et al. 2023 Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al.  Llama 2: Open foundation and fine-tuned chat models.  _arXiv preprint arXiv:2307.09288_ , 2023.
  * Gaulton et al. 2012 Anna Gaulton, Louisa J Bellis, A Patricia Bento, Jon Chambers, Mark Davies, Anne Hersey, Yvonne Light, Shaun McGlinchey, David Michalovich, Bissan Al-Lazikani, and John P Overington.  ChEMBL: a large-scale bioactivity database for drug discovery.  _Nucleic Acids Res._ , 40(Database issue):D1100–7, January 2012.  ISSN 0305-1048,1362-4962.  doi:10.1093/nar/gkr777.  URL http://dx.doi.org/10.1093/nar/gkr777.
  * Kinney et al. 2023 Rodney Kinney, Chloe Anastasiades, Russell Authur, Iz Beltagy, Jonathan Bragg, Alexandra Buraczynski, Isabel Cachola, Stefan Candra, Yoganand Chandrasekhar, Arman Cohan, Miles Crawford, Doug Downey, Jason Dunkelberger, Oren Etzioni, Rob Evans, Sergey Feldman, Joseph Gorney, David Graham, Fangzhou Hu, Regan Huff, Daniel King, Sebastian Kohlmeier, Bailey Kuehl, Michael Langan, Daniel Lin, Haokun Liu, Kyle Lo, Jaron Lochner, Kelsey MacMillan, Tyler Murray, Chris Newell, Smita Rao, Shaurya Rohatgi, Paul Sayre, Zejiang Shen, Amanpreet Singh, Luca Soldaini, Shivashankar Subramanian, Amber Tanaka, Alex D Wade, Linda Wagner, Lucy Lu Wang, Chris Wilhelm, Caroline Wu, Jiangjiang Yang, Angele Zamarron, Madeleine Van Zuylen, and Daniel S Weld.  The semantic scholar open data platform.  _arXiv [cs.DL]_ , January 2023.  URL http://arxiv.org/abs/2301.10140.
  * Segler et al. 2018c Marwin HS Segler, Mike Preuss, and Mark P Waller.  Planning chemical syntheses with deep neural networks and symbolic ai.  _Nature_ , 555(7698):604–610, 2018c.
  * _ja 2023 awesome-chemistry-datasets/code_of_conduct.md at main · kjappelbaum/awesome-chemistry-datasets, 2023.  URL https://github.com/kjappelbaum/awesome-chemistry-datasets/blob/main/code_of_conduct.md.
  * Mirza et al. 2024 Adrian Mirza, Nawaf Alampara, Sreekanth Kunchapu, Benedict Emoekabu, Aswanth Krishnan, Mara Wilhelmi, Macjonathan Okereke, Juliane Eberhardt, Amir Mohammad Elahi, Maximilian Greiner, Caroline T Holick, Tanya Gupta, Mehrdad Asgari, Christina Glaubitz, Lea C Klepsch, Yannik Koster, Jakob Meyer, Santiago Miret, Tim Hoffmann, Fabian Alexander Kreth, Michael Ringleb, Nicole Roesner, Ulrich S Schubert, Leanne M Stafast, Dinga Wonanke, Michael Pieler, Philippe Schwaller, and Kevin Maik Jablonka.  Are large language models superhuman chemists?  _arXiv [cs.LG]_ , April 2024.  URL http://arxiv.org/abs/2404.01475.
  * Gloriam 2019 David E Gloriam.  Bigger is better in virtual drug screens.  _Nature_ , 566(7743):193–194, February 2019.  ISSN 0028-0836,1476-4687.  doi:10.1038/d41586-019-00145-6.  URL http://dx.doi.org/10.1038/d41586-019-00145-6.
  * Liu et al. 2023a Shengchao Liu, Weili Nie, Chengpeng Wang, Jiarui Lu, Zhuoran Qiao, Ling Liu, Jian Tang, Chaowei Xiao, and Animashree Anandkumar.  Multi-modal molecule structure–text model for text-based retrieval and editing.  _Nature Machine Intelligence_ , 5(12):1447–1457, December 2023a.  ISSN 2522-5839.  doi:10.1038/s42256-023-00759-6.  URL https://www.nature.com/articles/s42256-023-00759-6.  Publisher: Nature Publishing Group.
  * Walters 2023 Pat Walters.  We need better benchmarks for machine learning in drug discovery, 2023.  URL http://practicalcheminformatics.blogspot.com/2023/08/we-need-better-benchmarks-for-machine.html.
  * Fang et al. 2023a Cheng Fang, Ye Wang, Richard Grater, Sudarshan Kapadnis, Cheryl Black, Patrick Trapa, and Simone Sciabola.  Prospective validation of machine learning algorithms for absorption, distribution, metabolism, and excretion prediction: An industrial perspective.  _J. Chem. Inf. Model._ , 63(11):3263–3274, June 2023a.  ISSN 1549-9596,1549-960X.  doi:10.1021/acs.jcim.3c00160.  URL https://pubs.acs.org/doi/full/10.1021/acs.jcim.3c00160.
  * 188 Therapeutics data commons.  https://tdcommons.ai/.  Accessed: 2024-6-13.
  * Huang et al. 2021 Kexin Huang, Tianfan Fu, Wenhao Gao, Yue Zhao, Yusuf Roohani, Jure Leskovec, Connor W Coley, Cao Xiao, Jimeng Sun, and Marinka Zitnik.  Therapeutics data commons: Machine learning datasets and tasks for drug discovery and development.  _arXiv [cs.LG]_ , February 2021.  URL http://arxiv.org/abs/2102.09548.
  * Velez-Arce et al. 2024 Alejandro Velez-Arce, Kexin Huang, Michelle M Li, Xiang Lin, Wenhao Gao, Tianfan Fu, Manolis Kellis, Bradley L Pentelute, and Marinka Zitnik.  TDC-2: Multimodal foundation for therapeutic science.  _bioRxiv_ , page 2024.06.12.598655, June 2024.  doi:10.1101/2024.06.12.598655.  URL https://www.biorxiv.org/content/10.1101/2024.06.12.598655v2.abstract.
  * Rich and Birnbaum 2023 Alex Rich and Ben Birnbaum.  Building adme benchmark datasets that drive impact, 2023.  URL https://www.inductive.bio/blog/building-better-benchmarks-for-adme-optimization.
  * Laurent et al. 2024 Jon M Laurent, Joseph D Janizek, Michael Ruzo, Michaela M Hinks, Michael J Hammerling, Siddharth Narayanan, Manvitha Ponnapati, Andrew D White, and Samuel G Rodriques.  LAB-bench: Measuring capabilities of language models for biology research.  _arXiv [cs.AI]_ , July 2024.  URL http://arxiv.org/abs/2407.10362.
  * Yoshitake et al. 2022 Michiko Yoshitake, Fumitaka Sato, Hiroyuki Kawano, and Hiroshi Teraoka.  MaterialBERT for natural language processing of materials science texts.  _Science and Technology of Advanced Materials: Methods_ , 2(1):372–380, December 2022.  ISSN 2766-0400.  doi:10.1080/27660400.2022.2124831.  URL https://doi.org/10.1080/27660400.2022.2124831.
  * Yu et al. 2022a Jiahui Yu, Chengwei Zhang, Yingying Cheng, Yun-Fang Yang, Yuan-Bin She, Fengfan Liu, Weike Su, and An Su.  SolvBERT for solvation free energy and solubility prediction: a demonstration of an NLP model for predicting the properties of molecular complexes.  July 2022a.  URL https://chemrxiv.org/engage/chemrxiv/article-details/62df4881a8e4dcc8f41cbadf.
  * Boobier et al. 2020 Samuel Boobier, David RJ Hose, A John Blacker, and Bao N Nguyen.  Machine learning with physicochemical relationships: solubility prediction in organic solvents and water.  _Nature communications_ , 11(1):5753, 2020.
  * Ross et al. 2022b Jerret Ross, Brian Belgodere, Vijil Chenthamarakshan, Inkit Padhi, Youssef Mroueh, and Payel Das.  Large-scale chemical language representations capture molecular structure and properties.  _Nat. Mach. Intell._ , 4(12):1256–1264, December 2022b.  ISSN 2522-5839,2522-5839.  doi:10.1038/s42256-022-00580-7.  URL http://dx.doi.org/10.1038/s42256-022-00580-7.
  * Huang and Cole 2022 Shu Huang and Jacqueline M. Cole.  BatteryBERT: A Pretrained Language Model for Battery Database Enhancement.  _Journal of Chemical Information and Modeling_ , 62(24):6365–6377, December 2022.  ISSN 1549-9596.  doi:10.1021/acs.jcim.2c00035.  URL https://doi.org/10.1021/acs.jcim.2c00035.  Publisher: American Chemical Society.
  * Maziarka et al. 2020 ukasz Maziarka, Tomasz Danel, Sławomir Mucha, Krzysztof Rataj, Jacek Tabor, and Stanisław Jastrzębski.  Molecule attention transformer.  _arXiv [cs.LG]_ , February 2020.  URL http://arxiv.org/abs/2002.08264.
  * Wang et al. 2019b Sheng Wang, Yuzhi Guo, Yuhong Wang, Hongmao Sun, and Junzhou Huang.  SMILES-BERT: Large Scale Unsupervised Pre-Training for Molecular Property Prediction.  In _Proceedings of the 10th ACM International Conference on Bioinformatics, Computational Biology and Health Informatics_ , BCB ’19, pages 429–436, Niagara Falls NY USA, September 2019b. Association for Computing Machinery.  ISBN 9781450366663.  doi:10.1145/3307339.3342186.  URL https://dl.acm.org/doi/10.1145/3307339.3342186.
  * Sultan et al. 2024 Afnan Sultan, Jochen Sieg, Miriam Mathea, and Andrea Volkamer.  Transformers for molecular property prediction: Lessons learned from the past five years.  _arXiv [cs.LG]_ , April 2024.  URL http://arxiv.org/abs/2404.03969.
  * Wu et al. 2023c Di Wu, Qi Chen, Xiaojie Chen, Feng Han, Zhong Chen, and Yi Wang.  The blood–brain barrier: structure, regulation, and drug delivery.  _Signal Transduction and Targeted Therapy_ , 8(1):1–27, May 2023c.  ISSN 2059-3635.  doi:10.1038/s41392-023-01481-w.  URL https://www.nature.com/articles/s41392-023-01481-w.  Publisher: Nature Publishing Group.
  * Bissantz et al. 2010 Caterina Bissantz, Bernd Kuhn, and Martin Stahl.  A medicinal chemist’s guide to molecular interactions.  _Journal of medicinal chemistry_ , 53(14):5061–5084, 2010.
  * Roughley and Jordan 2011 Stephen D Roughley and Allan M Jordan.  The medicinal chemist’s toolbox: an analysis of reactions used in the pursuit of drug candidates.  _Journal of medicinal chemistry_ , 54(10):3451–3479, 2011.
  * Doytchinova 2022 Irini Doytchinova.  Drug Design—Past, Present, Future.  _Molecules_ , 27(5):1496, February 2022.  ISSN 1420-3049.  doi:10.3390/molecules27051496.  URL https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8911833/.
  * 205 Introduction to Physical Polymer Science, 4th Edition | Wiley.  URL https://www.wiley.com/en-us/Introduction+to+Physical+Polymer+Science%2C+4th+Edition-p-9780471706069.
  * Newman and Cragg 2016 David J. Newman and Gordon M. Cragg.  Natural Products as Sources of New Drugs from 1981 to 2014.  _Journal of Natural Products_ , 79(3):629–661, March 2016.  ISSN 0163-3864.  doi:10.1021/acs.jnatprod.5b01055.  URL https://doi.org/10.1021/acs.jnatprod.5b01055.  Publisher: American Chemical Society.
  * Ferreira et al. 2023 Paulo Michel Pinheiro Ferreira, Daniel Dias Rufino Arcanjo, and Ana Paula Peron.  Drug development, Brazilian biodiversity and political choices: Where are we heading?  _Journal of Toxicology and Environmental Health, Part B_ , 26(5):257–274, July 2023.  ISSN 1093-7404.  doi:10.1080/10937404.2023.2193762.  URL https://doi.org/10.1080/10937404.2023.2193762.  Publisher: Taylor & Francis _eprint: https://doi.org/10.1080/10937404.2023.2193762.
  * Kolb and Sharpless 2003 Hartmuth C Kolb and K. Barry Sharpless.  The growing impact of click chemistry on drug discovery.  _Drug Discovery Today_ , 8(24):1128–1137, December 2003.  ISSN 1359-6446.  doi:10.1016/S1359-6446(03)02933-7.  URL https://www.sciencedirect.com/science/article/pii/S1359644603029337.
  * Castellino et al. 2023 Nathan J. Castellino, Andrew P. Montgomery, Jonathan J. Danon, and Michael Kassiou.  Late-stage Functionalization for Improving Drug-like Molecular Properties.  _Chemical Reviews_ , 123(13):8127–8153, July 2023.  ISSN 0009-2665.  doi:10.1021/acs.chemrev.2c00797.  URL https://doi.org/10.1021/acs.chemrev.2c00797.  Publisher: American Chemical Society.
  * Sharma et al. 2023 Komal Sharma, Krishna K. Sharma, Anku Sharma, and Rahul Jain.  Peptide-based drug discovery: Current status and recent advances.  _Drug Discovery Today_ , 28(2):103464, February 2023.  ISSN 1359-6446.  doi:10.1016/j.drudis.2022.103464.  URL https://www.sciencedirect.com/science/article/pii/S1359644622004573.
  * Reizman and Jensen 2016 Brandon J. Reizman and Klavs F. Jensen.  Feedback in Flow for Accelerated Reaction Development.  _Accounts of Chemical Research_ , 49(9):1786–1796, September 2016.  ISSN 0001-4842.  doi:10.1021/acs.accounts.6b00261.  URL https://doi.org/10.1021/acs.accounts.6b00261.  Publisher: American Chemical Society.
  * DiMasi et al. 2016 Joseph A. DiMasi, Henry G. Grabowski, and Ronald W. Hansen.  Innovation in the pharmaceutical industry: New estimates of R&D costs.  _Journal of Health Economics_ , 47:20–33, May 2016.  ISSN 1879-1646.  doi:10.1016/j.jhealeco.2016.01.012.
  * Lewars 2024 Errol G Lewars.  _Computational chemistry: Introduction to the theory and applications of molecular and quantum mechanics_.  Springer International Publishing, Cham, Switzerland, 4 edition, April 2024.  ISBN 9783031514425,9783031514432.  doi:10.1007/978-3-031-51443-2.  URL https://link.springer.com/book/10.1007/978-3-031-51443-2.
  * Brini et al. 2016 Emiliano Brini, S. Shanaka Paranahewage, Christopher J. Fennell, and Ken A. Dill.  Adapting the semi-explicit assembly solvation model for estimating water-cyclohexane partitioning with the SAMPL5 molecules.  _Journal of Computer-Aided Molecular Design_ , 30(11):1067–1077, November 2016.  ISSN 1573-4951.  doi:10.1007/s10822-016-9961-9.
  * Bidault and Chaudhuri 2023 Xavier Bidault and Santanu Chaudhuri.  How Accurate Can Crystal Structure Predictions Be for High-Energy Molecular Crystals?  _Molecules_ , 28(11):4471, January 2023.  ISSN 1420-3049.  doi:10.3390/molecules28114471.  URL https://www.mdpi.com/1420-3049/28/11/4471.  Number: 11 Publisher: Multidisciplinary Digital Publishing Institute.
  * Pyzer-Knapp et al. 2021 Edward O. Pyzer-Knapp, Linjiang Chen, Graeme M. Day, and Andrew I. Cooper.  Accelerating computational discovery of porous solids through improved navigation of energy-structure-function maps.  _Science Advances_ , 7(33):eabi4763, August 2021.  doi:10.1126/sciadv.abi4763.  URL https://www.science.org/doi/10.1126/sciadv.abi4763.  Publisher: American Association for the Advancement of Science.
  * Fredericks et al. 2021 Scott Fredericks, Kevin Parrish, Dean Sayre, and Qiang Zhu.  PyXtal: A Python library for crystal structure generation and symmetry analysis.  _Computer Physics Communications_ , 261:107810, April 2021.  ISSN 0010-4655.  doi:10.1016/j.cpc.2020.107810.  URL https://www.sciencedirect.com/science/article/pii/S0010465520304057.
  * Case et al. 2016 David H. Case, Josh E. Campbell, Peter J. Bygrave, and Graeme M. Day.  Convergence Properties of Crystal Structure Prediction by Quasi-Random Sampling.  _Journal of Chemical Theory and Computation_ , 12(2):910–924, February 2016.  ISSN 1549-9618.  doi:10.1021/acs.jctc.5b01112.  URL https://doi.org/10.1021/acs.jctc.5b01112.  Publisher: American Chemical Society.
  * Kazantsev et al. 2011 A. V. Kazantsev, P. G. Karamertzanis, C. S. Adjiman, and C. C. Pantelides.  Efficient Handling of Molecular Flexibility in Lattice Energy Minimization of Organic Crystals.  _Journal of Chemical Theory and Computation_ , 7(6):1998–2016, June 2011.  ISSN 1549-9618.  doi:10.1021/ct100597e.  URL https://doi.org/10.1021/ct100597e.  Publisher: American Chemical Society.
  * Huang et al. 2023 Guannan Huang, Yani Guo, Ye Chen, and Zhengwei Nie.  Application of Machine Learning in Material Synthesis and Property Prediction.  _Materials_ , 16(17):5977, January 2023.  ISSN 1996-1944.  doi:10.3390/ma16175977.  URL https://www.mdpi.com/1996-1944/16/17/5977.  Number: 17 Publisher: Multidisciplinary Digital Publishing Institute.
  * Martinez-Mayorga et al. 2024 Karina Martinez-Mayorga, José G. Rosas-Jiménez, Karla Gonzalez-Ponce, Edgar López-López, Antonio Neme, and José L. Medina-Franco.  The pursuit of accurate predictive models of the bioactivity of small molecules.  _Chemical Science_ , 15(6):1938–1952, February 2024.  ISSN 2041-6539.  doi:10.1039/D3SC05534E.  URL https://pubs.rsc.org/en/content/articlelanding/2024/sc/d3sc05534e.  Publisher: The Royal Society of Chemistry.
  * Wellawatte et al. 2023 Geemi P Wellawatte, Heta A Gandhi, Aditi Seshadri, and Andrew D White.  A PERSPECTIVE ON EXPLANATIONS OF MOLECULAR PREDICTION MODELS.  _J. Chem. Theory Comput._ , 19(8):2149–2160, April 2023.  ISSN 1549-9618, 1549-9626.  doi:10.1021/acs.jctc.2c01235.  URL http://dx.doi.org/10.1021/acs.jctc.2c01235.
  * Deng et al. 2023 Xiang Deng, Vasilisa Bashlovkina, Feng Han, Simon Baumgartner, and Michael Bendersky.  LLMs to the Moon? Reddit Market Sentiment Analysis with Large Language Models.  In _Companion Proceedings of the ACM Web Conference 2023_ , WWW ’23 Companion, pages 1014–1019, New York, NY, USA, April 2023. Association for Computing Machinery.  ISBN 978-1-4503-9419-2.  doi:10.1145/3543873.3587605.  URL https://dl.acm.org/doi/10.1145/3543873.3587605.
  * Schwaller et al. 2020a P Schwaller, Daniel Probst, A Vaucher, Vishnu H Nair, D Kreutter, T Laino, and J Reymond.  Mapping the space of chemical reactions using attention-based neural networks.  _Nat. Mach. Intell._ , 3(2):144–152, August 2020a.  ISSN 2522-5839,2522-5839.  doi:10.1038/s42256-020-00284-w.  URL https://www.nature.com/articles/s42256-020-00284-w.
  * Toniato et al. 2023 Alessandra Toniato, Alain C Vaucher, Philippe Schwaller, and Teodoro Laino.  Enhancing diversity in language based models for single-step retrosynthesis.  _Digital Discovery_ , 2(2):489–501, April 2023.  ISSN 2635-098X.  doi:10.1039/D2DD00110A.  URL https://pubs.rsc.org/en/content/articlelanding/2023/dd/d2dd00110a.
  * Schwaller et al. 2021a Philippe Schwaller, Benjamin Hoover, Jean-Louis Reymond, Hendrik Strobelt, and Teodoro Laino.  Extraction of organic chemistry grammar from unsupervised learning of chemical reactions.  _Science Advances_ , 7(15):eabe4166, April 2021a.  doi:10.1126/sciadv.abe4166.  URL https://www.science.org/doi/10.1126/sciadv.abe4166.  Publisher: American Association for the Advancement of Science.
  * Schwaller et al. 2021b Philippe Schwaller, Alain C Vaucher, Teodoro Laino, and Jean-Louis Reymond.  Prediction of chemical reaction yields using deep learning.  _Mach. Learn. Sci. Technol._ , 2(1):015016, March 2021b.  ISSN 2632-2153.  doi:10.1088/2632-2153/abc81d.  URL https://iopscience.iop.org/article/10.1088/2632-2153/abc81d/meta.
  * Zhang et al. 2022 Xiao-Chen Zhang, Cheng-Kun Wu, Jia-Cai Yi, Xiang-Xiang Zeng, Can-Qun Yang, Ai-Ping Lu, Hou T-j, Ting-Jun Hou, and Dong-Sheng Cao.  Pushing the Boundaries of Molecular Property Prediction for Drug Discovery with Multitask Learning BERT Enhanced by SMILES Enumeration.  _Research_ , 2022:0004, December 2022.  ISSN 2096-5168.  doi:10.34133/research.0004.  URL http://dx.doi.org/10.34133/research.0004.
  * Xiong et al. 2021 Guoli Xiong, Zhenxing Wu, Jiacai Yi, Li Fu, Zhijiang Yang, Changyu Hsieh, Mingzhu Yin, Xiangxiang Zeng, Chengkun Wu, Aiping Lu, Xiang Chen, Tingjun Hou, and Dongsheng Cao.  ADMETlab 2.0: an integrated online platform for accurate and comprehensive predictions of ADMET properties.  _Nucleic Acids Res._ , 49(W1):W5–W14, July 2021.  ISSN 0305-1048, 1362-4962.  doi:10.1093/nar/gkab255.  URL http://dx.doi.org/10.1093/nar/gkab255.
  * Sterling and Irwin 2015 Teague Sterling and John J. Irwin.  ZINC 15 – Ligand Discovery for Everyone.  _Journal of Chemical Information and Modeling_ , 55(11):2324–2337, November 2015.  ISSN 1549-9596.  doi:10.1021/acs.jcim.5b00559.  URL https://doi.org/10.1021/acs.jcim.5b00559.  Publisher: American Chemical Society.
  * Wolf et al. 2020 Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lhoest, and Alexander M. Rush.  Huggingface’s transformers: State-of-the-art natural language processing, 2020.
  * Yang et al. 2019b Kevin Yang, Kyle Swanson, Wengong Jin, Connor Coley, Philipp Eiden, Hua Gao, Angel Guzman-Perez, Timothy Hopper, Brian Kelley, Miriam Mathea, Andrew Palmer, Volker Settels, Tommi Jaakkola, Klavs Jensen, and Regina Barzilay.  Analyzing Learned Molecular Representations for Property Prediction.  _Journal of Chemical Information and Modeling_ , 59(8):3370–3388, August 2019b.  ISSN 1549-960X.  doi:10.1021/acs.jcim.9b00237.
  * Schwaller et al. 2019 Philippe Schwaller, Teodoro Laino, Théophile Gaudin, Peter Bolgar, Christopher A Hunter, Costas Bekas, and Alpha A Lee.  Molecular Transformer: A Model for Uncertainty-Calibrated Chemical Reaction Prediction.  _ACS Central Science_ , 5(9):1572–1583, September 2019.  ISSN 2374-7943, 2374-7951.  doi:10.1021/acscentsci.9b00576.  URL https://doi.org/10.1021/acscentsci.9b00576.
  * Vig 2019 Jesse Vig.  Bertviz: A tool for visualizing multihead self-attention in the bert model.  In _ICLR workshop: Debugging machine learning models_ , volume 3, 2019.
  * Skinnider 2024 Michael A. Skinnider.  Invalid SMILES are beneficial rather than detrimental to chemical language models.  _Nature Machine Intelligence_ , 6(4):437–448, April 2024.  ISSN 2522-5839.  doi:10.1038/s42256-024-00821-x.  URL https://www.nature.com/articles/s42256-024-00821-x.  Publisher: Nature Publishing Group.
  * Vermeire and Green 2021 Florence H Vermeire and William H Green.  Transfer learning for solvation free energies: From quantum chemistry to experiments.  _Chemical Engineering Journal_ , 418:129307, 2021.
  * Mobley and Guthrie 2014 David L Mobley and J Peter Guthrie.  FreeSolv: a database of experimental and calculated hydration free energies, with input files.  _J. Comput. Aided Mol. Des._ , 28(7):711–720, July 2014.  ISSN 0920-654X,1573-4951.  doi:10.1007/s10822-014-9747-x.  URL https://link.springer.com/article/10.1007/s10822-014-9747-x.
  * Marenich et al. 2020 Aleksandr V Marenich, Casey P Kelly, Jason D Thompson, Gregory D Hawkins, Candee C Chambers, David J Giesen, Paul Winget, Christopher J Cramer, and Donald G Truhlar.  Minnesota solvation database (mnsol) version 2012.  2020\.
  * Moine et al. 2017 Edouard Moine, Romain Privat, Baptiste Sirjean, and Jean-Noël Jaubert.  Estimation of solvation quantities from experimental thermodynamic data: Development of the comprehensive compsol databank for pure and mixed solutes.  _Journal of Physical and Chemical Reference Data_ , 46(3), 2017.
  * Grubbs et al. 2010 Laura M Grubbs, Mariam Saifullah, E Nohelli, Shulin Ye, Sai S Achi, William E Acree Jr, and Michael H Abraham.  Mathematical correlations for describing solute transfer into functionalized alkane solvents containing hydroxyl, ether, ester or ketone solvents.  _Fluid phase equilibria_ , 298(1):48–53, 2010.
  * Yang et al. 2019c Kevin Yang, Kyle Swanson, Wengong Jin, Connor Coley, Hua Gao, Angel Guzman-Perez, Timothy Hopper, Brian P Kelley, Andrew Palmer, Volker Settels, et al.  Are learned molecular representations ready for prime time?  2019c.  doi:10.26434/chemrxiv.7940594.v2.
  * Rong et al. 2020 Yu Rong, Yatao Bian, Tingyang Xu, Wei-Yang Xie, Ying Wei, Wen-Bing Huang, and Junzhou Huang.  Self-Supervised Graph Transformer on Large-Scale Molecular Data.  _Adv. Neural Inf. Process. Syst._ , 33:12559–12571, June 2020.  ISSN 1049-5258.  URL https://proceedings.neurips.cc/paper_files/paper/2020/hash/94aef38441efa3380a3bed3faf1f9d5d-Abstract.html.
  * Winter et al. 2022 Benedikt Winter, Clemens Winter, Johannes Schilling, and André Bardow.  A smile is all you need: predicting limiting activity coefficients from SMILES with natural language processing.  _Digital Discovery_ , 1(6):859–869, December 2022.  ISSN 2635-098X.  doi:10.1039/D2DD00058J.  URL https://pubs.rsc.org/en/content/articlelanding/2022/dd/d2dd00058j.  Publisher: RSC.
  * Jiang et al. 2024 Jing Jiang, Yachao Li, Ruisheng Zhang, and Yunwu Liu.  INTransformer: Data augmentation-based contrastive learning by injecting noise into transformer for molecular property prediction.  _Journal of Molecular Graphics and Modelling_ , 128:108703, May 2024.  ISSN 1093-3263.  doi:10.1016/j.jmgm.2024.108703.  URL https://www.sciencedirect.com/science/article/pii/S1093326324000032.
  * Chaves et al. 2024 Juan Manuel Zambrano Chaves, Eric Wang, Tao Tu, Eeshit Dhaval Vaishnav, Byron Lee, S Sara Mahdavi, Christopher Semturs, David Fleet, Vivek Natarajan, and Shekoofeh Azizi.  Tx-LLM: A large language model for therapeutics.  _arXiv [cs.CL]_ , June 2024.  URL http://arxiv.org/abs/2406.06316.
  * Yu et al. 2024 Botao Yu, Frazier N. Baker, Ziqi Chen, Xia Ning, and Huan Sun.  LlaSMol: Advancing Large Language Models for Chemistry with a Large-Scale, Comprehensive, High-Quality Instruction Tuning Dataset, April 2024.  URL http://arxiv.org/abs/2402.09391.  arXiv:2402.09391 [cs].
  * Kyro et al. 2024 Gregory W. Kyro, Anton Morgunov, Rafael I. Brent, and Victor S. Batista.  ChemSpaceAL: An Efficient Active Learning Methodology Applied to Protein-Specific Molecular Generation.  _Journal of Chemical Information and Modeling_ , 64(3):653–665, February 2024.  ISSN 1549-9596.  doi:10.1021/acs.jcim.3c01456.  URL https://doi.org/10.1021/acs.jcim.3c01456.  Publisher: American Chemical Society.
  * Frey et al. 2023 Nathan C. Frey, Ryan Soklaski, Simon Axelrod, Siddharth Samsi, Rafael Gómez-Bombarelli, Connor W. Coley, and Vijay Gadepally.  Neural scaling of deep chemical models.  _Nature Machine Intelligence_ , 5(11):1297–1305, November 2023.  ISSN 2522-5839.  doi:10.1038/s42256-023-00740-3.  URL https://www.nature.com/articles/s42256-023-00740-3.  Publisher: Nature Publishing Group.
  * Jablonka et al. 2024 Kevin Maik Jablonka, Philippe Schwaller, Andres Ortega-Guerrero, and Berend Smit.  Leveraging large language models for predictive chemistry.  _Nat. Mach. Intell._ , 6(2):161–169, February 2024.  ISSN 2522-5839,2522-5839.  doi:10.1038/s42256-023-00788-1.  URL https://www.nature.com/articles/s42256-023-00788-1.
  * Bagal et al. 2022 Viraj Bagal, Rishal Aggarwal, P K Vinod, and U Deva Priyakumar.  MolGPT: Molecular generation using a Transformer-Decoder model.  _J. Chem. Inf. Model._ , 62(9):2064–2076, May 2022.  ISSN 1549-9596, 1549-960X.  doi:10.1021/acs.jcim.1c00600.  URL http://dx.doi.org/10.1021/acs.jcim.1c00600.
  * Adilov 2021 Sanjar Adilov.  Generative Pre-Training from molecules.  _ChemRxiv_ , September 2021.  doi:10.26434/chemrxiv-2021-5fwjd.  URL https://chemrxiv.org/engage/chemrxiv/article-details/6142f60742198e8c31782e9e.
  * Hocky 2024 Glen M Hocky.  Connecting molecular properties with plain language.  _Nat. Mach. Intell._ , 6(3):249–250, March 2024.  ISSN 2522-5839,2522-5839.  doi:10.1038/s42256-024-00812-y.  URL https://www.nature.com/articles/s42256-024-00812-y.
  * Li and Vederas 2009 Jesse W.-H. Li and John C. Vederas.  Drug Discovery and Natural Products: End of an Era or an Endless Frontier?  _Science_ , 325(5937):161–165, July 2009.  doi:10.1126/science.1168243.  URL https://www.science.org/doi/abs/10.1126/science.1168243.  Publisher: American Association for the Advancement of Science.
  * Newman and Cragg 2012 David J. Newman and Gordon M. Cragg.  Natural Products As Sources of New Drugs over the 30 Years from 1981 to 2010.  _Journal of Natural Products_ , 75(3):311–335, March 2012.  ISSN 0163-3864.  doi:10.1021/np200906s.  URL https://doi.org/10.1021/np200906s.  Publisher: American Chemical Society.
  * A. Farha and D. Brown 2016 Maya A. Farha and Eric D. Brown.  Strategies for target identification of antimicrobial natural products.  _Natural Product Reports_ , 33(5):668–680, 2016.  doi:10.1039/C5NP00127G.  URL https://pubs.rsc.org/en/content/articlelanding/2016/np/c5np00127g.  Publisher: Royal Society of Chemistry.
  * Nigam et al. 2021 AkshatKumar Nigam, Robert Pollice, Mario Krenn, Gabriel dos Passos Gomes, and Alán Aspuru-Guzik.  Beyond generative models: superfast traversal, optimization, novelty, exploration and discovery (STONED) algorithm for molecules using SELFIES.  _Chemical Science_ , 12(20):7079–7090, 2021.  doi:10.1039/D1SC00231G.  URL https://pubs.rsc.org/en/content/articlelanding/2021/sc/d1sc00231g.  Publisher: Royal Society of Chemistry.
  * Gandhi and White 2022 Heta A Gandhi and Andrew D White.  Explaining molecular properties with natural language.  _ChemRxiv_ , October 2022.  doi:10.26434/chemrxiv-2022-v5p6m-v3.  URL https://chemrxiv.org/engage/chemrxiv/article-details/633731d1f764e6e535093041.
  * Du et al. 2024 Yuanqi Du, Arian R Jamasb, Jeff Guo, Tianfan Fu, Charles Harris, Yingheng Wang, Chenru Duan, Pietro Liò, Philippe Schwaller, and Tom L Blundell.  Machine learning-aided generative molecular design.  _Nat. Mach. Intell._ , pages 1–16, June 2024.  ISSN 2522-5839,2522-5839.  doi:10.1038/s42256-024-00843-5.  URL https://www.nature.com/articles/s42256-024-00843-5.
  * Houlsby et al. 2019 Neil Houlsby, Andrei Giurgiu, Stanislaw Jastrzebski, Bruna Morrone, Quentin De Laroussilhe, Andrea Gesmundo, Mona Attariyan, and Sylvain Gelly.  Parameter-efficient transfer learning for nlp.  In _International conference on machine learning_ , pages 2790–2799. PMLR, 2019.
  * Fuhr and Sumpter 2022 Addis S. Fuhr and Bobby G. Sumpter.  Deep generative models for materials discovery and machine learning-accelerated innovation.  _FRONTIERS IN MATERIALS_ , 9, MAR 22 2022.  ISSN 2296-8016.  doi:10.3389/fmats.2022.865270.
  * Han et al. 2023a Ri Han, Hongryul Yoon, Gahee Kim, Hyundo Lee, and Yoonji Lee.  Revolutionizing medicinal chemistry: The application of artificial intelligence (ai) in early drug discovery.  _PHARMACEUTICALS_ , 16(9), SEP 2023a.  doi:10.3390/ph16091259.
  * Koutroumpa et al. 2023 Nikoletta-Maria Koutroumpa, Konstantinos D. Papavasileiou, Anastasios G. Papadiamantis, Georgia Melagraki, and Antreas Afantitis.  A systematic review of deep learning methodologies used in the drug discovery process with emphasis on in vivo validation.  _INTERNATIONAL JOURNAL OF MOLECULAR SCIENCES_ , 24(7), APR 2023.  ISSN 1661-6596.  doi:10.3390/ijms24076573.
  * Kell et al. 2020 Douglas B. Kell, Soumitra Samanta, and Neil Swainston.  Deep learning and generative methods in cheminformatics and chemical biology: navigating small molecule space intelligently.  _BIOCHEMICAL JOURNAL_ , 477(23):4559–4580, DEC 2020.  ISSN 0264-6021.  doi:10.1042/BCJ20200781.
  * Bilodeau et al. 2022 Camille Bilodeau, Wengong Jin, Tommi Jaakkola, Regina Barzilay, and Klavs F. Jensen.  Generative models for molecular discovery: Recent advances and challenges.  _WILEY INTERDISCIPLINARY REVIEWS-COMPUTATIONAL MOLECULAR SCIENCE_ , 12(5), SEP 2022.  ISSN 1759-0876.  doi:10.1002/wcms.1608.
  * Gangwal et al. 2024 Amit Gangwal, Azim Ansari, Iqrar Ahmad, Abul Kalam Azad, Vinoth Kumarasamy, Vetriselvan Subramaniyan, and Ling Shing Wong.  Generative artificial intelligence in drug discovery: basic framework, recent advances, challenges, and opportunities.  _FRONTIERS IN PHARMACOLOGY_ , 15, FEB 7 2024.  doi:10.3389/fphar.2024.1331062.
  * Vogt 2022 Martin Vogt.  Using deep neural networks to explore chemical space.  _EXPERT OPINION ON DRUG DISCOVERY_ , 17(3):297–304, MAR 4 2022.  ISSN 1746-0441.  doi:10.1080/17460441.2022.2019704.
  * Talluri et al. 2024 Sekhar Talluri, Mohammad Amjad Kamal, and Rama Rao Malla.  Novel computational methods for cancer drug design.  _CURRENT MEDICINAL CHEMISTRY_ , 31(5):554–572, 2024.  ISSN 0929-8673.  doi:10.2174/0929867330666230403100008.
  * Polykovskiy et al. 2020 Daniil Polykovskiy, Alexander Zhebrak, Benjamin Sanchez-Lengeling, Sergey Golovanov, Oktai Tatanov, Stanislav Belyaev, Rauf Kurbanov, Aleksey Artamonov, Vladimir Aladinskiy, Mark Veselov, Artur Kadurin, Simon Johansson, Hongming Chen, Sergey Nikolenko, Alán Aspuru-Guzik, and Alex Zhavoronkov.  Molecular sets (MOSES): A benchmarking platform for molecular generation models.  _Front. Pharmacol._ , 11:565644, December 2020.  ISSN 1663-9812.  doi:10.3389/fphar.2020.565644.  URL https://www.frontiersin.org/journals/pharmacology/articles/10.3389/fphar.2020.565644/full.
  * Brown et al. 2019 Nathan Brown, Marco Fiscato, Marwin H S Segler, and Alain C Vaucher.  GuacaMol: Benchmarking models for de novo molecular design.  _J. Chem. Inf. Model._ , 59(3):1096–1108, March 2019.  ISSN 1549-9596, 1549-960X.  doi:10.1021/acs.jcim.8b00839.  URL http://dx.doi.org/10.1021/acs.jcim.8b00839.
  * Preuer et al. 2018 Kristina Preuer, Philipp Renz, Thomas Unterthiner, Sepp Hochreiter, and Gunter Klambauer.  Fréchet chemnet distance: a metric for generative models for molecules in drug discovery.  _Journal of chemical information and modeling_ , 58(9):1736–1741, 2018.
  * Haroon et al. 2023 Suhail Haroon, Hafsath C.a., and Jereesh A.s.  Generative Pre-trained Transformer (GPT) based model with relative attention for de novo drug design.  _Computational Biology and Chemistry_ , 106:107911, October 2023.  ISSN 1476-9271.  doi:10.1016/j.compbiolchem.2023.107911.  URL https://www.sciencedirect.com/science/article/pii/S1476927123001020.
  * Wang et al. 2023b Jianmin Wang, Jiashun Mao, Meng Wang, Xiangyang Le, and Yunyun Wang.  Explore drug-like space with deep generative models.  _Methods_ , 210:52–59, February 2023b.  ISSN 1046-2023.  doi:10.1016/j.ymeth.2023.01.004.  URL https://www.sciencedirect.com/science/article/pii/S1046202323000129.
  * Mao et al. 2024 Jiashun Mao, Jianmin Wang, Amir Zeb, Kwang-Hwi Cho, Haiyan Jin, Jongwan Kim, Onju Lee, Yunyun Wang, and Kyoung Tai No.  Transformer-Based Molecular Generative Model for Antiviral Drug Design.  _Journal of Chemical Information and Modeling_ , 64(7):2733–2745, April 2024.  ISSN 1549-9596.  doi:10.1021/acs.jcim.3c00536.  URL https://doi.org/10.1021/acs.jcim.3c00536.  Publisher: American Chemical Society.
  * Raffel et al. 2020 Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu.  Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer.  _Journal of Machine Learning Research_ , 21(140):1–67, 2020.  ISSN 1533-7928.  URL http://jmlr.org/papers/v21/20-074.html.
  * Mao et al. 2023 Jiashun Mao, Jianmin Wang, Kwang-Hwi Cho, and Kyoung Tai No.  iupacGPT: IUPAC-based large-scale molecular pre-trained model for property prediction and molecule generation.  _ChemRxiv_ , pages 1–13, May 2023.  doi:10.26434/chemrxiv-2023-5kjvh.  URL https://chemrxiv.org/engage/chemrxiv/public-dashboard.
  * Zhang et al. 2023a Wenyi Zhang, Kaiyue Zhang, and Jing Huang.  A Simple Way to Incorporate Target Structural Information in Molecular Generative Models.  _Journal of Chemical Information and Modeling_ , 63(12):3719–3730, June 2023a.  ISSN 1549-9596.  doi:10.1021/acs.jcim.3c00293.  URL https://doi.org/10.1021/acs.jcim.3c00293.  Publisher: American Chemical Society.
  * Wang et al. 2023c Xun Wang, Changnan Gao, Peifu Han, Xue Li, Wenqi Chen, Alfonso Rodríguez Patón, Shuang Wang, and Pan Zheng.  PETrans: De Novo Drug Design with Protein-Specific Encoding Based on Transfer Learning.  _International Journal of Molecular Sciences_ , 24(2):1146, January 2023c.  ISSN 1422-0067.  doi:10.3390/ijms24021146.  URL https://www.mdpi.com/1422-0067/24/2/1146.  Number: 2 Publisher: Multidisciplinary Digital Publishing Institute.