These authors contributed equally to this work.

These authors contributed equally to this work.

These authors contributed equally to this work.

[1]Kuangbiao Liao

1]Guangzhou National Laboratory, Guangzhou, Guangdong, PR China, 510005

2]AIChemEco Inc., Guangzhou, Guangdong, PR China, 510005

###### Abstract

The field of natural language processing (NLP) has witnessed a transformative
shift with the emergence of large language models (LLMs), revolutionizing
various language tasks and applications, and the integration of LLM into
specialized domains enhances their capabilities for domain-specific
applications. Notably, NLP has made significant strides in organic chemistry,
particularly in predicting synthetic tasks, paving the way for the development
of LLMs tailored to the organic chemistry field. In this work, we introduce
SynAsk, a comprehensive organic chemistry domain-specific LLM platform
developed by AIChemEco Inc. By finetuning an LLM with domain-specific data and
integrating it with a chain of thought approach, SynAsk seamlessly accesses
our knowledge base and advanced chemistry tools in a question-and-answer
format. This includes functionalities such as a basic chemistry knowledge
base, molecular information retrieval, reaction performance prediction,
retrosynthesis prediction, chemical literature acquisition, and more. This
novel methodology synergizes fine-tuning techniques with external resource
integration, resulting in an organic chemistry-specific model poised to
facilitate research and discovery in the field. Accessible via
https://synask.aichemeco.com, SynAsk represents a significant advancement in
leveraging NLP for synthetic applications.

###### keywords:

Large Language Model, AI in Chemistry, organic synthesis, retrosynthesis

##  1 Introduction

In recent years, the field of natural language processing (NLP) has undergone
a revolutionary shift with the emergence of large language models (LLMs),
advanced artificial intelligence systems trained on massive datasets to
understand and generate human-like text across various language tasks and
applications. At the core of LLMs lies the remarkable technology of generative
pre-trained transformers (GPT) [1]. Developed by OpenAI, GPT models like
ChatGPT [2] have gained widespread attention and adoption for their capacity
to produce coherent and contextually relevant text. ChatGPT, in particular,
represents a milestone in conversational AI, enabling human-like interactions
that go beyond scripted responses. Evolving from ChatGPT to GPT-4 [3] through
continual learning from vast datasets allows these models to grasp nuances of
language and context, making them versatile tools for diverse tasks, from
assisting in creative writing to generating videos. While GPT models have
dominated the landscape, other models like Qwen [4] and LLaMA [5] also make
significant contributions to the field, and these models are open-sourced for
the community to utilize. Qwen, primarily trained from Mandarin Chinese
language sources, is renowned for its robustness in question-answering tasks,
leveraging a different architecture and training approach. On the other hand,
LLaMA specializes in language understanding and inference tasks, offering
unique capabilities in semantic analysis and knowledge extraction.

Beyond ChatGPT and other models, LLMs encompass a spectrum of applications
across vertical domains. Domain-specific and customized data have been
collected and labeled to fine-tune these LLMs. One of the key benefits of
vertically specialized LLMs is their capacity to bolster domain-specific
applications. By refining their expertise within a particular domain, these
models possess the capability to delve deeply into the nuances of the subject
matter, rendering them invaluable tools for professionals operating in
specialized domains. For instance, a legally specialized LLM, namely DISC-
LawLLM [6], can provide precise legal counsel, draft contracts, and facilitate
intricate legal research, thereby streamlining processes and conserving
resources for legal practitioners. Similarly, a medically specialized LLM,
namely MultiMedQA [7], can assist physicians in diagnosing rare conditions,
proposing tailored treatment plan, and staying updated on the latest
technologies in medical research.

The integration of NLP into organic chemistry has brought about a revolution
in research and discovery. Molecules and reactions can now be represented
using SMILES (Simplified Molecular Input Line Entry System), a textual
notation for depicting high-dimensional chemical structures [8]. NLP
techniques have been employed to tackle organic synthesis tasks using SMILES
strings, treating the synthesis problem as a sequence generation task. This
approach involves training machine learning models to predict the sequence of
molecules and reactions necessary to synthesize a target molecule based on
desired products. These models learn from extensive datasets of annotated
reactions, where each reaction is represented as a sequence of SMILES strings.
Leveraging the patterns and rules encoded in the data, these models can
generate plausible synthesis pathways [9, 10].

LLMs have found applications in organic chemistry as well. However, without
further tuning with organic chemistry domain-specific data, researchers have
evaluated five LLMs in tasks related to organic chemistry, including reaction
prediction and retrosynthesis. While these models provide reasonable results
in classification or ranking tasks like yield prediction and reagent
selection, they face challenges in generative tasks that require a deep
understanding of molecular structures [11]. This difficulty may stem from the
highly experimental nature of organic chemistry, the lack of labeled data, and
the limited scope and applicability of computational tools in this field [12].
To bridge this gap and motivate further exploration of LLM potential in
chemistry, several domain-specific LLMs for organic chemistry have been
developed. ChemCrow [12] was the first proposed LLM in chemistry aimed at
enhancing its capabilities through external tools. It employs chain-of-thought
(CoT) strategies [13], which are a series of intermediate reasoning steps to
improve LLMs’ ability to understand tasks from prompts. ChemCrow also utilizes
LangChain[14], a framework to connect the LLM with multiple external tools
downstream to solve specific tasks and return answers back to the LLM.
However, this method relies on the reliability of tools, and general LLMs may
not comprehensively understand prompts and link to the correct tools to solve
specific tasks.Another approach, ChemLLM [15], was proposed to transform
structured chemical data into forms suitable for LLMs to fine-tune the LLaMA
model. ChemLLM excels in tasks such as cheminformatic programming. However,
its performance may not be as robust as comprehensive models like ChatGPT-4,
possibly due to human biases in the collection of incomplete structural
chemical data.

Figure 1: The overview of SynAsk platform.

We has long been dedicated to AI in chemistry research, developing a series of
machine learning and computational based tools to solve fundamental organic
chemistry tasks. However, we recognize that directly connecting these tools to
large language models (LLMs) may not yield appropriate results. Here we
introduce a comprehensive domain-specific LLM for organic chemistry developed
by AIChemEco, named SynAsk, as shown in Figure 1. An LLM was refined using a
limited set of domain-specific chemistry data and integrate it with a chain-
of-thought approach to understand user prompts. Our aim is to utilize
Langchain to seamlessly connect SynAsk with our existing suite of tools,
addressing specific user inquiries, drawing on the framework of
Langchain$-$chatchat [16]. This methodology allows us to combine fine-tuning
techniques with the integration of external resources, resulting in the
development of an organic chemistry-specific model. The model can be accessed
at https://synask.aichemeco.com.

##  2 Methods

To construct the comprehensive model integration platform, our approach
unfolds along three primary dimensions: utilizing a powerful foundation LLM as
the base for SynAsk, crafting more effective prompts and implementing fine-
tuning to the foundation model, and connecting with multiple tools to assemble
a chemistry domain-specific model platform.

###  2.1 Selection of a foundation LLM

Through various experiments, we have recognized that for the foundation LLM to
effectively understand prompts from end-users and apply insights to decide
whether to provide LLM inference answers or use specific tools to resolve
downstream tasks, it needs to have at least 14 billion parameters. Therefore,
only foundation models with over 14 billion parameters were considered. The
capabilities of the LLM was assessed using indicators such as Massive Multi-
task Language Understanding (MMLU) [17], Multi-level multi-discipline chinese
evaluation (C-Eval) [18], GSM8K [19], (BIG-Bench-Hard) BBH [20] and Measuring
massive multitask language understanding in Chinese (CMMLU) [21], as
elaborated in Section S1 of Electronic Supplementary Information (ESI). These
indicators collectively offer a comprehensive assessment of a model’s
proficiency, covering areas such as linguistic understanding, mathematical
reasoning, contextual comprehension, multi-modal integration, and the
application of Chain-of-Thought (CoT), which evaluates the fluency of LLMs’
integration with external tools. This evaluation framework underscores the
essential and diverse skills a model must possess to adeptly address complex
real-world problems.

As indicated in Table S1 [4], the Qwen series [4] outperforms other models
with equivalent parameter counts, including LLaMA2 [22], ChatGLM2 [23],
InterLM [24], Baichuan2 [25] and Yi [26] in these areas. Additionally, our
testing has confirmed that the Qwen series is more compatible with our
framework, especially with the release of Qwen-1.5, which provides us with
more options. We acknowledge that GPT series [2], particularly GPT-4 [3],
scores higher than Qwen. However, at the time of this work, GPT-4 has not been
open-sourced and requires paid API tokens to use as a foundation model. To
ensure SynAsk remains publicly accessible, we opted to use only open-sourced
foundation LLMs and developed an architecture that allows for smooth switching
of the foundation LLM, as discussed in Section 2.4.

###  2.2 Refinement to more Reasonable Prompt

To improve the model’s performance in two key areas—providing more targeted
responses in the chemical domain and enhancing its ability to efficiently
utilize tools—we refined our prompt templates through iterative testing and
adjustments. We guide the model to generate responses that are not only
accurate but also consistent with specific demand expectations. This process
encourages the model to become more deeply involved in the task at hand,
reducing ambiguity and focusing its attention. These optimized guidance models
function as both competent chemists and skilled tool users, establishing a
more focused, efficient, and effective interaction between the model and the
user.

In our integrated platform, utilizing the classification function of LLMs is
particularly crucial, as illustrated in Figure 2. Since this platform extends
from our existing NLP project, we believe it inherently possesses enhanced
capabilities. To further train it, we employ a tailored hint project, where
the model’s role is set as a chemist evaluating and scoring the generated
results. This project provides several examples to guide the model. This setup
enables the model to discern whether responses augmented by the knowledge
database meet the criteria, thereby classifying the results into those that
meet expectations and those that do not.

###  2.3 Fine-tuning of the LLM

The selected model underwent fine-tuning to specialize it further in the field
of chemistry, ensuring its engagement in professional chemical dialogues,
particularly in organic synthesis. The fine-tuning process comprised two
iterations, with data processed accordingly for each iteration.

  * •

The first iteration was supervised fine-tuning: This stage focused on
enhancing the model’s cognitive abilities, reinforcing its identity as an
expert in chemistry. The objective was to delve deeper into the model’s
capabilities within the chemistry domain without expanding its original data
source. This approach allowed the model to utilize existing data more
effectively to solve chemical problems.

  * •

The second iteration was instruction-based fine-tuning: The aim here was to
improve the model’s reasoning and tool invocation capabilities, thereby
enhancing its chain of thought. It learned to differentiate between various
types of chemical identifiers, such as SMILES and CAS numbers, rather than
treating them as ordinary words or sequences of numbers.

The rationale for dividing the fine-tuning into two stages is threefold:

  * •

Clear and Controllable Training: Each fine-tuning task addressed a specific
sub-problem, ensuring clarity and controllability in the training process and
outcomes. This approach facilitates adjustments and improvements based on the
results of previous fine-tuning, gradually enhancing the model’s performance
on specific tasks.

  * •

Prevention of Interference: Segregating the tasks prevents confusion and
interference between them. Combining all tasks into a single fine-tuning
session might lead to instability in training or reduced performance.

  * •

Accelerated Training: This approach speeds up the training process. By
simplifying each fine-tuning task, the training becomes more efficient,
yielding quicker results and feedback. The shorter training times for each
task contribute to a faster overall training cycle.

After fine-tuning, detailed techniques, procedures, and the necessary
equipment are elaborated in Section S2 of the ESI. Post-fine-tuning, our
emphasis mainly lies on the model’s ability to demonstrate Chain of Thought
(CoT) in its output. Following the fine-tuning process, we provide two
examples of the model’s simplified output format:
Prompt: What is the SMILES of toluene? Response:

    Action: get_SMILES
    Action Input: {"query": "toluene"}

Prompt: What is the name of CC1=CC=CC=C1? Response:

    Action: CAStoName
    Action Input: {"query": "CC1=CC=CC=C1"}

Notably, the power of these fine-tuned results is significantly enhanced when
used in conjunction with appropriately designed prompting strategies and
specially designed tool formats. These responses demonstrate the model’s
ability to identify the required action and its corresponding input from the
prompts. However, within our framework, these responses are not the final
outcome. Instead, they serve as intermediate prompts to be re-fed into the
model. This intermediary step is pivotal, enabling the model to discern the
specific tool it requires (e.g., ‘get_SMILES’ for the initial example) and to
process the "Action Input" (e.g., "query: ’toluene’") utilizing the designated
tool. Subsequently, the expansive model amalgamates the tool’s output with its
vast knowledge base, culminating in the generation of a final answer.

###  2.4 SynAsk Architecture

In the final phase, we implemented the LangChain framework to seamlessly
integrate our local knowledge base with both internal and external open-source
tools and APIs. Its primary role is to interpret the outputs from the language
models, converting them into a format understandable by external tools, thus
facilitating the execution of corresponding actions. Simultaneously, it
translates the responses from these tools back into a form comprehensible by
the language models. Furthermore, LangChain’s support for context management
enables it to track the interaction history between users and the system. This
enhances the system’s ability to understand user intentions and maintain
session continuity during interactions with external tools. Its scalability
ensures that the system can adapt to technological advancements and changing
user demands, providing a dynamic and responsive framework for our integration
needs. The LangChain framework serves as a pivotal bridge, culminating in a
logically coherent and systematically robust integration platform known as
SynAsk.

The structural framework of SynAsk is illustrated in Figure 2. Initially, it
can accept both voice and text inputs as queries, which are then segmented
into multiple tasks by a LLM and matched against our knowledge base. At this
stage, users also have the option to upload their local files as supplementary
knowledge or directly engage in conversations with the uploaded files. Once
matching texts are obtained, the large model synthesizes the content along
with its understanding of the question to deduce a conclusion, thereby
generating a result. Subsequently, the model evaluates this result to
determine if it meets the expected criteria. If the outcome is deemed
satisfactory, it is directly outputted as the Final Answer. Conversely, if the
results do not meet expectations, we will enter our customized Agent Q&A mode
and call our tools to answer. Finally, the tool output is combined with the
LLM’s self-knowledge to generate the final answer.

Figure 2: The workflow of the SynAsk platform: from the input to the final
answer.

In the SynAsk architecture, although we currently utilize Qwen-1.5 as the
foundation LLM, we recognize the ongoing revolutions in LLM technology.
Consequently, we have developed a workflow to swiftly adjust the foundation
model and fine-tune the domain-specific data. This approach ensures that
SynAsk can continuously update and iterate, leveraging the latest advancements
in foundation LLMs.

###  2.5 SynAsk Toolsets

Chemoinformatic tools are seamlessly connected with SynAsk through LangChain
to provide comprehensive organic synthesis answers. This includes a variety of
machine learning-powered tools developed both internally and by external
teams, all dedicated to solving organic synthesis tasks. At the time of
publishing this work, 12 internal tools and 10 external tools have been
integrated into SynAsk. External tools are appropriately cited with their
origins. With the rapid development of this field, we anticipate an increasing
influx of tools joining SynAsk. These tools are categorized into molecular,
reaction tools, and others, with a number of advanced in-house tools
elaborated in Section 2.5.5.

####  2.5.1 Molecular Information Retrieval

This category encompasses tools designed for querying various molecular
identifiers and properties. Functions include retrieving Chemical Abstracts
Service (CAS) numbers, Simplified Molecular Input Line Entry System (SMILES)
strings, molecular weights, assessing molecular similarity, identifying types
of functional groups, and checking the regulatory status of molecules. The
respective tools for these purposes are:

  * •

get_cas – for CAS numbers retrieval [27]

  * •

get_smiles – for obtaining SMILES strings [27]

  * •

CAStoName – to convert CAS numbers to chemical names [28]

  * •

SMILEStoName – to convert SMILES strings to chemical names[28]

  * •

get_mol_weight – for calculating molecular weights

  * •

get_mol_similarity – to determine molecular similarity

  * •

check_functional_groups – for functional group identification

  * •

ControlmolCheck – to check if a molecule is controlled

####  2.5.2 Chemical Reaction and Retrosynthesis

This category aids in querying chemical reaction conditions, planning chemical
reaction pathways, predicting chemical reaction yields, performing
retrosynthetic analysis, and predicting reaction derivatives. Tools provided
for these functions include:

  * •

Get_condition – to query chemical reaction conditions

  * •

ReactionPlanner – for planning chemical reaction pathways [29]

  * •

ReagentsPredict – to predict reagents in chemical reactions

  * •

YieldPredict – for predicting chemical reaction yields

  * •

Retrosynthesis – to perform retrosynthetic analysis

  * •

DerivatePredict – to predicts the derivatives from a chemical reaction, using
reactants’ names or SMILES, enhancing the exploration of reaction outcomes.

  * •

AutoMapping – to identify the position of each atom in the molecules before
and after a chemical reaction [30][31]

####  2.5.3 Acquisition of Chemical Literature and Knowledge

Dedicated to acquiring chemical literature and extracting chemical knowledge,
tools in this section include:

  * •

Get_literature – for retrieving literature [32][33]

  * •

get_knowledge – to obtain chemical knowledge [33]

  * •

Rxn_literature – for sourcing reaction-specific literature

####  2.5.4 Miscellaneous

This section covers a diverse array of functions including drawing chemical
molecular structures and balancing chemical equations. Tools include:

  * •

Moldraw – for drawing chemical molecular structures

  * •

calculate – a general-purpose calculation tool

  * •

automatic_balance – to automatically balance chemical equations [34]

  * •

image_gen – for generating and searching images [33]

####  2.5.5 Advanced In-House Analytical Tools

##### YieldPredict

This is an API tool linked with our self-developed reaction yield prediction
tool. By inputting at least two substrates, either in their molecular name or
molecular SMILES, this tool can identify the possible reaction types of the
molecules by querying our reaction template library. With the known reaction
types, the molecules are passed into the reaction models as substrates. The
models then suggest products and the most suitable reaction reagents and
conditions for the substrates. For example, by asking the reaction yield of
triethoxy(naphthalen-1-yl)silane and 5-bromobenzothiazole, the tool first
parses the two molecules into the reaction templates as substrates Figure 3.
This suggests Hiyama cross-coupling reactions. The two substrates are then
parsed into the Hiyama reaction models, generating products and possible
reaction yields under specific reaction reagents and conditions.

Figure 3: An example of the YieldPredict tool workflow for predicting the
reaction yield of triethoxy(naphthalen-1-yl)silane and 5-bromobenzothiazole:
(a) the user interface of SynAsk, (b) the thinking process of the YieldPredict
tool.

We have dedicated our efforts to developing data-driven reaction yield
prediction models for common reaction types [35, 36, 37, 38]. For each model
of a specific reaction type, we conduct chemical reaction experiments using
high-throughput experimentation (HTE) techniques with various substrates. This
enables us to draw insights from existing literature data and identify areas
where experimental data collection is necessary to augment an equitable data
space for refining model training, thus facilitating more robust
interpolation. We develop reaction models using machine learning techniques
such as support vector machine (SVM) and NLP deep learning models like BERT
(Bidirectional Encoder Representations from Transformers) [39]. These models
are validated using external literature test data, achieving reasonable Mean
Absolute Error (MAE), commonly below 0.15. As of the publication of this work,
we have included 18 reaction types in this tool.

##### Get_Conditions

This tool is a simplified version of YieldPredict. Instead of predicting the
reaction product and yield, it provides rapid responses and suggests only the
suitable reaction conditions and reagents for the substrates.

##### Retrosynthesis

By inputting the desired target products, this tool generates numerous
reaction pathways of molecules starting from buyable precursors. We have
developed our own retrosynthesis model for this purpose. For a desired
product, it is parsed into the reaction template library to find possible
substrates and, consequently, the suitable reaction site for bond breakage. A
reinforcement learning-trained agent selects the most suitable reaction from
the candidates based on the forecasted synthesis difficulty and predicted
reaction yield of the substrates (desired products at the previous step). This
process is conducted recursively until the last substrates are buyable. At the
output, we present the results in both textual form and as retrosynthetic
route images. The algorithm of our retrosynthesis model will be published
elsewhere.

##  3 SynAsk Performance

We evaluate the performance of SynAsk from two perspectives: its general
ability as a large language model (LLM), and its proficiency in synthetic
chemistry. Additionally, we provide several examples of SynAsk’s outputs to
demonstrate the platform’s comprehension capabilities.

###  3.1 General ability of SynAsk

We evaluate the performance enhancements achieved through our first fine-
tuning method on the SynAsk model based on OpenCompass [40], which serves as a
universal evaluation platform for foundation LLMs. The efficacy of the method
is demonstrated by its superior scores across various assessment indicators,
particularly in its application to chemistry. The definitions of the general
indicators used in Figure 4 are provided in Section S1 of the ESI, while the
chemistry-related indicators are outlined in Section S3 along with examples.
It’s noteworthy that indicators such as College Chemistry, High School
Chemistry, and Middle School Chemistry in the figure all stem from C-Eval..
SynAsk significantly outperforms its foundation model predecessors. For
example, in the area of College Chemistry, SynAsk achieves a remarkable score
of 70.83%, compared to 50% by both Qwen-14B-Chat and Qwen1.5-14B-Chat. This
signifies a substantial improvement, highlighting the model’s enhanced ability
to effectively utilize existing data sources for solving complex chemical
problems..

Figure 4: The comparison of general ability between SynAsk and Qwen in seven
aspects, including its applications in chemistry.

Furthermore, the scores in other key benchmarks such as MMLU, GSM8K and CMMLU
also reflect the overall enhancement of the SynAsk model. In CMMLU, which
assesses cross-model multitask learning, SynAsk scored 75.03%, indicating its
proficiency in integrating textual and visual information, crucial for multi-
model applications. Similarly, its performance in MMLU and GSM8K benchmarks
demonstrate its improved global knowledge comprehension and multi-step
mathematical reasoning, respectively.

The advancements in SynAsk are attributed to the fine-tuning approach that
leverages existing data sources more efficiently, thus enhancing the model’s
ability to address nuanced chemical contexts and complex reasoning tasks. This
is particularly crucial for applications requiring deep understanding and
contextual awareness, as indicated by the improvements in C-Eval scores.

These results collectively underscore the effectiveness of our fine-tuning
methodology, confirming its potential to significantly boost performance
across diverse linguistic and cognitive challenges, thereby reinforcing the
model’s utility in academic and practical applications.

###  3.2 Proficiency in synthetic chemistry

The primary proficiency of SynAsk in synthetic chemistry lies in its ability
to predict reaction performance, such as yield, and to conduct retrosynthetic
planning of target molecules, utilizing the embedded tools within SynAsk.
Several case studies are presented and compared with benchmarks to evaluate
the model’s performance. Additionally, the other functions of SynAsk are
compared with ChatGPT-4.0 answers to highlight its advancements in various
areas.

####  3.2.1 Reaction yield prediction

A number of reaction yield prediction models have been developed and widely
used to forecast the performance of reactions for frequently encountered
reaction classes. For instance, Doyle et al.’s palladium-catalysed
Buchwald–Hartwig cross-coupling reaction model [41] and Richardson et al.’s
Suzuki-Miyaura cross-coupling reaction model [42] are among the notable
examples. These models were trained using self-developed high-throughput
experimentation (HTE) reaction data employing machine learning algorithms.
Schwaller et al. [43] further enhanced the performance of these models using
the same datasets through pre-trained BERT model. While these methods
effectively predict the product yield within the self-developed HTE reaction
dataset, their applicability to predicting the product yield of external
literature recorded reactions may be limited.

We tested our in-house nucleophilic aromatic substitution (SNAr) reaction
model embedded in SynAsk with tboth a test set and external literature
reaction data. The model test set comprises unseen HTE reaction data, yielding
a mean absolute error (MAE) of 11.5%. For the external literature reaction
data, to minimize bias, we randomly collected 60 recently published SNAr
reactions from the last three years (2022-2024), including new substrate
molecules never seen by the reaction model. The comparison between the model-
predicted yield and literature-reported yield is presented in Figure 5b,
yielding an MAE of 14.1%. These recent published reactions encompass seven
different reaction conditions. For example, N-methyl-1-phenylmethanamine
reacting with 2-fluoro-5-methoxybenzaldehyde under K2CO3 and DMF is
illustrated in Figure 5c. The literature-reported yield of the product
2-(benzyl(methyl)amino)-5-methoxybenzaldehyde is 75% [44], whilst our model
predicts 80% and our HTE experimental yield is 70%. Additional results are
provided in Section S4 of the ESI.

The decay in prediction accuracy observed when transitioning from HTE
reactions to literature-reported reactions is primarily attributed to the
increased complexity of substrates in literature reactions. These substrates
are often more intricate and unseen by the model, thereby encompassing a wider
range within the chemical space, as depicted in Figure 5a. To compute the
chemical space, we digitized the reactions using RXNFP pretrained reaction
fingerprint [45] and reduced into two dimensions. Figure 5a also weakly show
three clusters of the SNAr reaction. Despite the challenges posed by the
complexity of literature-reported reactions, our in-house SNAr model
demonstrates the capability to predict the reaction performance of these
reactions. This is particularly valuable as it enables predictions closer to
the reactions of interest to synthetic chemists.

Figure 5: The SNAr reaction model results: (a) the chemical space of SNAr
reactions under the HTE and literature recorded datasets, (b) the predicted
yield versus experimental yield of the test dataset from the three different
models, and (c) an example of SNAr reaction: N-methyl-1-phenylmethanamine
reacting with 2-fluoro-5-methoxybenzaldehyde.

####  3.2.2 Retrosynthetic route planning

We tasked SynAsk with planning retrosynthetic routes for 11,549 small molecule
drugs recorded in the ChEMBL database [46]. SynAsk successfully predicted
retrosynthetic routes for 6,358 molecules, suggesting step-by-step routes
starting from buyable precursors. This accounts for 55% of the queried
molecules. In contrast, the State of the Art (SOTA) open-sourced
retrosynthetic planning tool, AIZynthFinder [47], only suggested 3,118
retrosynthetic routes, covering 27% of the queried molecules.

As a case study, let’s consider the retrosynthesis of Gilmelisib, a novel
small molecule under investigation as a selective inhibitor of PIK3C$\alpha$,
potentially treating cancers characterized by PIK3C$\alpha$ mutations. SynAsk
proposes a seven-step synthetic route with four precursors (as shown in Figure
6a), matching the route suggested by an experienced human chemist in terms of
length and number of precursors (as shown in Figure 6b). SynAsk utilizes
inexpensive precursors to rapidly obtain key heterocyclic fused ring
intermediates through straightforward Knoevenagel condensation and addition-
elimination reactions. In contrast, AIZynthFinder fails to provide a synthesis
route for the target molecule, even after enriching its starting materials
with our lists of buyable precursors. Additional synthetic routes for small
molecule drugs are detailed in Section S5 of the ESI.

While we refrain from concluding that SynAsk is smarter or approaching the
intelligence of a human chemist in retrosynthesis, as this determination would
necessitate passing the Turing test [48, 49] or experimental validation, we
acknowledge that SynAsk’s retrosynthetic ability offers valuable insights for
synthetic chemists and assists in synthesis planning.

Figure 6: The comparison among synthetic routes of the target molecule
Gilmelisib: planned by (a) SynAsk’s retrosynthetic tool and (b) an experienced
synthetic chemist.

###  3.3 Examples of the SynAsk platform outputs vesus other LLMs

Here we present a comparative analysis of the performance of three LLMs –
SynAsk, ChatGPT-4.0, and ChemCrow – in addressing synthetic chemistry queries.
We evaluated their capabilities by inputting a set of synthetic questions,
encompassing both general and professional inquiries, to assess their aptitude
in providing accurate and relevant responses.

####  3.3.1 General inquiries

Queries such as “Can you recommend me some reaction conditions for Suzuki
cross-coupling?" or “Please help me find some literature related to C-H
activation" were presented to all three LLMs. Across the board, each model
exhibited proficiency in generating appropriate responses, showcasing their
utility in aiding chemists with routine inquiries, details in Section S6 of
the ESI.

####  3.3.2 Professional synthetic questions

A more rigorous evaluation was conducted by inputting a specific synthetic
question: “Tell me what reaction can occur between
Nc1ccc2nccnc2c1.O=C(O)Cc1cc(F)cc(F)c1 and what the product is." Here
“Nc1ccc2nccnc2c1.O=C(O)Cc1cc(F)cc(F)c1" represents the SMILES syntax for
quinoxalin-6-amine and 3,5-Difluorophenylacetic acid as substrates. The
deliberate use of SMILES allows us to assess the LLMs’ ability to recognize
molecules from SMILES.

As shown in Figure 7, SynAsk demonstrates its specialization in organic
chemistry by providing a comprehensive list of potential reactions and their
corresponding products. Leveraging its domain-specific knowledge, SynAsk
offers a diverse array of feasible transformations, including N-acylation,
Buchwald-Hartwig amination, Minisci reaction, among others. This exhaustive
output underscores SynAsk’s capacity to analyze complex molecular interactions
and propose multiple viable pathways.

In contrast, ChemCrow delivers a singular response, identifying the reaction
as N-acylation and providing the corresponding product. While ChemCrow offers
a concise solution, its limitation in providing alternative reaction pathways
restricts its utility in scenarios where multiple transformation possibilities
exist.

ChatGPT-4, although proficient in understanding the query, encounters a
misinterpretation in identifying the compounds involved. While it accurately
delineates the structure and classification of the provided molecules, it
erroneously labels Nc1ccc2nccnc2c1 as nicotinic acid derivative, instead of
recognizing it as quinoxalin-6-amine. This discrepancy underscores the model’s
susceptibility to misinterpretation of chemical structures, particularly in
complex contexts.

Figure 7: The comparison of SynAsk, ChatGPT-4, and ChemCrow output on a
professional synthetic question.

SynAsk distinguishes itself as a specialized LLM tailored specifically for
organic chemistry tasks. Its domain-specific training and integration of fine-
tuning techniques result in a robust model capable of providing detailed
insights and accurate predictions for complex synthetic queries. While
ChatGPT-4 and ChemCrow offer general language processing capabilities, they
lack the nuanced understanding and domain expertise exhibited by SynAsk in the
context of organic chemistry applications. Therefore, for researchers seeking
nuanced insights and comprehensive analyses in organic synthesis, SynAsk
stands as a valuable tool for augmenting chemical exploration and discovery.

##  4 Conclusions and Future Works

In this work, we have developed SynAsk, a specialized LLM-powered platform for
synthetic chemistry. It represents the first publicly accessible chemistry
domain-specific LLM, fine-tuned with selected chemistry data and connected
with both in-house and external chemoinformatic tools. Through comparative
analyses with foundation LLMs, we have demonstrated SynAsk’s proficiency and
specialization in synthetic chemistry. Results obtained in reaction yield
prediction and retrosynthesis further validate SynAsk’s capability in
providing valuable chemical insights to synthetic chemists across multiple
domains.

Looking ahead, our future endeavors aim to enhance the functionality of SynAsk
by empowering the language model and fine-tuning it with additional data for
more seamless and appropriate responses. Additionally, we envision SynAsk
playing a pivotal role in driving autonomous reaction laboratories [50].
Traditionally, reaction robots have been constrained by written scripts to
define their scopes. Recent research has showcased the potential of LLMs to
drive robotic chemists effectively [51]. Leveraging SynAsk’s capabilities such
as retrosynthesis, inference, and programming script writing, we foresee it
being instrumental in driving autonomous laboratories, representing the next
phase of our fusion of LLM and hardware research.

##  5 Acknowledgements

We are grateful for financial support from Guangzhou National Laboratory, and
the National Natural Science Foundation of China (22071249, 22393892).

##  6 Conflict of Interest

We have a patent application in China with the application number
202410714040.6 titled "A Human-Computer Interaction Method and Electronic
Device Based on a Large Language Model".

## References

  *   * Radford et al. [2018] Radford, A., Narasimhan, K., Salimans, T., Sutskever, I.: Improving language understanding by generative pre-training (2018)
  * Brown et al. [2020] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al.: Language models are few-shot learners. arXiv preprint arXiv:2005.14165 (2020)
  * OpenAI et al. [2023] OpenAI, :, Achiam, J., Adler, S., Agarwal, S., Ahmad, L., Akkaya, I., Aleman, F.L., Almeida, D., Altenschmidt, J., Altman, S., Anadkat, S., Avila, R., Babuschkin, I., Balaji, S., Balcom, V., Baltescu, P., Bao, H., Bavarian, M., Belgum, J., Bello, I., Berdine, J., Bernadett-Shapiro, G., Berner, C., Bogdonoff, L., Boiko, O., Boyd, M., Brakman, A.-L., Brockman, G., Brooks, T., Brundage, M., Button, K., Cai, T., Campbell, R., Cann, A., Carey, B., Carlson, C., Carmichael, R., Chan, B., Chang, C., Chantzis, F., Chen, D., Chen, S., Chen, R., Chen, J., Chen, M., Chess, B., Cho, C., Chu, C., Chung, H.W., Cummings, D., Currier, J., Dai, Y., Decareaux, C., Degry, T., Deutsch, N., Deville, D., Dhar, A., Dohan, D., Dowling, S., Dunning, S., Ecoffet, A., Eleti, A., Eloundou, T., Farhi, D., Fedus, L., Felix, N., Fishman, S.P., Forte, J., Fulford, I., Gao, L., Georges, E., Gibson, C., Goel, V., Gogineni, T., Goh, G., Gontijo-Lopes, R., Gordon, J., Grafstein, M., Gray, S., Greene, R., Gross, J., Gu, S.S., Guo, Y., Hallacy, C., Han, J., Harris, J., He, Y., Heaton, M., Heidecke, J., Hesse, C., Hickey, A., Hickey, W., Hoeschele, P., Houghton, B., Hsu, K., Hu, S., Hu, X., Huizinga, J., Jain, S., Jain, S., Jang, J., Jiang, A., Jiang, R., Jin, H., Jin, D., Jomoto, S., Jonn, B., Jun, H., Kaftan, T., Kaiser, Kamali, A., Kanitscheider, I., Keskar, N.S., Khan, T., Kilpatrick, L., Kim, J.W., Kim, C., Kim, Y., Kirchner, H., Kiros, J., Knight, M., Kokotajlo, D., Kondraciuk, Kondrich, A., Konstantinidis, A., Kosic, K., Krueger, G., Kuo, V., Lampe, M., Lan, I., Lee, T., Leike, J., Leung, J., Levy, D., Li, C.M., Lim, R., Lin, M., Lin, S., Litwin, M., Lopez, T., Lowe, R., Lue, P., Makanju, A., Malfacini, K., Manning, S., Markov, T., Markovski, Y., Martin, B., Mayer, K., Mayne, A., McGrew, B., McKinney, S.M., McLeavey, C., McMillan, P., McNeil, J., Medina, D., Mehta, A., Menick, J., Metz, L., Mishchenko, A., Mishkin, P., Monaco, V., Morikawa, E., Mossing, D., Mu, T., Murati, M., Murk, O., Mély, D., Nair, A., Nakano, R., Nayak, R., Neelakantan, A., Ngo, R., Noh, H., Ouyang, L., O’Keefe, C., Pachocki, J., Paino, A., Palermo, J., Pantuliano, A., Parascandolo, G., Parish, J., Parparita, E., Passos, A., Pavlov, M., Peng, A., Perelman, A., Avila Belbute Peres, F., Petrov, M., Oliveira Pinto, H.P., Michael, Pokorny, Pokrass, M., Pong, V., Powell, T., Power, A., Power, B., Proehl, E., Puri, R., Radford, A., Rae, J., Ramesh, A., Raymond, C., Real, F., Rimbach, K., Ross, C., Rotsted, B., Roussez, H., Ryder, N., Saltarelli, M., Sanders, T., Santurkar, S., Sastry, G., Schmidt, H., Schnurr, D., Schulman, J., Selsam, D., Sheppard, K., Sherbakov, T., Shieh, J., Shoker, S., Shyam, P., Sidor, S., Sigler, E., Simens, M., Sitkin, J., Slama, K., Sohl, I., Sokolowsky, B., Song, Y., Staudacher, N., Such, F.P., Summers, N., Sutskever, I., Tang, J., Tezak, N., Thompson, M., Tillet, P., Tootoonchian, A., Tseng, E., Tuggle, P., Turley, N., Tworek, J., Uribe, J.F.C., Vallone, A., Vijayvergiya, A., Voss, C., Wainwright, C., Wang, J.J., Wang, A., Wang, B., Ward, J., Wei, J., Weinmann, C., Welihinda, A., Welinder, P., Weng, J., Weng, L., Wiethoff, M., Willner, D., Winter, C., Wolrich, S., Wong, H., Workman, L., Wu, S., Wu, J., Wu, M., Xiao, K., Xu, T., Yoo, S., Yu, K., Yuan, Q., Zaremba, W., Zellers, R., Zhang, C., Zhang, M., Zhao, S., Zheng, T., Zhuang, J., Zhuk, W., Zoph, B.: GPT-4 Technical Report (2023)
  * Bai et al. [2023] Bai, J., Bai, S., Chu, Y., Cui, Z., Dang, K., Deng, X., Fan, Y., Ge, W., Han, Y., Huang, F., Hui, B., Ji, L., Li, M., Lin, J., Lin, R., Liu, D., Liu, G., Lu, C., Lu, K., Ma, J., Men, R., Ren, X., Ren, X., Tan, C., Tan, S., Tu, J., Wang, P., Wang, S., Wang, W., Wu, S., Xu, B., Xu, J., Yang, A., Yang, H., Yang, J., Yang, S., Yao, Y., Yu, B., Yuan, H., Yuan, Z., Zhang, J., Zhang, X., Zhang, Y., Zhang, Z., Zhou, C., Zhou, J., Zhou, X., Zhu, T.: Qwen Technical Report (2023)
  * Touvron et al. [2023] Touvron, H., Lavril, T., Izacard, G., Martinet, X., Lachaux, M.-A., Lacroix, T., Rozière, B., Goyal, N., Hambro, E., Azhar, F., Rodriguez, A., Joulin, A., Grave, E., Lample, G.: LLaMA: Open and Efficient Foundation Language Models (2023)
  * Yue et al. [2023] Yue, S., Chen, W., Wang, S., Li, B., Shen, C., Liu, S., Zhou, Y., Xiao, Y., Yun, S., Huang, X., Wei, Z.: DISC-LawLLM: Fine-tuning Large Language Models for Intelligent Legal Services (2023)
  * Singhal et al. [2023] Singhal, K., Azizi, S., Tu, T., Mahdavi, S.S., Wei, J., Chung, H.W., Scales, N., Tanwani, A., Cole-Lewis, H., Pfohl, S., et al.: Large language models encode clinical knowledge. Nature 620(7972), 172–180 (2023)
  * Weininger et al. [1989] Weininger, D., Weininger, A., Weininger, J.L.: Smiles. 2. algorithm for generation of unique smiles notation. Journal of Chemical Information and Computer Sciences 29(2), 97–101 (1989) https://doi.org/10.1021/ci00062a008 https://doi.org/10.1021/ci00062a008
  * Schwaller et al. [2019] Schwaller, P., Laino, T., Gaudin, T., Bolgar, P., Hunter, C.A., Bekas, C., Lee, A.A.: Molecular transformer: A model for uncertainty-calibrated chemical reaction prediction. ACS Central Science 5(9), 1572–1583 (2019) https://doi.org/10.1021/acscentsci.9b00576 https://doi.org/10.1021/acscentsci.9b00576. PMID: 31572784
  * Weber et al. [2021] Weber, J.M., Guo, Z., Zhang, C., Schweidtmann, A.M., Lapkin, A.A.: Chemical data intelligence for sustainable chemistry. Chem. Soc. Rev. 50, 12013–12036 (2021) https://doi.org/10.1039/D1CS00477H
  * Guo et al. [2023] Guo, T., Guo, K., Nan, B., Liang, Z., Guo, Z., Chawla, N.V., Wiest, O., Zhang, X.: What can Large Language Models do in chemistry? A comprehensive benchmark on eight tasks (2023)
  * M. Bran et al. [2024] M. Bran, A., Cox, S., Schilter, O., Baldassari, C., White, A.D., Schwaller, P.: Augmenting large language models with chemistry tools. Nature Machine Intelligence, 1–11 (2024)
  * Wei et al. [2022] Wei, J., Wang, X., Schuurmans, D., Bosma, M., Xia, F., Chi, E., Le, Q.V., Zhou, D., et al.: Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems 35, 24824–24837 (2022)
  * Topsakal and Akinci [2023] Topsakal, O., Akinci, T.C.: Creating large language model applications utilizing langchain: A primer on developing llm apps fast. In: International Conference on Applied Engineering and Natural Sciences, vol. 1, pp. 1050–1056 (2023)
  * Zhang et al. [2024] Zhang, D., Liu, W., Tan, Q., Chen, J., Yan, H., Yan, Y., Li, J., Huang, W., Yue, X., Zhou, D., Zhang, S., Su, M., Zhong, H., Li, Y., Ouyang, W.: ChemLLM: A Chemical Large Language Model (2024)
  * [16] chatchat-space: Langchain-Chatchat. https://github.com/chatchat-space/Langchain-Chatchat
  * Hendrycks et al. [2020] Hendrycks, D., Burns, C., Basart, S., Zou, A., Mazeika, M., Song, D., Steinhardt, J.: Measuring massive multitask language understanding. arXiv preprint arXiv:2009.03300 (2020)
  * Huang et al. [2024] Huang, Y., Bai, Y., Zhu, Z., Zhang, J., Zhang, J., Su, T., Liu, J., Lv, C., Zhang, Y., Fu, Y., et al.: C-eval: A multi-level multi-discipline chinese evaluation suite for foundation models. Advances in Neural Information Processing Systems 36 (2024)
  * Cobbe et al. [2021] Cobbe, K., Kosaraju, V., Bavarian, M., Chen, M., Jun, H., Kaiser, L., Plappert, M., Tworek, J., Hilton, J., Nakano, R., et al.: Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168 (2021)
  * Suzgun et al. [2022] Suzgun, M., Scales, N., Schärli, N., Gehrmann, S., Tay, Y., Chung, H.W., Chowdhery, A., Le, Q.V., Chi, E.H., Zhou, D., , Wei, J.: Challenging big-bench tasks and whether chain-of-thought can solve them. arXiv preprint arXiv:2210.09261 (2022)
  * Li et al. [2023] Li, H., Zhang, Y., Koto, F., Yang, Y., Zhao, H., Gong, Y., Duan, N., Baldwin, T.: CMMLU: Measuring massive multitask language understanding in Chinese (2023)
  * Touvron et al. [2023] Touvron, H., Martin, L., Stone, K., Albert, P., Almahairi, A., Babaei, Y., Bashlykov, N., Batra, S., Bhargava, P., Bhosale, S., et al.: Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288 (2023)
  * Du et al. [2022] Du, Z., Qian, Y., Liu, X., Ding, M., Qiu, J., Yang, Z., Tang, J.: Glm: General language model pretraining with autoregressive blank infilling. In: Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 320–335 (2022)
  * Team [2023] Team, I.: Internlm: A multilingual language model with progressively enhanced capabilities (2023)
  * Yang et al. [2023] Yang, A., Xiao, B., Wang, B., Zhang, B., Bian, C., Yin, C., Lv, C., Pan, D., Wang, D., Yan, D., et al.: Baichuan 2: Open large-scale language models. arXiv preprint arXiv:2309.10305 (2023)
  * AI et al. [2024] AI, ., :, Young, A., Chen, B., Li, C., Huang, C., Zhang, G., Zhang, G., Li, H., Zhu, J., Chen, J., Chang, J., Yu, K., Liu, P., Liu, Q., Yue, S., Yang, S., Yang, S., Yu, T., Xie, W., Huang, W., Hu, X., Ren, X., Niu, X., Nie, P., Xu, Y., Liu, Y., Wang, Y., Cai, Y., Gu, Z., Liu, Z., Dai, Z.: Yi: Open Foundation Models by 01.AI (2024)
  * Kim et al. [2023] Kim, S., Chen, J., Cheng, T., Gindulyte, A., He, J., He, S., Li, Q., Shoemaker, B.A., Thiessen, P.A., Yu, B., et al.: Pubchem 2023 update. Nucleic acids research 51(D1), 1373–1380 (2023)
  * Pence and Williams [2010] Pence, H.E., Williams, A.: ChemSpider: an online chemical information resource. ACS Publications (2010)
  * for Chemistry team [2023] Chemistry team, I.R.: rxn4chemistry: Python wrapper for the IBM RXN for Chemistry API. https://github.com/rxn4chemistry/rxn4chemistry (2023)
  * Schwaller et al. [2020] Schwaller, P., Hoover, B., Reymond, J.-L., Strobelt, H., Laino, T.: Unsupervised attention-guided atom-mapping (2020)
  * Chen et al. [2024] Chen, S., An, S., Babazade, R., Jung, Y.: Precise atom-to-atom mapping for organic reactions via human-in-the-loop machine learning. Nature Communications 15(1), 2250 (2024)
  * Ginsparg [2011] Ginsparg, P.: Arxiv at 20. Nature 476(7359), 145–147 (2011)
  * SerpAPI [2023] SerpAPI: SerpAPI - Google Search Results API. https://serpapi.com/ (2023)
  * Dahlgren [2018] Dahlgren, B.: Chempy: A package useful for chemistry written in python. Journal of Open Source Software 3(24), 565 (2018)
  * Xu et al. [2023] Xu, Y., Gao, Y., Su, L., Wu, H., Tian, H., Zeng, M., Xu, C., Zhu, X., Liao, K.: High-throughput experimentation and machine learning-assisted optimization of iridium-catalyzed cross-dimerization of sulfoxonium ylides. Angewandte Chemie International Edition 62(48), 202313638 (2023) https://doi.org/10.1002/anie.202313638 https://onlinelibrary.wiley.com/doi/pdf/10.1002/anie.202313638
  * Qiu et al. [2023] Qiu, J., Xu, Y., Su, S., Gao, Y., Yu, P., Ruan, Z., Liao, K.: Auto machine learning assisted preparation of carboxylic acid by tempo-catalyzed primary alcohol oxidation. Chinese Journal of Chemistry 41(2), 143–150 (2023) https://doi.org/10.1002/cjoc.202200555 https://onlinelibrary.wiley.com/doi/pdf/10.1002/cjoc.202200555
  * Xu et al. [2023] Xu, Y., Ren, F., Su, L., Xiong, Z., Zhu, X., Lin, X., Qiao, N., Tian, H., Tian, C., Liao, K.: Hte and machine learning-assisted development of iridium (i)-catalyzed selective o–h bond insertion reactions toward carboxymethyl ketones. Organic Chemistry Frontiers 10(5), 1153–1159 (2023)
  * Yu et al. [2023] Yu, Z., Kong, Y., Li, B., Su, S., Rao, J., Gao, Y., Tu, T., Chen, H., Liao, K.: Hte-and ai-assisted development of dhp-catalyzed decarboxylative selenation. Chemical Communications 59(20), 2935–2938 (2023)
  * Devlin et al. [2018] Devlin, J., Chang, M., Lee, K., Toutanova, K.: BERT: pre-training of deep bidirectional transformers for language understanding. CoRR abs/1810.04805 (2018) 1810.04805
  * Contributors [2023] Contributors, O.: OpenCompass: A Universal Evaluation Platform for Foundation Models. https://github.com/open-compass/opencompass (2023)
  * Ahneman et al. [2018] Ahneman, D.T., Estrada, J.G., Lin, S., Dreher, S.D., Doyle, A.G.: Predicting reaction performance in c–n cross-coupling using machine learning. Science 360(6385), 186–190 (2018)
  * Perera et al. [2018] Perera, D., Tucker, J.W., Brahmbhatt, S., Helal, C.J., Chong, A., Farrell, W., Richardson, P., Sach, N.W.: A platform for automated nanomole-scale reaction screening and micromole-scale synthesis in flow. Science 359(6374), 429–434 (2018)
  * Schwaller et al. [2021] Schwaller, P., Vaucher, A.C., Laino, T., Reymond, J.-L.: Prediction of chemical reaction yields using deep learning. Machine learning: science and technology 2(1), 015016 (2021)
  * Zaitseva et al. [2021] Zaitseva, E.R., Smirnov, A.Y., Myasnyanko, I.N., Mineev, K.S., Sokolov, A.I., Volkhina, T.N., Mikhaylov, A.A., Baleeva, N.S., Baranov, M.S.: Imidazol-5-ones as a substrate for [1, 5]-hydride shift triggered cyclization. New Journal of Chemistry 45(4), 1805–1808 (2021)
  * Schwaller et al. [2021] Schwaller, P., Probst, D., Vaucher, A.C., Nair, V.H., Kreutter, D., Laino, T., Reymond, J.-L.: Mapping the space of chemical reactions using attention-based neural networks. Nature Machine Intelligence 3(2), 144–152 (2021)
  * European Bioinformatics Institute [2024] European Bioinformatics Institute: ChEMBL Drug Database. https://www.ebi.ac.uk/chembl/g/#browse/drugs. Accessed: 2024-04-22 (2024)
  * Genheden et al. [2020] Genheden, S., Thakkar, A., Chadimová, V., Reymond, J.-L., Engkvist, O., Bjerrum, E.: Aizynthfinder: a fast, robust and flexible open-source software for retrosynthetic planning. Journal of cheminformatics 12(1), 70 (2020)
  * Mikulak-Klucznik et al. [2020] Mikulak-Klucznik, B., Gołębiowska, P., Bayly, A.A., Popik, O., Klucznik, T., Szymkuć, S., Gajewska, E.P., Dittwald, P., Staszewska-Krajewska, O., Beker, W., et al.: Computational planning of the synthesis of complex natural products. Nature 588(7836), 83–88 (2020)
  * Segler et al. [2018] Segler, M.H., Preuss, M., Waller, M.P.: Planning chemical syntheses with deep neural networks and symbolic ai. Nature 555(7698), 604–610 (2018)
  * Burger et al. [2020] Burger, B., Maffettone, P.M., Gusev, V.V., Aitchison, C.M., Bai, Y., Wang, X., Li, X., Alston, B.M., Li, B., Clowes, R., et al.: A mobile robotic chemist. Nature 583(7815), 237–241 (2020)
  * Boiko et al. [2023] Boiko, D.A., MacKnight, R., Kline, B., Gomes, G.: Autonomous chemical research with large language models. Nature 624(7992), 570–578 (2023)

SynAsk: Unleashing the Power of Large Language Models in Organic Synthesis
Electronic Supplementary Information
Chonghuan Zhang1†, Qianghua Lin1†, Biwei Zhu2†, Haopeng Yang2, Xiao Lian2, Hao
Deng2, Jiajun Zheng2, Kuangbiao Liao1*
1Guangzhou National Laboratory, Guangdong, PR China, 510005
2AIChemEco Inc., Guangdong, PR China, 510005
* Corresponding author(s). E-mail(s<EMAIL_ADDRESS>
†These authors contributed equally to this work.

##  S1 The indicators used to assess LLMs

We evaluated the LLM’s capabilities using various metrics including Massive
Multi-task Language Understanding (MMLU), Multi-level multi-discipline chinese
evaluation (C-Eval), GSM8K, BIG-Bench-Hard (BBH), and Measuring massive
multitask language understanding in Chinese (CMMLU). These metrics
collectively provide a thorough assessment of a model’s proficiency,
encompassing linguistic understanding, mathematical reasoning, contextual
comprehension, multi-modal integration, and the application of CoT, which
examines the fluency of LLMs’ integration with external tools. This evaluation
framework emphasizes the diverse and essential skills a model needs to
effectively tackle complex real-world problems.

  * •

Massive Multi-task Language Understanding, MMLU represents a comprehensive and
multifaceted initiative that aims to evaluate and enhance the performance of
language models across a broad range of linguistic challenges, providing an
extensive evaluation of global knowledge and problem-solving abilities.

  * •

Multi-level Multi-discipline Chinese Evaluation, C-Eval tests models in
scenarios that necessitate an understanding of subtle context, which is
crucial for applications involving natural language understanding and
generation.

  * •

Grade School Math 8K, GSM8K is a widely recognized test set designed to assess
the mathematical capabilities of language models. It comprises problems that
require 2-8 steps of basic mathematical operations to test the models’ multi-
step mathematical reasoning.

  * •

BIG-Bench-Hard, BBH evaluates language models’ capabilities in applying Chain
of Thought to humanistic knowledge. It measures how effectively a model can
navigate through complex humanistic concepts and ideas, emphasizing its
ability to perform sequential reasoning that mirrors human-like understanding
in tasks with cultural and historical depth.

  * •

Measuring massive multitask language understanding in Chinese, CMMLU is a
comprehensive Chinese evaluation benchmark specifically used to evaluate the
knowledge and reasoning capabilities of language models in the Chinese
context. CMMLU covers 67 topics from basic subjects to advanced professional
levels.

Model |  MMLU (5-shot) |  C-Eval (5-shot) |  GSM8K (8-shot) |  BBH (3-shot) |  CMMLU
---|---|---|---|---|---
LLaMA2-7B |  46.8 |  32.5 |  16.7 |  38.2 |  31.8
LLaMA2-13B |  55.0 |  41.4 |  29.6 |  45.6 |  38.4
LLaMA2-32B |  62.6 |  - |  42.2 |  44.1 |  -
ChatGLM2-6B |  47.9 |  51.7 |  32.4 |  33.7 |  -
InterLM-7B |  51.0 |  53.4 |  31.2 |  37.0 |  51.8
InterLM-20B |  62.1 |  58.8 |  52.6 |  52.5 |  59.0
Baichuan2-7B |  54.7 |  56.3 |  24.6 |  41.6 |  57.1
Baichuan2-13B |  59.5 |  59.0 |  52.8 |  49.0 |  62.0
Yi-34B |  76.3 |  81.8 |  67.9 |  66.4 |  85.6
Qwen-1.8B |  45.3 |  56.1 |  32.3 |  22.3 |  52.1
Qwen-7B |  58.2 |  63.5 |  51.7 |  45.0 |  62.2
Qwen-14B |  66.3 |  72.1 |  61.3 |  53.4 |  71.0
Qwen-72B |  77.4 |  83.3 |  78.9 |  67.7 |  83.6
Table S1: Performance of Different Models on Various Benchmarks

##  S2 Fine-tuning techniques and procedures

In our experiments, we explored two distinct fine-tuning methodologies for
LLMs. The first approach involved techniques such as quantization to enable
the operation of a 14-billion-parameter model within a 24GB GPU environment.
The second approach was direct fine-tuning without additional quantization
techniques.

For our experiments, we selected a model with 14 billion parameters. We
applied Low-Rank Adaptation (LoRA) by incorporating low-rank matrices into the
fully connected layers. The parameter details are presented in Table S1.

Total Parameters | Trainable Parameters | Percentage of Total
---|---|---
14,209,134,120 | 41,843,040 | $\approx 0.294\%$
Table S1: Parameter quantity of the 14-billion-parameter model

The fine-tuning with quantization process, conducted on a dataset of 200
entries with a batch size of 2, was completed within an hour. This method is a
viable solution for managing large model training on hardware with limited
memory without significantly compromising precision.

| Before Quantization | After 4-bit Quantization
---|---|---
During loading |  $14\times 10^{9}\times 4$ bytes |  $14\times 10^{9}\times 0.5$ bytes
During computation |  $14\times 10^{9}\times 2$ bytes
Memory Consumption |  $\approx 56$ GB |  $\approx 7$ GB during loading
Table S2: Memory usage before and after quantization

Leveraging a single GeForce RTX 4090 with 24GB of VRAM for fine-tuning a
14-billion-parameter model, we initially applied quantization to reduce the
memory usage and accelerate inference, though at the potential cost of
precision loss. During loading, the model was quantized to 4-bit precision and
subsequently converted to 16-bit for computations. Post-loading, neither the
original nor the quantized weights were retained in memory.

The fine-tuning without quantization approach utilized LoRA under the
deepspeed’s ZeRO-3 optimization. We employed three GeForce RTX 4090 GPUs, each
with 24GB of memory, which allowed the fine-tuning of the model on a dataset
of over 4,000 entries. The process took approximately seven hours to complete.

Both fine-tuning methodologies proved to be effective, demonstrating the
practical applicability of our approaches to large-scale model optimization.

##  S3 Chemistry related indicators and examples

We assessed the chemistry ability of the LLMs using the chemistry test
questions from C-Eval, which comprises multiple discipline questions in
multiple levels in Chinese (Section S1). This test was completed in Chinese
since SynAsk’s original language is Chinese. However, we acknowledge that with
the LLMs’ powerful language ability, testing of the LLMs with different major
languages in the world would reach close results.

We provide a set of example questions for the chemistry question in C-Eval at
multiple levels. Sections Section S3.1, Section S3.2, and Section S3.3 refers
to the chemistry questions at college, high school and middle school levels,
respectively. The dataset format consists of multiple-choice questions and
answers. The Prediction contains the answers predicted by three models:
SynAsk, Qwen1.5-14B-Chat, and Qwen-14B-Chat.

###  S3.1 C-Eval (College Chemistry)

Problem: 以下是中国关于大学化学考试的单项选择题，请选出其中的正确答案。 下列说法中，正确的是： (A) 单质的焓为零 (B)
反应的热效应就是该反应的摩尔焓变 (C) 单质的摩尔生成焓为零 (D) 由最稳定单质生成1 mol化合物时，该化合物的标准摩尔生成焓
$\Delta_{\mathfrak{f}}H_{\mathfrak{m}}^{\mathfrak{e}}$ 等于该生成反应的
$\Delta_{\mathfrak{r}}H_{\mathfrak{m}}^{\mathfrak{e}}$ English translation:
The following are single-choice questions on university chemistry exams in
China. Which of the following statements is correct? (A) The enthalpy of an
element is zero (B) The heat of reaction is equal to the molar enthalpy change
of the reaction (C) The molar enthalpy of formation of an element is zero (D)
When 1 mole of a compound is formed from the most stable elements, the
standard molar enthalpy of formation
$\Delta_{\mathfrak{f}}H_{\mathfrak{m}}^{\mathfrak{e}}$ of the compound is
equal to the standard molar enthalpy of reaction
$\Delta_{\mathfrak{r}}H_{\mathfrak{m}}^{\mathfrak{e}}$ of the formation
reaction Answer: D Predictions: SynAsk: D Qwen1.5-14B-Chat: C Qwen-14B-Chat: C

###  S3.2 C-Eval (High School Chemistry)

Problem: 以下是中国关于高中化学考试的单项选择题，请选出其中的正确答案。 下列说法中，正确的是：
在一定温度下的恒容密闭容器中，当下列哪些物理量不再发生变化时，表明下述反应： $A(s)+2B(g)\rightleftharpoons
C(g)+D(g)$ 已达到平衡状态： ①混合气体的压强 ②混合气体的密度 ③B的物质的量浓度 ④气体的总物质的量 ⑤混合气体总质量 (A) ②③⑤ (B)
①②③ (C) ②③④ (D) ①③④⑤ English translation: The following are single-choice
questions on high school chemistry exams in China. Please select the correct
answer. Which of the following statements is correct? In a constant-volume
sealed container at a certain temperature, when which of the following
physical quantities no longer change, it indicates that the following
reaction: $A(s)+2B(g)\rightleftharpoons C(g)+D(g)$ has reached equilibrium: ①
Pressure of the mixed gases ② Density of the mixed gases ③ Concentration of
substance B ④ Total amount of gas ⑤ Total mass of the mixed gases (A) ②③⑤ (B)
①②③ (C) ②③④ (D) ①③④⑤ Answer: A Predictions: SynAsk: A Qwen1.5-14B-Chat: ②③⑤
Qwen-14B-Chat: A

It is noted while Qwen1.5-14B-Chat provides with the right answer, it predicts
with the context of the answer directly without showing the correct choice
“A".

###  S3.3 C-Eval (Middle School Chemistry)

Problem: 以下是中国关于初中化学考试的单项选择题，请选出其中的正确答案。 下列有关实验现象的描述正确的是： (A) 硫在氧气中燃烧发出淡蓝色火焰
(B) 无色酚酞试液遇稀盐酸变成红色 (C) 硫酸铜溶液和氢氧化钠溶液反应会产生蓝色沉淀 (D) 红磷在空气中燃烧产生白雾 English
translation: The following are single-choice questions on junior high school
chemistry exams in China. Please select the correct answer. Which of the
following descriptions about experimental phenomena is correct? (A) Sulfur
burns with a pale blue flame in oxygen (B) Colorless phenolphthalein turns red
when mixed with dilute hydrochloric acid (C) The reaction between copper
sulfate solution and sodium hydroxide solution produces a blue precipitate (D)
Red phosphorus burns in air to produce white smoke Answer: C Predictions:
SynAsk: C Qwen1.5-14B-Chat: C Qwen-14B-Chat: C

##  S4 Reaction yield prediction results

We randomly collected 60 recent published SNAr reactions from the last three
years (2022-2024), which includes new substrate molecules and never seen by
the reaction model. The model predicted yield versus literature reported yield
are compared in the attached spreadsheet file, SI.xlsx, with an MAE of 14.1%.
These recent published reactions consist of seven different reaction
conditions.

##  S5 Retrosynthetic pathway of selected target molecules

Figure S1, Figure S2, Figure S3 and Figure S4 shows numbers of retrosynthetic
pathways generated by SynAsk, which provide insights for synthetic chemists.
The routes indicate the ability of SynAsk in computer assisted synthetic
planning (CASP).

We are developing strategies towards generation of more reasonable
retrosynthetic pathways. This will be published elsewhere, and integrated into
SynAsk. Till now, no efforts were made to experimentally validate the
synthetic routes provided in the ESI, and more synthetic routes to other
target molecules can be generated via command to SynAsk.

Figure S1: The synthetic route of the target molecule mitoquinone planned by
SynAsk’s retrosynthetic tool. Figure S2: The synthetic route of the target
molecule L-778123 planned by SynAsk’s retrosynthetic tool. Figure S3: The
synthetic route of the target molecule trotabresib planned by SynAsk’s
retrosynthetic tool. Figure S4: The synthetic route of the target molecule
azaloxan planned by SynAsk’s retrosynthetic tool.

##  S6 Examples of the SynAsk platform outputs versus other LLMs

Figure S5: The first example of the outputs from the LLMs. Figure S6: The
second example of the outputs from the LLMs. Figure S7: The third example of
the outputs from the LLMs.


there exists some $u\in U_{i}$ such that $u\cdot A_{g}^{\pm}=A^{\pm}$.

###### Attempt to prove conjecture:.

We will do the case $A_{g}^{+}=g\cdot A^{+}$ for some $g\in B_{2,\infty}^{+}$
since the other cases are similar. We write more briefly $A$ for $A^{+}$ and
we write its ends as $e_{i}=e_{i,\infty}^{+}$ for $i=1,2$. We wish to show
there exists some $u\in U_{2}$ such that $uA_{g}=A$. This is trivially true
with $u=1$ when $g=1$. Assume $g\neq 1$ and that the result has been
established for all shorter elements $g=g_{i_{2}}\cdots g_{i_{n}}\in
B_{2,\infty}^{+}$. Using the correspondence between lines in the tree
$\mathcal{B}^{+}$ and pairs of ends in $\mathcal{B}_{\infty}^{+}$, we write
$A=[e_{2},e_{1}]$ and $A_{g}=gA=[ge_{2},ge_{1}]=[e_{2},ge_{1}]$ because $g\in
B_{2,\infty}^{+}$ fixes the end $e_{2}$.

If $g_{i_{1}}\in U_{2}\subset B_{2,\infty}^{+}$ then
$g_{i_{1}}^{-1}g=g_{i_{2}}\cdots g_{i_{n}}\in B_{2,\infty}^{+}$ so by
induction there is an element $u_{2}\in U_{2}$ such that $u_{2}g_{i_{2}}\cdots
g_{i_{n}}A=A$. Then $u=u_{2}g_{i_{1}}^{-1}\in U_{2}$ satisfies $uA_{g}=A$.

Now assume $g_{i_{1}}\in U_{1}\subset B_{1,\infty}^{+}$ then
$g_{i_{1}}^{-1}g=g_{i_{2}}\cdots g_{i_{n}}\in U_{1}B_{2,\infty}^{+}$. The
problem is that we have too little information about $g_{i_{2}}\cdots
g_{i_{n}}$ so we cannot apply the inductive hypothesis to it. ∎

If we could prove the conjecture above, we would have the following two
results:

######  Conjecture 9.14.

For $i=1,2$ we have the decomposition $B_{i,\infty}^{\pm}=$

$B^{\pm}_{W}U_{i}W^{even}=B^{\pm}_{W}W^{even}U_{i}=U_{i}W^{even}B^{\pm}_{W}=U_{i}B^{\pm}_{W}W^{even}=W^{even}U_{i}B^{\pm}_{W}=W^{even}B^{\pm}_{W}U_{i}\,.$

###### Proof.

  1. (1)

We have to establish the stated decomposition of $B_{i,\infty}^{\pm}$ into the
product of the subgroups $B^{\pm}_{W}=T_{\mathbb{C}}U^{\pm}_{W}$, $U_{i}$ and
$W^{even}$.

Hence there is some $bw\in B_{W}^{\pm}W^{even}$ such that $u_{n}\dots
u_{1}g=bw$. Thus we find the product expression $g=u_{1}^{-1}\dots
u_{n}^{-1}bw$ giving the decomposition

$B_{i,\infty}^{\pm}=U_{i}B^{\pm}_{W}W^{even}\,.$

  2. (2)

Using lemmas 9.9 and 9.10 yields the product decompositions

$B_{i,\infty}^{\pm}=U_{i}B^{\pm}_{W}W^{even}=U_{i}W^{even}B^{\pm}_{W}=W^{even}U_{i}B^{\pm}_{W}\,.$

  3. (3)

To get the remaining three decompositions for $B_{i,\infty}^{\pm}$ we have to
show that $B^{\pm}_{W}U_{i}=U_{i}B^{\pm}_{W}$. To do this use the notation of
part 1. Assume $g\in B_{i,\infty}^{\pm}$ has a decomposition $g=ubw$ such that
$u\in U_{i}$, $b\in B^{\pm}_{W}$ and $w\in W^{even}$. We start with the
apartment $A_{g^{-1}}^{\pm}=g^{-1}\cdot A^{\pm}$. Clearly $g\cdot
A_{g^{-1}}^{\pm}=A^{\pm}$. The argument in step 1 shows that there is some
$\overline{u}\in U_{i}$ such that $\overline{u}g^{-1}\cdot
A^{\pm}=\overline{u}\cdot A_{g^{-1}}^{\pm}=A^{\pm}$. Hence there is some
element $\overline{w}\overline{b}\in W^{even}B_{W}^{\pm}$ such that
$\overline{u}g^{-1}=\overline{w}\overline{b}$ so
$g=\overline{b}^{-1}\overline{w}^{-1}\overline{u}\in
B_{W}^{\pm}W^{even}U_{i}$. The use of lemmas 9.9 and 9.10 establishes the
other decompositions.

∎

For $1\leq i\leq 2$ we define $U_{i,\infty}^{\pm}=U_{W}^{\pm}U_{i}\,.$

######  Conjecture 9.15.

The hyperbolic Kac–Moody group $G$ admits a spherical Iwasawa decomposition

$G=KT^{ad}_{\mathbb{R}}U_{i,\infty}^{\pm}\,.$

###### Proof.

We know that $K$ acts transitively on chambers of $\mathcal{B}^{\pm}$ so it
acts transitively on $\mathcal{B}_{\infty}^{\pm}$. Let $g\in G$. This gives
chambers $c_{i}^{\pm}=g\cdot e_{i,\infty}^{\pm}$, $i=1,2$. Then by
transitivity of $K$ on $\mathcal{B}_{\infty}^{\pm}$ we find elements $k_{i}\in
K$ such that $k_{i}\cdot c_{i}^{\pm}=e_{i,\infty}^{\pm}$, so that
$(k_{i}g)\cdot e_{i,\infty}^{\pm}=e_{i,\infty}^{\pm}$. Thus $k_{i}g\in
B_{i,\infty}^{\pm}$, establishing

(18) $G=K\,B_{i,\infty}^{\pm}$

From
$B_{i,\infty}^{\pm}=W^{even}T_{\mathbb{C}}U_{i,\infty}^{\pm}=W^{even}TT^{ad}_{\mathbb{R}}U_{i,\infty}^{\pm}$
we get $K\cap B_{i,\infty}^{\pm}TW^{even}$ and in consequence a decomposition

(19) $G=KT^{ad}_{\mathbb{R}}U_{i,\infty}^{\pm}\qquad\textrm{such that}\quad
K\cap T^{ad}_{\mathbb{R}}=T^{ad}_{\mathbb{R}}\cap U_{i,\infty}^{\pm}=K\cap
U_{i,\infty}^{\pm}=1$

∎

######  Remark 9.16.

A spherical Iwasawa decomposition was established in Proposition 11.99 in
[AB08] for Euclidean buildings and follows from a spherical Bruhat
decomposition. For our Kac–Moody group $G$ over ${\mathbb{C}}$, we proved a
spherical Bruhat decomposition (lemma 9.6). In rank $2$, we may use the fact
that the hyperbolic and affine spherical buildings coincide. Hence we may
apply Proposition 11.99 in [AB08] to deduce the theorem.

Let $B_{i,K}^{\pm}=B_{i,\infty}^{\pm}\cap K$. In our next step towards the
proof of Theorem 9.2, we show that
$B_{1,K}^{+}=B_{1,K}^{-}=B_{2,K}^{+}=B_{2,K}^{-}=B_{K}$ is independent of the
choice of sign $\pm$ and of the choice of $i$, and establish a $K$-equivariant
bijection between $G/B_{i,\infty}^{\pm}$ and $K/B_{K}$.

######  Lemma 9.17.

We have $B_{i,K}^{\pm}=TW^{even}$, so $B_{K}=B_{i,K}^{\pm}$ is independent of
the choice of sign $\pm$ and of $i=1,2$.

###### Proof.

By lemma LABEL:lemma:structure_of_B_infty we have
$B_{i,\infty}^{\pm}=B_{W}^{\pm}U_{i}W^{even}$. We calculate the intersection
of each component with $K$. Recall that $B^{\pm}\cap K=T$. As $T\subset
B^{\pm}_{W}\subset B^{\pm}$, we see that $B_{W}^{\pm}\cap K=T$.

For $i=1,2$, set $(U_{i})_{K}=U_{i}\cap K=\\{u\in U_{i}\mid\omega(u)=u\\}$
where $\omega$ denotes the Cartan involution. Since
$\omega(\mathfrak{g}_{\alpha})=\mathfrak{g}_{-\alpha}$ we get
$\omega\circ(U_{\alpha})\circ(\omega)^{-1}=U_{-\alpha}$, and if $1\neq
u=exp(ad_{e_{\alpha}})\in(U_{i})_{K}$ for a basis vector
$e_{\alpha}\in\mathfrak{g}_{\alpha}$ then $u=exp(ad_{e_{-\alpha}})$ for some
basis vector $e_{-\alpha}\in\mathfrak{g}_{-\alpha}$. Then for any element
$h\in\mathfrak{h}$ such that $\alpha(h)\neq 0$, we get
$u(h)=h+[e_{\alpha},h]=h-[e_{-\alpha},h]$ so
$\alpha(h)e_{\alpha}=\alpha(h)e_{-\alpha}$ which is impossible, so $u=1$ and
$(U_{i})_{K}=\\{1\\}$. We have $W^{even}\subset W\subset K$. ∎

######  Corollary 9.18.

The chambers of $\mathcal{B}_{\infty}^{\pm}$ are in bijection with the
elements of $K/B_{K}$, hence from equation (16) we get
$\mathcal{B}_{\infty}^{\pm}=K/B_{K}\times\\{e_{i,\infty}^{\pm}\\}$ for either
choice of $i=1,2$.

###### Proof.

The group $K$ acts transitively on chambers which are the points of of
$\mathcal{B}_{\infty}^{\pm}$. Hence chambers correspond to the quotient of $K$
modulo the subgroup $B_{K}$ fixing the chamber $e_{i,\infty}^{\pm}$ for either
choice of $i=1,2$. ∎

######  Lemma 9.19.

For $\ \Xi_{\infty}=\Xi_{\infty}^{+}\cup\Xi_{\infty}^{-}$ the twin halo of
$\mathfrak{g}$, each element the set $\\{x^{\pm}_{i}\ |\
1=1,2\\}\subset\Xi_{\infty}^{\pm}$ is fixed by $B_{K}$, which is the pointwise
stabilizer of that set.

###### Proof.

The action of the even Weyl group $W^{even}$ fixes the four rays of
$\partial\mathcal{L}_{\mathfrak{t}}$, so its induced action on
$\\{x^{\pm}_{i}\ |\ 1=1,2\\}$ is trivial. The same is true of the Adjoint
action of $T=exp(\mathfrak{t})$ which fixes $\mathfrak{t}$ pointwise, so this
is true for $B_{K}$, which is contained in the pointwise stabilizer of this
set. Now suppose that $k\in K$ stabilizes the set pointwise. Then $k$ sends
two linearly independent points on the two lines of
$\partial\mathcal{L}_{\mathfrak{t}}$ to linearly independent points, a basis
for $\mathfrak{t}$, so $k$ normalizes $\mathfrak{t}$. But
$N_{K}(\mathfrak{t})=TW$, and the elements in the set $TW^{odd}$ exchange
$x^{\pm}_{1}$ and $x^{\pm}_{2}$, so $k\in TW^{even}=B_{K}$. ∎

We now have all technical ingredients to prove Theorem 9.2.

###### Proof of Theorem 9.2.

We begin to define $\Psi_{\infty}$ by setting

$\Psi_{\infty}(e_{1,\infty}^{+})=x_{1}^{+}\qquad\hbox{and}\qquad\Psi_{\infty}(e_{1,\infty}^{-})=x_{1}^{-}$

and we extend this to all of $\mathcal{B}_{\infty}$ by

$\Psi_{\infty}(kB_{K},e_{1,\infty}^{\pm})=k\Psi_{\infty}(e_{1,\infty}^{\pm})k^{-1}$

for any $k\in K$. It is clear that $\Psi_{\infty}$ is a well-defined
$K$-equivariant bijection. ∎

The results in this section suggest that on the Lie algebra level one could
study a non-standard Cartan decomposition

$\mathfrak{g}=\mathfrak{h}\oplus\bigoplus_{\alpha\in
P_{1}}\mathfrak{g}_{\alpha}\bigoplus_{\alpha\in P_{2}}\mathfrak{g}_{\alpha}$

based on a non-standard partition $\Phi=P_{1}\cup P_{2}$ such that
$\Phi_{i}\subset P_{i}$ for $i=1,2$, and such that

$\mathfrak{n}_{1}=\bigoplus_{\alpha\in
P_{1}}\mathfrak{g}_{\alpha}\qquad\hbox{and}\qquad\mathfrak{n}_{2}=\bigoplus_{\alpha\in
P_{2}}\mathfrak{g}_{\alpha}$

are subalgebras. Then $\mathfrak{b}_{i}=\mathfrak{h}\oplus\mathfrak{n}_{i}$,
$i=1,2$, would be non-standard Borel subalgebras related to the Borel
subalgebras $B_{i,\infty}^{\pm}$. That would be accomplished if each subset
$P_{i}$ is closed under addition. Since $\Phi$ is a subset of the dual of the
split real Cartan subalgebra, $\mathfrak{h}_{\mathbb{R}}^{*}$, one might
consider using a timelike line in the interior of the lightcone
$\mathcal{L}_{\mathfrak{h}_{\mathbb{R}}}^{0}$ to partition all of $\Phi$. If
such a line does not contain any roots, every root is on one side or the
other, and $\Phi_{1}$ and $\Phi_{2}$ are on opposite sides for any choice of
the line. But if the line contains roots, one would have to decide which ones
go in which part of the partition, and it must be done in such a way that each
$P_{i}$ is closed under addition. The solution would be to divide up the line
into two rays from the origin, and divide up the roots on the line according
to which ray they are in.

There are two obvious partitions determined by the two lightcone lines
themselves. For the line determined by $x^{+}_{2}$ the partition would be
$P_{1}=\Phi_{1}\cup(\Phi^{im})^{+}$ and $P_{2}=\Phi_{2}\cup(\Phi^{im})^{-}$,
while for the line determined by $x^{+}_{1}$ the partition would be
$P_{1}=\Phi_{1}\cup(\Phi^{im})^{-}$ and $P_{2}=\Phi_{2}\cup(\Phi^{im})^{+}$.
Associated to each of the four non-standard Borels at infinity,
$B_{i,\infty}^{\pm}$, is a choice of one of these four half-partitions.

Having found a non-standard Cartan decomposition as above, one gets a
corresponding non-standard decomposition of the universal enveloping algebra
$\mathcal{U}(\mathfrak{g})=\mathcal{U}(\mathfrak{n}_{1})\mathcal{U}(\mathfrak{h})\mathcal{U}(\mathfrak{n}_{2})$
and can construct induced Verma modules of two types,
$Verma^{i}(\lambda)=\mathcal{U}(\mathfrak{n}_{i})v^{i}_{\lambda}$, $i=1,2$,
where $v^{i}_{\lambda}$ are vectors such that $h\cdot
v_{\lambda}^{i}=\lambda(h)v_{\lambda}^{i}$ for any $h\in\mathfrak{h}$,
$\mathfrak{n}_{1}\cdot v^{2}_{\lambda}=0$ and $\mathfrak{n}_{2}\cdot
v^{1}_{\lambda}=0$. The quotient of such a Verma module by its maximal proper
submodule would be an irreducible module, $Irred^{i}(\lambda)$ generated by
$v^{i}_{\lambda}$. These would be integrable $\mathfrak{g}$-modules for
$\lambda=n_{1}\lambda_{1}+n_{2}\lambda_{2}$ in the weight lattice of
$\mathfrak{g}$ ($\lambda_{1}$, $\lambda_{2}$ are the fundamental weights of
$\mathfrak{g}$ such that $\lambda_{i}(h_{j})=\delta_{ij}$) but outside of the
lightcone, so that $n_{1}$ and $n_{2}$ have opposite signs. Examples of such
integrable modules, in addition to the adjoint representation, were mentioned
by Borcherds in Section 6 of [Bor86]. There is evidence that they occur in the
decomposition of the rank 3 hyperbolic algebra, $\mathcal{F}$, with respect to
its rank 2 hyperbolic subalgebras. That decomposition is currently under study
[Pen16].

##  10\. Conclusion and further directions

Our lightcone embedding of the twin building of a hyperbolic Kac–Moody group
is motivated by the conjectural existence of hyperbolic Kac–Moody symmetric
spaces. There have been some efforts recently to develop the geometry of
hyperbolic Kac–Moody symmetric spaces, building on work of the third author on
the construction of affine Kac–Moody symmetric spaces [Fre09, Fre11, Fre12b,
Fre13c, Fre13b].

Recalling the well–known finite dimensional theory (see for example [Hel01,
Ebe96]), the boundary of a symmetric space $M$ of non–compact type $IV$
corresponding to a complex simple Lie group $G$, can be identified with the
building over $\mathbb{C}$ associated to $G$. Via the isotropy representation
at a point $p\in M$, the building can be embedded into the unit sphere of the
tangent space $T_{p}M$. In this way, points in the building get identified
with directions in the tangent space of the symmetric space. Via the duality
between the compact type and non–compact types, we can identify spaces of type
$IV$ with spaces of type $II$ and thus also obtain an embedding of the
building into the tangent space of a compact symmetric space.

In the absence of hyperbolic Kac–Moody symmetric spaces, important properties
of the local geometry are captured via the embedding of the building into the
Lie algebra. Our embedding of the building into the Lie algebra gives local
pictures of the tangent spaces of conjectural hyperbolic Kac–Moody symmetric
spaces of types $II$ and $IV$. We note however, that since our twin building
embeds into the lightcone of the compact form of the Lie algebra, it captures
only the timelike directions in the tangent space.

Via Proposition 4.2, we have also obtained a hyperbolic analog of the notion
of a polar representation by the group $K$ on the $K$–conjugacy class of a
Cartan subalgebra. Recall that a group representation $G:V\longrightarrow V$
on a vector space $V$ is called polar if there exists a subspace
$\Sigma\subset V$, called a section, such that each orbit $G\cdot v$ for $v\in
V$ intersects $\Sigma$ orthogonally. Finite dimensional polar representations
are orbit equivalent to isotropy representations of finite dimensonal
Riemannian symmetric spaces [Dad85]. Similar observations were made in the
affine case [HPTT95, Gro00]. Our Proposition 4.2 shows that the action of $K$
on $\mathfrak{H}$ is a polar action with section $\mathfrak{t}$.

Further questions about polar representations for hyperbolic Kac–Moody groups
remain open and the full differential geometry need to develop hyperbolic
Kac–Moody symmetric spaces remains elusive. We hope to take this up elsewhere.

## References

  * [AB08] Peter Abramenko and Kenneth Brown, _Buildings_ , Graduate Texts in Mathematics, vol. 248, Springer Verlag, New York, 2008.
  * [AR98] Peter Abramenko and Mark Ronan, _A characterization of twin buildings by twin apartments_ , Geom. Dedicata 73 (1998), no. 1, 1–9. MR 1651854 (99m:51016)
  * [Ber85] Stephen Berman, _Real forms of universal Kac-Moody Lie algebras_ , Algebras Groups Geom. 2 (1985), no. 1, 10–25. MR 803743 (87a:17021)
  * [Bor86] Richard E. Borcherds, _Vertex algebras, kac-moody algebras, and the monster_ , Proc. Nat. Acad. Sci. U.S.A. 83 (1986), 3068–3071.
  * [BVBPBMR95] Valérie Back-Valente, Nicole Bardy-Panse, Hechmi Ben Messaoud, and Guy Rousseau, _Formes presque-déployées des algèbres de Kac-Moody: classification et racines relatives_ , J. Algebra 171 (1995), no. 1, 43–96. MR 1314093 (96d:17022)
  * [Cap09] Pierre-Emmanuel Caprace, _“Abstract” homomorphisms of split Kac-Moody groups_ , Mem. Amer. Math. Soc. 198 (2009), no. 924, xvi+84. MR 2499773 (2010d:20057)
  * [CG03] Lisa Carbone and Howard Garland, _Existence of lattices in Kac-Moody groups over finite fields_ , Commun. Contemp. Math. 5 (2003), no. 5, 813–867. MR 2017720 (2004m:17031)
  * [Che55] C. Chevalley, _Sur certains groupes simples_ , Tôhoku Math. J. (2) 7 (1955), 14–66. MR 0073602 (17,457c)
  * [CMS15] Lisa Carbone, Scott H. Murray, and Sowmya Srinivasan, _The geometry of rank 2 hyperbolic root systems_ , preprint, 2015.
  * [Dad85] Jiri Dadok, _Polar coordinates induced by actions of compact lie groups_ , Trans. Amer. Math. Soc. 288 (1985), 125–137.
  * [DH09] Thibault Damour and Christian Hillmann, _Fermionic Kac-Moody billiards and supergravity_ , J. High Energy Phys. (2009), no. 8, 100, 55. MR 2580182 (2011a:83112)
  * [DHN02] T. Damour, M. Henneaux, and H. Nicolai, _$E_{10}$ and a small tension expansion of M theory_, Phys. Rev. Lett. 89 (2002), no. 22, 221601, 4. MR 1949931 (2003k:83130)
  * [DMGH09] Tom De Medts, Ralf Gramlich, and Max Horn, _Iwasawa decompositions of split Kac-Moody groups_ , J. Lie Theory 19 (2009), no. 2, 311–337. MR 2572132 (2010j:20075)
  * [Ebe96] Patrick B. Eberlein, _Geometry of nonpositively curved manifolds_ , Chicago Lectures in Mathematics, Chicago, 1996.
  * [Fei80] Alex J. Feingold, _A hyperbolic GCM Lie algebra and the Fibonacci numbers_ , Proc. Amer. Math. Soc. 80 (1980), no. 3, 379–385. MR 580988 (81k:17009)
  * [FF83] Alex J. Feingold and Igor B. Frenkel, _A hyperbolic Kac-Moody algebra and the theory of Siegel modular forms of genus 2._ , Math. Ann. 263 (1983), 87–144.
  * [FF16] Alex Feingold and Walter Freyn, _Spherical parametrizations of cartan subalgebras_ , in preparation, 2016.
  * [Fre09] Walter Freyn, _Kac-Moody symmetric spaces and universal twin buildings_ , Ph.D. thesis, Universität Augsburg, 2009.
  * [Fre11] by same author, _Functional analytic methods for cities_ , submitted, 2011.
  * [Fre12a] by same author, _Kac-Moody geometry_ , Global Differential Geometry, Springer, Heidelberg, 2012, pp. 55–92.
  * [Fre12b] by same author, _Kac-Moody groups, infinite dimensional differential geometry and cities_ , Asian journal of mathematics 16 (2012), no. 4, 607–636.
  * [Fre13a] by same author, _Orthogonal-Symmetric affine Kac-Moody-algebras_ , accepted for publication in Transactions of the AMS, 2013.
  * [Fre13b] by same author, _Tame Fréchet structures for affine Kac-Moody groups_ , accepted for publication in Asian Journal of mathematics, 2013.
  * [Fre13c] by same author, _Tame submanifolds of co-Banach type_ , accepted for publication in Forum Mathematicum, 2013.
  * [GK81] Ofer Gabber and Victor G. Kac, _On defining relations of certain infinite-dimensional Lie algebras_ , Bull. Amer. Math. Soc. (N.S.) 5 (1981), no. 2, 185–189. MR 621889 (84b:17011)
  * [Gro00] Christian Groß, _$s$ -Representations for involutions on affine Kac-Moody algebras are polar_, manuscripta math. 103 (2000), 339–350.
  * [Hei06] Ernst Heintze, _Towards symmetric spaces of affine Kac-Moody type_ , Int. J. Geom. Methods Mod. Phys. 3 (2006), no. 5-6, 881–898. MR MR2264395 (2007g:17024)
  * [Hel01] Sigurdur Helgason, _Differential geometry, Lie groups, and symmetric spaces_ , Graduate Studies in Mathematics, vol. 34, American Mathematical Society, Providence, RI, 2001, Corrected reprint of the 1978 original. MR MR1834454 (2002b:53081)
  * [Hor09] Max Horn, _Involutions of Kac-Moody groups_ , Ph.D. thesis, Technische Universitẗ Darmstadt, 2009.
  * [HPTT95] Ernst Heintze, Richard S. Palais, Chuu-Lian Terng, and Gudlaugur Thorbergsson, _Hyperpolar actions on symmetric spaces_ , Conf. Proc. Lecture Notes Geom. Topology, IV, pp. 214–245, Int. Press, Cambridge, MA, 1995. MR MR1358619 (96i:53052)
  * [Jul85] Bernard Julia, _Kac-Moody symmetry of gravitation and supergravity theories_ , Lectures in Applied Mathematics, vol. 21, American Mathematical Society, Providence, 1985.
  * [Kac90] Victor G. Kac, _Infinite-dimensional Lie algebras_ , third ed., Cambridge University Press, Cambridge, 1990. MR MR1104219 (92k:17038)
  * [KM95] Seok-Jin Kang and D.J. Melville, _Rank $2$ symmetric hyperbolic Kac-Moody algebras_, Nagoya Math. J. 140 (1995), 41–75.
  * [KMW88] V.G. Kac, R.V. Moody, and M. Wakimoto, _On $e_{10}$._, Differential geometrical methods in theoretical physics, Proc. 16th Int. Conf., NATO Adv. Res. Workshop, Como/Italy 1987, NATO ASI Ser., Ser. C 250., 1988, pp. 109–128.
  * [KP85] V. G. Kac and D. H. Peterson, _Defining relations of certain infinite-dimensional groups_ , Astérisque (1985), no. Numero Hors Serie, 165–208, The mathematical heritage of Élie Cartan (Lyon, 1984). MR 837201 (87j:22027)
  * [KP87] Victor G. Kac and Dale H. Peterson, _On geometric invariant theory for infinite-dimensional groups_ , Algebraic groups Utrecht 1986, Lecture Notes in Math., vol. 1271, Springer, Berlin, 1987, pp. 109–142. MR 911137 (89b:22028)
  * [KPN12] Axel Kleinschmidt, Jakob Palmkvist, and Hermann Nicolai, _Modular realizations of hyperbolic weyl groups_ , Advances in Theoretical and Mathematical Physics 16 (2012), no. 1, 97–148.
  * [Kra02] Linus Kramer, _Loop groups and twin buildings_ , Geometriae Dedicata 92 (2002), 145–178.
  * [Kum02] Shrawan Kumar, _Kac-Moody groups, their flag varieties and representation theory_ , Progress in Mathematics, vol. 204, Birkhäuser Boston Inc., Boston, MA, 2002. MR MR1923198 (2003k:22022)
  * [KW92] V. G. Kac and S. P. Wang, _On automorphisms of Kac-Moody algebras and groups_ , Adv. Math. 92 (1992), no. 2, 129–195. MR 1155464 (93f:17041)
  * [LM79] James Lepowsky and Robert V. Moody, _Hyperbolic Lie algebras and quasiregular cusps on Hilbert modular surfaces_ , Math. Ann. 245 (1979), no. 1, 63–88. MR 552580 (81c:10030)
  * [Mit88] Stephen A. Mitchell, _Quillen’s theorem on buildings and the loops on a symmetric space_ , L’Enseignement Mathematique 34 (1988), 123–166.
  * [Mor88] Jun Morita, _Root strings with three or four real roots in Kac-Moody root systems_ , Tohoku Math. J. (2) 40 (1988), no. 4, 645–650. MR 972252 (89k:17044)
  * [MP95] Robert V. Moody and Arturo Pianzola, _Lie algebras with triangular decomposition_ , John Wiley Sons, New York, 1995.
  * [Pen16] Diego Penta, _The decomposition of the hyperbolic Kac-Moody algebra $\mathcal{F}$ with respect to its rank $2$ Fibonacci subalgebra_, Ph.D. thesis, Binghamton University, SUNY, 2016.
  * [PS86] Andrew Pressley and Graeme Segal, _Loop groups_ , Oxford Mathematical Monographs, The Clarendon Press Oxford University Press, New York, 1986, Oxford Science Publications. MR MR900587 (88i:22049)
  * [RC89] Enriqueta Rodriguez-Carrington, _Kac-Moody groups: An analytic approach_ , ProQuest LLC, Ann Arbor, MI, 1989, Thesis (Ph.D.)–Rutgers The State University of New Jersey - New Brunswick. MR 2638155
  * [Rém02] Betrand Rémy, _Groupes de Kac-Moody déployés et presque déployés_ , Société mathematiques de France, Paris, 2002.
  * [Ron89] Mark Ronan, _Lectures on buildings_ , Perspectives in Mathematics, vol. 7, Academic Press Inc, San Diego, 1989.
  * [Tic14] Alexander Tichai, _Some lightcone embeddings of Lorentzian Kac–Moody algebras_ , Masters thesis, TU Darmstadt, 2014.
  * [Tit86] Jacques Tits, _Ensembles ordonnés, immeubles et sommes amalgamées_ , Bull. Soc. Math. Belg. Sér. A 38 (1986), 367–387 (1987). MR 885545 (88j:20041)
  * [Tit87] by same author, _Uniqueness and presentation of Kac-Moody groups over fields_ , Journal of Algebra 105 (1987), 542–573.
  * [Wes01] Peter West, _$e_{11}$ and $m$-theory_, Classical and Quantum Gravity 18 (2001), 4443–4460.