# Enabling On-Device Smartphone GPU based Training: Lessons Learned

Anish Das  University of Cambridge
<EMAIL_ADDRESS>Young D. Kwon  University of Cambridge
<EMAIL_ADDRESS>Jagmohan Chauhan  University of Southampton
<EMAIL_ADDRESS>Cecilia Mascolo  University of Cambridge
<EMAIL_ADDRESS>
###### Abstract

Deep Learning (DL) has shown impressive performance in many mobile
applications. Most existing works have focused on reducing the computational
and resource overheads of running Deep Neural Networks (DNN) inference on
resource-constrained mobile devices. However, the other aspect of DNN
operations, i.e. training (forward and backward passes) on smartphone GPUs,
has received little attention thus far. To this end, we conduct an initial
analysis to examine the feasibility of on-device training on smartphones using
mobile GPUs. We first employ the open-source mobile DL framework (MNN) and its
OpenCL backend for running compute kernels on GPUs. Next, we observed that
training on CPUs is much faster than on GPUs and identified two possible
bottlenecks related to this observation: (i) computation and (ii) memory
bottlenecks. To solve the computation bottleneck, we optimize the OpenCL
backend’s kernels, showing 2x improvements (40-70 GFLOPs) over CPUs (15-30
GFLOPs) on the Snapdragon 8 series processors. However, we find that the full
DNN training is still much slower on GPUs than on CPUs, indicating that memory
bottleneck plays a significant role in the lower performance of GPU over CPU.
The data movement takes almost 91% of training time due to the low bandwidth.
Lastly, based on the findings and failures during our investigation, we
present limitations and practical guidelines for future directions.

###### Index Terms:

GPU, Training, Smartphones, OpenCL

##  I Introduction

Deep Learning (DL) has gained much significance in the last decade due to its
ability to model complex non-linear data efficiently. It has been successful
in many domains, including computer vision, speech processing and natural
language processing. Existing works are increasingly moving towards supporting
DL on resource-constrained mobile and Internet of Things (IoT) devices via
optimizations designed to reduce the computation and memory requirements of
Deep Neural Network (DNN) inference (i.e., forward pass) [1, 2]. To name a
few, weight quantization [1, 3], pruning [2], vector quantization [4], and new
neural architectures [5, 6] have been proposed to make inference more
efficient. However, the other side of the coin, DNN training (i.e., forward
and backward pass) on mobile systems such as smartphone’s heterogeneous
processors , has been largely neglected [7]. Moreover to address the
increasing need of personalizing the deployed model according to the different
user behaviors and changing environments effectively [8, 9], on-device
training become essential in enabling pervasive computing [10]. Further, on-
device training could provide personalization with lower latency and increased
privacy because the data does not need to be shipped off to a server farm and
the computation can be done on the device itself [11, 12].

Modern devices, especially smartphones and smartwatches, are equipped with an
array of accelerators such as GPU, which can realize the vision of enabling
training on-device as it supports SIMD operations, i.e. the ability to apply
the same operation on multiple data points in parallel, thereby expediting the
process of learning by DNNs. However, popular DL frameworks such as TensorFlow
[13] and PyTorch [14] primarily focus on DNN training on a server or a cloud
where abundant resources exist. Further, their lightweight counterparts for
mobile devices (i.e., TensorFlow Lite and PyTorch Mobile) optimize the
inference, leaving the training on mobile GPU not thoroughly optimized nor
studied yet. Moreover, the recent development of mobile DL frameworks (e.g.,
MNN [15], Core ML) enables on-device training on smartphones. Such frameworks
mainly focus on utilizing mobile CPU, which thereby could waste an opportunity
to exploit heterogeneous hardware accelerators on-device.

In this paper, we investigate the feasibility of on-device training on
smartphones using mobile GPUs for the first time: we evaluate to what extent
operations can be optimized on a mobile GPU and provide lessons learned while
doing the work. For our purposes, we adopt to optimize the MNN framework
(OpenCL backend) since MNN is open-sourced that can run on Android and iOS
devices while Core ML is proprietary. Initially, we executed MNN’s OpenCL
backend and ran the training with the MNIST dataset [16]. We observed that the
CPU was much faster than the GPU. Thus, we proceeded to identify potential
reasons for the poor performance and then improve the training performance on
GPUs compared to CPUs. Through our experiments on smartphones, we found out
that computation-bound and memory-bound bottlenecks are the two primary
reasons for the inefficiency of GPU on mobile platforms.

We firstly focused on solving computational bottlenecks. For this purpose, we
introduce kernel optimizations for the matrix multiplication kernels in OpenCL
to overcome the identified problems. After successfully implementing the
optimizations, the optimized GPU kernels show speedups (40 and 70 GFLOPs) over
CPU (15 and 30 GFLOPs) on One Plus 6 and One Plus 8 Pro. Nevertheless, we
failed to gain improvements in the performance of full DNN training on the
GPU. Upon further investigation, we discover that the memory bottlenecks
hamper the overall goal of faster training for DNN using GPU (over CPU) on
mobile devices. Our profiling experiments demonstrated that it takes about
10$\sim$15 $\times$ longer to copy data to and from the GPU than the actual
computation time on all three devices. Therefore, even though we made
significant improvements to the computation speed (optimizing matrix
multiplications), ultimately, we see little improvement. We envision that
solving this challenge will require the involvement of different disciplines,
including hardware, architecture and systems that can enable faster training
on mobile platforms using GPU.

The rest of the paper is organized as follows. Section II provides the
background knowledge of GPU architecture and the OpenCL programming model.
Then, Section III presents our kernel optimization techniques and study design
to evaluate our GPU kernel implementation. After that, Section IV presents
experimental results, and Section V discusses the limitations, practical
guidelines, and future work. Finally, we conclude the paper in Section VI.

##  II Preliminaries

This section provides a brief overview of the architecture of GPUs and the
OpenCL programming model.

###  II-A GPU architecture

Figure 1: High Level Overview of the Adreno GPU architecture for OpenCL

To explain the architecture of the smartphone GPUs, we use a high level
diagram of Adreno GPUs from the perspective of programming in OpenCL. As shown
in Figure 1, the GPU is comprised of a set of Shader Processors (SP). The SPs
are comprised of many ALUs which execute in lock-step. Each ALU has different
memory spaces available to it with drastically different latencies. The memory
spaces on the SP include registers, which is very fast, followed by Local
Memory, accessible to all ALUs. Outside the SP, there is the L1 cache which is
read-only and is used for Texture Processing, the L2 cache where the
Buffers/Image datatypes are stored and finally, the global memory (system
memory).

###  II-B OpenCL programming model

OpenCL is a framework for writing computationally-oriented kernels to run on
heterogeneous computing devices. It has a controller-device execution model, a
host process usually on CPUs, creating and managing tasks to be run on GPUs of
a computing device. The computing device can be any form of a hardware
accelerator such as GPU, Digital Signal Processor (DSP), or Field Programmable
Gate Arrays (FPGA). The host’s responsibility is to compile the kernel code
for the target device and then send an execution task to the command queue.
The command queue is responsible for scheduling tasks onto GPUs, and it runs
on the host process (sometimes asynchronously).

The four main parts of the OpenCL programming model are:

  1. 1.

Work Item: This can be regarded as an independent thread that runs the kernel
program. It is called fiber in the Qualcomm Adreno GPUs.

  2. 2.

Work Group: A collection of threads/work-item that execute in lockstep are
called a workgroup. These will be scheduled onto an SP (from Figure 1). The
workgroup has a limit due to the fixed number of SP.

  3. 3.

Local Size: The local sizes define the dimension of each workgroup. In OpenCL,
up to a three-dimensional arrangement of work-item is possible. The workgroup
size is the product of the dimensions.

  4. 4.

Global Size: Global size defines the total number of threads that need to be
run. It can also be up to 3 dimensional, and there is no limit.

The memory hierarchy in OpenCL closely tracks the memory arrangement on GPUs
as described previously. It has four types of memory as follows.

  1. 1.

Private Memory: The registers generally account for this memory. It is only
accessible to each thread, i.e. each SP has its private memory and any
intermediate variables created are stored here.

  2. 2.

Constant Memory: This is a fast read-only memory that can also be written to
by the host, generally used for textures. In the Qualcomm
guidebook111https://developer.qualcomm.com/qfile/33472/80-nb295-11_a.pdf, the
constant memory is stored in the L1 cache.

  3. 3.

Local Memory: This is a very fast memory that is cached on the SP. It is the
memory that is shared among all the threads in a workgroup.

  4. 4.

Global Memory: The largest and the slowest memory shared by all of the
threads.

Name | System-on-Chip (Qualcomm) | CPU | GPU
---|---|---|---
|  | Octa-core (1x2.4 GHz Kryo 475 Prime & |
One Plus Nord | Snapdragon 765G 5G (7 nm) | 1x2.2 GHz Kryo 475 Gold & | Adreno 620
|  | 6x1.8 GHz Kryo 475 Silver) |
|  | Octa-core |
One Plus 6 (OP6) | Snapdragon 845 | 4×2.8 GHz Kryo & | Adreno 630
|  | 4×1.8 GHz Kryo |
|  | Octa-core (1x2.84 GHz Kryo 585 & |
One Plus 8 Pro (OP8) | Snapdragon 865 5G (7 nm+) | 3x2.42 GHz Kryo 585 & | Adreno 650
|  | 4x1.8 GHz Kryo 585) |
TABLE I: The employed devices and their specifications.

##  III Methodology

In this section, we first present how we optimize the GPU kernels of MNN to
maximize the parallel computations of matrix multiplication (§III-A). We then
describe the experimental setup in §III-B. After that, we explain the study
design for evaluating our optimized GPU kernels compared to the naïve GPU
kernel and CPUs in §III-C.

###  III-A Kernel Optimizations

Matrix multiplication operation is one of the most crucial tasks in the DL
computation cycle, and hence we optimize this operation using kernel
optimizations in this subsection. We discuss the improvements made to a naïve
kernel that lead to the improved performance. For this step, we tried
optimization strategies for Single-precision matrix multiplication (SGEMM). We
mainly focus on and describe two types of optimizations: Tiling and
Vectorization, as they provide the best performance amongst all the available
optimization strategies in our experiments.

Figure 2:  The visualization of the tiling kernel optimization. A work group
is assigned to calculate the block $C_{(12)}$. During the calculation, the
kernel first loads blocks $A_{(11)}$ and $B_{(21)}$ and proceeds to calculate
the partial solution. Then it loads the next blocks $A_{(12)}$ and $B_{(22)}$
and uses them to compute the other half of the partial solution.

####  III-A1 Naïve Kernel

The naïve kernel assigns one work item to each output position in the output
matrix C in Figure 2. It then iterates over the common dimension in [MxK]
times [KxN]. It then iterates over all the K values in the rows of A and
columns of B. Thus, it is a very inefficient kernel because threads next to
each other on a workgroup will request the same row/column. Moreover, it does
not take advantage of the spatial and temporal locality related to the
computation of the kernel. Each workgroup executes in lockstep, and if all
work items continuously request indices from the input matrices, it would lead
to clashes, consequently, the data would have to be fetched from the global
memory. Two simple optimizations showing the best performance improvement are
described next.

####  III-A2 Vectorization

OpenCL provides functionality that allows for vector data types to be used in
the kernel code. Vector data types are, as the name suggests, data-types
arranged in a contiguous block of size 2, 4, 8 or 16. It is known that memory
is stored in cache lines (32 bytes long) for mobile GPUs. When data is read
in, the entire line is read in, and therefore, when we load the value at some
index, the data at the neighboring positions are also loaded in (‘Spatial
Locality’). When we use vector data types, we can exploit the spatial locality
as the load and store operations very closely mirror the actual operations in
the hardware. Through experimentation, we found that a vector size of four is
optimal.

In addition, the vectorization has a secondary benefit of having a lower
workgroup count because each work item is now responsible for multiple indices
in the output matrix. With a lower workgroup count, a larger number of kernels
can be scheduled to execute in each warp and consequently boost the overall
throughput.

####  III-A3 Tiling

For tiling, we leverage the local memory (shared among the work items in a
workgroup) to cache a portion of the input matrices and then reuse them for
each work item. This step is done iteratively, and it reduces the number of
load operations by a factor equal to the local size in each dimension. Thus,
the values at each index need to be loaded only a few times. In this step, we
create a matrix in local memory which has the same shape as the workgroup.
Next, each workgroup is assigned a task to load a specific index, and each
index is loaded concurrently. Then, the intermediate results are calculated
for the cached matrix portion for each index, respectively. Figure 2
visualizes the tiling kernel optimization.

Figure 3: Profiling results of the optimized OpenCL backend kernels for the
matrix multiplication operation.

###  III-B Experimental Setup

####  III-B1 Hardware

In our experiments, we use three devices: One plus Nord, One Plus 6, and One
Plus 8 Pro. The details of the mobile processors (i.e., CPU and GPU present on
smartphones) are shown in Table I. The One Plus 6 is the oldest phone and was
released in May 2018. The other two devices were released in 2020, with the
One Plus 8 released as a flagship and the Nord released as a cheaper
alternative. Therefore, the Nord has the older version of Adreno GPU but has a
newer CPU compared to the One Plus 6.

####  III-B2 Profiling Tools

We use two profiling tools: (1) GPU timer and (2) OpenCL profiler. Firstly,
the GPU timer is a tool present in the OpenCL API that allows an Event object
to be attached to a kernel invocation. The Event object is a bookkeeping tool
that tracks the time it takes to process the kernel call, i.e. time it takes
to add the kernel call onto the command queue. It also tracks the time the
command spends in the queue and its time to execute on the GPU. The GPU timer
is used in the profiling experiments to accurately measure the latency to run
the kernels on the GPU.

Secondly, the MNN framework provides an option to use the OpenCL profiler,
which tracks all the operations being performed and reports their time to
execute. It uses the GPU timer for execution time and further reports its time
to copy data to and from the GPU.

###  III-C Study Design

####  III-C1 Profiling Kernel Optimizations

To profile the matrix multiplication kernels, we ran the matrix multiplication
operation on square matrices of various sizes starting from [32$\times$32] to
[512$\times$512] in steps of 32. The operation was run multiple times by
queuing the operation multiple times with the same arguments. Keeping the
arguments the same ensures that we do not incur the cost of moving the data
to/from the GPU. We ran a few warm-up operations since the execution speed
usually takes some time to reach the maximum speed. We ran the matrix
multiplication for 500 hot runs and track the execution time following the
warm-up steps. After collecting the execution times, the GFLOPS value is
calculated using the standard formula: FLOPS = $2n^{3}$ / (avg. execution
time)

####  III-C2 Profiling Back Propagation

After we confirmed that the matrix multiplication operation on the GPU is
faster than the CPU on all the tested smartphones (after kernel
optimizations), the next step was to incorporate these new kernels into the
existing MNN training API. When we ran the experiments on the training
throughput, we employed the MNIST dataset [16] since it is a commonly used
dataset for an initial benchmark. In the context of training, the operation
that is most important and the one that takes the most amount of time is the
backward propagation. From our experiments, we report the backward propagation
time on all the devices for each of the models at a fixed batch size of 32.
Furthermore, we also report the ratio of execution time on the GPU over the
execution time on CPU. The ratios are meant to reduce the cognitive load when
trying to compare the CPU vs GPU. A simple feed-forward network model with
four layers (and nodes) as: $784\rightarrow 256\rightarrow 128\rightarrow 16$
was used to run the experiments. The time taken for each backward pass was
measured after discarding the first few warm-up runs. Our optimized kernels
did not help improve GPU’s performance over CPU for training and hence we
investigated further with different model architectures to find the underlying
cause. These details are described next.

Model Architecture. Having failed at beating the CPU in the experiments
devised above, we move on to trying out different architectures to test how
size affects the latency. Instead of finding a new dataset to meet our input
size requirements, we chose to use randomized inputs (matrices filled with
random numbers) because we are only interested in the average time it takes
for the forward and backward pass.

We used feed-forward neural networks with the following shapes in our
experiments:

  * •

Model1: $16\rightarrow 16$

  * •

Model2: $16\rightarrow 16\rightarrow 16\rightarrow 16$

  * •

Model3: $256\rightarrow 256$

  * •

Model4: $784\rightarrow 256\rightarrow 128\rightarrow 16$

Figure 4: The back propagation time on (a) CPU, (b) our improved kernel GPU
implementation, and (c) the baseline MNN GPU implementation for all the three
devices. The ratio of the execution time on GPUs vs. CPUs of (d) our improved
kernel implementation and (e) the baseline MNN GPU implementation.

####  III-C3 Profiling Compute and Memory Operations for Training

After we observed little change in the ratios (backward propagation time on
GPU/CPU) for the baseline-MNN kernels and the improved kernels, we devised a
final experiment to calculate the time taken for moving the matrices onto the
GPU memory and the time spent on actual computation. We employed the OpenCL
Profiler, part of the MNN framework, to measured the times for all models
mentioned before.

##  IV Results

###  IV-A Profiling Kernel Optimizatons

We now present the profiling results of kernel optimization for the matrix
multiplication operation. Figure 3 shows the speed of different matrix
multiplication kernels on CPU and GPU in terms of GFLOPS. To begin with, the
naive GPU kernel is slower than CPUs on all the three smartphones. Then, our
optimized GPU kernels improve the GFLOPS remarkably from 5, 10, 15 GFLOPS to
25, 40, 70 GFLOPS on One Plus Nord, One Plus 6, and One Plus 8 Pro,
respectively. In case of One Plus Nord which is based on a Snapdragon 765
processor, our optimized GPU kernel implementation shows similar performance
as CPU. But, for One Plus 6 and One Plus 8 Pro which are based on Snapdragon 8
series processors equipped with better mobile GPUs, our optimized GPU kernels
start to outperform CPU. The execution speed on GPU is at least 2x faster than
CPU on both the devices.

###  IV-B Profiling Back Propagation

After we obtained better performance on GPU than CPU for matrix
multiplication, we moved to profile the back propagation using our optimized
kernels. Figure 4 shows the overall latency of the backward pass (a-c) and the
ratio of running time with GPU over CPU (d-e). The comparison of the execution
time between the smaller and larger models (Model1 vs. Model3 and Model2 vs.
Model4) is roughly similar. This shows that the computation on the GPU is very
fast. However, the deeper models (Model2 and Model4) show the longer overall
back propagation latency on GPUs compared to the shallow models (Model1 and
Model3). This result indicates that the data movement is the major bottleneck
that hinders the overall performance of GPU. The data movement bottleneck
affects the deeper models more because each layer in the feed-forward neural
networks requires new matrix multiplication operations, and each operation
requires additional data movement. For example, the back propagation time of
the deeper models with three layers is about 3$\times$ longer than the shallow
models. Thus, the back propagation time grows linearly with the number of
layers. In addition, the higher ratio means CPU is faster than GPU. First of
all, we find that the ratios for the baseline MNN kernel and optimized GPU
kernel have little to no improvement in back propagation latency as optimized
GPU kernel does not show consistent improvement over the baseline as in
Figures 4d and 4e. We attribute this result to the fluctuations in the time
taken for data movement since the improvement in computation times are very
consistent as demonstrated in §IV-A. However, there is a rise in relative
performance for the larger models: Model3 and Model4, which have small ratio
values as larger models benefit more from computation parallelism than smaller
models (i.e., Model1 and Model2). This result reaffirms that the major factor
of low performance of back propagation latency on mobile GPUs is the memory
bound.

###  IV-C Profiling Compute and Memory Operations for Training

To confirm that the problem lies in the memory bandwidth between the CPU and
GPU, we estimate the time spent on computation versus time spent on data
movement between the main memory and the GPU memory. We use the OpenCL
profiler in MNN and check the average the time spent on copying and the time
spent on kernel execution during the training. The results indicate that the
computation time accounts for only about 9% of the total training time in all
three of the phones employed. Therefore, these results further confirm that
the primary factor of slower training time on mobile GPUs is the memory
bottleneck rather than the computation bottleneck.

##  V Discussion and Future Directions

Despite the best of efforts, this work has a few shortcomings and also
provides practical suggestions for future work. First, we need the minimum
matrix dimension to be 16 for all the matrices involved (inputs and outputs).
Although it does not affect the overall profiling, it would be interesting to
accommodate arbitrary matrix sizes. Further, since the focus was on
accelerating matrix multiplication operation and thus enabling feed-forward
network models, this work has not investigated convolutional neural networks.
However, since convolutional layers are computationally more expensive than
fully-connected layers (i.e., our target to improve in this work) as analyzed
in [17] and the real bottleneck of on-device training is memory bound as in
Section 4, analyzing and improving computationally expensive convolutional
layers can be a potential future direction. Our finding, memory bottleneck
issue, also suggests investigating how to enable on-device training with
memory optimization in terms of model, optimizer, and the activation [18].

##  VI Conclusion

We have investigated the feasibility of on-device training on resource-
constrained mobile GPUs with three different smartphones based on the OpenCL
backend of the mobile DL framework, MNN. First, we started with the
observation that training on mobile GPUs is much slower than that on mobile
CPUs. We then identified two obstacles regarding computation and memory.
Through extensive GPU kernel optimization to maximize the computation
parallelism, we improved the throughput of GPUs over CPUs, resolving
computation bound problem. However, our experimental results show that overall
training time on GPU remains higher than CPU, indicating that memory bound is
a major factor contributing to the inefficient training on mobile GPUs. Though
we failed to accelerate the full training on GPUs over CPUs, our research
findings could provide practical guidelines for further development of on-
device training and better utilization of heterogeneous mobile processors with
CPUs and GPUs in the near future.

## Acknowledgment

This work is supported by a Google Faculty Award 2019 and by Nokia Bell Labs
through their donation for the Centre of Mobile, Wearable Systems and
Augmented Intelligence to the University of Cambridge

## References

  * [1] S. Han, H. Mao, and W. J. Dally, “Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding,” _arXiv:1510.00149 [cs]_ , Feb. 2016.
  * [2] T. Zhang, S. Ye, K. Zhang, J. Tang, W. Wen, M. Fardad, and Y. Wang, “A Systematic DNN Weight Pruning Framework using Alternating Direction Method of Multipliers,” 2018, pp. 184–199.
  * [3] B. Jacob, S. Kligys, B. Chen, M. Zhu, M. Tang, A. Howard, H. Adam, and D. Kalenichenko, “Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference,” vol. abs/1712.05877, 2017\.
  * [4] J. Martinez, J. Shewakramani, T. W. Liu, I. A. Barsan, W. Zeng, and R. Urtasun, “Permute, Quantize, and Fine-Tune: Efficient Compression of Neural Networks,” 2021, pp. 15 699–15 708.
  * [5] M. Sandler, A. Howard, M. Zhu, A. Zhmoginov, and L.-C. Chen, “MobileNetV2: Inverted Residuals and Linear Bottlenecks,” in _2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , Jun. 2018, pp. 4510–4520.
  * [6] N. Ma, X. Zhang, H.-T. Zheng, and J. Sun, “ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Design,” 2018, pp. 116–131.
  * [7] J. Liu, J. Liu, W. Du, and D. Li, “Performance Analysis and Characterization of Training Deep Learning Models on Mobile Devices,” _arXiv:1906.04278 [cs, stat]_ , Sep. 2019.
  * [8] Y. D. Kwon, J. Chauhan, A. Kumar, P. Hui, and C. Mascolo, “Exploring system performance of continual learning for mobile and embedded sensing applications,” in _ACM/IEEE Symposium on Edge Computing_. Association for Computing Machinery (ACM), 2021.
  * [9] S. Servia-Rodriguez, C. Mascolo, and Y. D. Kwon, “Knowing when we do not know: Bayesian continual learning for sensing-based analysis tasks,” _arXiv:2106.05872 [cs]_ , Jun. 2021.
  * [10] Z. Zhou, X. Chen, E. Li, L. Zeng, K. Luo, and J. Zhang, “Edge Intelligence: Paving the Last Mile of Artificial Intelligence With Edge Computing,” _Proceedings of the IEEE_ , vol. 107, no. 8, pp. 1738–1762, Aug. 2019.
  * [11] J. Chauhan, Y. D. Kwon, P. Hui, and C. Mascolo, “ContAuth: Continual Learning Framework for Behavioral-based User Authentication,” _Proc. IMWUT_ , vol. 4, no. 4, pp. 122:1–122:23, Dec. 2020.
  * [12] Y. D. Kwon, J. Chauhan, and C. Mascolo, “FastICARL: Fast Incremental Classifier and Representation Learning with Efficient Budget Allocation in Audio Sensing Applications,” in _Proc. Interspeech 2021_ , 2021, pp. 356–360.
  * [13] M. Abadi, P. Barham, J. Chen, Z. Chen, A. Davis, J. Dean, M. Devin, S. Ghemawat, G. Irving, M. Isard, M. Kudlur, J. Levenberg, R. Monga, S. Moore, D. G. Murray, B. Steiner, P. Tucker, V. Vasudevan, P. Warden, M. Wicke, Y. Yu, and X. Zheng, “Tensorflow: A system for large-scale machine learning,” in _12th USENIX Symposium on Operating Systems Design and Implementation (OSDI 16)_. Savannah, GA: USENIX Association, Nov. 2016, pp. 265–283.
  * [14] A. Paszke, S. Gross, F. Massa, A. Lerer, J. Bradbury, G. Chanan, T. Killeen, Z. Lin, N. Gimelshein, L. Antiga, A. Desmaison, A. Kopf, E. Yang, Z. DeVito, M. Raison, A. Tejani, S. Chilamkurthy, B. Steiner, L. Fang, J. Bai, and S. Chintala, “Pytorch: An imperative style, high-performance deep learning library,” in _Advances in Neural Information Processing Systems_ , vol. 32. Curran Associates, Inc., 2019\.
  * [15] X. Jiang, H. Wang, Y. Chen, Z. Wu, L. Wang, B. Zou, Y. Yang, Z. Cui, Y. Cai, T. Yu, C. Lyu, and Z. Wu, “MNN: A Universal and Efficient Inference Engine,” _Proc. MLSys_ , vol. 2, pp. 1–13, Mar. 2020.
  * [16] Y. Lecun, L. Bottou, Y. Bengio, and P. Haffner, “Gradient-based learning applied to document recognition,” _Proceedings of the IEEE_ , vol. 86, no. 11, pp. 2278–2324, Nov. 1998.
  * [17] M. Alzantot, Y. Wang, Z. Ren, and M. B. Srivastava, “RSTensorFlow: GPU Enabled TensorFlow for Deep Learning on Commodity Android Devices,” in _Proceedings of the 1st International Workshop on Deep Learning for Mobile Systems and Applications_ , ser. EMDL ’17, New York, NY, USA, Jun. 2017, pp. 7–12.
  * [18] N. S. Sohoni, C. R. Aberger, M. Leszczynski, J. Zhang, and C. Ré, “Low-Memory Neural Network Training: A Technical Report,” _arXiv:1904.10631 [cs, stat]_ , Apr. 2019.


# Leap: molecular synthesisability scoring with intermediates

Antonia Calvi
Exscientia
<EMAIL_ADDRESS>
Théophile Gaudin
Exscientia
<EMAIL_ADDRESS>
Dominik Miketa
Exscientia
<EMAIL_ADDRESS>
Dominique Sydow
Exscientia
<EMAIL_ADDRESS>
Liam Wilbraham
Exscientia
<EMAIL_ADDRESS>

###### Abstract

Assessing whether a molecule can be synthesised is a primary task in drug
discovery. It enables computational chemists to filter for viable compounds or
bias molecular generative models. The notion of synthesisability is dynamic as
it evolves depending on the availability of key compounds. A common approach
in drug discovery involves exploring the chemical space surrounding
synthetically-accessible intermediates. This strategy improves the
synthesisability of the derived molecules due to the availability of key
intermediates. Existing synthesisability scoring methods such as SAScore,
SCScore and RAScore, cannot condition on intermediates dynamically. Our
approach, Leap, is a GPT-2 model trained on the depth, or longest linear path,
of predicted synthesis routes that allows information on the availability of
key intermediates to be included at inference time. We show that Leap
surpasses all other scoring methods by at least 5% on AUC score when
identifying synthesisable molecules, and can successfully adapt predicted
scores when presented with a relevant intermediate compound.

##  1 Introduction

The ability to predict whether a compound can be made is key to accelerating
drug discovery. Generative methods that neglect synthesisability often suggest
compounds that are challenging or impossible to make [8]. Attempting to
synthesise these problematic compounds is costly and time-consuming.
Therefore, it is crucial to assess synthesisability as part of the scoring
functions applied within generative methods to prioritise synthetically
accessible compounds.

Furthermore, the concept of synthesisability should be regarded as dynamic and
dependent on the compounds available at a specific time. Synthesis planning
algorithms [7, 11, 1, 19] are used to identify possible synthetic pathways for
specific molecules by using readily accessible compounds. However, these
methods are computationally expensive and slow to apply exhaustively within
generative workflows, which can involve the assessment of hundreds of
thousands to millions of compounds [8]. Therefore, it makes them unsuitable to
assess synthesisability when speed is required to provide feedback to
generative models. In practice, this limits the applicability of generative
workflows to real world scenarios where an fast, automated and accurate metric
of chemical tractability is critical to enable chemists to prioritise high
quality compounds that minimise development time and cost.

To address this challenge, scoring methods such as SAScore [6], SCScore [4],
and RAScore [20] were developed to assess the synthetic accessibility of
molecules. SAScore assumes that the complexity of synthesising a molecule is
correlated with the scarcity of its fragments in databases of bioactive
compounds (e.g. PubChem [13] or ChEMBL [10]) and checks for complex chemistry
like stereocenters and ring systems. SCScore is designed to ensure that, on
average, the products of chemical reactions exhibit greater synthetic
complexity compared to their reactants. RAScore [20] classifies whether a
synthetic route can be identified for a particular compound or not by the
retrosynthesis tool AiZynthFinder [11]. During inference, none of these
methods are aware of possible available intermediates for the query molecules.
In drug discovery projects, a common strategy is to identify, through the
synthesis of molecules around a specific chemical space, large quantities of
useful synthesisable intermediates from which further compounds can be readily
made [12]. When failing to adapt to this information, existing scorers can
overestimate the synthetic complexity of molecules produced in this way [16],
hindering their ability to identify compounds as synthesisable, particularly
when these intermediates are complex.

Figure 1: Schematic diagram of our workflow. a) shows an example of a
synthesis tree of depth 3, and its representation as a string; b) shows input
and expected output of our model during pre-training, while c) shows the input
and expected output of our model fine-tuned to predict tree depth for a
molecule with and without intermediate.

Our main contribution is a novel synthesisability scoring method, Leap, that
can adapt to available intermediates to better estimate the practical
synthetic complexity of a target molecule. We introduce a specific pre-
training objective, where we train a GPT-2 model to predict the entire multi-
step retrosynthesis route for a target compound. We use the retrosynthetic
planning tool AiZynthFinder to compute the routes of compounds sampled from
ChEMBL, and fine-tune the pre-trained model to predict a molecule’s synthetic
complexity with and without an intermediate. Figure 1 shows a schematic
representation of our method.

Two practically useful characteristics distinguish our method from prior art.
First, compared to existing scorers, Leap adapts its synthetic complexity
score in the presence of useful intermediates for a target molecule. There is
currently no existing tool that offers this capability. Second, we demonstrate
that even in the absence of intermediates, Leap outperforms existing scorers,
providing a strong baseline for future research in this area.

The rest of the paper is organised as follows: in Section 2 we expand on our
method and provide details on both the pre-training and fine-tuning steps; in
Section 3 we outline our experiments and discuss the results on both publicly
available data and molecules coming from real drug discovery projects; finally
we conclude and discuss further work in Section 4.

##  2 Methods

###  2.1 Problem Setting

Within this work, we also refer to a synthesis route as a synthetic tree.
Figure 1a. shows an example of this. The root of the tree is the target
molecule to be synthesised. The leaves are the building blocks and consist of
available compounds. All the other molecules are intermediates. The depth of
the tree (tree depth) is given by the maximum number of steps needed to go
from the target molecule to any of the building blocks.

Our method is based on the success of transformers [21] in predicting single-
step forward reactions (reactants to product), as well as single-step
retrosynthesis (product to reactants) [18, 22] by extending the approach to
multi-step routes. We achieve this by encoding synthesis routes as strings,
and training a transformer model (GPT-2 [17]) to predict them. To guide the
model towards the expected route, we prompt it with the expected building
blocks. Finally, we fine-tune this model using a regression head to predict
tree depth for target molecules.

###  2.2 Pre-Training

####  2.2.1 Data Representation

Our dataset consists of synthesis routes that we represent with a custom
string format. This representation encodes both the tree depth and
intermediate molecules that act as reactants and products of two reactions as
shown in Figure 1b.

Each molecule is encoded as Smiles. Inspired by the reaction smiles [2], we
represent these in a product>>reactants format, where each reactant is
separated by a dot. Reactions occurring at the same depth are separated by a ;
, while we use : to separate reactions at different depths of the tree.

Each molecule is given a unique identifier, in the format of [0],[1],[2] etc.
These are assigned in ascending order as we traverse the tree from the root to
the leaves. The target molecule (i.e. the root of the tree) is always assigned
the identifier [0]. When a molecule is encountered for the first time, it is
represented by its Smiles followed by its identifier. If that same molecule is
used in a subsequent reaction, only the identifier is used. This shortens the
length of the string, freeing GPT-2’s context window from repeated Smiles.
Figure 1a. shows an example.

To prompt the model with the expected building blocks for a route, we
introduce these at the beginning of the string, separating them from the
target molecule with a ?. As before, each building block’s Smiles is followed
by a unique identifier. We represent these with [0!],[1!],[2!] etc. Then, when
the building blocks are used along the synthesis route, we refer to them using
their identifier as shown in Figure 1.

####  2.2.2 Model Architecture and Training

Our model is a GPT-2 model with 8 attention heads and 5 hidden layers. We use
HuggingFace’s implementation [23]. The hidden states have a dimension of 512,
while the feed-forward layers have a dimension of 1024. We train the model for
16 epochs, use AdamW optimiser with a learning rate of 0.0003, and a cosine
learning rate scheduler with 2000 warmup steps. We train the model on a
language modelling task, where the goal is to predict the next token.

####  2.2.3 Dataset

To pre-train our model, we use a combination of two datasets. As no dataset of
multi-step synthetic routes exists, methods have been developed to generate
pathways from existing single-step reaction datasets. The first dataset from
Liu et al. [14], uses dynamic programming to extract synthesis routes from
patent reactions found within the USPTO-all [15] dataset. We use train, test
and validation splits from Liu et al. [14] which gives 39717, 5479 and 5493
routes for each.

The dataset from Liu et al. [14] is skewed both towards short routes of an
average depth of three, and routes that only extend down a single branch of
the synthesis tree (i.e. linear rather than convergent synthesis routes). To
increase diversity, and allow for longer and more branched routes, we extend
the training set with a second dataset based on Gao et al. [9]. This models
the problem of synthetic tree generation as a Markov decision process with a
random policy over reactants and reaction templates. Reaction templates are
rule-based patterns extracted from existing reactions to define
transformations on molecules that represent valid chemical reactions. From
this we obtain 116029 synthesis routes, which we add to the training set.

We convert all routes to the string format described in Section 2.2.1. We
augment the data by randomising the Smiles as suggested in Arús-Pous et al.
[3], and randomly permuting both the reactions located at the same depth of
the route, and the building blocks in the prompt. We apply augmentation three
times per route.

###  2.3 Fine-Tuning

The purpose of fine-tuning the model is to have it predict the expected tree
depth of a target molecule both when an intermediate is provided and when it
is not.

####  2.3.1 Data Representation

The input to the model consists of the target molecule and, when available, an
intermediate. Both molecules are represented with their respective Smiles. The
target molecule is prepended by the ? token that is used to separate it from
the potential intermediate. Therefore, the input format to the model will
either be intermediate?target or ?target depending on whether an intermediate
is included.

The output of the model is an integer that corresponds to the expected tree
depth. A simplified example of this is shown in Figure 1c.

####  2.3.2 Model Architecture and Training

We add a regression head on top of the pre-trained GPT-2 model to predict tree
depth. We tune the hyperparameters by hand. Fine-tuning only the regression
head, and the the regression head with last two hidden layers, showed worse
performance on the validation set than fine-tuning all layers of the model. We
use an AdamW optimizer with a learning rate of 0.0001, and a linear scheduler
with 4000 warmup steps. We minimise the mean squared error, and train for 16
epochs with early stopping on the validation loss.

####  2.3.3 Dataset

The fine-tuning dataset is a sample of 290418 ChEMBL [10] molecules that we
use to train our model. In addition, we curate a held-out dataset of 4989
project molecules coming directly from internal drug discovery projects, to
challenge the model on out-of-domain prediction.

We use AiZynthFinder [11] with default parameters to compute the routes of
both datasets. A route is considered to be solved if all leaves are
purchasable molecules. If more than one route is found for a molecule, the
route scored the highest by AiZynthFinder is retained. For solved routes, we
compute the depth of the tree. Since the maximum tree depth allowed by
AiZynthFinder’s default parameters is 7, we assign a depth of 10 for molecules
where a route is not found.

To create target molecule-intermediate pairs, we randomly sample a maximum of
three intermediate molecules for each route and recompute the depth
accordingly. We do the latter by treating the sampled intermediate as an
available building block, meaning that we can ignore the synthesis steps
needed to create it. At a synthesis tree level, this effectively results in
the removal of any nodes beyond the intermediate molecule. This reduces the
depth of the tree when the intermediate is found along the longest branch of
the tree.

To train the model to recognise that not every molecule is a viable
intermediate and pathologically reduce predicted route depth in response to
supplied intermediates, we also generate a set of "false" intermediates. We do
this by pairing target molecules with a random molecule sampled across all
intermediates in the dataset. For these negative pairs, the expected depth is
the same as that of the target molecule when no intermediate is supplied.

ChEMBL molecules are randomly split 70/15/15 into train, test, validation
sets, leaving 202421, 43563, 44434 molecules, respectively. After augmentation
we have a total of 646421, 139594, 142600 data points per set. The project
molecules are augmented in the same way, totalling 20277 project-intermediate
data points.

##  3 Experiments

We run experiments to assess the ability of Leap to adapt to the knowledge of
intermediates, its ability to classify the synthetic feasibility of molecules,
and whether pre-training helps. We compare to SAScore, SCScore and RAScore.
SAScore is normalised between 0 and 1 through min-max scaling. We explore
results on the ChEMBL test set as well as internal molecules set to assess
generalisation to potentially out-of-domain novel compounds.

###  3.1 Identifying synthesisable molecules

(a) Test molecule

(b) Project molecules

Figure 2: ROC-AUC curves for all scorers on both test and project molecules.

Figure 3: Distribution of scores assigned by Leap, SAScore, SCScore and
RAScore for molecules with synthetic routes of different depths.

Figure 4: Distribution of scores assigned by Leap, SAScore, SCScore and
RAScore for solved and unsolved compounds.

To determine molecular synthesisability we compute routes using AiZynthFinder
[11] for both test and project molecules. If a route can be found, the
molecule is classified as synthesisable.

To permit comparison between Leap and RAScore, SAScore and SCScore, and to
establish a baseline for further experiments, we do not consider intermediates
in this experiment. We score all molecules with all scorers, and use these to
compute the ROC-AUC curves shown in Figure 2. Leap outperforms all other
methods in terms of AUC score for both test and project molecules. This
demonstrates that our model remains useful even when intermediates are not
present. Leap also shows superior performance on the hold-out set of project
molecules, showing its ability to generalise beyond its training domain.
Conversely, even though RAScore was also trained on ChEMBL molecules, its
predictive performance deteriorates on out-of-domain molecules as previously
observed in Thakkar et al. [20].

We hypothesise that the ability of our model to better distinguish between
easy- and difficult-to-synthesise molecules is due to Leap using tree depth as
a proxy for synthetic complexity. We show, from Figure 4 that scores predicted
by Leap correlate with the tree depth predicted by AiZynthFinder. All other
scorers show very similar scores across molecules that require different
depths of synthesis. Therefore, this can make it harder to establish the
optimal threshold to differentiate between synthesisable and non-synthesisable
molecules: as shown in Figure 4 Leap is the only scorer that shows a clear
distinction in scores assigned to synthesisable and non-synthesisable
molecules.

###  3.2 Effects of intermediates

Figure 5: Barplot showing the AUC score for the various scorers when we do and
do not have a key intermediate for molecules.

In the previous experiments, we showed that even when not supplying
intermediates Leap performs better than other scorers in identifying whether a
molecule is synthesisable. Here, we explore how Leap performs when
intermediates are supplied. We use the test and project molecule sets obtained
from Section 2.3.3, both with and without intermediates. We then score all the
molecules with the scorers, however only Leap can make use of intermediates.
From Section 2.3.3, we mark molecules as synthesisable if their assigned depth
is not 10 and non-synthesisable otherwise. Note that the same molecule can go
from non-synthesisable to synthesisable if a useful intermediate is supplied.
It should also be noted that not all intermediates have this effect: aside
from including false intermediates, selecting intermediates at random can
result in some of them not contributing to the reduction of tree depth.

We compute the AUC for each scorer in identifying whether a molecule is
synthesisable, both when no intermediate is known, as well as when an
intermediate is provided. From Figure 5 we can see that for molecules for
which a key intermediate is given, Leap can differentiate between
synthesisable molecules with an AUC score of 0.89. Conversely, all other
scorers incur a decrease in their AUC scores. This is to be expected, but
highlights the limitations of such scorers: complex synthesis routes become
simpler if chemists already possess key intermediates. We further check that
our model does not pathologically lower the predicted score if any
intermediate is supplied by comparing the distributions between supplying
false and no intermediates. Figure 6 shows that distributions of predicted
scores are very close for both test and project molecules, indicating that
Leap’s predictions are robust against irrelevant intermediates.

(a) Test molecule

(b) Project Molecules

Figure 6: Distributions of predicted depths for both project and test
molecules when false, true and no intermediates are supplied.

###  3.3 Ranking molecules with intermediates

The purpose of this experiment is to investigate whether our model maintains
the ranking of the expected tree depth as different intermediates are
supplied. We group both test and project sets by their target molecules, and
then compare the expected ranking of tree depth for each target molecule with
the different intermediates, to that obtained by the prediction of our model.
If everything is ranked correctly, then that is considered a correct ranking.
For this experiment, we do not consider false intermediates. We obtained 77%
correct ranking on the test set, and 69% on the project set.

###  3.4 Comparison to baseline models

| Test | Project
---|---|---
Model | R2 | MAE | MSE | R2 | MAE | MSE
CatBoost | 0.42 | 1.32 | 3.63 | 0.19 | 1.98 | 7.18
BERT | 0.54 | 1.05 | 2.89 | 0.29 | 1.78 | 6.77
GPT-2 | 0.41 | 1.23 | 3.71 | 0.17 | 1.92 | 7.48
Leap | 0.64 | 0.88 | 2.29 | 0.41 | 1.56 | 5.59
Table 1: Comparison between performance of our model Leap to the baseline
models on regression metrics: R2, mean absolute error (MAE), mean squared
error (MSE)

To validate our architecture and our pre-training method, we conduct
experiments using three baseline models: CatBoost, GPT-2 (i.e. Leap with no
pre-training) and BERT [5]. We compare their performance on R2, mean absolute
error (MAE) and mean squared error (MSE). For CatBoost, intermediate and
target molecules are converted to ECFP4 fingerprints, and then concatenated.
For BERT, they are passed in as two sentences, separated by SEP token. The
results are reported in Table 1. We show that pre-training helped boost the
performance of Leap, and that the fine-tuned Leap outperforms both CatBoost
and BERT on all the metrics.

##  4 Conclusion

We introduced Leap, the first method for scoring the synthesisability of
molecules that considers intermediates. We have pre-trained a model on the
task of generating synthesis routes, and fine-tuned it on tree depth
prediction with and without intermediates. We have shown the ability of our
model to generalise to out-of-domain molecules and to take advantage of
intermediates provided from various points along a synthesis route. Future
work includes testing this scorer paired with a generative model, as well as
testing the effect of more challenging false intermediates, for example by
modifying intermediates along a synthesis route of a target molecule.

## References

  * [1] IBM RXN.  URL https://rxn.res.ibm.com/.
  * [2] Daylight theory: Reaction smiles and smirks.  URL https://www.daylight.com/meetings/summerschool01/course/basics/smirks.html.
  * Arús-Pous et al. [2019] Josep Arús-Pous, Simon Viet Johansson, Oleksii Prykhodko, Esben Jannik Bjerrum, Christian Tyrchan, Jean-Louis Reymond, Hongming Chen, and Ola Engkvist.  Randomized SMILES strings improve the quality of molecular generative models.  _Journal of Cheminformatics_ , 11(1):71, November 2019.
  * Coley et al. [2018] Connor W. Coley, Luke Rogers, William H. Green, and Klavs F. Jensen.  Scscore: Synthetic complexity learned from a reaction corpus.  _Journal of Chemical Information and Modeling_ , 58(2):252–261, 2018.  doi: 10.1021/acs.jcim.7b00622.  URL https://doi.org/10.1021/acs.jcim.7b00622.  PMID: 29309147.
  * Devlin et al. [2018] Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova.  BERT: pre-training of deep bidirectional transformers for language understanding.  _CoRR_ , abs/1810.04805, 2018.  URL http://arxiv.org/abs/1810.04805.
  * Ertl and Schuffenhauer [2009] Peter Ertl and Ansgar Schuffenhauer.  _Estimation of synthetic accessibility score of drug-like molecules based on molecular complexity and fragment contributions_ , volume 1.  June 2009.
  * Gao et al. [2018] Hanyu Gao, Thomas J. Struble, Connor W. Coley, Yuran Wang, William H. Green, and Klavs F. Jensen.  Using machine learning to predict suitable conditions for organic reactions.  _ACS Central Science_ , 4(11):1465–1476, 2018.  doi: 10.1021/acscentsci.8b00357.  URL https://doi.org/10.1021/acscentsci.8b00357.  PMID: 30555898.
  * Gao and Coley [2020] Wenhao Gao and Connor W. Coley.  The synthesizability of molecules proposed by generative models.  _Journal of Chemical Information and Modeling_ , 60(12):5714–5723, 2020.  doi: 10.1021/acs.jcim.0c00174.  URL https://doi.org/10.1021/acs.jcim.0c00174.  PMID: 32250616.
  * Gao et al. [2022] Wenhao Gao, Rocío Mercado, and Connor W. Coley.  Amortized tree generation for bottom-up synthesis planning and synthesizable molecular design, 2022.
  * Gaulton et al. [2011] Anna Gaulton, Louisa J. Bellis, A. Patricia Bento, Jon Chambers, Mark Davies, Anne Hersey, Yvonne Light, Shaun McGlinchey, David Michalovich, Bissan Al-Lazikani, and John P. Overington.  ChEMBL: a large-scale bioactivity database for drug discovery.  _Nucleic Acids Research_ , 40(D1):D1100–D1107, 09 2011.  ISSN 0305-1048.  doi: 10.1093/nar/gkr777.  URL https://doi.org/10.1093/nar/gkr777.
  * Genheden et al. [2020] Samuel Genheden, Amol Thakkar, Veronika Chadimová, Jean-Louis Reymond, Ola Engkvist, and Esben Bjerrum.  AiZynthFinder: a fast, robust and flexible open-source software for retrosynthetic planning.  _Journal of Cheminformatics_ , 12(1):70, November 2020.
  * Hughes et al. [2011] J P Hughes, S Rees, S B Kalindjian, and K L Philpott.  Principles of early drug discovery.  _Br J Pharmacol_ , 162(6):1239–1249, March 2011.
  * Kim et al. [2022] Sunghwan Kim, Jie Chen, Tiejun Cheng, Asta Gindulyte, Jia He, Siqian He, Qingliang Li, Benjamin A Shoemaker, Paul A Thiessen, Bo Yu, Leonid Zaslavsky, Jian Zhang, and Evan E Bolton.  PubChem 2023 update.  _Nucleic Acids Research_ , 51(D1):D1373–D1380, 10 2022.  ISSN 0305-1048.  doi: 10.1093/nar/gkac956.  URL https://doi.org/10.1093/nar/gkac956.
  * Liu et al. [2023] Songtao Liu, Zhitao Ying, Zuobai Zhang, Peilin Zhao, Jian Tang, Lu Lin, and Dinghao Wu.  Metro: Memory-enhanced transformer for retrosynthetic planning via reaction tree, 2023.  URL https://openreview.net/forum?id=9JjGZsDvHb.
  * Lowe [2012] Daniel Mark Lowe.  Extraction of chemical structures and reactions from the literature.  2012\.  doi: 10.17863/CAM.16293.  URL https://www.repository.cam.ac.uk/handle/1810/244727.
  * Makara et al. [2021] Gergely M Makara, László Kovács, István Szabó, and Gábor Pőcze.  Derivatization design of synthetically accessible space for optimization: In silico synthesis vs deep generative design.  _ACS Med Chem Lett_ , 12(2):185–194, January 2021.
  * Radford et al. [2019] Alec Radford, Jeff Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever.  Language models are unsupervised multitask learners.  2019\.  URL https://api.semanticscholar.org/CorpusID:160025533.
  * Schwaller et al. [2019] Philippe Schwaller, Teodoro Laino, Thé ophile Gaudin, Peter Bolgar, Christopher A. Hunter, Costas Bekas, and Alpha A. Lee.  Molecular transformer: A model for uncertainty-calibrated chemical reaction prediction.  _ACS Central Science_ , 5(9):1572–1583, aug 2019.  doi: 10.1021/acscentsci.9b00576.  URL https://doi.org/10.1021%2Facscentsci.9b00576.
  * Segler et al. [2018] Marwin H S Segler, Mike Preuss, and Mark P Waller.  Planning chemical syntheses with deep neural networks and symbolic ai.  _Nature_ , 555(7698):604—610, March 2018.  ISSN 0028-0836.  doi: 10.1038/nature25978.  URL https://doi.org/10.1038/nature25978.
  * Thakkar et al. [2021] Amol Thakkar, Veronika Chadimová, Esben Jannik Bjerrum, Ola Engkvist, and Jean-Louis Reymond.  Retrosynthetic accessibility score (rascore) – rapid machine learned synthesizability classification from ai driven retrosynthetic planning.  _Chem. Sci._ , 12:3339–3349, 2021.  doi: 10.1039/D0SC05401A.  URL http://dx.doi.org/10.1039/D0SC05401A.
  * Vaswani et al. [2023] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin.  Attention is all you need, 2023.
  * Wang et al. [2021] Xiaorui Wang, Yuquan Li, Jiezhong Qiu, Guangyong Chen, Huanxiang Liu, Benben Liao, Chang-Yu Hsieh, and Xiaojun Yao.  Retroprime: A diverse, plausible and transformer-based method for single-step retrosynthesis predictions.  _Chemical Engineering Journal_ , 420:129845, 2021.  ISSN 1385-8947.  doi: https://doi.org/10.1016/j.cej.2021.129845.  URL https://www.sciencedirect.com/science/article/pii/S1385894721014303.
  * Wolf et al. [2020] Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lhoest, and Alexander M. Rush.  Huggingface’s transformers: State-of-the-art natural language processing, 2020.