# Transfer learning for solvation free energies: from quantum chemistry to
experiments

Florence H. Vermeire  William H. Green  Department of Chemical Engineering,
Massachusetts Institute of Technology, Cambridge, MA, 02139, U.S.A

###### Abstract

Data scarcity, bias, and experimental noise are all frequently encountered
problems in the application of deep learning to chemical and material science
disciplines. Transfer learning has proven effective in compensating for the
lack in data. The use of quantum calculations in machine learning enables the
generation of a diverse dataset and ensures that learning is less affected by
noise inherent to experimental databases. In this work, we propose a transfer
learning approach for the prediction of solvation free energies that combines
fundamentals from quantum calculations with the higher accuracy of
experimental measurements. The employed model architecture is based on the
directed-message passing neural network for the molecular embedding of solvent
and solute molecules. A significant advantage of models pre-trained on quantum
calculations is demonstrated for small experimental datasets and for out-of-
sample predictions. The improved out-of-sample performance is shown for new
solvents, for new solute elements, and for the extension to higher molar mass
solutes. The overall performance of the pre-trained models is limited by the
noise in the experimental test data, known as the aleatoric uncertainty. On a
random test split, a mean absolute error of 0.21 kcal/mol is achieved. This is
a significant improvement compared to the mean absolute error of the quantum
calculations (0.40 kcal/mol). The error can be further reduced to 0.09
kcal/mol if the model performance is assessed on a more accurate subset of the
experimental data.

###### keywords:

transfer learning , solvation free energy , COSMO-RS , quantum chemistry ,
aleatoric uncerainty

##  1 Introduction

Deep learning has emerged as an effective technique for property prediction in
the field of chemical engineering and material science. In the last decade,
many efforts have been made to replace structure-based estimation methods by
deep neural networks [1, 2]. One major problem that is often encountered is
data scarcity. Compared to other disciplines like image recognition and
natural language processing, the availability and size of datasets in chemical
engineering and material science are very limited. Transfer learning has been
proposed as a technique to solve the problem of the low data regime [3].
Success has been demonstrated in other disciplines, such as the transfer of
knowledge from general image recognition to more specific medical imaging.
Data scarcity is not the only problem related to the experimental nature of
databases in chemical engineering and material science. They are often biased
towards certain groups of components, cover only a limited domain of chemical
space, and have an uncertainty associated with the experimental nature of the
data. With transfer learning and the use of quantum chemical calculations, one
can compensate for this bias and cover a larger chemical space by generating
additional and diverse data.

The advantage of transfer learning with respect to data scarcity in chemical
engineering and material science has been demonstrated in recent work. Within
quantum machine learning, transfer learning has been used to calculate
thermodynamic properties of molecules in vacuum at the coupled cluster level
of theory, the gold standard of quantum chemical calculations. This has been
done by Grambow et al. [4] and Smith et al. [5] Large DFT-based datasets have
been used to pre-train models that were further fine-tuned on computationally
expensive coupled cluster calculations. Ma et al. [6] demonstrated the
advantage of transfer learning for gas adsorption on metal organic frameworks.
Parameters were transferred from a model trained on a large dataset of
hydrogen gas adsorption at 100 bar and 243 K to initialize the parameters of a
model fine-tuned at 130 K and for methane adsorption with a smaller dataset.
This technique has also been effective at transfering knowledge between
disciplines of materials. Yamada et al. [7] proposed a shot-gun transfer
learning approach where models trained on small molecules were used to aid
learning of polymer properties, and models trained on organic materials were
used to aid learning of inorganic material properties. Jha et al. [8] used
transfer learning to predict the formation enthalpy of crystal structures
starting from the elemental composition. In their approach, a model was pre-
trained on a large dataset of DFT calculations. All model parameters were used
to initialize a new model that was fine-tuned on two other smaller DFT
databases and an experimental database.

In this work, we introduce an inductive transfer learning approach with the
transfer of model parameters from models trained on quantum chemistry
calculations to models trained on experimental data, similar to the approach
reported by Jha et al. [8] The transfer learning algorithm uses inductive
biases from the quantum chemistry data to improve learning of small and biased
experimental datasets. In the present work, the transfer learning method is
applied to the prediction of solvation free energies in a variety of solvents.
For the purpose of this work we provide two databases in supporting
information: (i) a quantum chemistry database with solvation free energies for
1 million solvent/solute combinations, calculated according to the COSMO-RS
theory, and (ii) an experimental database with 10145 solvent/solute
combinations compiled from publicly available databases. For solvation free
energies a rather large amount of data is available compared to other chemical
or material properties. This allows us to investigate the effect of the
dataset size and the influence of data noise or the aleatoric uncertainty on
model predictions.

Solvation free energies have been used before for the construction of deep
neural networks. The FreeSolv database [9], with only hydration free energies,
has often been used as a benchmark to compare different molecular
representations in deep learning [10, 11]. Some reported deep neural network
architectures that account for multiple molecules are trained on data from the
Minnesota Solvation database [12]. Hutchinson and Kobayashi [13] used features
for the representation of the solvent and functional class fingerprints for
the solute. Lim and Jung [14] and Pathak et al. [15] proposed architectures
with explicit solvent and solute embedding and an interaction layer to account
for pair-wise interactions between solvent and solute latent representations.
Both studies reported overall good predictions on random test splits with each
an RMSE of 0.57 kcal/mol on their respective test sets. To test the
transferability of the model to new solvents and solutes, Lim and Jung [14]
clustered the solvents and solutes, re-trained the model while leaving out one
cluster at a time, and tested performance on that cluster. On average, the
RMSE of the model increased to 1.45 kcal/mol for solvent clustering and 1.61
kcal/mol for solute clustering. Pathak et al. [15] tested the transferability
to new solvents in a similar manner by excluding certain solvents from the
training set. They achieved overall good performance on test sets that include
the left-out solvent. However, many of the considered solvents had chemical
structures very similar to other solvents in the training set. In this work,
the transferability of the model is tested by excluding specific solvents,
solutes with certain elements, and solutes based on their molar mass from the
training and validation sets. The transferability of the pre-trained models is
compared to the transferability of the models trained solely on experimental
data.

##  2 Methods - databases

###  2.1 Quantum chemical data generation

The Gibbs free energy of solvation ($\Delta G_{solv}$) at 298 K for generated
data is determined for different solvent/solute combinations using the
commercial software COSMOtherm [16]. COSMOtherm computes thermophysical data
of liquids based on the COSMO-RS theory [17, 18, 19]. Using this software,
$\Delta G_{solv}$ is calculated from the chemical potential of the solute in
the ideal gas phase and at infinite dilution in the considered solvent. The
chemical potential is determined by considering pair-wise interactions between
segments of the quantum chemical COSMO-surfaces of the solute and solvent
molecules. The COSMO-surfaces used in this work are computed at BP-TZVPD-FINE
level of theory, i.e. using a geometry optimization on the density functional
theory BP-TZVP level, a single point calculation on the BP-def2-TZVPD level
and a FINE cavity for the construction of the surface segments. Different
molecular conformations for the solvent and solute are accounted for in the
calculation of $\Delta G_{solv}$. The conformer generation workflow considers
conformers relevant for thermodynamic properties in the gas and in the liquid
phase, as implemented in the commercial software COSMOconf. The $\Delta
G_{solv}$ values in this work are calculated in the molar reference state,
meaning that $\Delta G_{solv}$ is the free energy for transferring a solute
molecule from the ideal gas phase at 1 mol/L concentration into an ideal
solution at the same solute concentration.

One of the advantages of using COSMOtherm for the calculation of $\Delta
G_{solv}$ is that once the expensive quantum calculations are done for a new
molecule and its conformers, the COSMO-surface of the new species can be
quickly combined with already available COSMO-surfaces. This allows fast
computation of new solvent/solute combinations to extend the quantum chemical
database according to the user’s needs. For the purpose of this work, a
database is generated with 1 million combinations of 284 commonly used
solvents and 11029 solutes. Solvents and solutes with elements H, B, C, N, O,
F, P, S, Cl, Br and I are included with a solute molar mass ranging from 2.02
g/mol to 1776.89 g/mol. All calculations are performed with resources of the
National Energy Research Scientific Computing Center (NERSC). The complete
database, with solvents and solutes represented by SMILES, can be found in the
Supporting Information.

The most commonly used QM database to optimize machine learning architectures
for molecular property prediction is the QM9 database published by
Ramakrishnan et al. [20] in 2014. The QM9 database contains QM properties for
134 thousand molecules in vacuum, calculated at a DFT level of theory. The QM
dataset published in this work contains 1 million solvation free energies for
different solvent/solute combinations and can be used to further optimize
machine learning architectures that consider multiple molecules and pair-wise
interactions between those molecules.

###  2.2 Experimental database

To construct the experimental database employed in this work, experimental
data from different sources are combined. Those sources include the Minnesota
Solvation database (MNSol) [12], the hydration free energy database published
by Mobley et al. [9] (FreeSolv), the database published by Moine et al. [21]
(CompSol) and a collection of data published by the Abraham research group at
University College London [22]. For each of those databases, the reported
molecular identifiers are converted to computer-readable identifiers, SMILES
and InChI. Only solvation free energies at temperatures of 298K ($\pm$2K) are
accounted for. When the gas-liquid solvation equilibrium coefficient $K$ or
$log(K)$ is reported instead of $\Delta G_{solv}$, the former is converted by
the relation $K=\exp({\Delta G_{solv}}/{RT})$.

The database is further curated by removing ionic liquids and ionic solutes.
In addition, only molecules with the same elements as available in the quantum
chemical database are allowed. Even though it would be beneficial to train a
machine learning model on a larger and more diverse experimental dataset, to
demonstrate the advantage of transfer learning, the self-solvation
experimental data are excluded from the final experimental database.

The data from different sources are combined and duplicate entries are
averaged or removed from the dataset if the standard deviation is larger than
0.20 kcal/mol. By applying this constraint, 181 entries are removed from the
database. The selected standard deviation to remove entries from the database
is based on reported experimental uncertainty of 0.20 kcal/mol for $\Delta
G_{solv}$ of neutral components [23, 24, 25]. For some components however, the
experimental uncertainty is larger and can go up to 1 kcal/mol [26]. The final
database consists of 10145 solvent/solute combinations for 291 solvents and
1368 solutes. The database is provided as part of the Supporting Information,
excluding the data from the proprietary MNSol database. The number of entries
from each of the respective data sources are given in Table 1.

Table 1: Number of database entries from each of the data sources Data source | Number of entries
---|---
MNSol | 2275
FreeSolv | 560
CompSol | 3548
Abraham | 6091

###  2.3 Comparison of the experimental and QM database

One of the advantages of using a QM database over an experimental one is the
lack of bias towards certain solvents. Over 11.8% of the experimental data has
water as a solvent and more than 10.0% is in hydrocarbon solvents such as
hexane, heptane, octane, decane and hexadecane. In the QM database, the data
is more evenly distributed between solvents, and they each appear in about
0.35% of the data.

Another advantage of using a QM database is that more and new solutes can be
considered compared to those that are experimentally available. The
experimental database has 1368 different solutes, while 11029 different
solutes are considered in the QM database. The molar mass distribution of
solutes in the experimental database and the QM database are compared in
Figure 1 (left). Note that the molar mass distribution is higher for the QM
database. The component with the highest molar mass in the experimental
database is hexabromobenzene with a molar mass of 551.49 g/mol, while in the
QM database this is hexadecabromophthalocyanine with a molar mass of 1776.89
g/mol. The QM database can be extended at a computational cost but cheaper and
faster than new experimental measurements to include more high molar mass
components and to extend the application range of the model.

The values of the solvation free energies are compared between the
experimental and QM database. There are 3164 overlapping solvent/solute
combinations between the two databases. With the COSMO-RS theory, we can
predict the experimental solvation free energy with a root-mean-square-error
(RMSE) of 0.67 kcal/mol and a mean-absolute-error (MAE) of 0.40 kcal/mol. This
is close to the MAE reported by Klamt et al. [27] of 0.42 kcal/mol for
comparison to the SM8 dataset [28]. A parity plot can be seen in Figure 1
(middle).

The distribution of $\Delta G_{solv}$ for both databases is given in Figure 1
(right). The distribution is similar between the experimental and QM database,
although a broader distribution is observed for the QM database.

Figure 1: Comparison of the experimental (exp) and computational (QM)
database. left: molar mass distribution of the solutes, middle: parity plot
for solvent/solute pairs which appear in both databases, right: distribution
of $\Delta G_{solv}$.

##  3 Methods - machine learning

###  3.1 Model architecture

The model architecture used in this work is based on the state-of-the-art
directed message passing neural network (D-MPNN) as implemented in the
software chemprop. Only details specific to this work are discussed below. For
more general information on the D-MPNN, the reader is referred to the work
published by Yang et al. [11] The software is written using the package
PyTorch and is available open-source at https://github.com/chemprop/chemprop.

A molecular identifier, SMILES or InChI, of the solvent and solute molecule
are converted into a graph-based structure by the open-source cheminformatics
software RDKit [29]. Atom and bond feature vectors are constructed for each of
the atoms and bonds in the graph representations. Those feature vectors are
adapted from the standard version of chemprop to make them more specific to
solvation related properties. The atom feature vectors contain information on
(i) the atomic number, (ii) the number of neighboring atoms, (iii) the formal
charge, (iv) the number of connected hydrogen atoms, (v) the hybridization,
(vi) the number of lone pairs, (vii) the hydrogen bond donating or accepting
character, (viii) the ring size, (ix) the aromaticity, (x) the
electronegativity and (xi) the atomic molar mass. The bond feature vectors
contain information on (i) the bond type, (ii) the conjugation, (iii) the ring
type and (iv) the stereo-chemistry. The values of the atom and bond features
are assigned by RDKit.

The feature vectors are converted into a molecular latent representation by
passing them through a convolutional neural network, more specifically a
D-MPNN. This is done separately for the solvent and solute molecules and the
latent representations of both molecules are concatenated. Some additional
molecular features, such as the RDKit-calculated topological polar surface
area and the RDKit-calculated molecular size, are concatenated with the
molecular latent representations of the solvent and solute to improve the
predictions. The concatenated embedding for the solvent and solute molecules
are passed through a second neural network for the property prediction that is
made up of linear feed forward layers. A schematic overview of the model
architecture is given in Figure 2. Note that the D-MPNN and the network for
property prediction, further referred to as the feed forward network (FFN),
are treated differently during transfer learning.

Figure 2: Demonstration of the directed message passing neural network
(D-MPNN) for solvent and solute molecular representation and the subsequent
feed forward network (FFN) for property prediction

Since this work focuses on demonstrating the benefits of transfer learning
rather than optimizing the neural networks for property prediction, many of
the hyperparameters for the model architecture and optimization of the neural
network are fixed. For the D-MPNN, the depth of the message passing is set to
4 and the size of the hidden layers to 200 for both solvent and solute
embedding. The D-MPNN linear layers have no bias and the best results are
obtained without considering dropout. A LeakyReLU activation function is used
to connect the different layers of the neural network. For the FFN, 4 layers
are considered each with a hidden size of 500. The linear layers have a bias,
no dropout, and are also connected with a LeakyReLU activation function.

Prior to training the neural network, all targets are normalized using the
standard score. The parameters of the neural network are initialized randomly
by a normal distribution as published by Glorot et al. [30], except in the
case of transfer learning where the model parameters are initialized using
parameters from the other neural network. Training of the neural network is
done in batches of 50 datapoints for 200 epochs. A Noam learning rate
scheduler is used with piece-wise linear increase and exponential decay, based
on the learning rate scheduler in the Transformer model for Natural Language
Processing [31]. The model parameters are optimized with stochastic
optimization as implemented in the Adam algorithm [32] and based on the mean-
squared-error loss. All models are trained on Nvidia Volta V100 GPUs on MIT
SuperCloud [33].

###  3.2 Transfer learning

The design of the transfer learning approach is similar to the implementation
by Grambow et al. [4] In this work, the parameters of the models trained on QM
data (the QM models), are used to initialize the parameters of new models that
are further refined using experimental data (the pre-trained models). The
parameters of the D-MPNN are frozen during optimization, while the parameters
of the FFN are allowed to optimize for 20 epochs. The methodology is
schematically presented in Figure 3. In later sections, the performance of
these pre-trained models is compared to models trained on only experimental
data (the experimental models).

Figure 3: Schematic representation of the transfer learning approach. The
learned parameters from the directed message passing neural network (D-MPNN)
are transferred from the model trained on the quantum chemical calculations to
the new model fine-tuned on experimental data. The learned parameters of the
feed forward network (FFN) for property prediction are used to initialize the
parameters of the new model trained on experimental data.

##  4 Performance of the quantum machine learning model

The QM database is used to train 10 different models with 10-fold cross
validation and random initiation of the model parameters. The train,
validation, and test set are a 80/10/10 % random split of the initial QM
database. Further, the parameters from those 10 different models will be used
for transfer learning.

###  4.1 Size of training data for the quantum chemistry model

The final models for transfer learning are trained on the complete QM database
with 1 million solvent/solute combinations. The rather large size of this new
QM dataset provides an opportunity to investigate the influence of the size of
the dataset to the model accuracy. To this purpose, the same training
procedure is done starting from randomly selected subsets of the QM database.

The size of the dataset used for training, validation, and testing is varied
between $10^{3}$ and $10^{6}$. The results of predictions on random test
splits are presented in Figure 4 as a function of the dataset size. The
presented RMSE and MAE are the average of the RMSE and MAE of each of the
models on their respective randomly selected test split. The uncertain shaded
area in Figure 4 is defined by the maximum and minimum RMSE and MAE over the
set of tested models. The average RMSE/MAE are 1.43/0.95 for $10^{3}$ data
points, 0.72/0.39 for $10^{4}$ data points, 0.25/0.12 for $10^{5}$ data points
and 0.10/0.05 for $10^{6}$ data points. Similar trends for errors as a
function of the dataset size were observed by von Lilienfeld et al. [34] who
reviewed predictions on the QM9 dataset and by Jha et al. [35] who performed
deep learning for DFT enthalpies of formation for crystal structures.

Figure 4: Model for $\Delta G_{solv}$ trained and tested using varying amounts
of COSMO-RS (quantum-derived) data. Root-mean-square-error (RMSE) and mean-
absolute-error (MAE) in kcal/mol on a randomly selected 10% test set as a
function of the dataset size. The shaded area is defined by the maximum and
minimum RMSE and MAE over the set of tested models.

###  4.2 Physical interpretation of solvents

A principal component analysis (PCA) is performed on the solvent molecular
embeddings for the 284 solvents in the QM database to demonstrate the physical
behavior learned by the QM model. The first 2 principal components (PC1 and
PC2) are graphically represented in Figure 5. Together they explain 41.9% of
the variance. The solvents are manually classified according to the presence
of features specific to their chemical structure that are considered important
for physical interactions in solvation, such as hydrogen bonding. Water is
separated as a unique solvent, as well as carbon disulfide (CS2), sulfur
dioxide (SO2), and dimethyl sulfoxide (DMSO). The first class, orange in
Figure 5, includes all molecules that can have intra-molecular hydrogen
bonding between oxygen and/or nitrogen atoms. The second class, yellow in
Figure 5, includes solvents with one oxygen or nitrogen atom containing a
hydrogen bond accepting and donating site. All other oxygen- and nitrogen-
containing components are grouped in a third class, green in Figure 5. The
first three classes include some hetero-atomic aromatic structures, for
example 2-furfuryl alcohol, pyrolle, phenol and pyridine. Those types of
solvents are, according to the first two principal components, better
classified in the first three groups rather than based on their aromaticity.
All other aromatics are included in a fourth group, indigo in Figure 5. The
two remaining groups are the halogen-containing solvents, blue in Figure 5,
and the hydrocarbon solvents, grey in Figure 5.

The first two principal components clearly separate physical properties
specific to solvation. PC1 distinguishes solvents based on their hydrogen
bonding capability and the polarity of the solvents. The highest value of PC1
is for water, followed by some molecules with the ability to make intra-
molecular hydrogen bonding, such as propane-1,2-diol, glycerol, and formamide.
The lowest values of PC1 are for a group of unsaturated hydrocarbon solvents
in the lower left corner of Figure 5, including, for example, bicyclohexane
and hexadecane. PC2 separates molecules within one class based on the length
of their hydrocarbon backbone. For example, within the halogens, carbon
tetrachloride has the highest value of PC2, whilst the lowest values of PC2
for halogenes are for fluorooctane and bromooctane. The alcohol and amine with
the lowest value for PC2 are dodecanol and dibutyl amine respectively.

Figure 5: First two principal components (PC1 and PC2) of the learned solvent
molecular embedding. The solvents are classified manually based on the
presence of features important to solvation in their chemical structure.

###  4.3 Prediction of experimental data by the quantum chemistry models

The QM models have an associated uncertainty when used to predict experimental
solvation free energies. This absolute error is made up of the error inherent
to the quantum chemical calculations and the additional error from the
structure of the machine learning model. To evaluate those uncertainties, the
10 QM models are used to predict the complete experimental dataset without
further fine-tuning of the model.

The predictions of the 10 models are averaged and the ensemble is used to
calculate a RMSE of 0.81 kcal/mol and a MAE of 0.47 kcal/mol for the QM model
predictions on experimental measurements. The absolute error of the
predictions is expected to be close to the sum of the error of the QM
calculations and the error of the model. The MAE of the former is 0.40
kcal/mol, see Figure 1 (middle), and the MAE of the latter is 0.05 kcal/mol as
can be evaluated by the performance on a random test set, see Figure 4.

##  5 Transfer learning from quantum chemical calculations to experimental
data

The parameters of the QM models are used for transfer learning according to
the procedure explained in Figure 3. The model is fine-tuned on the complete
set and subsets of the experimental dataset to demonstrate the advantage of
transfer learning for small dataset sizes. Furthermore, we elaborate on the
limited accuracy that can be reached with the employed experimental dataset
caused by the noise inherent to the experimental nature of the data (i.e. the
aleatoric uncertainty).

###  5.1 Random splits of the complete experimental dataset

The uncertainty associated with the quantum chemical calculations can be
reduced by fine-tuning the model on experimental data. The performance has
been evaluated using 10-fold cross validation. Fine-tuning the parameters of
the neural network for property prediction improves the RMSE/MAE from
0.81/0.47 kcal/mol to 0.44/0.21 kcal/mol.

For comparison with the pre-trained models, 10 experimental models are trained
solely on the experimental database with random initiation of the model
parameters. Those are trained using 10-fold cross validation to prediction the
same random test splits as the pre-trained models. The predictions of the
models on the random test splits can be seen in Figure 6 for the pre-trained
models (left) and the purely experimental models (right). Even though
differences are observed between the predicted values, the overall performance
of the experimental and pre-trained models are the same.

Figure 6: Predictions of the pre-trained models (left) and the purely
experimental models (right) on random splits of the experimental database.

###  5.2 Influence of the experimental dataset size

One of the advantages of transfer learning is the improved model performance
on small datasets. For chemical and material properties, large datasets are
often not available. For example, if a model would be trained for solvation
free energies in specific solvents, for a common solvents such as dimethyl
sulfoxide (DMSO) only 60 data points would be available. Moreover, in many
industrial processes mixtures of solvents are used instead of one-component
solvents. The more complex the solvent system gets, the more scarce the
experimental data will be. Here we demonstrate the advantage of transfer
learning with respect to small dataset sizes and that the same accuracy can be
achieved as a model trained on solely experimental data for much smaller
datasets.

A random 10% of the experimental dataset is held out and used as separate test
set. The size of the training and validation dataset is varied between 0.5%
and 70% of the remaining experimental data, corresponding to 46 and 6392 data
points respectively. In Figure 7 the RMSE of the pre-trained and purely
experimental models on the separate 10% test set (1014 data points) are given
as a function of the training and validation set size. The reported RMSE is
calculated from the ensemble of the different models. The shaded uncertainty
area in Figure 7 is defined by the minimum and maximum RMSE from each of the
individual models in the ensemble.

Figure 7 shows the decreasing error as a function of dataset size on a linear
scale (left) and a logarithmic scale (right). The training data has
significant noise leading to errors in the model predictions. This effect will
be reduced by averaging when a large training dataset is used. A simplistic
scaling argument says that effect should give a slope of -1/2 in Figure 7
(right), in rough accord with the observed slope for the purely experimental
models.

The performance of the purely experimental models significantly increases with
increasing size of the experimental dataset for the smallest fractions. This
is the case until $\sim$50% of the initial training set size, or $\sim$4567
data points. Once more data is added, the RMSE is slightly improved from 0.52
kcal/mol to 0.47 kcal/mol, but no significant further improvement of the model
is observed. The pre-trained models perform significantly better for smaller
dataset sizes. The same accuracy of 0.52 kcal/mol is reached at $\sim$20% of
the initial training set size, or $\sim$1826 data points. The resulting RMSE
on a separate test set for 1%, 5% and 10% of the initial training data (or 91,
457 and 913 data points) is improved from 2.64, 1.45 and 1.14 kcal/mol for the
purely experimental models to 0.76, 0.61 and 0.57 kcal/mol for the pre-trained
models.

Figure 7: Performance of the pre-trained (blue) and purely experimental (red)
models on a fixed 10% random test split as a function of the size of the
training and validation set in linear (left) and logarithmic (right) scale.
The shaded uncertainty area is defined by the minimum and maximum RMSE from
each of the individual models in the ensemble.

###  5.3 Aleatoric uncertainty as a limit to model performance

When working with experimental datasets, the uncertainty related to the data
noise should be accounted for in addition to the uncertainty associated with
quantum chemical calculations and the model-related uncertainty (i.e.
epistemic uncertainty). Even a perfect model will have deviations from the
test data due to noise in those data (i.e. the aleatoric uncertainty). The
aleatoric uncertainty is inherent to the experimental dataset and limits the
accuracy of predictions on that data. This limit can be observed clearly in
Figure 6 and Figure 7. A lower limit of the model performance on the
experimental test set with a MAE around 0.21 kcal/mol is approached by both
the experimental and pre-trained models. This limit is the result of the
aleatoric uncertainty in the dataset shared by both models. Note that the
quantum chemistry dataset also has an aleatoric uncertainty, but much smaller
than the one of the experimental dataset.

As a demonstration of how noise inherent to the dataset can affect the
prediction results, a subset of the experimental dataset with more accurate
experimental measurements is constructed. Entries in the database that exhibit
a low degree of variability are chosen. Those included have at least 3 unique
measurements and a standard deviation below 0.15 kcal/mol. This results in a
new experimental dataset with 629 solvent/solute combinations for 82 solvents
and 166 solutes. More than 20% of the entries has water as a solvent and more
than 10% has linear hydrocarbon solvents. This data is used as a separate test
set, while the QM models and the purely experimental models are retrained on
the remaining experimental data (i.e. excluding the new, more accurate test
set). The RMSE and MAE on the test set are calculated based on the model
ensembles. For a more accurate experimental test set, the RMSE/MAE can be
reduced from 0.44/0.21 kcal/mol to 0.14/0.09 kcal/mol for the pre-trained
models and to 0.18/0.10 kcal/mol for the purely experimental models.The
aleatoric uncertainty or noise inherent to the experimental nature of the test
set, limits the error that can be achieved for the predictions on this test
set. This result implies that the models trained on large amounts of data are
significantly more accurate than one might infer from Figure 6 and Figure 7.
Most of the error seen in those figures is due to noise in the test set, not
due to error in the model prediction. However, in the absence of less noisy
test data it is impossible to quantify exactly how accurate the models are.

##  6 Transfer learning to improve out-of-sample predictions

On random test splits, the advantage of transfer learning is clear for small
experimental dataset sizes. Besides this advantage, they are also expected to
perform better on out-of-sample predictions. To demonstrate the advantage of a
more physical model on a different region of chemical space, the QM models are
fine-tuned on different pre-defined splits of the experimental data and
compared to models trained on solely experimental data. The pre-trained models
are constructed according to the transfer learning procedure used before
(Figure 3), and the experimental models are constructed with random initiation
of the model parameters. The model parameters are refined with a pre-defined
split of the experimental dataset and used to predict $\Delta G_{solv}$ of
pre-defined test sets. The reported RMSEs and MAEs are based on the model
ensembles.

###  6.1 Solvent splits

To test the transfer learning methodology for out-of-sample solvents, some
commonly used solvents are left out of the experimental training and
validation dataset. The pre-trained and experimental models are trained with
10 different model initiations on the reduced experimental dataset and used to
predict a test set that contains all experimental data of the left-out
solvent. Note that depending on the left-out solvent, the training,
validation, and test set differ in size. The results are summarized in Table
2.

For some solvents, such as hexane, acetone, ethanol, and benzene, the
performance of the pre-trained models and the experimental models are similar.
Each of those solvents are part of a class of solvents that has a high
occurrence in the experimental database. Many other hydrocarbon, ketone,
alcohol, and aromatic hydrocarbon solvents are present in the experimental
database. This demonstrates that the D-MPNN is effective in learning
similarity in chemical structures even starting from an experimental database.
For other solvents like ethylacetate, dichloromethane (DCM), and acetonitrile,
the pre-trained models have an improved performance compared to the
experimental models. These solvents also have counterparts in the experimental
database with similar structures but not as many as the first set of solvents.
At last, for solvents with a more special chemical structure such as
tetrahydrofuran (THF), dimethyl sulfoxide (DMSO), and water, the pre-trained
models outperform the experimental models significantly.

The comparison of predictions for out-of-sample solvents demonstrates that the
D-MPNN is able to learn some similarities in chemical structures and make
accurate predictions based on those similarities. However, for solvents with
chemical structures less represented in the experimental training data, the
performance of the D-MPNN diminishes. Transfer learning from QM data can
significantly improve the model performance in this case.

When more accurate predictions are required for a new type of solvent, quantum
calculations with this solvent can be added to the QM database at low
computational cost. The pre-trained model can in this way be used to make
predictions for the new solvent quickly and more accurately than either the
direct application of the QM calculations or the experimental models.

Table 2: Performance of pre-trained models and experimental models on out-of-sample solvent test sets. The reported RMSE and MAE are calculated on the predictions of the model ensembles. THF: tetrahydrofuran, DCM: dichloromethane, DMSO: dimethylsulfoxide Solvent | Pre-trained model | Experimental model | Test data
---|---|---|---
RMSE | MAE | RMSE | MAE | size
hexane | 0.27 | 0.13 | 0.24 | 0.13 | 201
acetone | 0.24 | 0.15 | 0.23 | 0.18 | 100
ethanol | 0.50 | 0.20 | 0.51 | 0.27 | 144
benzene | 0.58 | 0.25 | 0.51 | 0.23 | 105
ethylacetate | 0.21 | 0.13 | 0.31 | 0.17 | 131
DCM | 0.25 | 0.16 | 0.34 | 0.21 | 51
acetonitrile | 0.13 | 0.10 | 0.23 | 0.18 | 67
THF | 0.24 | 0.17 | 0.47 | 0.37 | 116
DMSO | 0.34 | 0.27 | 0.95 | 0.85 | 60
water | 1.31 | 0.82 | 3.93 | 2.93 | 1200

###  6.2 Solute splits

The advantage of transfer learning on out-of-sample solutes is demonstrated
using two different types of splits. The first split type is extreme, where
certain elements are left out of the experimental training and validation set.
The second split is a more commonly used split, where the models are trained
on $\Delta G_{solv}$ for low molar mass solutes and used to predict those of
high molar mass solutes. Extension of the chemical space to include solutes
with a higher molar mass is useful for the determination of dry partitioning
coefficients and solid solubility. Latter properties are valuable in the
design of purification steps or for the selection of optimal solvents in, for
example, drug discovery and synthesis. Moreover, gas-liquid solvation free
energies for high molar mass components are often more difficult to measure
experimentally and have a higher aleatoric uncertainty as a result.

####  6.2.1 Element-based splits

In the out-of-sample solute element split, the model performance is evaluated
on unknown elements. Solutes with those elements are left out of the
experimental training and validation dataset and used to test the model
performance. Depending on the identity of the excluded solute element, the
training, validation, and test sets vary in size. The results are summarized
in Table 3.

Table 3: Performance of pre-trained models and experimental models on out-of-range solute element test sets. The reported RMSE and MAE are calculated on the ensemble predictions. Excluded | Pre-trained model | Experimental model | Test data
---|---|---|---
solute element | RMSE | MAE | RMSE | MAE | size
O | 0.91 | 0.52 | 1.97 | 1.37 | 4684
N | 1.11 | 0.62 | 2.13 | 1.45 | 1559
F | 0.84 | 0.60 | 3.66 | 3.17 | 363
S | 0.94 | 0.64 | 1.71 | 1.34 | 369
Cl | 0.63 | 0.45 | 0.83 | 0.54 | 1124
Br | 0.51 | 0.28 | 0.56 | 0.41 | 216
I | 0.44 | 0.26 | 1.23 | 0.98 | 133

In all cases the pre-trained models outperform the experimental models, since
they learned the element representation in the neural network from the QM
data. Especially for elements that have hydrogen bonding accepting and
donating sites, i.e. O, N and F, the difference between the pre-trained and
experimental models is significant. For more similar halogene elements, i.e.
Cl, Br and I, the gain from transfer learning is less significant. The
experimental models are in this case able to learn some atom features, for
example the electronegativity, from the similar halogene components.

####  6.2.2 Molar mass-based splits

As could be seen in Figure 1 (left), the QM database includes more high molar
mass solutes compared to the experimental database. Moreover, the QM database
can be extended to include more high molar mass components, such that it
covers more of the higher molar mass components for which $\Delta G_{solv}$
needs to be predicted. The advantage of transfer learning to predict $\Delta
G_{solv}$ of out-of-sample, high molar mass, solutes is demonstrated by
splitting the experimental database according to the solute molar mass.

The results for different molar mass splits are reported in Figure 8. The
training and validation dataset include all solutes with a molar mass below
the cut-off value, while the model performance is tested for all solutes with
molar mass above the cut-off value. The reported RMSE is determined from the
model ensemble and the shaded uncertainty area is defined by the maximum and
minimum RMSE found for the individual models within the ensemble. The size of
the training and validation dataset differs between 4656 and 9860 for a cut-
off molar mass equal to 100 g/mol and 300 g/mol respectively.

For all cut-off solute molar mass splits, the pre-trained models outperform
the models trained solely on experimental data. The trend in the RMSE as a
function the cut-off molar mass is defined by a combination of the increasing
size of the training dataset and the increasing complexity (or higher molar
mass) of the test set. Note that experimental data for most of the highest
molar mass components are hydration free energies, i.e. the considered solvent
is water. Typically, the model performance is worse when predicting hydration
free energies compared to other solvents (see also Table 2). As a result, the
RMSE reported in Figure 8 is significantly higher compared to the RMSE
reported on random splits.

Figure 8: Performance of the pre-trained (blue) and experimental (red) models
on a test set with solutes that have a molar mass above the cut-off value. The
models are trained on experimental data containing only solutes with a molar
mass below the cut-off value. The shaded uncertainty area is defined by the
maximum and minimum RMSE found for the individual models within the ensemble.

##  7 Conclusions

A transfer learning approach is presented that combines the advantages of
quantum chemical and experimental databases in machine learning. Transfer
learning from the quantum chemistry improves model predictions for molecules
outside the range of the experimental training data, and for all molecules if
the experimental training set is small. This is demonstrated for the
prediction of solvation free energies in a variety of solvents. A QM database
is reported with COSMOtherm calculations for 1 million solvent/solute
combinations. An experimental database is compiled from different data sources
for 10145 solvent/solute combinations.

For the purpose of this work, the directed message passing neural network
(D-MPNN) as developed by Yang et al[11] is extended to include multiple
molecules. During transfer learning, the deep neural network is first trained
using the QM database. The model parameters of the QM models are used to
initialize the parameters of new models which are refined using the
experimental database, while the parameters of the D-MPNN for the solvent and
the solute are frozen.

To demonstrate the improved performance of the pre-trained models for small
experimental dataset sizes, fractions of the experimental database are used
for training and predicting a fixed 10% test set. Especially for small
experimental datasets, with up to 2000 data points, a clear advantage of
transfer learning is demonstrated. For larger experimental datasets, the
accuracy that can be achieved is limited by the noise in the experimental
dataset. The model becomes sufficiently accurate such that the deviations
between the model predictions and the test set are limited by the noise in the
experimental test set. A significant improvement in performance, measured by
deviations between model predictions and test set data, is gained when a
subset of accurate experimental measurements with 629 datapoints is used for
testing. The excellent performance of the model at predicting this high-
accuracy data suggests that the model predictions of $\Delta G_{solv}$(298 K)
have errors of less than $\sim$0.1 kcal/mol for in-scope molecules, but even
more accurate test data would be needed to reliably determine this number.

The superior out-of-sample performance of the pre-trained models is
demonstrated with pre-defined splits of the experimental data. For left-out
solvents, left-out solute elements and a solute molar mass-based split, the
pre-trained models outperform the purely experimental models. The experimental
models are able to learn similarities in chemical structures compared to other
entries in the database. However, for true out-of-sample data, the pre-trained
models clearly have a superior performance.

## Acknowledgements

The authors acknowledge the Belgian American Educational Foundation (BAEF),
the Machine Learning for Pharmaceutical Discovery and Synthesis Consortium
(MLPDS), and the DARPA Accelerated Molecular Discovery (AMD) program (DARPA
HR00111920025) for funding. This research used HPC resources to perform
quantum calculation of the National Energy Research Scientific Computing
Center (NERSC), a U.S. Department of Energy Office of Science User Facility
operated under Contract No. DE-AC02-05CH11231. The MIT SuperCloud Lincoln
Laboratory Supercomputing Center is acknowledged for providing HPC resources
to train the deep neural networks that have contributed to the research
results reported within this paper. We thank Yunsie Chung for her help with
the database construction, and Charles McGill and Lagnajit Pattanaik for
useful discussions and revision of the manuscript.

## References

  * Butler et al. [2018] K. T. Butler, D. W. Davies, H. Cartwright, O. Isayev, A. Walsh, Machine learning for molecular and materials science, Nature 559 (7715) (2018) 547–555.
  * Toyao et al. [2020] T. Toyao, Z. Maeno, S. Takakusagi, T. Kamachi, I. Takigawa, K. I. Shimizu, Machine Learning for Catalysis Informatics: Recent Applications and Prospects, ACS Catalysis 10 (3) (2020) 2260–2297.
  * Pan and Yang [2010] S. J. Pan, Q. Yang, A Survey on Transfer Learning, IEEE Transactions on Knowledge and Data Engineering 22 (10) (2010) 1345–1359.
  * Grambow et al. [2019] C. A. Grambow, Y.-P. Li, W. H. Green, Accurate Thermochemistry with Small Data Sets: A Bond Additivity Correction and Transfer Learning Approach, The Journal of Physical Chemistry A 123 (27) (2019) 5826–5835.
  * Smith et al. [2019] J. S. Smith, B. T. Nebgen, R. Zubatyuk, N. Lubbers, C. Devereux, K. Barros, S. Tretiak, O. Isayev, A. E. Roitberg, Approaching coupled cluster accuracy with a general-purpose neural network potential through transfer learning, Nature Communications 10 (1) (2019) 2903\.
  * Ma et al. [2020] R. Ma, Y. J. Colón, T. Luo, Transfer Learning Study of Gas Adsorption in Metal–Organic Frameworks, ACS Applied Materials & Interfaces 12 (30) (2020) 34041–34048.
  * Yamada et al. [2019] H. Yamada, C. Liu, S. Wu, Y. Koyama, S. Ju, J. Shiomi, J. Morikawa, R. Yoshida, Predicting Materials Properties with Little Data Using Shotgun Transfer Learning, ACS Central Science 5 (10) (2019) 1717–1730.
  * Jha et al. [2019] D. Jha, K. Choudhary, F. Tavazza, W.-k. Liao, A. Choudhary, C. Campbell, A. Agrawal, Enhancing materials property prediction by leveraging computational and experimental data using deep transfer learning, Nature Communications 10 (1) (2019) 5316\.
  * Mobley and Guthrie [2014] D. L. Mobley, J. P. Guthrie, FreeSolv: a database of experimental and calculated hydration free energies, with input files, Journal of Computer-Aided Molecular Design 28 (7) (2014) 711–720.
  * Wu et al. [2018] Z. Wu, B. Ramsundar, E. Feinberg, J. Gomes, C. Geniesse, A. S. Pappu, K. Leswing, V. Pande, MoleculeNet: a benchmark for molecular machine learning, Chemical Science 9 (2) (2018) 513–530.
  * Yang et al. [2019] K. Yang, K. Swanson, W. Jin, C. Coley, P. Eiden, H. Gao, A. Guzman-Perez, T. Hopper, B. Kelley, M. Mathea, A. Palmer, V. Settels, T. Jaakkola, K. Jensen, R. Barzilay, Analyzing Learned Molecular Representations for Property Prediction, Journal of Chemical Information and Modeling 59 (8) (2019) 3370–3388.
  * Marenich et al. [2012] A. V. Marenich, C. P. Kelly, J. D. Thompson, G. D. Hawkins, C. C. Chambers, D. J. Giesen, P. Winget, C. J. Cramer, D. G. Truhlar, Minnesota Solvation Database (MNSOL), 2012.
  * Hutchinson and Kobayashi [2019] S. T. Hutchinson, R. Kobayashi, Solvent-Specific Featurization for Predicting Free Energies of Solvation through Machine Learning, Journal of Chemical Information and Modeling 59 (4) (2019) 1338–1346.
  * Lim and Jung [2019] H. Lim, Y. Jung, Delfos: deep learning model for prediction of solvation free energies in generic organic solvents, Chemical Science 10 (36) (2019) 8306–8315.
  * Pathak et al. [2020] Y. Pathak, S. Laghuvarapu, S. Mehta, U. D. Priyakumar, Chemically Interpretable Graph Interaction Network for Prediction of Pharmacokinetic Properties of Drug-like Molecules, Proceedings of the AAAI Conference on Artificial Intelligence 34 (01).
  * Dassault Systemes [2020] Dassault Systemes, BIOVIA COSMOtherm, 2020\.
  * Klamt [1995] A. Klamt, Conductor-like Screening Model for Real Solvents: A New Approach to the Quantitative Calculation of Solvation Phenomena, The Journal of Physical Chemistry 99 (7) (1995) 2224–2235.
  * Klamt et al. [1998] A. Klamt, V. Jonas, T. Bürger, J. C. W. Lohrenz, Refinement and Parametrization of COSMO-RS, The Journal of Physical Chemistry A 102 (26) (1998) 5074–5085.
  * Eckert and Klamt [2002] F. Eckert, A. Klamt, Fast solvent screening via quantum chemistry: COSMO-RS approach, AIChE Journal 48 (2) (2002) 369–385.
  * Ramakrishnan et al. [2014] R. Ramakrishnan, P. O. Dral, M. Rupp, O. A. von Lilienfeld, Quantum chemistry structures and properties of 134 kilo molecules, Scientific Data 1 (1) (2014) 140022\.
  * Moine et al. [2017] E. Moine, R. Privat, B. Sirjean, J.-N. Jaubert, Estimation of Solvation Quantities from Experimental Thermodynamic Data: Development of the Comprehensive CompSol Databank for Pure and Mixed Solutes, Journal of Physical and Chemical Reference Data 46 (3) (2017) 33102.
  * Grubbs et al. [2010] L. M. Grubbs, M. Saifullah, N. E. De La Rosa, S. Ye, S. S. Achi, W. E. Acree, M. H. Abraham, Mathematical correlations for describing solute transfer into functionalized alkane solvents containing hydroxyl, ether, ester or ketone solvents, Fluid Phase Equilibria 298 (1) (2010) 48–53.
  * Kelly et al. [2005] C. P. Kelly, C. J. Cramer, D. G. Truhlar, SM6: A Density Functional Theory Continuum Solvation Model for Calculating Aqueous Solvation Free Energies of Neutrals, Ions, and Solute-Water Clusters, Journal of Chemical Theory and Computation 1 (6) (2005) 1133–1152.
  * Nicholls et al. [2008] A. Nicholls, D. L. Mobley, J. P. Guthrie, J. D. Chodera, C. I. Bayly, M. D. Cooper, V. S. Pande, Predicting Small-Molecule Solvation Free Energies: An Informal Blind Test for Computational Chemistry, Journal of Medicinal Chemistry 51 (4) (2008) 769–779.
  * Geballe et al. [2010] M. T. Geballe, A. G. Skillman, A. Nicholls, J. P. Guthrie, P. J. Taylor, The SAMPL2 blind prediction challenge: introduction and overview, Journal of Computer-Aided Molecular Design 24 (4) (2010) 259–279.
  * Guthrie [2009] J. P. Guthrie, A Blind Challenge for Computational Solvation Free Energies: Introduction and Overview, The Journal of Physical Chemistry B 113 (14) (2009) 4501–4507.
  * Klamt and Diedenhofen [2015] A. Klamt, M. Diedenhofen, Calculation of Solvation Free Energies with DCOSMO-RS, The Journal of Physical Chemistry A 119 (21) (2015) 5439–5445.
  * Cramer and Truhlar [2008] C. J. Cramer, D. G. Truhlar, A Universal Approach to Solvation Modeling, Accounts of Chemical Research 41 (6) (2008) 760–768.
  * 202 [2020] RDKit: Open-source cheminformatics, 2020.
  * Glorot and Bengio [2010] X. Glorot, Y. Bengio, Understanding the difficulty of training deep feedforward neural networks, Journal of Machine Learning Research 9 (2010) 249–256.
  * Vaswani et al. [2017] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, I. Polosukhin, Attention is All you Need, in: I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, R. Garnett (Eds.), Advances in Neural Information Processing Systems, vol. 30, Curran Associates, Inc., 5998–6008, 2017.
  * Kingma and Ba [2015] D. P. Kingma, J. L. Ba, Adam: A method for stochastic gradient descent, in: ICLR: International Conference on Learning Representations, 2015.
  * Reuther et al. [2018] A. Reuther, J. Kepner, C. Byun, S. Samsi, W. Arcand, D. Bestor, B. Bergeron, V. Gadepally, M. Houle, M. Hubbell, others, Interactive supercomputing on 40,000 cores for machine learning and data analysis, in: 2018 IEEE High Performance extreme Computing Conference (HPEC), IEEE, 1–6, 2018\.
  * von Lilienfeld et al. [2020] O. A. von Lilienfeld, K.-R. Müller, A. Tkatchenko, Exploring chemical compound space with quantum-based machine learning, Nature Reviews Chemistry 4 (7) (2020) 347–358.
  * Jha et al. [2018] D. Jha, L. Ward, A. Paul, W.-k. Liao, A. Choudhary, C. Wolverton, A. Agrawal, ElemNet: Deep Learning the Chemistry of Materials From Only Elemental Composition, Scientific Reports 8 (1) (2018) 17593\.